Tim Swast posted
Introducing partial ordering mode for BigQuery DataFrames (bigframes)
Set the bigframes.pandas.options.bigquery.ordering_mode = "partial"
option to prevent BigQuery DataFrames from creating a deterministic
ordering and sequential index. Later in this post, I show an example of where this has a 4,000,000x
speed up in terms of bytes scanned.
Tim Swast posted
Announcing google-cloud-bigquery Version 1.17.0: Query
Results to DataFrame 31x Faster with Apache Arrow
Upgrade to the latest
google-cloud-bigquery and google-cloud-bigquery-storage
packages to download query results to a DataFrame 4.5 times faster
compared to the same method with version 1.16.0. If you aren't using the
BigQuery Storage API yet, use it to download your query results 15 times
faster compared to the BigQuery API. (31 times, if you don't mind using the
default Arrow to pandas conversion.)
Tim Swast posted
When to use concurrent.futures.wait versus Queue.get (blocking) in Python
I had some multi-threaded code that was actually slower than the single-threaded version. Multiple worker threads published their results to a queue and were processed by the main thread. Because of how I was waiting for the work to finish, calls to put the results onto the queue were blocking.
Tim Swast posted
Ssite for Handwritten Static Sites - Five Years Later
Five years ago, I announced that I had made the ssite static site enhancement tool public. In this post, I’ll reflect on my experience of using ssite to maintain this website for the past half decade.
Tim Swast posted
What are the most popular Citibike destinations for NYC
couples?
For this Valentine’s Day themed data doodle, let’s use BigQuery and the New York City public dataset on Citibike trips and to find a romantic spot to bike to with a significant other.
Tim Swast posted
Introducing Ssite: An Engine for Handwritten Static Sites
Say hello to ssite, the little blog engine that doesn't do much. This past weekend I built a blog engine to keep my site organized. It differs from existing engines in that it does much less. It generates an index without templating or modifying the articles at all.
Tim Swast posted
Building a PBS streaming device with Raspberry Pi and OpenELEC XBMC : Part 2, Software
In this article, I assume you already have setup all the hardware you need for a Raspberry Pi. In this tutorial, you will flash an OpenELEC XBMC Operating System, configure XBMC to install video addons and themes, and set up your smart phone as a WiFi remote control.
Tim Swast posted
Building a PBS streaming device with Raspberry Pi and OpenELEC XBMC : Part 1, Hardware
In this series, I'll go from start to finish on how to build a home-theater computer with Raspberry Pi and the OpenELEC XBMC Operating System. At the end we'll be able to install XBMC plugins to stream videos, including PBS.
Tim Swast posted
Matplotlib 1.1.0 for Solaris on SPARC and x86, Part 3: Packaging
This is part 3 of a three-part series on building and packaging matplotlib as a multi-architecture, Solaris IPS package. In this tutorial, I go through creating a multiple architecture (fat) IPS package.
Tim Swast posted
Matplotlib 1.1.0 for Solaris on SPARC and x86, Part 2:
Setting up Solaris IPS servers for multiple architecture (fat)
packages
This is part 2 of a three-part series on building and packaging matplotlib as a multi-architecture, Solaris IPS package.
Tim Swast posted
Matplotlib 1.1.0 for Solaris on SPARC and x86, Part 1: Compiling
This is part 1 of a three-part series on building and packaging matplotlib as a multi-architecture, Solaris IPS package. At the end of this series, you should be able to install matplotlib as easily as any other Solaris 11 package, on both SPARC and x86 machines.
Tim Swast posted
Setting up KeePassX on Mac OS X Lion
This guide walks through the steps I followed to build KeePassX on my Mac. I found I needed to install many prerequisites in order for it to build.
Tim Swast posted
Plain text tables with Python's % operator
So, say you had a console Python application where you
are trying to print a table of numbers as your output. Printing the numbers
without any formatting is messy. By using formatting options for the
% operator, you can make tables that are much easier to
read.
Tim Swast posted
More fun with Python (the ternary operator)
The ternary operator isn't necessary (Python didn't have one for years), but it can make code clearer when put to good use. The ternary operator consists of three parts: a condition, an expression to evaluate if the condition is true, and another expression to evaluate if the condition is false.
Tim Swast posted
Python List Comprehensions are Easy!
I've been aware list comprehensions for a while now, but they seemed complicated, so I was afraid to use them. Today, I found that they really aren't that bad. In fact, if you already know how to populate a python list inside a for loop, you're 95% there already.