Set the bigframes.pandas.options.bigquery.ordering_mode = "partial"
option to prevent BigQuery DataFrames from creating a deterministic
ordering and sequential index. Later in this post, I show an example of where this has a 4,000,000x
speed up in terms of bytes scanned.
Upgrade to the latest
google-cloud-bigquery
and google-cloud-bigquery-storage
packages
to download query results to a DataFrame 4.5 times faster
compared to the same method with version 1.16.0. If you aren't using the
BigQuery Storage API yet, use it to download your query results 15 times
faster compared to the BigQuery API. (31 times, if you don't mind using the
default Arrow to pandas conversion.)
I had some multi-threaded code that was actually slower than the single-threaded version. Multiple worker threads published their results to a queue and were processed by the main thread. Because of how I was waiting for the work to finish, calls to put the results onto the queue were blocking.
Five years ago, I announced that I had made the ssite static site enhancement tool public. In this post, I’ll reflect on my experience of using ssite to maintain this website for the past half decade.
For this Valentine’s Day themed data doodle, let’s use BigQuery and the New York City public dataset on Citibike trips and to find a romantic spot to bike to with a significant other.
Say hello to ssite, the little blog engine that doesn't do much. This past weekend I built a blog engine to keep my site organized. It differs from existing engines in that it does much less. It generates an index without templating or modifying the articles at all.
In this article, I assume you already have setup all the hardware you need for a Raspberry Pi. In this tutorial, you will flash an OpenELEC XBMC Operating System, configure XBMC to install video addons and themes, and set up your smart phone as a WiFi remote control.
In this series, I'll go from start to finish on how to build a home-theater computer with Raspberry Pi and the OpenELEC XBMC Operating System. At the end we'll be able to install XBMC plugins to stream videos, including PBS.
This is part 3 of a three-part series on building and packaging matplotlib as a multi-architecture, Solaris IPS package. In this tutorial, I go through creating a multiple architecture (fat) IPS package.
This is part 2 of a three-part series on building and packaging matplotlib as a multi-architecture, Solaris IPS package.
This is part 1 of a three-part series on building and packaging matplotlib as a multi-architecture, Solaris IPS package. At the end of this series, you should be able to install matplotlib as easily as any other Solaris 11 package, on both SPARC and x86 machines.
This guide walks through the steps I followed to build KeePassX on my Mac. I found I needed to install many prerequisites in order for it to build.
So, say you had a console Python application where you
are trying to print a table of numbers as your output. Printing the numbers
without any formatting is messy. By using formatting options for the
%
operator, you can make tables that are much easier to
read.
The ternary operator isn't necessary (Python didn't have one for years), but it can make code clearer when put to good use. The ternary operator consists of three parts: a condition, an expression to evaluate if the condition is true, and another expression to evaluate if the condition is false.
I've been aware list comprehensions for a while now, but they seemed complicated, so I was afraid to use them. Today, I found that they really aren't that bad. In fact, if you already know how to populate a python list inside a for loop, you're 95% there already.