Streak Developer Tools Chrome Extension

Streak is built on top of powerful platforms — Gmail and AppEngine, and wherever possible we like to build on top of those platforms. We recently started using Google BigQuery internally to perform logging analysis (see our logging framework Mache) and wanted to build a few tools on top of BigQuery’s great web interface.

The Streak Developer Tools (SDT) is a chrome extension that currently adds functionality to BigQuery (see features below) and in the future will add other tools used internally at Streak.

Here’s what the SDT can do today with BigQuery:

Save Named Queries

We felt the need to be able save queries and run them again later. While the query history in BigQuery is nice, it’s sometimes hard to parse all the queries to find the one you want. We added a section in the left sidebar for saved queries.

To save a query simply type it into the query box. Also, make sure you add one line to the query which is a comment (starts with ‘ — ‘). Whatever is in that comment will be the name of the saved query. Next, hit the save query button. It’s as simple as that. This stores the query in your local storage so it is not shared with other users of the project and will not persist if you clear your local storage. We may store this online in the future.

Query Across Datasets

At Streak, we have several tables in a dataset representing log information. Often we want to query across the entire dataset. While we could manually type all of the tables into the query, we added an option to the dataset drop down that does that for you. Simply select “Query Dataset” and the query box will be filled in for you with all the tables in the dataset.

Show Cost of Each Query

We also added the ability to see how much each query that you ran costs. Once the query completes, we added the cost (in US cents) next to the summary status of the query. The cost is calculated based on $0.035/GB processed.

Expand Multi-line Results

Since we store a lot of log data inside of BigQuery, including stack traces, many of our fields are multi-line strings which BigQuery doesn’t display very well. The SDT allows you to click on any result row and expand it to show multi-line strings. Simply click on the row number in the results table to toggle multi-line on and off.

In order to put multi-line strings into BigQuery, make sure that your line endings are ‘\r’ and not ‘\n’ as those aren’t supported by BigQuery.

Automatically Convert Timestamps to Human Readable Format

AppEngine and BigQuery store timestamps as epoch time which is great for computers and computation, but not so good for a human to read. The extension automatically finds, parses and replaces epoch timestamps with a human friendly date/time rendering.

Insert All Columns into SELECT Statements

Sometimes you want to select every column in your query but typing each of them out manually takes time especially for tables with a large number of columns. In the schema description for a table, we’ve added a final yellow field called “Add All Columns” which you can press to insert all of the column names into your query.

So that’s a summary of the Streak Developer Tools and the features currently available. We’re constantly adding features to the dev tools so be sure to install the extension from the Chrome Web Store so you get auto updates. We’re always looking for help — so feel free to contribute back to the project on GitHub.