Despite its importance in multiple industries, time series data continues to be one of the more challenging datasets to handle at scale. Time series help us identify trends in data, letting us demonstrate concretely what happened in the past and make informed estimates about what will happen in the future.

But for TIMESLIDERS, time series data was the backbone of their business, but also the bane of their users. TIMESLIDERS is a mobile-application performance library, built for developers. …


From a performance perspective, one of the most important things to consider while designing your Bigtable Schema is how to properly design your index. And our friends at Urban Fitness Forge ran into this headlong recently.

Urban Fitness Forge is one of the latest in awesome real time fitness tracker products. Their advanced clothing line carries over 50 sensor measurements for the human body, each sampled 10 times a second. After their initial series A funding, they saw a huge boom in usage for users as they were picked up by a local clothing company. …


FlightMight was having trouble with Cloud Bigtable performance from their clients around the globe. Their system for monitoring aircraft information meant sampling lots of instrument data hundreds of times a second, submitting it all back to a server for storage and analysis.

Sadly though, their aircrafts tended to work all over the world, with shoddy connections and quickly changing latency situations, leaving their performance when interfacing with Cloud Bigtable, less than ideal.

Location, location location.

The issue here is that Cloud Bigtable, in order to be as performant as possible, is local to a specific region. As such, accessing Cloud Bigtable from clients…


When you’re dealing with something as big and powerful as Cloud Bigtable, it’s important to make sure you’re looking at the right data and tests to ensure performance is awesome. As we talked about before, most of the magic of how Cloud Bigtable handles such scale is all behind the scenes; where frontends can coordinate, balance, and optimize your access patterns by moving tablets of data between various compute nodes as needed.

Because so much of the heavy lifting of Cloud Bigtable performance happens behind the scenes, it might not be obvious how to figure out whether an instance is…


When it comes to performance at scale, there’s very few products that can truly represent the “at scale” part the way that Cloud Bigtable does. To date, Bigtable is the technology behind the majority of Google products, including Gmail, Maps, Youtube; Each of which serves multi-billion active users.

What’s really amazing is that this same technology that powers Google’s most popular products is also available for you to use for your applications. …


With the boom of mobile devices across the world, we’ve been seeing a lot of areas where smartphone adoption outpaced the ability for the telecom providers to update their networks. The result is a less-than-fast experience for the users, and a deep need to re-evaluate how our current internet protocols are working on the modern web.

This is some of the fundamental reasoning behind Google’s development of the QUIC protocol.

QUIC (Quick UDP Internet Connections) is a modern transport layer protocol that provides congestion control similar to TCP and security equivalent to SSL/TLS, along with improved performance (reduced connection and…


One of my favorite things about App Engine is that your requests are automatically traced, so you can see where during a request that your execution time is going. Shortly after, I showed a small technique that allowed you to insert custom trace blocks (since most trace data is RPC based).

Now, I’m really excited to talk about how tracing can be rolled out to all your other systems, even the ones not running on app engine.

Recap: Stackdriver Tracing

Stackdriver Trace allows you to analyze how customer requests propagate through your application, and is immensely useful for reducing latency and performing root…


Understanding the performance of production systems is notoriously difficult. Not only is it difficult to replicate the exact scenario in which the product is being used in the wild, but adding profiling adds an additional load on the production system, which could inadvertently hurt your performance.

This is a catch-22, since if you don’t analyze code execution in production, unexpectedly resource-intensive functions increase the latency and cost of web services every day, without anyone knowing or being able to do anything about it.

This predicament is where many of us find our mental energy : We need to constantly perf-test…


Nomad Socialite was a new geo-location game, where players traveled all around the world to rack up points, catch the latest in urban events, and meet a whole community at the same time. So, catching a plan to San Fran, hitting the hot new Poké popup, and catching their rare collectible digital item can help rack up some huge points.

The challenge, however, was that from a producer perspective, it was getting hard to track analytics on all these events to hand back to the proprietors running events. …


Urban IoT was is a sensor and spatial representation company. Their work is simple, they deploy a whole load of sensors in an urban area that monitor everything from noise, temperature, humidity, pressure, (and more) .They were noticing issues when pushing the time stamped data to GCP Datastore, saying that after a while, their reads slowed down significantly on their dashboards.

Stamped out.

Cloud Datastore is built on top of Google’s NoSQL database, Bigtable, and is subject to Bigtable’s performance characteristics. Bigtable scales by sharding rows into separate tablets (aka a contiguous unit of storage); When a Bigtable tablet experiences a…

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store