Building Analytics at 500px

A look at building analytics from the ground up at a mid-sized start-up and some of the challenges faced along the way

Some Background

The Business

The Tech Stack

The Road to Better Analytics

Two Sources of Data

Two Headaches

The Exec Dashboard — Run For The Hills

Image for post
Image for post
Our Exec Dashboard. Numbers have been randomized

New Infrastructure, Part 1 — Redshift

A Primer on Dimensionality Modelling

The Dimensional Data Warehouse Schema

ETL

Creating The Schema

Does This Schema Fit Our Needs?

Hosting It — Amazon Redshift

Building the ETL

Image for post
Image for post
The Luigi directed graph

The Finished Data Warehouse

New Infrastructure, Part 2 — Periscope

Image for post
Image for post
An example of a dashboard
Image for post
Image for post
Writing SQL to create charts

Performance Assessment and Reporting

Deciding on Format

Image for post
Image for post

On Defining Metrics

Implementation

Periscope Training

Metrics Evangelism

Data Accuracy

Working with Engineering

20% Rule — Analytics Needs Evangelism

Reflection

What’s Next?

Written by

A Canadian living in San Francisco exploring the world samsonhu.com

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store