“Data Thinking”: How I do product Analytics

A case study with real examples.

There are three steps to this work: The Analytics Strategy, The Event Specification and The Dashboard.

Part 1: The “Analytics Strategy”

An analytics strategy is a common first phase of a project before beginning the expensive process of building usage analytics into your product. After delivering this work for 4 project teams at Infor, I finally did it. I re-designed the format of the “Analytics Strategy.”

This is what it looks like:

You can read a copy of it here.

These are the principles it follows:

1. Visual. I was keen on adding abstract additional data visualizations to the document for clarity, but mainly just for the sake of being visual, which keeps viewers reading and engaged.
2. Minimal. Many problems and metrics were left off of this document.
3. Prioritized. The first scroll of the page is the top of the funnel and I want the reader to get hooked.
4. Self serve. I focus my efforts on the narrative version of the document. Basically, just not a deck.
5. Framed as problems and solutions. This is my favorite one. It prevents confusion around the tough question of “so what do I do with this data?” This focus, this intent, is so important to get done before you get swept away into analyzing the data you do have.

Here are the inspirations:

Left: WeWork dashboard. Right: My App dashboard sketch in same style, but grouped into problems.

1. The WeWork member experience dashboard. I was struck by epiphany when I saw this: Basically, I realized that I needed a design system.
2. The exercise of “Problems, then solutions” (read more here , here and here ) is one of my favorite wisdoms that is part of the field of Human Computer Interaction . It was uncanny and exhilarating to realize that it could have such a role in building out product analytics for apps and brands.

How to do it yourself:

1. First, identify and prioritize the problems your solution will solve. Then, sketch or simply describe solutions for these problems. This is just the regular process that UX people are trained to do.
2. Then, come up with as many metrics that should correlate with the problems which are being solved. I like to say that coming up with metrics is an under-appreciated skill — you can be creative. Remember to not confine yourself to things you can necessarily measure. It’s a brainstorm.
3. Scale back to what can be measured, and maybe include some “optional” metrics that are more expensive to calculate.
4. And boom you are pretty much done. Now, package up the problems and metrics in some kind of digestible format!
5. Finally, Add in your generic “adoption” category. Adoption, engagement itself indicates that users are seeing value and are having their problems solved. (And depending on the sophistication of your org, connect up to sales data and pair adoption with ACV!)

p.s. Remember to circle back and iterate. Technically the analytics strategy should be a one-to-one with the evolving key drivers the product group is working on.

Part 2: Create the event specification

Here is what it looks like:

LEFT: Event Map showing the paths that trigger each event. CENTER: Event table with Properties. RIGHT: Property Table with Values and which Events they are used for.

LEFT: Event path description, with Syntax for each event. RIGHT: Metrics Summary.

Principles:

1. I work backwards from the land of interesting metrics I identified, and simply ensure my tool can visualize it.
2. Resist breaking minimalism and not adding extra stuff I don’t need.
3. Put care into the document. Show respect to the developers implementing the tags.

Inspirations:

1. The Mixpanel “Death of the page view” article recording where a user is (i.e. “page”) indiscriminately, we lack intent in each item we choose to record, and entropy increases. By doing so, we create data noise that analysts could waste time analyzing, and proliferate a mind-set of not having intent in our research.
2. A quote from a somewhat obscure page on the Segment.io docs site. It reads:

“The mistake most people make when installing tracking for SaaS is trying to record everything immediately. Adding tracking on an iterative cycle leads to quick wins and a much better analytics system.”

This quote clicked with me and reinforced my ability to resist temptation to add more and more events and properties.

Part 3: Building the dashboard

Basically, the analytics strategy did the heavy lifting for you. Just go out and execute. I use Google Data Studio to build the dashboard, and rely on modifying the SQL that builds the sources, as opposed to using the GUI “data modeling” tools that Data Studio offers.

Here is what I have right now:

Left: Dashboard cover page: Right: Drill ins by Customer.

Principles:

First off, the dashboard is basically a straightforward execution from the strategy portion. Metrics, trended, w/ options to drill in. Here are some other notes as I got to the execution.

1. Label metrics with explainer text directly in the report. (Don’t fully rely on an appendix).
2. I love trend or spark lines. I think they draw the eye and are one of the most effective way to engage the viewer to see a story.

There is a lot more to say here, but I will leave it at that!

Recap

Why I titled this piece “Data thinking”

I call this “data thinking” because you can do a lot with “data” without having well…“data.” A well thought out analytics strategy, created well ahead of product work can be a rudder. This is very similar to the often blogged about “OKR.” My case study is about how when someone asks you to track their app, you can flip the conversation into much more.

Coming up with metrics is a creative exercise, one that can be learned

It took me a while to get more creative and wholly appreciate the skill of creating metrics. I read a lot of analytics click bait and I just hadn’t run into the idea of “hey, be divergent about coming up with metrics!”

Minimalism and “Problems & Solutions”

It is not a coincidence that some of my favorite themes from working in software surfaced in this work.

Learn more about me at http://rickswette.info