The Umbrella Approach to Analytics
If you’re an Analytics team at a growing start-up. How do you manage prioritizing all of the things you’re working on?
You’re a small analytics team in your early days, data is becoming a strategic and tactical necessity, and there’s a soul crushing amount of work to do. Actually stack ranking and prioritizing that work is a delicate dance, and one that usually leaves some amount of data needs unmet. Once an organization is sold on the value of data, how do you pick a conceptual framework to guide what projects you support and how deep you go?
The Umbrella framework allows teams to map product verticals that need basic analytics coverage, and then selectively pick and attack projects that allow for deep, meaningful impact.
Analytics is hard.
I know…you don’t ever have enough people to do all of the things that you want to do. Spoiler. You’ll always be understaffed. You’ll default to tactical and ad-hoc. You’ll constantly be re-prioritizing all of the things. It really doesn’t matter what your official function is when you’re in the technology space…engineer, analyst or PM…you’re always having to juggle responsibilities between what’s urgent and what’s important. This means that as much as you can fantasize about it, you won’t ever have the opportunity to be all things to all people, and you will have to say no, a lot.
So, for a small startup Analytics team, what does this mean?
Analysts, and sometimes to a greater degree Data Scientists, have a tendency to err on the side of Perfect, and not just Good Enough. Get that R2 higher, the regression model tighter, the powerpoint cleaner. If you’re spending an hour on the color palette of your presentation, I’m sorry, you’re doing it wrong. In the early days of Analytics work, the value of the first 30% of the work is greater than the value of the last 30%, so if we’re looking at the added value of our timespent as the input, it’s clear we should be working on the parts that have the most marginal value, up until the analysis is actionable.
Do The Basics Well, For Everyone
Products and Features all need a basic level of analytics coverage. When using data to track the success (or otherwise) of products and features, there will always be a core group of people that care about performance at a holistic level, and this means Growth and Engagement metrics around user acquisition, churn, and retention are a must. These interested parties range from executives and management to individual product owners and engineers, and a standard set of metrics to track overall performance is key to understanding your organization, product, or feature.
I should note here that operational reporting may or may not fall under the “Analytics” function. This can mean things like catastrophic crashes, anomalies, real time monitoring, etc. If you’re not offering these need-to-haves, you’re going to have a bad time. Without providing every product a minimal toolkit of metrics (accurate metrics to be clear, which require solid logging), shit is going to rain down on the people and teams who ideate, create, and support your organization’s products and features. They won’t be able to defend, promote, or properly frame decisions from a quantitative position.
What you don’t want to do is fall into a pattern where it’s so important that you as an analytics team provide the impact directly that you don’t facilitate other teams by providing them the data they need to make an impact themselves.
People Will Always Remember Being Shit On
This is why the Umbrella approach is so important. Everyone needs some sort of basic coverage when it comes to managing their product or features so that they’re able to operate with some level of data enlightenment. The impact that you create as a team isn’t always direct, and that’s ok. But…you felt this urge to lock down for a month, drill-drill-drill, create a beautiful slide deck with a solid recommendation, and do the talking tour. An important part of what analytics does is facilitate others to make impact using data, and you didn’t do that. You decided your direct impact was more important, so you went deep quick and didn’t prove the breadth of your value, so now you’ve got trust issues. People were flying blind.
Don’t forget a part of your job is facilitating others.
The handle part of the Umbrella represents providing deep analysis. While you’re providing the breadth part (reporting), it’s important to pick a single project and drill deep. There are a couple of reasons this is important:
- You don’t want to turn into the “reporting” or “dashboarding” team.
- You prove the deeper value of Analytics to others by following through to impact, not just throwing data over the fence, but this requires a history of delivering on the basics to have the prerequisite trust.
- Successes create more demand.
Always remember, deep analysis and product understanding don’t matter unless you actually do something with them. Understanding just for the sake of understanding is academic, and insights are only valuable to an organization if you use them to drive action. Without a well established baseline of delivering on the basics, you’ll constantly be fighting for your end user’s trust, and struggling to prove why your team matters. You will regularly find yourself defending the accusation: “You can’t give the sums and the counts accurately, why would we ever believe your regression models and predictive analysis?”
Using the Umbrella framework allows us to easily understand how Analytics as a function adds value at the places that matter, and forces us to map and provide that basic level of coverage that allows people to understand their own products, services, or features. Frameworks like this also keep Analytics teams focused on picking single projects that force us to provide deeper impact, and follow that impact through to decision.
Interesting in building the future? Oculus is hiring.