Anyone can publish on Medium per our Policies, but we don’t fact-check every story. For more info about the coronavirus, see cdc.gov.

Image for post
Image for post

How We Use Metrics At The Liberators

And how it is helping us get through the Corona crisis

Christiaan Verwijs
May 25 · 8 min read

We always tell teams to track metrics that matter to them. For Scrum Teams, this could be cycle time, team morale, and stakeholder happiness. For a marketing team, this could be a conversation rate or NPS. When you have data, you can formulate hypotheses about the experiments you’re doing and when they are successful (or not). When you have more objective data available, you can reduce the impact of social and cognitive biases on what is that you are doing. If all you have to go on our assumptions, opinions, and intuitions, it's hard to be successful.

“If all you have to go on our assumptions, opinions, and intuitions, it’s hard to be successful.”

When are we successful?

In a previous post, we made the distinction between output-oriented and outcome-oriented metrics. Where the former is mostly about how busy you are (e.g. velocity, hours spent, revenue), the latter is about how effective the outcomes of being busy actually are (e.g. stakeholder value, profit).

It's easy to measure everything and still not know much about what really matters. The challenge of metrics is to pick outcome-oriented metrics that tell you when you’re successful (or not). The first step is to consider what success actually looks like.

“It’s easy to measure everything and still not know much about what really matters.”

From a financial perspective, we agreed that success required at least sufficient revenue to cover our costs (including salaries), build buffers to weather future storms, and generate a modest profit. Finally, we agreed that we wanted to create enough space in our agenda to try new things, run experiments, and create valuable free content.

How we decided to measure “success”

With a clear definition of success in mind, we then translated it into indicators that we could measure (our ‘metrics’). For example:

  • When we are successfully able to support more people, we should see increases in metrics like the number of followers on various platforms, (re)shares, orders in our webshop, and participants in our various events. In other words; growth.
  • When people like what we do and it offers value to them, we should see increases in their engagement with us, our content, and our community over time.
  • When we are successfully able to generate enough revenue, we should be able to cover our costs and make enough profit to use that to create space to do work that isn’t paid (like writing, podcasts, research, free apps).
  • When we are successfully able to build a community, we should see the number of members of The Liberators Network grow over time. More importantly, if we do a good job, more and more of these people should engage with us and each other.

Our dashboard

With a set of indicators in mind, we set up a dashboard in Cyfe and built a service that connects with the platforms we use. At The Liberators, we rely on over 25 platforms to deliver value to our customers and community. For example, we have Buzzsprout for our podcasts, Medium for our blog posts, Shopify for our webshop, and YouTube for videos. These are the obvious ones. Behind the scenes, we use Exact for accounting, Harvest for invoicing, Bit.ly for link-shortening, AppVeyor for deploying our codebase and Fotojet for creating collages. And there’s also use Patreon and Paypal for donations, EventBrite for our events, Trello for coordinating our work and several more.

We collect metrics every hour. They are aggregated once a day. Below is a snapshot of the dashboard at the time of writing:

Image for post
Image for post
A snapshot of our current dashboard. We’ve blurred out personal information.

As you can see, each widget captures one element of how we defined success. We grouped the widgets into activities (how busy we are), output (how much content we generate), outcomes (how valuable that content is), and impact. There are also some critical indicators of class quality and taxes that we need to pay. Let’s explore two widgets in more detail.

Metric: How much space we have

One of our most important widgets is the one that tells us how much space is created by our revenue and profit. The “space” is represented by the yellow bar. The higher it is, the more time we can spend on unpaid work.

Image for post
Image for post
Note the axis; it dips to -50K. If things proceed as they are now, we drop below zero before the end of the year.

During the Corona-crisis, this bar took on another meaning. Because we had to cancel all our paid in-person events, we effectively saw most of our revenue evaporate, save for some modest streams from donations, a few virtual events, and our webshop. At the moment, we’re surviving mostly on our buffers. If we are unable to generate additional revenue, we’ll likely run out by November. Even though that is scary, the good news is that the information from this widget allows us to breathe, consider our options and, and prevent chaotic shifts in order to make some money.

It's also the reason why we’ve more actively started asking for people to support us through a time where our primary source of paid work (in-person workshops and classes) are gone.

Image for post
Image for post
The Corona crisis obliterated our profit-loss balance. Currently, we have little income — only costs.

Metric: Are people increasing their engagement with us?

When our work and content is helpful and valuable, we expect people to engage more with it over time. So if someone likes our blog posts on Medium, we hope they’ll join one of our meetups, participate in a class and/or support us on Patreon. Although you could simply measure “engagement” by counting the members on our Meetup, or the followers on Medium, this alone is a very indirect measure. So we started looking into ways to capture engagement within and across platforms.

After a lot of experimentation, we developed a service that breaks down how people engage with our work into three groups:

Image for post
Image for post
Breakdown of the people we’re connecting with through our work, as well as “conversion rates”.
  • “Fans” are people that frequently engage with us, our content, or our services over many platforms. For example, they participate in classes, join meetups, order products from our webshop, or even support our work financially.
  • “Followers” are people that engage with our work on multiple occasions over time and on a few platforms. They like what we do, but it isn’t super valuable to them yet.
  • “Intrigued” are people that have engaged with our work once or twice and on a very limited number of platforms.

Once we identified this break-down and figured out how to count it with some reliability, we calculated the percentages of people that flow from one category to another. The widget shows how many people become “fans” or “followers” from just being “intrigued” (27.3%) and the number of “followers” and “intrigued” that become fans (4.5%). If we do a good job, both should remain the same or increase.

This widget is helpful because it is as close as we can currently get to measuring how successful we are in achieving our mission. The metric is imperfect in how it collects data — there’s much we don’t know — and it’s lagging in that it takes a while to see how experiments affect it.

Trends over time

Metrics on their own have no meaning. You have to interpret them together, and especially the trends over time. Thankfully, we’ve been collecting data since September 2018. Our dashboard allows us to show trends:

Image for post
Image for post

How we use the metrics

Barry Overeem and I receive a weekly report with a screenshot of our dashboard. Every so often, we do a What, So What, Now What on the metrics and determine what else is needed from us to adapt to what is happening.

During the Summer- and Christmas-break, we do a more thorough inspection to make bigger shifts in our strategy and start new experiments. For this, we use Strategy Knotworking (as explained in this post). The good thing about having metrics available is that we can create hypotheses about when an experiment is successful. For example, we restructured how we post content to LinkedIn and Twitter in October and expected to see more engagement in terms of clicks and shares. The metrics clearly show that this happened.

Closing words

If you find yourself reading this post and thinking “Wow, cool, what tools did they use to do this?”, then you’re asking yourself the wrong question. No matter how cool the tools are, they are only as helpful as the data you put in. When we started The Liberators, we asked ourselves “How do we know when we are successful in achieving our mission?”. This helped us to identify metrics that we now inspect frequently.

When we started The Liberators, we asked ourselves “How do we know when we are successful in achieving our mission?”

We hoped you liked this post, maybe even found some inspiration. Give it a try! And let us know in the comments what happened.


The Liberators

The Liberators: Unleashing Organisational Superpowers

Christiaan Verwijs

Written by

I aim to liberate teams & organizations from de-humanizing and ineffective ways of organizing work. Professional Scrum Trainer & Steward @ Scrum.org

The Liberators

The Liberators: Unleashing Organisational Superpowers

Christiaan Verwijs

Written by

I aim to liberate teams & organizations from de-humanizing and ineffective ways of organizing work. Professional Scrum Trainer & Steward @ Scrum.org

The Liberators

The Liberators: Unleashing Organisational Superpowers

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store