The value of a service: data science and user experience investigate the good, good life
A crucial part of building a product is understanding exactly how it provides your customers with value. Understanding this is understanding how you fit into the lives of your customers, and should be central to how you build on what already exists. It is a way of ensuring that every decision taken will be positive and ultimately improve the value you deliver.
In the last couple of months the value question has been hot on the lips of some of us at Jeff. Considering the current global situation and the resulting difficulty of continuing to expand a rapidly growing platform, it was a good moment to take a step back and really think about how our customers were taking advantage of “the good, good life”.
How we got here
The company wide project started with the new user experience team, who got data science involved after some early conversations. Understanding our service’s value was one of their first initiatives, and they saw that a necessary part of this was drawing the typical “customer journey”. These are the typical life cycles that users have on the platform and they help us clearly pinpoint the different moments where customers are delivered value. To draw this they needed a general overview of our different users’ behaviours.
Getting in touch with the data science team to see if we could demystify some aspects, everyone quickly realised that one of the biggest challenges facing us was that most of the understanding of our company was dispersed, based on intuition, and not easily accessible. Hypotheses were unconfirmed, and complex topics were relatively unexplored. This is pretty typical for many organisations, especially when they are still young, rapidly changing, or don’t have a strong research culture.
While this made the task at hand more difficult, it also has several repercussions for decision making. The first main issue is that teams are mostly blind to research not done by themselves, and are doomed to either waste time on ad hoc investigations, miss out on what other teams already know, or make decisions for the wrong reasons. The other is a struggle to gauge the impact of any changes to the platform. Should we encourage more users to subscribe? Is that more important than improving onboarding of new users? This is hard to understand on the fly and makes prioritising a vague process.
With both of our teams being relatively young, we saw this as a good opportunity to not only analyse our value proposition, but also to deliver a broad, unified understanding that could be used by anyone in the company when it came to decision making.
Segmenting our customers
One of the beautiful things about the data — user experience partnership is that both sides can readily contribute to a common goal in ways that the other cannot.
Part of the initial problem was understanding exactly what the status quo was — understanding what users come to us for. This is a daunting task, considering the thousands of different users with all of their peculiarities. As a data scientist however, detecting and quantifying diverse behaviours should be your bread and butter.
This seemed like a typical case that calls for a user segmentation, which is basically the division of users into different common behaviours. This classic concept is quite simple, but in practice, a good segmentation is nuanced and defined by a central trade off. For it to be useful, we need to design segments that contain specific, uniform behaviours, that all carry some business meaning. The trade off is that many tiny groups, created using all of the variables available to you, are all specific and uniform — but a few big groups designed using only a few company level KPIs are far, far easier to understand and use practically. The bonus difficulty is that there isn’t a single metric that will evaluate the quality of your segments.
This problem typically arises when data scientists are too quick to shove a whole database into their favourite algorithm. In our particular case, the platform combines online, offline, subscribers, and occasional users — without even mentioning our other customers, the franchise owning partners — a lot of combinations and the need to create a more or less unified framework. Considering that to start with we were interested in a general overview of behaviour, we alleviated the dilemma by focusing on variables that reflect the core of the business, and by creating the segments focusing on interpretable “cut off” points — the limits we used to define different behaviour groups. All of this while taking into account the entire lifecycle of our users on the platform.
Two good examples are the frequency and number of orders for a user. The frequency is easily divisible into interpretable “categories”, like users who order once a week or once a month — especially since this lines up with how our subscriptions work. Looking at how many users followed different behaviours, we can easily make more or fewer frequency segments like this. The number of orders was a bit more complicated. We saw that the more orders a user had, the more likely they were to be retained long term, but only marginally. Comparing users with more than 5 and 20 orders, for example, we saw that while users with more than 5 were slightly less likely to churn, there were way more of them than those with over 20. We accepted this trade off to define a “retained users” segment that had plenty of customers, only marginally losing out on uniformity of behaviour.
This approach meant different behaviour groups are easy to understand and immediately relevant to current strategy, while being defined in a purposeful and meaningful way thanks to our interpretable “cut offs”. For example, it becomes very clear how much we stand to gain from converting new users to “retained” users, the potential target audience for subscription up-selling (frequently ordering users without a subscription), as well as how users’ behaviour evolves (how they move from one segment to another over time).
Throughout the process of building our segments, we also adopted a methodology of continuously delivering concrete insights about what we were finding, ranging from how long users take to make their first order, to how much subscribers contribute to the financial health of a hub. These were all key pieces of knowledge that we found to be missing when we started the project. The continuous delivery of these conclusions gave our young team plenty of low hanging fruit to make some quick impact.
The next step specific to our segments is to understand how they can best be used directly to improve the development of new features and strategy. We are also in the process of understanding how they might be useful for our franchise partners, so that they can improve how they run their businesses.
User experience and our value proposition
With these findings, we were able to get rid of the vague, general context that we started with, and replace it with a detailed, specific overview of the different customer behaviours. This allowed the user experience team to start to really flesh out their understanding of the user journey, adding new paths, expanding on existing ones, and supplementing the design with concrete statistics on behaviour. We could now understand where we are fulfilling our promised value, where we are not, and where users are finding value in unexpected ways.
Having reached this point, the user experience team is also in a much better position to plan future research activity. They can interview our customers and ask exactly why they do certain things, and deliver the company insights that data science alone could not. It allows us to understand the motivations behind user behaviour, including external factors that affect this.
We are confident that this will be the first major step on our way to fully understand how we improve the lives of our customers. By taking advantage of data science’s ability to summarise how users behave, user experience is able to both fully understand the customer journey with the service, as well as round off our overall understanding with their own differential research. This puts us well on the path to understanding what role Jeff plays in the lives of its customers, and will hopefully drive real, research backed improvements to the service in the future.
We will be sure to post an update on how things are going in the near future, but in the meantime don’t hesitate to get in touch if you have any questions, comments, or feedback!
