Part II: How to build customer love without building software
[We ran two experiments over the course of 6 weeks. Within those six weeks, we tested our assumptions on healthy habits at work. We tested once with our own office and a second time with four different offices in San Francisco and New York. The goal of our project ‘Balance’ was to help people have healthier workdays.]
Testing one’s assumptions before building software seems obvious. Of course you should learn from the market and real people! However, as ‘makers’ can resonate with, it requires a lot of discipline not to jump to software. Working on Balance, our team definitely faced this tension. We were four technical practitioners. For us, running an experiment by hand using software off-the-shelf is like asking Michael Phelps to swim with floaties.
As Neons we are well-versed in early stage product development processes and experimentation. From our experiences, we have a guidelines of what works and what doesn’t when turning new ideas into products and businesses. Here a few of them:
- We assess the opportunity. Is it worth pursuing?
- We talk to humans. How are our potential customers describing their existing behaviors?
- We look for real data. How do we design our experiments in order to find credible signals?
- We set up small advisory boards. How can we keep ourselves accountable to a critical perspective?
In this post we will focus on number three :looking for real data. In particular we will showcase how to use a concierge experiment to test product assumptions and create customer love.
In this kind of concierge, the customer feels like they are using a real product, but behind the scenes the product is mostly manual labor by the team. Concierge experiments require a lot of human labor and do not scale, but they do give you insights very quickly.
The goal of our concierge at Balance was to create an experience that people would not only participate in, but one that people would enjoy. Inspired by Jesse Schell’s work on ‘pleasure’ and BJ Fogg’s work on habit formation, we designed an experience that integrated pleasurable interaction with a coach we designed an experience where users planned with their coaches a route to success to tiny healthy habits and were encouraged through micro-conversations to execute on that plan.
The results of our experiment were promising:
- 31 people tried the Balance experiment for 5 consecutive workdays
- 84% of users were able to accomplish at least one daily health goal
- 80% would recommend balance to a friend/family member
- 90% said their level of health awareness increased during the workday.
Through concierge experimentation you learn about the individual parts of a problem set in a way that software can’t teach you. Running a concierge places you in a state of “beginner’s mind”, building each part of the process based on what our users were doing and saying, not what our preconceptions were. At an early stage of product development concierge experiments provided an opportunity to help our team reality check all of our personal assumptions, as well as the assumptions we made based on customer interviews.
Here are a few tips on how to run a concierge that will drive customer love:
- Clearly define a goal for each stage of your experiment — At Neo what we call an ‘opportunity sprint’ is an assessment of the market to see if our idea is unique enough to drive new value and be sustainable. We then test to see if people would use the service, and then test to see if we can design the product in such a way for people to truly enjoy their experience.
- Determine how you would ‘measure love’ quantitatively. — We looked at several different quantifiable measure of ‘love.’ We settled on recommendation to friend or family member. Another classic measure is Sean Ellis’ “very disappointed metric”.
- Simplify your product to key features (you can always try other features the next round) — In our concierge we focused on getting our users to reach their tiny habit healthy goals through coaches. Keeping it simple helped us truly assess what worked and didn’t work with our design, and how to make the experience better.
- Do a thorough assessment of software that already exists — For balance we used a combination of software: mailchimp, typeform, google spreadsheets, Slack and Zapier to capture and communicate data to and from our users.
- Hack solutions for real-time analytics — Learning as you build and continually improving the experience is critical to developing customer love. However, concierge experimentation can preclude you from easy access to real time data. Our team was relentless in keeping in touch with our coaches. We set up a coach community where coaches could share best practices, integrated our tracking system with slack so we would be pinged when people entered their progress, and had our coaches send their messages daily so we could run analysis on their conversations with users.
Searching for signals can be a long and arduous process. There is an abundance of research and best practices that seem easy enough to apply to make your product a roaring success. However, when applied en masse it becomes difficult to see what is working, what’s not working and why. What concierge experimentation does is force you to keep your experiments simple, so you can learn what you should and should not translate into developer time. What we learned throughout our experience of running Balance was that our users most appreciated the short, yet rich experience they had with their coach, and what we could make more efficient with software was the coaching experience.
More on the strategy behind Balance’s place in the wellness at work market next!