5 Beta lessons — from a multi award winning service

Carl Burton
Nov 28, 2017 · 6 min read

Medium is full of posts on lessons from design sprints and MVP’s but a bit light on beta lessons so I thought I’d share some big learnings from my work at The Co-op. These lessons are valuable for any Beta.

1) Trust and decision making

Data is essential to inform decisions but at the start of a Beta quite often you don’t have the data or insight at hand to manage stakeholders. You have to play the long game to earn their trust in the early days. This means building features they want, even when you know they aren’t what users need. As long as you time box the work to 1 or 2 weeks and use it to test the stakeholders hypothesis then it is win, win.

If it turns out they were right and you were wrong then great, you have a valuable feature for your users. However if it means you have hard evidence to pushback against the “I want” and show the user need then you start earning trust with your stakeholders. No one wants to build the wrong thing or waste money. With growing trust you earn permission to manage the backlog without having detailed stakeholder engagement in decisions.

Our trust journey during Beta

Above is a crude graph of our journey. Our stakeholders were used to waterfall projects so they were initially sceptical of our mystical agile ways of working. They loved seeing the progress in regular show and tells but the real lift off came when we demonstrated how by testing assumptions quickly and proving they can be wrong, we avoided building the wrong thing. By focusing on outcomes over outputs we were saving money, speeding up delivery and increasing confidence of success.

There is no magic formula for this. You need to be able to read people and learn fast, which can take time and comes with experience. If you’re struggling it’s ok to ask for help as you will have a team around you who have knowledge and experience to share.

2) Taking action when things go wrong

Things are going to go wrong, probably quite often, especially in the early days. These are opportunties to shine, not bury your head in the sand.

In our service there was one part where we went very deep based on business wants, not user needs, and guess what when we let real users test it they hated it and we had to take action. Credibility of the Beta was on the line with our users and stakeholders. It was still early days and they had experience of past project failures.

This became the most important thing on our backlog and prompted us to immediately replan. We did the research, design and development in 8 days. 8 days from receiving the problem to releasing a working solution. What was a mini crisis quickly turned into a mini triumph.

If we had stuck to the plan and hid the problem I’m certain our users would have become disengaged and the bad news would have found its way back to our sponsor anyway. It could have turned a mini crisis into a catastrophe which could have derailed the whole beta.

3) Overcoming your critics

When you’re doing fortnightly show and tells, you very quickly get good at doing presentations and slick demos. However you need to have the voice of the user as part of the message, otherwise your critics will think what you’re saying is too good to be true. There are many ways of doing this from showing clips of user research or, as in the case above, getting real users to present.

We reached a point in our journey where there were rumours going around that we were 6 months behind plan. However as we have never had a 6 month plan or a fixed one (we reprioritise as we learn all the time) I had no idea where this had come from. What it showed was there were mixed messages and lots of assumptions. People don’t come to all the show and tells plus even if they do it is hard to see how all the parts and iterations fit together, so we needed a way to communicate a clear message to overcome our critics.

So I arranged an end to end demo of the service and paired up real users with different members of the team to demo and discuss different parts of the service. We invited questions and it was the users who got asked most of them. Their answers really brought to life the value of the service. Stats are powerful but stories of how your service is helping your users are undeniable and even better.

4) Building a winning team

The team we have built has achieved great things, none of which would have been possible without the team being a true self organising multidisciplinary one.

People write books about this stuff but in short the key for us has been having a culture where people feel safe to call each other out knowing that it is driven by the desire to build the best product we can. Sure sometimes we fall out but healthy tension is fine as long as we make up quickly and don’t carry baggage into the next day. We also have a great sense of fun in the team which helps lighten the mood at points of stress and also motivates people to put in the extra effort when it is needed.

This sounds easy but requires a lot of investment both when setting up a new team and on boarding a new team member. Food is a great enabler to get people talking and sharing

5) Embracing the struggle

When you are building something with a great bunch of people and seeing how much of a difference it is making to users you can’t help getting deeply emotionally invested. When things are going well you get this amazing sense of pride, but when they aren’t it can feel like the world is against you.

We have faced adversity throughout our journey and this takes a real toll. We all know when projects go well lots of people come of the woodwork to take some credit but when they don’t you are often left isolated. We have been very lucky that our work has received external recognition, making the roll the team has played undeniable.

However, we have taken our fair share of bruises along the way and that is why it is important to be able to let go and make the most of your down time away from work. If you don’t then your mental health could be at risk which is why this is the most important of these lessons.

Whatever your thing is and no matter how it’s going please don’t lose perspective. So you miss a deadline or a demo fails or your code crashes . . . life goes on, the world doesn’t end and the sun still rises in the morning.

As a product lead I constantly have to think about what the next most important thing is. This is just as important at the weekend, as it is during work hours. Do I choose to spend a great weekend with my family or spend all weekend stressing about things at work that I have no control over?

Thanks for reading.

Carl Burton

Written by

I'm Head of Product at Co-op Digital. Tackling tough problems, making mistakes, learning, iterating and coming up with user centred solutions. All views my own.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade