ProductStories #2–200 users, 2 months of testing: our App beta test program

After having revealed some Luko product manufacturing secrets in the first episode of our Product Stories, I propose for this new episode a zoom on a project that has been at the heart of Luko in recent months: our new mobile application.

Amélie, our lead app, would probably explain the details and technical issues of this application much better than I do.

That’s why I propose, instead, to share with you our secret recipe: our large-scale beta test program, which has allowed us to design a really useful application, made by and for our users.‍

A strange idea

Why make an insurance app?

Of course, it may seem strange to think that you are going to launch an insurance application. Especially with the thousands of mobile applications that already exist… on subjects a little more exciting than insurance :)

Especially when you consider that insurance applications have not necessarily been historically successful.

So why did we start the adventure?

Because our vision was not to release a simple insurance application that digitized a contract. Our vision was to go to the extreme to be as close as possible to our users and to be present for them everywhere, all the time.

As for our ambition, it goes far beyond the idea of incorporating a contract into an application. We want to protect homes and make prevention an ambition that is becoming a little more real every day. And to follow the consumption advice of our electricity or door sensors, the desktop didn’t seem really suitable… ( have you ever been to Strava on your computer?)

A small technical challenge

To include our vision in this new app, we set ourselves an ambitious program from the beginning, despite the fact that resources are still quite limited. We have just reached 20 employees and it was impossible to dedicate all of our technical resources to a single product. We have many challenges on which we must focus in parallel: adding new insurance ( PNO insurance), optimizing our existing product, perfecting the artificial intelligence of our sensors, etc.

It was therefore a real challenge to successfully allocate a full-time dev resource to the realization of a product that is not a direct vector of growth.

However, to achieve our ambitions and offer a real protection solution, it was necessary to start unlocking resources for this first version of the application one day.

This current version is only a first step towards what we have planned. Without revealing too much, the application will include in the future:

• key insurance features(such as an inventory of valuable items for better and faster reimbursement in the event of a claim)
• unique protection features(notifications when the door is left open, consumption tips to reduce your electricity bill and your ecological footprint, etc.)

All these features, both those we have already developed and those we have designed for the future, come from one place: the imagination of our best allies, our users.‍

How we built our app thanks to our best allies

So here we are back to the past: mid-November 2018.

The decision to develop a new application was made and the programming language chosen. All that was left was to….

Yes, but… What if we’re wrong? What if we don’t integrate the features our users want? What if…

Questions that were difficult to answer with internal meetings. So the key was somewhere else. In this case, at the same place where we had already gone a few months earlier to create our online subscription process: with our users.

1. First user feedback on the existing system

So we started by asking them for feedback on the application we already had on the Protection section. From there came many ideas. In parallel, we analyzed the data on the use of the online personal space: what were our users doing on this space? What features were most commonly used?

Based on these quantitative and qualitative feedback, we had sufficient basis to imagine a first application.

2. Definition of a precise roadmap

With our designer, we imagined what the application would look like in an ideal world (a world full of mobile developers to infinity 🤓). Then we broke it down into small parts that could be done as we went along, knowing that our vision would only come true if we made our first draft.

In view of the work ahead, one of the keys to success was to define a precise roadmap. And to stick to it. So we decided that we would start user testing in February, and that the app would be released at the end of April, no matter what.

Our development plan was defined:

We designed this first version of the app in 2 months.

We wanted a prototype that was simple enough but interactive enough for our users to test it in production. We could have gone on something less technically expensive. But the resources were there, at the right time, and it was also an opportunity to try React Native.

3. Selection of beta testers

In parallel with the development of the app, we have selected our users for the tests. Those who had shown the most interest in us, those who had already given us constructive feedback, those who were passionate about the user experience… a total of 300 people were contacted to join the adventure.

When you ask a few people you only know through a few interposed chat exchanges if they could give you some time to help you and more than half of them answer present, with a lot of enthusiasm, it’s a real special moment!

With 200 beta testers on the starting blocks, we were ready to get lots of feedback.

4. Feedback feedback feedback and iterations

To move forward on the application while collecting continuous feedback, we proceeded in waves.

A first wave of 50 people gave us the first feedback: such and such a feature was missing, such and such was not displayed correctly on their phone (iPhone X made us pull more than a hair).

During this phase, we were lucky to have one more developer. The dev team therefore separated the tasks: one person made sure that user bug returns were fixed as quickly as possible, while the other continued to develop the most requested features.

Every week, we would make a new release and inform users who had encountered bugs that their problems were fixed.

Then came the next wave, and so on. Until last week.

We collected the feedback via different channels:

• a classic Typeform with fairly open-ended questions
• for those who preferred it by simply returning an email, or via the application’s chat directly

For me, it was quite important to be extremely available, and not to limit our users to express themselves, and also to reassure them that we were really listening to everything they were telling us.

Some of them lent themselves to the game at 300% and gave us incredibly complete feedback. We are fortunate to have product designers and UX specialists among our users, who have really adopted the application as their own and shared with us their vision of what they would have done.

5. Assessment and prioritisation

The key was to follow the feedback as it happened and to continuously integrate everything that seemed most urgent to us: bugs, display problems….

At the end of the test, we still have many features to develop. If we look at the list of what has been requested/reported as missing, there are more than 35 features that we have in mind and that we will prioritize in the coming months based on the volume of feedback we have received… and the development time.

A whole project awaits us, but despite everything, we have a lot of faith in what we have produced. The application released today is the one our users wanted: it integrates their current needs and will soon integrate all their future needs.

‍

What if we have to do it all over again?

If we had to do it all over again… we’d do it again with our eyes closed!

For me, it would be inconceivable to make the product without our users. Most of the decisions we make at Luko are driven by feedback from our users. Even internally, we rarely release a new feature or design without the whole team giving their opinion.

Of course, sometimes you have to be able to innovate and listen to your instincts, propose new things. Ford’s well-known expression stating that “if (one) had asked people what they wanted, they (…) would have responded: faster horses” makes sense with our Protection section, on which we innovate, iterate and want to propose solutions that do not exist anywhere else among our competitors. But on insurance, we must cover our policyholders as best we can and meet their needs above all.

Does it take time? Of course! Of course!

Is there not a risk of being drowned in user suggestions, including the most eccentric ones? Not really! The more we think about a problem, the more solutions emerge.

Beyond that, what really stands out for me is the privileged relationship we have been able to maintain with our users. Little by little, we are building a movement, a community of people who, until now, had only seen insurance as an obligation. If we can change the insurance together and make it more beneficial, it will be a real gain, and not only for the Product!

Acknowledgements 💙

A huge thank you to our 200 users who took part in the test. Thank you to them for taking the time to download the application, for playing with it, for analyzing it in detail and for sharing all their opinions with us.

Thanks also to the people without whom this application would never have been possible: Amélie and Loris, who worked hard on the development to meet the app’s launch deadlines, and Emmanuel and Quentin, our great Product Designers who brought their pretty graphic paws.

‍

Originally published at https://fr.luko.eu on March 30, 2019.