Unlocking User Insights by Reaching Out Early

Getting a first taste of the market🍷 by talking to potential users👥, gaining insights into usability and market fit quickly⚡️ — to make improvements to the MVP before the launch😎.

A typical challenge for any app is to make it easy to use. When you develop something, step by step, you intimately know how the product works, where to look and where to tap or click. While it is essential to know your product well, this level of experience also makes you prone to overlook shortcomings.

Knowing about the risk of being too involved to see some obvious usability and product issues, we wanted to look at UnscrewMe with some fresh sets of eyes. For this purpose, we set out to ask a few friends for their opinions – not necessarily representative, but still insightful enough to help us improve UnscrewMe at this early stage.

Learning about the market and user preferences

Our goal was to look beyond the web app and also learn more about the market in general. So we decided to devise a semi-structured questionnaire as our initial test plan. The more open questions should supplement our market research and mainly focused on personal preferences and experiences. And a few well-defined tasks should generate actionable feedback about how we could improve the user experience going forward.

Questionnaire design in itself is a tricky subject. We wanted to gather enough information to gain deeper insights into how people organise their time and how they discover events in general. Looking at UnscewMe specifically, we also aimed at identifying how intuitive our web app can be used right now, and what new features users might value the most, what features would make users come back.

At the same time, we wanted to keep the list of questions and tasks short, yet also open, allowing our test subjects to freely express their opinion on a range of topics. To make the sessions easier to schedule and less intimidating, our objective was to be able and finish a testing session within half an hour. As a result, balancing the granularity of the questions and tasks with the time available was critical to collect the user experience data we needed.

We ended up with a set of questions and tasks such as “How far in advance do you usually plan for events?” and “Please find the next tasting you could attend.” to gather some information about user behaviour and usability.

Following our standard approach to writing, we treated the test plan no different to any article we publish and went through our questions and tasks with two friends – kind of like testing the test plan!

Selecting early testers

After we made up our minds about what we wanted to ask, we had to look for suitable testers. This would be our first formal test situation, so we did not want to ask too many people for their feedback. We also thought it might be sensible to start with people we know, and who we know are at least a little bit interested in wine.

We put together a list of potential testers, including people who signed up for our mailing list. To create a friendly and relaxed atmosphere, our idea was to do the testing sessions not in a lab environment, but instead at a wine bar or cafe – matching the theme of the app. One incentive for our early testers: the wine we drink is on us 🍷! (Or the coffee or whatever they might want to have.)

Doing this user testing was really important to us, to check how our product addresses the needs of real people, besides ourselves. But we also did not want this testing phase to hinder further development, so we limited the number of testing sessions to three to ten, depending on how easy it would be to find testers, willing to be entertained for at least 30 minutes.

At first, we were surprised about how easy and quick it was to arrange two test sessions. Unfortunately, additional test sessions turned out to be slower to set up. But two was already better than none, so we went ahead with the valuable feedback we got.

Analysing the user feedback

As a first step after listing the main feedback from the user testing sessions, we sorted the feedback into four categories with a decreasing level of specificity:

• change requests – for feedback that addresses details that could be changed immediately, possibly even with just little effort
• feature requests – for feedback outlining new features, which might be entirely new or already be on the roadmap
• usability feedback – for comments about the usability in general, which could be used to write up or expand change or feature requests
• generic feedback – for everything else, be it about the web app, the scope of the service or even the business model

After sorting all into the four buckets, we reviewed the four lists to highlight which were completely new ideas (💡), which were improvements or new features we actually want to implement (👍🏻), which features need more research (❓) and which comments we might decide to ignore (👎🏻).

Finally, we looked at all the feedback we wanted to implement (👍🏻), and started to integrate these items into our existing roadmap. Most importantly here was to define what we want to implement immediately as part of the Minimum Viable Product (MVP), and which improvements we will be adding in later releases.

Once all feedback had been analysed, categorised and sequenced, we could start to create individual “issues” in GitLab. This was important to make sure we won’t forget about the feedback by the time we reach the “milestone” which we assigned to a release, where we wanted to implement the features and improvements.

Gaining new usability insights

After we have outlined our process, it might be interesting to also look at some of the insights we have gained. So, one pattern was that our testers liked to have a closer integration with Google Maps and preferred the native app over the mobile web app.

Another common theme was the mobile user experience. While we did optimise the mobile web app design, we did usually start designing for a bigger tablet or laptop screen. In a second step we then adjusted that design to make it better to use on a smartphone. As it turns out, all our testers would almost exclusively use UnscrewMe on their phones. Based on our article about preparing for user testing, we perhaps should have assumed this mobile-focused user behaviour from the beginning.

Starting with a mobile design and adjusting that to a bigger screen might be the better workflow from now on. This could be reflected in many small changes, such as larger buttons, bigger font sizes in general and similar adjustments.

Possibly related to this approach, one tester in particular would like a more colourful design and would appreciate a more advanced, yet simple visual presentation. While we want to stick to our minimal layout, we do agree that there is some room for improvement, to spice it all up a bit.

A third improvement that has been suggested is that the search should be a bit more powerful. For example, in the release we used for the testing, users could not search for a month.

In general, searching and filtering should be simplified, to make specific kinds of tastings easier accessible. This is something we already have in our roadmap, but the user feedback might help us prioritising this feature.

Another feature we already had planned to implement was popular with our testers: calendar integration. Of course, this should work across different calendar apps, but a basic implementation should not be too complicated. And since we now know that users will like it, it will also be worth the effort.

Next, we planned to explore what it means to take a side project seriously and how we reflect on the business side of UnscrewMe. But we changed our mind and wrote instead about learning more about wine to build a better product.

(We started putting together the first part some time ago, we do not apologise for the long notes — you might discover some new wines you never heard of before.
One evening we checked out newly opened pasta restaurant Lina Stores in Soho and enjoyed a beautiful, easy-drinking white, the 2016 Roero Arneis DOCG from Batasiolo winery in the heart of the Langhe in the Piedmont region in Northern Italy. Then we switched over to red and were delighted by the intense red fruits of the Barbera d’Alba DOC, also from Batasiolo winery.
Changing our desk for something else, we wrote up the second half of the draft on holiday in the Bulgarian🇧🇬 city of Varna by the Black Sea, after we had evaluated some user testing sessions. Great flat whites from CoffeeCup fuelled the writing.
But of course, we also took this opportunity to venture into Bulgarian wines. At the excellent The Sea Terrace restaurant with an unrivalled view over the Black Sea, a tip by my Airbnb host, we followed the recommendation of the staff and had a local white wine, a 2017 “White Story” Varna Misket from Staro Oryahovo Winery located by the Black Sea Coast close to Varna. Varna Misket is a Bulgarian aromatic grape variety and the wine had a fruity nose, not unlike other Muscat grapes, and tasted refreshing with notes of green apple, grass and a bit of lemon. Later, we moved on to red and had the 2016 Pinot Noir by the Tsarev Brod winery — another splendid example of local wine production with notes of cherry, perfectly balanced.
More of a surprise was the 2017 No Man’s Land “Lava Rose” from Damianitza Winery near the Southern border of Bulgaria we had at the rustic Staria Chinar in the Port of Varna. Boiling in the sun, the chilled rose with its slightly fruity nose and notes of strawberry and a bit of lime is made from regional grape varieties Broad Leaved Melnik and Early Melnik amongst others and seemed like an ideal summer drink.)