Giving users a seat at the table
If you want to build products users love, get them to help you do it.
Asana recently launched its first native Android app and we relied heavily on our users throughout the development process. This started out of necessity: nobody on the team was an Android power user at first, so we didn’t trust our product instincts the way we may have on web or iOS. We had a vague idea of the features required to deliver a great app experience, but the details were fuzzy. So, we optimized our development process around learning — collecting and incorporating feedback from a special group of beta testers early and often. What resulted has shaped the way we approach product development at Asana. Here’s how:
Smaller MVP with room to grow
When we initially built the product roadmap, we knew we had a lot to learn. Sure, some information was available to us from the beginning — user research, competitive analysis, lessons we learned on other platforms, etc. However, we acknowledged that lots of new information would become available to us later. For example, our team was small, but growing fast, and we expected to have a couple of Android experts weighing in on key decisions soon. We also wanted to factor in any advancements in Android technology or design that Google was planning to announce later that quarter. Most importantly though, we wanted to incorporate feedback from beta testers on the app as it developed.
So we negotiated down to a tiny set of features we knew we’d need in order to test the app ourselves. For us, this was simply an inbox that presented a feed of task updates. By only committing to building the inbox, we avoided the costs inherent in gaining consensus on longer term roadmap decisions. We chose to make those calls closer to the time of actually executing on them, when we’d presumably have more information available to us. We left plenty of room in our schedule to expand scope beyond inbox for that reason.
Design with the intention to iterate
Knowing that our plans were subject to change based on the arrival of new information, we approached design with a light heart. Without the pressure to “get it right” straight out of the gate, we were free to experiment with fresh ideas and take chances that otherwise may have felt too risky. We considered our initial designs to be learning opportunities more than anything else.
Our approach not only opened us up to design experimentation, but also allowed us to move faster than usual. We didn’t have to bother with redlines or other documentation work about work because we expected most things to change soon. Instead, our designer put together rough mocks and tweaked them in code alongside an engineer to save time.
Since we were starting with just an inbox, we were able to narrow our focus on one feature rather than designing an entire app end to end, which would have required a longer lead time ahead of implementation. Of course, we knew we’d eventually add more features, so extensibility was always top of mind.
Extend the roadmap incrementally
Focusing on one feature first also allowed us to gather meaningful feedback from users early. We started with inbox because we believed testing that feature internally would answer a lot of open questions we had about Android specific use cases and allow us to make more informed roadmap decisions. So, once the inbox was fully functional and decently polished, we distributed it internally.
Because the inbox only supported a single use case, the feedback we received was very focused and provided a clear signal for next steps. We heard things like, “Getting notifications about task updates is great, but I need to see the rest of the task for more context,” and, “Once I get a notification, I’d like to be able to create a task for myself to follow up.”
While it felt great to have more information than when we started, we knew that we still had a lot to learn from real users.
So, we used the internal feedback to extend our roadmap a bit further out, but decided to hold off on committing to anything longer term. We would only build the features necessary to get us to our next goal, which was an app we could distribute to a small group of external beta testers in order to collect more feedback.
We added a couple of features on top of the inbox before distributing the app more broadly, but we didn’t build them all at once. Instead, we added each new feature one at a time and then collected feedback before moving on to the next feature. This prevented us from building more than what was absolutely necessary to reach our next goal of external distribution. It also allowed us to ship as soon as we recognized that our feature set was sufficient — we were never stuck with a large set of partially built features that all needed to be polished before shipping. Instead, we had a small set of fully functional and polished features ready to go as soon as we recognized our next MVP.
Fast feedback loops with real users
Once we had the right set of features required to collect meaningful feedback from people outside our company, we kicked off phase one of our beta program. First, we invited a handful of friendly customers into our office to meet the team, eat dinner, and get access to the app. This was not only a great opportunity to build the foundation for longer term relationships we hoped to have with our Android community, but it also gave our team (not just user research, but designers, engineers, and PMs) a chance to see first hand how our work was received. It was an incredibly humbling experience to watch users stumble through certain interactions. We realized that many of the assumptions we were making up front were not always true and learned a lot about ways we could improve.
We kept in touch with these users over the next few weeks and incorporated their feedback into our roadmap. Our user research team used Get Feedback to send out surveys gauging reactions to design iterations and new features every week. We also installed Instabug to collect one-off bug reports and feature requests. The app was transforming rapidly for these users and they were excited to be a part of the process. Working in sync with them was rewarding for us, too, because we got to see the impact we were making in real time, rather than building in the dark.
Once the app was in good enough shape to collect feedback from a broader group of external testers, we kicked off the second phase of our beta program. This was similar to the first phase, but included a larger group of about 50 testers. We hosted another dinner, observed their first experiences with the app, and then continued to collect and incorporate their feedback over the next several weeks. We eventually expanded the group of testers a third and final time to maximize learning before launch.
Knowing when to launch
To help us determine the best time to advance our app from internal testing to a small group of external testers to even larger groups and then eventually to launch, we used a sentiment score that tracked our progress.
Each week we asked our users to rate their experience on a scale from “tough going” to “totally awesome.”
Initially we saw huge gains in the sentiment score after each update to the app; this wasn’t surprising since we started with just an inbox and then quickly added features that significantly improved the overall experience for the majority of users.
Eventually, we started seeing diminishing returns. At that point, our updates either marginally improved the experience for everyone (design polish) or significantly improved it for only a subset of users (editing capabilities for power users). For us, this was a good signal that it was time to launch. We knew we’d learn much more about how to make great leaps again once we got feedback from users at large.
Never stop learning
One of the best parts about giving your users a seat at the table is that you know they will welcome your launch with open arms:
Hundreds of beta testers were already getting a lot of value from the app so we felt confident that thousands of others would follow. We are by no means finished, though. Some of the most interesting insights from users are yet to come. Our plan is to build an even stronger relationship with our Android community, continue to incorporate their feedback into our lightweight roadmap, and incrementally improve our app to better serve them.
If you’re passionate about user driven development, join us.
