The user research behind AWeber’s new mobile app: How we all became product owners

In the fall of 2015, AWeber’s mobile team faced a huge challenge: building an entirely new app from the ground up that our 100,000+ customers would want to use.

Though we had a path we wanted to go down, we didn’t have a clear idea of what the app should do. We were starting from scratch. Essentially, as a product team, this put us in the admittedly fun position of being a startup even though we existed inside an established company.

As a “startup,” our team was the perfect testing ground for a new way (well, new to us) to develop a product. Via the lean startup model, we knew that staying inside of a customer feedback loop was going to make the end product much stronger, which we all wanted.

We took this idea one step further, because we decided that everyone on the team — developers (devs) included — were going to be exposed to user research in real time instead of hearing about the results afterwards. And this was key, considering we wanted users to steer the product’s focus. We steadfastly maintained this user feedback loop throughout every stage in the app’s development.

The results exceeded our expectations and led to everyone on the product team feeling like a product owner. I’ll explain why.

The cycle basically looked like this:

drawing board → prototype testing with users → drawing board → development → alpha study & development → beta study & development → MVP launch

(It won’t end there. Curate MVP just launched, and it’s just the beginning as we continue to listen to our customers and add features.)

Prototype Testing

To start the project, we did prototype testing for our first plausible idea.

The concept for our prototype mobile app arose from the drawing board. We’ll call this first app concept “George.” (It doesn’t really matter what George did, as you’ll see in a minute.) We quickly turned George into a set of wireframes which were then dropped into a prototyping app called Marvel. We put the prototype in front of AWeber customers.

The prototype gave our customers just enough of an understanding of George in order to offer a reaction. We tested with customers who live all over the U.S., so the tests were run remotely. We broadcast the live tests over big screen TVs in our office so that the team could see and hear the unfiltered user feedback as it happened.

(Spoiler alert: George failed miserably. No one wanted George. This is the beauty of a lean model; we were pivoting off of a cheap, 2-week prototype, not an expensive, bloated, fully functional app. Win!)

As a UX Designer, I did not have to spend the time writing up or packaging the prototype test results (which may or may not have included inadvertent biases of mine), a step which can take several days and leave a team of devs waiting. Nope; we all saw the user reactions live without biases, gathered as a team, discussed what we heard, and were onto the next step lickety-split.

Alpha Study

After George, we did a lot of brainstorming and product concepting. We used the user feedback we had gathered with George and developed an alpha concept, called Henry. We felt confident it was something our customers wanted (that’s the power of research!), and fortunately, Henry became an instant hit.

We pushed Henry onto our coworkers, as is the tradition of an alpha study to keep it limited to friends/families/associates. Henry was crude and raw, but its purpose and value was obvious to all those who jumped into alpha testing (more on what Henry did in a bit). Tons of feature requests rolled into our team, a sure sign of high engagement. All feedback was distributed team-wide in its raw format, then compiled, prioritized and discussed. Development continued concurrently as alpha feedback came in and MVP started to come into sharper focus.

Beta Study

Our alpha product, Henry, rapidly evolved to become our beta product, Curate. The concept behind Henry, which became Curate, is that AWeber customers can quickly build a curated newsletter by using the native sharing functionality built into their mobile devices. See content you want to share with your subscribers? Share it into Curate!

As we opened the beta study for Curate, we really accelerated user research. We invited hundreds of AWeber customers into the beta pool, opting for a closed beta study so that the feedback was manageable, and strictly coming from engaged AWeber customers.

As the UX researcher overseeing the beta study, I wanted to find a channel through which we could obtain user feedback in a way that allowed the entire product team to be involved, but kept the public blocked out. After considering a few options, such as a Facebook or Google group, I settled on Slack. (In a future blog post, I’ll go into deeper detail about my experience using Slack to collect UX research feedback.)

With Slack, I created private rooms and invited the beta testers in. We also included every member of our product team, as well as individuals from the design and marketing departments. I really wanted everyone at AWeber who was going to have a hand in Curate’s product launch to have full access to real time user feedback.

It was beyond exciting to see user feedback start arriving in the Slack room where my entire team and I were reading it at the same time, regardless of whether it was a Saturday night or a Tuesday morning. Devs were immediately starting direct messages with me in Slack in order to help address customer questions or bugs. Discussions around the office were often sparked by customer compliments or criticism. Prioritizing app features became much easier because we were all advocating for what customers wanted, not what we each individually wanted.

The End Result

The decision to rely on user feedback in each step of the product development cycle, plus including the whole product team in that feedback in real time meant we were ALL responsible for internalizing and understanding what users were saying and applying it toward the product — and I don’t think it could have gone any better.

In this way, we were all product owners. Plus, it eliminated the time-suck of having the UXD translate the results into some kind of digestible format for the team, or baking it into wireframes and UI where it might spend unnecessary time being challenged and defended later.

Instead, everyone was able to set their ego aside and feel empathy for what our customers were telling us. The internal team conversations were ongoing, lively, passionate and empathetic. Sometimes the entire room would burst into conversation the moment something provocative or positive came into the Slack room.

As a UXD, I can say that it made my job more rewarding. When I asked the devs how it changed their work on the project, they felt the same way.

In fact, one developer said, “It made the connection to our customers very real. It was extremely motivating to hear that customers liked what we are creating. And when customers identified needs that the app didn’t fulfill, I feel it really enabled me and the team to put this information to use first-hand in a quick turn around.”

But most importantly, putting real time user feedback in front of the whole product team made product development for Curate faster, easier and reduced common product development friction. Ultimately, it made Curate better.

Note: This is a post in the AWeber Engineering Blog series “Behind the App,” which reveals important pieces of the behind-the-scenes story of our latest mobile app Curate.

Curate is available for download on both the Apple App Store and the Google Play Store.