Building the case for overhauling Avvo’s account creation experience
I’m a UX data junkie — I want numbers, quotes, trends… whatever I can get. This served me well last year when I found myself needing to make a case for redesigning the account creation experience on Avvo. I was working to improve funnel abandonment issues for one of Avvo’s services and noticed a sizable exit rate at the registration step. My team and I were busy focusing on other, higher priority issues though and this problem just never made the cut for prioritization. We didn’t know what it would take to solve it and if it would be worth it.
Here’s how I steadily built a case for overhauling the Avvo registration experience using iterative quantitative and qualitative testing on a hypothesis developed from qualitative insights.
Funnel abandonment at registration
When you have a multi-step funnel in a product, especially one with a blocking registration step, you expect to have some drop-off along the way. At Avvo we have a couple messaging experiences that require a visitor to create an account. Last year, we had a standard account creation page: email, password, confirm password, and an option to sign in with social accounts.
The abandonment at registration wasn’t huge, but it was there with 31% of people exiting at the registration step:
Usability testing yields initial qualitative insights
To get some insight into why people might be dropping off, I ran our service through usability testing. The results were pretty expected:
- There’s a lot of hesitance to create an account on a website someone doesn’t know much about. Avvo isn’t a household name, and participants were concerned about how we’d be using their information (“I know I’m getting spam now”).
- Social sign-in causes concern in sensitive situations. Legal matters are often very private and our participants were immediately turned off by the thought of connecting their social accounts to a site where they’d be entering lots of sensitive information.
A/B test shows quick wins from usability insights
In the initial usability test, social sign-in was presented as the primary way to create an account. Since that was sending negative signals to visitors, I ran an A/B test with 2 variants:
- Moving the social buttons under the primary sign up fields
- Removing the social buttons completely
Removing the social buttons completely yielded the highest net gain, but simply moving them down also resulted in a fair lift. Moving the social buttons was the easiest approach from a development perspective, so we went with that and I moved on to other projects for a while.
A new hypothesis based on qualitative insights
One day, I was running the same service through usability for another project and I heard this little nugget:
“Password? I don’t want to enter something so… personal”
As with many comments we hear in usability, this took some interpretation. Isn’t your email the personal part? Or all the data about your legal issue? I still can’t tell you exactly what the participant meant by that, but it made me think about the password field. Do we need it? Lots of websites have been experimenting with new ways of authenticating (Medium, for example). I also remembered a competitive usability study we’d done in which a site asked for “contact info” (no password) as opposed to asking the user to create an account and participants breezed right through.
Based on all of this information, I developed a new hypothesis:
If we reframe account creation as “contact details” and remove the initial password requirement, we can decrease funnel abandonment and increase registrations.
In retrospect it seemed very obvious — someone using Avvo wants to be contacted by an attorney, they don’t want to create an account on Avvo.
Qualitative concept testing in stealth mode
As I was working on other projects related to this service, I would sometimes change up the registration step just to see how people would respond. I tested changes such as:
- Reframing account creation as providing contact details
- Just asking for an email
- Giving the option to enter a phone number
- Just asking for email and then taking them to a screen post-flow with a notice to set a password via an emailed link
It was amazing — participants cruised through these variations with no issue.
Our UX research team compiled findings from all of the usability tests we had run on registration over the year— my stealth mode tests, competitive usability tests, and tests on the existing service funnel. The pattern supported my hypothesis, and having months of studies backing up my ideas helped me buy the time to spend a few days building an A/B test to test my idea.
Quantifying the opportunity with an A/B test
I wanted to be able to measure what effect the password field and account creation framing had on registration completion. I created an A/B test where I split the registration form (email, password, confirm password) into 2 steps and reframed it as “contact information”.
For the sake of the test, if someone entered an email that was already associated with an account (a very low % of our traffic), it would kick them back to the old form with the email pre-filled. Otherwise they were asked to create a password:
The test indicated that if registration was limited to only asking for email, we’d see a 14% improvement in registration completion. Surprisingly, I also found that just splitting the registration form into two steps and reframing it yielded an 8% lift in overall form completion.
This was great — we could get a solid lift without changing our entire authentication approach which made it much easier to sell.
Releasing the MVP & seeing real results
My team was able to release the 2-step version of the registration experience from my A/B test with minimal technical re-work — most of the time was spent dealing with edge cases related to removing the social account registration options.
Our messaging funnel now consists of more steps, but the completion rate is higher.
This service is currently free to the public, so we left it here even though we know there are more gains to be had by removing the password step entirely. One of our other services has a similar issue though, with higher potential for immediate monetary gains. Redesigning that experience based on these learnings is now prioritized, with a chance that we’ll take it further to not requiring our visitors to set a password initially.
More insights needed for future plans
What we haven’t learned yet is how removing the password step from the initial registration to an email-driven transaction would impact people’s behavior after account creation. One concept prototype didn’t reveal any big concerns, but we don’t have much solid evidence yet.
Closing thoughts
Here are my biggest takeaways from this project:
- Sometimes you learn that your big idea isn’t worth pursuing (now, or maybe ever). If someone ever says “have we thought about passwordless account creation?” I can say “yes” and give them the data.
- Think about A/B testing as a tool for gathering data vs a tool for testing your solution. A 2-step form wasn’t what I had in mind as my solution initially, it was just a convenient way to gather the data I needed.
- Look at your qualitative data holistically — having the combination of usability testing, concept testing, and competitive usability all supporting each other and leading to the same hypothesis painted a stronger picture than a single study would have.
- Once you’ve got some hard data, you can flip the conversation from the cost of doing it to the cost of not doing it
This project was a perfect example of where qualitative insights can be paired with quantitative experiments to build a strong case for getting work prioritized. Tying the quantitative data to monetary gains is even better — I haven’t met any product managers who will leave money sitting on the table for too long.
