Case Study: How Fixing Email Verification Issues Restored Our 100% Email Verification Success Rate
Email verification during sign-up — a simple action that should be muscle memory in this digital age. Almost every platform requiring sign-up comes with a verification step, be it a link or an OTP. Yet, as familiar as it might be, here’s a case study of how we uncovered a series of issues creating the ‘perfect storm’ resulting in a below-average task success rate (TSR), which led to a low conversion rate of users actually making it to experience our platform.
Problem
It was an exciting night celebrating the official launch of our terrific platform. Marketing engines were on, and users were signing up daily. Two weeks in, numbers looked good, but we discovered a problem: the conversion rate into actual platform users was strangely low compared to sign-ups. Users weren’t getting past the verification step to use our platform. Something was wrong!
Our goal
Uncover the root cause(s) and release a hotfix ASAP to bring the TSR of verifying an account to 100%, thus funnelling all users to experience the platform.
Plan of action
In the war room, I (UX Lead and Product Owner), co-founders, and lead developers discussed root causes and devised a plan:
- Reach out to affected customers to find out what went wrong and offer a solution.
- Comb through telemetry to connect any dots.
- Run a remote usability study to uncover insights.
- Explore tech solutions and restrictions.
The findings allowed quick fixes in the next development sprint. Read about these findings in the chart below:
As the UX lead and product owner, I led the remote usability study and consolidated insights for wireframes of an improved verification page design.
Solutions
From the insights gathered, we can easily group the main user scenarios as follows:
- Users in the ‘happy flow.’
- Users finding verification emails in junk/spam.
- Users needing to resend the email.
- Users forgetting their signed-up email.
- Users mistyping their email.
- Users verifying emails on mobile, then continuing on desktop.
- Users not receiving verification emails due to email firewall.
By studying the scenarios above, it was clear what we had to change on the verification page. However, the challenge was cramping so much information into a short and simple page.
After some ideation, I deferred to using a natural language approach, where I designed an accordion-style verification page akin to FAQs. Users only see relevant info, reducing confusion.
The solution, well-received by the team, balanced usability and development resources. We skipped prototype testing as it was not possible to replicate a real-world email verification. We rolled out the hot-fix in two weeks, and results were positive. From our CRM, we observed that after a week, email verification success rate — measured by new users entering the platform for the first time — improved to 100%. In other words, there was a significant drop in user drop-offs after verification, which was a resounding success compared to our initial findings.
Learning and conclusions
Reflecting on this challenge, I’ve learned that:
- Designing a user flow that accounts for all exception scenarios is impractical. Learning from past experiences, we aim to address as many as possible during development.
- Users commonly verify emails on their mobile phones, regardless of the sign-up device, as smartphones have become powerful tools for messaging and emails.
- Acknowledging the finite attention and memory capacity of users, it’s common for them to forget the email address used for sign-up.
- Human errors like mistyping emails are expected. Ensuring visible system status and providing clear recovery instructions is essential.
- Adhering to Jakob’s Law, we emphasize not reinventing the wheel for common procedures like email verification. Familiar and intuitive designs align with users’ muscle memory in this digital age.
What could have been improved?
There was plenty that could have been done better in hindsight. Notably, I know much better today about UX metrics measurement, which I would have applied to this case study if I could go back and improve on what I did. We could:
- Implement event tracking for clicks on the 3 accordions as an approximation of users encountering an issue with that scenario.
- Measure the number of clicks on the buttons introduced in the 3 accordions. This will give us a clear indication of how common those scenarios occur.
- Measure the time on task required to complete the sign up form, verify their email address, then entering our platform. The shorter time spent on the email verification page, the better.
Thereafter, we can further diagnose and improve the sign up and verification process with proper UX metrics.
In conclusion
thoroughly enjoyed the discovery and design phases to resolve the uncovered issues. I am also delighted to see how the solution is both simple and elegant. As a team, we collectively addressed the issues, rolled out a stable hot-fix in under three weeks, and successfully achieved our goal.
A key takeaway for me is recognizing that we cannot cover all bases on the first attempt. As the saying goes, Rome was not built in a day. The crucial lesson is how, as an organization, we adopt an agile approach to collaboratively resolve issues with speed and grace.
Article written by me, proofread by ChatGPT.
