Launching a Vitamin with Confidence
Author’s note: a version of this article originally appeared on mabl.com as “How mabl helped Ritual launch a new men’s multivitamin”
Quality plays a critical role in how many enterprises operate and build their products. At Ritual, quality is at the core of our brand and a big part of the reason we built our company in the first place.
Ritual is a health-meets-technology company that reinvented the multivitamin from the ground up. We pride ourselves on challenging the status quo with our traceable, high-quality, and evidence-based nutrients backed by the first visible supply chain of its kind. Bottom line: we are obsessed with quality — in all facets of our business. Our multivitamins, of course, but also our website and customer experience. Since we sell our products direct-to-consumer on Ritual.com, providing a seamless customer experience that both informs and delights is a cornerstone of our company.
Preparing for a BIG launch
We recently launched our highly-anticipated line of men’s multivitamins that expanded our reach and relevance. To prepare for this launch, stakeholders from every department needed to be on the same page to ensure a successful roll-out that lived up to our high expectations, including our software development team. Our technology stack doesn’t just revolve around ecommerce — it empowers every individual at Ritual to move quickly and autonomously, iterating on a myriad of initiatives across the company to turn our vision of this launch into a true moment. We knew technology would be at the core of this launch and we needed to be prepared.
I lead our small-but-mighty QA team here at Ritual. While many associate digital quality assurance with software testing (and you wouldn’t be wrong), we operate with much greater latitude than many traditional QA groups. Whether it be testing transactional and marketing emails or deploying new API functionality and monitoring uptime, we’re responsible for testing efforts across the entire digital product team. Quality is a team sport, and we like to think of ourselves as QA coaches. We help define what quality looks and feels like on our ecommerce platform and socialize those standards across the entire organization so that other departments can operate as independently as possible with a high level of confidence. We also provide the tools the rest of the company needs to uphold and quantify those standards. In the context of our men’s product launch, I knew we would have to do a lot of prep work in order to effectively deliver on one of our biggest product launches to date.
We had some issues to tackle right out of the gate. For starters, our entire company has been operating remotely due to the COVID-19 pandemic, which introduced an extra layer of complexity to many of our cross-functional processes. In addition, we were faced with rapidly evolving schedules and priorities. The timeline given to the digital team was aggressive and new functionality required extensive testing at every layer of our tech stack, which meant we were constantly trying to keep up. As the customer journey evolved with feedback from user testing, some architecture changes made (or even planned) became irrelevant. To handle the rapidly evolving situation, we needed to strategically balance in-depth exploratory testing with automated tests that were resilient and could stand up to changes on-the-fly; enter our intelligent testing platform, mabl.
Scaling from 1 to n with 2 clicks
As frontend features began taking shape, our backend team was hard at work integrating financial software that was a prerequisite to the product launch. This particular integration required a significant amount of data and stress testing to evaluate effectively and time was of the essence, so the first thing we did in was scale up our test runs. We needed to test the system’s concurrency limits to see how much transaction volume we could handle at any given time. What may have taken days or even weeks using traditional automation took 10 seconds in mabl — we simply increased the test run multiplier on our checkout regression suite from 1x to 10x so each run would simulate 100 customers checking out simultaneously. This helped us identify resource bottlenecks in our configuration that may have caused days-long delays if shipped to production and we ended up increasing concurrency to handle transaction volume as a result of mabl’s findings. After validating expected results against the data created by the test runs, the integration was deployed successfully, paving the way for our men’s launch a month later.
Mabl’s integration with Segment allows us to prioritize test runs based on changing site traffic and page complexity.
“One of these things is not like the other”
As we worked through the launch’s evolving priorities, we leaned heavily on mabl’s visual change detection to gracefully handle copy updates that were being updated by our product and marketing teams. This AI-driven screenshot comparison runs passively in the background when running mabl tests, alerting us quickly to unanticipated UI changes and proactively identify issues early in the development cycle with minimal effort from our small team.
Mabl’s visual regression detection flags an issue where the yellow circle was not rendering on a product page.
As work began shipping behind feature flags multiple times each day, we found ourselves frequently context switching between testing feature work and testing updates made by other departments; no easy task for a team of two. When copy changes were made by other internal teams, they were often identified by mabl visual regression detection and were able to stay up-to-date on changes even when we weren’t explicitly notified. This inadvertent “shadow testing” became our canary in the coal mine (code mine?) and allowed us to focus on core functionality while giving the teams making the changes another layer of confidence. The ability to leverage branching logic in mabl meshed extremely well with our deploy/release cycle and made testing incremental feature roll-outs much easier because our feature flagging platform (LaunchDarkly) integrates seamlessly with mabl’s plan (test suite) configuration settings.
Mabl highlighting the visual delta between a previous cart feature and the current state in the nav. We were able to amend the feature flag logic based on this finding.
As we worked to meet the looming launch deadlines, we noticed turnaround time from design to engineering was becoming exponentially shorter at a rate we hadn’t previously seen. A thorough deep dive into sprint metrics compared to our mabl initiatives during the course of the project revealed that bugs identified by mabl were fixed at the earliest stages of development in almost all cases, significantly reducing the amount of developer time and effort that would have been required if these issues had been caught behind production flags. The combination of all of these features working in tandem helped us stay ahead of a fast-moving project.
I’m happy to say that this work culminated in a successful roll-out of our new Essential for Men’s multivitamins. At the end of the day, mabl was able to scale with the magnitude of our men’s multivitamin launch and enable our small team to do the work of one double its size. Throughout this launch process, we found that mabl acted as a force multiplier. Although there are only two dedicated testers on our QA team, mabl helped us get a lot more done — and be confident in our approach.
