Two Mobile Mistakes Costing Companies Millions of Dollars

Every year, Google holds a conference focused on sales conversions at their EMEA headquarters in Dublin. This year’s theme was “converting mobile consumers” and Craig Sullivan, a self-declared ‘optimizer of everything’, delivered a humorous, yet rich talk about frequent mistakes and missed A/B testing opportunities that are killing companies.

Sullivan is the former Optimiser in Chief at Optimal Visit and an experienced speaker on product optimization and A/B testing. In his talk at the Google conference, he identified the biggest digital slip ups that are costing companies millions of dollars, and shared actionable steps to optimize platforms and mitigate that large risk.

1) Ignoring the macro significance of micro patterns

When building products, a common critical mistake is overlooking the little details that contribute to the user’s experience with it. As we’re experiencing rapid growth in the digital industry, we should look to learn from an industry that succeeded in meeting ever-evolving customer demands: manufacturing. Popular strategies such as agile, waterfall, lean, and Kaizan were all born from the boom of the industrial era when car manufacturers had to learn to work efficiently to meet consumer demands.

Just as they did in the industrial revolution, new efficient ways of building great products that meet evolving demands must emerge in this digital era. For example, retailers rely on what Sullivan refers to as ‘micro patterns’ such as lighting, clean bathroom or even the spacing between racks to convert shoppers into purchasers. Without optimizing these little details of their in-store experience, it would be difficult to get shoppers to browse and complete purchases, no matter how great the products being offered are.

“All of these thousands of tiny little details must work without fail for you to have a decent retail experience”

Likewise, digital platforms cannot ignore the potential impact of seemingly small components like logins, navigation, sizing or form fields. Sullivan emphasizes that optimizing micro patterns like copy, buttons, placement of content or onboarding flow has a macro effect on the app’s success.

2) Ignoring broken stuff

In the rapidly evolving digital environment, everyone gets so caught up in trying to make bigger and better things that they miss a huge opportunity to move the needle.

“If your front doors don’t work on your shop, [no one is] going to buy anything… An experience is only as good as the crappiest part.”

Rarely do mobile companies talk about getting rid of features that aren’t converting customers. While it’s true that persuading and motivating your users or making your app easy to use can make an impact, Sullivan observes that eliminating broken elements that create roadblocks for users will make the biggest impact. In companies he’s worked with, Sullivan observed millions of dollars in ROI achieved through fixing a few bugs on their platform. In fact, one company spent less than 5 days fixing defects which achieved a 91,150% ROI!

Another company discovered that their bugs were costing them more money than their annual budget for the entire IT team. Evidently, fixing or getting rid of bugs and defects on your product is a high-reward actionable that could save or make your company of millions of dollars.

Never blindly copy — Always validate hypotheses!

How do can we stop making these costly mistakes and start optimizing like an expert? During his talk, Sullivan outlines 10 A/B testing and 11 Optimization power-ups, but the most impactful one is

“Stop [blindly] copying… It’s okay to copy stuff, but you have to validate it!”

Even if the outcome of implementing a cool new idea seems obviously bound for success, Sullivan emphasizes how customers, site, advertising/ traffic and many other factors differ from company to company. Without validating the hypothesis on your platform with your users, all hypotheses are unproven because what works for one company may not work for another, as Michael Ferreira from Booking.com pointed out. Always hypothesize, test and validate to achieve true optimization.

How are you supposed to know if an idea is worth testing? You need to build a proper hypotheses. With the help of experienced testers from companies like Booking.com and Skyscanner, Sullivan shaped the following hypothesis kit:

If the idea cannot be presented in this format, it likely will not create effective tests, which signals that more details must be iterated before the experiment can be conducted. Like a hypothesis of an essay, a strong test hypothesis is the crucial foundation to a strong experiment with actionable results.

Stop Making Mistakes; Make Changes!

When it comes to creating a competitive product, many companies aren’t doing enough. They are leaving big gaps in optimization by overlooking small details or bugs that seem negligible to their app’s success.These mistakes are costing companies millions of dollars, but luckily there is a solution to mitigate the risks of such costs: test everything that goes into building your product! Validate all product hypotheses, regardless of how confident you are that it will be successful. Every company’s unique set of customers, traffic, business model and product makes test validation invaluable in creating success for your organization. Moreover, if you’re not sure how to tell whether your hypothesis is strong enough, refer to the simple and advanced hypothesis kit that Sullivan developed with the industry’s expert testers.

Taplytics is a product experimentation solution committed to helping teams build digital products for their users and get the most out of their technology budget. By encouraging companies to execute, listen then iterate, Taplytics helps them validate product decisions with live user data to prove that they are making a positive impact.