Case study: How I designed an $83K revenue MVP in just 4 days
The perfect solution is not always the best solution
Logistics are one hell of a hurdle when the entire European Union could fit inside your country’s border twice over.
This is OFF Premium’s ordeal: a multi-million fashion marketplace working with hundreds of stores spread throughout Brazil’s 8.5 million² kilometres, resulting in shipping costs of up to $400 for a single shirt!
Customers were furious on social media, claiming they were abandoning packed carts and visiting competitors — and I honestly believe they should. 😅
The Problem
- Shipping used to be free but commercial policies suddenly changed, making prices go from $0 to $XXX out of nowhere. (Clients went mad!)
- Built on VTEX, OFF’s checkout process is severely “code-locked”, affording minimal CSS tweaks. (The best solutions take months to develop!)
- A consistent buying process that hadn’t changed for years, so any changes would heavily impact the hundreds of thousands of users.
- If that wasn’t enough, we needed to fix it before Black Friday, otherwise it could be a disaster — and we had no more than 4 days to do so!
No room for failure: only guerrilla UX Research and development remain. (Seriously, talk about Agility!)
The Design Process
Nielsen proposes you test with 6 users (per user flow) before making any design decisions so you avoid biased results.
Ain’t nobody had time for that. Once again, 4 days, so we had to go extreme!
What I call “guerilla research and development” is a quick round of test > redesign > test > redesign > while development occurs, regardless if you have statistical significance.
This sounds pretty insane(risk of technical debt + design bias), but there’s an argument for that: all humans are affected by the same cognitive biases, so if a user has a given problem, it’s likely that many others also do!
Also, remember I researched the subject before? I had enough data to say this MVP would work better than the original design, so I defined some basic UI parameters the developer could work on regardless of test results.
Original Design & Task Flow
Hypothesis Statement
Dropdown carts should 1) only provide a summary of your clothes, and 2) clients must know how each item affects shipping price and its estimates.
Since VTEX’ dropcart is unable to provide detailed shipping, customers would have to test how each item affects pricing (by adding and removing stuff repeatedly), but that probably never happens.
A solution would be to remove the ZIP field, reducing the burden of seeing added items as a prelude to a purchase. The less customers see it as a buying process, the more likely they are to add more items — and the less likely they will be to remove them. We call this the Sunk Cost Effect.
The Sunk Cost Effect is a cognitive bias that makes people stick to choices they invested time or money in, for instance: if you pay $400 for a concert, a dangerous frozen road is not enough to make you stay at home.
And if you spend an hour adding items to your shopping cart, you also be less willing to remove stuff since you have already grown attached to it.
Final Assumption: less cart abandonment and more items purchased.
New Dropdown Cart UI
As I told you in “how Tesler’s Law makes your designs fail”, every system has a given amount of complexity you must respect — so no matter how good your new design is, it will fail if you don’t account for Tesler.
That’s why we simply couldn’t remove the ZIP field from the dropdown cart. Users would feel lost and potentially mad (“Not only the price is abusive, now you are hiding charges from me?!”), so I had to try a different strategy:
Replacing the ZIP field with a temporary friendly warning telling users all shipping info can still be found on the next page (cart), as they always did.
The warning is pretty bold (heavy black) but it’s intentional as it couldn’t be missed and, fortunately, all testees noticed it and proceeded to “checkout”!
Development instruction: I requested our developer to implement it right on the first day as “hiding” those bizarre prices would be enough to get people moving beyond the product page.
New Cart UI: before testing
Relying on previous research I had done and lots of conversations with devs, we came up with a new design that I was “sure” would work. Here you go:
Although the “Best price” box remains (we couldn't remove it), we now have a set of cards explaining it. Each card represents a package (yes, that’s what “many estimates” meant) so users can see why packages won’t arrive at the same time and how prices vary.
I bring another bold warning to ensure people notice the changes. This one explains how orders are divided, but even if you don’t click on it, it draws your attention to the set of cards that makes shipping info much clearer than before.
As I told you, this is not the best solution (the final one says items’ location, shipping company, etc.) but that’s what VTEX’ code let us play with.
Development instruction: “Start working on this set of cards with pricing and estimates. Oh, and we might need a ‘remove’ button, so prepare the code. Regardless of test results, I’ll find a way to make these cards work.”
New UI after testing: 1st & 2nd user
Remember we only had 4 days to develop this? That’s why I designed in-between tests, saving all the time that would be wasted recruiting users.
Task: “Please, tell me the shipping cost of [given item].” (Placed in the last card, so users must swipe to see it.)
The first testee (an eComm heavy user) simply didn’t notice she couldn’t swipe right to see more — and as I told you a hundred times, we only had 4 days, so if a user’s problem is all users’ problem!
That’s when I remembered 93% of human vision is dedicated to border detection to mitigate Ensemble Coding. By switching from dropshadows to black borders, I hoped our users’ eyes would never ignore the hidden cards— and fortunately they didn’t!
Last, you see a new shipping text on the centre. Having “1/3” on the left and “shipping” on the right required a bi-directional eye movement (top-down, then left-right), thus compromising scannability.
Development instruction: “add a border to it and make our dropdown cart’s warning work as ‘proceed to cart’ .” (Yes, I user tried it repeatedly.)
Last testees: Task: “How could you make it more affordable?”
Here’s a puzzle: if I add a big “remove” button to every package, I’m nudging users towards emptying their cards. If I don’t, then I’m providing a bad experience as people will have to find their way around it.
I only made up my mind after noticing users didn’t realise they could remove an expensive package, saying “I’d abandon the cart altogether.” (That’s exactly what we are fighting against!)
That’s why I added that discrete “remove” button on the upper-right corner. Yes, we have the bi-directional gaze once again, but having the most relevant info on the left ensures users won’t miss it — while the light-grey “remove” is only noticed by those looking for it. Now we have it!
(Tests continued successfully after the 3rd testee, so let’s skip those.)
A/B Test Results: +$83K after a month
Mobile resulted in a $54 thousand increase, led by an additional 5% conversion rate. Just as I predicted, average purchases rose by $30 as people didn’t need to remove each item to find out why shipping was so expensive.
An unexpected result is that more carts were abandoned and yet OFF sold more.
I’m not sure, but it probably has to do with VTEX having two shipping pages: the one you just saw (cart) and a second one right before the payment page. This second one has more info than the former, but it’s also so intricate that many users simply quit when they reach it.
So although disclosing shipping info on the cart made more people quit it, those who continued felt so safe that they ignored the confusing pre-payment page, dropping its bounce rate from 17% to 6%.
The desktop version could only run for 2 weeks but it was enough to generate $29K revenue and a +15% conversion rate. It seems it would beat the mobile version if it ran for a month, and that’s awesome!
The secret here is that OFF’s desktop site had no cart (you’d go from dropdown cart to payment) because some stakeholders didn’t want it. That was the perfect recipe for disaster as customers relied on that misleading dropdown to estimate their shipping.
As you’ve noticed, I had to cut corners to get this working in four days but that was certainly worth it. (This required huge alignment between myself, POs and development, so I seriously discourage you from doing the same unless you reaaally need it.)
I’d like to thank my colleagues for this amazing run and also you, for reading this story until the end! ♥
If you have any doubts, please, share it in the comments and I’ll be glad to lend you a hand! =)
Oh, feel free to connect: https://www.linkedin.com/in/romarioverbran/
