What Happened When We Treated Our Checkout Redesign Like the Hunger Games
May the odds be ever in your favor, form fields.
DonorsChoose.org handles a lot of donations to classrooms. Last year, our checkout flow, which included viewing cart and payment page, handled over 2.6 million transactions (whoa!) — which is exactly why we needed to look at it with a critical eye.
We didn’t decide to redesign checkout because anything was broken or buggy with it, but because it is such a core part of our site. Since every single donor has to go through checkout in order to make a donation, we knew it needed to be as frictionless as possible. Our goal with the redesign was to strip away anything that might slow down or inhibit a donor from completing their donation.
Advocating to remove existing asks that stakeholders had grown attached to proved tricky, so I flipped the script.
Old Checkout: 2 page loads. 16 asks. A whole lotta scrolling.
Over the last 19 years, our checkout process had stealthily expanded: a new form field here, a new UI element there. Before we knew it, it had turned into this long scrolling behemoth. But advocating to remove existing elements that stakeholders had grown attached to proved tricky, so I flipped the script: we would start from a clean slate and put the onus on team members to convince us if they wanted to include any elements that they felt were essential to checkout.
In order to be a part of the checkout flow, elements had to survive the Checkout Hunger Games.
Only the strongest asks, or tributes, would survive and be allowed to thrive in our redesigned checkout.
The Checkout Hunger Games
In Suzanne Collins’ YA novel The Hunger Games, a resource-constrained dystopian nation forces a bunch of school children, aka “tributes,” to compete in a battle to the death until one sole survivor remains. Similarly, our checkout is resource-constrained (the resources, in this case, are donors’ time and attention), so we forced the asks to compete in a battle to the death. Only the strongest asks, or tributes, would survive and be allowed to thrive in our redesigned checkout.
The Tributes
We had 16 tributes. That is, we had 16 open fields or checkboxes vying for a chance to stay in our checkout redesign.
Some of these tributes, like credit card info, were absolutely required to make a donation. Other tributes, like asking for a donor’s birthday, were not required to make a donation but had the potential to bring donors back to our site. But would they add enough value to survive our redesign?
Rounds of the Checkout Hunger Games
We weighed the cost and benefit of each tribute through a combination of A/B testing and usability testing.
A/B Testing
We ran A/B tests on our existing checkout to hide any asks that weren’t required to make a donation. We wanted to see if removing these asks made any difference to donor conversion. If removing an individual ask showed no impact or positive impact to conversion, we’d remove it.
In A/B testing, we found that removing Credit a Friend and asking for a donor’s Birthday gave us a statistically significant increase in donor conversion by 1.0% to 2.8%, depending on the donor’s device type. We analyzed the revenue that those asks generated from reengagement marketing campaigns, but they didn’t generate enough donations to classrooms to justify the difference in conversion.
Zip code, however, proved it could survive. Asking for a zip code helped us recommend better classroom projects to donors so that a significant amount would come back to our site to donate again, so it got to stay in our redesigned checkout.
Removing Display name and Create password made no significant difference in conversion, but we erred on the side of not keeping asks even if they seemed “neutral” because we were confident that even if the removal of a single ask didn’t appear to have a positive effect, the simplicity created by the removal of a bunch of asks would have a net positive effect.
Usability Testing
The value of some fields were harder to quantify, so we ran some live usability tests to make a more informed decision about their survival. There were some tributes that might not impact conversion but perhaps brought value in other ways.
We tested an ultra paired down version of our payment form and discovered that many donors weren’t comfortable with moving forward with their donation if they didn’t have the option to mark their donation as anonymous. Similarly, many donors didn’t want to donate if they couldn’t dedicate their donation.
We also learned that though the promo/gift code field was useful for some donors, many didn’t need it and skipped over it, so we considered how we could minimize its prominence.
Also as a result of our research sessions, we validated the fact that we could get away with combining some fields to reduce the amount of space they took up on the screen. First name and Last name became one Full name field. Similarly, a credit card’s Expiration month and Expiration year became one MM/YY field.
Crowning the Winners
After 8 rounds of testing with dozens of participants and A/B testing almost every field to ensure we’d only help our checkout flow, we reduced our cohort of tributes from 16 to only 10. I know it’s not one final winner, but [***HUNGER GAMES SPOILER ALERT***] it’s just like when Katniss and Peeta both survived the first games. [***END SPOILER ALERT***]
We took these 10 and redesigned them into a simpler experience:
The first thing you’ll notice is that what was previously 2 different page loads is just one — and we got rid of all of that pesky scrolling. We were able to simplify checkout by removing some asks altogether (like the password field, which we removed in favor of a secure and passwordless sign in) or reducing the visual weight of others (like putting Promo/Gift code behind a link).
A New Home for Tributes Who Didn’t Make It
Many of the tributes who didn’t survive the Checkout Hunger Games found a new life in a different district — or, elsewhere on our site. We recognized that some of them still brought value, even if checkout wasn’t the right place for them.
We moved some of the marketing asks to later in the flow, once a donor has already placed their donation and in places that still feel natural. For example, we moved a donor’s “Display name,” which we previously asked donors to fill out in addition to their first name and last name, to the page where donors write a message to the teacher. Now donors can see how their name appears in context. But this now appears after we’ve already processed a donor’s donation.
Then there’s the birthday ask. We moved it into a brand new module we created for the Thank You Page that asks donors getting-to-know-you questions that are fun and easy to fill out.
And these questions aren’t just there to be delightful — we use donors’ answers to send them project recommendations every month going forward.
The Results of the Checkout Hunger Games
All of these changes make our checkout flow feel quicker and easier. Nowhere is this more apparent than in the mobile experience.
With our redesign, we succeeded in doing something that might seem like a paradox: through a creative mix of building out new homes for marketing asks elsewhere on our site, slashing some form fields, and combining other form fields, we were able to simplify the process and reduce the number of fields that donors had to fill out to make a donation, while simultaneously learning so much more about our donors than we were before.
The argument for having a simplified checkout lies in the numbers. Thanks to our redesign, we’ve seen a conversion rate increase of 8.25% and an increase in average donation size by 13.33%. Our mobile conversion rate increased by 6.61% with a mobile average donation size increase of 4.35%.
Thankfully, the result of our Checkout Hunger Games isn’t bloodshed (like it is in the novels). Instead, it’s more books, lab equipment, field trips, and more for teachers and students.
