How TeamSparta Increased User Conversion to 85.5% by A/B Testing their Critical User Pathways

TeamSparta is one of the fastest-growing EdTech startups in Korea that provides online courses on coding for beginners. By adopting Hackle’s A/B Test from their early development stages, they were able to create better products backed up by data-driven decisions by focusing their resources on their core products.

Producing over 500 learners in a single round and accumulating over 100,000 cumulative learners, TeamSparta has experienced massive growth since they launched their first product SpartaCodingClub. TeamSparta adopted Hackle’s A/B Test at the end of 2020 when their team size only consisted of 10 members in order to seamlessly integrate the culture of experimenting with data in order to create the best possible digital experience for their customers.

(After using Hackle’s A/B Test) By the metrics :

• From 1–2 A/B tests a month to 1–2 A/B tests a week
• 500% increase in revenue compared to the previous year
• Average course completion rate 85.5% of learners

From 1–2 A/B tests a month to 1 every week

Although ideal, adopting a proper, data-driven, experimentation culture can be challenging. Before using Hackle’s A/B Test, they were manually coding and running A/B tests in-house. They also tried out Google Optimize to run A/B tests on various new features they wanted to release. However, they faced a few major problems that TeamSparta faced when they were trying to run experiments without Hackle’s A/B Test.

• Running A/B tests in-house took a toll on our limited development resources.
• From creating the organization’s database to analyzing results, a data engineer was required on top of a data analyst.
• Their alternative fix, Google Optimize, was too limited in features and they could only use the most fundamental A/B testing functions.
• Google Optimize was session-based, meaning that it was hard to get valid results from the experiments.

Although they wanted to run more, as a startup with limited resources they could only run 1 to 2 A/B tests per month.

After adopting Hackle’s A/B Test, they were able to run at least 1 A/B test per week and trust the results to make sound decisions.

Reducing Developer, Data Analyst Resources From Many to None

With the switch to Hackle’s A/B Test, even without the rare developer and data analyst resources, once they had a feature or product they wanted to test out, they could get the test up and running within the same day.

Using accurate, fast data to confidently adopt new decisions

The previous tool TeamSparta was using, Google Optimize, was session-based, where each data was tracked by session. This meant that it was difficult to obtain accurate data as it was hard to trace a certain action back to a user. Hackle’s A/B Test can identify users based on the device allowing for more sophisticated experiments as users were assigned a designated user ID. Due to this, members of TeamSparta trusted the numbers on the Hackle dashboard more than the ones on Google Optimize. Moreover, it took about half a day for Google Optimize to reflect the A/B test configuration to their digital service, but with Hackle’s A/B Test the changes made to any of the A/B test settings were applied to their service in real-time.

A data-driven team culture and improved product

Experimentation and making data-driven decisions define the team’s development speed and culture. At TeamSparta, this means that this emergence of culture in building and developing products with data followed naturally once Hackle’s A/B Test was adopted. Everyone from the TeamSparta team was content with Hackle’s A/B Test because of:

• Hackle’s intuitive dashboard
• All team members were encouraged data-driven manner
• High-quality customer support and expertise by obtaining some of the best practices of A/B Testing from some of the most experienced experts

The team at TeamSparta was given the opportunity to experience the best practice of A/B testing and obtain related domain knowledge at the same time while experimenting around with the Hackle dashboard.

Check out Hackle at www.hackle.io in order to start creating your own A/B tests and create a similar impact as TeamSparta.