Customer research strategies behind Busuu’s 120 million learners

Kirsten Campbell-Howes, ex-CLO of Busuu, explains the importance of continuous customer research, how to translate customer insights into a roadmap, and the critical role of a CLO in edtech. Find out how these elements combined to form Busuu’s winning formula to reach and maintain product-market fit.

Welcome to the Emerge Product-Market Fit Academy series, where we bring you practical insights and advice from our community of Venture Partners, who are world-leading edtech founders and operators. This series is dedicated to helping early stage edtech founders navigate the challenging path to achieving product-market fit.

In this case study, we introduce Kirsten Campbell-Howes, former Chief Learning Officer at Busuu, a category leader in the language learning space. With a strong track record in both B2C and B2B products, Busuu has empowered more than 120 million learners worldwide and partnered with renowned clients such as Inditex, Adidas and Unilever. Notably, Busuu achieved a remarkable exit in 2022 when it was acquired by Chegg for $436M, marking one of the largest edtech exits in Europe to date.

Kirsten Campbell-Howes, ex Chief Learning Officer at Busuu (acq by Chegg), Venture Partner at Emerge

Kirsten’s expertise lies in collaborating with cross-disciplinary teams to develop and refine product features and courses based on user feedback. As the head of education and localisation, she successfully scaled the teams to over 50 members, fostered academic research partnerships, and spearheaded learning design initiatives with corporate giants such as Uber.

With over two decades of experience in the edtech industry, Kirsten has dedicated her career to integrating new digital technologies into education and driving positive outcomes for learners and businesses alike. She has built up a wealth of expertise on how edtech founders should approach customer research and solution building in the early days to not only reach, but also maintain, product-market fit.

By the end of this article you’ll understand more about:

• The role of a Chief Learning Officer (CLO)
  Gain insights into the role of a CLO and when to bring one on board to focus on learning outcomes, learning science, and ensuring a customer-centric learning experience.
• Designing effective customer research programmes
  Learn how to create research programmes that deliver valuable insights with minimal effort, including strategies for accessing hard-to-reach markets.
• Leveraging early customer feedback
  Discover how to effectively collect, prioritise, and act on customer feedback to drive product improvements and shape the development of your product.
• Building a strong product roadmap
  Explore frameworks such asRICE (Reach, Impact, Confidence and Effort) and agile tools like story points, that can help you prioritise features and allocate resources effectively, ensuring your roadmap aligns with customer needs.

THE ROLE OF A CHIEF LEARNING OFFICER (CLO)

As an edtech founder, understanding the role of a CLO can significantly impact your product’s success and long-term competitive advantage. Confusingly, as Kirsten points out, there are two types of CLOs. First, there’s the type of CLO who focuses on employee learning and development within large corporate organisations. Second, there’s the type of CLO, like Kirsten was at Busuu, who is dedicated to driving learning outcomes and ensuring a superior learning experience for customers.

As an early-stage edtech founder, it can be difficult to know when to bring on a CLO with expertise in learning science and content development. In the early days, you may not require a CLO with specialised qualifications. However, it is crucial to recognise the value of learning expertise within your leadership team. Kirsten observes that it is common for edtech founders to become enamoured with the technology, inadvertently overlooking the primary purpose of their product: facilitating effective learning. Neglecting to incorporate learning design and science early on can result in challenges with customer retention, negative feedback and difficulty attracting future investments.

“There’s a tendency in edtech startups for people to focus on the tech first and forget that the customer is there to learn; the majority of the time they spend in your product, they are learning. If you don’t bring on learning science expertise early and make that a core part of your leadership team and strategy, you are going to struggle later on, in particular with things like retention. You may start to get negative customer feedback because you can have the most exciting, brilliant technology, but if people aren’t feeling the learning outcomes, if they’re not feeling that strong sense of progress, they’re not going to stick with you. You may also struggle to raise investment rounds in the future because, as you grow in stature as an edtech business, investors will start caring much more about learning outcomes.”

By integrating learning specialists, content creators, and learning designers from the outset, you can prioritise and optimise the learning outcomes that truly matter to your customers. While a full-time CLO may not be necessary initially, considering a part-time CLO or contracting learning experts can be immensely beneficial to your product’s development.

When Kirsten started her journey at Busuu almost a decade ago, the concept of a learning designer was virtually non-existent; she had to mould teachers into instructional and learning designers herself. But now, she exclaims with excitement, that times have changed. A vast pool of talent exists, comprising professionals specialising in designing and building effective learning experiences, many of whom hold advanced qualifications, such as master’s or PhD degrees, or have experience working with other edtech companies. She advises edtech founders to tap into this expertise early in their journey and make the most of the flexibility to engage contractors or part-time specialists to incorporate their learning design.

DESIGNING EFFECTIVE CUSTOMER RESEARCH PROGRAMMES

A critical part of Kirsten’s role as CLO was conducting effective customer research with Busuu’s users. When it comes to building a successful startup, understanding your target customers and their needs is paramount. Underestimating the importance of this is one of the most common reasons startups fail. Conducting effective customer research can be a challenging task, especially for early-stage founders with limited resources. However, Kirsten shares four key considerations for founders to conduct effective customer research that delivers maximum results with minimal effort, without needing a dedicated research team.

1. Striking the right balance between quantitative and qualitative data
2. Avoiding biases and fostering constructive feedback through research hygiene
3. Regularly experiencing the product from the user’s perspective
4. Recognising the importance of ongoing customer research to adapt to evolving needs and maintain a competitive edge

When it comes to customer research, it can be hard to strike the right balance of quantitative and qualitative data. However, both are extremely valuable in understanding customer behaviour. Kirsten points out that while quantitative data provides metrics and trends, qualitative research uncovers the reasons behind customer actions and allows founders to make informed decisions about their product roadmap.

“At Busuu, we used Amplitude to collect a lot of quantitative data so we could see, for example, that 50% of people were not completing a certain lesson. But that didn’t tell us why they weren’t completing that lesson. It was only by doing qualitative research, by interviewing customers, that we were able to understand that there was a particular exercise type that was very tricky. The user interface was challenging for users, which was why so many people were not completing this particular lesson. Without the qualitative insight, the quantitative data, didn’t help us or might have led us to make incorrect hypotheses.”

Busuu used Amplitude to collect and analyse large amounts of quantitative customer data

Tools like Amplitude can be helpful, but founders should be careful to avoid the trap of focusing too much on numerical data and ending up distanced from the experience of their end users. One strategy to avoid this, which worked particularly well for Busuu over the years, was getting the team to regularly sign up and go through the first user experience of the product, taking notes on their experience. Whenever they did that at Busuu, Kirsten recalls, the team always walked away with valuable insights, from new ideas to usability issues and potential blind spots.

While going through the user experience can provide excellent insights, it is no replacement for talking to actual users, a process that requires good research hygiene to prevent bias and foster an environment of constructive feedback. The Mom Test by Rob Fitzpatrick is a great resource on this topic, to understand how to make time spent on customer research a worthwhile exercise in data gathering. As a founder, the importance of learning how to talk to customers and get reliable, actionable feedback on your idea to drive product development, should not be underestimated. A research hygiene strategy can be something as simple as declaring before a customer interview “We really, really welcome constructive feedback, however critical. You’re not going to offend me if you say something critical about this product. I didn’t design it and I am open to any feedback that you want to give.”

In their customer research, Kirsten notes, founders often make the mistake of asking questions such as, “Do you want features X, Y and Z?” and the customers almost always say yes. People want more features. However, that data doesn’t help translate into your product roadmap and help you prioritise which features are most important. Instead, good research hygiene would be:

1. Asking a question such as, “What feature would you like to see?”
2. Observe what emerges naturally from customers’ own needs, rather than the thoughts of the founder or team on what the most important features are.
3. Once you have a list of most requested features from customers, then ask, “If you had to choose one, would you rather have feature X, Y or Z?”, to determine what the priority should be when it comes to your product roadmap.

This process should be repeated often, regardless of stage. A mistake that Kirsten often sees founders make as they scale, is viewing customer research as a one-off process, relegated to the pre-product-market fit stage.

“It’s really tempting to dial down on that research, especially the qualitative research. That’s dangerous because you lose sight of your customer and, the way the world evolves so rapidly right now, that means you can lose sight of growing dissatisfaction or you can become complacent. Product market fit is not static. It could grow, but it can also disappear. In the learning design team at Busuu, we found that if we didn’t do customer research for a period of time, that was always a bad idea and we needed to get back to it because we would discover that things had changed.”

Founders should continuously gather feedback, validate hypotheses, and make informed decisions about future product development. By incorporating research as a regular practice, founders can ensure their product evolves in line with customer needs and maintains a competitive advantage in the fast-changing edtech landscape.

LEVERAGING EARLY CUSTOMER FEEDBACK

Collecting strong customer feedback is step one. Leveraging that data is the critical step two. Armed with a plethora of data, it is important to avoid analysis paralysis, where teams become stuck in a loop of reviewing the data but not acting on it — or simply ignoring it all together, warns Kirsten. The core purpose of customer research should be actionable product changes that reach end users. Founders should set clear research objectives and focus on gathering data that directly contributes to answering key questions, while paying close attention to the time it takes for insights gleaned from research to make their way into the product roadmap.

At Busuu, Kirsten recalls continuously prioritising gathering large amounts of research data and translating that into actionable product changes. They implemented a Customer Feedback Bank, a centralised system for collecting and categorising customer feedback, from all possible sources including app store reviews, customer service interactions, user interviews etc. They then hired a research lead, whose focus was to collate the data and translate it into actionable insights, using the following steps:

1. Feedback collection: the research lead reviews customer feedback from all sources, expressed in multiple languages, phrased in different ways, on different platforms. They then identify patterns in the feedback and find the common requests, for example, ‘I’d like more speaking practice’.
2. Categorisation: for each piece of feedback, the research lead codifies the data with either a number or letter that relates to the request. For example, they might use ‘S’ to represent ‘more speaking practice’.
3. Ranked feature list: the process is conducted for all feedback to ultimately generate a list of the most common requests, ranked in order of frequency they are requested by customers.
4. Team review: each month, the research team meets with the product and learning design teams to look through the customer feedback bank and review the top feature requests that need to be added to the product roadmap.

Through this process, the team was able to translate a large quantity of data into tangible, actionable insights to put into the product roadmap.

The process at Busuu for translating customer research into actionable insights on the product roadmap

BUILDING A STRONG PRODUCT ROADMAP

Prioritising features is crucial for effective product development. At Busuu, Kirsten used the RICE framework for prioritising features in a product roadmap to ensure the most impactful and feasible initiatives were given priority in product development. By evaluating and scoring all competing priorities, you can produce a concrete number to determine which initiatives should be prioritised and in which order. RICE assesses items based on four factors (Reach, Impact, Confidence and Effort).

1. Reach: measures the potential audience or user base that will be affected by the product or feature. It helps determine the scale of impact and the number of users who will benefit from its implementation. To assign a value to reach, estimate the number of individuals who will directly benefit from or be impacted by the project. This can be based on the project’s target audience, market size, etc.
2. Impact: evaluates the expected positive outcome or benefit that the product or feature will have on the users and the business. It assesses the significance of the change or improvement it brings. To assign a value to impact, consider the expected outcomes, benefits or changes the project will deliver. This can include factors such as revenue growth, cost savings, user satisfaction, etc.
3. Confidence: reflects the level of certainty or confidence in the expected impact. It considers the amount of data, research, or evidence available to support the expected outcomes. To assign a value to confidence, evaluate the level of confidence you have in the project’s success based on factors such as market research, technical feasibility, team expertise, etc.
4. Effort: gauges the amount of resources, time and effort required to develop and implement the product or feature. It helps assess the feasibility and cost-effectiveness of the initiative. To assign a value to effort, estimate the level of effort required in terms of time, budget and team resources. Consider factors such as the complexity of the project, required skill sets, dependencies, potential risks, etc.

RICE Score = (Reach x Impact x Confidence) / Effort

Once you have assigned values to each factor (Reach, Impact, Confidence and Effort), you can also add an assigned weighting to each factor based on its relative importance. For example, you might give higher weight to impact and confidence while assigning lower weight to effort. Though the weighting is not necessary, it is highly recommended, as it allows you to reflect the relative importance of each factor in the overall assessment and prioritise them accordingly. It helps to differentiate the significance of different factors and make more informed decisions about project prioritisation, as factors with higher importance receive more consideration in the final RICE score calculation.

However, if you choose not to assign weightings, you can skip this step and still calculate the RICE score, following the steps below:

1. Assign values to each factor (Reach, Impact, Confidence and Effort) based on your assessment.
2. Assign weightings to each factor based on their relative importance. The weightings should reflect how much each factor contributes to the overall priority of the project. You can assign weightings as numerical values or percentages.
3. Multiply each factor by its respective weighting. For example, if the weightings for Reach, Impact, Confidence and Effort are 0.3, 0.4, 0.2, and 0.1 respectively, you would multiply each factor by its weighting.
4. Calculate the RICE score by summing up the weighted values of Reach, Impact, and Confidence and then dividing the result by Effort. The formula would be: RICE Score = (Reach x Weighting for Reach + Impact x Weighting for Impact + Confidence x Weighting for Confidence) / Effort

This calculation provides an overall priority score for the project, which can help prioritise projects based on their potential impact and resource requirements. The higher the RICE score, the higher the priority of the project. It’s important to note that the specific values and weightings assigned to each factor are subjective and can vary based on the context, organisation, or project requirements. By using this model, founders can make data-driven decisions, reduce biases, and effectively communicate priorities to stakeholders.

Once you’ve used RICE to prioritise where you want to focus your efforts, you need to map that ranked feature list against a tangible timeline that teams can follow and map their actual day-to-day tasks against. When leading the learning design team at Busuu, Kirsten used story points.

Story points are a tool in agile product development for estimating the effort and complexity of work items, such as features or tasks. They focus on the size and complexity rather than specific time durations, acknowledging that these can vary between teams and individual experience levels.

Teams assess the complexity and effort required for each work item. Teams often use the Fibonacci sequence (1, 2, 3, 5, 8, 13, etc.) to represent story point values as this sequence helps capture the exponential increase in complexity as work items become larger, allowing for more accurate estimation and prioritisation. These estimates help determine the team’s capacity and velocity. Capacity refers to the amount of work a team can complete within a given timeframe, while velocity represents the average number of story points the team can complete in a sprint or iteration.

By using story points, teams can have meaningful discussions about workloads, trade-offs, and prioritisation. It facilitates collaboration, enables efficient resource allocation, and provides a more accurate estimation of effort compared to specific time durations. As Kirsten experienced with Busuu, incorporating story points into your product development process can enhance planning, coordination, and alignment among team members as they map their efforts against the prioritised feature list.

Example of how to determine story points of a task

SUMMARY

In this article, Kirsten shares her strategies for attaining and maintaining product-market fit in edtech, from her experience as CLO at Busuu. Emphasising the importance of comprehensive and continuous customer research, Kirsten covers the concept of research hygiene, encouraging founders to focus on gathering unbiased insights to inform product development. To effectively use customer feedback, she breaks down how Busuu built a Customer Feedback Bank, a system for collecting and categorising customer feedback. She shares the process through which those insights are translated into a product roadmap, for which Kirsten recommends the RICE framework to prioritise features and story points to manage workloads and set timelines across teams. Finally, Kirsten underscores the vital role of the CLO in an edtech startup, advising early-stage startups to incorporate learning science expertise from the onset to ensure effective customer retention and to later answer investors’ questions around measurable learning outcomes.

Emerge is a global pre-seed fund backed by 100+ of the world’s best edtech operators. Our vision is to democratise access to opportunity — by being a catalytic partner for early-stage edtech founders. If that’s you, get in touch and submit your deck on our website.

This article is part of our Product-Market Fit Academy, where we bring you practical insights and advice from our world-leading community of Venture Partners. To keep up to date with our episodes, follow us on LinkedIn, Twitter, Medium, YouTube and Spotify. You can also subscribe to our newsletter to stay up to date with our latest insights for early-stage edtech founders.