Building an internal user testing process at Teachable

One new hire at a time

At Teachable, I’m fortunate to work on a product that our users care deeply about. 22,000+ instructors use Teachable to teach all sorts of skills: flying drones, blacksmithing, watercolor painting—even how to create VR games. Since our platform is tied so closely to people’s passions and livelihoods, our users (understandably) have very strong opinions about what we do.

The challenge of working on a product with so many use cases is what drew me to move across the country ten months ago to join Teachable. As our first product design hire, one of the goals I wanted to achieve was to build a repeatable product design process, with regular user testing being a major part of that.

Not to say we weren’t previously talking to people— we were doing a good job of talking to users to gauge how effective our product was and find pain points. In fact, there’s a fairly vibrant community of school owners on our Facebook group that don’t hesitate to tell us when they like (or vehemently dislike) something. What we were lacking was a consistent step in our design process where we ran feature prototypes by real people, before we started development.

Internal vs. external testing

User testing is usually thought of as an external process, but for us I thought it made sense to start building our testing process with internal stakeholders. We’ve been (and still are) hiring aggressively; as a result, there’s a good mix of people who know a ton about the product and recent hires who may not know much about us, other than that we’re in the online course space.

Having non-product/engineering employees participate in user testing can serve as a way to gain familiarity with different parts of the product while also taking the veil off the ~design process~. Since we’re not a huge org (70+ people) and pretty much all of our company is located in a central office, it’s also great for soliciting quick feedback; I can set up a clickable InVision prototype and get in-person results from three or four people in a couple hours.

Another benefit to starting internally is that we’re able to take time to build our testing process in accordance with best practices before applying it to customers. We can strategize around minimizing bias, refine testing scripts to encourage open-ended feedback and collectively get a better sense of which kinds of feedback to filter out. It’s also an opportunity to have different members of the product team practice leading tests in a low-stress environment—equipping anyone to be able to conduct quick tests if need be.

It helps that experiments like this are encouraged throughout the company. All I had to do was throw a message out on Slack to see if anyone wanted in on being a feature tester. To be honest, I was expecting maybe 10 responses max.

My first #general post

The actual number blew me away, with over 30 people (a sizable chunk of our 70-ish person operation) volunteering to join our internal testing group!

The results

Internal user testing has been used during the development of five features so far, with multiple members of the product team conducting the tests. We’ve interviewed volunteers across multiple roles—including finance, customer care, engineering and marketing. As with any new process, there have been details to work out, but there have been a few quality takeaways from starting it internally.

Greater context around a feature

Getting feedback from diverse roles has helped me to gain greater context on what I’m designing and how it fits into the product as a whole. Receiving firsthand insight around user behavior from customer care, learning about the needs of larger schools around a particular workflow, or fully understanding the business implications of a feature results in immediately applicable design takeaways. These viewpoints have made me approach problems differently than I would have if I was just conducting cursory primary/secondary research.

Team involvement

People outside the product team aren’t afflicted with the tunnel vision that comes from staring at a particular feature for months, so the learnings from these internal sessions is super practical. But in previous startups I’ve seen that people from outside product can be hesitant to offer up feedback because of lack of opportunity, bureaucracy, or even intimidation. Internal user testing opens up the feature design process to a wider audience.

Its also been a way to meet members of teams that I normally wouldn’t interact with—people that have strong design acumen and thoughtful feedback around usability and design. When people all over the org are empowered to be curious and ask questions around a feature, that makes the end result all the better.

Increasing the visibility of design

Conducting user tests shows firsthand the importance of a quality user experience within our product (nothing makes people empathize with users as much as unbridled frustration), while giving those outside of the product and engineering teams a primer on what constitutes good design. As we conduct more and more user testing sessions, everyone comes away with a clearer understanding of what the bar is for usability and polish. The result is that with every new feature, user experience becomes more of a consideration by people both in and out of the product/engineering team.

Next Steps

Internal user testing is only the beginning. As we refine our testing methodology, we can use it as a blueprint to shape how we go about building out our external user testing process with customers (be on the lookout for that!). Having both internal and external user testing as a consistent part of feature design can help ensure that we’re getting real feedback sooner, with the goal of building the best possible features to empower our school owners.

If you’d like to join a fast-growing startup in the education space that values hustle, candid feedback and continually trying to do right by our users— we’re hiring!