Developing Education Technology Is Its Own Fresh Baked Hell: A Product Developer’s POV
Way back in 2013, back when Caseworx was little more than some wireframes and a few discussions with educators, one of our first “jobs” was for the Education Writers Association. We were sent to document the panels and workshops led by some of the foremost education journalists around the world on Stanford’s idyllic campus. The theme was all about disruptive technology. Specifically…MOOCs. This new iteration of the online course was a fast-moving novelty then (so much so that many people were pronouncing them as “moks”), but the excitement was palpable. The best education minds from the New York Times, Wall Street Journal, the Chronicle of Higher Education, and beyond were grappling with the exciting, complex future that MOOCs would issue forth.
But we know what happened: the backlash happened. Completion and attrition rates were measured, and they weren’t good. Questions arose about who actually benefited from this technology. Charges of scale at the cost of efficacy flew back and forth. Major companies like Coursera and Udacity cast about, looking for profitability. There was little discussion of MOOCs at the 2014 EWA Conference, and the tool suffered a bit of a malaise through much of the 2010s. Now…things have changed. Coursera is now a unicorn. Many other MOOC platforms have found their footing in things like certifications and credentials. It’s safe to say that MOOCs are not the panacea that some claimed at first, but they have become more sophisticated and found a way to bring real value to learning. That is, they behaved a lot like almost every new innovation, education or otherwise.
But the stories of MOOCs and CD-ROMs and LMS platforms are unique in that they are built for learning. Education can be a maddening vertical, especially for technology products. It’s a space often blinded by passion, caught between warring agendas, plagued by low adoption rates and anemic price points…but given to grand claims and zealous missionaries. There are easier ways to make money, in other words. As a small team who’s built an education platform that’s won some acclaim over the past few years, we want to give some insights on best practices to develop edtech products for the long haul. Because it is a long haul, and even before you start the journey, it’s important to understand something about incentives in education. They are perverse.
You want to cut the lawn? Which mower does it best at the right price? You want a pizza? Who delivers around here and makes good pies? Selling enterprise SaaS software gets harder, of course, because there are more decision makers, but nothing really compares to education in terms of asymmetrical incentives between stakeholders. Take a look at the website for any major university, or ask any student or professor how they feel about their learning management system if you want to see bewildering products torn between many masters. What’s more, profit is often a buried or irrelevant concern in learning (hang out at any faculty meeting if you don’t believe this), and what does take place instead is a dizzying array of “wants” from wildly divergent constituencies. Curriculum, scheduling, billing, privacy, transparency, on and on…most education products have to at least think about all of these issues.
Getting an edtech product embedded in a learning environment requires aligning those incentives to some extent. That’s not simply BizDev or Marketing’s issue; it’s a huge concern for Product and Engineering. You may have designed the greatest math learning tool on Earth, but if it’s not ADA-compliant or the instructor can’t hook it up to her LMS for grading, it’s not going anywhere. How do you balance your constrained resources and avoid mission-creep, while creating an excellent product that brings delight?
You Gotta Problem?!
First, it’s important to be very clear what specific problem the product is solving. This may seem obvious, but it never ceases to amaze how often Product and Sales exist in different universes here. It may also be true that your product has to attack different problems for different groups. In this common case, it’s important to sequence the problems in order to get your foot in the door (and keep it there). For example, it may be necessary to solve the L&D instructor’s lack-of-AR-curriculum problem before you can attack the student-evidence-of-learning problem. Presenting your product in front of a cross-section of stakeholders in the institution will leave you open to so many demands that your head will spin. The only way to stand up to this is to have (a) a rock solid problem-solution-value prop axis that everyone agrees on internally, and (b) a clear understanding of how solving the varied problems — in sequence — will not only get you in the door, but reordered year after year. Again: this happens in a lot of enterprise software, but in education it’s turned up to 10.
This may mean saying “NO” to early deals, if they are trying to get you to solve a different problem(s) than you’re trying to solve. That’s a very painful thing to do for early-stage companies. But getting into “mission quicksand” with an educational institution is a long, slow death. Be very clear about the problem you attack; do not deviate when trying to close deals. Know the problem. Have evidence it exists in the world. Hunt it every day.
Separating Stakeholders From Users by Design
For the majority of education products, there are many stakeholders, but only two core users: learners and educators (there are ancillary users — those tasked with implementation, course designers, etc. — but here we will discuss the two primary user groups). Learners have things to learn, and educators have things to teach. For the edtech product, it’s essential that these users feel supported in the use of the platform from the get-go. Training materials and first-time tutorials on how to use the platform are helpful, but having a human available at all times to answer questions and help troubleshoot is a requirement. From our perspective, you cannot automate or AI your way out of this, at least not in the early days. Product teams should not oversell how ably the product can process user needs, and company leadership should always be factoring some layer of human-centered customer success, which may span across curriculum development, troubleshooting, user engagement, and more.
But you cannot give all stakeholders the same level of care with startup resources. So perhaps there’s a good rule of thumb: personalize user success for your core users, while automating for the many stakeholders beyond them. This starting point can help you preserve bandwidth while giving full attention to the most important folks on the customer side.
Not-So-Rapid Iteration
Products always begin with the basic functionalities, intriguing features, and core structures from your initial “best-guess” design. After your product works and it is in the hands of customers, is that it (hint: this is a rhetorical question)? There are always updates and changes that must be made for customers, and improvements to stay ahead of the competition. Hardware products such as cars, laptops, and phones have a new version every year, with enticing features to keep the customers coming back for more. But the “move fast and break things” motto, while a romantic thing for entrepreneurs to write in their AngelList bio, is just not so useful for edtech product development. No one actually wants you to break things in education. No one. In most walks of life, people want painless innovation, but here it’s on another level. There really are high stakes if a student’s grades mis-transfer to Moodle or the answers to a med school test are jumbled. From an engineering standpoint, this means being very sensitive to how all of your features connect internally and externally. It means being very clear about which features have changed, and being prepared to do a lot of continuing user education.
On the flip side — and this is where good edtech products can stand out — being lean is a great asset. Again, so many education products are bloated, bewildering, do-everything-mediocre kinds of experiences. Doing one thing really, really well — attendance, curriculum, note taking — not only results in better UX, but also happens to make your product a compelling acquisition target (this is how education products usually exit, you know). So it’s not to say that you shouldn’t iterate in education…you must, in order to survive. But it can’t be the chaotic, mad dash for the optimal experience that you see in many other industries. It must be deliberate, clear, scheduled, and with lots of aftercare.
Semester-to-Semester Loop-d-Loop
Finally…an advantage! Systematized feedback is where developing an edtech product finally becomes a bit more straightforward and easier than other products. By formalizing a survey and feedback session with real learners and educators at the end of a learning period, you gather that critical information at a logical time in the product life cycle. This is the time to digest that feedback, compare the direct statements from your users with the analytics data that you collect on the platform, and design your next round of features. Edtech platforms can develop a very natural iteration-testing-feedback loop in this way.
For Caseworx, each learning period becomes a new chance to refocus our energy and resources on what is most important to our users. With the platform analytics we get a quantitative look at the results almost immediately, and make that quick decision on initial success. But qualitatively, we will typically wait until the end of the term to survey our users to get more detail. The reason for this is simple — learners and educators are busy during the term, and any written feedback we get is going to be brief and incomplete. Catching them at the end of the learning period gives a nice user narrative that is great for product development assessment.
Here’s an example of the feedback loop in action: Let’s take a real-world decision based on instructor feedback: Building a filtered search mechanism should be our next major feature. Once the feature is complete and rolled out, the next term is when instructors will get the chance to use it. Looking at the data from the platform itself, we can immediately answer several questions — “Are they using it at all?”, “How efficiently is it working?”, and “Are we getting customer service questions about it?” All of these data points can inform whether we need to make any immediate changes or patches to fix problem areas. But in this example, there’s nothing on fire, and the feature is being used appropriately (if not a ton). At the end of the period, our survey for the instructors includes several direct questions about the feature. The warning sign to look for is a consistent set of messages in the answers. Provided that there’s nothing jumping out at us in those answers, it’s time to process the data and decide on smaller tweaks to our new filtered search feature to prioritize along with our other goals for the next period. So, the rhythmic nature of learning cycles becomes a key part of our own development process. Nice!
So…Why Are We Doing This Again?
The education space can seem like it combines the worst aspects of all the other verticals. The sales cycle is not only rigid, but long. Multiple stakeholders complicate clear value props. Stakes are high. ROI is hard to gauge. Price points continually suffer from downward pressure. But there is one massive upside, in that education provides advancement and mobility to our fellow humans like nothing else. There is no more powerful tool you can give someone to help them improve their situation. It is, quite literally, a game changer for people…and it changes the game most radically for the people who need it the most. That’s why edtech people deal with the — let’s say eccentricities — of the industry. Having these development concepts in mind can help future education entrepreneurs navigate some of the most treacherous pitfalls. Good luck!
Rudy Luthi is the Co-Founder and Head of Platform for Caseworx. He has also been lead designer for major projects at Evite, Fabric Interactive, and beyond. This piece was co-written by Steven Lang, our 2019 Spring Intern at Caseworx (hire him!).
