Behind the Scenes of Engramo, Pt. 4

Engramo English is a mobile learning app. It’s the smart solution for the English learner of the 21st century. You download it, open it, and it does its job — it teaches you English. But how does it actually work? What is hidden behind that nice & clean Dashboard?

Review Process

Over years of development, thousands of exercises have been created by our team, though some don’t make the cut and end up scrapped or recycled. Mostly, it’s the latter, though, because no one likes throwing away the result of their work and because modifying (sometimes heavily) canned exercises takes less time than creating new ones, which includes selecting suitable sentences from a corpus (you can read more about corpora here and here, importing them into our systems, etc. But how does an exercise end up on the scrap pile?

First, a fellow Content Team member goes through my exercises and ranks their quality on a three-point scale. It’s either OK, to be modified, or to be redone completely. The reviewer also notes what’s wrong with the exercise so that I know what to focus on. When I’m done, the reviewer (hopefully) gives it an OK ranking at which point the exercise passes onto the test realm of the Engramo English app. There it has to pass two more rounds of review. Firstly, by a linguist for compliance with the GB and the actual state of the language “in the wild” and then also by a native speaker without specialisation in linguistics so that we know that it’s not just “technically correct”, but that it also “feels right” (this step resembles acceptability judgement tests common in linguistic studies). The main difference between these reviews and the first round with the rankings is that these reviewers access the exercise the same way a real user would, through the app as part of a mission, and this helps them tune into the authentic language experience and filter out the content that’s only “technically correct”. Only after these reviews are completed is the exercise published and made live.

But that’s not all! Internal Quality Assurance is one thing, but we really want to be sure that nothing slips through the cracks, so we’ve also enlisted the help of independent reviewers from Charles University’s Faculty of Arts, who are given a random sample of our exercises for scrutiny. Once that’s done, we’re finally happy with the fruits of our labour.