Chinegraphy: Hear, See, Do!

The development process of a Chinese training App

Published in

Apple Developer Academy | Federico II

5 min readMar 19, 2020

Learning a new language is a passion for many, a necessity for most, but for all it always represents an adventure, and for some people this adventure is called Chinese.

According to 2020 estimates, almost a billion people speak Chinese as their first language, and another 450 million are currently studying it at school or university, and the one thing they all agree about is that it’s not a simple feat.

The Defense Language Institute has estimated Mandarin Chinese to be a Category IV Language, meaning it usually takes an English native speaker about 2200 hours to reach a proficient level.
This is unanimously attributed to the writing system, a way of representing the language that is so hard, it literally takes natives a lifetime to master.

Researches

Traditionally, children from the age of three start school and are immediately introduced Hanzì, that is how Chinese characters are called, and they never really stop learning and revising them.

A common way to practice Hanzì is writing them on a “Tian zi ge”, a particular notebook designed specifically for Chinese.

Unfortunately, this results in an irresponsible waste of paper and so, with the advent of modern technology, there have been attempts at finding greener solutions.

In order to understand this trend, we conducted a series of interviews on a number of individuals, all of which were involved with Chinese on different degrees and levels, and our research showed the following:
despite the abundance of applications and programs, none of those solutions actually feel like convenient tools to learn or revise on their own merit.

The current state of things is causing students to pick one of two inconvenient alternatives: adopt multiple solutions at once or ditch these innovations altogether and rely only on the obsolete ways of training.

Our studies made one thing clear: a gap is there and we want to fill it.
The solution we propose is called Chinegraphy, an app designed to help people train their Chinese memorizing and writing skills in a simple and intuitive way.

The Result

Decks

When you open the app, the first thing you stumble upon is the list of decks, whose design is broadly inspired upon flashcards, which are a tool commonly used for mnemonic learning of Vocabulary.
Our goal was to inspire familiarity to users who may have used Language training apps in the past, while keeping the style simple and easily comprehensible by anybody, regardless of their background.

Drawing Field

The main training area is the drawing field, with a design inspired by the Tian zi ge; in this area, you can live the complete experience of the application:
in it, you can hear the Hanzì, you can see the Hanzì, but most importantly you can do the Hanzì, living an overall whole and fulfilling adventure.

Wise Master

If you do your homework and train daily, you too can become a master of Chinese and complete multiple decks!
To celebrate your efforts, you will be complimented by the Wise Master at the end of each deck, with a quote from the wisest thinkers from China.

The Chinese writing system is not simple on its own, and abstracting an entire writing system into an application is already a hard task, and we ended up facing two main complication when engineering for Chinegraphy.

Technical Feats

The main technical challenge we encountered was translating Chinese characters into a mathematical model for analysis and storing purposes.
The first step in this direction was to meticulously study the subject, and the main realisation that came from it was that Hanzì are deeply varied as far as difficulty goes, and present a complex set of rules to be written properly.
An example of the variety is represented by the two Hanzì below:

A simple Hanzì on the left, a much more complex Hanzì on the right

In order to tackle this problem, we had two choices: the majority of programs uses a technique called OCR, which is usually done by training a Neural Network to recognise a given set of Characters.
After testing this solution for a while, we realised it would be too impervious to apply for Hanzì, because of their sheer number, so we decided to approach and implement a study we found from The University of Tokyo, which breaks down the general structure of Hanzìs into 16 different patterns. That was the breakthrough we needed.
After some amount of difficulties, we finally made the main algorithm work.

Making the model usable by everyone

The second challenge that we had to overtake was making the application actually enjoyable for the users.
In order to achieve this task, we released the application onto TestFlight, so that users could test it and adjust the algorithm to the way they actually saw and wrote the glyphs.
In addition, we were committed to take what was structurally an utility app and make it into a fun experience, or in other words, we gamified our application: our efforts focused mainly on implementing visual and auditory feedback, such as a nice sound effect and a confetti animation for each correctly drawn character.

Future Additions

The applications is currently in the last phase of Beta Testing and we have plenty of plans for the future: firstly, customisation is the priority for us, and many news are up to come that will make the application truly yours, namely you will be able to customise your own decks and share them with your friends.
Furthermore, progress is being made on generalising the algorithm, and we’re confident that the application may soon be extended to include multiple languages, such as Japanese or Korean, and to be cross-platform between your iPhone and iPad.
Finally, we want to improve on the process of gamification, adding more elements to make the application more game-like and offer an overall better experience.

And as always, we stand by our motto: