Build a Talking Emoji with React

Have you seen this excellent article by Maurizio Carboni recently? How cool is it to build a talking emoji with regular JavaScript? So I thought, what would be cooler is… if we slap a framework on top of it!

Check out the talking emoji app I built with React — using more or less the same idea, with some additional features (audio and… a British accent).

The idea behind making this work is to:

1. Map characters of a phrase to lip movements
2. Animate the emoji so that it seemt like it’s talking
3. Put audio so that it’s… actually talking

Step 1 and 2 is already done in the original article, all we have to do is implement the same solution in React (using setState to cause re-render instead of generating a time interval from a Date object).

This character map was from the original article, and I loved how accurate it is. I don’t think I could have figured this out on my own, so I kept the original implementation.

To animate the emoji, all we have to do was render the right emoji at the right interval so that it looked like it was talking. This is where the implementation is slightly simpler in React, as we can just use setState.

Essentially, this component has the state values of phrase (the value of the form), currentIndex (the current emoji being rendered), and speaking (a boolean to indicate if it’s in the middle of a speech.

When the user submits a phrase, we trigger a new interval object which calls the animation function. This increments the index where the emoji is rendered, and clears the interval if it’s done.

I find the interval value of 55ms here works really well with Javascript’s SpeechSynthesis’ pace and these settings.

Setting up speech synthesis is pretty much a breeze. I find the British accent the easiest to synch with. One slight problem is SpeechSynthesis is smart enough to pause at punctuations — which is harder to do with intervals, but I find these settings work well enough.

With all that set-up, the emoji app works pretty well in React. This can be a nice UI feature to have in chat apps or kids games. If you like this implementation, you can check out the entire repository here.

Thanks for reading!