A new generation of AI powered, personalized avatars are currently enjoying a moment in the spotlight. Apple and Snap’s Memoji and Bitmoji have respectively cemented digital mini-mes into popular culture, which is why we’re seeing them pop up everywhere in the form of games, apps, even digital concerts.
The avatars of today are a far cry from those introduced in the early Sim games or the popular Second Life world of the early 2000’s. Advancements in AI technology have allowed this new generation of avatars to be more life-like and capable of many more functions.
ObEN’s Personal AI (PAI) showed what the next generation of AI powered avatars will look like when we competed against Microsoft’s XiaoIce AI on the hit Chinese game show “My Future”. Each week, the show features futuristic technologies that compete for audience votes.
The focus of this episode was “Companions of the Future”. Using our proprietary, full stack AI technology, ObEN created the PAI of famous Chinese actress Dai Chunrong , known for her starring role in the internationally acclaimed drama My Fair Princess.
Dai’s PAI represents the type of avatar ObEN wants to create for everyone in the world. As seen in the video, Dai’s PAI not only looks and sounds like her — as Dai interacts and converses with her PAI, and shares her personal data with it, the PAI becomes smarter and capable of holding an entire conversation with her. With enough data, it could become the ultimate digital companion.
Unlike most avatars currently on the market, ObEN’s PAI is so much more that a digital sticker for messaging and social media (though that is certainly one of its many uses). Using cutting-edge NLP and machine learning technology, PAI’s can be used by celebrities to create more personalized interactions with fans, for doctors to provide remote patient monitoring and triaging via smartphones or tablets, even used by you or I to perform common everyday tasks like managing schedules or relaying simple messages.
During the episode, the audience got to follow along as Dai created her PAI.
Creating a “celebrity grade” PAI only takes a few hours of the talent’s time. We start with a full body scan, which we then feed into our AI algorithm, producing a 3D, animated avatar of the person. ObEN is also developing “motion estimation” AI technology that would allow our AI algorithms to mimic movement drawn from a video and apply it directly to the PAI. Our work in “visual speech” technology will also enable our AI to generate facial and mouth animation that matches with any given speech or text input — adding another layer of realism and emotionality to our PAI.
Next, we collected voice data from Dai — which enables our AI to generate a PAI that speaks in her voice as well. Unique to ObEN is the vastly smaller quantity of data required to mimic a human voice in both color and prosody — while other companies many require hundreds of hours of voice data to create a talking avatar, we only need about an hours worth. In addition to Text-to-Speech technology, our AI can also mimic the singing voice of a person. We referred to this as Speech-to-Singing — a magical-seeming technology that can make anyone’s PAI sound pitch perfect!
From that voice and image data, a high quality PAI is born. Like most things at birth, the PAI is still immature — capable of a few fundamental actions, but no operating at its full potential. This is where NLP and chatbot technology comes it. By training the PAI with information about Dai, it slowly learns more about her. Much like children learn by watching and mimicking their parents, Dai’s PAI learns by consuming data about Dai.
We were excited to share the stage with a technology powerhouse like Microsoft. XiaoIce — one of its flagship AI systems in Asia — has successful developed skills as a poet, singer, and storyteller over six successive generations of development. And though we ended up losing to XiaoIce by a single point, the fact that we stood toe to toe with a technology juggernaut was a satisfying validation of the R&D coming out of the ObEN team.
In addition to our “celebrity grade” PAI, ObEN is also creating consumer friendly PAI technology that allows use to generate a 3D avatar from a single selfie and brief voice recording. Soon, everyone in the world will have the ability to create, teach, and enjoy their very own Personal AI. So strap in — the avatar craze is just getting started, and ObEN is wave of the next generation of digital avatar technology.
