Who Does What In The 3D Avatar Ecosystem?
Since we started, we have witnessed the birth of multiple startups working on look-alike 3D avatars. Honestly, if you are not familiar with this ecosystem, it is easy to think that we are all working on more or less the same thing, competing for the same users/customers, and facing the same tech challenges.
Obviously, that is not the case, and to fix this misconception, we have put together an overview of the players for use by anyone interested in taking a closer look at this domain, even those with no specific 3D expertise.
Avatars For The Entertainment Industry
This overview is solely focused on companies that work either directly or indirectly with the entertainment industry for a general consumer audience: gaming, social, videos, etc. We do not address the players in the health, fitness, or apparel industries.
In order to keep the overview easy the understand, we had to make a few choices:
- Scanning booths are not included in our review. Alongside expensive, industrial-type, and handheld systems, they have been the primary scanning solution for 15 years, prior to the recent miniaturization of depth sensors and the advances of computer vision. Even if some companies have succeeded in the consumer space (e.g. with 3D figurines) most of them are used for B2B or high grade use cases, and we have chosen to focus on the solutions that will ultimately be the most scalable — hence, mobile-based.
- Many companies in the ecosystem are not very vocal about their current projects. We based our analysis on publicly available information and we are happy to update our review if necessary.
- Simplicity excludes exhaustiveness. In order to deliver on our promise to give a quick and understandable overview for an outsider, we did not address every specific subtlety or use case.
Now, the giant elephant in the room: yes, we are part of this ecosystem.
We started out at a time when people thought that having a 3D double was a fantasy. We met many people working in the field, from startups to giant tech companies. Our first product made it possible for us to work within several sub-domains (gaming, VR, AR, enterprise) before we finally landed in our sweet spot (videos!) So, we believe we have a legitimate claim to take a step back and analyze the ecosystem from a neutral standpoint.
Lastly, at some point, someone has to go out on a limb and do the job that no one else dares to do, so here it is!! :]
Most players fall into two really broad categories: Enablers and Content Providers.
A. ENABLERS
This first category regroups companies that provide solutions to capture, enrich, and distribute a look-alike avatar.
CAPTURE: creating the 3D scan of the user from either a depth-sensor or from a 2D source (photo or video).
Among the challenges faced:
- compatibility with as many devices as possible,
- dealing with capture conditions in widely different lighting environments,
- delivering consistent results for any user skin tone and hairiness.
Depth sensing solutions: Apple Face ID, Bellus3D, Google Tango, Intel RealSense, Scandy, Structure
Scan from video: Microsoft 3D Capture, Scann3D, Sony 3D Creator, Trnio
Scan from picture: Didimo, FaceGen, Itseez3D, Loomai, Oben, Pinscreen, Wolf3D
Obviously, these capture solutions do not provide identical results. To date, scans from depth sensors are still “generally” more accurate than scans from videos, which are “generally” more accurate than reconstructions from pictures. This hierarchy, however, is changing, and it is changing FAST.
It would be completely irrelevant to judge which solution is better or worse. At the end of the day, what matters is the end-user application of the scan, and whether or not the capture solution fits with the likeness-level desired.
Also, this classification is only a rough indicator. Some companies are working on different topics, like ItSeez3D, which provides software to exploit depth sensing data, but also has a photo-based solution, or Trnio, which started with scans from video, but is now digging into pictures.
ENRICH: convert the 3D scan into a character compatible with real-time animation.
Depending on the capture solution used, the resulting 3D asset generally must be processed to fit with the 3D application criteria. Operations include:
- modification of the density of the mesh,
- correction/edits of the texture, rigging/skinning of the face, head or body,
- blending with a generic head and/or body (our take on this).
Several of the companies mentioned above perform enriching operations while creating an avatar, with varying levels of complexity.
Companies that specialize in 3D scan enrichment: Uraniom(1), Mixamo (Adobe) Auto-Rigger
DISTRIBUTE: provide an SDK for integration of the avatars into 3rd party applications.
Once the avatar is generated, there must be an easy way to import and use it within a consumer application. Most companies listed above provide an SDK, however the full spectrum of potential requirements is broad and represents a tough challenge:
- level of proficiency necessary to use the SDK,
- compatibility with different game engines (commercial or in-house),
- investment necessary to integrate into current or future projects,
- features supported by the avatar (e.g. customizations or animations),
- offline integration vs. online dependency.
You can see that within the ENABLERS category, the issues are numerous and diverse (and difficult!) The whole avatar creation segment (and our collective contribution) is likely to be challenged faster than we think by phone makers’ internal projects and open-source alternatives in the coming months and years.
So far, the facts go along with this prediction. See, for example, this open-source project on reconstruction from a single image, or the work from this USC ICT lab on full-body scan enrichment. Companies like Microsoft and Sony, which are on our list, revealed their projects within the last 12 months. We expect to see more projects of this type in the future, until an inevitable consolidation takes place.
In terms of availability, there is an interesting parallel with panoramic pictures — even if the tech hurdles were arguably fewer and simpler. Early on, you had to use specific apps to make cool panoramas with your phone, and before you knew it, this became a de-facto feature of all mobile devices.
B. CONTENT PROVIDERS
In this category, we regroup companies that provide end-user applications based on look-alike avatars.
CONTENT PROVIDERS make use of solutions from the ENABLERS to build their avatar-based applications. Due to the different types of capture options, the resulting avatar’s likeness with a user varies, so they fall into two different categories: playful or realistic.
PLAYFUL
In most cases created from a single picture — thus facing no device limitation — “playful” avatars are colorful, highly customizable and their intent is not to match Pixar-grade characters. The applications using them are generally casual, and some have achieved tremendous success.
Notable companies in different segments: Gabsee (AR), EmbodyMe (VR), My Idol (video)
REALISTIC
Applications in this category bet on the most realistic avatar solutions to provide experiences that would have been considered sci-fi-esque only a few years ago.
Considering the very recent availability of realistic capture solutions for consumers, very few companies have taken this approach: Morphin (2), WithMe
Also, even if they are not exactly interactive avatars, we must mention the companies working on realistic holograms: 8i, Matsuko, Mimesys.
THE LIKENESS THRESHOLD
Sorting the content providers in two distinct families (Playful / Realistic) is not just with respect to the avatar’s style. The fact is, the experience with the avatars does not impact each user the same way, and it depends on how well they can recognize themselves. After many trials and errors with different capture solutions, we have learned that emotional reaction does not grow linearly with avatar likeness.
Let me illustrate my point. Take an avatar that “kind of” looks like you. Depending on the angle you choose, the face is similar, but the hair and head shape does not precisely match yours. Still, you know it was based on your picture and yes, a friend might recognize you if you tell her that it’s your avatar. It’s mostly funny, and some solutions in that range work better than others.
However, at some point, when the character gets realistic enough, both for the face features and the hair, and your avatar can clearly be recognized from any angle by any of your family members, then the reaction shifts. You go from people saying, “Haha, that’s funny.” To shouting, “WOW! That’s me here!”. This is the likeness threshold.
Beyond that point, you can see more than ever the fascination people have with their self-representation, a powerful emotional lever that has yet to be taken advantage of in interesting ways.
We believe we can spearhead a new branch of storytelling. In gaming, communication and other domains of the entertainment industry, thanks to the power of self-casting content, there is room for unheard-of experiences, from Star Wars-ish holograms to facing a lifelike avatar of your best friend as the final boss in your VR game (3).
A Burgeoning Ecosystem
The 3D look-alike avatar ecosystem is very young, and most companies are still focused on the tech stack that will power future end-user applications.
It will be highly interesting to see which companies — enablers or content providers — will prevail in the coming years.
We are also looking forward to seeing the reaction of the current kings of selfie-based content, namely Snapchat, which recently introduced Bitmoji to the 3D world, and Facebook, with its playful avatars for VR.
What about you? Any insight for the future of look-alike avatars? Please share your thoughts!
(1) full disclosure: that’s us
(2) Morphin tech and app was still WIP when that post was published
(3) only caveat, beyond the likeness threshold, is getting close to the uncanny valley
Pictures credit: Processing pipeline for avatar creation, Engadget
