Hyprmeet — an Avatar Video Chat App for OBS/Zoom

Have you been curious about virtual avatars? From Digital Popstars selling out stadiums and digital venues to Virtual Influencers selling products with major brands, we are currently experiencing a revolution in the way we engage with content and these types of characters are popping up everywhere.

Image for post
Image for post

Insights about the Current and Potential Faceless YouTubers

Have you ever imagined your life as a YouTuber? What is the biggest concern you would have about starting your channel? For some people, the answer would be the burdens of revealing their faces in public. We all get it — it’s not an easy decision to make considering how brutal the internet can be.

When you search under ‘becoming a YouTuber without showing your face,’ more than 5 million results come out on Google.

Project done by Yoonsung, a machine learning intern at Hyprsense

Before We Get Into

Hyprsense develops real-time facial motion capture software, Hyprface SDK. It is a real-time facial landmark tracker built with machine learning, running on CPUs, Central Processing Units. We have never stopped developing our technology to make better and more accurate tracking solutions.

For this purpose, we have been recently working on new landmark tracking software on GPUs, Graphical Processing Units, for Windows, and finally released the new SDK called HyprfaceX. As the new product came out, we would like to share some parts of the development process focusing on the subproject our intern Yoonsung conducted.

Why Do We Try To Use GPUs?

First of all, let’s take a look at some differences between a CPU and a GPU. Both of them are basically microprocessors that handle data. However, as you can see in the picture below, they have different architectures for processing their tasks. …

Image for post
Image for post
Source

Project done by Jupiter Zhu, a deep-learning intern at Hyprsense

Along with the significant improvement of object detection/segmentation technology from deep learning, and the rising trend of virtual celebrity, Hyprsense is experimenting with whole-body tracking for virtual-beings developers.

At the core of it is the body/limb tracking network, and here is the report on the implementation of a network base on the architecture of the Openpose model, and how the training could be further accelerated onto a cloud platform like GCP.

Here is a Brief Overview of the Project:

Virtual celebrity landscape update & analysis

Last year, we witnessed the virtual celebrity trend starting to bloom. As 2020 arrived, we want to share with you some updates about how the industry is doing so far.

Who is still growing fast and what is their secret in growth? Is YouTube still the only platform going strong? Which talent agencies are thriving? For those who haven’t read the previous postings, check them out to better understand this article.

Related article 1: ‘How to Become a Virtual YouTuber/Influencer
Related article 2: ‘How Many Virtual Celebrities Are There?
Related article 3: ‘Digital Human is Becoming Prevalent

Edit(Aug 24th, 2020): Along with the researches, we finally released an avatar video-chat/streaming app for potential VTubers! Currently, the app, Hyprmeet, is free to download as a beta app so check it out and let us how you think. …

Concepting, 3D modeling/texturing, blendshapes, and integrating Hyprface facial mocap solution

Image for post
Image for post

What makes a character alive and beloved? We asked this simple and powerful question — and from here, Sandie & Avery were born.

In this article, we will follow the steps of creating Sandie & Avery from concepting, 3D modeling/texturing, blendshapes, and integrating our facial mocap, Hyprface SDK.

Edit(Aug 24th, 2020): We finally released an avatar video-chat/streaming app that you can create your own 3D avatars! Currently, the app, Hyprmeet, is free to download as a beta app so check it out and let us how you think.

Image for post
Image for post
hyprmeet.com

It contains a built-in avatar customizing tool, a virtual webcam feature for streaming/video call software such as Zoom, OBS or Discord, and our real-time facial mocap technology. We are planning to add more features such as VRoid 3D model support, button-triggered animation, etc. …

Available both on Android & iOS

If the lack of engineering resources was blocking you from testing our technology, there is no need to worry about it anymore. We just released the Hyprface demo app on both Android & iOS! Simply download the app and test our latest version of the mobile Hyprface on your phone.

Image for post
Image for post

What can we test from Hyprface app?

  • Live facial mocap:
    You can animate 3D characters with your own expression in real-time.
  • Multi-face tracking:
    You can use our app with multiple people up to five at the same time.
  • Character switch:
    There are five characters with their own unique blendshapes and you can switch characters by tapping the bottom center button. …

List of technical solutions from avatar creation to motion capture

According to ‘Think with Google’ article, “as the technology used to create VTubers becomes more accessible, their numbers are expected to rise.

Image for post
Image for post
Example of a virtual human, Avery, animated in real-time using Hyprsense facial motion capture software

Becoming a virtual character is a charming alternative for those who seek for fame, yet, do not want to reveal their face. One question arises — how can we become a virtual celebrity in a practical sense? What kind of technical solutions can we utilize?

Related article 1: ‘How Many Virtual Celebrities Are There?
Related article 2: ‘Digital Human is Becoming Prevalent
Related article 3: ‘Is the Virtual Celebrity Industry Still on the Rise in 2020?

Edit(Aug 24th, 2020): Along with the researches, we finally released a VTuber PC app on our own! Currently, the app, Hyprmeet, is free to download as a beta app so check it out and let us how you think. …

Virtual Celebrity Fun Facts & Statistics

Have you ever heard of Kizuna AI and Lil Miquela? If so, you would know about Virtual Celebrities! Virtual Celebrities — so-called virtual YouTubers or virtual influencers — seem pretty much like existing YouTubers and Influencers. They will live stream or upload daily postings on SNS. Except that they are not real but pure 3D graphics.

Image for post
Image for post
(left) Lil Miquela, (right) Kizuna AI

As virtual celebrities gain global popularity, we started to wonder from when they started and how many there are. These questions led us to dive deep to research the trends and found out the unexpectedly vibrant ecosystem and insightful data about virtual celebrities. …

Immerging technologies leading the new wave of innovation

This past March, Google presented Stadia and the world got wildly excited at the release announcement. Meanwhile, 5G is expected to be generalized sooner or later. Graphics chipset, display technology, and software based on AI/deep learning are advancing at a fast-growing rate.

All these immerging technologies are leading us to the new wave of innovation in mobile industries: real-time 3D graphics.

Image for post
Image for post

A Brief Journey to Real-time 3D graphics

Once there was a time when people would consider these 2D graphics of the highest quality.

Image for post
Image for post

About

hyprsense

Hyprsense develops real-time facial expression tracking technology to light up the creation of live animation.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store