Deepfakes from a Positive Standpoint | SoftServe
As numerous deepfake videos are flooding streaming platforms, people start to wonder. What are those videos? How are they created? And how can one tell the real content apart? If you have seen those videos of Mark Zuckerberg admit to controlling “billions of people’s stolen data” or Barack Obama calling Donald Trump names, then you have seen deepfakes. But did you realize you were fooled? The quality of artificial videos is so impressive that people are having a hard time spotting them.
Deepfakes and the technology behind them became famous because of its illegal usage such as harming people’s reputation by spreading falsified content. However, all those infamous uses of the technology are not the topic discussed here. There’s a different side of the story with deepfakes — let’s consider positive use cases that the deepfake technology offers that can bring enormous value to the business and even transform industries.
Face reenactment — the technology behind deepfakes
Face reenactment aims at generating face image with the identity of one person (avatar) and facial expression from another person (actor) preserving the background in the image of an avatar. Face reenactment has three scenarios:
1. “One-to-one” — avatar and actor identities are the same
2. “Many-to-one” / “One-to-many” — either avatar or actor identity is arbitrary
3. “Many-to-many” — avatar and actor identities are different, arbitrary
The “one-to-one” scenario is rarely useful for practical purposes. The second scenario applies to the real-world problems. However, it is limited to specific identities and therefore is resource-hungry when those persons are changed. The last scenario — “many-to-many” — overcomes those limitations while being the most challenging task.
Following recent advancements in the Image-to-image translation techniques, the SoftServe R&D team developed an algorithm that can translate a face image of an arbitrary avatar to the expression of any actor’s face. The resulting synthesized face image is highly realistic, has accurate expression transfer from the actor, and simultaneously preserves identity and background as in the image of an avatar. The proposed solution is very efficient in terms of speed, memory, and time-to-train. The entire pipeline is end-to-end, meaning only one image of an avatar and an actor is needed to generate the reenacted avatar face image.
Taking an alternative perspective
Now, when the solution works efficiently in terms of computational costs and quality, it could be easily scaled to many domains.
For example, museums competing with other numerous leisure time options are looking for ways to satisfy the needs of the modern visitor who craves sophisticated new experiences and ideas. Also, art imagery is now instantly available to everyone online, so museums need to sell a reason for people to pay to view the originals. And the face reenactment technology can give a completely new experience of getting the information right from the artists. Imagine that instead of reading long text labels, the visitor has an opportunity to get to know the long-lived artist and his or her work by watching reenacted videos. Given that most people are visual and audial learners, such form of interaction can be more appealing.
In education, there is also a need to adjust to generation Z audiences who grew up in a hyper-connected world where the smartphone is their preferred method of communication. Learning can be made fun and engaging for them with historical figures telling their exciting stories in the video format.
Entertainment and advertising. Making movies or video games with historical figures or actors that are no longer alive can be streamlined with face reenactment. Studios will save costs on altering video footage thanks to AI capabilities.
Personalization. In huge enterprises, employees often feel the lack of appreciation while face reenactment allows creating personalized content like the message straight from the CEO with minimum resources.
To conclude — nowadays, companies are urged to always keep up and satisfy the needs of the user to stay on track. Modern customers are demanding more and more impressive and engaging experiences. Face reenactment technology can be the answer your company is looking for. To view a demo of Face Reenactment for museums and get started, contact SoftServe today.
Read the paper on Face Reenactment by SoftServe R&D Team Marian Petruk https://arxiv.org/abs/2003.13840
