NEON: humanity’s next best friend
Let’s discover how NEON might become your cashier, your makeup advisor, and even maybe your friend
First of all what is Neon? Neon is a subset of Samsung that aims at creating an artificial virtual human. The goal is to enable the use of artificial virtual humans in daily life and to revolutionize the HMI (human-machine interface).
Why is it important?
Today the way that we are interacting with computers when we are buying train or a bus ticket, checking in at an airport for a flight or checking out at the supermarket is not efficient. It isn’t intuitive and most would prefer to talk to a human. However, to increase productivity, humans have been removed from such functions. During COVID-19, more and more trivial activities and daily tasks have become contactless.
An example that appeared during COVID: automated phone systems. This system developed and all society needed to deal with it, we all saw that this was very inefficient and not adapted to human needs (questions, faces, speed, etc…)
COVID-19 showed us how much we need a better HMI, people need to see a face, need to hear a human voice, and need to communicate more naturally, it’s why I think Neon has even greater potential now.
Another reason why is think it’s important, it’s the scientific one. Is think working to recreate emotions, memories and human interaction is fascinating and crucial for the AI revolution. Also sometimes trying to recreate something is very often leading us to better understand the thing.
What’s the tech behind it:
First of all, what I’m going to detail about the technology is based on the short R3 whitepaper and speculation that I’ve made. There is little information about the real tech behind Neon’s technology as it is in active development.
Neon is using several technologies to create an artificial human, they divided this tech into three systems: Core R3, Spectra, and Nebula.
Core R3 :
The core R3 system is the pillar of Neon tech, R3 stands for Reality, Realtime, Responsive. It’s from what Neon is saying the three big pillars of natural interactions. The R3 part is the rendering part, what the user is seeing. From what the whitepaper is saying it’s very impressive tech.
To achieve a human-like rendering there are not using CGI or any other 3D tech for photorealistic rendering. They are saying that they are rendering each individual frame and that is not some image modification tech like face-swap or deep-fake. From this, we can suppose that they are using a styleGAN type of neural network. GAN is a type of artificial neural network where two networks compete each other in zero-sum game. StyleGan is a proprietary technology from NVIDIA, which is using this type of neural network to create fake human faces.
We can imagine that there are using a similar type of technology for the creation of the Neon’s face and that same AI rendering technic is used with their behavior network to create a video feed of a human with accurate facial expressions.
Another critical thing for a natural interaction to occur is a life-like human voice for this we can suppose that there a using some already sean type of neural network like Google did with Deep Mind or a most famous example with the demo of google duplex at the Google I/O of 2018.
It’s also important that the facial expression, the voice, and the lips match what the NEON is saying (the subject, the emotions) it’s here that we realized the core difficulty of Neon, cause everything that I’ve just said has already be made individually but never been assembled together. The difficulty here is to stack all this technology together and to make with all of this a consistent and efficient AI. But I will go deeper with that in the challenge section.
There is another tech for the R3 core but I will just skip them cause they are not that important and relevant.
Spectra :
Spectra is the brain of Neon, its goal is to bring intelligence, understanding, and memory to R3. No less!
Spectra is one of the most crucial steps for NEON, cause it’s this that will definitely make a difference and enable human-like interaction.
We can guess that it will use for this Natural Language Processing. The understanding part is already used Alexa, google assistant or other known technology but for the creation of the answer, we should go and see from experimental technology like Open AI’s GPT-3 or Google’s Lambda. We could imagine the same type of technology to be used in this part. For the memory one, this type of model already has a type of memory but it’s not a very consistent one and not a long-term one. Another problem is that NEON will need a visual memory and understanding to advise someone in a makeup shop, for example, liked they’ve shown in one of their trailers. We could imagine some descriptive AI, but it would have to adapt depending on the situation.
Nebula :
Nebula is the “oxygen” of Neon, it’s the cloud part of this system. It’s very important cause there is no Neon process that is done on the client-side. All of this will be done remotely, it’s a great example of 5G usage by the way. Neon is a very ambitious project and the cloud side is going to be extremely powerful to run thownsend of neon simultaneously and with extremely low latency. It’s will need and gigantic money investment to deploy NEON on a cloud-scale (low latency so …., gigantic compute power)
What are the biggest challenges?
Like you saw NEON is made up of already know technology but which are interconnected and stack up together and it’s why it’s becoming such a hard task. I will put here some of the biggest challenges that NEON has to go throw on the tech side :
1) The accuracy problem :
AI models are not that accurate they can be easily misleaded if somethings is very different from the training dataset the AI will usually produce a random output. Usually, an AI model as to have a low error rate to be used, but it’s okay if sometimes it failed or is giving an inaccurate result (which append pretty often with AI models tested with randomly or nerver-trained-on data) The problem here is that it’s kinda like a waterfall, like we saw earlier every one of this models is crucial to a natural conversation and there are all interconnected and that’s mean that all models need to be super accurate if you want to have an overall Neon that is working fine.
Mathematically speaking, the more you’re stacking models together the more you have a chance that the global output will be inaccurate or wrong. It’s why all of these crucial systems need to be super accurate plus able to deal with unexcepted inputs. And this is really hard to do.
2)The ambition :
The trailer and the website are making very impressive claims, and are even talking about a NEON that would be your friend. I think that if we consider the technology that we have today it’s will not be possible to have a real artificial human that can perfectly imitate a friend before some time. But I think Neon will be a great leap forward in this direction and an amazing improvement over HMI. Neon will have a human look, feel, gesture and will be able to converse naturally and do a great job (like selling tickets) but their ability to talk on various subjects and to understand and react will not be at the level of a human. I think this way cause the actual technology, hardware and software are not ready a tall, and I’m not even talking about an AI that would achieve singularity, in 2022 (the release date of Neon friend) an AI that would able to act, look and feel the same way a human does and to be able to be your friend will not be possible. We could take GPT-3 for example, it’s very impressive and you can actually hold a conversation with it, but it doesn’t understand anything and cannot understand difficult analogies, things that are requiring a reflexion. Neon might be able to create this kind of illusion with tasks like advising someone on his clothes but for other things like being a friend or being able to work with you on a complicated task, I’m still skeptical.
Why working on this?
I’m generally passionate about Artificial Intelligence, and I think that AI is going to impact the world like never before and that it’s going to help us take better decision everywhere and saving millions of lives (agricultural optimization, autonomous cars, economic predictions, improved political choice, better meteorological prediction, etc…)
Neon is a very ambitious project and a very complicated one, and its what we need to make breakthroughs.
Creating a general AI would make humanity jump to the next level, and Neon is a step further to this.
What impact on society we can expect from Neon?
We can except neon to make easier lots of interaction with machines and to reduce the number of people required to assist peoples in their interaction with machines (less support required). We can except if there is wide adoption of Neon a “Will robots take my jobs?” type of fear. But if Neon is only replacing old machines it’s shouldn’t be big deal. But like Neon might replace some humans jobs, society as a whole might realize the potential of AI and Neon might be the concrete” things that were missing for a general enthusiast and the creation of jobs vocations. If things are getting like this way it’s might be the start of the 4th industrial revolution, the AI one. (A big number of people with qualifications in AI before the transition of jobs, it’s the key thing for a good transition)
What I’m supposing here is an ideal scenario and it’s not that realistic but I think that it’s this kind of technology that will push the 4th industrial revolutions.
