One more way to touch the digital world — How innovative technologies and smart devices build a new image of our world
Introduction
New innovative solutions and technologies are quickly appearing in your life. Likewise, that is not an unexpected fact that most humans want to share their personal life with others even if there are some questionable doubts regarding this. For instance, according to the research done recently, the number of social media users worldwide changed significantly from 2010 to 2018 — a 250% increase in the number of users has been noted. In addition, more than 2.62 billion people used social media accounts at the beginning of 2018, and a 50 percent increase is foreseen by 2021.
But social media is not the main topic of this article — smart devices and gesture control systems.
Gesture control devices
And what devices are people using to connect to their digital world? As part of 2017 year-end roundup, there were up to 4.78 billion mobile users in the world. As the world becomes increasingly interconnected, both socially and economically, technology adoption remains one of the defining factors in human progress. The technology usage rates are increasing rapidly in emerging markets but there is still a lag behind rich countries where more than 87%[1] of adults have access to the Internet.
Overall, now the adult internet users worldwide want to receive something more from the technology industries than just access to the Internet and smartphones. While the Internet is growing, a new technological sector is rising together. Global companies are developing new solutions focused on the physical interaction with the virtual world. This popularity, as a result of digital games, will be developed increasingly in the future because of the engagement through its stimulating audiovisual effect and adaptability. Physical interaction through its ability to design game dynamics will be used in many different industries and solutions very soon. The most common example of this is an ability to operate physical objects remotely using only hands — you can control devices (e.g. car radio) with the wave of a hand. In 2015, one of the most popular German car producers showed a new technically innovative product which would be installed in all its new cars. A new solution based on the “gesture control system” will detect a driver`s hand gestures to operate some functions of the car such as taking a phone call or turning up the radio. Let`s go forward and think about other possible solutions which can be connected to these innovative developments.
[1] Spring 2015 Global Attitudes Survey Q71 & Q 72

The ability to control devices with gestures alone has a broad range of applications in IoT (IoT — Internet of things). You can control a TV set, a drone, a car toy or other stuff based on touchless gesture control systems. Does it sound good to you? It`s amazing from my point of view. Additionally, this solution has one more upside — a natural user interface system. A natural user interface (NUI) is a system for computer-human interaction that the user operates through intuitive actions related to natural human behavior. It is critically important to let users quickly master control with minimum learning. Overall, “the system should always be designed with the use of context in mind”, said Bill Buxton, a principal researcher at Microsoft.
But let`s go a little back. In the 1970s, wired gloves were invented to capture hand gestures and motions. The first gloves to use multiple sensors was developed by Dr. Gary Grimes in 1983. According to the research information, the gloves used a different sensor mounted on cloth. Over the past 25 years, that technology has resulted in many successful products that offer total wireless connection.

During the 1980s, the sensor technology developed rapidly mainly because of Cold War fears and the natural expansion of industry in many European countries. After 1993 the first gesture recognition system was built based on human-computer interaction and then 10 years later the first commercial product was released. The EyeToy product for Sony PlayStation 2 consists of a 2D camera that translated human body movements into game controls was released on the market in 2003.

It was the first step to building a new market sector and the beginning of the digital world. Generally, we can indicate two types of technologies for human interaction with a computer such as 2D and 3D gesture recognition. Basically, the 2D system does not require special cameras or even sensors. The system relies on the following steps:
· Segmentation (separate pixels around the hands or body from the background)
· Feature detection (statistical — based on a center of equivalent rectangle and contour — based on the outline of users` hands or body)
· Gesture recognition (directional, action based on point, grab or select, custom libraries)
A 3D gesture system is a little bit more complicated. It requires special cameras, projectors, and sensors to sense depth and human motion. The measurement is based on microwave, light or ultrasonic waves.
Vision-based gesture recognition
As was mentioned the 3D gesture recognition system is based on advanced cameras. But what exactly happens inside this system? The process is simple in itself. A camera captures an image, this information is used for generating the x, y, z coordinates for each pixel. The acquired image is being processed and compared to itself in the loop until the system recognizes a change. When a change is recognized in the pixel matrix, the system is sending the comment such as left, right, down, up.

In recent years, a number of 3D vision-based applications have been developed widely. It is necessary to indicate a few downsides of this technology. The important issue concerned is how to extract the human hand form the image captured effectively in real time from the complicated background. Additionally, the process requires a lot of power and generates heat which has to be neutralized effectively. To implement image processing in real time, huge amounts of the same data (comparing process) requires rapid speed technological processing. A customized algorithm built on the FPGA architecture works out better as compared to the standard architecture (x86). The algorithm based on FPGA receives the acquired image and processes it effectively, generating low heat and using a low amount of power.
Color segmentation technique known as color thresholding is used in this system to differentiate human hands from the background. The color technique is being carried out based on the adaptation and slight modification of the grey level algorithm. The segmentation technique is transferred to the RGB color information (red, green, blue) of the object extracted from the background. This technique consists of the following three steps: the original image is being acquired, the system calculates the range of RGB intensities for thresholding and thresholding is applied.
Where are we now
The 3D gesture recognition system could have a significant impact on our future. So far, the companies have released a lot of IoT products based on this technology. The simple example here can be an Xbox 360 which sold about 14 million units. The hand gesture recognizer system presented in this paper is suitable for controlling hardware application in real time. The system relies on FPGA architecture boards to speed up the computing or computer vision challenges. The system is able to track and detect objects and can be used in many different market sectors such as healthcare, education, automotive, robotics, home media, security, video games, and mobile phones as well.
Written by Lukasz Kudlak
Expert Janusz Wolak

