The Future of Work and Hospitals

Imagine a patient walks into your hospital room. His heart is racing. He is sweating profusely. He is hunched over, looking down. He is trying to explain himself, but he is having a hard time. You observe him, glance at your tablet and make a quick decision — this man needs emergency help!

You press a button on your tablet and calmly say: “Sir, you are experiencing a panic attack.” The patient looks surprised. You continue: “I have already called the nurse who is on her way to take you to another room, where we can offer you immediate assistance.” The nurse opens the door and warmly says: “Follow me, sir. I will take care of you.”

What just happened? Let’s analyze how artificial intelligence (AI) saved this man’s life.

The room is equipped with multiple cameras, recording data in real-time. This data is available on-demand to the doctor though the tablet. The data is analyzed on-site and in the cloud. The healthcare data platform is powered by artificial intelligence. This system runs multiple AIs, each AI passing data to the next, in order to give the doctor superhuman powers.

The first model, powered by deep learning technology, was used for human detection, a category of object detection. This deep learning model detected people in the video, including the doctor and patient. Object detection is a computer technology related to computer vision and image processing that deals with detecting instances of semantic objects of a certain class (such as humans, buildings, or cars) in digital images and videos. Why object detection and not image classification? Image classification models classify images into a single category, usually corresponding to the most salient object. Object detection models are therefore more appropriate to identify multiple relevant objects in a single image. The second significant advantage of object detection models versus image classification ones is that localization of the objects is provided.

The second and third models was used for face detection and face identification. These models were created to identify which doctor was in the room and if the patient had a previous health record at the hospital.

The fourth model used pose recognition to interpret the movements of the body on a physical level, i.e. reconstruction of the 3D articulated motions. This model was used to transfer gestures of the patient onto a 3D skeleton. This model determined that the hunched over body language of the patient was indicative of depression or panic attacks.

The fifth model used action recognition to interpret the movements of the body on higher, semantic level, i.e. understanding the body’s movements over time. This model determined the patient exhibits several poses related to a panic attack.

When the patient spoke, the video camera detected his face and analyzed his emotional state. The sixth model was used to identify facial expressions in order to recognize emotions. The seventh model was used to perform facial analysis. These four models leveraged neural networks in their architecture.

Since the person looked down while speaking, the video was not enough. The camera also captured audio signal. Eight model used signal processing to separate audio from video, isolate multiple voice signatures and clean the audio recording, powered by deep learning. The ninth model translated audio signal into text using natural language processing. The tenth model used sentiment analysis to analyze emotional categories of patient’s speech.

Once the computer completed it’s calculations, the computer send results to the doctor’s tablet using a private API. The results were displayed in a tablet running an AI-driven app. The solution was showed the diagnosis with it’s probability score, a percentage to represent the probability of diagnosis.

Thanks to ten artificial intelligence models, the doctor received actionable advice and took immediate action to save the man’s life!

Start an AI project

At Produvia, we produce intelligent software. We specialize in artificial intelligence, machine learning, and deep learning. Let’s chat! Visit us at produvia.com