Acquiring data to develop in-cabin monitoring systems — The challenges

Whether it’s DMS, OMS, or any other interior camera system, acquiring data to develop in-cabin monitoring systems is challenging.

But… Why is that? Why is acquiring real-world data particularly hard for the in-cabin monitoring use case?

Data challenges when developing In-cabin monitoring systems

Advanced perception developers need data (lots of data) to train and validate their deep learning models. Crafting the right data is not an easy task… and even more so when we talk about interior monitoring.

Real-world data is very hard to produce for in-cabin

Imagine generating thousands upon thousands of real images for the different use cases:

• Driver operating status (on a call, eating, drinking, etc.)
• Driver driving status (fatigue/tension, drowsiness/arousal, drinking, etc.)
• Sideways/looking away detection (driver posture, etc.), driver authentication, etc.
• Lack of ground truth
• Occupant movement detection (food and drink, smartphone operation, smoking)
• Occupant condition (ride state, posture/safety, etc.)
• And many more…

If you need real images of each use case, you need to have cars with real cameras and sensors, you need to have real drivers and occupants, and you need to continuously record what’s happening inside the car.

This leads us to the first challenge:

1. Privacy

Privacy is an important issue when gathering data from the driver and other occupants.

For example, in Europe, every system that uses personal information (and a picture of a person is very personal…) needs to be GDPR compliant. That means you can take those pictures, but once you process them, you have to destroy them, or if you plan to store them, it needs to meet all the GDPR requirements.

And we reach a whole new level when those images are of children. It greatly limits your opportunities to collect data from the real world because the protection of children is even stronger.

But let’s suppose that in a hypothetical case, you manage to acquire these images, are you sure that you are going to have enough to train your systems to perform to the level these systems require? Do you have enough variability?

2. Variability

Variability does matter for training an accurate in-cabin monitoring system:

• Target Position
• Camera Location
• Car brand and interior
• Environment (including the car)
• Time of day
• Occupant distribution
• Driver and occupant’s age or ethnicity

If you are going to generate the data yourself, you’ll need to get a lot of different actors and actresses to play the roles of being in the car doing a lot of different things.

As well as all this, depending on the use cases, the variability may be completely different, such as the type of objects, their distribution, and how the occupants interact with them.

At the end of the day, a lack of variability leads to a bias in the system that car manufacturers and interior monitoring system developers can’t afford if they want to be globally competitive.

You need enough variability and close to reality accuracy to develop in-cabin monitoring systems so that they can apply themselves to the real world they are going to face in the final production stage.

3. Accuracy

Real-world data is limited, expensive, and time-consuming to get, curate, and maybe not even completely accurate when you have to manually label it.

Accuracy is always key when developing new technology, but in the case of in-cabin monitoring, where human safety is at stake, having accurate data with ground truth is critical.

This means generating physically correct data, automatic annotations (avoiding non-free-from-error manual annotations), sensor-specific data, unlimited scene variations, and ultimately, helping the system understand or be able to interpret the images coming from the real sensors and optical systems.

Can we afford to risk the system not recognizing that the baby seat is not properly anchored with the seat belt?

The system is going to apply itself to the real world according to the data used to train the AI behind it, and there is no room for error if we want to develop a trustworthy in-cabin monitoring system.

About Anyverse™

Anyverse™ helps you continuously improve your deep learning perception models to reduce your system’s time to market applying new software 2.0 processes. Our synthetic data production platform allows us to provide high-fidelity accurate and balanced datasets. Along with a data-driven iterative process, we can help you reach the required model performance.

With Anyverse™, you can accurately simulate any camera sensor and help you decide which one will perform better with your perception system. No more complex and expensive experiments with real devices, thanks to our state-of-the-art photometric pipeline.

Need to know more?

Visit our website, anyverse.ai anytime, or our Linkedin, Instagram, and Twitter profiles.