AI can detect distracted drivers on the road

“Texting is the most dangerous and alarming distraction on the road ”

National Highway Traffic Safety Administration (NHTSA)

According to the NHTSA, reading a text while driving at 55 miles per hour takes one’s eyes off the road the equivalent amount of time as driving across a football field blindfolded.

What Is Distracted Driving?

NHTSA defines distracted driving as any activity that diverts attention from driving, including talking or texting on your phone, eating and drinking, talking to people in your vehicle, fiddling with the stereo, entertainment or navigation system, anything that takes your attention away from the task of safe driving.

You cannot drive safely unless the task of driving has your full attention. Any non-driving activity you engage in is a potential distraction and increases your risk of crashing.

Nowadays, thanks to the help of new technologies, car companies are integrating systems with cutting edge technologies that can prevent crash accidents and reducing the number of deaths and injuries.

The next video shows a solution called drowsiness detection system as a new car safety feature. Check it out:

Just think about what a small camera, Internet of Things (IoT) and machine learning can do together to create a revolutionary product that can impact safe driving.

State Farm Distracted Driver Detection

State Farm launched a competition years ago through Kaggle. The competition was called “State Farm Distracted Driver Detection”, in which you are given driver images, each taken in a car with a driver doing something in the car (texting, eating, talking on the phone, makeup, reaching behind, etc). The goal was to predict the likelihood of what the driver is doing in each picture.

The dataset consist of 2D dashboard camera images that fall in the following categories:

  • c0: Safe driving
  • c1: Texting — right
  • c2: Talking on the phone — right
  • c3: Texting — left
  • c4: Talking on the phone — left
  • c5: Operating the radio
  • c6: Drinking
  • c7: Reaching behind
  • c8: Hair and makeup
  • c9: Talking to passengers

The train and test data are split between the drivers, such that one driver can only appear on either train or test set.

Taking advantage of the last days of my free trial on Custom Vision from Microsoft Azure, created a model to detect different types of distracted drivers.

Uploading the Dataset

Custom Vision is not allowing me to upload all the images to the project, so I decided to use 400 images Per Category.

After adding the images and tagging into different categories, it’s time to train the model. You will see that it take a few seconds, depending on the size of your dataset. The results are below:

If you go below the Precision and Recall Diagram, you can see the Precision Per Tag:

As you can see, “Drinking”, “Talking to passenger”, “Texting-Right”, “Hair and Makeup” and “Safe Driving” are the categories with low numbers.

The image below belongs to the “Talking to passenger” class but once tested through the model, it gets classified as “Texting Right”. The second guess is indeed “Talking to passenger”. In the picture, the right hand seems like is holding a phone. That’s the kind of problems in computer vision models, there are some classes that look very similar and the model struggles while making a prediction.


Check the next predictions, I think the model did pretty good!

Talking on the phone-Right
Talking to a passenger
Operating the radio
Safe Driving

Experiment: Model in PyTorch

Now, let’s try to create our model in PyTorch and use transfer learning for our purpose:

Experiment Details:

Platform: Google Colab

Data: Training, Validation and Testing Sets

Batch Size: 64

Data Augmentations: Random Rotation (15 degrees), Random Resized Crop (224), Random Horizontal Flip

Technique: Transfer Learning

Pre-trained Model: Resnet-152

Criterion: Cross Entropy Loss

Optimizer: Adam

Learning Rate: .0001

Number of Epochs: 20

Accuracy: 82%


Accuracy reached 82%. We can still get a better number though.

Talking on the phone-Left
Talking on the phone-Right

Future Work

  • Implement Distracted Driver Detection in Real-Time with a Raspberry Pi
  • Try to improve the accuracy by tuning the model
  • Try different pre-trained models

You can check the full project on my GitHub here:


StateFarm Distracted Driver Kaggle Competition

Real-time Distracted Driver Posture Classification

Data Driven Investor

from confusion to clarity, not insanity

Viridiana Romero Martinez

Written by

Artificial Intelligence and Machine Learning enthusiast. Data Driven Investor writer. Healthy lifestyle lover ♥

Data Driven Investor

from confusion to clarity, not insanity

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade