Why and How We Measure Driver Performance
Driver performance is at the heart of modern fleet management systems. This is the first in a series of KeepTruckin blog posts about driver performance, where we explain the driver performance evaluation paradigm, its challenges, and KeepTruckin’s approach in tackling them. In this blog, we aim to capture the essence behind driver performance, the need for tracking it, and a high-level overview of the architecture.
Why Track Driver Performance?
Every year, 1.3 million people die in road accidents, and one of the major reasons is driver distraction. This is a problem self-driving cars are trying to solve. But while we wait for self-driving technology to mature, why not reduce this number as much as possible?
The Federal Motor Carrier Safety Administration (FMCSA) identified driver fatigue as one of the main reasons for road accidents involving long-haul trucks in the U.S. As a result, in 2017 the FMCSA implemented an electronic logging device (ELD) mandate, aimed at reducing road accidents by enforcing the hours of service (HOS) rule. Using ELDs, drivers’ duty hours are logged electronically, eliminating the risk of human error, lost or damaged logs, and illegible manual entries.
While tracking hours does address driver fatigue, it does nothing to illuminate us about driving errors resulting from lack of training, negligence, or impatience. This is where driver performance comes into play. By studying data gathered from different sensors on the vehicle, we can determine the driver’s actions on the road. We can verify and quantify each driving action, and use that information to assign the driver a safety score. This helps fleet managers identify risky drivers and coach them as necessary, to avoid future accidents. Insurance companies use scores like this to offer driver insurance discounts.
Driver Performance Events (DPEs)
A driver performance event, or a DPE, is an event of interest that is recorded for the purpose of analyzing driving patterns on the road. A DPE can be positive; for instance, changing lanes to avoid an accident. Naturally, DPEs can also be negative; for instance, hard braking, speeding, failing to stop at a stop sign, and so forth. A DPE can fall into one or more of the following categories:
- Hard brake, hard acceleration, hard cornering
- Tailgating, cutoff lane changes
- Speeding
- Distracted driving
How We Detect DPEs
At KeepTruckin, we calculate driver scores by combining data gathered from:
- GPS sensors
- Inertial measurement unit (IMU) devices that track the force applied on a vehicle
- Electronic control units (embedded internal sensors that control a vehicle’s electronic systems)
- Dual-facing dash cams
But to assess a driver’s performance, we must examine each of the driver’s actions within its context. This can occasionally be difficult if there is missing information caused either by the unavailability of a given sensor, an unreliable or noisy sensor, or insufficient processing power in the vehicle’s embedded systems.
How We Process DPEs
As you’ll see in the diagram below, we keep our code separate, identifying DPE candidates on the ELD and then applying filters on the server side. This is because the server-side development iteration loop is faster than it is on the ELD hardware. We can also filter bugs and other unwanted behavior more easily on the server. Finally, this approach gives us more computation power for the heavy AI processing required to identify a DPE.
We Detect DPE Candidates on the ELD Side
Refer to the diagram above for our core workflow in creating a DPE. First, data from multiple sensors is used to generate a DPE candidate on the ELD in the vehicle.
To keep our ELD-side processing to a minimum, we have a constant threshold programmed into our ELDs that defines severe events. Because this threshold is applied across the board on all ELDs, regardless of how each fleet manager defines a severe event, we set it relatively low to cast a wide net. This way, we capture anything that is likely to be an event, and then process our clients’ individual thresholds on the server side.
The ELD compares the event’s intensity to the threshold, and if the event exceeds that threshold, the ELD saves the video generated during that time.
We Filter and Validate On the Server Side
The DPEs are transferred to our server, where we apply various filters for validation. In addition to validating the DPE at this stage, we also compare event intensities against the fleet manager’s threshold (because every company has its own definition of severe events, KeepTruckin provides an easy-to-use interface for fleet managers to customize their event intensity thresholds).
If an event’s intensity surpasses the fleet manager’s threshold, the ELD uploads the relevant video to the cloud and generates a DPE event. The video, along with the telematics data, is moved to an annotation pipeline, where our human-in-the-loop AI tags the video based on image and telematics data.
Our human-in-the-loop AI puts our algorithms through their paces. We employ two annotation teams to validate our algorithms: one team watches and annotates, or tags, videos of DPEs; another team second-checks events at random, whether those events were accepted or rejected by our algorithms, and whether or not they generated a video, to verify the algorithms’ accuracy. For context, these teams compare what they see in videos (when available) to accompanying sensor reports of speed, acceleration, angular velocity, jerking, gyroscope ratings, trajectory measures, and more.
How We Deliver DPE Reports and Driver Scores to Fleet Managers
The video of the DPE (see the example below), along with its assigned tags, then becomes available for fleet managers to view on their dashboard. On average, the entire process to collect DPE data and populate an annotated video on a fleet manager’s dashboard takes less than five minutes.
We trained our AI algorithm on millions of annotated events, and use it to assign a driver score based on all the DPEs the driver generated (see the screenshot below for an example of our driver scorecards.) Fleet managers use the driver performance score as part of a holistic view of their drivers’ performances, and as a trigger to provide additional training to low-performing drivers.
Coming Soon on This Topic
In this blog, we explained why driver performance is important, and how we at KeepTruckin measure it to improve road safety. In our next post on this topic, we’ll discuss the difficulties of estimating driver performance, and what type of data, systems, and engineering problems we have encountered in our quest to measure it reliably.
