Logging FRC robot data to a USB thumb drive

Example logs as displayed in the FRC Message Console.

The obvious first question that arises is why? Why log your robot debug info to a USB drive instead of to the console or to the roboRIO? Well, as it turns out, there are many benefits to storing your logs on an easily removable drive which could make debugging problems at competitions significantly smoother for your team. In this article, I’ll be detailing the good, the bad and the ugly as it relates to this type of logging.

The Good

Custom Log Formats

When logging to the console, you have very little input as to how your data is logged. It comes out as a string, and the contents of that string are about as much as you’re able to control. Sure you could add a timestamp or a log level, but graphing that data or using it in any other programs would likely be a pretty big pain.

This year, our team used badlog (thanks team 1014!) to save our data in a CSV format which, when used with the accompanying badlogvis, can output some pretty extraordinary looking visualizations of your logs from the match.

An example output from badlogvis. More examples available here.

Ease of Use

In comparison to having to power up the robot and SSH into the roboRIO every time you want to download a log file, pulling out a thumb drive and plugging it into your computer is pretty painless. It allows for quicker access to the critical log information that needs to be seen and processed before the next match. There were certainly a fair number of matches over the past season where I unplugged the drive as soon as I was able to get on the field, and quickly fled to the dark corner of the pit with my laptop to process the data before the robot even arrived. In total, having a USB drive likely saved our team at least a few hours of booting up the robot, SSHing in and downloading the latest log file.

No Network Throttle

Numerous people on ChiefDelphi have reported lag due to a large number of console logs flooding the network. Luckily, since your files are stored on a USB drive and never sent over the network, you can rest easy knowing your logging won’t cause any additional network latency.

The Bad

Corrupt USB Drives

This could have simply been an issue with how we implemented our logging (or the format of the drive) but on occasion, it would show up as corrupt when being plugged in to our computers in order to process the data. Luckily, the Windows automatic repair always took care of the issue for us, but it was still a little off-putting every time the corruption happened.

One definite way we found to reduce the likelihood of corrupt USB drives is to turn off the robot before unplugging the drive, though, unfortunately, that still didn’t entirely solve the problem.

Corrupt Log Files

Corrupt log files usually occur when the robot is turned off in the middle of writing a log. It’s more common than you might imagine (especially if you’re logging at a fairly high frequency, as we were), but thankfully, due to badlog’s design, the log files typically only require a minimal amount of work in order to become usable again. Still, this issue would not occur if you were only logging your data to the console.

Potentially High CPU Usage

This mainly depends on the number of logs you have and the frequency at which you’re logging. Writing to a disk (even if it’s not the roboRIO disk) can eat up a fair amount of CPU, so if you’re doing CPU intensive things on your roboRIO, such as vision processing, you might consider logging fewer things, or at a slower rate.

An example robot log (as seen in the Driver Station Log Viewer) displaying high CPU usage.

The Ugly

Unplugged USB Drive

With external drives, there’s always the possibility that it come unplugged at some point before or during the match causing important logs to be lost. Thankfully this only happened once or twice during the season. The more common situation, though, was forgetting to plug the drive back in after processing the logs. In hindsight, this issue likely could have been mitigated by using the roboRIO as a backup place to store logs if the USB drive was disconnected, but we never got around to implementing that code, and it wouldn’t have fixed the drive coming unplugged mid-match.

Summary

So, with all this said, should you start logging all-out to a USB thumb drive instead of to the roboRIO or the console? Potentially, but probably not. In reality and in the heat of competition, things happen: drives are left unplugged or come loose after a hard hit, files get corrupt, or the entire USB port on the roboRIO falls out (unfortunately that last one is a little too real for us). Therefore, combining all three options and adding fallbacks where necessary will greatly increase your chance of having reliable, complete and useful logs at the end of each match.

If you’re interested in seeing an implementation of badlog in Kotlin, you can check out our 2019 code here. If you’re a Java team, team 1014 (the Bad Robots, also the authors of badlog) have their 2019 code available here.