Using Robot Scouter with Tableau To Collect and Analyze Scouting Data
A powerful scouting system that works for the smallest of teams
Scouting is an important asset for any team that wants to do well at competition. Data collected by scouts provide valuable insight into the capabilities of every robot on the field that translates into creating an effective match strategy. During alliance selection, scouting data allows captains to discover teams that are much better than what their competition ranking suggests and robots with unique characteristics that could never be shown by simply looking at their OPR or win-loss ratio. Scouting data is immensely important, but good things are often difficult to obtain.
For some, the process of collecting and transforming data into a usable format can be more of a hassle than what it may be worth. I marvel at all the innovative ways teams have made a scouting process work well for them: from manually entering scouting sheets to scanning QR codes to creating massive applications that takes care of every imaginable step from collection to analysis and more. However, I can’t imagine wading through hundreds of paper scouting sheets each competition or devoting massive amounts of time to creating custom collection, storage, and analysis programs for each year’s game.
Our team is by no means small, but we simply don’t have enough resources to allocate that much time and effort to creating a new scouting system each year. We wanted something both powerful yet ridiculously simple at the same time. A system we could re-use with minimal adjustment each season. We wanted a collection process that wasn’t tedious on scouts; a way of compiling data that didn’t require frantic data entry. We wanted a way to easily visualize our data in all kinds of ways that weren’t just specific to a particular year’s game. After several years of working on this problem, we finally had a solution that could realize all of this.
Most scouting systems can be broken down into three steps: data collection, data storage, and data analysis. Our team uses two different tools to make all three steps go smoothly.
Using Robot Scouter to Collect and Store Scouting Data
When our team first started scouting, like many teams, we just printed out paper scouting sheets and told people in the stands to fill them out whenever they felt like it. The sheets would all go in a pile, never to be seen again until a bored freshman tasked with organizing totes discovered the dust-filled stack several years later. Data collection? Yes. Data storage and analysis? No and No.
Software Makes (Almost) All Things Better
Two years ago, things began to change. One of our former members got fed up with paper scouting sheets that were always so poorly organized and never used. And thus Robot Scouter was born.
Robot Scouter is an android application that serves as both a collection mechanism and storage database for scouting data. With Robot Scouter, our team creates custom scouting templates that are accessible on every device linked to the same Google account. Scouts follow the template to collect locally stored data that is automatically synced together with data from other devices/scouts whenever there is Internet access. At the press of a button, all the collected data is exported in spreadsheet and JSON format (more on this later) that can be easily moved from the exporting device to a computer.
Robot Scouter’s automatic syncing makes it really easy to see all of the data collected by different scouts in one place without fussing with data entry or setting up databases. The customizable templates also allow our team to easily modify what data we are scouting for, something that’s really nice when the game changes every year.
Scouting Logistics
Our team uses 6 inexpensive Android tablets to collect our scouting data. Each one is labeled with a different position on the field (ex. Red Alliance Far) and the scouter using that particular tablet is responsible for scouting the robot in that position. After someone is done scouting, they simply pass the tablet on to another available team member, ensuring that there is always one person scouting each of the 6 robots in a match. Whenever we want the scouting data out of the app, we briefly turn on a mobile hotpot in between matches to sync the data, and then press the export button. Now that wasn’t too bad, was it?
Everyone on our team scouts when they’re in the stands watching — regardless of whether they’re new or a subteam lead. Before competitions, team members go through a training presentation and a scouting practice run to ensure that we are collecting consistently high quality data (Here is the link to our presentation this year if you’re interested).
This is just our team’s approach to organizing scouting and it’s definitely not the only way of doing things nor is it the absolute best. Teams should always organize things according to what works best for their specific needs. Here is an amazing post on Chief Delphi that goes much more in depth about how to organize scouting effectively.
So now have a way to easily collect and store scouting data (Hooray!). But this is only two of three steps in our scouting system checklist. How can we actually use the data we’re collecting?
Using Tableau to Analyze Scouting Data
Initially, our team’s definition of ‘analyzing’ the scouting data was just looking through the Robot Scouter user interface at each team’s performance for each match we had data on, and accessing the analysis spreadsheet that’s exported by default. Don’t get me wrong, this was by no means a bad way of using the data (after all, we managed to form a successful alliance that got us to division semi-finalists at Houston using this approach). The default analysis spreadsheet is great and contains a lot of useful information. However, things can always be improved upon, and we wanted to be able to do more things with our scouting data.
Enter Tableau.
For those who aren’t as familiar, Tableau is a data visualization software that makes it super easy to see data in all sorts of different formats like graphs, charts, maps, and more.
Our team had always heard of other teams using Tableau for their data analysis but never looked into it for two main reasons. Firstly, we had no idea whether Tableau was even compatible with the data exported by Robot Scouter. Secondly, we thought that it was kinda expensive — $70 dollars a month for a tool that wasn’t guaranteed to work for us is not something we were exactly jumping to get.
In the middle of the build season this year, we stumbled upon something unexpected in the Virtual KOP. Teams had access to Tableau Desktop for free?! Now, some of you guys might be laughing at the fact that we didn’t realize this until 4 years after the software was first included in the KOP (Thanks Skunkworks!). (In our defense, we’d probably seen it in the past but never really paid much attention to it because we didn’t really have a use for it at the time)
Sweet! The Virtual KOP has solved all of our problems; onwards to a glorious season of analyses and graphs! But wait… there was another problem too. Would Tableau be able to interpret how scouting data is exported by Robot Scouter? Early attempts at answering this question were not very promising. It turns out that Tableau does not like the format of the JSON file that’s exported by default (The program literally stops responding for a bit because it’s trying in vain to figure out how the data is structured). But what if we re-organized the scouting data a little bit before it’s fed into Tableau?
Converting Robot Scouter Exports Into a Format Understood by Tableau
Our team wasn’t to be disheartened by our initial failures. We were so close and there had to be a way. After some fiddling around, we had a solution.
Before we put our scouting data into Tableau for analysis, our team runs a simple Python script that converts the Robot Scouter data exports into CSV files formatted in a way that’s understandable by the software. The code can be found here and is fully open source. Not only does the script convert the data, it can also help filter out empty forms that were accidentally added but never filled out by scouts.
We go through the entire process of exporting data step-by-step here, but if you just want a concise summary for using the transformation program, feel free to skip it for now and just consult the README on the Github page.
Effectively Visualizing Data in Tableau
Tableau can really be an extremely powerful tool for data analysis because it’s insanely versatile and allows for data to be visualized in so many different ways. This last season, our team used Tableau visualizations of our scouting data primarily to help with two aspects at competition: match strategy and alliance selection.
Before each one of our matches, our team uses scouting data to prepare a ‘briefing’ containing an overview of all the teams playing in the match to help our drive team determine what should be focused on. This year, these briefings contained information on each team’s ability to score game elements in various locations on the field in addition to other things like their climbing ability.
Our team also creates an alliance selection workbook to help find alliance partners that will work well with us. We create ‘vizzes’ that compare each team’s performance in all sorts of different categories ranging from the total number of points they score to how often they perform certain actions on the field like defense. Additionally, we create a dashboard that allows us to view all of the scouting data and comments we have on one or two specific teams.
We won’t go more into more detail about data visualization with Tableau past a brief overview of what our team has done because we’re still relatively new to the software and there are far better resources created by teams that have much more experience than we do. Tableau has a designated webpage filled with amazing tutorials that are tailored for FIRST teams, and the Bionic Black Hawks recently gave a great conference about how they visualize their data to gain competitive success.
Looking Ahead and Final Thoughts
This year has been a big one for our team in regards to our scouting system and we really have learned a lot. There’s a lot that can be improved upon and we will continue to work on those problems continuing into our next season and beyond. One of the biggest goals is to redesign our data conversion program into something that is easier to install but more powerful at the same time. Make sure to look out for that in the near future!
The scouting system we’ve been working on these past years is definitely not the best one a team could ever make; like everything else in the world, it has its advantages and drawbacks. Still, its combination of both power and simplicity is makes it really difficult to look past.
Though we focused on FRC specific games, there’s also no reason for why such a system wouldn’t work for FTC as well — you’ll just have to purchase Tableau instead of getting it from the KOP. Alternatively, you could use our team’s own analytics application designed specially for Robot Scouter exports and can be found here. Click here to check out the code!
Whether, you’re on the hunt for an easy scouting system for your team, simply interested in how other teams scout, or are just bored, I hope that this article has been helpful for you! If you have any questions or want to know more about using Robot Scouter for your own team, leave a comment below or send an email to sertscouting@gmail.com.
Thanks for reading, and happy scouting!
