The Pathway Forward for Mapping Broadband Speeds in America
As a country, we need to do a better job mapping broadband availability and speeds. Local policy makers and digital inclusion champions need this information to make decisions about where to focus their efforts and where additional broadband infrastructure should be built. Furthermore, they need to understand what paying customers are actually getting for their money. Right now, FCC 477 data is the best resource available, but it doesn’t provide a clear picture about what is actually available in certain neighborhoods and it definitely doesn’t provide on-the-ground truth about the actual broadband speeds customers receive.
I am here today to tell you we’ve built something that can provide your community with more accurate broadband data. Working with our civic tech and startup communities, Louisville Metro created an open-source solution called SpeedUp (see: SpeedUpLouisville.com) for you to see the actual speeds that residents are receiving at a census tract level. And, Tech Oregon is about to take on the task of building it for the whole country!
Read below to learn more about the application and some general background information about broadband mapping and the SpeedUp development process.
About SpeedUp
The solution has three primary functions:
- Test internet speeds using the M-Lab speed test (this is what google.com uses when you type in “speed test”)
- Visualize M-Lab data on a map as well as the visuals provided at M-Lab site
- Collect anonymized survey data about service quality and pricing.
You can see the deployed product at SpeedUpLouisville.com & SpeedUpSanJose.com.
Open Source Information:
Read more about it at the Open Gov Coalition GitHub Repo
And, check out the Open Source Code
Screenshots:
Background
How did we get here?
The FCC provides 477 data in an attempt to provide data to local and state governments about the availability of broadband in their community, but it has its limitations. Not satisfied with the FCC 477 data, communities started crowd-sourcing their data from their residents to fill the gaps. We built SpeedUpLouisville.com, San Jose deployed SpeedUpSanJose.com, and North Carolina set out to map what broadband looks like in their state (I am sure we are missing other efforts). Even the telcos will tell you that we, as a country, aren’t doing good enough mapping availability (again, this isn’t what we are getting for download or upload speeds).
Here is a quick syllabus of articles of people complaining about poor broadband mapping:
- USTelecom: Reinventing Broadband mapping is needed to close the digital divide
- How bad maps are ruining American Broadband
- Microsoft says the FCC ‘Overstates’ Broadband Availability in the US
- NDIA proposes community-friendly changes in FCC Form 477 data
- (Podcast) Municipal Equation: The Map is Wrong
History of the Project
Three years ago, at a Code for America hackathon, the Louisville civic tech community, in partnership with Louisville Metro Government, created SpeedUpLouisville.com. This project followed in the footsteps of an Open Seattle and Open Technology Institute (OTI) project to map broadband speeds in in the Emerald City. After the initial push at the hackathon, we worked with a local start up to build out the tools, features and visualizations to make it a more useful including upgrades to the maps and the data being collected. We launched SpeedUp in April 2016 and gathered a few thousand speed tests in a couple months. Usage plateaued after our small social media advertising budget ran out.
Pivoting to M-Lab
We quickly learned that creating our own speed test provided valuable data, but we were never going to get the number of users that someone like, say, Google could get. Talking with OTI about possible next steps, we started to investigate switching our speed test from using a custom application to using the Measurement Lab (M-Lab) speed test to feed our data into their crowd-sourced, open data repository.
Providing some synergy to our efforts, Google uses the M-Lab platform as their speed test. When you type in “speed test” into the Google search, it pulls up the M-Lab speed test. The results from the speed test are then routed to the M-Lab servers and made available to the public in a free open source database. This provides real, accurate data about what speed tests look like across our country. It follows best practices it has been used in research, and it’s better than anything that’s available to local and state broadband advocates, if you can wrangle the data.
Seeing the opportunity with M-Lab data, we turned our application into an M-LAB speed test and visualization tool for their crowd-sourced data. After a month, the results spoke for themselves, we increased our speed test data capture by over 1000%. We had a real data about the digital divide. But, we wanted to go beyond a few communities, we wanted to see SpeedUpUSA become a tool for the whole country. We wanted everyone interested in broadband to have actual data that they could use at their fingertips. As a local government team, we could not justify using our limited resources to build it for the whole country. In order to move forward we had to find a partner, and I am excited to say after a year plus of searching we have found someone to carry the torch forward and lead us into the next great frontier in broadband mapping.
The Future
The future is bright for this project because we have someone with the vision, time, and resources to make this project what we hoped it could be. After a year or so of off-and-on seeing if someone would take on the project, we have finally found a partner with the vision, organization, and the funding to take the project on.
We are excited to share that the next step of development will kick off in April at Hack for a Cause in Eugene, Oregon. They’ll work to advance the technology originally created in Louisville, and make it available to
all cities and rural communities across America.
We believe that Hack for a Cause, which is backed by US Ignite and the Technology Association of Oregon, is going to give this application the best chance of being successful. This project started as a scrappy civic tech attempt to do something well-funded institutions couldn’t do. And, it is with this spirit that we hand the figurative leadership baton to our new friends in Oregon to build a tool that has the potential to map the digital divide more than any other map or tool on the market.
Amnesties
SpeedUpLouisville didn’t originally start as an open-source project. The start-up we worked with built the tool with us with the purpose of re-selling it to other communities but met with limited success. Unfortunately, I think they were a little ahead of the market or there just wasn’t appetite from local governments to pay for the tool. Part of that may be because digital inclusion programs are just taking root in local governments and non-profits. Whatever the reason, they decided to stop selling the tool and currently only maintain a couple instances. We didn’t want to see the project die so we purchased the code from them to make their updates to the tool open source and available to everyone (that is if we could find the right partner to build it out and maintain it).
During the transition from being a managed application to open-source project, there was a lapse in maintenance, and we lost the data we collected. This obviously frustrated us, but we knew that most of the historical data the tool visualized was safe and sound on M-Lab servers. Also, M-Lab made tweaks to their tool so our integration on SpeedUpLousivlle isn’t exactly what it should be. But, the purpose of this post and this project is to share what we learned and cheer-lead the Tech Oregon efforts.
Blog will be updated periodically as necessary
