Why Every US Carrier Is ‘Winning’ Fastest 5G Awards

RootMetrics, OpenSignal, and Ookla gave ‘fastest 5G’ awards to three different carriers. What’s up with that?

By Sascha Segan

Which carrier has the fastest 5G network in the US? It depends who you ask. Because of the messy state of 5G in the US—and because of the different ways tests are run—it’s now possible for any US carrier to call itself the 5G leader.

The quick takeaway is that 5G networks are changing so fast that the ratings organizations are getting tripped up in the time between when they run their tests and when they publish their reports. Differences about where they test, and what devices they do it with, can come into play.

But don’t let any of these wins confuse you: the state of 5G right now in the US is not good. While our carriers are duking it out, they’re battling for last place. The US has some of the slowest 5G in the world. Ookla and OpenSignal are in sync on the issue. Our nation’s reliance on reusing odds and ends of old 4G airwaves for 5G, as opposed to opening up new mid-band channels, has resulted in a nationwide 5G experience that looks and feels just like 4G.

There is a light at the end of this tunnel. Verizon’s millimeter-wave 5G is the fastest in the world; it’s just that relatively few people can get it. T-Mobile’s mid-band, “ultra capacity” 5G will also help pull us up toward global standards. At the end of this year, C-band 5G, which uses fresh airwaves untouched by 4G, will likely pull Verizon’s performance (and possibly AT&T’s) up again.

The Key Factors

So why are our carriers claiming they’re the best for now?

Two key factors are probably how much Verizon millimeter-wave and how much T-Mobile mid-band each test hit. As those forms of 5G are much faster than others, the percentage of tests specifically on those networks make a lot of difference in final results.

AT&T says it’s the fastest based on Ookla Speedtest scores. Ookla uses crowdsourced testing, which means there can be some differences based on where different providers’ subscribers are testing and on which phones. The score AT&T cites averages results over Q4 2020. My guess is that their T-Mobile testers were supplying a lot of results before mid-band came to their towns, and that there was heavy testing of Verizon’s slow low-band network shortly after its very public launch in mid-October, swamping the Verizon millimeter-wave tests.

T-Mobile cites two reports: one from OpenSignal, which uses crowdsourced testing, and one from Umlaut, which uses drive testing. OpenSignal’s report claims to use more recent data than Ookla’s did—so there may have been more T-Mobile mid-band involved, and thus T-Mobile would end up rating higher on speed.

Umlaut’s testing involved only four cities: Washington D.C., New York, Chicago, and Houston. All four cities have T-Mobile’s new “ultra-capacity” 5G using Sprint’s old airwaves. As Umlaut points out, that’s much faster than T-Mobile’s broader low-band 5G. In Houston, for example, mid-band was 2.7 times faster than low-band. So a test like Umlaut’s, which looks only at areas with T-Mobile mid-band 5G, would show great results for T-Mobile.

Verizon is leaning on RootMetrics, a major drive-testing firm that drives across 125 metro areas in all 50 states. Root is particularly strong in rural drive testing, which has tended in the past to favor Verizon and AT&T over T-Mobile.

Root’s report does not appear to separate 4G and 5G. Unlike other testing firms, it factors calls and texting into its overall scores. And it uses a system where it aggregates statewide results and metro results into different buckets. Root also says these are results from the “second half” of the year, and the situation was very different for the carriers in July versus December. Results from earlier in the second half might have pushed Verizon’s comparative speeds higher, as they would have more (fast) Verizon millimeter-wave, no (slow) Verizon low-band, and relatively little T-Mobile mid-band.

OpenSignal’s report gives a median T-Mobile 5G download speed of 58.1Mbps, which would be totally impossible in Root’s testing, where T-Mobile had no metro areas with greater than 50Mbps. That really makes me think a lot of the Root testing happened before the T-Mobile mid-band rollout.

Ookla has a good article justifying its testing setup, which brings up even more variables, such as the design of the apps used to do speed testing, and their ability to detect whether or not a connection is actually on 5G. Before Android 11, it was very difficult to tell whether a connection was really 5G or not because of the design of the Android OS, and software would often get that wrong. In my mind, Ookla is too harsh on drive testing, though, because drive testing lets you get results across carriers at the exact same time and location, which crowdsourced testing cannot do.

Here’s another example of how methodologies change results. In our Fastest Mobile Networks study, we tend to get higher average speeds than any of the crowdsourced studies do—even ones that use the same testing engine. That’s because we only use the best possible phones and test outdoors in cars, while the crowdsourced databases have a lot of older phones and indoor tests, which tend to be slower.

OK, Which Carrier Is Really Best?

It’s a mess, and it’s a mess that’s changing month by month.

Until the US wireless operators have similar 5G technologies—which is going to take several years—it’s going to continue to be very difficult to easily compare them. The difference between millimeter-wave speeds and low-band is so huge, and the difference between mid-band and low-band is also significant.

That means you need to turn down the volume on these “nationwide” awards and look clearly at which technologies are available near you. If you spend time outside in Verizon mmWave areas, that amazing form of 5G will blow the others away. If your city has T-Mobile mid-band, that’s looking good right now. If your rural town is only served by a nearby AT&T tower and the other two carriers are three miles away over a hill, AT&T is your top choice.

Eventually all of us testers will start to agree with each other, as we did much more often in the 4G era. Until then, though, the race to 5G in the US is going to look a lot more like roller derby.

Disclosure: Ookla is owned by PCMag parent company, Ziff Davis.

Originally published at https://www.pcmag.com.