Tag Archives: speed test

An Alternate to the FCC Maps

I’ve written a lot of blogs about FCC broadband mapping. It’s been easy to criticize the maps since they are still full of errors and fantasy. I don’t foresee the maps getting any better as long as ISPs can continue to decide what they want to report in terms of broadband coverage and speeds. Too many ISPs have reasons for reporting maps they know are inaccurate, and it’s hard to think that’s going to change.

Perhaps having accurate maps doesn’t matter all that much. If BEAD comes somewhat close to solving the rural broadband gap, the FCC maps will quickly lose relevance. The FCC is always going to need some version of the maps to report to Congress each year on the state of broadband. But when the maps stop being a tool for deciding who gets grant funding, most local and state governments will stop caring about the maps – and nobody is going to much care what ISPs report to the FCC. Truth or fantasy won’t really matter, just like nobody cared about the maps just five years ago.

But I still think the FCC owes it to the public to provide a way to judge and compare the local ISPs. I’ve thought of a simple FCC tool that can accomplish that.

I think the FCC should buy the entire Ookla speed test database every month and make it available to the public in a portal where folks can see the speed tests actually reported for the ISPs working in their neighborhood. If a speed test comparison portal is made easy to use, it would be one of the best gifts the FCC can give the public. The FCC spent a huge amount of money developing the new broadband mapping system, and after all of that money, the maps are largely useless for the general public.

I will be the first to say that speed tests aren’t perfect. There are slow speed tests recorded for every ISP for reasons out of the control of the ISP. The biggest reason is WiFi routers that don’t deliver the speed to a computer that is delivered to the home. But the generic flaws of speed tests apply across the board to every broadband technology, meaning that speed tests are a great tool for comparing specific ISPs or technology.

Consider the following chart that represents Ookla speed tests taken in a suburban county over the last year. These results reflect over a million speed tests. The speeds shown for each technology are the overall average of all speed tests for each technology.

Download

Upload
DSL

37

7
Fiber

357

357
Cable

264

24
FWA Wireless

99

16
Fixed Wireless

12

7
Satellite

69

8

These results are similar to what I see in a lot of counties. There are some counties where DSL isn’t as fast as in this county. There are counties where fixed wireless ISPs perform much better than in this particular county. This is the only county where I’ve seen fiber have an identical average upload and download speed. But the overall performance of the various technologies is pretty typical.

These speed test results tests tell a great story about the local differences between technologies. This is information that is not readily available to the public. The FCC has ordered ISPs to create broadband labels – but those labels allow ISPs to report marketing speeds and will be no more useful to the public than the broadband maps. An FCC-sponsored speed test portal would allow the public to see how various ISPs perform in and around their neighborhood.

Detailed speed test results also can tell us a lot about any given ISP. It’s interesting to look at the fastest speed tests delivered for a given ISP. When I look at fiber based ISPs there are invariably some speed tests that are near the gigabit speed claimed by the ISPs in the FCC map. But the same is not always true for cable companies. I find some cable companies that are also delivering the top speed claimed in the FCC mapping. But it’s not unusual to find markets where a cable company claims gigabit speeds but doesn’t have any speed tests faster than 600 – 700 Mbps. That is still blazingly fast, but that is not a gigabit network.

Making the Ookla data available to everybody would be a huge public service. Local politicians always tell me that they have no way to judge the ISPs in their community, and this would give them a tool to do so. When used on a smaller scale, speed tests can also be used to show that some neighborhoods get faster broadband than others from the same ISP. As I wrote in a recent blog, folks can use speed tests to see that broadband speed on FWA cellular broadband die quickly as the distance between a cell tower and customers increases. This information would let the public understand broadband in a way that has never been understood before. Speed test data, when used in mass, will expose ISPs that exaggerate their speeds – and highlight ISPs that are honest.

So, FCC keep your maps for reporting to Congress, but please give the public this readily available tool so that everybody can get usable facts about broadband. ISPs that exaggerate their speeds will hate this idea – but honest ISPs will welcome it.

Measuring Mobile Broadband Speeds

I was using Google search on my cellphone a few days ago and I thought my connect time was sluggish. That prompted me to take a look at the download speeds on cellular networks, something I haven’t checked in a while.

There are two different companies that track and report on mobile data speeds, and the two companies report significantly different results. First is Ookla, which offers a speed test for all kinds of web connections. Their latest US speed test results represent cellphone users who took their speed test in the first half of this year. Ookla reports that US cellular download speeds have increased 19% over the last year and are now at an average of 22.69 Mbps. They report that the average upload speeds are 8.51 Mbps, an improvement of 4% over the last year. Ookla also found that rural mobile broadband speeds are 20.9% slower at urban speeds and are at an average of 17.93 Mbps.

The other company tracking mobile broadband speeds reports a different result. Akamai reports that the average cellular download speed for the whole US was 10.7 Mbps for the first quarter of 2017, less than half of the result shown by Ookla.

This is the kind of difference that can have you scratching your head. But the difference is significant since cellular companies widely brag about the higher Ookla numbers, and these are the numbers that end up being shown to regulators and policy makers.

So what are the differences between the two numbers? The Ookla numbers are the results of cellphone users who voluntarily take their speed test. The latest published numbers represent tests from 3 million cellular devices (smartphones and tablets) worldwide. The Akamai results are calculated in a totally different way. Akamai has monitoring equipment at a big percentage of the world’s internet POPs and they measure the actual achieved speeds of all web traffic that comes through these POPs. They measure the broadband being used on all of the actual connections they can see (which in the US is most of them).

So why would these results be so different and what are the actual mobile broadband speeds in the US? The Ookla results are from speed tests, which last less than a minute. So Ookla speed test measures the potential speed that a user could theoretically achieve on the web. It’s a test of the full bandwidth capability of the connection. But this is not necessarily the actual results for cellphone users for a few reasons:

  • Cellphone providers and many other ISPs often provide a burst of speeds for the first minute or two of a broadband connection. Since the vast majority of web events are short-term events this provides users with greater speeds than would be achieved if they measured the speed over a longer time interval. Even with a speed test you often can notice the speed tailing off by the end of the test – this is the ‘burst’ slowing down.
  • Many web experts have suspected that the big ISPs provide priority routing for somebody taking a speed test. This would not be hard to do since there are only a few commonly used speed test sites. If priority routing is real, then speed test results are cooked to be higher than would be achieved when connecting to other web sites.

The Akamai numbers also can’t be used without some interpretation. They are measuring achieved speeds, which means the actual connection speeds for mobile web connections. If somebody is watching a video on their cellphone, then Akamai would be measuring the speed of that connection, which is not the same as measuring the full potential speed for that same cellphone.

The two companies are measuring something totally different and the results are not comparable. But the good news is that both companies have been tracking the same things for years and so they both can see the changes in broadband speeds. They also both measure speeds around the world and are able to compare US speeds with others. But even that makes for an interesting comparison. Ookla says that US mobile speed test results are 44th in a world ranking. That implies that the mobile networks in other countries make faster connections. Akamai didn’t rank the countries, but the US is pretty far down the list. A lot of countries in Europe and Asia have faster actual connection speeds than the US, and even a few countries in Africa like Kenya and Egypt are faster than here. My conclusion from all of this is that ‘actual’ speeds are somewhere between the two numbers. But I doubt we’ll ever know. The Akamai numbers, though, represent what all cell users in aggregate are actually using, and perhaps that’s the best number.

But back to my own cellphone, which is what prompted me to investigate this. Using the Ookla speed test I showed a 13 Mbps download and 5 Mbps upload speed. There was also a troublesome 147 ms of latency, which is probably what is accounting for my slow web experience. But I also learned how subjective these speeds are. I walked around the neighborhood and got different results as I changed distances from cell towers. This was a reminder that cellular data speeds are locally specific and that the distance you are from a cell site is perhaps the most important factor in determining your speed. And that means that it’s impossible to have a meaningful talk about mobile data speeds since they vary widely within the serving area of every cell site in the world.

What’s the Real Speed of Bandwidth in the US?

Polk County SignRecently there were two very different reports published about the speed of broadband in the US. Comparing these two figures tells us some interesting things about the politics of broadband.

The first report was the most recent quarterly report from Akamai. They rated the US as 16th globally with average landline broadband download speeds at 12.6 Mbps. This number is up 10% from the previous year’s average speed of 11.5 Mbps. Akamai derives their speeds from looking at millions of actual downloads around the world.

At about the same time the FCC released its latest report on broadband and they said that the average download speed in the US for September 2014 is now 31 Mbps, almost triple the 10 Mbps reported in March of 2011. That’s a huge disparity with the FCC’s numbers at almost 2.5 times higher than what was measured by Akamai. How can the numbers be that far apart?

Akamai is measuring the actual download speed of millions of transactions on the Internet. This is how fast things are actually downloading and, because of the huge number of transactions that they look at, the number should be very accurate.

The FCC on the other hand is using a very odd measuring technique. First, they rely on speeds that come from 5,000 homes in the US that have volunteered to have their speeds measured. It’s pretty obvious in looking at the FCC results that these homes are not a random sample. I would suspect they aren’t measuring too many homes with dial-up, for example. If their sample is not random, then the results of the FCC tests cannot be multiplied out to accurately represent the whole country. That is one of the basic precepts of statistical sampling.

Another reason for the difference is that the FCC is basically looking at speed tests and not at actual downloads. Speed tests measure the highest speed that can be achieved to a specific test site created for that purpose. The FCC further says that their sample shows customers’ actual speeds are not much different than subscribed speeds. But my recent experience is that this disparity still widely exists. For example, in helping a western city look at broadband issues, we recently asked citizens there to take the Ookla speed test and report to us the results of the speed tests along with the speed they are paying for. The results were eye-opening with about 80% of customers seeing speed test results much slower than what they were subscribing to. In some cases the results were drastically different with households that subscribe to 100 Mbps services seeing actual speeds under 20 Mbps.

Another difference between the two sets of numbers is the little-understood fact that you can’t download anything faster than what the Internet is sending you. Somebody with a gigabit fiber connection from Google is going to see the same download speed from watching a Netflix HD movie as somebody on a 20 Mbps cable modem connection. Netflix doesn’t care what connection you have and sends the same stream to everybody, as does a lot of the Internet. Akamai is measuring this real speed and not the theoretical speed from speed tests.

The problem with the FCC’s method from a mathematical perspective the FCC’s method will skew the average towards the faster speeds in the sample. In a very simple example, if you sampled one gigabit customer and nine dial-up customers you would say that the average data speed was a little over 100 Mbps. While that would be true for those ten people, that is obviously not a number that has any practical relevance.

What is troubling is that the FCC makes policy decisions based upon the results of these tests. For instance, they routinely publish the percentage of homes that now meet their new 25 Mbps download standard to be considered as broadband. On the surface there is nothing wrong with what the FCC is doing, and for those 5,000 customers the reported speeds really is the average for that group. The problem is that those results show a different Internet than is really out there. For example, the FCC just also declared that 39% of the homes in the country can’t get real broadband at all. If the FCC is going to report an average speed for the whole country, those homes ought to be represented somehow in the the average.

There might be help on the way since the FCC is finally starting to require large ISPs to tell their customers the truth about their networks, including actual download speeds. There are new reporting formats circulating now at the FCC that the ISPs are going to have to provide to every customer.

There is one promising take away from the FCC numbers which is that the large ISPs are increasing customer speeds. 2016 promises to be a year when many households will see a boost in speeds. I know Comcast just recently upgraded my speed, with no announcement, and my 50 Mbps cable modem now seems to regularly be getting 88 Mbps download. Almost every big cable companies has announced initiatives to increase customer speeds this year, across the board.

But this difference in speed reports does remind us that you always have to dig a little deeper when you see a statistic like average download speeds. There are different ways of measuring speeds and the FCC way of measuring speeds is suspect in several ways. What they published is not an untrue number for the way they measured it – but the number is just not relevant when talking about the US broadband experience.