The FCC recently released the reports from its speed test program for both 2017 and 2018. The reports summarize the results of the FCC’s Measuring Broadband America (MBA) that samples the actual performance of broadband customers by installing measuring devices at their homes. This program began in 2011 and these are the 7^thand 8^threport from the program. These used to be issued as separate reports, but these reports along with a number of other FCC reports are now being released together in one large annual filing. The link to the reports can be found here. The 2017 report begins at page 349 of the document and the 2018 report on page 463.

These tests are given to volunteer households of large ISPs only – those that cover 80% of all broadband customers in the country. This list of ISPs contains the big cable companies, big telcos, satellite broadband providers.

The primary conclusion of both reports is that “For most of the major broadband providers that were tested, measured speeds were 100% of advertised speeds or better between the peak hours (1 p.m. to 11 p.m. local time).

Frankly, that conclusion is impossible for me to believe and indicates that there is something in this testing program that is different than the reported experience by many customers in the real world. Consider all of the following:

• It’s possible that the FCC is somehow doctoring the speed data, or at least not reporting all of the data they gather. Ars technica reports that SamKnows, the firm doing the measuring for these tests said they have been collecting data from between 6,000 and 10,000 homes during the time of these tests. But the reports are basing data on about 4,500 locations. This is an FCC that seems adverse to reporting things it doesn’t like, so there is certainly a chance that there is selective editing of the data used to create the report.

• It’s clear that the reported users in these test results are not from rural America. My experience over the last decade is that virtually nobody in rural America is receiving the advertised broadband speeds. It’s virtually impossible for a rural DSL customer to getthe advertised speeds since they live far away from the core DSLAM modems that provide broadband. It’s worth noting that both reports admit that satellite broadband underperforms.

My experience comes from working extensively in rural America across the country. When we do broadband studies we elicit households to take speed tests so we can see actual performance. Admittedly speed tests have issues and are not as accurate as the measuring being done by SamKnows. They are likely connecting directly to the incoming broadband signal at the modem while most households today use WiFi, which affects self-administered speed test results. But in the many rural speed tests we’ve seen households perform, it’s rare to see a rural customer getting the speed they are paying for, and often they get just a tiny fraction of that speed, with results sometimes barely better than dial-up.

In general I test speed tests because we also do broadband studies for larger towns, and sometimes the speed tests show good performance by the ISP. For example, we studied a City recently with about 40,000 homes and for the most part Comcast was delivering speeds that often exceeded the advertised speeds. This makes me believe that the major speed test sites, while not perfect, are not terrible and can be believed to represent a whole community.

However, I’ve also studied larger communities where a major ISP underperforms across the board. I’ve rarely seen DSL meet advertised speeds for the majority of customers in a community. And I’ve studied communities where the cable company was slower than advertised for everybody.

The FCC results are also hard to believe because we know from the press that there are whole communities where a major ISP underperforms. As an example is a long-running battle in upstate New York where Charter has been delivering speeds at a fraction of the advertised speeds – the performance was so poor that the State is trying to kick Charter out of the state.

I have similar anecdotal evidence at my own house. My ISP is also Charter. They currently tell me that my speed ought to be 200 Mbps but I’m getting about 135 Mbps. Before the recent upgrade I was also getting less than what they said. I’m not unhappy with the 135 Mbps, but if my house was part of the FCC test it would show a connection getting only 2/3 of the advertised speeds.

The ars technica article I cited above is worth reading because they dig deeper into the data. I must admit that I got stopped at the first page of each report where they said that the large ISPs are mostly delivering the speeds they are advertising, because I know for much of the country that is not true. That makes me suspect that there is doctoring of data somehow. Perhaps the results mostly come from larger communities where the speeds are okay. Maybe the FCC is doctoring the data and excluding poor test results. Perhaps the ISPs know which homes are being measured and give them special attention. I don’t know the details of how the report was generated, but I have too much experience in the real world to accept the conclusions that big ISPs deliver the speeds they advertised.