Monthly Archives: February 2021

Broadband Usage Explodes in Fourth Quarter of 2020

OpenVault just released its Broadband Insights Report for the 4th quarter of 2020. The report shows continued staggeringly fast growth of broadband usage in the country. The growth has been so fast that OpenVault applies the word ‘exploded’ to describe the giant increase in customer usage.

The monthly average household usage leaped to 482.6 gigabytes – which is a combination of both the upload and download usage in the average home across the whole country. It’s impossible to put that number into perspective without comparing it to the average household usage in recent quarters, as follows:

1st quarter 2018           215 Gigabytes

1st quarter 2019           274 Gigabytes

4th quarter 2019           344 Gigabytes

1st quarter 2020           403 Gigabytes

2nd quarter 2020          380 Gigabytes

3rd quarter 2020           384 Gigabytes

4th quarter 2020           483 Gigabytes

The fourth-quarter number is up 26% from the third quarter this year and up 40% from the fourth quarter of last year – the last time that usage wasn’t impacted by the pandemic. These are growth rates that network engineers can’t really grasp – how does one design a network to be ready for a 40% growth in traffic volume in a single year?

OpenVault also saw that there wasn’t a big difference between homes with data caps at 472.3 gigabytes per month and homes with unlimited usage at 496.6 gigabytes per month. You might expect homes with data caps to try to curtail usage, and the numbers show they only do so moderately.

Median usage has also grown significantly over the last year – this is the level at which 50% of homes use less broadband and 50% use more. Median broadband usage per home was 190.7 gigabytes at the end of 2019 and grew to 289 gigabytes by the end of 2020. The median usage in the US is going to remain a lot lower than the average usage due to homes with exceedingly slow rural broadband that are incapable of using a lot of data.

The number of households that are heavy users of broadband continues to explode upward. At the end of the fourth quarter, 14.1% of homes consumed more than 1 terabyte of data per month (1,000 gigabytes), up from 7.3% of homes just a year earlier. OpenVault has started also counting what they call extreme users or homes using more than 2 terabytes per month – 2.2% of all homes were extreme users at the end of 2020, up from 1% only a year earlier. There is no mystery about why Charter wants to start billing for data caps – there is a lot of money to be made from homes that use a lot of data. Looking back four and five years, and the cable companies claimed that homes that used a terabyte of data were anomalies and should pay extra. That argument loses steam when 1 out of 7 homes is routinely using more than a terabyte of data.

Of all of the statistics gathered by OpenVault, the fastest-growing category is the number of homes subscribed to a gigabit-speed service. At the end of the fourth quarter that has grown to 8.5% of all households, triple the 2.8% of homes that were buying gigabit products at the end of 2019.

Finally, the statistic that best defines broadband during the pandemic is the growth of average household upload usage each month. At the end of 2019, the average US homes uploaded 19 gigabytes of data per month. By the end of 2020 that had grown to 31 gigabytes. It’s likely that the demand for upload has grown even more than this, but huge numbers of homes report being restricted on the upload path. There are stories everywhere of homes still unable to connect to school or work servers, or home broadband connections that won’t support multiple people using the upload path at the same time.

Broadband is seasonal it’s likely that usage will drop some by the end of the second quarter in 2020 when school is out for most students. It’s going to be really interesting to see how usage changes by the end of 2021 as we hopefully leave the pandemic behind us.

Solving the Urban Digital Divide

TR007241

We suddenly have a new way to tackle the urban digital divide. The Consolidated Appropriations Act of 2021 allocated $3.2 billion to bring broadband to homes that need it during the pandemic. Further, a recent editorial in the Washington Post suggests that we expand this program and make it permanent. I have conflicted feelings about the plan.

It’s wonderful for the federal government to finally recognize that there is a huge digital divide in urban America. The FCC has rightfully concentrated its effort on rural homes that have no broadband option, but this means there is barely any official acknowledgement that an urban digital divide even exists. Finally recognizing the issue is a great step towards starting the process of finding permanent solutions to the problem.

This feels like the absolute right solution for 2021 while we are still under the pandemic. The proposed solution would pay $50 for a home broadband connection for homes that need it. There are still millions of homes with students struggling with schooling from home and millions more were sent home to work. Broadband is also essential for the millions who have lost jobs during the pandemic. $3.2 billion will pay for broadband to 10 million homes for six months or 20 million homes for three months. I wonder if anybody even knows how many homes need broadband right now – I’m guessing it’s far more than 10 million.

But I can’t help feeling like this is a big giveaway to the biggest ISPs which are likely to claim most of this money. It’s impossible for Washington DC outsiders to understand where ideas like this came from. I’d feel better about this idea if it came from digital divide advocates than if this idea was hatched by big ISP lobbyists who see a way to snag billion in federal money.

That’s a legitimate concern because this plan largely ignores the other two issues faced by homes without broadband. Many such homes don’t have a computer and many people don’t know how to use computers. Urban digital divide proponents will tell you that we need to solve all three issues before a home can meaningfully take advantage of getting broadband.

I also hope that any money used this way goes towards real broadband. It will be maddening if Comcast can charge the federal $50 monthly for what it already provides with its $9.95 Internet Essentials program. Comcast has crowed loudly about the good done by this program, but Comcast recently announced it is upgrading the program for 25/3 to 50/5 Mbps – a speed that is still inadequate for multiple people to use at the same time. I thought that was a decent program before the pandemic, but it no longer delivers what a home needs to stay connected during the pandemic.

I also hope that none of this money goes to AT&T DSL since the company stopped offering DSL to the public in October. None of this money should go to DSL in general since most DSL connections have upload speeds even slower than the Comcast Essentials.

I have reservations against making this a permanent solution. I might be wrong about this and perhaps I can be convinced that this is the best solution for the urban digital divide. I can’t help but wonder about what happens the day this subsidy ends and millions of homes lose their broadband connection. Perhaps the big ISPs already envision that day and know the temptation will be for the government to continue funding the program endlessly. My other big concern is that if this is made permanent that it will always be an annual target for politicians to eliminate.

My gut tells me that billions could be better spent by building a permanent broadband solution for poor urban neighborhoods. Before we make $50 payments into a permanent solution, as has been proposed by the Washington Post, I’d like to at least explore using federal funds to create urban broadband cooperatives or non-profit ISPs that could bring permanent broadband to neighborhoods that most need it. We already have a great historical example of how this can work. Rural America basically got electricity because the federal government loaned money to cooperatives to build electric networks. My guess is that we’d see the same thing happen in urban America if the government offered the oans. It makes a lot more sense to loan the money needed to let neighborhoods help themselves than to create a permanent subsidy.

Keeping Up With Broadband Upgrades

It’s been clear for many years that the US has an infrastructure problem due to having not put money into roads, bridges, dams, and other key public infrastructure. Those of us in the broadband industry also understand we have a huge infrastructure deficit when we look around at the state of broadband that’s limping along on sixty-year-old copper networks. An article was published in the Washington Post in December that points out that the bandwidth deficits affect a lot more than home broadband.

The article points out that the National Weather Service admits it has a bandwidth problem and is seeking to throttle back the public’s ability to gain access to real-time weather data. The NWS is the collector of the huge amount of weather data that is reported from numerous weather stations, sensors, and satellites.

Numerous other businesses collect data from the NWS and disseminate weather information to the public. Airline pilots rely on real-time data in plotting a course around bad weather. We use real-time data to know when to pause sporting events from little league games to pro sports. People rely on real-time weather information during bad weather – it’s vital for folks living in or near the path of a tornado to see where the storm is heading in real-time. And we all have built real-time weather forecasting into our daily lives. The weather apps we use don’t just tell us that it’s going to rain later today, they tell us it’s likely to start raining in 14 minutes, allowing us to plan appropriately. Society is clearly harmed in we no longer have access to real-time weather information.

It turns out the National Weather Service problems are solvable by updating computers and broadband connections. It’s a shame that the agency didn’t figure out a way to solve the problem before it was time to start rationing weather data, but the same statement can be made about all of our infrastructure problems.

For example, the telephone copper networks in Germany are roughly the same age as the ones in this country, and perhaps even a little older on average. Where the big telcos in the US simply stopped maintaining and upgrading copper networks, German telcos have continued with maintenance programs and continued to upgrade the electronics. Where DSL speeds in the US are generally under 30 Mbps (and often far slower) in towns and are often at speeds far under 10 Mbps in rural areas, the German DSL networks deliver much faster broadband. It’s not unusual to find DSL in cities at 100 Mbps and rural broadband at speeds up to 40 Mbps in Germany – all due to the fact that there was a push to get the best possible speeds out of the networks.

The Germans know that copper networks are near to the end of life and are in the process of upgrading to fiber, but rather than suffer from slow broadband speeds in the meantime the German telcos are milking the best speeds they can out of the old copper networks.

I am asked at least several times each month why upgrading to fiber is important. Underlying this question is the fear that the speeds on fiber will be obsolete in a few years in the same manner that has happened to DSL. After all, DSL has only been around for twenty years and went from state-of-the-art technology to become obsolete in a relatively short period of time. Communities don’t want to invest in fiber technology if it has the same path forward.

My answer to this question brings both good news and bad news. The good news is that fiber is almost infinitely upgradeable. I have no doubt that scientists will keep finding ways to milk more speeds out of laser technology. A new fiber network today can be built to deliver a symmetrical gigabit to everybody, and the sky is the limit on fast the network can go – the glass is capable of multi-terabit speeds.

But my answer also comes with a warning. Anybody that operates a fiber network has to be prepared to upgrade the electronics periodically. If somebody builds a fiber network and treats it the same way that big telcos have treated DSL, then that network is going to start having problems in a decade and might be barely limping along in twenty years. This is one of my biggest concerns about the big telcos taking grants to build rural fiber networks – are they going to keep up with the needed maintenance and upgrades? Their broadband history would indicate otherwise. Fiber is not automatically a great technology – it can be a great technology when operated by a great ISP.

5G a Fizzle With Consumers

The cellular companies have made an unprecedented push to get customers interested in 5G. Back in November, I recorded a college football game which enabled me to go back and count the twelve 5G commercials during the game. Advertising during sports events is relatively expensive, so these ads were not purchased at bargain-basement prices. The amount of money being spent to advertise 5G must be gigantic.

It looks like all of that advertising is not having the impact that the cellular companies want. JD Powers conducted a series of large surveys near the end of 2020 and found that the public was less than enamored by 5G, even after all of the advertising.

In good news for the cellular carrier, the advertising has created awareness of 5G and 92% of those surveyed had heard of 5G. However, only 26% believe that 5G is faster than existing 4G cellphone broadband. The response that cellular carriers will find troubling is that only 5% of those surveyed would pay more to get 5G. Only 4% are willing to switch cellular providers to get 5G.

The only companies that are making money on 5G are the cellphone manufacturers. Throughout the fall all of the big cellphone companies put a big push on having 5G in the phone. However, that advertising might not be having the desired impact and I’ve noticed that recent cellphone ads focus on the cameras in the newer phones instead of 5G.

None of this is particularly surprising because the web is full of stories about how 5G speeds are disappointing. Numerous reporters have compared 4G and 5G coverage in the same locations and often found that 5G is slower than 4G.

But there is a more fundamental question that the cellular companies have never addressed with customers. Why do customers need faster cellphone data speeds? The biggest bandwidth functions performed on most cellphones are watching a video or playing games – and 4G data speeds are more than adequate for these needs. Cellphones don’t suffer from having multiple users trying to use the bandwidth at the same time. Unlike with home broadband connection, I can’t recall hearing of people complaining that cellphone data speeds are too slow. People complain about coverage gaps where they can’t get service, but there doesn’t seem to be any groundswell asking for faster cellphone data.

Most people don’t realize that the cellular companies have no choice in the way they are rolling out 5G. The 4G cellular networks are swamped and overloaded, and if the cellular companies didn’t act, they were facing the collapse of the 4G network during busy hours.

The carriers have taken some of the stress off the 4G network by deploying small cell sites. But like with many other things, small cell site deployment slowed down during the pandemic. The carriers have introduced new spectrum bands, and that is what they are currently labeling as 5G. The real point of the 5G advertising is to lure people to buy and use phones that use the new spectrum bands, which reduces the pressure on the traditional cellular spectrum.

Eventually, the carriers will deploy the real 5G, which means using the 5G specifications. That will complete the third leg of cellular improvements. Some of those 5G features will significantly improve cellular networks. For example, a single cell site will be able to eventually handle up to 100,000 connections at the same time. Cellphones will not only try to connect to the nearest cell site but will be able to connect to other cell sites and will even be able to connect to more than one cell site at the same time. The 5G improvements are all aimed at helping urban cellular coverage and won’t make much difference in rural markets.

For now, the only new 5G feature that has been deployed is dynamic spectrum sharing (DSS). This feature lets a carrier mix 4G and 5G customers in the same spectrum bands. This feature is allowing the cellular companies to shuttle customers away from busy spectrum to relieve pressure on the network.

I don’t know that we’ll ever learn the extent to which these various efforts are helping the cellular networks. The cellular companies have been careful for years to not publicly discuss the 4G crisis and they are not now likely going to divulge the details of how they are fixing network problems.

Mediacom Threatens ‘Excessive Uploaders’

Far too often in this industry I read something that makes me roll my eyes. The latest comes from a warning by Mediacom to subscribers about excessive upload usage. The company is calling and writing to customers who use upload bandwidth and warning them that they will be throttled or disconnected if they use too much bandwidth.

This is outrageous during the pandemic. The vast majority of upload traffic is coming from those who work from home or have students working from home. The heaviest users are households with two or more people trying to work simultaneously – something that millions of households are trying to do. Upload bandwidth means connecting to school and work servers and getting on Zoom calls – something that is hard for anybody working from home to cut back on. The next biggest use of upload bandwidth is backing up data on local hard drives into the cloud – something that is universally recommended to anybody working from home.

Mediacom already uses data caps as a way to throttle heavy usage. The data caps on a 60 Mbps broadband connection is one of the stingiest in the industry, with monthly data caps starting as low as 200 gigabytes of data per month. However, the data caps on speeds greater than 100 Mbps are larger – the cap for 300/30 Mbps is 2 terabytes and for a gigabit download product can be as high as 6 terabytes. These data cap plans put a cap on total download and upload traffic – but it appears there is also a secret unpublished cap placed on upload data usage.

Like all cable companies, Mediacom has slow upload speeds which are proving to be inadequate during the pandemic. The advertised upload speed for 100 Mbps download broadband is only 5 Mbps. That’s likely the most popular product being sold by Mediacom, and a 5 Mbps upload path is not adequate to support multiple family members working and schooling at the same time.

During the pandemic, it’s been reported that millions of households nationwide have upgraded to more expensive broadband products in order to get an upload data path that will work. In the case of Mediacom, subscribing to a gigabit broadband product comes with a 50 Mbps upload speed – a speed that ought to be sufficient to support people working from home. But now the company is punishing customers that pay more for the extra speed and then try to use what they’ve purchased.

This is the kind of ISP behavior that cries out for broadband regulation. If the FCC regulated broadband, it could step in and tell Mediacom to stop this anti-customer behavior. This is a company that undoubtedly accepted a lot of upgrade orders from customers wanting faster broadband during the pandemic and then turned around and told those same customers they couldn’t use what the extra fees had purchased.

Mediacom could offer faster upload speeds but has decided not to. I have no doubt that Mediacom is still using the upload technology that came with DOCSIS 3.0 from 2006. That technology deploys upload bandwidth in the band of frequency inside the cable transmission between 5 MHz and 42 MHz. This is a relatively small band of frequency and doesn’t support much bandwidth. It’s also the noisiest frequency inside of a cable network and is subject to a lot of ambient interference.

Since Mediacom is offering gigabit broadband it means it upgraded the download path to DOCSIS 3.1. But the company clearly elected to not upgrade the upload bandwidth speeds. DOCSIS 3.1 allows for using what is labeled as a mid-split option to enable frequency as high as 204 MHz to be used for better upload speeds.

I’m sure the company elected to not undertake the upload upgrade to save money. Like most of the cable industry, Mediacom didn’t think customers needed more upload speed. But now, during the pandemic, the public clearly needs faster upload speeds and is even willing to pay more to get it. Mediacom could have spent 2020 making the upgrade to the mid-split to meet increased customer demand. Instead, they are threatening and throttling customers who actually use what they have purchased. For those that are keeping tabs – this is what monopoly abuse looks like. I hope the new FCC finds a way to stop this nonsense.

The Consolidated Appropriations Act of 2021

The headlines claimed that the most recent $900 billion COVID-19 relief package includes nearly $7 billion in funding for broadband. That is a lot of money and is one of the biggest awards ever aimed at broadband. Following in my first impressions about the usefulness of the funding.

Replace Chinese Electronics – $1.98 billion. Over a quarter of the funding is being used to ‘rip and replace’ electronics from Huawei and other Chinese manufacturers. This money doesn’t bring any broadband benefit, so it’s disingenuous to call this broadband funding. I’ve wondered about this concept from day one. Network engineers tell me that it would not be hard to firewall this equipment and eliminate any risks that such equipment is spying on us – so this feels more like a political gimmick than anything that’s really needed.

Subsidize Low-Income Connections – $3.2 billion. This provides $50 per month to low-income homes to get a broadband connection. I’m really torn on this one. First, this seems like a big victory in that it’s the first time that the federal government has recognized that there is an urban digital divide. It still looks like the pandemic will be with us for much of this year and this funding can get broadband to homes that really need it.

But at the same time, this also feels like a giveaway to the big ISPs. For instance, Comcast already has a low-income program that provides inferior broadband for $9.95 per month. Does this bill let them bill the federal government $50 for that same program? I also worry that some of this money will go to DSL. AT&T doesn’t deserve a penny of this money after walking away from DSL in October. DSL in general should not be funded since DSL upload speeds are generally set at 1 Mbps or less – which makes the technology worthless for working or schooling from home.

I also worry about what happens when this funding ends. This could fund broadband to 10 million homes for six months or 20 million homes for 3 months and then the program will run out of money. What happens to these homes then? This is a lot of money to spend on a temporary program.

Infrastructure and Adoption in Tribal Areas – $1 billion. This is my favorite part of the funding package because this can lead to building a permanent broadband solution. But this funding has two caveats that could be gotchas. First, the funding can’t be used any place where other federal funding was already assigned. Unfortunately, the FCC has previously funded stupid solutions like Viasat satellite broadband in some tribal areas, and hopefully that can be ignored. This funding also comes with a ticking clock and needs to be spent by the end of 2021. If a broadband project is not already shovel-ready this is going to be a challenge to use.

Digital Inclusion in Minority Communities – $285 million. This money is being earmarked to historic black university and colleges, tribal colleges, and similar institutions. Again, this item seems to be a victory in that the FCC is finally recognizing that broadband has bypassed inner cities and tribal areas. This money could do a lot of good if it goes towards establishing permanent programs. But it also runs the risk of being wasted if the money goes to conducting studies instead of creating programs that will continue to help solve the digital inclusion gap.

FCC Broadband Mapping – $98 million. Everybody has been calling on the FCC to fix the damned FCC broadband maps. But as a consultant, this seems like an extraordinarily large amount of money to get this done. I’m frankly surprised that this couldn’t be done for a tiny fraction of this money – some handful of consultants are going to have a very good year.

Other Awards. Probably the best of the other awards is $300 million for rural infrastructure projects not covered by other specific awards. I would hope the NTIA will use this wisely and award broadband grants to areas that have been misclassified by the FCC maps as having broadband. There is also $250 million to bolster the FCC’s COVID-19 Telehealth grants. Hopefully, that money is mostly used to bring a permanent fiber connection to rural hospitals and health clinics.

Bottom Line. Overall, this funding is disappointing in that it doesn’t live up to the hype that hit the press the day after the funding was announced. But there are good things in here, and hopefully, some of this money helps to fund permanent solutions. But I feel like the money could have been used more wisely than for the big-ticket items like ripping and replacing Chinese electronics or of like overpaying big ISPs to provide low-income broadband. This doesn’t feel like we’re getting $7 billion of value.

Misconceptions of Title II Regulation

It seems likely that a newly formed FCC will tackle the reinstitution of regulation. We can only speculate how they’ll do it, but the easiest path would be to declare broadband to be a “telecommunications service’ under Title II regulations. A cleaner path would for Congress to finally pass an updated telecom act that would address broadband regulation, but nobody is holding their breath waiting for Congress to do the right thing.

This is a topic that clients and peers have recently wanted to talk about. I can already see that the big ISPs have turned up the PR machine to lobby against the reregulation of broadband and we can expect the press to be full of warnings about how regulation will destroy the Internet. I thought it would be useful to discuss just what Title II regulation of the Internet means and doesn’t mean. It’s something we don’t have to speculate about since Title II regulation of broadband was in place before Chairman Pai’s FCC got rid of it.

Most importantly, declaring broadband a regulated service doesn’t regulate the ‘Internet”. It means broadband would be regulated, not Internet content. This means that the FCC would again have to power to regulate ISPs or companies that transmit data on long-haul fiber networks. The FCC is not seeking to regulate Facebook, which would make many people nervous – they want to regulate Comcast, which I imagine a lot of people think is a good idea.

We have a good idea of what the FCC would put back in place once broadband is regulated because we know what disappeared when broadband was deregulated:

  • The FCC will begin again taking complaints from customers about abuses by ISPs. The FCC complaint process provided a mechanism for customers to get redress for bad instances of overbilling or other bad ISP behavior. The fact that a complaint process existed made the big ISPs at least try to not abuse customers.
  • The FCC will also be able to again mediate disputes between carriers. You may remember that there was a bunch of complaints at the FCC when broadband was regulated looking at issues like the big ISPs that were strong-arming companies like Netflix to pay more by throttling their signals as a negotiating tactic. We stopped hearing about such issues when the FCC stopped being the referee in such complaints. It’s not likely that bad carrier behavior ceased, just that we stopped hearing about it.
  • The FCC is almost certainly going to reinstate net neutrality, which basically says that ISPs can’t discriminate between different kinds of web traffic. ISPs don’t really hate this and the CEO of every major ISP went on the record at some point in the past saying they could live with net neutrality.
  • With regulation, the FCC could have told ISPs how to react to the pandemic. Numerous other federal agencies were able to impose emergency rules to protect the public and the economy, but the FCC was powerless to do anything.

There are some other things the FCC could do with reinstated Title II regulation:

  • The FCC could regulate broadband prices but is extremely unlikely to do so. Doing this would finally awaken Congress via industry lobbyists. However, the FCC could use the threat of rate regulation to get ISPs to pare back on the worst practices like hidden fees or exorbitant fees for going over data caps. I am positive that all of the ISP anguish about regulation is a worry that there will be constraints on rates. The big cable companies clearly envision $150 monthly broadband rates in a decade, and they don’t want anybody telling them they can’t do this – at a time when the cable companies are wiping out the vestiges of urban DSL and become real monopolies.
  • I find it likely that the FCC would use regulation to push the consumer privacy issues harder.

The list above is all that we are likely to see from a reintroduction of Title II regulation. But the ISP arguments against regulation aren’t likely to mention any of the above issues – all of which are popular with the public. Instead, the ISP arguments against regulation will make the following spurious claims.

  • ISPs will threaten that increased regulation will mean less investment in infrastructure. This is the funniest thing the big ISPs have ever argued because there has never been a big ISP board meeting where somebody suggested cutting back on investments due to regulation – this never ever happened in any boardroom.
  • ISPs will warn that the FCC will ‘break the Internet’ by imposing heavy regulations. The above list of regulations doesn’t come close to being heavy regulation when you realize that broadband is one of the key industries in the country, operated largely by monopolies. Nobody blinks an eye over regulators keeping an eye on automobiles or airlines.

The big ISPs have been extremely clever and call the current regulatory regime ‘light-touch’ regulation, when in fact it is no-touch regulation – broadband is as close as possible to being fully unregulated. The real issue in play is that ISPs have powerful enough lobbyists to get what they want. But as a country, it’s crazy to not regulate perhaps our most important industry that is controlled by monopolies. The public deserves better than that.

Fixing the RDOF

A bipartisan group of legislators recently sent a letter to the FCC asking for transparency during the due diligence review of the long forms for the RDOF grants. The group was led by Representatives James E. Clyburn (D-SC) and Tim Walberg (R-MI) and Senators John Thune (R-SD) and Amy Klobuchar (D-MN) and signed by 160 members of the House and Senate. Here is the key paragraph of the letter:

As responsible stewards of USF funds, we ask that the FCC redouble its efforts to review the long-form applications that will now be submitted.  We urge the FCC to validate that each provider in fact has the technical, financial, managerial, operational skills, capabilities, and resources to deliver the services that they have pledged for every American they plan to serve regardless of the technology they use. We also strongly encourage the FCC to make as public as possible the status of its review and consider opportunities for public input on the applications. Such transparency and accountability will be essential to ensure the success of this program and to minimize any opportunities for fraud or abuse.

The RDOF grants were awarded near the end of last year in a reverse auction where bidders who were willing to take the least amount of grant subsidy won grant funding to build broadband to some of the least populated parts of the country. This letter was prompted by major concerns about some of the winners of the grant being able to execute on what they had promised in the winning bid. Just a few of the concerns that prompted the letter include:

  • Sketchy Technologies. Some bidders claimed to be able to deploy gigabit fixed wireless technology in rural America – a technology that nobody I know thinks exists.
  • Huge Financial Commitment. Since the grants are paid over 10 years but must be built faster, grant winners not only have to raise the money not provided by the grant but must borrow against future grant payments. There are some big awards made to small companies that look impossible to finance. This issue is magnified by cases where the winning bidders took only a small fraction of the grant funds that were on the table.
  • Build Commitment. The grants require quick construction, with 40% of the build to be completed by end of the third year. It doesn’t seem reasonably feasible for small companies that won big awards to build that quickly. Can satellite companies launch enough satellites to get the needed coverage in 3 years?

The letter asks for a few important things. First, it asks that the review process be made transparent. The process of approving grant recipients is referred to as the long-form process, and they are asking that these forms be made available for public review and comment. In the past, long-form reviews were done behind the scenes, with no public scrutiny of the promises made by grant winners.

The legislators also asks the FCC to conduct due diligence to made sure that grant winners demonstrate the technical, financial, managerial, and operational capacity to do what they’ve promised. This is going to be hard for the FCC since this is a regulatory agency and is not stocked with the experts that can make such complicated and nuanced determinations. I’ve always recommended that big federal grants be made as block grants to the states for this exact reason – the FCC doesn’t really belong in the business of picking commercial winners and losers.

The biggest question that my clients are asking is what happens if the FCC disqualifies a grant winner. Would the grant funding revert to the second-highest bidder? That could be a problem because there are quite a few examples where multiple questionable bidders were going after the same Census blocks. If a grant award is canceled would the funds and the Census blocks get pushed into the second round of RDOF? That seems unfair to the people that live in these areas because it seems likely there is going to be a significant delay until the next round of RDOF. It also seems unfair to bidders who were willing to build rural fiber but got outbid by questionable winners.

The FCC has tied itself a Gordian knot and it’s going to take a long time to sort out this first round of the grant. This grant is such a mess that the agency ought to strongly consider something different for the second round of RDOF – because the reverse auction brought too many problems that could have been avoided.

Taking the Long View

I have to wonder if this year is making the big ISPs rethink their business plans. For years, many big ISPs have foregone making long-term investments in broadband and instead chased the quick return.

A good example is CenturyLink. Before the merger with Level 3, the company had started a program to aggressively replace the copper plant in urban markets with fiber. At the peak, the company built fiber to pass 700,000 homes a year. This was not a surprising direction for a company that had its roots as a rural telco. The company’s executive team understood the huge benefits of building a business that spins off cash year after year. The company clearly envisioned growing to tens of millions of satisfied urban fiber customers.

But that strategy stopped dead cold when the company merged with Level 3. Within a year, the Level 3 team wrestled away control of the company, and I recall a quote by new CEO Jeff Storey that the business was no longer going to chase growth with ‘infrastructure returns’. Like most big ISPs, the new CenturyLink management started chasing the quick hit returns. For the last few years, CenturyLink press releases have highlighted the number of urban buildings that the company has added to the network.

And then the pandemic turned that strategy on its head. Everything I read in the business press says that many companies are not going to be returning in full force to downtown offices. The business real estate market is likely facing a bleak upcoming decade of vacant spaces until the industry right-sizes itself. The likely big downturn in the business real estate market also means a big downturn in the urban broadband market – a strategy for selling to downtown businesses can’t be as effective when the businesses are sending staff permanently home to work.

What is so odd about the strategy of the big ISPs is that they would love broadband customers that spin off big piles of cash. That is the precise business plan that most fiber overbuilders are chasing – make the big investment in fiber-to-the-premise, and then reap the rewards for decades with good cash returns.

Most big ISPs share the same philosophy as CenturyLink, where quarterly earnings and short-term investments are preferred over capital intensive but long-term steady returns. AT&T has built tiny clusters of fiber in markets all over the country instead of replacing all of its copper in its historic market. Verizon has always been the most disciplined ISP and has only built broadband in neighborhoods that meet its cost profiles. This has resulted in a hodgepodge of FiOS fiber scattered throughout the northeast. It’s hard to think the company won’t use this same discipline in building its fiber-to-the-curb wireless product – some blocks will get the new network while adjoining neighborhoods will be bypassed.

The only big ISP that seems determined to expand by grabbing every possible customer is Charter. The company has clearly recognized that it has won the battle against DSL and is becoming a de facto monopoly in most of its markets. But rather than sit back and collect cash, Charter is aggressively planning to grow to the outer suburban and even rural areas surrounding its markets. To some degree, Charter seems to be the only big ISP that is pursuing a strategy of maximizing economy of scale, where efficiency and profitability are maximized by getting as many customers as possible in a geographic region.

It’s interesting to compare AT&T and Charter. AT&T has a few thousand fiber customers in practically every market in the country. Altogether that adds to millions of customers on fiber, but it also means a widely dispersed technician base to service the customers. That’s drastically different than Charter which seems to want to serve every customer within big circles around major markets. My experience in building business plans tells me that the Charter strategy will be far more profitable in the long run.

None of this would matter much except for the fact that a handful of giant ISPs control most of the broadband customers in the country. The combination of Charter, Comcast, AT&T, and Verizon currently serve 72% of all broadband customers in the country. The decisions of these few big ISPs determines the only broadband options available to millions of us.

Why Fiber?

As much as I’ve written about broadband and broadband technology, it struck me that I have never written a concise response to the question, “Why Fiber?”. Somebody asked me the question recently and I immediately knew I had never answered the question. If you’re going to build broadband and have a choice of technologies, why is fiber the best choice?

Future-proofed. This is a word that gets tossed around the broadband industry all of the time, to the point that most people don’t stop to think about what it means. The demand for broadband has been growing at a blistering pace. At the end of the third quarter of 2020, the average US home used 384 gigabytes of data per month. That’s up from 218 gigabytes per household per month just two years earlier. That is a mind-boggling large amount of data and most people have a hard time grasping the implications of fast growth over long periods of time. Even fiber network engineers often underestimate future demand because the growth feels unrealistic.

As a useful exercise, I invite readers to plot out that growth at a 21% pace per year – the rate that broadband has been growing since the early 1980s. The amount of bandwidth that we’re likely to use ten, twenty, and fifty years from now will dwarf today’s usage.

Fiber is the only technology that can handle the broadband demand today and for the next fifty years. You can already buy next-generation PON equipment that can deliver a symmetrical 10 Gbps data stream to a home or business. The next generation already under beta test will deliver a symmetrical 40 Gbps. The next generation after that is likely to be 80 Gbps or 100 Gbps. The only close competitor to fiber is a cable company coaxial network, and the only way to future proof those networks would be to ditch the bandwidth used for TV, which is the majority of the bandwidth on a cable network. Even if cable companies are willing to ditch TV, the copper coaxial networks are already approaching the end of economic life. While there has been talk of gigabit wireless to residents (which I’ll believe when I see it), nobody has ever talked about 10 gigabit wireless.

Fiber Has Solved the Upload Problem. Anybody working or schooling from home now needs fast and reliable upload broadband. Fiber is the only technology that solves the upload needs today. Wireless can be set to have faster uploads but doing so sacrifices download speed. Cable networks will only be able to offer symmetrical broadband with an expensive upgrade with technology that won’t be available for at least three years. The industry consensus is that cable companies will be loathe to upgrade unless forced to by competition.

Is the Easiest to Operate. Fiber networks are the easiest to operate since they transmit light instead of radio waves. Cable company and telco copper networks act like giant antennas that pick up interference. Interference from other wireless providers or from natural phenomenon is the predominant challenge of wireless technologies.

A fiber network means fewer trouble calls, fewer truck rolls, and lower labor costs. It’s far faster to troubleshoot problems in fiber networks. Fiber cables are also surprisingly strong, and fiber is often the only wire still functioning after a hurricane or ice storm.

Lower Life Cycle Costs. Fiber is clearly expensive to build, but the cost characteristics over a fifty-year time frame can make fiber the lowest-cost long-term option. Nobody knows how long fiber will last, but fiber manufactured today is far superior to fiber built a few decades ago. When fiber is installed carefully and treated well it might well last for most of a century. Fiber electronics are likely to have to be upgraded every 10-12 years, but manufacturers are attuned to technology upgrades that allow older customer devices to remain even after an upgrade. When considering replacement costs and ongoing maintenance expenses, fiber might be the lowest-cost technology over long time frames.