Verizon’s Network Performance

Verizon has been posting a weekly report of how COVID-19 has been impacting their network. The weekly blogs are rather short on facts and it’s clear that the intent of this weekly report is to put investors at ease that the company’s networks are coping with the burst of traffic that has come as a result of the pandemic. With that said, the facts that are discussed are interesting.

Verizon lead off the weekly entry for 5/21 saying that voice and text traffic are starting to return to pre-COVID levels. On the most recent Monday Verizon saw 776 million voice calls, down from 860 million calls at the peak of COVID-19. That falls under the category of interesting fact, but heavier telephone call volumes are not the cause of undue stress on the Verizon network. Telephone calls use tiny amounts of broadband – 64 kbps. Thirty telephone calls will fit into the same-size data path as one Netflix stream. Additionally, once voice calls reach a Verizon hub, telephone calls are routed using a separate public switched telephone network PSTN to transport calls across the country. Text messages use much less data than a telephone call and are barely noticed on telco networks.

The bigger news is that some other traffic is staying at elevated levels. Verizon reported for 5/21 that gaming is still up 82% over pre-pandemic levels and VPN connections used to connect to school and work servers are up 72%. The use of collaborative tools like Zoom and Go-to-Meeting are up ten times over pre-COVID levels (1,000%).

One of the more interesting statistics is that network mobility (people driving or walking and switching between cell towers) has increased in recent weeks and that one-third of states now have higher levels of mobility than pre-COVID. At first that’s a little hard to believe until you realize that in pre-COVID time students and employees were largely stationary at the school or office much of the day – any roaming by stay-at-home people is an increase.

Reading back through the weekly statistics shows that most web activities are at higher levels than pre-COVID. Fir example, in the 4/22 report the volumes of downloading, gaming, video usage, VPNs, and overall web traffic were higher than normal, with the only decrease being the volumes used for social media.

What none of these reports talk about is the stress put on the Verizon networks. It’s easy in reading these reports to forget that Verizon wears many hats and operates many networks. They are still a regulated telco in the northeast and still have a lot of telephone customers. That also means they still operate a sizable DSL network. The company, through Verizon FiOS is still the largest fiber-to-the-home provider. The company also owns and extensive enterprise and long-haul fiber network. Verizon also operates one of the largest cellular networks in the world.

When Verizon says all is well, they can’t mean that for each of these networks. The web is full right now of complaints from DSL customers (Verizon’s and other big telcos) complaining how inadequate DSL is for working at home. The Verizon DSL network was already overstressed in evenings and has to be near the point of collapse due to the big increases in VPNs and collaboration connections. Any Verizon DSL customer reading this Verizon blog that says everything is fine is probably spitting fire.

By contrast, Verizon’s FiOS networks are likely handing the pandemic traffic with ease. Verizon FTTH products have offered symmetrical data for years, with the upload data path was lightly utilized. The big uptick in VPN connections and collaboration connections ought to be handled well in that network. Any glitches might come from older FiOS neighborhoods where the backhaul oaths out of neighborhoods are too small.

What Verizon or AT&T haven’t talked about is the different impact on their various networks. For example, what’s the overall change in data usage on their cellular networks compared to other networks? The big telcos have been moot on this kind of detail, because admitting that some of the networks are handing the pandemic well might lead to an admission that other parts of the company are not doing so well. Instead we get the very generic story of how everything is fine with the company and their networks.

These companies probably do not have any obligations to report about their various networks in detail. Verizon DSL customers don’t need company pronouncements to know that their broadband experience has nearly collapsed since the pandemic. FiOS customers are likely happy that their broadband has weathered the storm. One of these days I’ll hopefully have a beer with some Verizon engineer who can tell me what really happened – both good and bad – behind the scenes.

Easing Fiber Construction

Almost every community wants fiber broadband, but I’ve found that there are still a lot of communities that have ordinances or processes in place that add cost and time to somebody trying to build fiber. One of the tasks I always ask cities to undertake is to do an internal review of all of the processes that apply to somebody who wants to build fiber, to identify areas that an ISP will find troublesome. Such a review might look at the following:

Permitting. Most cities have permitting rules to stop companies from digging up the streets at random, and ISPs expect to have to file permits to dig under streets or to get onto city-owned utility poles. However, we’ve run into permitting issues that were a major hindrance to building fiber.

  • One of my clients wanted to hang fiber on city-owned poles and found out that the city required a separate permit for each pole. The paperwork involved with that would have been staggering.
  • We worked in another city where the City wanted a $5,000 non-refundable fee for each new entity wanting to do business in the city. Nobody at the City could recall why the fee was so high and speculated that it was to help deter somebody in the past that they didn’t want working in the city.
  • I’ve seen a number of cities that wanted a full set of engineering drawings for the work to be done and expected no deviance from the plans. Very few ISPs do that level of engineering up front and instead have engineers working in front of construction crews to make the final calls on facility placement as the project is constructed.

Rights-of-Way. Cities and counties own the public rights-of-way on the roads under their control. Most cities want fiber badly enough to provide rights-of-way to somebody that is going to build fiber. But we’ve seen cities that have imposed big fees on getting rights-of-way or who want sizable annual payments for the continued use of the rights-of-way.

I’ve seen fiber overbuilders bypass towns that overvalue the rights-of-way. Many cities are desperate for tax revenues and assume anybody building fiber can afford high up-front fees or an ongoing assessment. These cities fail to realize that most fiber business plans have slim margins and that high fees might be enough to convince an ISP to build somewhere else.

Work Rules. These are rules imposed by a city that require work to be done in a certain way. For example, we’ve seen fiber projects in small towns that required flagmen to always be present even though the residential streets didn’t see more than a few cars in an afternoon. That’s a lot of extra cost added to the construction cost that most builders would view as unnecessary.

We’ve seen some squirrelly rules for work hours. Many cities don’t allow work on Saturdays, but most work crews prefer to work 6-day weeks. We’ve seen work hours condensed on school days that only allow construction during the hours that school is in session, such as 9:00 to 2:00. Anybody who has set up and torn down a boring rig knows that this kind of schedule will cut the daily feet of boring in half.

Timeliness.  It’s not unusual for cities to be slow for tasks that involve City staff. For example, if a City does their own locates for buried utilities it’s vital that they perform locates on a timely basis so as to not idle work crews. In the most extreme case, I’ve seen locate put on hold while the person doing the locates went on a long vacation.

We’ve also seen cities that are slow on inspecting sites after construction. Fiber work crews move out of a neighborhood or out of the town when construction is complete, and cities need to inspect the roads and poles while the crews are still in the market.

In many cases, the work practices in place in a city are not the result of an ordinance but were created over time in reaction to some past behavior of other utilities. In other cases, some of the worst practices are captured in ordinances that likely came about when some utility really annoyed the elected officials in the past. A city shouldn’t roll over and relax all rules for a fiber builder because such changes will be noticed by the other utilities that are going to want the same treatment. But cities need to eliminate rules that add unnecessary cost to bringing fiber.

We always caution ISPs to not assume that construction rules in a given community will be what is normally expected. It’s always a good idea to have a discussion with a city about all of the various rules long before the fiber work crews show up.

Expanding the Universal Service Fund

A bipartisan bill has been introduced in Congress that would expand the size of the FCC’s Universal Service Fund by adding a fee on top of broadband bills. This fund is currently funded by fees added to landline telephone and cellular bills. The USF assessment on Interstate traffic recently increased to 26.5% – which is an extraordinarily high tax.

The bill was introduced by Collin Peterson (D-Minn.) and Don Young (R-Alaska). Also sponsoring the bill are T.J. Cox (D-Cal.), Hal Rogers (R-Ky.), Angie Craig (D-Minn.), Frank Lucas (R-Oklahoma), Luis Correa (D-Cal.) Jeff Van Drew (R-N.J.), Ed Case (D- Hawaii), and Vicente Gonzalez (D-Texas).

I’ve been advocating this for a decade because the Universal Service Fund is the FCC’s only tool to tackle the rural broadband issue. The USF already does a lot of good. The Fund is used to bring affordable gigabit broadband to schools. It’s used to bring affordable broadband to rural health care facilities. And even though the FCC keeps fighting it, the USF is used to hold down broadband bills for low-income households, with the Lifeline program that makes ISPs whole for providing lower prices.

In the past the Fund was used to fund two large-dollar broadband expansion projects – one successful and one a total bust. The successful program was ACAM, which has provided the funding to build rural fiber networks by small telcos. I see people around the industry praising the rural broadband in states like North and South Dakota – and that fiber was largely funded by the ACAM program.

Unfortunately, the USF doesn’t always get used wisely. This was the source of funding for the CAF II program that handed $11 billion to the big telcos to ostensively upgrade rural broadband speeds to 10/1 Mbps. It appears that money was largely frittered away or pocketed by the telcos because it’s still hard to find rural households with DSL speeds of 10/1 Mbps. The entire project basically shoveled billions to the bottom line of the telcos.

The Universal Service Fund is about to be used again in big ways. USF is the source of the $16.4 RDOF grants that will be awarded later this year, with another $4 billion to be awarded next year. Assuming this reverse auction doesn’t go cockeyed by awarding money to satellite providers instead of fiber networks, then this will be the biggest boost to rural broadband ever. I’ve been working with a lot of ISPs planning to use this money to build fiber in rural counties all over the country.

The Universal Service Fund is also the source of the proposed $9 billion 5G Fund with a goal of bringing cellular coverage to everybody in the US. Again, assuming the FCC does this right, this would make it a lot easier to live in rural America. Done poorly, this could instead line the pockets of the giant cellular companies.

What nobody is talking about is that those two programs – the RDOF grants and the 5G Fund will use all of the dry powder in the Universal Service Fund. These programs will both award funding over 10 years, and if we don’t find a new source of funding, there will be no additional big grants coming from the USF for the next decade.

What’s even scarier is that the revenues into the Universal Service Fund are dropping as people continue to drop landline telephones. Without some bolstering, there is no assurance that future FCCs will be able to meet the obligations to the recipients of the RDOF and 5G grants.

The revenue impact of imposing a $1 fee on broadband connections is gigantic. There are currently around 106 million broadband customers in the US. A $1 monthly fee on broadband would add $1.3 billion annually to the USF, or over $13 billion over the next decade. That would allow for another big rural broadband grant program.

The members of Congress sponsoring this bill seem to trust the FCC to disperse grant funding. Honestly, their track record on choosing winning grants is mixed. There are also plenty of policy people who think we should take every step possible to keep broadband affordable and that even a $1 monthly fee helps to push broadband out of the affordability range for homes.

If the Universal Service Fund is not expanded, then the only other source for funding rural broadband is Congress. There is a lot of talk about broadband funding coming out of the various COVID-19 stimulus packages. But if that doesn’t happen, we are likely facing an economy with a lot of problems for the next few years. In that environment, rural broadband funding might get shuttled behind other priorities.

Putting COVID-19 Traffic Growth into Perspective

Nokia Deepfield is another company that works in the background on the web and that analyzes data traffic patters for the big ISPs. Their June 4 report on web traffic reports about the same thing we’re hearing from most large ISPs – that the volume of web traffic suddenly shot up since the onset of the pandemic.

Nokia Deepfield says that the increase in traffic has settled in at about a 25% increase over pre-COVID levels. The most important aspect of the increase has been that almost all of the increases have been during the daytime, including on weekends. Networks have not seen any surge (or decrease) in the evening busy hour traffic.

To people who don’t follow the industry, those increases likely sound astronomical. Any other businesses would find a sudden 25% increase in business to be an extraordinary event. Imagine the impact of a sudden and sustained 25% increase in customer demand at a coffee shop, a bank, or a drug store. A business would have to scramble to increase inventory and staff to keep up with the new demand.

But in the world of ISPs this kind of growth is a lot less astounding. Cisco has been reporting for years that residential web traffic has been growing by 21% annually and business broadband by 24%. The ISP industry just absorbed in a single month the growth that would have normally been expected for all of 2020 – but any ISP worth their salt was already braced for this kind of growth this year.

It’s probably hard for the average person to digest that fact that the ISP industry has been coping with this kind of sustained growth for decades. If an ISP makes an expensive investment to double network capacity they’ll see the newly-created capacity filled within three or four years. ISP network engineers face a never-ending task of staying a step ahead of constant and relentless broadband growth.

It’s also worth noting that the growth due to COVID was less dramatic than the industry press might make you believe. Networks are engineered to satisfy the demands at the busy hour – those times of the day when networks are the busiest. During the rest of the day much of the network sits idle since the data pipes aren’t as full. The business hour for residential neighborhoods has been the evenings when homes watch video. Almost all of the growth from COVID came during the workday as students and employees worked from home. For most ISPs, the busy hour is still the evenings, and so there has been far less than a 25% increase in busy-hour demand. Most network should have been able to absorb this burst in growth.

This is not to say that all networks handled COVID growth well. For example, it’s been clear that the big telcos haven’t been investing money in their DSL networks for many years. Performance in those networks has been degrading every year as broadband usage increases. Customers in neighborhood with any significant number of DSL customers have seen broadband speeds decrease year after year as their the demand for broadband has increased. Anybody who has been working at home on DSL during the pandemic saw the network performance in the daytime nearly die.

The Nokia Deepfield blog introduces a new fact that I’d not heard before. They report that distributed denial of service (DDoS) attacks are up 50% during the COVID crisis. At first blush this seems counterintuitive because a lot of businesses have been shut down during the pandemic. Nokia Deepfield says the increased DDoS traffic comes from gamers. Apparently gamers can pay $30 to launch a custom 5-minute DDoS attack against an opponent. Anybody that has seen their neighborhood broadband become useless for five minutes might have been the unintended victim of such an attack. If we had an FCC that regulated broadband they might be investigating this kind of destructive web practice – but this is something they will leave to somebody else.

The bottom line on traffic growth is that. overall, most networks should have been prepared to absorb the growth in traffic due to the pandemic. Most of the growth happened during non-busy hours, and so, while the networks saw a lot of growth in traffic volumes they didn’t see an equal growth in network stress. The bad news for network engineers is that a lot of the recent growth looks like it will stick around, and the overall volumes of web traffic will probably continue to grow at 20% annually on top of the COVID growth.

What is Light-Touch Regulation?

One thing I’ve noticed recently is that a lot of people are climbing on board the idea of building better broadband to rural America. A lot of people seem to think that the FCC can somehow act to fix a lot of the shortcomings of rural broadband – but in doing so they have missed the entire point of what the FCC calls ‘light-touch’ regulation – because, from a practical perspective, broadband is not regulated at all.

It’s not hard to understand why people would misunderstand the situation, because ‘light-touch regulation’ is one of those euphemisms that governments invent to disguise what they are really doing. Chairman Pai at the FCC never misses an opportunity to talk about his regime of light-touch regulation. I have to wonder if there would be as much support for the light-touch regulation if that phrase was replaced with the simpler and more descriptive phrase ‘deregulated’. I suspect a lot of people would be uncomfortable that one of our largest industries is largely deregulated.

The FCC pulled off this huge change by hiding the deregulation inside of their move to undo net neutrality. The FCC didn’t just reverse the net neutrality rules put into place by the previous FCC, they killed Title II regulation – which is the authority given to the FCC by Congress to regulate ISPs as common carriers. The FCC gave up Title II authority and gave the tiny remaining vestiges of broadband regulation to the Federal Trade Commission – even though the FTC is not a regulatory agency. The FTC doesn’t create or enforce new rules – they are more like corporate police that fine corporations when they’ve abused their customers too egregiously.

What does it mean to give up Title II authority? The FCC can no longer judge, or even track broadband prices. ISPs are free to raise rates to any level they want and make any profits they want. There hasn’t been an FCC since the dawn of the broadband industry that has invoked price regulation – but the fact that they could always acted as a brake on bad ISP behavior.

The FCC can no longer intervene in disputes between ISPs or with their biggest customers. Under Title II regulation, the FCC could decide if an ISP was fairly dealing with Netflix or some other large user of broadband. When Title II regulation was killed, the FCC stopped acting as the arbiter in industry disputes – ISPs are free to act in any way they want.

The FCC can’t even intervene when ISPs abuse customers. I recently wrote about the FCC complaint process. Before the end of Title II regulation, ISPs would try to resolve issues raised during the complaint process. The FCC had the authority to make ISPs treat customers fairly if they decided to exercise it – and it was the threat of the FCC creating new rules that made ISPs willing to curb some of their worst behavior. But now the FCC is nothing more than a gatekeeper – they lamely pass on consumer complaints to ISPs, which the ISPs largely toss into the wastebasket since the FCC no longer has any regulatory teeth.

It’s not completely fair to say that ISPs are 100% deregulated because there are a few areas of regulation that were not created under Title II authority that are still in place. For example, the Patriot Act created the requirement that ISPs have to allow federal law enforcement to be able to ‘wiretap’ broadband connections in the same way they used to wiretap telephone connections. The FCC can’t shed that responsibility and so they still enforce the CALEA rules that require ISPs to respond to subpoenas.

There are also various types of privacy and billing rules that were the result of other acts by Congress, and the FCC still oversees these remaining vestiges of regulation. As an example, the whole recent controversy over removing Section 230 protections for online companies like Twitter also applies to ISPs and is enforced by the FCC.

But the core basis for FCC regulation of ISPs was due to the fact that ISPs are common carriers, similar to telephone companies or cellular carriers. ISPs provide a two-way communications path with customers, and it is that basic function that justified regulating them under Title II regulations.

The FCC undertook one of the most bizarre steps in regulatory history when their voluntarily neutered themselves as broadband regulators. They no longer have the authority to force the big telcos to provide better rural broadband. They no longer have the authority to stop an ISP from raising rates to the point of unaffordability. They no longer even have the power to stop an ISP from billing customers for non-existent products. This is all easy to remember if you replace the term light-touch regulation with unregulated.

Beware of Deceptive Surveys

I was sent a copy of the results of a survey about 5G from Sykes, a market research company. In glancing through the survey, it looked like they were asking great questions. For example, the survey showed that over 85% of people have heard of 5G, meaning the cellular companies have done a good job creating brand awareness with the 5G name.

The second question is another good one. It asked if people currently pay for 5G service. 20% said they do. I think I know as much about 5G as anybody and I don’t know how to answer that question. I know my AT&T phone has been telling me for a year that I have 5G, when I know I only have 4G. Am I paying for 5G? I know I’m not paying anything extra for it.

The survey asked what people think of when they hear ‘5G’. 50% said they thought this means a faster and more reliable cellular data connection. A surprising 38% said they thought 5G means a faster and more reliable connection for anything that requires the Internet. That answer surprised me, but most 5G advertising is so generic I guess it’s not hard to see how people might think that 5G is a landline alternative.

The survey asked about the most significant benefits of 5G. 57% said faster speeds. 11% said lower latency, which is a bit surprising since I would bet you can’t find anybody who can point to an example of poor cell phone latency. 19% said the most important aspect is the capacity to use multiple devices – another allusion to 5G being a landline replacement.

An interesting question asked about people’s concerns about 5G. 36% said they were worried about exposure to 5G frequency, the increased presence of 5G transmitters in the environment, and the environmental effects if 5G infrastructure. Interestingly, 17% said they were concerned about the complexity and cost of the 5G infrastructure. I have a really hard time thinking that many people are really worried about what 5G is costing AT&T and Verizon – but that’s the answer they picked out of a list of choices.

Unfortunately, at this point the survey went sideways. Question 6 was: A 5G connection is more reliable and reportedly 100 times faster than 4G. If those claims are true, what impact do you expect 5G to have on your daily life in the next 2 years? This is a classic push poll question. A push poll question plants an untrue fact into a survey and then asks people to react to it. There is nobody in the industry who thinks that 5G is going be this fast for most people within 2 years, and perhaps not even within 10 years, or possibly ever. The cellular companies might never invest in the fiber needed to put a small cell site every 1,000 feet in city neighborhoods, suburbs, or anywhere rural.

A more honest question would have been: The cellular carriers have introduced millimeter wave spectrum in small sections of big city downtowns. This technology is as much as 50 times faster than 4G cellular. It requires a user to buy an expensive new phone and it only works outdoors within perhaps 500 feet of a cell site. Do you think you would pay extra for a phone and a monthly fee to use this technology if it comes to the neighborhood where you live or work?

The problem with a push poll question is that it pollutes every question that follows. People taking the survey were influenced with the idea of 100 times faster speeds within 2 years when they answer any additional questions about 5G. For example, the tenth question in the survey asks: Would you consider switching your Internet provider and/or mobile phone carrier this year in order to have 5G? 23% said yes and 46% said maybe. But they are answering this question with the influence of question 6 about 5G being 100 times faster. I would change my Internet provider for 100 times faster speed – but I know that is not an option that is coming from 5G. I don’t expect any 5G connection in my neighborhood to match my cable modem speed within the next decade – and even when they do, they’ll match it outdoors and not through the walls of my home. This is another ludicrous question.

They survey snuck in another push poll question when it asked: If you were guaranteed to never experience any speed or connectivity issues again with your home connected devices, would you pay more for 5G service this year? The killer wording in this question is the ‘this year’ part. I can’t wait for a cellular company rep to try to sell me on the pitch that cellular coverage this year can handle my computers, smart TV, tablets, and the other twenty connected devices I have in my home.

My experience of cellular coverage in the last month is that the networks are having a hard time keeping up with the demands created by the COVID-19 crisis. I’ve been on numerous conference calls when one of the parties had to drop and reinitialize the call due to a poor-quality cellular connection. We are a still a number of years away from seeing the technology improvements that will make 5G better than 4G. The cellular carriers are in no position to meet any of the expectations implied by this survey.

This survey results don’t disclose who sponsored the survey. Market research firms don’t generally conduct surveys without a paying client. This likely came out of an overzealous marketing department at one of the cellular carriers, or perhaps one of the 5G vendor. Whoever wrote this survey knew the answers would be bogus, but they obviously have an agenda to use the results to influence the general public, or perhaps politicians with the result. I’d hate to think that anybody thinks this poll represents real market research.

Look Out for those Grant Gotchas

One of the most disappointing things in the broadband world is when broadband grants have ‘gotchas’ that make it hard to use the money as intended. North Carolina has named its state broadband grant program the “GREAT” grants. That’s a pretty bold name, and I’m sure that the homes and businesses in the states that are getting broadband as a result of these grants think it’s great. But like other grant programs around the country, there are some gotchas in the grants.

Even though I live in North Carolina, I hadn’t looked into the details of the program until I got a call from a part-time professor at NC State. He was struggling while working at home due to COVID-19 and trying to hold remote college classes from a slow home broadband connection. He and his neighbors were upset because the State had awarded a grant in 2019 to bring broadband to his area of Caswell County, and yet nothing has happened. The rumor was that there was something amiss about the grant, but nobody locally knew why they weren’t getting broadband.

It didn’t take long to find that Open Broadband, a wireless ISP, had won a $1.54 million Great grant to bring broadband to 1,194 homes in the County – roughly one-fourth of the rural households in the county that the FCC says don’t have 25/3 broadband in the 2019 Broadband Deployment Report. This state grant was to have been matched with another $1.54 million from the ISP.

Open Broadband told me that the state had yanked the grant due to a change in the grant rules that was introduced after the grant award – a rule that had not been disclosed in the original grant rules. The state administrator of the grant, the Broadband Infrastructure Office, notified Open Broadband after the grant had been awarded that the ISP had to meet one of the following financial tests to receive the funds:

  • Letters from the applicant’s Board of Director or investors guaranteeing the total project cost.
  • A letter or actual statements from a bank or other financial institution verifying total available cash or lines of credit meet or exceed the anticipated total project costs.
  • Letters from a verifiable third-party entity guaranteeing the total project cost.

In case those rules didn’t sink in, the state is requiring an ISP to guarantee 100% of the cost of the project, including the portion of grant being supplied by the State. In this case, the state wants Open Broadband to guarantee over $3 million, even though they are only receiving $1.5 million of grant funding. Most grants that seek proof of financial viability would ask an ISP to guarantee their out of pocket costs – in this case that’s significantly less than $1.5 million since customer installation fees will cover a large portion of the ISP’s grant matching.

It’s ludicrous that the state wants an ISP to guarantee the portions of the grant paid by the State – is North Carolina worried that its own grant money is no good? The state wants Open Broadband to have $3.08 million in free cash or else a guarantee of that amount from a bank or a private investor. Whoever came up with this after-the-fact rule doesn’t understand ISPs or the banking industry.

ISPs don’t sit on much free cash – even fairly large ISPs. We are living at a time when every ISP I know is expanding – something that governments at all levels should support because government constantly says that the private sector should solve the broadband problems in rural America. Few ISPs are sitting on the free cash needed to make this guarantee.

The State will alternatively accept a guarantee from an owner or investor. This request is massively out-of-line with the nature of the project. A personal guarantee means putting your home and retirement savings on the hook. Many small and medium ISPs are already partially financed by personal loan guarantees by owners, and they can’t pledge this twice – even the rare owner that has over $3 million in net personal wealth.

The idea of an ISP having $3 million in an unused line of credit is even more absurd. Only a really large ISP would have a $3 million line of credit. A business line of credit is a loan that hasn’t yet been drawn. Banks are not like credit card companies and they don’t set big credit limits without an expectation that a borrower will use the money quickly.

The only other way to meet this requirement from a bank is with a bank letter of credit. A letter of credit is a formal negotiable instrument – a promissory note like a check. A letter of credit is a promise that a bank will honor the obligation of the buyer of letter of credit should that buyer fail to meet a specific obligation. Banks consider a letter of credit to be the equivalent of a loan. The banks must set aside the amount of pledged money in case they are required to disburse the funds. Most letters of credit are only active for a short, defined time. A letter of credit for a 2-year grant would be unusual and expensive – the ISP would likely to have to pay full interest expense as if this was a loan. The bottom line is that banks don’t issue a letter of credit for this kind of purpose.

I never heard of the concept of guaranteeing grants in this manner until last year. Out of the blue, the FCC suggested that winners of the $16.4 RDOF grants should be required to guarantee matching funds by either holding the cash or supplying a letter of credit. There was such an immediate outroar from the industry that this idea was killed in a matter of weeks. The industry conjectured that the idea came from the big ISPs, which have the FCC’s ear. The big ISPs know this requirement would stop most ISPs from applying for an RDOF grant – which would allow the incumbents to keep milking money out of rural properties with lousy and overpriced broadband. This may not be the reason for the North Carolina requirement, but the timing is suspicious.

This is not the only gotcha in the grant. The state also only reimburses funds that have been spent by a grant awardees once per quarter. Every other grant program I know of reimburses ISP expenses monthly. It’s a major financial penalty to make ISPs wait for months to get expenses reimbursed.

I was surprised to see major gotchas in North Carolina grants. This comes from a state that brags about its history of being first in broadband. For example, North Carolina says it was the first to have brought fast broadband to every school. I have no idea where the State got the idea for the grant guarantees, but it’s an absurd requirement that will do little more than discourage ISPs from investing in the state. ISPS can easily move across the border to nearby Virginia or Tennessee where the grant process is friendlier. North Carolina needs to kill this absurd rule. Those homes in Caswell County deserve the broadband they were promised, and it’s still not too late to make this grant work.

Frontier’s Lack of Fiber

The primary reason that Frontier cites for going into bankruptcy is the lack of fiber. They are finally acknowledging that customers are bailing on them due to the poor broadband speeds on their copper networks. This is being presented as if this is a sudden revelation – as if the company woke up one day and realized that it’s selling services that nobody wants to buy. I must admit this gave me a chuckle and there are some giant flaws with this argument.

Rural customers don’t hate DSL – they hate DSL that doesn’t work. If Frontier had implemented the CAF II upgrades as had been promised, then rural customers would all be using the 10/1 Mbps or faster rural DSL that would have been created as a result of those upgrades. Instead, customers have gotten disgusted by overpriced DSL that is so slow that they can’t stream video or connect to a school or work server. We’ve been doing speed tests all over the country and it’s rare to find rural DSL in many markets that delivers even 5 Mbps download – much of it is far slower than that, some barely faster than dial-up. If Frontier had provided 10/1 Mbps DSL to millions of homes, those households would gratefully be buying that broadband during the COVID-19 crisis.

Frontier blames its woes on lack of fiber with no mention of their reputation for unconscionably bad customer service. I’ve talked to customers who talk about routine network outages that lasts for many days. Customers complain about losing broadband and having to wait weeks to get it repaired – or worse, are told that the electronics needed to replace a bad DSL modem are out of stock. This is a company that has trimmed, then trimmed again its maintenance staff to the bone. Talk to any rural Frontier technician and they’ll tell you that they don’t have the time or resources available to address routine customer problems.

Frontier complains about lack of fiber, but as recently as 2015 they purchased another huge pile or dilapidated Verizon copper networks as part of a $10.5 billion acquisition. While that acquisition came with some FiOS fiber networks, the company also doubled down on buying non-functional copper networks. The speculation in the industry was that Frontier continued to buy lousy properties because it created opportunities for huge management bonuses – the company never had any plans to make the purchased copper networks any better.

And that’s the real issue with Frontier’s claim – they have no fiber because they’ve made almost no effort to migrate to fiber. The company burned all of its cash on trying to service the debt for overpriced acquisitions rather than rolling cash back into its networks.

It’s interesting to compare Frontier to the many smaller independent telephone companies. The FCC brags about places like the Dakotas that have a huge amount of rural fiber to homes. But that rural fiber didn’t happen all at once. It happened over decades. Most rural telcos went through two rounds of investment where they invested to improve rural DSL. In doing so they built fiber to go deeper into the rural areas, the first build brought fiber within maybe ten miles of homes, the second got fiber to within 3 miles of most homes. When the rural telcos decided to take fiber the rest of the way, it was reasonably achievable because they already had fiber deep into rural neighborhoods.

Frontier has done very little of that kind of incremental improvements over the years. They found it more enticing to keep borrowing to buy new rural properties rather than roll cash back into the existing networks. It doesn’t even look like they did all of that much new fiber as part of the CAF II upgrades. I’m sure Frontier would refute that statement and say they are fully compliant with CAF II, but if they had built fiber deep into the network then rural DSL would have gotten better – and for the most part, it hasn’t.

I can’t how the bankruptcy will benefit frontier’s customers. The company will likely get to walk away from a lot of the debt that was provided for the last few acquisitions – and it’s hard to feel bad for lenders who thought it was a good idea in 2015 to lend to buy copper networks. But bankruptcy won’t fix any of the fundamental problems with the Frontier networks. Customers are going to continue to bail on inferior and nonfunctional broadband products. The upcoming RDOF auction is going to give a lot of money to ISPs that are going to overbuild Frontier copper with something better (even though Frontier made a last-minute filing at the FCC to block grant funding by claiming they had magically upgraded 16,000 rural census blocks).

Is Frontier going to somehow start investing in rural fiber? My best guess is that they won’t even after bankruptcy. If they can raise any money for new capital spending they’ll likely try to salvage some of the county seats and other markets where there is a mass of customers. However, in many of those markets they’ve already lost the battle to the cable companies.

Frontier is right in that they are failing from lack of fiber. But that statement doesn’t tell the full story. They are failing because the company decided decades ago to not invest capital into their own networks – and now they are paying the price.

Say No to Data Caps

Last week I had a blog that asked why the FCC is seemingly supporting data caps by allowing caps on broadband built with federal grant money. The FCC has established grants that place premium value on fast broadband speeds and low latency but that ignores one of the most important aspects of broadband today – usability.

A broadband connection that doesn’t let homes partake in the same online world as the rest of America is inferior broadband – and there is no better example of an unusable data plan than one a low data cap. The FCC’s RDOF rules support monthly data caps of 250 gigabytes for plans offering 25/3 or 50/5 Mbps. The FCC is clearly saying to rural America – we’ll give grant money to ISPs to bring you better broadband, but we’re going to let the ISPs cripple that broadband so that they can bill you an extra $50 or $100 per month if you want to use that broadband like everybody else in the country.

Recall that ISPs that win the RDOF grants have six years to build the new networks. How will a 250 GB data caps look by the time these networks are built? OpenVault says that the average home used 274 GB per month in 2018 – already higher than the FCC’s proposed data cap. By the end of 2019 average usage had grown to 344 GB for the average home and exploded to 402 GB by March due to people and students working from home during the pandemic. Trending household usage forward until 2026 would suggest that the average home will be using more than a terabyte of data each month by then. That’s not a big stretch since more than 10% of homes are already using a terabyte or more of data today.

The FCC is not the only one to point a finger at. There are plenty of state broadband programs that have awarded grants to ISPs that have data caps. This has happened because policymakers have not viewed data caps as providing inferior broadband. This is easy to understand since just a few years the vast majority of homes used a lot less broadband than the data caps. You might recall in 2015 when there was big public pushback when Comcast tried to introduce a 300 GB data cap. At that time, Comcast said that only a tiny number of customers used more data than 300 GB per month – but in five short years, the national average data usage is significantly higher than the cap Comcast wanted to impose in 2019.

We need a new policy at the state and federal level that says that ISPs with data caps are not welcome to broadband grant funding. Not only should they not be able to impose data caps on grant-funded networks – an ISP that has routine data caps for others should be prohibited from participating in any grant funding anywhere.

There are still ISPs that say that data caps help to protect the integrity of their network. This argument went out the door when most ISPs stopped billing for data caps during the pandemic – the one time when protecting the network would have been important.

What hasn’t been said enough is that a broadband connection with a data cap is an inferior broadband connection. A home with data caps faces the monthly choice of either curtailing broadband usage or else spending a lot more for broadband. Are we going to fix the rural broadband gap by transitioning rural homes with slow broadband connections to ones with tiny data caps that cost a lot more than everybody else in the country?

It’s always been clear that data caps are mostly about greed. There is no better example of this than the AT&T rural hotspot that has a data cap that allows for as much as an extra $200 in monthly fees to a subscriber. It’s outrageous that the FCC gave grant money last year to Viasat which has tiny monthly data caps. The whole proposed 5G Fund is outrageous if billions of federal money will allow the cellular carriers to sell more rural hotspots with tiny data caps and huge monthly fees.

It’s time for broadband policymakers at all levels to categorically say no to data caps. Shame on the FCC for allowing data caps into the discussion of the RDOF grant. But also shame on Congress for not issuing a new telecom bill to stop all of this nonsense. And shame on any state policymaker or regulator who has allowed state resources to be used to support broadband with data caps. It’s time to say no.

Can LTE Fixed Wireless Solve the Rural Digital Divide?

I’ve been working in rural counties all over the US and have found that a lot of rural homes are using a fixed 4G LTE wireless product for home broadband. All three big cellular companies have a fixed LTE product for the home. The product typically involves installing a small dish outside of a home that receives broadband using 4G LTE broadband from a nearby cell tower.

The big cellular companies have bombarded the press for the last several years touting how 5G cellular might solve rural broadband gaps. The way the wireless companies price and market these products is a precursor for what they will likely do with 5G (assuming 5G even comes to rural places). I haven’t looked into the LTE products in a while and so I researched the LTE broadband products intended as home broadband. I knew these products were expensive, but I had forgotten how stingy these plans are with broadband.

Verizon. Verizon’s hotspot product has four available pricing tiers based upon the monthly data allowance. The 10 GB plan is $60, the 20 GB plan is $90, the 30 GB plan is $120, and the 40 GB plan is $150. The real cost killer is that Verizon bills additional gigabits for $10 each.

Verizon says that broadband speeds average from 5 – 12 Mbps download and 2 – 5 Mbps upload. I recently talked to a customer using this plan who told me that when a customer doesn’t agree to pay the overage charges that Verizon throttles speeds to a crawl once the monthly data limit has been reached.

T-Mobile. T-Mobile has six hotspot pricing plans based upon the monthly data usage. The 2 GB plan is $10. The 6 GB plan is $25, the 10 GB plan is $40, the 14 GB plan is $55, the 18 GB plan is $70, and the 22 GB plan is $85. Each plan offers a $5 discount for customers who authorize autopay. The killer with this plan is that speeds revert to 3G speeds when the cap has been met. The plans also include unlimited texting.

It’s worth noting that T-Mobile will offer a plan that provides 100 GB of monthly data to qualified students for the next 5 years as one of the promises made to merge with Sprint. I haven’t seen the definition of eligible households, but the company estimated 10 million homes would be eligible.

AT&T. AT&T has two plans. In areas where AT&T is the incumbent telephone provider and accepted CAF II funding, the company decided to provide 4G LTE fixed cellular broadband to satisfy their CAF II requirements. The company had accepted $2.56 billion from the FCC’s Universal Service fund with payments spread over six years. That funding covered 1,117,806 rural homes, providing AT&T with a subsidy of $2,296 per home.

That CAF II product is priced at $50 per month and comes with a 250 GB data cap. AT&T says that speeds are at least 10/1 Mbps (since that was the FCC requirement). The killer with this plan is that customers pay $10 for each additional 50 GB block of data, and payments aren’t capped until a customer spends $200 extra.

Outside of the CAF II areas, AT&T has three hotspot plans. That includes 3 GB of data for $25, 10 GB of data for $50, and 18 GB of data for $75. Extra data ranges from $10 for 1 GB with the $25-dollar plan to $10 for 2 extra GBs with the $72 plan.

Do These Products Solve the Rural Digital Divide? I think not – the core hotspot products define the digital divide. Rural customers who have no other options are paying for some of the most expensive broadband in the world. You have to look at distressed third world countries to see similarly high prices per gigabyte. Recall in looking at these prices that these products are for home broadband – not for cellphones.

  • Verizon’s plans range from $3.75 to $6.00 per gigabyte. Additional gigabytes are $10 each.
  • T-Mobile data prices range from $3.86 to $5 per gigabyte. After hitting the data cap, the company throttles customers rather than provide more expensive data.
  • AT&T hotspots are the most expensive and range from $4.16 to $8.22 per gigabyte. Extra gigabytes on AT&T range between $5 and $10 per gigabyte.
  • AT&T’s CAF II product is much more affordable. However, customers can still spend up to $250 per month. Additionally, AT&T charges $100 for installation, which is outrageous considering the company collected $2,296 from the FCC for each of the 1.1 million potential customers. The CAF II money had to have gone almost entirely to AT&T’s bottom line.

What is somewhat ironic is that the cellular companies don’t aggressively advertise the product in rural America, even at these incredibly high prices. I guess that the cellular carriers don’t want to sink any money into rural cell sites or pay for more backhaul, so they are reluctant to sell very much of these products in any one location.