Monthly Archives: June 2024

Regulatory Costs of Fiber Construction

At the federal level there has been an ongoing battle over the level of federal regulations. The Ajit Pai FCC strove to eliminate almost all broadband regulations that were not specifically mandated by Congress. Pai referred to his concept as light touch regulation. The FCC before and after Ajit Pai believe that there needs to be some regulations in place to protect the general public.

However, there are a lot of regulations other than the ones created by or enforced by the FCC. Anybody who builds fiber networks can describe the litany of state and local regulations involved in constructing fiber. Following are the primary kinds of such regulations – and there are others in some places.

  • National and State Codes. Fiber builders must meet various national and state codes related to electricity, safety, and specific fiber specifications.
  • Safety. Work sites must comply with safety standards set by OSHA and States.
  • Permits. Most jurisdictions have a formal permitting process. This is where a contractor will specify the planned construction of the network.
  • Rights-of-way. Federal rules allow fiber to be constructed in any existing public right-of-way. However, many local jurisdictions require a fiber builder to pay fees and obtain a right-of-way agreement before undertaking construction.
  • Easements. Contractors are required to acquire an easement from private landowners, which is permission to construct on private land.
  • Financial Requirements. Some jurisdictions require that an entity that wants to cut into a street to satisfy specific financial requirements. This might mean obtaining a bond or providing a deposit before construction. There might be requirements for contractors to carry specific amounts of insurance and name the government entity as a covered entity under the policy. Some jurisdictions treat a fiber network like other infrastructure and charge property or related taxes on the asset.
  • Business License / Franchise Agreements. Some jurisdictions require anybody that wants to operate a fiber network must obtain a business license. Communities often require a franchise agreement that lists the various construction parameters and details the fees associated with building and owning a fiber network.
  • Deployment Codes. A city might require fiber to be buried in some neighborhoods. It might require handholes instead of pedestals. It might require that huts go through the same permitting process as any other building. There may aesthetics requirements for huts and cabinets such as hiding infrastructure with shrubs.
  • Locating. It’s always mandated that existing underground utilities are located before doing any underground work.
  • Public Notification. Many communities require a contractor to notify the public before construction. This might include a requirement to knock on doors and leave notices.
  • Traffic Control. Many communities require flag people or other ways to manage traffic during times when construction will block traffic lanes.
  • Site Requirements. There are often specific rules about what must be done at any site when a street is excavated. This might mean taking measures to control dirt runoff if it rains. It might mean covering construction holes for safety purposes.
  • Restoration. Most communities expect any entity that excavates in a right-of-way to restore the area as nearly as possible to the conditions before the construction.
  • Inspection. There can be government inspections required at any step of the construction process. Inspectors typically have the ability to shut down a construction site that is not meeting the expected codes and standards.
  • Mapping. Many jurisdictions require drawings or electronic files showing completed construction.
  • Licenses. Operators of heavy machinery may be required to have specific licenses and certifications. Engineers that design a fiber project might be required to be licensed by the state.
  • Environmental Studies. Local, state, or federal rules might require an environmental study when constructing in sensitive areas. The studies might look at the likely impact on endangered species or the impact of construction on sensitive waterways.
  • Cultural Review. A cultural review might be mandated if construction is to be done in areas with burial grounds, archeological sites, or fossil beds.
  • Historic Site Review. There can be a review required if construction is to be done close to a designated historical site.

Will We Ever End Legacy Telephone Networks?

Anybody not involved in the telephone business will probably be surprised to find that the old TDM telephone networks are still very much alive and in place. The old technologies were supposed to be phased out and replaced by digital technologies. The FCC started talking about this before 2010. In 2013, Tom Wheeler, the FCC Chairman at the time, announced an effort to force the needed changes, which was dubbed the IP Transition. The goal of the transition was to upgrade and replace the public switched telephone network that was used by every telco, CLEC, and cable company for exchanging voice traffic.

The FCC made good on the promise and released an order in 2015 that described the process for telcos to retire copper networks, and that also discontinued the requirement that big telcos offer wholesale TDM services. In 2016 the FCC released that full IP Transition Plan that was aimed at replacing the TDM voice infrastructure with an all-IP network.

But somewhere along the line, AT&T and Verizon highjacked the IP Transition. AT&T announced a few trials to eliminate the TDM network, but then the transition process stopped and was never discussed again. Industry insiders speculated that all the big telcos really wanted from the IP Transition was the ability to retire copper telephone networks, and once they won that issue they lost interest in the costly process of replacing or upgrading legacy systems. The FCC didn’t help when it poured billions of dollars into upgrading the old copper networks of the big telcos through the CAF II subsidy program.

The problem is that all of the problems that were identified with old TDM technology in 2013 are still around today. Any telephone company or CLEC that wants to exchange traffic with the big telephone companies must still do so using TDM technology (technology based on T1s). Telephone features like caller ID still rely on the SS7 network, which is a separate network used to exchange data associated with a telephone call.

Small carriers have been begging for years for the big telcos like AT&T, Verizon, and CenturyLink to allow digital SIP connections instead of TDM connections – but the big companies are deaf to these pleas. There are big parts of the national telephone networks that have been upgraded – for example, large cellular companies have digital SIP connections to the big telcos. It’s interesting how something that benefits AT&T and Verizon was upgraded, just not the connections used by everybody else.

It’s getting more costly every year for traditional carriers to keep using the TDM networks. The prices being charged for T1s have steadily climbed since the 2015 order that eliminated mandatory wholesale T1 prices. The cost of ancillary services needed to support voice, like the SS7 network, is climbing even faster. A handful of large companies that provide most of the SS7 services have squeezed smaller providers out of the market. The cost to access to the databases that include things like the name of calling parties needed for Caller ID has skyrocketed.

Meanwhile, smaller telcos have done everything suggested by the original IP Transition order. Most of them have replaced, or are in the process of replacing copper networks with fiber. These companies have modernized everything except for the legacy connections that are still the only option for completing local calls with neighboring carriers.

What is probably most amazing (or maybe not amazing at all) is how the FCC ordered the IP Transition and then just let the biggest telcos walk away from the process with no repercussions. This is partly due to the big telcos that just stopped working on the issue, but also on the Ajit Pai FCC that entered the picture in 2017 with the agenda of not regulating big companies. It’s time for the FCC to pick this back up and finally make this happen.

My consulting firm is still highly attuned to these issues. Anybody having a problem with TDM trunking, or access to SS7 or SS7 databases should contact me. I will be the first to tell you that I never expected to be still talking about these issues in 2024.

Competing Against FWA

At the end of the first quarter of this year, T-Mobile and Verizon together have accumulated 8.6 million customers nationwide on FWA cellular home broadband. This is amazing success for a product that was just launched in 2021. The combined FWA customers represent 7% of the entire U.S. broadband market, and if FWA was a single ISP it would be the fourth largest ISP in the country behind Comcast, Charter, and AT&T.

Nobody knows exactly where the companies are finding the new customers because they aren’t telling, and the companies losing customers are mum about it. The FWA technology isn’t everywhere, and T-Mobile claims to cover over 50 million households with the technology, and Verizon 40 million. The appeal of FWA is obvious. The companies offer broadband between 100 Mbps and 300 Mbps in in most markets with prices from $50 to $70 depending on bundling with a cellphone and agreeing to use autopay. The two carriers have also selectively been paying customers to break contracts with other ISPs.

It’s obvious in looking at the claimed coverage of FWA in the FCC broadband maps that a lot of FWA coverage is in rural areas where there aren’t a lot of broadband alternatives. The two carriers are likely snagging customers from DSL, fixed wireless ISPs, and satellite companies, as well as migrating their own rural hotspot customers to the much-improved broadband. However, FWA also covers a lot of towns, suburbs and cities. In these markets, the FWA carriers are touting low prices and faster speeds to lure customers who stayed with telephone company DSL to save money. With low prices, FWA is also clearly targeting cable companies.

It’s been interesting to watch how competitors have been dealing with FWA. In an article in FierceNetwork, Comcast CEO Brian Roberts characterized FWA by saying “Three companies are all simultaneously within a short period of time are all offering a home connectivity product by their own admission a lower speed, more easily congested network.” Comcast is reacting to FWA by advertising the differences between the products. The company has also launched its NOW line of products. This starts with broadband priced at 100 Mbps for $30 or 200 Mbps for $45.

Charter’s CFO Jessica Fischer characterized FWA technology as “lower quality but also lower cost.” She went on to say that FWA will not be able to keep up with increased household demand in future years. Fischer characterized the impact of FWA as “temporary.”

Cox Communication has been advertising against FWA since the end of 2022. In it’s first ads the company said that “FWA is just phone Internet, not home Internet” and isn’t as fast or reliable as Cox’s cable Internet service. The ads went on to warn the public not to “put a cell tower in charge of your home Internet connection.”

Frontier’s Executive Chairman of the Board was quoted at a J.P. Morgan conference as saying that FWA is having almost no impact on Frontier’s fiber business, which is believable since Frontier offers symmetrical 500 Mbps broadband for a standard rate of $64.99, with an introductory rate of $44.99. But Frontier hasn’t been saying anything about the impact of FWA on its DSL service, which is an obvious target for FWA where Frontier has not yet converted to fiber.

It’s an interesting set of reactions. Only Comcast is trying to openly compete with price. It’s likely that the others are quietly offering price deals to keep customers. However, lowering prices has a downside by lowering average revenue per customer – a key financial metric for the industry. ISPs have to decide which is worse – losing customers or lowering prices.

The primary thing that FWA has done to the industry is to shake up the price point for broadband. The big cable companies have all increased list prices annually over the last decade to goose prices to $90 and more. Charter just announced another $3 rate increase across the board. However, a shrinking number of cable customers are paying the list price for broadband.

The cable companies all warn about the ability of FWA technology to serve a lot of customers. The cable companies seem to believe (or at least want the public to believe) that many people will try FWA and not like the broadband experience. The cellular carriers have enough capacity to have gained 7% of the U.S. market in an incredibly short time, and only time will tell if the cell carriers can hang on to these customers over the long haul.

Inside the Telecom Bubble

A recent Harris-Guardian poll shows that the public’s perception of the economy is different than economic reality. Most things that the majority of Americans believe about the economy are wrong. I have to say this surprised me more than it probably should have.

To give some examples

  • 55% of poll respondents think that the economy is shrinking. Almost the same percentage of people believe we are in a recession. The reality is that the economy has grown consistently every month over the last two years. The recession was in 2020 during the pandemic, and the U.S. economy is outperforming the economies of every major economic power.
  • 49% of poll respondents believe that unemployment is at a 50-year high. The opposite is true, and unemployment has stayed below 4% for over two years – something that hasn’t happened since the early 1960s.
  • 49% also think the stock market is down this year, when the market has grown by over 10%, and most markets have reached an all-time high in recent months.

How can the public’s perception be so different than the economic reality?  There are a number of possible reasons. For one, many people base their opinion on issues like the economy according to their own situation and personal feelings. They might know somebody who lost a job and take that to mean that the economy is doing poorly. They might see higher prices for things they regularly buy and think that inflation is still high, while the reality is that inflation was in the past  but has currently slowed back down to historical levels. It’s harder to explain why people think the stock market is down, with the best explanation being the large amount of misinformation in our digital world that gets magnified on social media.

What does all of this have to do with broadband? I often write about the broadband policy issues that are being widely discussed within the industry. People have strong feelings about whether ACP should be renewed and how it should be funded. Inside the industry there are hot and heavy debates occurring about the issue of letting people in apartments opt-out of landlord provided broadband. Industry folks have strong opinions about the BEAD grant program, the FCC, and a host of other industry issues.

But it’s easy for industry insiders to forget that insiders live in a telecom bubble. If half of the people in the country have the wrong beliefs about basic economic facts, then how many people know anything about the end of ACP? In the press outlets that I regularly read, the ACP issue only garnered a minor mention once or twice – certainly not enough for a non-industry person to understand or even notice the issue.

It’s hard to break out of the telecom bubble. I’ve been writing this blog for over eleven years, and I understand it’s written for telecom folks. The only blog I ever wrote that got noticed outside the telecom bubble was a blog that went viral and described how squirrels and gophers like to chew on fiber.

The only telecom issue that reached huge numbers of the general public in the last decade was net neutrality – and most people both for and against the issue got the facts wrong. The arguments for net neutrality argued that reins had to placed on ISPs or they would sell priority access to the point that it would ruin broadband for everybody else. Opponents of net neutrality said that putting any curbs on ISPs would kill broadband investments and innovation.

These arguments for and against implementing net neutrality were far off base. Industry insiders understood that this was not a big fight about fast lanes or monopoly abuse, but about the simple question of whether there should be any regulation of ISPs. In all of the loud back and forth about this issue, the question of regulating or not regulating basic ISP behavior was barely mentioned.

I’ve concluded over the years that industry insiders can be as vulnerable to hyperbole and misunderstanding of the facts as the general public is about the economy. Practically every FCC docket includes exaggerated claims from industry trade groups on the impact of whatever regulatory change is being proposed. Insiders often think the sky is falling over issues that people outside the industry have never heard of.

A Mature Broadband Market?

In several recent blogs, I said that it is becoming clear that the broadband market is reaching maturity. This is already causing havoc in the industry for ISPs that relied on year-over-year customer growth to prop up stock prices.

Earlier this year, the New Street Research, a company that specializes in research in the telecommunications and technology sectors, said that it estimated that new broadband customers would grow by about 1 million this year. That’s roughly equal to the number of new households expected to be created during the year. This is down from the two to three million new broadband subscribers added every year since 2017.

It’s always risky to say that the broadband market has matured, and experts who predicted this in the past turned out to be wrong. We might find that if interest rates go low again that the ISP industry will return to the historical growth pattern. But we are clearly approaching the point where households that can afford to buy home broadband probably have it – and that is the definition of a mature market.

Any student of economics knows that there are behaviors that should be expected in a mature market that differ from a growing market:

  • Growth is Regional. When broadband growth mostly comes from new households, then growth is regional and is concentrated in places where the population is increasing. U.S. New and World Report says that growth is continuing in parts of Colorado, North Carolina, Florida, and Texas. Other markets seeing growth are in Tennessee, Oregon, Georgia, and Arizona. A few cities are also growing the fastest, including Virginia Beach, Miami, Cleveland, and Detroit. The corollary to regional growth is that other parts of the country are losing population, adding pressure to ISPs there.
  • Price Competition. Slowing and tightening markets usually bring price competition when the only opportunity for growth comes from taking customers from competitors. Lower prices also help to reduce churn. We are seeing lower prices across the industry as cable companies are pushing low-price special packages to stave off FWA wireless providers and fiber overbuilders that offer lower prices.
  • Changes in Capital Spending. When markets get tight, a normal reaction is for companies to pare back on capital spending. However, ISPs are facing other challenges that make this hard to predict. Cable companies feel compelled to increase upload speeds to be more in parity with fiber overbuilders. As I wrote in a blog last week, it looks like many cable companies are taking the least expansive paths to increase upload speeds. Some ISPs will react to market stagnation by pouring money into expanding their broadband footprint. Charter is a good example of this and has been growing to some degree through expansion into rural markets through participation in grant and subsidy programs.
  • Mergers and Acquisitions. Mature markets usually lead to market consolidation since one of the few remaining paths to increased profitability is to improve the economy of scale by getting larger. It seems inevitable that we’ll see an increased pace of large ISPs buying smaller ones. It might be tougher for larger ISPs to merge in the current regulatory environment that frowns on monopoly consolidation.
  • Big ISPs in Distress. Continued lower earnings will cause distress at ISPs, and this might eventually lead to bankruptcies. Many of the ISPs in the industry rely on balloon financing where they must periodically retire and replace large amounts of debt. Lower earnings and today’s higher interest rates are going to wreak havoc with debt refinancing.
  • Layoffs. We’ve seen staff reductions at Lumen, AT&T, Verizon, and T-Mobile. I even noticed a layoff last week announced by Charter – a rare event in the past that will likely become more common.
  • Cuts in Maintenance. When companies stop growing, budgets get tighter. That almost always manifests in the broadband industry as a reduction in maintenance since spending on marketing goes up when markets get tough. This translates into more network problems over time and more waiting for repairs.

Upgrades to DOCSIS 4.0

Cable companies often make it sound like DOCSIS 4.0 is right around the corner. This is the technology upgrade that will increase overall speeds to multiple gigabits while also providing the option for symmetrical speeds.

ATX Networks, a leading manufacturer of electronics for cable companies published the results of a survey in May of cable company executives that shows almost half of cable companies intend to implement DOCSIS 4.0 by the end of 2025. That includes 10% of cable companies that say they have already started the upgrade in 2023.

The upgrade to DOCSIS 4.0 is interesting because vendors are not yet offering gear with a full DOCSIS 4.0 solution. Cable companies seem happy to make a partial upgrade now to get somewhat faster speeds and will likely layer on the rest of the upgrade later.

This says a lot about the mindset of cable companies today. They are being besieged from all directions. The public clearly has formed an opinion that fiber is a superior technology to cable technology. The shift to DOCSIS 4.0 will allow cable companies to claim the same speed capabilities as their fiber competitors. If a cable company can upgrade and serve a small portion of customers with a 2-gigabit broadband subscription, they hope they will have evened the playing field in the marketing battle with fiber providers.

Cable companies are also losing customers at the bottom end of the market to FWA cellular wireless as people settle for slower speeds to get significant price savings. A few cable companies, like Charter, face another big customer loss due to the end of the ACP program.

The survey showed that cable companies are taking different approaches on how to upgrade to DOCSIS 4.0. About one-third of the respondents to the survey said they would be happy to get symmetrical speeds using a network with the overall capacity of 1.2 GHz. This is being referred to as a full duplex (FDX) upgrade) since it will provide symmetrical broadband speeds over an existing 1.2 GHz network. The FDX upgrade will get as much extra speed out of a current cable company network as possible, but it saves cable companies from having to go through the expensive process of updating the outside network and amplifiers.

Other cable companies intend to also upgrade the overall bandwidth capability of the network to 1.8 GHz. This is being referred to as an Extended Spectrum (ESD) upgrade. For those wondering why this is expressed in gigahertz, cable networks operate by using a radio network that is captive within the coaxial cable. The DOCSIS technology assigns different portions of this spectrum to different functions, like delivering broadband or cable TV channels. Cable companies upgrading the overall capacity of the network have significantly increased the overall bandwidth available for broadband.

This is the third annual survey from ATX Networks, and the survey shows that more cable companies are now favoring the FDX upgrade, which will be less costly.

Finally, the survey showed that 15% of the cable companies don’t have plans to upgrade to DOCSIS 4.0. There are some interesting alternate upgrades that can milk faster speeds out of an existing DOCSIS 3.1 network. One intriguing patch is to use a DOCSIS 3.1 headend and connect to DOCSIS 4.0 modems for only the customers who want speeds faster than 1 gigabit. This is being labeled as DOCSIS 3.1 Extended and only involves upgrading modems for customers who are willing to buy the fastest speeds.

There are still cable companies that believe the best option is to convert directly to fiber. For example, Cox is upgrading to fiber in some of its largest markets. But that is by far the most expensive option.

The bottom line is that it’s going to be hard in the future to understand the capability of a local cable market. Cable companies are likely to label all their upgrades as having DOCSIS 4.0, even though the different upgrade paths will create networks with widely different capabilities.

Relaxing Letter of Credit Rules

The FCC issued a Notice of Proposed Rulemaking (NPRM) that looks to ease some issues with the letters of credit that must be maintained by ISPs that have accepted FCC grants or subsidy awards. This applies to programs like RDOF, the CAF II Reverse auction, and the upcoming 5G Fund. These changes will not apply to BEAD recipients.

The NPRM recognizes some issues with the current letter of credit process. One key issue is nearly half of the banks that were eligible to issue a letter of credit two years ago no longer meet the FCC’s criteria. A bank must have a Weiss bank safety rating of at least B- in order to qualify for issuing a letter of credit. Weiss Ratings is an independent rating agency that analyzes and provides a rating for over 8,500 commercial banks, savings banks., and S&Ls.

According to the FCC NPRM, there were 3,600 banks that had a B- or better rating from Weiss when the FCC first adopted the Letter of Credit rules. Almost half of those banks now have a lower rating or have failed in the last few years.

Most people don’t understand the stress that higher interest rates put on smaller banks. The natural assumption is that higher interest rates are good for banks – and it is good for the giant national banks. However, smaller banks experience several negative impacts from higher interest rates. While they can charge more for loans they make, the volume of new loans generally drops significantly when interest rates are higher. Smaller banks also don’t use their own money for many of the loans they make and borrow the funds for making loans at an interbank rate from larger banks. The profit margin on such loans doesn’t improve from higher interest rates and can actually shrink. Finally, when interest rates are higher, banks also have to pay higher interest rates to depositors – something that comes out of their own bottom line. All of these various impacts of higher interest rates contribute to banks now failing the Weiss rating test.

The FCC also noted one of the biggest problems for letters of credit. When a bank gives a letter of credit to an ISP, the bank expects the ISP to keep a large percentage of the guaranteed funds on deposit. That is mind-boggling if you think about it. An ISP wins a grant, gets a letter of credit for something like half that amount, and is then expected to have a deposit in the bank for the amount of the letter of credit. Anybody who knows anything about small ISPs should understand that they don’t hoard cash. ISPs generally roll excess cash back into investing in infrastructure. The letter of credit rules are particularly insidious because grant recipients have to somehow use the grant funding to build the promised infrastructure while also finding other cash to deposit in the bank that gave the letter of credit.

You probably wonder why banks charge interest on a letter of credit if the ISP must maintain collateral equal to the amount of the letter of credit. A letter of credit is a guarantee by a bank to repay the FCC for grant funds if a grant project goes sour. A bank that issues a letter of credit is forced to set aside the guaranteed cash, almost the same as if they had made a loan. That money sits without earning interest. Worse, on the bank’s balance sheet, this is categorized as money that has been loaned, not as cash in the bank. To some degree, a large volume of letters of credit contributes to banks having lower ratings.

The purpose of an NPRM is for the FCC to consider alternatives to its current letter of credit rules. The document asks the following types of questions to the industry:

  • Should the FCC have easier letter of credit rules for small ISPs versus large ones?
  • Should the FCC consider something other than Weiss ratings. There are other rating agencies that are subject to SEC regulation, which Weiss is not.
  • Are the letter of credit amounts set too high? For example, RDOF recipients must have a letter of credit to cover all future RDOF subsidies. Should this be reduced to a letter of credit to only guarantee the current year RDOF subsidy? Should ISPs be able to lower the amount of the letter of credit as the project in constructed?

Most ISPs don’t follow FCC proceedings and rarely file comments. But if you are an RDOF recipient you should tell the FCC your story if letters of credit are harming your business. The FCC has opened the door to hear alternate ideas, so this is the chance to be heard.

New NTIA Broadband Survey

The National Telecommunications and Information Administration (NTIA) updated its national broadband survey in November 2023. NTIA has been doing broadband surveys for thirty years, with the last survey in 2021. It has some interesting results since NTIA summarizes the results differently than most other broadband surveys.

The headline finding in the survey is that broadband usage is up. NTIA says that 83% of all people older than three years old used the Internet in 2023, compared to 80% in 2021. That’s an increase of 13 million people. The NTIA survey tries to count everybody in calculating these percentages, from the young to the elderly, the homeless, and folks in group living arrangements.

The survey shows that the increase largely comes from increased use of the Internet for the segments of the population on the wrong side of the digital divide. For example, 83% of American Indians and Alaska Natives used the Internet in 2023, up from 75% in 2021. Internet usage in low-income households grew from 69% in 2021 to 72% in 2023. I have to wonder if these gains will be reversed by the end of the ACP program.

The survey also counted households with broadband and concluded that only 12% of households have no Internet connection. This compares well to an estimate I recall from the FCC last year that estimated that 87% of households have a broadband connection.

There is still a big difference in broadband connectivity based on household incomes. For example, 80% of households that made $100,000 or more had both a fixed and a mobile broadband connection. Only 54% of households with incomes under $25,000 have both.

There are a lot of people who get most or all of their Internet connectivity from smartphones. That includes 25% of Hispanics, 22% of American Indians and Alaskan Natives, and 16% of Black Americans. This contrasts with 12% of White Americans.

There is also a big gap in the percentage of homes with devices like desktops, laptops, and tablets. For example, 72% of White non-Hispanics and Asians used one of these devices in 2023 while 62% of Black Americans, 57% of American Indians and Alaskan Natives, and 54% of Hispanics used the devices.

For those who like to dig deeper into the NTIA data, like me, they have created a tool to make that easier. This tool lets you look in detail at the most recent survey results and also dig into and compare to prior years. Statistics can be displayed as a map, chart, or data table.

Unending Broadband Growth

I wrote a recent sequence of blogs that look at the increasing demand for broadband usage. In today’s blog I’m going to look at some concrete examples of situations where broadband demand has expanded a lot faster than expected.

The first example is in schools. Ten years ago, there was a scramble to get gigabit broadband access to schools. Because of the use of the FCC’s E-rate money, a lot of schools across the country got connected to fiber and were able to buy faster broadband. The original goal was to get a gigabit connection to each school, and I remember a few years ago seeing a report that almost every school in many states met that goal. More recently, the FCC created an updated goal that schools ought to have access to at least 1 Mbps of simultaneous capacity for each student. Connected Nation published a report for 2023 saying that 74% of school districts in the country meet or exceed that new goal – an increase of 57.4% since 2020.

My consulting firm interviews a lot of schools every year, and we’re hearing that the 1 Mbps goal is no longer adequate. Just recently, we heard from a school that meets that target but still can’t have all students take mandatory state performance tests on the same day. The school still has to ration broadband to make sure that too many classrooms aren’t working online at the same time. I’ve talked to schools that have established goals between 3 and 5 Mbps per student to accommodate the way that teachers and students really use broadband.

Another example of fast-growing demand is ISP backhaul. These are the broadband connections that connect local networks to the Internet. I work with a lot of small ISPs. I can remember helping folks find backbone connections a decade ago, and a typical small ISP might have purchased a connection with an overall 10-gigabit capacity but only provisioned a few gigabits of capacity on the connection. Many were amazed at the 10-gigabit capacity when they first ordered it since it felt so oversized. They assumed that the connection capacity was going to be good for many years.

These ISPs turned out to be wrong, and broadband demand grew to swamp the 10-gigabit connections a lot sooner than expected. It’s not hard to understand why. OpenVault has been reporting on the overall average usage of nationwide customers. According to the OpenVault data, the average broadband consumption for homes and businesses has more than tripled just since the end of 2017. At the end of 2023, the average consumption was 641 gigabytes per customer – a number that ISPs would never have believed a decade ago. However, the size of backbone connections is not based on overall broadband consumption but on the busy hour consumption – the time of the day when an ISP’s network is the busiest. Many small ISPs tell me that busy-hour traffic has grown even faster than the average consumption reported by OpenVault.

The final example of broadband demand inflation is the broadband speeds being subscribed by homes and businesses. OpenVault also reports on the subscribed speeds that people are buying nationwide and reported the following statistics for the end of the years between 2019 and 2023.

This chart shows a rapid migration of households now buying faster broadband connections. Some of the increases came when cable companies unilaterally increased customer speeds. Since 2019, many cable companies increased 100 Mbps subscriptions to 200 or 300 Mbps. The chart also shows a big migration of customers buying gigabit broadband. There is nobody in the country who predicted in 2019 that in four short years, there would be an 11-fold increase in households subscribed to gigabit speeds. Most of these gigabit customers pay a premium price to get the faster speed.

There is proof of increasing broadband demand almost everywhere I look. I often talk to businesses that have upgraded to faster speeds only to find out that within a few years they need even more speed. I hear from farmers, photographers, newspapers, and others who send and receive gigantic data files that they are having a problem buying a broadband product that meets their needs.

911 Consolidation

I’ve written several times in the past about how network consolidation and centralization of networks is putting our broadband and voice networks in increased jeopardy. Outages like the latest one that hit AT&T are evidence that we’ve probably gone too far with consolidation. Networks are always going to have problems and glitches. However, problems that would have affected only one city or region in the past can now affect huge parts of the country due to carriers having consolidated electronics hubs and network control at a small number of locations nationwide. It’s easy to understand why carriers are in favor of the savings that come from consolidation, but it’s vital that we recognize and acknowledge the increased risk that comes as a consequence of choosing efficiency over other factors.

There is one area of network consolidation that I think is of particular concern. Carriers have very quietly rearranged the 911 network configuration in the country in a way that I think puts 911 callers in jeopardy.

Historically, 911 was the ultimate local network arrangement. Each city or county had at least one 911 call center (PSAP). Over time, 911 call centers were consolidated regionally. For example, the 911 centers for a dozen cities in a metropolitan area might have been consolidated into one regional 911 center for efficiency. But the consolidated metropolitan 911 call centers were still local.

The 911 PSAP call centers were connected to the closest tandem switch. These were switches that didn’t serve customers but only switched calls between carriers. When a customer of a telephone company, cable company, or CLEC called 911, the call was routed to the closest tandem. The local tandem housed a database that could translate a calling telephone number into an address. Once the tandem identified the address of a caller, the 911 call was routed to the appropriate 911 PSAP. Cellular calls were more of a challenge, but over time the cellular carriers developed software that could triangulate the originating point of a cellular call to 911. Routing 911 calls got more complicated when Congress passed Kari’s Law and the Ray Baum Act, which requires multi-tenant buildings, including businesses and hotels, to identify the specific location within a building for each handset. In this historical network, 911 calls did not leave the region where the call was made.

Every carrier that had voice customers – telephone companies, cable companies, CLECs, VoIP companies, and cellular companies – had to buy a local dedicated connection between their switching center and each local 911 tandem. There was always a risk that a cable cut could stop a 911 call from reaching the 911 tandem, and most 911 rules required carriers to create a diverse path for reaching the 911 tandem. There were inevitably some carriers in every market where the voice switch was located outside of the market, and that carrier had a greater risk of being unable to reach 911 services than local carriers.

Over time, voice services have been consolidated as carriers moved voice functions to regional or national switching platforms. The biggest carriers gained a lot of efficiency by consolidating and reducing the number of voice switches, and a customer call might route calls to a switch in a distant state to initiate any voice call, including 911. In creating these efficiencies, carriers also increased the distance between customers and the electronics used to complete voice calls.

The next big change in 911 networks came when SIP-based trunks were allowed for 911. Historically, 911 trunks were placed on T1s or the equivalent. The T1s were directly routed on a dedicated path between a carrier and each local tandem close to its customers. The change to SIP trunks meant that voice routing to the 911 centers started to use the Internet (or IP-based transport) instead of a dedicated physical path – calls to 911 can be routed in multiple ways. The shift to SIP has resulted in the creation of SIP 911 tandems that can serve a huge footprint, even nationwide.

What does all of this mean? It’s now fairly routine when a customer calls 911 that the call is routed to a 911 tandem that is many states away before ultimately being routed back to the appropriate local 911 center. A call to 911 can travel great distances, and in the network world, greater distance means greater risk. A network outage in a distant city can stop or block a local call to 911. This awkward and unsafe system is why the first thing carriers announce in the middle of a big outage is that 911 calls are still safe. That statement was part of the first announcement made in the recent AT&T outage. Carriers say this because they fully understand how fragile the 911 system has become.

There is no doubt that the current 911 configuration is more efficient. However, I suspect the government folks in charge of 911 centers and public safety have little idea about the great distances covered by a routine 911 call. There would be a big improvement in 911 safety if the 911 tandems and databases were closer to customers, like the historical 911 networks. If I was a government 911 official, I’d make every carrier show me the route taken when a customer dials 911. I think they’ll be shocked, and maybe that can lead to putting the genie back in the bottle. Carriers will always choose consolidation and saving money over other factors. However, saving money is not as important as making sure that 911 calls get delivered when people are in trouble.