Shrinking Cellular Backhaul Revenues

There are a few carriers that rely on cellular backhaul as a major part of their revenue stream, but there are many more carriers that provide transport to a handful of cell sites. In all cases these are some of the highest-margin and lucrative products sold on the market today, and a business line that every carrier wants to keep. However, there are big changes coming in the cellular market and today I will look at the trends that are going to affect this market over the next decade.

Increasing Bandwidth Demand. The growth in bandwidth demand at many cell sites is explosive with the overall growth in cellular data doubling every 18 months. This growth is not the same everywhere with growth coming in cell sites serving residential customers and not in older cell sites built to satisfy highway phone coverage.

The demand growth is being driven by several factors. First, it’s becoming far more prevalent for customers to use cellphones to watch video. Part of that growth in demand comes directly from the big cellular companies which are bundling in access to content as part of the service. But a more important reason for the growth in demand is that the historic reluctance of customers to use cellular data is eroding as the cellular companies push ‘unlimited’ data plans.

Demands for Lower Transport Costs. Cellular service has become a commodity. The industry is no longer adding many new customers since almost everybody has a cellphone. This has led to price wars between cellular providers, and lower average customer prices are driving the cellular companies to look to cost reductions. At least in urban areas they are starting to also lose significant customers to Comcast, with Charter just entering the fray.

Recently I’ve seen cellular companies ask for lower prices as contracts get renewed or else demand greater bandwidth for the prices already in place. This means that fiber owners are not likely to see increases in revenues even as the bandwidth they are delivering grows.

Cellular Carriers Building Fiber. I’ve had several clients tell me recently that Verizon or AT&T is building fiber in their area. While this construction might be to reach a new large customer, the most likely reason these companies are building right now is to eliminate leased transport at cell sites. This is not just happening in urban areas and one of my clients who serves a market of 10,000 homes tells me that Verizon is building fiber to all of the cell sites in the area.

Verizon made headlines last year when they ordered $1 billion in fiber. AT&T is also building furiously. If you believe the claims made by T-Mobile and Sprint as part of the proposed merger – they also will be expanding their own fiber.

I also expect the cellular carriers to make reciprocal deals to swap fiber connections at cell sites where they now own fiber. If Verizon and AT&T each build to 2,000 cell sites they could easily swap transport and both gain access to 4,000 cell sites – that’s a huge nationwide decrease in transport revenues for others.

Growth of Small Cells. Layered on top of all of this is the predicted growth of small cell sites. I don’t think anybody knows how big this might market grow. I’ve seen optimistic predictions that small sell sites will be everywhere and other predictions that the business case for small cell sites might never materialize. Many of my clients are seeing the deployment of a few small cell sites to relive 4G congestion, but it’s hard to predict in smaller markets if this will ever expand past that.

One thing we can know for sure is that the cellular carriers will not be willing to pay the same prices for connection to small cell sites that they’ve been paying for the big cell tower sites. By definition, a smaller cell site is going to serve a smaller number of customers and the pricing must be reduced accordingly for it to make sense for the cellular providers.

Conclusion. My best guess is that cellular transport will be hit and miss depending up the specific local situation. There are many who will lose all cell site transport where the cellular carriers decide to build their own fiber. But even where they don’t build fiber I would expect the cellular carriers to bring the threat of physical bypass into price negotiations to drive transport prices far below where they are today.

This is a natural economic consequence of cellular becoming a commodity. As the cellular industry tightens its belt it’s going to demand lower costs from its supply chain. Transport costs are one of the major costs of the cellular industry and the most natural place for them to look to reduce costs. The big cell companies already understand this future which is one of the primary reasons they are furiously building fiber today while they have the cash to do so.

The Growing Dislike of Big ISPs

The annual ratings from the American Consumer Satisfaction Index came out recently, and they show that consumer dislike for the big ISPs is increasing. This survey looks at how consumers feel about a wide range of businesses, and the ISPs have been ranked as some of the most disliked corporations for a number of years.

The survey asks numerous questions and creates a satisfaction scale from 1 to 100. The survey looks at several different categories of telecom companies and has separate rankings for for cable TV providers, broadband providers and a new category for streaming video providers.

Among the big ISPs that offer cable TV service, the rank of every provider except AT&T U-Verse sank compared to last year. AT&T was the highest rated company in this group with a rating of 70. At the bottom was Mediacom with a rating of 55, down from 56 a year ago. The two giant cable companies both saw a drop in consumer satisfaction: Charter had a huge drop from 63 down to 58, Comcast dropped from 58 to 57.

The rankings for how consumers feel about their broadband provider were similar. The only big ISP that didn’t drop was Comcast that stayed at a ranking of 60 for two years running. Everybody other big ISP dropped. At the top of the list was Verizon FiOS which dropped from 71 to 70. At the bottom was Mediacom again which had a big drop from 58 to 53. Charter also had a big drop from 63 to 58. Rounding out the bottom rankings were Frontier (54), Windstream (56) and CenturyLink (58)

Streaming services got significantly higher rankings. Topping this first time list were Netflix, Playstation Vue and Twitch with a ranking of 78. At the bottom were Sony Crackle (68), Showtime Anywhere (70) and DirecTV Now (70), all still significantly better than traditional cable companies.

It must be frustrating for the big ISPs to see their customer satisfaction drop year after year. The rankings of the ISPs are lower than other unpopular industries like airlines, banks, insurance companies and even the Internal Revenue Service.

If there is any upside to the low customer satisfaction rankings it’s that it creates opportunities for competitors. It’s been conventional wisdom for years that a new competitor will get up to 30% of a market just for showing up with an alternative network – assuming they know how to sell and have decent customer service.

They survey doesn’t dig into the reasons for the sinking dissatisfaction, but it’s easy to speculate on some of the reasons. People are certainly unhappy with traditional cable TV due to the ever-rising prices. High prices are the number one factor cited for consumers who are cutting the cord, and the dropping satisfaction shows there is likely another growing pile of future cord cutters.

It’s a little harder to understand the dissatisfaction with broadband. At least in major metropolitan areas the ISPs have continued to unilaterally increase download speeds with only modest rate hikes. One would expect satisfaction with the the broadband product to be higher and my guess is that the low ranking deal more with the pain involved in having to ever call these big companies. Compared to other businesses we all deal with, the interaction with the cable company / ISP is often the one we dread the most. The other likely cause for dissatisfaction is that ISPs often don’t deliver the speeds they promise. This varies by market, but we’ve seen cities where consumers only get a fraction of the speed they are paying for.

It’s much easier to understand unhappiness with ISPs immediately outside of big cities. Broadband is smaller towns is often still generations behind and is inadequate for what households expect today in terms of download speeds and latency. Anybody who reads this blog will understand the near-hatred for the ISPs in rural areas. The cable companies don’t come to rural America and the big telcos have abandoned maintenance of the copper networks for decades. Rural broadband is either poor or nonexistent with practically everybody hating the companies that won’t bring them broadband.

 

Predicting Broadband Usage on Networks

One of the hardest jobs these days is being a network engineer who is trying to design networks to accommodate future broadband usage. We’ve known for years that the amount of data used by households has been doubling every three years – but predicting broadband usage is never that simple.

Consider the recent news from OpenSource, a company that monitors usage on wireless networks. They report a significant shift in WiFi usage by cellular customers. Over the last year AT&T and Verizon have introduced ‘unlimited’ cellular plans and T-Mobile has pushed their own unlimited plans harder in response. While the AT&T and Verizon plans are not really unlimited and have caps a little larger than 20 GB per month, the introduction of the plans has changed the mindset of numerous users who no longer automatically seek WiFi networks.

In the last year the percentage of WiFi usage on the Verizon network fell from 54% to 51%; on AT&T from 52% to 49%, and on T-Mobile from 42% to 41%. Those might not sound like major shifts, but for the Verizon network it means that the cellular network saw an unexpected additional 6% growth in data volumes in one year over what the company might normally have expected. For a network engineer trying to make sure that all parts of the network are robust enough to handle the traffic this is a huge change and means that chokepoints in the network will appear a lot sooner than expected. In this case the change to unlimited plans is something that was cooked-up by marketing folks and it’s unlikely that the network engineers knew about it any sooner than anybody else.

I’ve seen the same thing happen with fiber networks. I have a client who built one of the first fiber-to-the-home networks and use BPON, the first generation of electronics. The network was delivering broadband speeds of between 25 Mbps and 60 Mbps, with most customers in the range of 40 Mbps.

Last year the company started upgrading nodes to the newer GPON technology, which upped the potential customer speeds on the network to 1 gigabit. The company introduced both a 100 Mbps product and a gigabit product, but very few customers immediately upgraded. The upgrade meant changing the electronics at the customer location, but also involved a big boost in the size of the data pipes between neighborhood nodes and the hub.

The company was shocked to see data usage in the nodes immediately spike upward between 25% and 40%. After all, they had not arbitrarily increased customer speeds across-the-board, but had just changed the technology in the background. For the most part customers had no idea they had been upgraded – so the spike can’t be contributed to a change in customer behavior like what happened to the cellular companies after introducing unlimited data plans.

However, I suspect that MUCH of the increased speeds still came from changed customer behavior. While customers were not notified that the network had been upgraded, I’m sure that many customers noticed the change. The biggest trend we see in household broadband demand over the last two years is the desire by households to utilize multiple big data streams at the same time. Before the upgrades households were likely restricting their usage by not allowing kids to game or do other large bandwidth activities while the household was video streaming or doing work. After the upgrade they probably found they no longer had to self-monitor and restrict usage.

In addition to this likely change in customer behavior the spikes in traffic also were likely due to correcting bottlenecks in the older fiber network that the company had never recognized or understood. I know that there is a general impression in the industry that fiber networks don’t see the same kind of bottlenecks that we expect in cable networks. In the case of this network, a speed test on any given customer generally showed a connection to the hub at the speeds that customers were purchasing – and so the network engineers assumed that everything was okay. There were a few complaints from customers that their speeds bogged down in the evenings, but such calls were sporadic and not widespread.

The company decided to make the upgrade because the old electronics were no longer supported by the vendor and they also wanted to offer faster speeds to increase revenues. They were shocked to find that the old network had been choking customer usage. This change really shook the engineers at the company and they feared that the broadband growth curve was going to now be at the faster rate. Luckily, within a few months each node settled back down to the historic growth rates. However, the company found itself instantly with network usage they hadn’t expected for at least another year, making them that much closer to the next upgrade.

It’s hard for a local network owner to predict the changes they are going to affect the network utilization. For example, they can’t predict that Netflix will start pushing 4K video. They can’t know that the local schools will start giving homework that involves watching a lot of videos at home. Even though we all understand the overall growth curve for broadband usage, it doesn’t grow in a straight line and there are periods of faster and slower growth along the curve. It’s enough to cause network engineers to go gray a little sooner than expected!

AT&T’s Fiber Strategy

On the most recent earnings call with investors, AT&T’s EVP and CFO John Stevens reported that AT&T has only 800,000 customers nationwide remaining on traditional DSL. That’s down from 4.5 million DSL customers just four years ago. The company has been working hard to work its way out of the older technology.

The company overall has 15.8 million total broadband customers including a net gain of 82,000 customers in the first quarter. This compares to overall net growth for the year of 2017 of only 114,000 customers. The company has obviously turned the corner and after years of stagnant growth is adding broadband customers again. The overall number of AT&T broadband customers has been stagnant for many years, and if you go nearly a decade the company had 15 million broadband customers, with 14 million on traditional DSL.

The 15 million customers not served by traditional DSL are served directly by fiber-to-the-premises (FTTP) or fiber-to-the-node (FTTN) – the company doesn’t disclose the number on each technology. The FTTN customers in AT&T are served with newer DSL technologies that bond two copper pairs. This technology generally has relatively short copper drops of less than 3,000 feet and can deliver broadband download speeds above 40 Mbps download. AT&T still has a goal to pass 12.5 million possible customers with fiber by the end of 2019, with an eventual goal to pass around 14 million customers.

The AT&T fiber buildout differs drastically from that done by Verizon FiOS. Verizon built to serve large contiguous neighborhoods to enable mass marketing. AT&T instead is concentrating on three different customer segments to reach the desired passings. They are building fiber to business corridors, building fiber to apartment complexes and finally, offering fiber to homes and businesses that are close to their many existing fiber nodes. Homes close enough to one of these nodes can get fiber while those only a block away probably can’t. It’s an interesting strategy that doesn’t lend itself to mass marketing, which is probably why the press has not been flooded with stories of the company’s fiber expansion. With this buildout strategy I assume the company has a highly targeted marketing effort that reaches out only to locations it can easily reach with fiber.

To a large degree AT&T’s entire fiber strategy is one of cherry picking. They are staying disciplined and are extending fiber to locations that are near to their huge existing fiber networks that were built to reach large businesses, cell sites, schools, etc. I work across the country and I’ve encountered small pockets of AT&T fiber customers in towns of all sizes. The cherry picking strategy makes it impossible to map their fiber footprint since it consists of an apartment complex here and a small cluster of homes there. Interestingly, when AT&T reports these various pockets they end up distorting the FCC’s broadband maps, since those maps count a whole census block as having gigabit fiber speeds if even only one customer can actually get fiber.

Another part of AT&T’s strategy for eliminating traditional DSL is to tear down rural copper and replace DSL with cellular broadband. That effort is being funded to a large extent by the FCC’s CAF II program. The company took $427 million in federal funding to bring broadband to over 1.1 million rural homes and businesses. The CAF II program only requires AT&T and the other telcos to deliver speeds of 10/1 Mbps. Many of these 1.1 million customers had slow DSL with typical speeds in the range of 1 Mbps or even less.

AT&T recently said that they are not pursuing 5G wireless local loops. They’ve looked at the technology that uses 5G wireless links to reach from poles to nearby homes and said that they can’t make a reasonable business case for the technology. They say that it’s just as affordable in their expansion model to build fiber directly to customers. They also know that fiber provides a quality connection but are unsure of the quality of a 5G wireless connection. That announcement takes some of the wind out of the sails for the FCC and legislators who are pressing hard to mandate cheap pole connections for 5G. There are only a few companies that have the capital dollars and footprint to pursue widespread 5G, and if AT&T isn’t pursuing this technology then the whole argument that 5G is the future of residential broadband is suspect.

This is one of the first times that AT&T has clearly described their fiber strategy. Over the last few years I wrote blogs that wondered where AT&T was building fiber, because outside of a few markets where they are competing with companies like Google Fiber it was hard to find any evidence of fiber construction. Instead of large fiber roll-outs across whole markets it turns out that the company has been quietly building a fiber network that adds pockets of fiber customer across their whole footprint. One interesting aspect of this strategy is that those who don’t live close to an AT&T fiber node are not likely to ever get their fiber.

Who’s Pursuing Residential 5G?

I’ve seen article after article over the last year talking about how 5G is going to bring gigabit speeds to residents and give them an alternative to the cable companies. But most of the folks writing these articles are confusing the different technologies and businesses cases that are all being characterized as 5G.

For example, Verizon has announced plans to aggressively pursue 5G for commercial applications starting later this year. The technology they are talking about is a point-to-point wireless link, reminiscent of the radios that have been commonly used since MCI deployed microwave radios to disrupt Ma Bell’s monopoly. The new 5G radios use higher frequencies in the millimeter range and are promising to deliver a few gigabits of speed over distance of a mile or so.

The technology will require a base transmitter and enough height to have a clear-line-of-sight to the customer, likely sited on cell towers or tall buildings. The links are only between the transmitter and one customer. Verizon can use the technology to bring gigabit broadband to buildings not served with fiber today or to provide a second redundant broadband feed to buildings with fiber.

The press has often confused this point-to-point technology with the technology that will be used to bring gigabit broadband to residential neighborhoods. That requires a different technology that is best described as wireless local loops. The neighborhood application is going to require pole-mounted transmitters that will be able to serve homes within perhaps 1,000 feet – meaning a few homes from each transmitter. In order to deliver gigabit speeds the pole-mounted transmitters must be fiber fed, meaning that realistically fiber must be strung up each street that is going to get the technology.

Verizon says it is investigating wireless local loops and it hopes someday to eventually use the technology to target 30 million homes. The key word there is eventually, since this technology is still in the early stages of field trials.

AT&T has said that it is not pursuing wireless local loops. On a recent call with investors, CFO John Stevens said that AT&T could not see a business case for the technology. He called the business case for wireless local loops tricky and said that in order to be profitable a company would have to have a good grasp on who was going to buy service from each transmitter. He says that AT&T is going to stick to it’s current network plans which involve edging out from existing fiber and that serving customers on fiber provides the highest quality product.

That acknowledgement is the first one I’ve heard from one of the big telcos talking about the challenges of operating a widespread wireless network. We know from experience that fiber-to-the-home is an incredibly stable technology. Once installed it generally needs only minor maintenance and requires far less maintenance labor that competing technologies. We also know from many years of experience that wireless technologies require a lot more tinkering. Wireless technology is a lot more temperamental and it might take a decade or more of continuous tweaking until wireless local loop become as stable as FTTH. Whoever deploys the first big wireless local loop networks .better have a fleet of technicians ready to keep it working well.

The last of the big telcos as CenturyLink and their new CEO Jeff Storey has made it clear that the company is going to focus on high-margin enterprise business opportunities and will stop deploying slow-payback technologies like residential broadband. I think we’ve seen the end of CenturyLink investing in any last-mile residential technologies.

So who will be deploying 5G wireless local loops? We know it won’t be AT&T or CenturyLink. We know Verizon is considering it but has made no commitment. It won’t be done by the cable companies which have upgraded to DOCSIS 3.1. There are no other candidates that are willing or able to spend the billions needed to deploy the new technology.

Every new technology needs to be adopted by at least one large ISP to become successful. Vendors won’t do the needed R&D or crank up the production process until they have a customer willing to place a large order for electronics. We’ve seen promising wireless technologies like LMDS and MMDS die in the past because no large ISP embraced the technologies and ordered enough gear to push the technology into the mainstream.

I look at the industry today and I just don’t see any clear success path 5G wireless loop electronics. The big challenged faced by wireless local loops is to become less expensive than fiber-to-the-home. Until the electronics go through a few rounds of improvements that only come after field deployment, the technology is likely to require more technician time than FTTH. It’s hard to foresee anybody taking the chance on this in any grand way.

Verizon could make the leap of faith and sink big money into an untried technology, but that’s risky. We’re more likely to keep seeing press releases talking about field trials and the potential for the 5G technology. But unless Verizon or some other big ISP commits to sinking billions of dollars into the gear it’s likely that 5G local loop technology will fizzle as has happened to other wireless technologies in the past.

The False 5G Narrative in DC

The FCC and some members of Congress have adopted a false narrative about our need for the rapid deployment of 5G. The narrative says that rest of the world is already ahead of the US with 5G deployment and warns about the huge downsides to our economy should we not sweep aside all barriers for deploying 5G.

This is the narrative being used to justify giving wireless carriers cheap and ubiquitous access to poles for 5G transmitters. The FCC and others want to sweep away all state and local rules for pole-related issues. They want rules that will allow wireless carriers to deploy electronics first and straighten out the paperwork later. They argue that all of this is needed so that the country can keep up with the rest of the world in 5G deployment, with some horrific, yet unspecific disastrous result should we fail to make this happen.

The big problem with this narrative is that it’s based upon false premises. The narrative is nothing more than a fairy tale spun by the wireless industry as a way to justify bypassing the normal regulatory process, to hand them fast and cheap connections on poles for wireless devices.

First, there is no big impending needed to deploy huge numbers of 5G devices, because the technology doesn’t yet exist. There are two distinct 5G technologies – 5G cellular and 5G millimeter wave broadband. The industry agrees that it’s going to take a decade until we have a 5G-compliant cellular technology available. There are thirteen key aspects of the new 5G standard that must now be tackled by engineers and then woven into the next generation electronics. We made numerous gradual incremental improvements in technology to evolve from 3G to 4G and it was only last year that we finally saw the first deployments of 4G technology that meets most of the original 4G specifications. There is no reason to think that we are going to progress any faster towards 5G and we will upgrade over time to 4.1G, 4.2G, etc. until a decade from now we finally have a 5G cellular network. By then we will no doubt start over and begin implementing 6G.

There is similarly no pressing need to deploy millimeter wave 5G. This is a technology that promises to potentially offer a gigabit alternative in residential neighborhoods. We have a long way to go before we are going to see wide-spread deployments of this technology. We are just now seeing the first early trials of the technology and it’s going to take years before electronics are widely available and affordable. Further, this technology is going to require a lot of concurrent fiber deployment, and that is likely to be the biggest cost barrier to deployment – not getting onto poles. I even have to wonder who is going to be deploying the 5G millimeter wave radios on a big scale – every one of the big telcos has made it clear that they are backing away from residential broadband, and the big cable companies have, or will soon have, gigabit-capable networks. We might never see the gigabit wireless networks that are the bait being used to tout 5G, because there might not be any deep-pocket ISPs willing to tackle such a large infrastructure investment.

What the wireless carriers are starting to deploy today are 4G small cell sites. These cell sites are being used to supplement and boost the existing cellular networks. The original big-tower cellular network was built to provide voice services and the cell site spacing is terrible for delivering broadband, which uses frequencies that don’t carry as far as the lower frequencies used for voice. The exploding demand for cellular broadband is driving the need for more cell sites just to accommodate the number of users and the amount of bandwidth that can be deployed in a given neighborhood.

The existing cellular networks are clearly under stress in urban areas. But the real issue we should be talking about is how to bolster 4G networks, not how we are already behind in the mythical 5G race. The cellular carriers are crafty and they are using the 5G race narrative as a way to get politicians to support their demands. They are promising wireless gigabit cellular speeds in just the next few years and cheap wireless gigabit broadband soon coming into every home. They have created a feigned panic that the current regulatory rules will stop this progress dead in it’s track unless carriers get fast and cheap pole access.

If this 5G narrative was true we’d be seeing a collapse of cable company stock prices. Cable companies have the most to lose if they are suddenly faced with gigabit cellular and gigabit wireless to the home. We are probably decades away from seeing cellular speeds approaching anything close to a gigabit – that’s the biggest myth in this narrative. And even when the new technology is developed for wireless gigabit to the home one has to ask what ISPs are going to spend the huge billions needed to build that network to compete against the entrenched cable companies.

I don’t want to minimize some of the barriers faced by wireless companies when trying to get onto poles today. Wireless carriers have cited a few horror stories in FCC filings. But like anything else brand new, most pole owners aren’t sure yet how to respond to requests for wireless attachments. There are a lot of issues to work through including safety, pricing, aesthetics and the long-term impact on the real estate space on poles. These are all issues that need solutions, but I can’t find one reason why we need to tackle this at breakneck speed or why we need to give the wireless carriers everything on their wish list. It’s important to bolster the stressed 4G network and we will want to be ready for the 5G technology when it is finally available. We have the time to make the needed regulatory changes in the deliberative manner that makes sure that all aspects of the issues are considered. We don’t need a fast knee-jerk response to a false 5G narrative that might create more problems than it solves.

Are There Any Level Playing Fields?

If you follow regulatory filings, one of the most common arguments you will encounter from the big ISPs is the concept of a level playing field. The idea behind the level playing field is that every competitor in the industry should be working from the same set of rules and nobody should have a market advantage due to regulatory rules. AT&T and Verizon have both rolled out the argument many times when arguing to tighten rules against potential competitors.

There are several good examples of the level playing field argument anywhere that the big ISPs fight to keep municipal entities from building fiber networks. They argue, for example, that municipal entities have an unfair market advantage because they don’t pay state and federal income taxes. But this argument falls apart quickly under examination. First, many municipal ventures such as electric or broadband entities pay in lieu of taxes. This is a tax-like fee that the local government charge to a municipal business. While it’s not really a tax, the fees ac like taxes and can be substantial.

Even more importantly, I can remember many years when AT&T or Verizon made the news due to paying no federal income taxes. Big corporations have numerous tax shelters that allow them to shield income from taxes, and the telcos have gotten numerous favorable rules into the tax code to allow them to walk away from most of their expected tax obligations. You can’t really fault a big corporation for legally avoiding taxes (unless you fault them for the lobbying that slanted the tax codes in their favor to begin with). It’s dishonest for these big ISPs to claim that a municipality has an advantage due to their tax-free status when they pay little or no taxes themselves. Under deeper examination, a municipal fiber venture paying 5% of revenues for in lieu of taxes is often paying a larger percentage of taxes than the big ISPs.

The big ISPs also claim that municipalities have an unfair advantage due to being able to finance fiber networks with municipal bonds. While it’s true that bonds often have a lower interest rate, I have compared bond and bank financing side-by-side many times and for various reasons that are too long to discuss in a blog, bond financing is usually more expensive than commercial loans. It’s also incredibly difficult for a municipality to walk away from a bond obligation while we have numerous examples, such as the Charter bankruptcy a few years back that let a big ISP walk away from repaying the debt used to build their networks.

The big ISPs don’t only use this argument against municipal competitors. AT&T is using the argument as a way to justify hanging 5G wireless devices on poles everywhere. They think there should be a level playing field for pole access, although at this early stage they are one of the few companies looking to deploy 5G small cells. Interestingly, while AT&T wants the right to easy and cheap pole access everywhere, in those places where they own the poles they fight vigorously to keep competitors from getting access. They effectively stopped Google Fiber plans to build in Silicon Valley by denying them access to AT&T poles.

Every time I hear the level playing field argument my first thought is that I would love it if we really had a level playing field. I look at the way that the current FCC is handing the big ISPs their wish list of regulatory rule changes and wish that my clients could get the same kind of favorable regulatory treatment.

A good case in point is again the 5G small cell deployment issue. The FCC has already said that they are in favor of making it cheap and easy for wireless carriers to deploy 5G cell sites. It seems likely that the FCC is going to pass rules to promote 5G deployments unless Congress beats them to the punch. Yet these regulatory efforts to make it easier to deploy 5G conveniently are not asking to make it easier to deploy fiber. If things go in favor of the big ISPs they will have a market advantage where it’s easier to deploy last mile 5G instead of last mile fiber. This will give them a speed-to-market advantage that will let them try to squash anybody trying to compete against them with a FTTP network.

The FCC is supposedly pro-competition, and so if we really had a level playing field they would be passing rules to make it easier to deploy all broadband technologies. They have had decades to fix the pole attachment issues for fiber deployment and have not done so. But now they are in a rush to allow for 5G deployments, giving 5G ISPs a market advantage over other technologies. The consequences for this will be less competition, not more, because we’ve already seen how AT&T and Verizon don’t really compete with the cable companies. In markets where we have both Verizon FiOS and Comcast cable networks both companies charge high prices and are happy with high-margin duopoly competition. There is no reason to think these big ISPs won’t do the same with 5G.

I look around and I don’t see any level playing fields – particularly not any that give small competitors any advantages over the big ISPs. I do, however, so scads of regulatory rules that provide unequal protection for the big ISPs, and with the current FCC that list of advantages is expanding quickly. The big ISPs don’t really want a level playing field because they don’t want actual competition. There are many reasons why other countries have far more last-mile fiber deployed than the US – but one of the biggest reasons are regulatory rules here that protect the big ISPs.

Carrier-of-Last-Resort Obligations

Earlier this month the U.S. District Court for the District of Columbia upheld FCC orders that still require large telcos to be the carrier-of-last-resort provider of telephone service for at least some of their service territory. The ruling is the result of appeals made by CenturyLink and AT&T that required them to provide telephone service to new rural households.

The idea of carrier of last resort has been part of the telephone industry for nearly as long as the FCC has been regulating the industry. The concept was a key component of spreading the telephone network to all corners of the country – the Congress and the early FCC understood that the whole country was better off if everybody was connected.

Over the years the FCC and various state regulatory commissions ruled that telcos had to make a reasonable effort to connect rural customers. Telcos always had the option to petition against adding customers in really hard to reach places like mountaintops, but for the most part telcos routinely added new homes to the telephone network.

Carrier-of-last-resort started to weaken with the introduction of competition from the Telecommunications Act of 1996. Since that time the big telcos have been able to walk away from carrier-of-last-resort obligations in most of their territory. This court order ruled that in areas where the telcos are still receiving federal high cost support that the telcos are still obligated to connect homes that request service.

I worked for Ma Bell pre-divestiture and there was a real pride in the telephone industry that the network reached out to everybody. Telcos then also deployed huge numbers of pay telephones throughout the network to reach those that couldn’t afford phone service – even though they lost money on many of the payphones. The Bell company and the smaller independent telcos made it their mission to extend the network to everybody.

This order made a few comments, though, that puzzled me. They point out that many of the high-cost areas served by the big telcos are up for new funding from the upcoming CAF II auctions. Any winners of that auction are required to file to become the Eligible Telecommunications Carrier (ETC) for any areas they receive funding. The discussion in the court order implies that these new ETCs will become the carrier-of-last-resort in these areas.

That surprised me because there are plenty of carriers that have ETC status and yet are not the obligated carrier-of-last-resort. The best example is the same big telcos examined in this case who are the ETC of record for their whole footprints but now only have carrier-of-last-resort obligations for the last most rural areas covered by this case. There have been stories for years of people who built new homes, even in urban areas, and are refused service by both the telco and cable company. The cable companies have no carrier-of-last-resort obligations, but it’s clear that in many places the telcos have been able to walk away from the obligation.

I think that companies seeking the CAF II reverse auction funding might be surprised by this interpretation of the rules. Being carrier-of-last-resort means that a carrier is obligated to build to reach anybody in the covered area that requests telephone service. The reverse auction doesn’t even require total coverage of the covered census blocks and that seems to be in conflict with the court’s interpretation. The reverse auction census blocks are some of the most sparsely populated areas of the country and building to even one remote customer in some of these areas could be extremely expensive.

Unfortunately, the carrier-of-last-resort obligation only applies to telephone service and not to broadband. It would be nice to see this concept applied to broadband and the FCC missed a good opportunity to do this when they handed out billions of federal dollars in the CAF II plan. With that plan the big telcos are only required to make their best effort to reach customers with broadband in the areas that got the CAF II funding – I’m hearing from rural people all over the country that a lot of the CAF II areas aren’t seeing any upgrades. For the most part the idea of carrier-of-last resort and universal coverage are becoming quaint concepts of our past.

SDN Finally Comes to Telecom

For years we’ve heard that Software Defined Networking (SDN) is coming to telecom. There have been some movement in that area in routing on long-haul fiber routes, but mostly this network concept is not being used in telecom networks.

AT&T just announced the first major deployment of SDN. They will be introducing more than 60,000 ‘white box’ routers into their cellular networks. White box means that the routers are essentially blank generic hardware that comes with no software or operating systems. This differs from the normal routers from companies like Cisco that come with a full suite of software that defines how the box will function. In fact, from a cost perspective the software costs a lot more than the software in a traditional router.

AT&T will now be buying low-cost hardware and will load their own software onto the boxes. This is not a new concept and the big data center companies like Facebook and Google have been doing this for several years. SDN let’s a provider load only the software they need to support just the functions they need. The data center providers say that simplifying the software saves them a fortune in power costs and air conditioning since the routers are far more efficient.

AT&T is a little late to the game compared to the big web companies, and it’s probably taken them a lot longer to develop their own proprietary suite of cell site software since it’s a lot more complicated than switches in a big data center. They wouldn’t want to hand their cell sites over to new software until it’s been tested hard in a variety of environments.

This move will save AT&T a lot of money over time. There’s the obvious savings on the white box routers. But the real savings is in efficiency. AT&T has a fleet of employees and contractors whose sole function is to upgrade cell sites. If you’ve followed the company you’ve seen that it takes them a while to introduce upgrades into their networks as technicians often have to visit every cell site, each with different generics of operating hardware and software.

The company will still need to visit cell sites to make hardware changes, but the promise of SDN is that software changes can be implemented across their whole network in a short period of time. This means they can fix security flaws or introduce new features quickly. They will have a far more homogeneous network where cell sites use the same generics of hardware and software, which should reduce glitches and local problems. The company will save a lot on labor and contractor costs.

This isn’t good news for the rest of the industry. This means that Cisco and other router makers are going to sell far fewer telecom-specific routers. The smaller companies in the country have always ridden the coattails of AT&T and Verizon, whose purchase of switches and routers pulled down the cost of these boxes for everybody else. These big companies also pushed the switch manufacturers to constantly improve their equipment, and the volume of boxes sold justified the router manufacturers to do the needed R&D.

You might think that smaller carriers could also buy their own white box routers to also save money. This looks particularly attractive since AT&T is developing some of the software collaboratively with other carriers and making the generic software available to everybody. But the generic base software is not the same software that will run AT&T’s new boxes. They’ve undoubtedly sunken tens of millions into customizing the software further. Smaller carriers won’t have the resources to customize this software to make it fully functional.

This change will ripple through the industry in other ways. For years companies often hired technicians who had Cisco certification on various types of equipment, knowing that they understood the basics of how the software could be operated. But as Cisco and other routers are edged out of the industry there are going to be far fewer jobs for those who are Cisco certified. I saw an article a few years ago that predicted that SDN would decimate the technician work force by eliminating a huge percentage of jobs over time. AT&T will need surprisingly few engineers and techs at a central hub now to update their whole network.

We’ve known this change has been coming for five years, but now the first wave of it is here. SDN will be one of the biggest transformational technologies we’ve seen in years – it will make the big carriers nimble, something they have never been. And they are going to make it harder over time for all of the smaller carriers that compete with them – something AT&T doesn’t mind in the least.

Convergence

Even a decade ago it was apparent that the telecom industry was headed towards convergence. By that, I mean that the various cable companies, telcos and wireless companies are expanding service lines and are starting to compete with each other in areas that were unimaginable even a few years ago.

Comcast is the best example of this. Their CEO Brian Roberts was quoted last year as saying that the company was now in all of the business lines available to it. Compare today’s Comcast with the company a decade ago. Then they were just becoming a triple play provider by adding voice to their product line-up. Since then they have added a lot more business lines.

A decade ago Comcast barely went after business customers and didn’t even own network in many business districts and industrial parks – and now they are a major provider of business services. The company recently added cellular service and it appears they are adding customers at a furious pace. They are becoming a major player in home security. The company has a thriving product line selling residential smart home services. They even started bundling home solar panels with their residential product line recently.

The company has even stepped up their traditional cable service to do better against the satellite providers. They’ve developed their own settop box that is said to be the best in the industry. And they have bought a number of the cable programmers like NBC, giving them a margin advantage over any competitor for video.

It seems to me like everybody else wants to be Comcast. Consider AT&T. A decade ago they were a traditional telco. They operated a huge copper network for residential broadband and telephone service and owned the country’s largest fiber network for providing wholesale transport and business services. They were also one of the two largest cellular companies, and with Verizon controlled the vast majority of that business.

AT&T not only added cable TV service to their product line, but they bought DirecTV and become a major video provider. They are trying hard to buy programming and content by merging with Time Warner. The company has been aggressive building fiber to large apartment complexes and has become a major player in the MDU market that used to be almost exclusively controlled by the cable incumbents. The company has also been building a lot of fiber to better compete head-to-head with Comcast and other cable companies that have faster residential broadband.

Verizon took a different path and competed head-to-head with Comcast in the northeast even a decade ago with its FiOS fiber network. The company continues to buy smaller regional fiber providers like XO to beef up its business and fiber networks. Verizon has announced that it intends to roar back into the residential market by use of small cell 5G over the next decade. And Verizon continues to thrive as a cellular carrier.

Even smaller companies like CenturyLink are looking a lot like their bigger competitors. The company had added cable to its bundle. They built fiber past almost a million passings last year to provide more robust competition for broadband speeds. And they bought Level 3 to become a major player for transport and business services.

But these big ISPs are not the only ones crossing into new product lines. Consider T-Mobile. In a move that was unthinkable even a few years ago they are making a major play to bundle video content with their cellular service – making them a direct competitor of all of the ISPs for the market segment of folks who are happy with mobile video rather than a landline connection. T-Mobile is pushing the other cellular providers to do the same.

And there are other national competitors on the horizon. For example, there are several satellite companies like SpaceX and OneWeb that are likely to compete nationally with bundles similar to the other ISPs. I also think we’ll see new competitors spring up and compete with 5G last-mile networks as that technology matures.

It’s going to be interesting to see the winners and losers over the next decade. Right now the cable companies are approaching a near monopoly in many markets for broadband. The only way these other competitors are going to survive and thrive is to chop away at Comcast and the other large cable companies. But at the same time the cable companies will be carving cellular customers. For those like me who follow the industry it’s going to be interesting to watch.