Easing of the Supply Chain

One of the biggest broadband issues in 2021 and 2022 was that ISPs suddenly found themselves unable to buy the materials and electronics needed to construct networks. Within six months after the onset of the pandemic, wait times for many fiber and electronics components had stretched to a year or longer. Before the pandemic, ISPs knew they could order materials a few months ahead of construction and have the material on-site when needed.

The Fiber Broadband Association recently announced a dramatic improvement in the supply chain for fiber networks. The FBA cited the following decreases in the waiting times for the delivery of fiber materials by comparing the average wait times for material deliveries in the summer of 2022 to March 2023.

  • The lead time for fiber cable dropped from 52-60 weeks to 4-10 weeks.
  • The delivery time for handholes decreased from 22-26 weeks to 8-14 weeks.
  • The wait times for fiber cabinets and splitters decreased from 10-20 weeks to 4-8 weeks.
  • Delivery times for fiber multiport terminals dropped from 20-35 weeks to 4-8 weeks.
  • Conduit backlogs decreased from 15-20 weeks to 3-7 weeks.
  • The wait time for home electronics and wiring decreased from 12-24 weeks to 4-10 weeks.

This is all good news for the many ISPs building fiber networks. Lead times have almost returned to pre-pandemic levels, and somebody building fiber no longer has to worry about being able to obtain the needed materials.

There are a lot of reasons for the supply chain improvements:

  • New Capacity. New factories have been built to manufacture fiber, conduit, and other components.
  • Re-Sourcing. Before the pandemic, a lot of fiber electronics were produced in the Wuhan province in China. When the pandemic largely shut down the province, manufacturers of electronics found other factories around the world ready to take on the manufacturing. This process took a few years, but China is no longer the only source of some key components.
  • Raw Materials. The pandemic also shut down the mining and shipping of rare metals and other raw materials that are key components of electronics. Those supply chains are now back and operating at pre-pandemic levels. There has also been a worldwide push by geologists to find new sources of ores, and there are new mines being opened around the world.
  • Logistics. During the pandemic, I saw daily stories about how the ports and the trucking industry in the U.S. were backed up and paralyzed. Even when materials were delivered from overseas, it could take months to get them through the ports. At the same time, the pandemic saw a lot of older truckers retire and created a trucker shortage. Logistics are getting back to normal, but I’ve noticed that the radio is still full of ads from trucking companies looking for drivers.
  • Reduced Demand. The largest ISPs in the county collectively trimmed their projected construction for 2023 by millions of passings. That alone frees up a lot of demand for materials that can be used elsewhere.

We may not be out of the woods completely for the supply chain. Future increases in demand from both the big ISPs and the billions that will be spent on grants are going to keep the pressure on manufacturers. But as long as the pandemic-related factors don’t return, this might mean slightly longer delivery times, but nothing like we saw during the pandemic. I recall getting calls from folks who were aghast that they were being given delivery times of over a year for fiber. It seems like those ugly times are behind us.

Rural Cellular Coverage

When working in rural areas, I find invariably that any county that has poor rural broadband also has poor cellular coverage. If you plot a 2 or 3-mile circle around the existing cell towers in many counties, it becomes quickly obvious that cell coverage is non-existent in many places. The real cellular coverage in rural areas is drastically different than the national coverage maps that cellular carriers have been advertising for years.

The FCC announced a process to address this issue in October 2020 when it announced the creation of a 5G Fund for Rural America. This will be a $9 billion fund that comes from the Universal Service Fund and that will provide subsidies for wireless carriers to build and equip new rural cell towers. This fund would work through a reverse auction in the same manner as RDOF, with the only bidders in the auction being licensed cellular carriers. The first reverse auction will be for $8 billion, with the rest specifically set aside for tribal areas.

The FCC tried this a few years earlier and abandoned the process when it became obvious that the cellular coverage maps created by the big cellular companies had little to do with reality. As part of that effort the FCC required cellular carriers to submit maps of cellular coverage as a prelude to launching this fund. The smaller cellular companies all complained that the big cellular company maps were wrong and were aimed at locking them out of the reverse auction. The FCC agreed and canceled plans for the fund until the 2020 announcement.

I haven’t been following this issue closely enough at the FCC to understand why it’s taking so long to launch the endeavor, but I have to think that mapping is still a primary issue. Then FCC has now included cellular coverage in the same BDC mapping process used for broadband. When the new maps were released there were a lot of public complaints that the new FCC cellular maps still overstate rural coverage.

There is a map challenge process for the public to provide feedback to try to fix the cellular maps by taking speed tests from rural locations – but the process is cumbersome, and it’s likely that few people know about it or are providing the speed tests in the specified way. The speed tests must be logged through an FCC app.

There is no question that something like this funding is badly needed. It’s hard to justify building rural cell towers and installing radios at a tower will only see a handful of homes. Remote rural cell sites can’t possibly generate enough money to justify the cost of the radios and backhaul, let alone the towers. One of the issues that the FCC is going to have to face is that any subsidy for this issue might need to be permanent if the goal is to keep cell towers operating where few people live.

Poor cell coverage is devastating to an area. There are huge swaths of the country where folks can’t reach 911 by cellphone. We can’t get serious about smart agriculture without the bare minimum network to provide connectivity. No cell coverage makes it hard to do tasks that the rest of us take for granted.

One of the interesting things about the timing of this effort is how the rural cellular industry will benefit from the BEAD grants. There is no fiber near many of the best spots for rural towers, and the BEAD grants will fund the construction of a lot of fiber in rural areas that could be used to provide backhaul to new cell sites.

Interestingly, one of the things that was missed in creating the BEAD rules was any requirement for BEAD grant winners to provide fiber connectivity to rural cell towers at a fair price. That would have been a good opportunity for these different federal programs to mesh together for the benefit of both wireless and wireline rural broadband. One of the legitimate complaints made by cellular companies is that they are often quoted extremely high prices for broadband connectivity at cell towers – a lot of ISPs look at cell towers as a chance to make a lot of money.

Communities with poor cellular coverage need to keep an eye on this FCC program to make sure that some cellular carrier seeks the funding for building in their county. Just like with the BEAD grants, I have no idea of $9 billion is enough to get cellular coverage everywhere – but it is a good start.

Digital Payments

When the iPhone first hit the market, the pundits started touting the huge benefits that would come from carrying around a computer in our hands. Some of those benefits have been transformational. There used to be a rack with maps inside every gas station and convenience store to help travelers figure out directions. The map industry has been completely displaced by online GPS and driving instructions that have brought huge efficiency and a lot fewer lost travelers wandering rural roads.

We were also told that the Rolodex was dead and that you would be carrying everybody’s contact information with you – something that quickly became true. When was the last time that you called information to get somebody’s telephone number?

At the top of the claimed benefits was the promise that we’d quickly be paying for everything seamlessly with our smartphone. We’d be able to buy from a vending machine or shop at a store by just waving our phone.

There has been some movement in recent years to make this easier. You can load credit or debit cards into your phone and use Apple Pay, Google Pay, or Samsung Pay at places that accept the payments. There is a more recent movement to allow people to seamlessly pay each other through direct bank debits without having to use an intermediary service.

But we are nowhere near universal acceptance of payments through a phone. There are a number of reasons why this is still the case 16 years after the promise that this was right around the corner.

  • A bank survey in 2022 showed that 38% of Americans would refuse to use such a payment system. But that is not an excuse for making it easier for everybody else.
  • For many years, financial institutions didn’t have any interest in accepting micropayments. Banks were not interested in enabling a system that would generate millions of $1 transactions at vending machines or other types of small transactions. The fees the banks wanted for the transaction were too high to make this reasonable.
  • There were always a lot of concerns about security. Somebody could steal a phone with an automatic payment system and spend it without scrutiny. That’s being solved in many cases by phones tied into biometrics.
  • All of the proposed payment solutions require sellers and retailers to foot the bill for the electronic readers that can accept payments. This is particularly challenging when there isn’t a universal reader that would accept payments for multiple payment systems. The different payment systems have been pushing unique hardware solutions. This has led to many merchants unwilling to embrace electronic payments.
  • It’s even more of a challenge to equip millions of vending machines, gas pumps, and other payment portals with readers, particularly those in an outdoor environment.
  • There are still plenty of merchants in rural areas that have problems accepting credit cards the traditional way. A credit card transaction doesn’t require the transfer of a lot of data, but it requires a stable connection to be held during the length of a transaction. A lot of rural broadband fluctuates and kills a lot of credit card transactions.

Perhaps the most important reason it’s not widespread here is that the U.S. took the high-technology approach, like we do with many things. Requiring a new set of payment readers is good business for the merchant service companies that provide the readers and the software for merchants.

To demonstrate how we might have taken the wrong path, we only need to look at India. A common payment method for outdoor street vendors is to have a QR code posted. A buyer scans the QR code, which sends them to a portal where they approve the amount of payment. When the payment is complete, a message is sent and is usually played out loud on the merchant’s cell phone. When somebody buys food from a food cart, the payment can be completed by the time the seller is ready to hand over the food. Maybe we are just making this too complicated.

The Need for Mid-Band Spectrum

5G Americas recently released its annual white paper discussing the lack of activity at the FCC in making more mid-band spectrum available for cellular broadband. The group is made up of large cellular carriers and various vendors or other companies associated with the cellular business.

Midband spectrum is an industry-defined term for the spectrum between 1 GHz and 7 GHz spectrum. This is the sweet spot for cellular broadband because these bands of spectrum can cover the distances needed for cell phone data along and carry a decent amount of bandwidth.

The paper laments that are no actions currently at the FCC to consider any new bands of spectrum in the range for cellular data. This is a concern for the cellular industry because it takes many years to open up a frequency for a new use. All parts of mid-band spectrum are currently in use. Any plan to free existing spectrum for cellular use means either relocating the current users to a different frequency or finding a way to accommodate them to coexist with cellular carriers.

The report does a great job of looking at the status of each mid-band spectrum block. Reading through the uses, it becomes quickly apparent that a lot of these spectrum bands are reserved for the U.S. government and includes uses like air traffic control, commercial and military radar, airplane altimeters, and numerous military applications.

I’m always instantly leery of any statistics, but the paper cites a report by Ericsson that the worldwide demand for cellular data is growing at 40% annually. Even if that number is true, I have to imagine that most of the increased demand comes in third-world countries where cellular is the predominant way to use the Internet and where the technology in many networks is still far behind what we have here. This statistic feels like a scare tactic, because 40% growth per year would mean a doubling of network demand every 2.5 years. If that growth was true in the U.S., we’d have heard a lot more about this growth outside of this whitepaper.

But I don’t know anybody who doesn’t think that we’ll eventually need more spectrum for mobile services. All uses of broadband keep growing, and it’s not hard to look out ten and twenty years and see a much larger demand for using wireless spectrum.

The report includes one statistic that I hadn’t seen anywhere else. It says that at the end of 2022 that North America had 108 million connections on the spectrum carriers have labeled as 5G and 506 connections that are still using 4G LTE. The initial goal for using the new mid-band 5G spectrum was to de-load 4G networks – the goal was never to move everybody to 5G. I would have expected more users of the 5G spectrum bands, but there still are a lot of cell sites that have not been upgraded to the 5G spectrum.

I think cellular carriers are going to have a challenge making their case to the public. The carriers have done a magnificent job, at least in cities, of increasing cellular speeds. According to the latest report from Ookla, the median nationwide download speed in March 2023 was over 81 Mbps, with speeds in cities over 100 Mbps.

It’s going to be more of a challenge since cellular carriers have lost some credibility with the public and politicians. They badly needed additional spectrum five years ago, but rather than openly plead that case, the carriers invented an imaginary 5G war with China and convinced the public that giving them more spectrum would transform the world. The dilemma for cellular companies now is that it’s clear that most of the public isn’t willing to spend more to get faster cellular speeds. There is no public outcry supporting more spectrum for cellular companies.

But the public has a short memory. Five years ago, a lot of markets were having huge cellular problems. It was so bad in some places that it was getting hard to make and hold mobile voice calls. The new spectrum bands that we’re labeling as 5G had a big role in solving that problem. As this whitepaper argues, we don’t want to wait until the networks degrade to have the conversation again.

Buy America and BEAD

In the State of the Union speech earlier this year, President Biden made it clear that he wants to see the monies spent on infrastructure projects follow the Buy America rules. The Buy America rules were enacted in 1933. The Act says that purchasing funded by the U.S. government should have a preference for using American-made products. The rules allow for waivers from this provision, but the presumption is that without a waiver that American goods must be used.

The NTIA reacted to the president’s speech by writing a blog talking about the use of the Buy America rules in the upcoming $42.5 billion BEAD grants. The blog states, “The president made clear that while Buy America has been the law of the land since 1933, too many administrations have found ways to skirt its requirements. We will not.”

The NTIA requested waivers from Buy America rules when administering past grant programs, including the recent $1 billion middle-mile grants. The USDA sought a 6-month waiver of these rules that applied to some earlier rounds of the ReConnect grants. But the NTIA has made it clear that it doesn’t see any need for a waiver to buy American fiber optic glass or cable. The NTIA says there should be sufficient time for manufacturers to re-shore or expand U.S. manufacturing to meet the demands from the BEAD grants.

In the requested waiver for the Middle Mile Grant Program, the NTIA identified components of a fiber network that are sourced almost exclusively in Asia. This includes electronics like broadband switching equipment, broadband routing equipment, dense wave division multiplexing transport equipment, and broadband access equipment. It doesn’t seem likely that U.S. vendors are going to step up to create an American source for these components in time to meet the needs of the BEAD grants. And while the BEAD grants are substantial, they are not alone enough inducement to manufacture these goods in this country.

The market reality is that most of the costs of any broadband grant project will be spent on American inputs. The cost of labor is usually the largest component of network costs, and the grants require this work be done by American firms. As the NTIA points out, there are plenty of sources for American fiber and conduit. There are American sources of cabinets, huts, and enclosures. There are American vendors making handholes and pedestals.

But the sticky item is going to be electronics. If the NTIA plays hardball on fiber electronics, it will be nearly impossible that any ISP can fulfill the Buy American provision. I’m not as familiar with where wireless electronics are manufactured, but I assume that WISPs have a lot of the same concerns. Electronics are a relatively tiny slice of the total cost of a fiber network but a larger percentage for a new wireless network. .

The arbiter of the Buy American rules is the U.S. Office of Management and Budget (OMB), which recently solicited nationwide comments about how firmly the Buy American rules should be enforced for projects that will be funded by the Infrastructure Investment and Jobs Act. There is a possibility that the OMB will be stingy with waivers even if the NTIA asks for them, but that’s a bridge that can’t be crossed until it happens.

What’s most disturbing is that this joins a list of other issues that create a lot of uncertainty for ISPs considering the BEAD grants. If we don’t start clearing up the uncertainties, states might find that the ISPs they are hoping will request grants will sit out the BEAD grants. ISPs are naturally attracted to grants, but not if the hurdles are too hard to overcome.

GM Wants to Curate Your Car Experience

General Motors recently announced that it is going to stop supporting Apple CarPlay and Android Auto in some of its vehicles. These are smartphone mirroring apps that let a driver use their cellphone to connect to music, get driving directions, listen to eBooks, etc. GM announced that it plans to block the smartphone connection capability and will instead run a Google infotainment suite that includes Google Maps, Google Assistant, Spotify, and other apps that will be built into the dashboard display.

The company is not alone, and other companies like Mercedes and VW don’t like smartphone mirroring. GM says that it is doing this to take back control over customers and the in-car experience. I had to pause at that statement because I can’t think of a time when carmakers had that kind of control.

An article in Light Reading quoted an analyst saying that this means that the bandwidth used by the average car would grow from a few hundred megabytes per month to 4-8 gigabytes per month. That seems like a gigantic increase in bandwidth to me to take over the functions that were already going through a cellphone. Does this mean that the average driver really uses 4-8 gigabytes per month on the cellphone while driving? That can’t be true, and there is more at play here.

This raises a lot of questions for me. Does this finally mean that AT&T will reach its dream of requiring car owners to subscribe to a cellular subscription? That’s something the company has been angling for since the first conversations about smart cars and 5G. It seems likely that the cost of this service will be embedded in the cost of the car for the first year, but will all car owners be required to subscribe to this service when the paid year lapses? You might not have a choice if you can’t use your cell phone. Perhaps the car makers will pay this for a longer period if gaining control of the customer experience can generate additional monetary benefits higher than the cost of the cellular subscription.

Car companies have been trying to force subscriptions on car owners for years with the OnStar service. But most people drop that service at the end of the free period after buying a new car. I may be wrong, but I can’t see most car owners willing to buy a new monthly data subscription. There is no doubt that a 4–8 gigabyte cellular subscription is not going to come cheap.

Carmakers wouldn’t be considering this unless it will make them money. I can think of several ways this could financially benefit them. They might get a share of any revenues paid to AT&T for a subscription. I have to imagine Google will pay them for getting access to a car’s data – having a car connected to a cellular plan will let car makers gather detailed analytics on how the car is being driven, and I imagine that creates a revenue opportunity for selling driver data to insurance companies and others. A car is not going to use 8 gigabytes of data monthly by connecting only to GPS and listening to music. That much data has to mean transferring a lot of base analytics about the car and the driver. I can’t imagine paying for a subscription that would let GM and Google spy on me.

This also raises questions about tying my car to a cellular carrier. The new FCC maps for the big cellular companies are a joke. There are huge areas of the country that have little or no cellular coverage. I live in Appalachia, and I don’t have to drive far to find areas with no cell coverage. One town we visit is Boone, NC, and over half of the drive between here and there has zero cell coverage. How will car companies deal with irate customers that require a service that doesn’t function where they live? My wife listens to an eBook from her phone on that drive – I know how upset she would be if that no longer works because she can’t connect her cellphone to the car speakers.

I’m not sure why carmakers think folks want or will accept this. I might be the exception, but I would never buy a car that forced this on me unless I had the option to disable it. I don’t want to be curated and monitored by my carmaker. Their relationship with me ends the day I pay for the car. My wife avidly dislikes Android and wouldn’t buy a car that forced her to connect to Google and Android instead of her preferred IOs. If GM or any other company mandates this, we’d take them off our list of cars to consider.

Shutting Down Obsolete Technologies

There was an interesting statement during the recent Verizon first quarter earnings report call. The company admitted that shutting down the 3G cellular networks cost it about 1.1 million retail cellular customers along with the corresponding revenues.

This was long expected because there are still a lot of places where 3G technology was the only cellular signal available to rural customers living in remote areas. There were also a lot of people still happy with 3G flip phones even where 4G was available. Some of these customers will likely come back with 4G phones, but many might be angry with Verizon for cutting them off and go elsewhere.

Verizon has been trying to shut down the 3G network for at least five years. Its original plans got delayed due to discussions with the FCC and then got further delayed because of the pandemic – it didn’t seem like a good idea to cut folks dead when cellular stores were shuttered.

This change was inevitable. The bandwidth that can be delivered on the 3G networks is tiny. Most of you remember when you used 3G and a flip phone to check the weather and sports scores. Cellular carriers want to repurpose the spectrum used for 3G to support 4G and 5G. This is something that is inevitable – technologies become obsolete and have to be upgraded or replaced. The 3G transition is particularly abrupt, because the only possible transition is to cut the 3G signal dead, and 3G phones become bricks.

All of the technologies used for broadband and telecom eventually become obsolete. I remember when we used ISDN to deliver 128 Kbps broadband to businesses. I remember working with n-carrier and other technologies for creating data connections between central offices. Telephone switches took up a big room instead of being housed inside a small computer. The earlier version of DOCSIS technology were largely abandoned and upgraded to new technology. BPON became GPON and is now becoming XGS-PON.

Most transitions to new technologies are phased in over time. You might be surprised that there are still working ISDN lines chugging along that are being used to monitor remote sensors. There are still tiny rural cable companies operating the early versions of DOCSIS. But the industry inevitably replaces ancient technology in the same way that none of you are reading this blog on an IBM 5150 or a Macintosh 128k.

But some upgrades are painful. There were folks who lost cellular coverage when 3G was cut dead since they lived in a place that might not be able to receive the 4G replacement. A 3G phone needed only a tiny amount of bandwidth to operate – at levels that newer phones would perceive to be far under one bar of service.

The other painful technology replacement that keeps getting press is the big telcos killing off the copper networks. When copper is cut off in an area, the traditional copper landlines and DSL go dead. In some cases, customers are offered to move to a fiber network. The price might be higher, but such customers are offered a good permanent technology replacement. But not every DSL customer in a city that loses copper service is offered a fiber alternative. Customers find themselves likely having to pay $30 or $40 more to move to the cable company.

In rural areas, the telcos often offer to move customers to wireless. For a customer that lives within a decent distance from a cell tower, this should be an upgrade. Fixed wireless delivered for only a few miles should be faster than rural DSL. But like all wireless technologies, there is a distance limitation around any given tower, and the FWA signal isn’t going to work for everybody. Some customers that lose rural copper are left with whatever alternatives are available – because the telephone company is basically abandoning them. In many rural areas, the broadband alternatives are dreadful – which is why many were sticking with slow rural DSL.

I hear a lot of complaints from folks who lose traditional copper who are upset that they lose the ability to use services that work on copper technology, such as fax machines and medical monitors. It may sound uncaring, but these folks need to buy something newer that works with today’s broadband. Those are the kind of changes that are inevitable with technology upgrades. Just like you can’t take your old Macintosh to get fixed at Best Buy, you can’t keep using a technology that nobody supports. That’s an inevitable result of technology getting better over time. This is not a comfort to the farmer who just lost his 3G cell coverage – but there is no way to keep older technology operating forever.

Matter – The New IoT Standard

Anybody that uses more than one brand of Internet of Things (IoT) device in the home understands that there is no standard way to connect to these devices. Each manufacturer chooses from a range of different protocols to communicate with and control its devices, such as BLE, LoRa, LTE-M, NB-IoT, SigFox, ZigBee, and others. Every family of devices, and typically every different brand requires a separate app on your smartphone, which means managing a pile of different apps, passwords, and logs-ins to control your devices.

The situation is tougher on businesses. Consider a farmer that might need a dozen sets of software to control the different smart devices and systems installed in a modern dairy or chicken farm. Farmers have complained to me that it’s been growing increasingly complex to manage the electronics in their operation from day to day. Not only must they master different systems to control each set of devices, but the outputs of the various systems are not in a compatible format to communicate with other systems. A farmer must manually intervene if an alarm from one set of devices needs a response from other devices.

This is a big problem also for larger businesses that deploy IoT devices. It’s not uncommon for the makers of smart devices to retool their products over time, and a large business might find over time that it has multiple generations of smoke alarms, security cameras, smart door locks, or other devices from the same manufacturer that each require a different set of software to control. Companies have sometimes resorted to ripping and replacing older but still functional devices that are incompatible with the newest generation of devices.

Big companies also have the same problems as farmers in that there is no easy way to tie devices together onto one platform. The benefit of using smart sensors loses a lot of appeal if people are needed to manually interpret and intervene when trying to coordinate alarms or other events. Some companies have spent a lot of money to develop unique software to make sense of the outputs of different kinds of smart sensors – but that software has to constantly be tweaked for new devices.

The manufacturers of smart devices recognized that the chaos in the industry is holding down sales. Amazon, Apple, Google, and more than 200 other makers of home electronics and smart devices got together to develop a common IoT platform. These manufacturers agreed that it is important for them to work together, even though they are market rivals because the confusion created by the multiple communications platforms for IoT devices is hurting sales for the industry as a whole.

The new IoT platform that addresses the problems of the industry has been named Matter. There were hundreds of new devices using Matter at this years CES from a variety of vendors. Matter has created a standard language for interpreting the outputs from IoT devices. This means that the commands to operate a smart door lock will be identical from every manufacturer of smart door locks that joins the Matter consortium.  Matter also tests and certifies that devices adhere to the new standard.

This has huge potential for users of IoT. It will be possible to have one app on a smartphone that can communicate with all Matter-enabled devices in the home. This will make it easy and almost instantaneous to connect a new Matter device into your home network of devices. It also will make it easier to coordinate interactions between devices. For example, let’s say that you want your smart blinds to be lowered any time the inside temperature rises to some predetermined level. That can be made to work even if your smart thermostat and smart blinds equipment come from different vendors – commands will be unified across Matter devices, regardless of who made them. The implications for the farmer and the businesses are even more profound. They might finally be able to have a suite of interactive smart devices instead of disparate devices that can’t communicate with each other.

Interestingly, there were folks calling for this from the beginning of the IoT industry. But vendors all chose to take separate paths, and some competitors chose a different path so they wouldn’t be compatible with anything else. In the early days, manufacturers had a vision that people would buy a whole integrated suite of products from them – but the industry didn’t go in that direction. If this catches on, vendors that use Matter ought to have a major advantage within a few years of anybody that refuses to use the new standard.

The Increasing Cost of Building fiber

Diana GoovaErts recently cited Pascal Desroches, the CFO of AT&T, as saying that the cost of building fiber has increased. He said that increased costs are getting close to hitting the company’s goal of not spending more than $900 – $1,000 per new fiber passing.

Any time I see an ISP talking about fiber costs, my first question is what is included in the costs. Does AT&T’s number cover only the fiber on the street? Does it also include a fiber drop, customer electronics including Wii, and installation labor? AT&T operates a PON fiber network – does the cost include field splitters, cabinets and other such costs? We don’t have any context to judge AT&T’s number and that makes it impossible to compare to costs claimed by other ISPs.

To put the AT&T numbers into perspective, I work with ISPs that are building aerial fiber in county seats that hope to hold all-in costs to $2,000 per passing when building to everybody, but often go higher. That number includes all of the costs I listed above. But it also differs from AT&T because the higher number includes the cost of building to everybody in a community. We know that AT&T only builds to small pockets of customers, and it probably rarely builds to any parts of a city that are challenging or expensive.

The other big difference is that AT&T is mostly overlashing fiber onto its existing copper. That is a construction method that is not available to other overbuilders who have to pay for make-ready on poles. The only times when costs are low for other ISPs is when the poles are in great shape, with minimal make-ready work needed. AT&T’s low target number highlight two things – its advantage from being able to overlash, and a willingness to skip neighborhoods with higher costs.

AT&T’s low target price also highlights that AT&T is shooting for a higher margin goal than most overbuilders. There is a big difference in the short-term return between an ISP paying $1,000 and one paying $2,000 per passing. AT&T is clearly under pressure to make fiber profitable as quickly as possible. Interestingly, when looking out at a ten-year horizon, there is very little difference in the cash flow generated for the low or higher cost build. Most ISPs that overbuild fiber recognize that the business has relatively low-returns for the short run but eventually cranks a lot of cash flow.

The $1,000 top target of cost also tells us a lot about AT&T’s market plan. To stay under that number means being very careful about where the company builds. This explains why AT&T is building to small pockets of customers in its markets and not building to everybody. The low target cost number also tells us that there is very little buried fiber in AT&T’s plans.

To some degree, AT&T is following the model established fifteen years ago by Verizon FiOS. Local communities were incensed when the Verizon built some streets but not the ones a block away, or when Verizon built fiber in one subdivision but not the one immediately next door. I don’t recall Verizon in those days ever mentioning a target price for construction, but it was clear that it had a cost metric that was driving where the company decided to build.

Desorches also said that AT&T is only forging ahead because the company is seeing higher than expected customer penetration rates on fiber. That fact must be creating a chill in cable company board rooms. It explains why cable companies are moving as quickly as possible to boost broadband speeds through upgrades. Cable companies are hoping that matching the speeds on fiber will fend off fiber overbuilders. That’s going to be an interesting marketing challenge because it seems to me that a lot of the public now believes that fiber is superior to other broadband technologies.

Desroches said that AT&T is still holding to its goal to pass 30 million homes by the end of 2025. The company closed 2022 with 24 million passings and will need to pass 2 million new homes per year to meet that target.

It seems likely to me that inflation isn’t the only reason that AT&T’s costs are rising. I would guess that the company has already constructed to the locations with the lowest cost per passing and that the remaining 6 million passings  likely have higher costs than the places already built.

It’s going to be interesting to see what AT&T does when it hits 30 million passings. The company could do what Verizon did with FiOS and sit on the fiber portfolio and generate a lot of cash. It’s anybody’s guess if the company will roll any of those profits back into building more fiber.

AT&T announced recently that it is interested in pursuing some of the $42.5 billion BEAD grant funding to build in rural markets. I don’t foresee the company finding any grant opportunities where its cost for matching funds will be under its $1,000 target per passing. But I think all the big telcos are considering that a higher out-of-pocket cost for grant areas will be offset by the benefits of creating a virtual monopoly in those places.

AI and Telecom Jobs

I’ve seen a lot of articles recently predicting that artificial intelligence will bring about a massive upheaval in the U.S. job market. Such predictions are not new, but the recent introduction of ChatGPT and other language models has elicited a new round of predictions. We already know that software can displace people. In 2019, Wells Fargo predicted that efficient software would replace 200,000 jobs in the banking industry. Much of this has already come to pass as software has replaced a lot of bond traders and behind-the-scenes analysts at banks. The question I’ve been pondering today is how artificial software will impact the telecom industry.

This industry has seen major retooling over the years. My first industry job was as an RF technician, and almost every function I tackled in the early 70s has been replaced by software. There is great software today that can pop out a propagation study or quickly estimate the link budget for a wireless connection. Similar changes have happened across most jobs in the industry. Folks proficient in copper technologies have been nearly phased out. There is no longer an army of certified Cisco techs working in every network engineering office. Rooms full of draftspeople have been replaced by fiber network design software.

Many of the past changes to industry jobs are solely due to the introduction of new technologies, such as copper jobs being replaced by fiber jobs. But a lot of the changes to jobs are due to productivity software, where computers can figure things out faster and more accurately than people.

The web is currently full of predictions that the next wave of innovations will impact office workers much more than craft jobs. Outside of the telecom industry, there are some drastic predictions of big changes in the next five years. One of the most immediate jobs that will be under fire is coders. There will always be a place for the smart innovators that come up with unique software ideas, but folks who write the fill-in code or people that debug software are likely to be replaced by AI software that can do the same functions faster and more accurately.

There are predictions that call centers will be emptied out over the next decade when voice software becomes as good at answering customer questions as a live person. The same is true for jobs that deal with a lot of paperwork. Jobs like paralegals, insurance claims specialists, and anything else that means processing repetitive information can be replaced by AI software.

One of the direst predictions is that AI can replace a lot of the work done by high-proficiency experts. For example, the prediction is that medical diagnosis software will be faster and far more accurate than doctors at diagnosing and recommending treatment for diseases. In the telecom world, this might mean replacing jobs like network engineers since software can monitor and react to network issues in real-time. A lot of this has already happened, and it’s amazing how few people it takes today to operate a NOC or data center.

Not all of the predictions are dour. I read one prediction that AI would eliminate 12 million U.S. over the next decade. But these predictions don’t talk about the new jobs that will be created in a world with prevalent AI. I don’t know what those jobs will be, but they are bound to materialize.

Innovation from AI is likely to impact large corporations far sooner than small ones. It’s not hard to envision some of the giant ISPs fully automating the backoffice function to eliminate many customer service, accounting, and other office workers. Little companies are not going to easily duplicate this transition. Employees in smaller ISPs tend to wear many hats and usually don’t perform just one function. The cost for a small company to implement an AI solution might be a lot higher than the savings.

One consequence of improved efficiency for big ISPs might be that it will become easier to justify buying small ISPs and eliminating everybody except the field technicians.

Interestingly, there is one area where most of the predictions agree – that AI will not replace innovators and experts who see the big picture. Nobody believes that software is going to have any creative spark in the coming decades, and maybe never. But that raises an interesting question. How do we grow the next generation of experienced veterans in an industry where a lot of the functions are done by AI? All of the smartest people I know in the broadband industry have worn many different hats during their careers. It is the accumulated experience of working in many parts of the business that makes them an expert.

One thing is sure. ChatGPT and similar software is new, and we’re at the very beginning of the AI revolution. But if this new software meets only a fraction of the early claimed benefits, we’re going to see huge changes across the economy. Whatever is coming is going to be massively disruptive, and working in telecom or any other industry will never be the same.