Tag Archives: supply chain

Easing of the Supply Chain

One of the biggest broadband issues in 2021 and 2022 was that ISPs suddenly found themselves unable to buy the materials and electronics needed to construct networks. Within six months after the onset of the pandemic, wait times for many fiber and electronics components had stretched to a year or longer. Before the pandemic, ISPs knew they could order materials a few months ahead of construction and have the material on-site when needed.

The Fiber Broadband Association recently announced a dramatic improvement in the supply chain for fiber networks. The FBA cited the following decreases in the waiting times for the delivery of fiber materials by comparing the average wait times for material deliveries in the summer of 2022 to March 2023.

  • The lead time for fiber cable dropped from 52-60 weeks to 4-10 weeks.
  • The delivery time for handholes decreased from 22-26 weeks to 8-14 weeks.
  • The wait times for fiber cabinets and splitters decreased from 10-20 weeks to 4-8 weeks.
  • Delivery times for fiber multiport terminals dropped from 20-35 weeks to 4-8 weeks.
  • Conduit backlogs decreased from 15-20 weeks to 3-7 weeks.
  • The wait time for home electronics and wiring decreased from 12-24 weeks to 4-10 weeks.

This is all good news for the many ISPs building fiber networks. Lead times have almost returned to pre-pandemic levels, and somebody building fiber no longer has to worry about being able to obtain the needed materials.

There are a lot of reasons for the supply chain improvements:

  • New Capacity. New factories have been built to manufacture fiber, conduit, and other components.
  • Re-Sourcing. Before the pandemic, a lot of fiber electronics were produced in the Wuhan province in China. When the pandemic largely shut down the province, manufacturers of electronics found other factories around the world ready to take on the manufacturing. This process took a few years, but China is no longer the only source of some key components.
  • Raw Materials. The pandemic also shut down the mining and shipping of rare metals and other raw materials that are key components of electronics. Those supply chains are now back and operating at pre-pandemic levels. There has also been a worldwide push by geologists to find new sources of ores, and there are new mines being opened around the world.
  • Logistics. During the pandemic, I saw daily stories about how the ports and the trucking industry in the U.S. were backed up and paralyzed. Even when materials were delivered from overseas, it could take months to get them through the ports. At the same time, the pandemic saw a lot of older truckers retire and created a trucker shortage. Logistics are getting back to normal, but I’ve noticed that the radio is still full of ads from trucking companies looking for drivers.
  • Reduced Demand. The largest ISPs in the county collectively trimmed their projected construction for 2023 by millions of passings. That alone frees up a lot of demand for materials that can be used elsewhere.

We may not be out of the woods completely for the supply chain. Future increases in demand from both the big ISPs and the billions that will be spent on grants are going to keep the pressure on manufacturers. But as long as the pandemic-related factors don’t return, this might mean slightly longer delivery times, but nothing like we saw during the pandemic. I recall getting calls from folks who were aghast that they were being given delivery times of over a year for fiber. It seems like those ugly times are behind us.

The Challenge of Accepting RDOF

I’ve been wondering lately if some of the RDOF reverse auction winners are having second thoughts about accepting the RDOF awards. It’s amazing how much the broadband world has changed since the end of that auction in December 2020.

It’s gotten more expensive to build fiber projects over the last year. The supply chain has played havoc with the costs of the raw components needed to build fiber networks. Many clients tell me that the cost of fiber components like conduit are collectively up by 40% or more over the last year. As somebody who has worked through several periods of inflation in the past, there is not a big likelihood that prices will return to the old levels even after the supply chain gets back to normal.

A bigger concern is the cost of labor. The explosion in the volume of fiber construction projects is almost too hard to grasp. The demand for construction crews is going to soon outstrip the number of experienced technicians. That means big challenges for finding and keeping construction crews. Shortages always lead to higher labor costs.

The federal government also layered on a new requirement that didn’t exist at the time of the auction. The Buy American Act now applies to infrastructure projects awarded with federal funds after November 18, 2021. These rules will apply to any RDOF winner that is approved by the FCC after that date. These rules don’t automatically add to the cost of building a fiber network, but they kill any thoughts of using lower-cost foreign fiber or components. The Act makes it clear that components like fiber and conduit must be 100% sourced to U.S. manufacturers. The new rules also make it seem unlikely that there will be many waivers allowed – these new rules have teeth.

The biggest kick in the teeth to an RDOF winner are the huge new grants are offering far more funding than anybody won in the RDOF auction. The giant BEADS grants can fund up to 75% of the cost of building a network for a rural project. Grants like ReConnect also have a 75% grant option. An RDOF winner that was unopposed in the auction got 60% of the FCC’s bid price – and that price was not set at the full cost of building a network but based upon some screwball federal cost models. An electric cooperative that won the RDOF auction could get a lot more funding from ReConnect grants or the upcoming BEAD grants – but nobody in the industry knew this at the time of the RDOF auction.

Another issue to consider is that RDOF winners might have missed out on the opportunity for matching state grants. While some states might make matching grants to go along with RDOF awards, many will not. That means the RDOF funding is all such winners will see.

What I’ve never figured out is why some RDOF winners bid the awards down to ridiculously low levels. There are places where bidders accepted RDOF awards under 10% of the expected cost of building a network – in some cases as low as 1%. In one county I’m familiar with, an RDOF bidder accepted less than 5% of the cost of building the network. This is a county with some of the easiest costs in the country to bury fiber. But this county is typical of rural areas and is sparsely populated, so the cost per passing is still high. Even considering the relatively low construction costs in the area, I couldn’t make a business case in this county for accepting less than 50% outside funding to make the project viable. I’m still scratching my head, wondering how this RDOF winner expects to make a business plan out of such a low award.

It’s not hard to imagine that some RDOF winners are having second thoughts. There are penalties for walking away from RDOF, but those penalties might be a lot smaller than the downside of being forced to build a rural network that will never generate enough revenue to cover the cost of construction. I was mystified by some of the winning RDOF bids in 2020 – and those bids make a lot less sense when viewed from 2022.

The Coming Year of Confusion

July 2020 Calendar

I’ve had a number of people ask me about how I think COVID-19 will impact the ISP industry over the next six months. It’s an interesting question to consider because there are both positive and negative trends that ISPs need to be concerned about. The chances are that these various trends will affect markets and ISPs differently – making it that much harder for an individual ISP to understand what they are going to see over the next six months. Following are some of the trends I think ISPs will need to deal with:

People Want Faster Broadband.  Many households came to the realization that their home broadband is inadequate when parents and students tried to work from home simultaneously. OpenVault reported that the number of households subscribing to gigabit service nearly doubled in the first quarter of this year. Clients are reporting an increased demand from first-time customers as well as customers wanting to upgrade to faster speeds.

Downturn in Small Businesses. Everything seems to indicate that a lot of small and medium businesses are not going to survive the pandemic. There have already been a number of businesses like restaurants and small retail stores that have gone under. The anchor stores at malls are failing right and left. There seems to be an expectation that travel-related businesses are going to take a long time to come back. Everything I read says that there is a coming crisis in the fall for business landlords when the finally digest that business tenants are either disappearing or will want to negotiate cheaper rent. That’s likely to have a secondary ripple effect as strip malls and other business landlords start declaring bankruptcy. Over time, new businesses will grow to fill many of the voids, but there has been a huge shift to shopping online that will likely not retreat to pre-COVID levels.

People Will Continue to Work from Home. Every day I read about businesses that say that working from home, at least part time, will become the new normal in many industries. The latest was a survey of law firms that said that a lot of lawyers are not going to return to the office full time when the pandemic is over. This is good news for ISPs that provide residential broadband, because people working from home are going to demand speeds and latency that will support their work. OpenVault just reported that as of the end of the first quarter of 2020 that the percentage of homes subscribing to gigabit broadband doubled over the last year and is now at 3.75% of all homes and growing rapidly. This is not such great news for ISPs that serve law offices.

The Big Unknown is the Impact of Unemployment. As businesses fail or downsize a lot of people are not going to be returning to their original job. ISPs are already reporting that people are ditching telecom products like landlines. The cord cutting in the last month of the first quarter of this year was record-setting. The big unknown will be the number of households that can no longer afford to buy landline broadband. Obviously, unemployment isn’t going to stay at the current 40 million people, but it’s not quickly going to return to pre-COVID levels. A secondary impact of a degraded economy will be a surge in bad debt as customers hang onto to home broadband as long as they can. We’re likely to see a big impact when the Keep America Connected pledge ends. If ISPs present a bill for multiple back months of billing we ought to see a lot of customers forced to default and cancel broadband.

The Pandemic is the Dagger That Will Finally Kill DSL. Homes that have an option of using DSL or something faster like cable broadband or fiber are going to be bailing on DSL in big numbers. Many people in towns have stuck with DSL because it is priced cheaper than cable broadband. However, for a lot of homes, the most important factor in broadband has become speed and performance.

The Rural Broadband Gap Will Keep Getting Headlines. COVID-19 made it clear to elected officials at all levels of government that the rural broadband gap is badly hurting the economy. Even if schools return to normal, businesses in rural areas are not going to have the same flexibility to send employees home, and unemployed people in rural area are not going to easily be able to accept at-home jobs. That’s going to keep a sizable slice of the economy from fully participating in any recovery. Almost everybody I talk to is hopeful that this might translate to increased grant money for rural broadband – but that’s no guarantee.

We’re Going to Have Unexpected Shortages in the Supply Chain. The best way to describe the supply chain right now is spotty. Manufacturers of telecom electronics are going to suddenly find they can’t buy one or two components, and manufacturing will come to unexpected halts. Anybody building a broadband network needs to expect delays, and if history is a good teacher, the delays will last longer than expected. This is going to play havoc with anybody that has financed a new network and needs to install customers to meet debt payments.

Banks Are Going to Tighten Lending. It’s inevitable that as banks digest bad loans from failing businesses that they are going to get more cautious about making new loans. Even if interest rates don’t rise, banks will do what they always do under stress and get more conservative. Some local banks are likely to get into real trouble and will fail if their portfolio was heavily invested  into businesses that are failing.

This all makes for an interesting short-term future. There will be more people yelling for faster broadband at the same time there will be more customers unable to afford broadband. There will be grants awarded for rural markets at a time when banks might not provide the matching funds. All in all, it’s going to be a mess for most ISPs who will see both good and bad things affecting them at the same time.

 

 

 

 

The Fragile Supply Chain

The recent outbreak of the coronavirus reminded us how fragile the supply chain is for telecom. As it turns out, the Hubei province of China is where much of the world’s optics and lasers are built that are the key component in every device that is used to communicate in a fiber network. Within days after the reality of the virus become apparent, the stocks of tech companies that rely on lasers took a hit.

The supply chain for electronics manufacturing stretches worldwide. The lasers are made in one place. The chips in devices are made somewhere else. Other electronic components come from a third geographic source. Components like cellphone screens and other non-electric components come from yet a different place. And the raw materials to make all of these devices come from markets all over the world.

The virus scare made the world wake up to the fragility of the supply chain. Without lasers, there would be no fiber-to-the-home devices manufactured. There would be no new servers in data centers. There would be no new small cell sites built or activated. Major industries could be brought to their knees within weeks.

It’s not hard to understand why I say the supply chain is fragile. Consider smartphones. There are probably a dozen components in a smartphone that must be delivered on time to a smartphone factory to keep the manufacturing process going. If any one of those components can’t be delivered, smartphone manufacturing comes to a halt. The manufacturing floor can be crippled by a lack of screens just as much as it can suffer if the chips, antennas, or other key electronic components become unavailable.

It’s impossible to know if the coronavirus will cause any major disruption in the supply chain for fiber optics – but the point is that it could. If it’s not a virus today, disruptions could come from a wide range of natural disasters and manmade problems. I remember a fire that destroyed a fiber optic cable factory a few decades ago that created a major shortfall of optic cables for a year. Floods, fires, earthquakes, and other disasters can knock out key manufacturing sites.

Manmade disruptions to the supply chain are even easier to imagine. We saw the price of electronics components shoot up over the last year due to tariff battles between the US and China. The supply chain can be quickly cut if the country making devices goes to war, or even undergoes an ugly regime change. It’s also now possible to weaponize the supply chain and threaten to cut off key components when negotiating other issues.

I’m sure that very few Americans realized that the Wuhan region has a near-monopoly on the manufacture of lasers. A worldwide economy rewards the creation of monopolies because components are cheapest when an industry takes the most advantage of the economy of scale. The companies in the Wuhan region can likely manufacture lasers cheaper than anybody else.

From a strategic position, countries like the US should foster their own industries to manufacture vital components. But that’s not easy or practical to achieve. A new US company trying to compete on the world stage by making lasers is likely to be more expensive and unable to compete when the supply chain is humming at normal capacity. It’s hard to picture creating a competitor to the Wuhan region that can manufacture lasers in the quantities, and at a price the market is willing to pay.

In the long run, the world always finds alternate solutions to any permanent changes in the supply chain. For example, if China is ever unable to export lasers, within a few years other countries would pick up the slack. But the fiber industry would be devastated during the lull needed to find a new source of components. Bank of America reported last year that 3,000 major manufacturing companies were already reconsidering their supply chain due to tariff and other concerns. Some of these companies, particularly electronics companies have been considering bringing production back to the US now that factories can be heavily robotized. I’m sure the coronavirus has accelerated these decisions.