Zayo announced the installation of an 800 Gbps fiber route between New York and New Jersey. This is a big deal for a number of reasons. In my blog, I regularly talk about how home and business bandwidth has continued to grow and is doubling roughly every three years. It’s easy to forget that the traffic on the Internet backbone is experiencing the same growth. The routes between major cities like Washington DC and New York City are carrying 8-10 times more traffic than a decade ago.

Ten years ago, we were already facing a backhaul crisis on some of the busiest fiber routes in the country. The fact that some routes continue to function is a testament to smart network engineers and technology upgrades like the one announced by Zayo.

There is not a lot of new fiber construction along major routes in places like the northeast since such construction is expensive. Over the last few years, a major new fiber route was installed along the Pennsylvania Turnpike as that road was rebuilt – but such major fiber construction efforts are somewhat rare. That means that we must somehow handle the growth of intercity traffic with existing fiber routes that are already fully subscribed.

You might think that we could increase fiber capacity along major fiber routes by upgrading the bandwidth capacity, as Zayo is doing on this one route. But that is not a realistic option in most cases. Backhaul fiber routes can best be described as a hodge-podge. Let’s suppose as an example that Verizon owns a fiber route between New York City and Washington DC. The company would use some of the fibers on that route for its own cellular and FiOS traffic. But over the years, Verizon will have leased lit or dark fibers to other carriers. It wouldn’t be surprising on a major intercity route to find dozens of such leased arrangements. Each one of those long-term arrangements comes with different contractual requirements. Lit routes might be at specific bandwidths. Verizon would have no way of knowing what those leasing dark fiber are carrying.

Trying to somehow upgrade a major fiber route is a huge puzzle, largely confounded by the existing contractual arrangements. Many of the customers using lit fiber will have a five 9’s guarantee of uptime (99.999%), so it’s incredibly challenging to take such a customer out of service, even for a short time, as part of migrating to a different fiber or a different set of electronics.

Some of the carriers on the major transport routes sell transport to smaller entities. This would be carriers like Zayo, Level 3, and XO (which is owned by Verizon). These wholesale carriers are where smaller carriers go to find transport on these existing busy routes. That’s why it’s a big deal when Zayo and similar carriers increase capacity.

I wrote about the first 400 Gbps fiber path in March 2020, implemented by AT&T between Dallas and Atlanta. Numerous carriers have started the upgrade to 400 Gbps transport, including Zayo, which has plans to have that capacity on 21 major routes by the end of 2022. The 800 Gbps route is unique in that Zayo is able to combine two 400-Gbps fiber signals into one fiber path using electronics from Ciena. Verizon had a trial of 800 Gbps last year using equipment from Infinera.

In most cases, the upgrades to 400 Gbps or 800 Gbps will replace routes lit at the older standard 100 Gbps transport. While that sounds like a big increase in capacity, in a world where network capacity is doubling every three years, these upgrades are not a whole lot more than band-aids.

At some point, we’re going to need a major upgrade to intercity transport routes. Interestingly, all of the federal grant funding floating around is aimed at rural last-mile fiber – an obviously important need. Many federal funding sources can’t be used to build or upgrade middle-mile. But at some point, somebody is going to have to make the needed investments. It does no good to upgrade last-mile capacity if the routes between towns and the Internet can’t handle the broadband demand. This is probably not a role for the federal government because the big carriers make a lot of money on long-haul transport. At some point, the biggest carriers need to get into a room and agree to open up the purses – for the benefit of them all.