Tag: Covid-19

Categories
What Customers Want

Being Stingy with Broadband Speeds

I’ve never understood ISPs that build fiber networks and then sell small-bandwidth products. The fiber technologies in place today can easily provide gigabit speeds to every customer without straining the network. The cost of providing 10-gigabit electronics keeps dropping and is now only a few hundred dollars extra per customer. Why would a fiber-based ISP have speed tiers that provide 50 Mbps?

I’ve found that it’s not unusual for an ISP with a low-bandwidth product on fiber to also charge a lot for gigabit bandwidth. There are a number of ISPs that charge $150 to $200 for a residential gigabit bandwidth product.

Bandwidth pricing philosophies differ around the industry. There are ISPs like Google Fiber, and Ting that only offer gigabit broadband. These ISPs are declaring that their fiber is a faster technology and they are marketing based upon that technology advantage.

The big ISPs in the country have trained the public that extra bandwidth is expensive. The cellular companies are the king at this game and will sell an extra gigabyte of usage for as much as $10. The big ISPs like Comcast and AT&T charge a lot for any customer that exceeds an arbitrary data cap. These pricing philosophies make a lot of money for the big ISPs, but this pricing conveys the false message that extra customer usage drives up costs for an ISP, and that customers that use more data ought to pay more.

Anybody who understands how ISPs operate realizes that there is little or no incremental cost for a given customer to use more bandwidth. It seems counterintuitive, but a household that uses a terabyte of download for a month doesn’t cost the ISP any more than the customer that uses 200 gigabytes per month – all due to the way that ISPs pay for wholesale broadband. ISPs buy enough bandwidth to satisfy the busiest hour of the day – and the rest of the day a lot of the bandwidth sits unused. There was zero incremental cost to ISPs for wholesale broadband when their customers started downloading and uploading a lot more data during the daytime due to COVID-19 – because the daytime usage still didn’t exceed the evening busy hour.

An argument can be made that faster speeds are more efficient for an ISP.  Consider the difference between two customers that each download a 1-gigabyte data file. A customer with a 50 Mbps product is using the network, and potentially interfering with other traffic for twenty times longer than the customer with a gigabit bandwidth product. Faster speeds reduce collisions between data streams, and a fiber network with fast customer speeds is significantly more efficient than one with slower speeds.

This is not to say that I’m advocating that ISPs should sell only the gigabit product since that is the most efficient use of a network. I think ISPs with only a gigabit product are leaving revenues on the table – unless the product is priced low enough to be affordable for everybody. I think companies that only a gigabit product at $70 or $80 are pricing out of the financial reach of many homes.

I get to peek behind the curtain of a lot of ISPs, and I know that an ISP with a smart tier of products can have more customers and more revenues than the ISP with only one product. I’m positive that an ISP with a $60, a $70, and an $80 product will do better than an ISP with only a $70 gigabit product.

The COVID-19 pandemic has finally forced the industry to confront broadband affordability. Even in markets where there is fast broadband available, we found out during the pandemic that there are a lot of homes without broadband for students because their parents can’t afford it. ISPs have not cared much about the homes that can’t afford broadband – and in most markets, that’s anywhere from 10% to 40% of the market. ISPs have been happy selling expensive broadband to those that can afford their prices and have given little thought to those that can’t.

I am truly puzzled why ISPs with fiber networks have broadband products between 25 Mbps and 75 Mbps. That’s like buying a race car and driving it on the freeway at 25 miles per hour. A cable company is not afraid of a competitor that wants to fight the market battle at speeds the cable company can match. The cable companies are afraid of ISPs of affordable symmetrical data products they can’t match.

Categories
The Industry

The Homework Gap is Not Just a Rural Problem

It’s been interesting watching the States react to the COVID-19 pandemic. There are headlines from all over the country about how States are taking emergency steps to bring some broadband to rural students using steps like beefing up external broadband at schools, creating temporary hotspots installed in school buses, and giving out computers and hot spots to rural students. These steps are all needed at a time when students are being asked to learn at homes in communities that have no broadband.

It’s easy to forget that there are many more students in our towns and cities without home broadband than rural households. Consider the list of the ten cities with the worst household broadband penetration rates, as compiled by NDIA – the National Digital Inclusion Alliance.

1          Brownsville, TX                     47.13%

2          Pharr, Texas                            46.24%

3          Rossier, LA                             45.30%

4          Lorain, OH                             37.29%

5          Compton, CA                         37.05%

6          Lynwood, CA                         36.19%

7          Newport News, VA                35.58%

8          Shreveport, LA                       35.49%

9          Flint, MI                                  35.16%

10        East Los Angeles, CA            33.06%

Right below this list are plenty of other cities with high numbers of home with no broadband. For example, sitting at number 20 on the list is Memphis, TN at 29.38%. Add to this the staggering statistic from the National Center for Homeless Education that over 1.5 million students are homeless in 2020 and are not counted in the above statistics – a statistic that’s likely to skyrocket due to high unemployment.

Solving the urban homework gap is often a huge challenge due to the sheer number of student homes with no broadband. Getting computers and hotspots for huge numbers or urban students can be an overwhelming problem that cities have been struggling with for years.

We know that lack of home broadband directly correlates to poor performance in school. Earlier this year, the Quello Center at Michigan State University released a definitive study that was able to isolate the lack of home broadband from other factors like poverty. That study showed that:

  • Students with home Internet access had an overall grade point average of 3.18 while students with no Internet access at home had a GPA of 2.81.
  • During the study, 64% of students with no home Internet access sometimes left homework undone compared to only 17% of students with a high-speed connection at home.
  • Students without home Internet access spend an average of 30 minutes longer doing homework each evening.
  • The study showed that students with no Internet at home often had no alternative access to broadband. 35% of students with no broadband also didn’t have a computer at home. 34% of students had no access to alternate sources of broadband such as a library, church, community center, or homes of a neighbor or relative.
  • The most important finding of the study was that there is a huge gap in digital skills for students without home broadband. To quote the study, “The gap in digital skills between students with no home access or cell phone only and those with fast or slow home Internet access is equivalent to the gap in digital skills between 8th and 11th-grade students.” It’s almost too hard to grasp that the average 11th-grade student without home broadband had the equivalent digital skills an 8th grader with home broadband.

A recent article talks about steps that Detroit has been taking to try to bring broadband to more students.

  • A local ISP, 123Net has been bringing gigabit access to southwest Detroit. A pilot project has brought Internet access to a few student homes and the goal is to find funding to expand the initiative.
  • There is a petition drive underway to create a ballot initiative that would ask the public to have the city address the digital divide as a basic city service.
  • The Public Lighting Department has been installing streetlights that come equipped with Internet connectivity. The hope is that over time that low-income neighborhoods can be blanketed with outdoor WiFi.

There are no easy answers for solving the urban homework gap because the issue is generally wrapped into the larger issue of addressing poverty. But as the Quello study points out, it’s vital that we try.

Categories
Current News

The Coming Year of Confusion

July 2020 Calendar

I’ve had a number of people ask me about how I think COVID-19 will impact the ISP industry over the next six months. It’s an interesting question to consider because there are both positive and negative trends that ISPs need to be concerned about. The chances are that these various trends will affect markets and ISPs differently – making it that much harder for an individual ISP to understand what they are going to see over the next six months. Following are some of the trends I think ISPs will need to deal with:

People Want Faster Broadband.  Many households came to the realization that their home broadband is inadequate when parents and students tried to work from home simultaneously. OpenVault reported that the number of households subscribing to gigabit service nearly doubled in the first quarter of this year. Clients are reporting an increased demand from first-time customers as well as customers wanting to upgrade to faster speeds.

Downturn in Small Businesses. Everything seems to indicate that a lot of small and medium businesses are not going to survive the pandemic. There have already been a number of businesses like restaurants and small retail stores that have gone under. The anchor stores at malls are failing right and left. There seems to be an expectation that travel-related businesses are going to take a long time to come back. Everything I read says that there is a coming crisis in the fall for business landlords when the finally digest that business tenants are either disappearing or will want to negotiate cheaper rent. That’s likely to have a secondary ripple effect as strip malls and other business landlords start declaring bankruptcy. Over time, new businesses will grow to fill many of the voids, but there has been a huge shift to shopping online that will likely not retreat to pre-COVID levels.

People Will Continue to Work from Home. Every day I read about businesses that say that working from home, at least part time, will become the new normal in many industries. The latest was a survey of law firms that said that a lot of lawyers are not going to return to the office full time when the pandemic is over. This is good news for ISPs that provide residential broadband, because people working from home are going to demand speeds and latency that will support their work. OpenVault just reported that as of the end of the first quarter of 2020 that the percentage of homes subscribing to gigabit broadband doubled over the last year and is now at 3.75% of all homes and growing rapidly. This is not such great news for ISPs that serve law offices.

The Big Unknown is the Impact of Unemployment. As businesses fail or downsize a lot of people are not going to be returning to their original job. ISPs are already reporting that people are ditching telecom products like landlines. The cord cutting in the last month of the first quarter of this year was record-setting. The big unknown will be the number of households that can no longer afford to buy landline broadband. Obviously, unemployment isn’t going to stay at the current 40 million people, but it’s not quickly going to return to pre-COVID levels. A secondary impact of a degraded economy will be a surge in bad debt as customers hang onto to home broadband as long as they can. We’re likely to see a big impact when the Keep America Connected pledge ends. If ISPs present a bill for multiple back months of billing we ought to see a lot of customers forced to default and cancel broadband.

The Pandemic is the Dagger That Will Finally Kill DSL. Homes that have an option of using DSL or something faster like cable broadband or fiber are going to be bailing on DSL in big numbers. Many people in towns have stuck with DSL because it is priced cheaper than cable broadband. However, for a lot of homes, the most important factor in broadband has become speed and performance.

The Rural Broadband Gap Will Keep Getting Headlines. COVID-19 made it clear to elected officials at all levels of government that the rural broadband gap is badly hurting the economy. Even if schools return to normal, businesses in rural areas are not going to have the same flexibility to send employees home, and unemployed people in rural area are not going to easily be able to accept at-home jobs. That’s going to keep a sizable slice of the economy from fully participating in any recovery. Almost everybody I talk to is hopeful that this might translate to increased grant money for rural broadband – but that’s no guarantee.

We’re Going to Have Unexpected Shortages in the Supply Chain. The best way to describe the supply chain right now is spotty. Manufacturers of telecom electronics are going to suddenly find they can’t buy one or two components, and manufacturing will come to unexpected halts. Anybody building a broadband network needs to expect delays, and if history is a good teacher, the delays will last longer than expected. This is going to play havoc with anybody that has financed a new network and needs to install customers to meet debt payments.

Banks Are Going to Tighten Lending. It’s inevitable that as banks digest bad loans from failing businesses that they are going to get more cautious about making new loans. Even if interest rates don’t rise, banks will do what they always do under stress and get more conservative. Some local banks are likely to get into real trouble and will fail if their portfolio was heavily invested  into businesses that are failing.

This all makes for an interesting short-term future. There will be more people yelling for faster broadband at the same time there will be more customers unable to afford broadband. There will be grants awarded for rural markets at a time when banks might not provide the matching funds. All in all, it’s going to be a mess for most ISPs who will see both good and bad things affecting them at the same time.

 

 

 

 

Categories
The Industry

Verizon’s Network Performance

Verizon has been posting a weekly report of how COVID-19 has been impacting their network. The weekly blogs are rather short on facts and it’s clear that the intent of this weekly report is to put investors at ease that the company’s networks are coping with the burst of traffic that has come as a result of the pandemic. With that said, the facts that are discussed are interesting.

Verizon lead off the weekly entry for 5/21 saying that voice and text traffic are starting to return to pre-COVID levels. On the most recent Monday Verizon saw 776 million voice calls, down from 860 million calls at the peak of COVID-19. That falls under the category of interesting fact, but heavier telephone call volumes are not the cause of undue stress on the Verizon network. Telephone calls use tiny amounts of broadband – 64 kbps. Thirty telephone calls will fit into the same-size data path as one Netflix stream. Additionally, once voice calls reach a Verizon hub, telephone calls are routed using a separate public switched telephone network PSTN to transport calls across the country. Text messages use much less data than a telephone call and are barely noticed on telco networks.

The bigger news is that some other traffic is staying at elevated levels. Verizon reported for 5/21 that gaming is still up 82% over pre-pandemic levels and VPN connections used to connect to school and work servers are up 72%. The use of collaborative tools like Zoom and Go-to-Meeting are up ten times over pre-COVID levels (1,000%).

One of the more interesting statistics is that network mobility (people driving or walking and switching between cell towers) has increased in recent weeks and that one-third of states now have higher levels of mobility than pre-COVID. At first that’s a little hard to believe until you realize that in pre-COVID time students and employees were largely stationary at the school or office much of the day – any roaming by stay-at-home people is an increase.

Reading back through the weekly statistics shows that most web activities are at higher levels than pre-COVID. Fir example, in the 4/22 report the volumes of downloading, gaming, video usage, VPNs, and overall web traffic were higher than normal, with the only decrease being the volumes used for social media.

What none of these reports talk about is the stress put on the Verizon networks. It’s easy in reading these reports to forget that Verizon wears many hats and operates many networks. They are still a regulated telco in the northeast and still have a lot of telephone customers. That also means they still operate a sizable DSL network. The company, through Verizon FiOS is still the largest fiber-to-the-home provider. The company also owns and extensive enterprise and long-haul fiber network. Verizon also operates one of the largest cellular networks in the world.

When Verizon says all is well, they can’t mean that for each of these networks. The web is full right now of complaints from DSL customers (Verizon’s and other big telcos) complaining how inadequate DSL is for working at home. The Verizon DSL network was already overstressed in evenings and has to be near the point of collapse due to the big increases in VPNs and collaboration connections. Any Verizon DSL customer reading this Verizon blog that says everything is fine is probably spitting fire.

By contrast, Verizon’s FiOS networks are likely handing the pandemic traffic with ease. Verizon FTTH products have offered symmetrical data for years, with the upload data path was lightly utilized. The big uptick in VPN connections and collaboration connections ought to be handled well in that network. Any glitches might come from older FiOS neighborhoods where the backhaul oaths out of neighborhoods are too small.

What Verizon or AT&T haven’t talked about is the different impact on their various networks. For example, what’s the overall change in data usage on their cellular networks compared to other networks? The big telcos have been moot on this kind of detail, because admitting that some of the networks are handing the pandemic well might lead to an admission that other parts of the company are not doing so well. Instead we get the very generic story of how everything is fine with the company and their networks.

These companies probably do not have any obligations to report about their various networks in detail. Verizon DSL customers don’t need company pronouncements to know that their broadband experience has nearly collapsed since the pandemic. FiOS customers are likely happy that their broadband has weathered the storm. One of these days I’ll hopefully have a beer with some Verizon engineer who can tell me what really happened – both good and bad – behind the scenes.

Categories
The Industry

Putting COVID-19 Traffic Growth into Perspective

Nokia Deepfield is another company that works in the background on the web and that analyzes data traffic patters for the big ISPs. Their June 4 report on web traffic reports about the same thing we’re hearing from most large ISPs – that the volume of web traffic suddenly shot up since the onset of the pandemic.

Nokia Deepfield says that the increase in traffic has settled in at about a 25% increase over pre-COVID levels. The most important aspect of the increase has been that almost all of the increases have been during the daytime, including on weekends. Networks have not seen any surge (or decrease) in the evening busy hour traffic.

To people who don’t follow the industry, those increases likely sound astronomical. Any other businesses would find a sudden 25% increase in business to be an extraordinary event. Imagine the impact of a sudden and sustained 25% increase in customer demand at a coffee shop, a bank, or a drug store. A business would have to scramble to increase inventory and staff to keep up with the new demand.

But in the world of ISPs this kind of growth is a lot less astounding. Cisco has been reporting for years that residential web traffic has been growing by 21% annually and business broadband by 24%. The ISP industry just absorbed in a single month the growth that would have normally been expected for all of 2020 – but any ISP worth their salt was already braced for this kind of growth this year.

It’s probably hard for the average person to digest that fact that the ISP industry has been coping with this kind of sustained growth for decades. If an ISP makes an expensive investment to double network capacity they’ll see the newly-created capacity filled within three or four years. ISP network engineers face a never-ending task of staying a step ahead of constant and relentless broadband growth.

It’s also worth noting that the growth due to COVID was less dramatic than the industry press might make you believe. Networks are engineered to satisfy the demands at the busy hour – those times of the day when networks are the busiest. During the rest of the day much of the network sits idle since the data pipes aren’t as full. The business hour for residential neighborhoods has been the evenings when homes watch video. Almost all of the growth from COVID came during the workday as students and employees worked from home. For most ISPs, the busy hour is still the evenings, and so there has been far less than a 25% increase in busy-hour demand. Most network should have been able to absorb this burst in growth.

This is not to say that all networks handled COVID growth well. For example, it’s been clear that the big telcos haven’t been investing money in their DSL networks for many years. Performance in those networks has been degrading every year as broadband usage increases. Customers in neighborhood with any significant number of DSL customers have seen broadband speeds decrease year after year as their the demand for broadband has increased. Anybody who has been working at home on DSL during the pandemic saw the network performance in the daytime nearly die.

The Nokia Deepfield blog introduces a new fact that I’d not heard before. They report that distributed denial of service (DDoS) attacks are up 50% during the COVID crisis. At first blush this seems counterintuitive because a lot of businesses have been shut down during the pandemic. Nokia Deepfield says the increased DDoS traffic comes from gamers. Apparently gamers can pay $30 to launch a custom 5-minute DDoS attack against an opponent. Anybody that has seen their neighborhood broadband become useless for five minutes might have been the unintended victim of such an attack. If we had an FCC that regulated broadband they might be investigating this kind of destructive web practice – but this is something they will leave to somebody else.

The bottom line on traffic growth is that. overall, most networks should have been prepared to absorb the growth in traffic due to the pandemic. Most of the growth happened during non-busy hours, and so, while the networks saw a lot of growth in traffic volumes they didn’t see an equal growth in network stress. The bad news for network engineers is that a lot of the recent growth looks like it will stick around, and the overall volumes of web traffic will probably continue to grow at 20% annually on top of the COVID growth.

Categories
The Industry

Home Broadband Usage Explodes

ISPs have all been reporting increased bandwidth usage due to employees and students being asked to work from home during the COVID-19 pandemic. Perhaps the best proof we’ve seen yet of the huge increase in home broadband usage comes from OpenVault, which has been tracking home broadband usage for several years.

The company reported that as of the end of March that the average US home used 402.5 gigabytes of usage, up 17% from the 344.0 gigabytes reported just 3 months earlier at the end of 2019, and up 47% from the 273.5 gigabytes measured a year earlier. OpenVault says that most of the growth was realized in the last two weeks of March as employees and students started working from home in earnest.

The OpenVault numbers represent total bandwidth used by a home, meaning the numbers are a combination of download and upload usage. OpenVault validated the widely reported phenomenon that the demand for upload bandwidth is increasing far more than the need for downloading.

Another interesting way to look at broadband usage is by considering the median usage – which is the speed at which half of homes use more and half less broadband. The median broadband usage is the US has always been lower than average usage because of the large number of rural homes that are stuck using slow broadband connections. A home with a 1 Mbps download speed cannot easily use the same amount of bandwidth as a home with a 100 Mbps connection. Median usage for the first quarter was at 233.6 gigabytes, up 60% from 146.0 gigabytes from a year earlier, and up 22% from the 190.7 gigabytes used at the end of 2019. The big news in the growth of median speeds is that even homes with slower broadband connections are burning through more broadband.

One of the most startling numbers to come from OpenVault is what they call power users – homes that are using more than 1 terabyte of data per month. At the end of March, 10% of all US homes were using a terabyte of data, an increase of 138% over the 4.2% of homes that used a terabyte of data just three months earlier at the end of 2019. Even more interesting, 1.2% of homes used 2 terabytes of data at the end up march, up 215% from the end of December. The big ISPs like Comcast are supposedly not billing for data caps during the pandemic – but they must be licking their chops at the flood of new revenues this is going to create if broadband usage doesn’t return to pre-COVID levels.

We saw the demand for faster broadband products also leap upward. At the end of March the percentage of homes subscribing to gigabit data products jumped to 3.75% of homes, up from 2.8% at the end of 2019 and up from 1.9% a year earlier. Amazingly, more than 1% of all homes in the US upgraded to a gigabit data plan in just the last three months – that’s something that’s been predicted for years. Those homes are not likely going to downgrade to slower speeds – so gigabit broadband is now becoming a significant segment of the market. OpenVault says that 12% of US homes now subscribe to speeds of 200 Mbps or faster.

The OpenVault data also validates what’s been reported widely by ISPs – that the patten of broadband usage is changing by time of day. In the recent past the peak period for broadband usage – the busy hour – was always in the evenings. In the first quarter the amount of usage in the evenings was flat and all of the increased usage came during the daytime as employees and students used broadband and video conferences to function.

OpenVault says that usage peaked in the third week of March. It will be interesting going forward to see the how home usage changes. OpenVault doesn’t have any better crystal ball than the rest of us, but they are predicting that broadband usage will never return to the historic patters. They predict that a lot of people will continue to work from home, meaning increased broadband demand during the day. They believe there will be continued pressure on the upload data paths. People who have learned to videoconference during the recent months are likely to continue that practice in the future. Companies and employees that realize they can be productive at home are likely to work more from home, even if only on a part-time basis.

Categories
Current News

The Upload Crisis

Carriers continue to report on the impact of COVID-19 on their networks. One of the more interesting statistics that caught my eye was when Comcast reported that upload traffic on their network was up 33% since March 1. Comcast joins the rest of big ISPs in saying that their networks are handling the increased traffic volumes.

By ‘handling’ the volumes they mean that their networks are not crashing and shutting down. But I think there is a whole lot more to these headlines than what they are telling the public.

I want to start with an anecdote. I was talking to a client who is working at home along with her husband and two teenagers. The two adults are trying to work from home and the two kids are supposed to be online keeping up with schoolwork. Each of them needs to create a VPN to connect to their office or school servers. They are also each supposed to be connecting to Zoom or other online services for various meetings, webinars, or classes.

These functions all rely on using the upload path to the Internet. The family found out early in the crisis that their broadband connection did not provide enough upload speed to create more than one VPN at a time or to join more than one video call. This has made their time working at home into a major hassle because they are being forced to schedule and take turns using the upload link. This is not working well for any of them since the family has to prioritize the most important connections while other family members miss out on expected calls or classes.

The family’s upload connection is a choke point in the network and is seriously limiting their ability to function during the stay-at-home crisis. But the story goes beyond that. We all recall times in the past when home Internet bogged down in the evenings when everybody in the neighborhood was using broadband to watch videos or play games. Such slowdowns occurred when the download data path into the neighborhood didn’t deliver enough bandwidth to satisfy everybody’s request for broadband. When that download path hit maximum usage, everybody in the neighborhood got a degraded broadband connection. When the download path got overloaded, the network responded by giving everybody a little less bandwidth than they were requesting – and that resulted in pixelating video or websites that lose a connection.

The same thing is now happening with the upload links, but the upload path is a lot more susceptible to overload.  For technologies like coaxial cable networks or telephone DSL the upload path leaving the neighborhood is a lot smaller than the download path into the area. As an example, the upload link on a coaxial network is set to be no more than 10% of the total bandwidth allowed for the neighborhood. It takes a lot more usage to overload the download path into the neighborhood since that path is so much larger. On the upload path, the homes are now competing for a much smaller data path.

Consider the difference in the way that homes use the download path compared to the new way we’re all using uploading. On the download side, networks get busy mostly due to streaming video. Services like Netflix stay ahead of demand by downloading content that will be viewed five minutes into the future. By doing so, the neighborhood download network can have cumulative delays of as much as five minutes before the video streams collapse and stop working. The very nature of streaming creates a buffer against failure – sort of a network insurance policy.

Homes are not using the upload links in the same way. Connecting to a school server, a work server, or a video chat service creates a virtual private network (VPN) connection. A VPN connection grabs and dedicates some minimum amount of bandwidth to the user even during times when the person might not be uploading anything. A VPN carves out a small dedicated path through the upload broadband connection provided by the ISP. There is no buffer like there is with downloading of streaming video – when the upload path gets full, there’s no room for anybody else to connect.

The nearest analogy to this situation harkens back to traditional landline telephone service. We all remember times, like after 911, when you couldn’t make a phone call because all of the circuits were busy. That’s what’s happening with the increased use of VPNs. Once the upload path from the neighborhood is full of VPNs, nobody else is going to be able to grab a VPN connection until somebody ‘hangs up’.

Residential customers have historically valued download speeds over upload speeds and ISPs have configured their networks accordingly. Many technologies allow an ISP to balance the upload and download traffic, and ISPs can help upload congestion by providing a little more bandwidth on the upload stream. Unfortunately for cable companies, the current DOCSIS standards don’t allow them to provide more than 10% of bandwidth on the upload side – so their ability to balance is limited.

As I keep hearing these stories from real users I am growing less and less impressed by the big ISPs saying that everything is well and that their networks are handling the increased load. I think there are millions of households struggling due to inadequate upload speeds. It’s true, as the big ISPs are reporting, that the networks are not crashing – but the networks are not providing the connections people want to make. No big ISP is going to admit this to their stockholders – but I bet a lot of those stockholders already understand this first-hand from having troubles trying to work from home.

Categories
Current News

Congress, Don’t Be Too Hasty

As Congress is handing out relief money for the COVID-19 crisis, rumors are flying around that rural broadband relief is one of the issues being discussed. The plight of employees and students unable to work from home has certainly bubbled up to a majority of those in Congress.

My advice to Congress is to not be to hasty. Don’t have the knee-jerk reaction of just tossing big bucks towards the rural broadband problem, because if you do much of the money will be wasted. There have been back-of-the-envelope estimates made that it would take anywhere from $60 billion to well over $100 billion to bring fiber to everywhere in rural America. Nobody knows the number and my guess is that it’s towards the upper end of that scale.

The typical Washington DC approach to the problem would be to earmark a pile of money to solve the problem, with no forethought of how to use the funds. This tendency is bolstered by the fiscal year spending nature of government funding, and Congress would likely expect broadband money to be spent quickly.

And that’s where the rub comes in. The broadband industry is not prepared to handle a sudden huge influx of funding. Consider all of the following issues that would quickly become apparent if this were to happen:

  • The first big question that would be asked with funding is where to spend the money – which parts of the country need the funding help? Unfortunately, the FCC will be of nearly zero help in this area, so you can’t run a giant grant program through them. The upcoming RDOF grants are supposedly aimed at bringing broadband to all of the places that don’t already have 25/3 broadband. But due to the dismal FCC mapping process, the current maps miss huge swaths of rural America that also need better broadband but that are misclassified by the FCC maps. If Congress gives the money to the FCC to disperse, the agency has no idea where to spend it according to its flawed data.
  • The next big question is how to award funds. The FCC’s RDOF grant program is using a reverse auction to award funds – but this only works when the funding is awarded to a specific footprint of grant areas. More traditional grant awards require the writing of extensive grant requests to prove the worthiness of a grant applicant and the worthiness of grant project. Anybody that remembers the Stimulus grants for broadband recalls that even at that time there were almost no qualified and experienced people available to review grant applications – and a lot of the Stimulus funding went to unworthy projects. A poorly run grant program also invites fraud and waste – the bigger the dollars the bigger the problems.
  • In perhaps the hardest issue for many to believe, there are not enough qualified ISPs ready and able to handle a big influx of funding and of operating the ensuing broadband businesses. We hear about small ISPs offering service all over the country, but all of them together don’t serve more than perhaps 5% of the broadband customers in the country today. Most small ISPs are already fully leveraged today as they’ve borrowed money to expand their footprint. Any grants that require matching funds might find a dearth of takers. If we throw money at the industry too quickly, it’s going to all end up going to the big telcos – and that likely just means pouring money down a black hole. It’s not hard to look back at the total bust of the CAF II program where the big telcos spent $11 billion in FCC funding and didn’t make any dent in the rural broadband problem. If Congress spreads awards out over time, then big new ISPs like electric cooperatives can prepare to go after the awards – most of them are not close to ‘shovel-ready’ today.
  • You can’t ask for broadband funding without some sort of engineering estimate of the cost of building a network and some sort of business plan showing that the network can operate profitably at the end of the funding. There are not a lot of ‘shovel-ready’ broadband projects laying around waiting for funding, and so the first step after a big Congress funding program would be to develop hundreds of new business plans. All of the consultants and engineers I know are already full-time busy helping companies to prepare for the $16.4 billion RDOF grants and the various state grant awards around the country.
  • The same is true of fiber construction companies. During this last construction season, we started seeing construction companies bidding up rates to go to the builder willing to pay the most for their services. There are not a lot (if any) idle fiber construction crews sitting around waiting for work. Fiber construction is not something that can be taught quickly to new workers – it takes years to develop a good fiber splicer or to train somebody to be able to determine pole make-ready.
  • We’re also starting to see backlogs for fiber materials. The waiting times for ADSS fiber that goes into the power space recently crept up to six months. The far bigger concern is electronics. Right now, the world supply chains are a mess due to COVID-19 and the industry is expecting delays in electronics delivery in the coming construction season. Supply houses and vendors aren’t talking about this, hoping it will magically go away, but there will likely be electronics shortages in the 2020 construction season even without pressure from new grants. Such shortages can cripple construction projects.
  • Finally, I am positive that any federal broadband grant money will come with stupid rules, many slapped on the funding by the big ISP lobbyists. There will be needless hoops to jump through and rules that make it hard to spend the money well. There is zero chance that federal grant funding wouldn’t come with ridiculous rules and ridiculous restrictions. If Congress is going to award big money they need to take a little time that the rules are fair and efficient.

There will be people reading this in amazement and wondering how a rural broadband advocate could be recommending caution. One only has to look back to the stimulus grants to recall that probably half of that money was wasted due to the haste of the grant programs. My fear is a knee-jerk federal reaction that will throw giant bucks at the problem when the proper solution would be a series of grants awarded over five or more years to allow ISPs time to get ready. Funding in one giant lump would result in a mess of epic proportions. I fear that DC would then wash their hands of rural broadband by saying that they already funded it, and any communities left behind after a flawed grant program would likely be left behind for decades to come. Congress, if you want to help your constituents, please ask for advice and get it right.

Categories
Current News

Predictions for a Post-COVID-19 World

While it might still be too early to make predictions, there are dozens of articles on the web predicting how the COVID-19 pandemic might change our long-term behavior. Here are some of the more interesting predictions I’ve seen that involve broadband and telecom:

An Outcry for Better Home Broadband. Millions of people were sent home for work or school to homes that didn’t have good broadband. These folks have been telling the world for years that they don’t have good broadband. When this crisis is over these people are going to insist on being heard, and they are going to take out their anger on politicians who don’t help to find broadband solutions. This means Mayors and City Councils that are not pro-broadband. This means County Boards and Commissions that don’t offer matching grants to attract ISPs. This means any state politician who votes against significant state broadband grants or who votes against municipal participation in broadband. And this means federal Senators and Representatives that support the big cable companies and telcos over their constituents. Folks are not likely to be fooled any longer by false legislation that supposedly is pro-broadband but which is the exact opposite – because folks are going to be paying attention to any news concerning their home broadband.

Digital Meetings Are Here to Stay. We are all seeing how effective it can be to meet online. People are going to be a lot less willing to travel for a one or two-hour meeting. I know my days of doing that kind of traveling are over. This means airline business travel is likely never coming back to former levels, but it means a lot more. I was talking to somebody in local government the other day who told me that they spend over 10 hours of every workweek driving between meetings around a large county. He said he thinks the day or required live attendance at such meetings is likely over.

Demand for Faster Upload Speeds. The permanent uptick in more video meetings means there will be an increased demand for faster upload broadband speeds. The FCC still talks about 25/3 Mbps as acceptable broadband, but a home or office getting only 3 Mbps upload is not able to hold multiple simultaneous video calls. Homes and businesses are going to favor technologies willing to meet that upload speed demand.

Telemedicine has Arrived. I have been watching the glacial acceptance of telemedicine for fifteen years. The biggest hurdles have been the reluctance of doctors to try telemedicine and the willingness of insurance companies to pay for it. We’ve broken both of those barriers and telemedicine is here to stay. There are numerous routine doctor visits that don’t require an office visit. It’s never made sense to force patients who aren’t sick to march through a waiting room that has been filled all day with those with colds, the flu, or worse.

Expect Contactless Payments. I can remember being promised twenty years ago that we’d be able to pay for things by waving a cellphone. Nobody wants to hand a credit card to a clerk or even pass a credit card through a device that other people have used all day – so stores that install touchless payment systems are quickly going to become preferred. Expect an expansion of telephone, voice, and vision interface at checkout locations and a phase-out of credit card swiping. Also, expect an increased reluctance to take cash. There were already stores in New York City last year that made headlines by refusing to accept cash – expect a lot more of that.

More Telecommuting. Businesses have seen that people can be effective when working from home. Expect to see businesses more easily allowing for working from home at least part-time. This likely means a downturn in business real estate. For example, my neighbor is an architect who works at a small local branch of a larger firm. They’ve already seen the effectiveness of working from home and have already discussed not reopening the local office when the crisis is over. More telecommuting means more daytime use of neighborhood bandwidth and an increased expectation of residential broadband signal quality.

A Reboot for Corporate Security. We just spent a decade moving corporate data behind firewalls and restricting access to data from outside the business. Many businesses scrambled to find ways to allow employees to work from home, and in doing so undid many of their security protocols. Expect a major reboot as companies implement security solutions that support telecommuting.

Categories
Current News Regulation - What is it Good For?

The FCC Releases Needed Spectrum

The FCC made two moves in the last week concerning spectrum. Chairman Ajit Pai announced intentions to vote later this month to release the entire 1,200 MHz band of 6 GHz spectrum for unlicensed usage. They also awarded special temporary authority for 33 WISPs to use 45 MHz out of the 5.9 GHz band to boost rural fixed broadband during the COVID-19 crisis.

It’s expected that the recommendation for the 6 GHz spectrum will be approved unanimously by FCC Commissioners. This announcement is huge news. This would increase the bandwidth available for WiFi by almost a factor of 5. The WiFi band already carries far more data than any other swath of spectrum and this bolsters WiFi for the next few decades. The order proposes to uses for the new spectrum. The entire 1,200 MHz of frequency would be released for indoor usage at low power. 850 MHz of the band would be released at standard power levels and can be used outdoors in hot spots and for point-to-multipoint fixed wireless networks.

The cellular carriers have been lobbying hard to have some of the bandwidth sold as licensed spectrum. Instead, the FCC order would allocate it all to public use, but allows anybody, including the cellular carriers to use the spectrum subject to automated frequency coordination. That’s the system that senses existing use of the spectrum before allowing a second interfering use. The cellular carriers might elect to use this spectrum heavily, on an as-needed basis, in urban areas, but likely won’t bother in rural areas – freeing this bandwidth mostly for rural broadband usage.

This is big news because until this announcement there was still the possibility that some of the spectrum would be allocated to a licensed auction. The Chairman did say that he was considering making this all public spectrum a year ago, but a decision was never official until now. This is big news for the whole WiFi industry as well, since any spectrum allocated to licensed spectrum would have been off-limits for indoor WiFi use. As I’ve written in other blogs, this new spectrum, along with the introduction of WiFi 6 technology means a massive upgrade in capability for home and office WiFi performance. This should enable multiple simultaneous large-bandwidth uses of bandwidth within the home or office without interference. WiFi 6 also uses techniques that cut down on interference from neighboring hotspots.

The second action by the FCC is interesting. They granted special temporary authority to 33 rural WISPs to use 45 MHz of the 5.9 GHz spectrum for the next 60 days. This will allow these WISPs to beef up rural bandwidth during the COVID-19 crisis. The WISPs report that they are seeing a 35% increase in traffic volumes along with a requests for more bandwidth due to students and employees suddenly working from homes.

The extra bandwidth will allow these ISPs to boost bandwidth since they use software-defined radios that already work in the nearby 5 GHz WiFi spectrum band. I would expect the FCC to continue the temporary use of the spectrum if shelter-in-place extends in some places past the 60-day window.

These temporary uses of the spectrum might presage a more permanent use of this spectrum band. The 5.9 GHz spectrum was set aside many years ago for vehicle-to-vehicle communications. The self-driving and assisted driving vehicle technology has advanced much more slowly than originally anticipated, plus some car manufacturers are using a different spectrum solution for communicating from car to car. The FCC was already considering splitting the spectrum band and cutting the amount of spectrum available to vehicles in half, with the rest likely going to public auction. The cellular carriers claim that they still only have half of the mid-range spectrum they need to support full deployment of 5G, and the FCC seems likely to grab this spectrum for that purpose.

Exit mobile version