The Terabyte Household

I was just in a meeting the other day with a bunch of ISPs were talking about household downloads. Several said that they were now seeing monthly data usage exceed a terabyte, and those with Comcast were lamenting that this is causing them a lot of money.

I wrote a lot about Comcast data caps a few years ago when the company experimented with really low data caps of 300 gigabytes per month. At that time a lot of households complained that they were exceeding those caps. Comcast was arguing at the time to end net neutrality and I think this persuaded them to back off of the low caps, which they set to 1 terabyte.

Here we are only a few years later and a lot of households are bumping up against and exceeding that data cap. Comcast absolutely knew this was coming and they just pushed the ability to monetize data caps a few years into the future. As an ISP the company knows better than most that the household demand for total downloaded data has been doubling every three years or so. That kind of growth will push a huge number of households over a terabyte within a decade – with many already hitting it now.

Comcast tries to justify data caps by arguing fairness – the same argument they made a few years ago. They say that those that use the Internet the most ought to pay the most. Even if you can buy that argument the penalty for exceeding the data caps are excessive. Comcast doesn’t charge a household for the first two months they exceed a terabyte. After that they have two plans. They will automatically bill $10 for every extra 50 Gigabytes over the data cap – with total excess charges capped at $200 per month. Customers who expect to exceed the data cap can also agree to pay $50 extra every month to get unlimited usage.

Comcast goes on to explain away the terabyte cap by describing what it takes to exceed the cap, as follows:

  • Stream between 600 and 700 hours of HD video
  • Play online games for more than 12,000 hours
  • Stream more than 15,000 hours of music
  • Upload or download more than 60,000 hi-res photos

This explanation is simplistic for a number of reasons. First, full Netflix HD broadcast at 1080p streams at over 7 Mbps and uses roughly 2.5 GB per hour, meaning a terabyte will cover about 400 hours of full HD video. If you have a good broadband connection the chances are that you are watching a lot of 4K video today – it’s something that Netflix and Amazon Prime offer automatically. It only takes only about 180 hours of 4K video in a month to hit the terabyte data cap – a number that is not hard to imagine in a cord-cutting home. The chart also misses obvious large uses like downloading games – with download sizes over 40 GB for one game becoming common.

The Comcast charts also fail to recognize the hidden ways that we all burn through bandwidth today. It’s not untypical for the average household to have a 30% to 40% overhead on Internet usage. That comes from the network having to transmit data multiple times to complete a download request. This overhead is caused for a number of reasons. First are inefficiencies inherent in the open Internet. There are always packets lost on transit that much be sent multiple times. There are also delays caused by the ISP network, particularly networks that are undersized in neighborhoods and that hit capacity during the busy hours. The biggest cause of delays for most of us is in-home WiFi networks that creates a lot of collisions from competing signals.

There are also a lot of background use of the Internet today that surprises people. We now routinely use web storage to back up files. All of the software on our machines upgrade automatically. Many now use applications like video cameras and home alarms that connect in the cloud and that ping back and forth all day. All sorts of other things go on in the background that are a mystery – I’ve noticed my house has significant broadband usage even when we aren’t home. I’ve estimated that this background communication probably eats about 150 gigabytes per month at my house.

When I consider those issues the Comcast terabyte data caps are stingy. A household with a lot of network noise and with a lot of background traffic might hit the data caps using only half of a terabyte of downloaded video or other services like those listed by Comcast. A home today might hit the cap with 200 hours of full HD streaming or 90 hours of 4K streaming.

The other amazing aspect of the terabyte data caps is the charge for using more than a terabyte in a month. As mentioned above, Comcast charges $10 for every extra 50 GB. I’ve done the math for dozens of ISPs and most of my clients spend between $2 and $4 per month on average for the bandwidth per broadband customer. That number includes not only residential users, but for most ISPs also includes some huge commercial broadband customers. The average price varies the most according to how far an ISP is away from the Internet, and that component of the cost is fixed and doesn’t increase due to higher data volumes by the ISP. After backing out this fixed transport cost, my math says that an extra 50 GB of broadband costs an ISP only a few pennies. For a large ISP like Comcast that cost is significantly lower since they peer with the big broadband companies like Netflix, Google and Amazon – and traffic exchanged in those arrangements have nearly zero incremental cost of extra bandwidth.

Finally, the Comcast website claims that less than 1% of their users exceed the terabyte data caps. Only they know the numbers, but I find that hard to believe. When you look at the amount of usage needed to hit that cap there has to be a lot of cord-cutter households already exceeding a terabyte.

The bottom line is that Comcast is extorting homes when they force them to spend $50 per month for unlimited data usage. That extra bandwidth costs them almost nothing. Unfortunately, there isn’t a damned thing any of us can do about this any since Comcast and the other big ISPs got their wish and broadband is no longer regulated by the FCC.

The Puzzling Lifeline Order

USACThe FCC has released more details of the revised Lifeline program order. It’s a long order and I won’t even try to summarize the order in this blog since the Internet will be full of such summaries in a few days.

Instead, I am going to highlight a few parts of the order that truly have me puzzled. The intent of the Lifeline order was to help to promote broadband adoption for low-income households. Unfortunately there are parts of the order that I think might accomplish the opposite of what is intended.

My primary beef with the plan (and it’s a huge one) is that the fund can be used to subsidize 3G cellular service. Not only that, but it will support cellular data plans with a monthly data cap of only 500 MB (half of 1 gigabit). This is mind-boggling to consider.

One of the stated purposes of the Lifeline plan is to help close the “homework gap” by providing data connections for school age children. What sort of homework gap does the FCC think it is closing with a 3G connection and a miniscule monthly data cap? The FCC is basically supporting a flip-phone data plan.

There has been a lot of recent press about how some broadband customers are now opting for mobile data over landline data, and I figure this has to mostly be to save money. The people who are choosing mobile data as their only option either aren’t big data users or else they have access during the day somebody’s WiFi on a landline data connection.

A few weeks ago I was in eastern Washington State at a hotel that had data speeds so slow that I couldn’t even open email. And so for two evenings I used my mobile data to connect my laptop. I didn’t watch any video and just conducted business, followed some election news and looked at Facebook a bit, and in two short evenings I used over 2.5 GB of data. It is impossible to use mobile data to do normal functions over the Internet.

And yet, somehow a family with school kids is supposed to be able to use a 3G mobile connection that has a data cap for the whole month of half of a gigabit? Have you ever tried opening a big web page on 3G? The FCC’s plan is beyond ludicrous. I’m picturing that AT&T and Verizon are either going to cut people off the Lifeline connection when they reach the tiny monthly cap or else they are going to nail the poorest households with data overage charges – and those households will end up spending more for mobile data than they do today.

I guess the FCC thinks the ½ gigabyte cap is too small and the cap will grow to 2 GB by the end of 2018. But even that will provide almost no real functionality for kids doing homework. I’m picturing kids watching assigned videos on their phone and using their monthly data cap on the first school day of the month. The FCC has caved in to special interests and has handed a huge revenue stream to the wireless carriers that is downright sickening. This one provision basically ruins the functionality of the Lifeline plan in my eyes because the wireless carriers are going to siphon off huge amounts of Lifeline fund for worthless data plans.

The other part of the plan that I dislike is the cap on wireline data. This requires that low income households be given connection speeds of at least 10 Mbps downstream and 1 Mbps upstream. This is not great (and not broadband according to the FCC) but it is good enough for the homework gap. Yet anybody getting this assistance can still be subjected to a monthly data cap of 150 GB.

And so, a household today that might already have a data plan with no cap is going to get a data cap slapped on their household due to taking advantage of a $10 per month subsidy from the FCC. Comcast just raised their data caps to 1 TB (terabit), something that I was very happy to see. But now the FCC comes along and imposes a much smaller data cap on Lifeline landline connections. Should a customer who is paying $40 today for a data connection be penalized that heavily because they accept a $10 subsidy on their broadband? This feels vindictive to me, as if the sentiment is “No on-line video for you poor people!”

I honestly don’t understand why the FCC would impose data caps on Lifeline plans, and particularly don’t understand why they would impose data caps that are more stringent than what the carriers already have in place today. Hopefully the carriers will ignore these caps and let customers have the same cap as anybody else with the same plan. But I fear otherwise, and that makes the practical application of the Lifeline order pretty rotten in my mind.

Comcast Trying Data Caps Again

comcast-truck-cmcsa-cmcsk_largeYet again Comcast is trying to introduce data caps. They have introduced what they are calling ‘data usage trials’ in Miami and the Florida Keys. For some reason most of their past trials for this have also been in the southeast. The new plan gives customers a monthly data cap of 300 gigabits of downloaded data. After you hit that cap then every additional 50 gigabits costs $10. For $30 extra you can get unlimited downloads.

When Comcast tried caps a few years ago they used a monthly cap of 250 gigabits. Since the average household has been doubling the amounts of data used every three years, the new cap is stingier than the old 250 GB cap since households would have normally almost doubled usage compared to the last time Comcast tried this. This means the 300 GB cap is going to affect a lot more people than the old cap.

What is probably most annoying about this is that Comcast is refusing this time to call these data caps. Instead they are calling this a ‘data usage trial’ and are trying hard to compare themselves to the plans sold by the cell phone companies. Of course, everybody in the world understands those cellular plans to be data caps.

It’s not hard to understand why Comcast wants to do this. While broadband subscriptions continue to grow, with the overall US market at an 83% broadband penetration there is not a lot of future upside in broadband sales. Further, I know that Comcast is eyeing the financial performance of the cellphone companies with envy since they can see the significant revenues generated by AT&T and Verizon with their data caps.

But Comcast also must understand that customers are absolutely going to hate these caps. Households are watching online video more and more and it is that usage that is driving the vast majority of downloads. There are other households that have big usage due to gaming, and some households that still engage in file-sharing, even though that is often illegal and riskier than it used to be.

The last time Comcast did this they saw a massive customer revolt and I certainly expect that to happen again. Take my case. I estimate that we probably use at least 500 GB per month. So for me this is basically means a $30 increase in my data rate. They have already pushed me to the edge of tolerance by forcing me to buy a basic TV package that I don’t use in order to get a 50 Mbps cable modem. If they introduce this cap they would push me over $100 per month just to get a broadband connection. At that point I start taking a very serious look at CenturyLink, the other provider in my neighborhood.

The biggest problem with any data caps is that, no matter where the cap is set, over time more and more customers are going to climb over it. We are just now starting to see the first proliferation of 4K video, and at download requirements of 18–22 Mbps this will blow past the data cap in no time.

What is most ridiculous about data caps either for cellular or landline data is that the US already has the most expensive Internet access of all of the developed countries. ISPs are already reaming us with ridiculously expensive broadband access and are now scheming for ways to make us pay more. The margins on US broadband are astronomical, in the 90% plus profit margin range. So data caps at a company like Comcast are purely greed driven, nothing else. There are zero network or performance issues that could justify penalizing customers who actually use the data they are paying for.

I am not entirely against data caps. For example, I have one client that has a 1 terabit cap on their basic data product and 2 terabits on their fastest product. They don’t use these caps to jack up customer prices, but instead use them as an opportunity to discuss usage with customers. For instance, they might convince somebody who is constantly over the 1 terabit cap to upgrade to a product with a higher cap. But mostly they use these caps as a way to force themselves to monitor customers. Their monitoring found a few customers who went over the cap because they were operating some kind of commercial retail server out of their home. Their terms of service prohibit operating a business service over a residential product and they upgraded such customers to a business product, which has no data cap.

If you want to get really annoyed, look at this Comcast blog which explains the new ‘data usage trials.’ It is frankly one of the worst cases of corporate doublespeak that I have read in a long time. You have to feel a bit sorry for the corporate communications people who had to write this drivel, but the ones to hate are their corporate bosses who are determined to make us all pay more for using data.