The Terabyte Household

I was just in a meeting the other day with a bunch of ISPs were talking about household downloads. Several said that they were now seeing monthly data usage exceed a terabyte, and those with Comcast were lamenting that this is causing them a lot of money.

I wrote a lot about Comcast data caps a few years ago when the company experimented with really low data caps of 300 gigabytes per month. At that time a lot of households complained that they were exceeding those caps. Comcast was arguing at the time to end net neutrality and I think this persuaded them to back off of the low caps, which they set to 1 terabyte.

Here we are only a few years later and a lot of households are bumping up against and exceeding that data cap. Comcast absolutely knew this was coming and they just pushed the ability to monetize data caps a few years into the future. As an ISP the company knows better than most that the household demand for total downloaded data has been doubling every three years or so. That kind of growth will push a huge number of households over a terabyte within a decade – with many already hitting it now.

Comcast tries to justify data caps by arguing fairness – the same argument they made a few years ago. They say that those that use the Internet the most ought to pay the most. Even if you can buy that argument the penalty for exceeding the data caps are excessive. Comcast doesn’t charge a household for the first two months they exceed a terabyte. After that they have two plans. They will automatically bill $10 for every extra 50 Gigabytes over the data cap – with total excess charges capped at $200 per month. Customers who expect to exceed the data cap can also agree to pay $50 extra every month to get unlimited usage.

Comcast goes on to explain away the terabyte cap by describing what it takes to exceed the cap, as follows:

  • Stream between 600 and 700 hours of HD video
  • Play online games for more than 12,000 hours
  • Stream more than 15,000 hours of music
  • Upload or download more than 60,000 hi-res photos

This explanation is simplistic for a number of reasons. First, full Netflix HD broadcast at 1080p streams at over 7 Mbps and uses roughly 2.5 GB per hour, meaning a terabyte will cover about 400 hours of full HD video. If you have a good broadband connection the chances are that you are watching a lot of 4K video today – it’s something that Netflix and Amazon Prime offer automatically. It only takes only about 180 hours of 4K video in a month to hit the terabyte data cap – a number that is not hard to imagine in a cord-cutting home. The chart also misses obvious large uses like downloading games – with download sizes over 40 GB for one game becoming common.

The Comcast charts also fail to recognize the hidden ways that we all burn through bandwidth today. It’s not untypical for the average household to have a 30% to 40% overhead on Internet usage. That comes from the network having to transmit data multiple times to complete a download request. This overhead is caused for a number of reasons. First are inefficiencies inherent in the open Internet. There are always packets lost on transit that much be sent multiple times. There are also delays caused by the ISP network, particularly networks that are undersized in neighborhoods and that hit capacity during the busy hours. The biggest cause of delays for most of us is in-home WiFi networks that creates a lot of collisions from competing signals.

There are also a lot of background use of the Internet today that surprises people. We now routinely use web storage to back up files. All of the software on our machines upgrade automatically. Many now use applications like video cameras and home alarms that connect in the cloud and that ping back and forth all day. All sorts of other things go on in the background that are a mystery – I’ve noticed my house has significant broadband usage even when we aren’t home. I’ve estimated that this background communication probably eats about 150 gigabytes per month at my house.

When I consider those issues the Comcast terabyte data caps are stingy. A household with a lot of network noise and with a lot of background traffic might hit the data caps using only half of a terabyte of downloaded video or other services like those listed by Comcast. A home today might hit the cap with 200 hours of full HD streaming or 90 hours of 4K streaming.

The other amazing aspect of the terabyte data caps is the charge for using more than a terabyte in a month. As mentioned above, Comcast charges $10 for every extra 50 GB. I’ve done the math for dozens of ISPs and most of my clients spend between $2 and $4 per month on average for the bandwidth per broadband customer. That number includes not only residential users, but for most ISPs also includes some huge commercial broadband customers. The average price varies the most according to how far an ISP is away from the Internet, and that component of the cost is fixed and doesn’t increase due to higher data volumes by the ISP. After backing out this fixed transport cost, my math says that an extra 50 GB of broadband costs an ISP only a few pennies. For a large ISP like Comcast that cost is significantly lower since they peer with the big broadband companies like Netflix, Google and Amazon – and traffic exchanged in those arrangements have nearly zero incremental cost of extra bandwidth.

Finally, the Comcast website claims that less than 1% of their users exceed the terabyte data caps. Only they know the numbers, but I find that hard to believe. When you look at the amount of usage needed to hit that cap there has to be a lot of cord-cutter households already exceeding a terabyte.

The bottom line is that Comcast is extorting homes when they force them to spend $50 per month for unlimited data usage. That extra bandwidth costs them almost nothing. Unfortunately, there isn’t a damned thing any of us can do about this any since Comcast and the other big ISPs got their wish and broadband is no longer regulated by the FCC.

Raising Data Caps

comcast-truck-cmcsa-cmcsk_largeBrace yourself, because I am about to say nice things about Comcast. Last week Comcast announced that it was raising its month data caps countrywide to 1 TB (terabyte). This is an increase from the current caps of 300 GB that the company has implemented in a number of markets starting last year. This is good news for me. My household easily exceeds the 300 GB data caps. It’s a relief to know that I am not going to be seeing the small data cap.

There are probably a few reasons why Comcast decided to raise the cap. First, the FCC just required that one of the conditions for Charter’s purchase of Time Warner is that they impose no data caps on customers for seven years. In making that statement the FCC said that they had serious concerns about ISP data caps if those same ISPs also owned video programming, like Time Warner. In such cases, the ISP imposing data caps is favoring their own content over Netflix, Amazon Prime and Hulu delivered over the Internet.

And of course, the ISP that owns the most content is Comcast. They not only own NBC and other TV networks, but they just announced last week that they are going to buy DreamWorks. And so the company probably raised the data caps voluntarily rather than have it imposed on them during any investigation of the DreamWorks purchase.

Comcast was also taking a lot of bashing about the data caps. Data cap complaints have soared to become the most common consumer issue at the FCC. People complained that Comcast wasn’t measuring their usage correctly and that the caps were penalizing them for watching online video rather than buying Comcast video.

I always found the numbers that Comcast quoted about data caps to be suspicious. When they imposed the 300 GB data caps they said that only 8% of their customers exceeded that cap each month. They said last week that 1% of their customers exceed the 1 TB limit. I always thought the 8% number sounded too small, and if the TB number is correct it probably is. It’s hard to think that any household that watches a significant amount of online video doesn’t hit the 300 GB cap.

In addition to video, anybody who downloads games and 4K movies are surely exceeding that cap. It’s not unusual for a game or 4K movie file to be between 40 GB and 60 GB, and it wouldn’t take long for files that large to blow the 300 GB data cap.

But what perplexes me is that if the FCC is generically against data caps, why did they just impose a cap on the new Lifeline data programs? They imposed a cap on any customer getting a landline data subsidy to a 150 GB monthly cap and imposed an unbelievably paltry cap on mobile data of ½ GB per month. I’ve been scratching my head since I read the order trying to figure out why there are any data caps at all on the Lifeline plan.

This is particularly perplexing since one of the major stated purposes of the Lifeline plan is to close the “homework gap.” From everything I read, a large part of homework these days is assigning videos for homework. Students watch schoolwork videos at home, saving valuable class time to then discuss the video. But having data caps on homework plans – or allowing mobile data to be used for this purpose – is puzzling.

There are still a few big players in the industry with data caps that the FCC is surely watching. Both Verizon and AT&T now have video products as part of their monthly service that don’t count against their mobile data caps. It’s hard to think that this is going to be allowed to stand. Mobile data in the USA is close to the most expensive data in the world and hopefully the FCC can find a way to get the wireless carriers to raise data caps in the same way that they are getting the big landline companies to do so. I think the FCC just missed a big chance by not requiring removal of data caps as a requirement to buy new spectrum.

People in the rest of the world are amazed at our data caps. For most of the world, if you have a mobile data plan you can use it pretty much as much as you want. Foreign cellular providers don’t make any promises that mobile data will always be available, but they expect customers to actually use it.  The fact that US cellular carriers impose incredibly stingy data caps is frustrating and I hope the FCC has the wireless carriers in their crosshairs.