Our Cellphone Data Usage

HTC-Incredible-S-SmartphoneLast week I looked at what we are downloading on home Internet connections. Today I am taking the same look at the latest statistics about what we download on our cellphones. These numbers represent downloading only through cellular connections and don’t include using cellphones on WiFi.

The first thing that strikes you about the numbers is how small the numbers are compared to landline data connections. We are literally using 100 times more data, on average, for a landline connection compared to the average cell phone. It’s obvious that people are still very cautious using their cellphone data. But the overall use of data on cellphones is rising and it is not going to take very many years of growth until the average person is bumping up against the normal 2 gigabit monthly cap on smaller cellphone plans. It’s clear in looking at these numbers that the cellular carriers have trained us not to use cellular data, but as I recently wrote, they are now trying to figure out ways to reverse that trend.

I compared the statistics from the first half of 2014 to the first half of 2012 just to see how things have changed. Consider the following chart that compares average cellular downloads in the US for the two periods.

Rank           1st Half 2012      Pct                1st Half 2014      Pct

1                   YouTube         27.2%             YouTube          17.6%

2                   HTTP               19.9%             Facebook         14.0%

3                   Facebook         8.7%              HTTP                 12.7%

4                   MPEG               7.2%              MPEG                 8.6%

5                   Pandora           5.4%              SSL                     6.5%

6                   SSL                    4.8%             Google Market   5.3%

7                   Google Market 3.5%             Pandora              5.2%

8                   Netflix               2.2%             Netflix                  5.1%

9                   Flash Video       1.7%             Instagram           3.5%

10                 Windows           1.7%             iTunes                  3.1%

  1. 3%             78.5%

To put this into perspective it’s also important to look at average Internet usage per cellular customer. Consider the following numbers:

.                                                        1st Half 2012   1st Half 2014     Pct Increase

Average Monthly Download              280 MB          404 MB               44%

Average Monthly Upload                     33 MB            69 MB              109%

Aggregate Monthly Usage                  313 MB          465 MB                50%

This shows that average cellphone download increased only 44% over a two year period. This is much slower than the 33% annual increase we see in landline data use. But uploads doubled over two years and maybe it’s all those selfies! Combining these two sets of statistics tells the real story. Following is the average download in megabits used by cellphone users for each of the top data applications in 2012 and 2014.

Rank                  1st Half 2012                         1st Half 2014

1                   YouTube         76.7 MB          YouTube            71.1 MB

2                   HTTP                55.7 MB          Facebook          56.6 MB

3                   Facebook         24.4 MB          HTTP                 51.3 MB

4                   MPEG               20.2 MB          MPEG                34.7 MB

5                   Pandora          15.1 MB           SSL                    26.3 MB

6                   SSL                   13.4 MB           Google Market 21.4 MB

7                   Google Market  9.8 MB           Pandora            21.0 MB

8                   Netflix                6.2 MB           Netflix                20.6 MB

9                   Flash Video        4.8 MB           Instagram          14.1 MB

10                 Windows            4.8 MB           iTunes                12.5 MB

If you look at the first table you would suppose that YouTube downloads are way down. But the second table shows YouTube is down only 7% over two years. There are a few uses of cellphones that are way up. Netflix is up 235% over two years. Facebook is up 132% over two years. And Google Market is up 118% over two years. The only thing other than YouTube that is slightly down over the last two years is HTTP, or web browsing.

If you trended this two years forward then Facebook will clearly become the predominant use on cellphones, followed by Netflix. YouTube and MPEG pictures would trend to become third and fourth. It’s obvious that video overall is growing on the cellphone faster than anything else, with social networking also a significant use.

It will be interesting to see what impact is felt over time as the wireless carriers push more data usage. Both AT&T and Verizon have been pushing bigger family plans in the attempt to get people off their WiFi and back onto their 4G networks. It’s pretty obvious that, on average, people are not using a lot more data on their cellphones and continue, on the whole to be cautious. This is not to say that there are not many people who use a lot of data or that there are not already a lot of people who exceed their monthly data plans. These statistics represent the nationwide averages. Cellular companies report that data sales are way up, while these numbers show that average usage is not. One has to think that perhaps people are buying more data than they actually use.

Who Will Own the Internet of Things?

Tribrid_CarYesterday’s blog talked about the current Internet that is falling under the control of a handful of large corporations – Apple, Amazon, Facebook, Google and Microsoft. This leads me to ask if the upcoming Internet of Things is also going to be owned by a handful of companies

This is not an idle question because it has become clear lately that you don’t necessarily own a connected device even though you might pay for it. As an example, there was recently an article in the New York Times that reported that a car company was able to disable cars for which the owners were late in making payments. The idea of Ford or General Motors still having access to the brains of your vehicle even after you buy it is unsettling. It’s even more unsettling to think access is in the hands of somebody at your local car dealer. Imagine them turning off your car when you are far away from home or when you have a car full of kids. But even far worse to me is that if somebody can turn off your car then somebody else can hack it

The car companies are able to do this because they maintain access to the root directory of your car’s computer system. Whether you financed the car with them or paid cash, they still maintain a backdoor that lets them get remotely into your car’s computer. They might use this backdoor to disable the vehicle as in this example or to download software upgrades. But the fact is, as long as they have that ability, then to some degree they still have some control over your car and you. You have to ask if you truly own your own car. As an aside, most people don’t realize that almost all cars today also contain a black box, much like the recorder in airplanes that records a lot of data about your car and your specific driving habits. It records data on how fast you drive or if you are wearing your seatbelt – and this data is available to the car companies

Perhaps the car is an extreme example because car is probably the most complicated device that you own. But it’s likely that every IoT device is going to have the same backdoor access to the root directory. This means that the company that made an IoT device is going to have a way to gain access. This means every smartphone, appliance, thermostat, door lock, burglar alarm and security camera can be controlled to some degree by somebody else. It makes you seriously ask the question if you entirely own any smart device

Over time it is likely that the IoT industry will consolidate and that there will be a handful of companies that control the vast majority of IoT devices just like the big five companies control a lot of the Internet. And it might even be the same companies. Certainly Apple, Google and Microsoft are all making a big play for the IoT

I’ve written before about the lack of security in a most IoT devices. My prediction is that it’s going to take a few spectacular failures and security breaches of IoT devices before the companies that make them pay real attention to security. But even should they tighten up every security breach, if Google or Apple maintains backdoor access to your devices, then they are not truly secure

I think that eventually there will be a market for devices that a buyer con control and that don’t keep backdoor access. It certainly would be possible to set up an IoT network that doesn’t communicate outside the home but where devices all report to a master controller within the home. But it’s going to take people asking for such devices to create the market for them

If people are happy to have Apple or Google spy on them in their homes then those companies will be glad to do it. One of the first things that crossed my mind when Google bought Nest was that Google was going to be able to start tracking a lot of behavior about people inside their homes. They will know when you wake and sleep and how you move around the home. That may not sound important to you, but every smart device you add to your house will report something else about you. With the way that the big companies mine big data, the more they know about you the better they can profile you and the easier it is for them to sell to you. I don’t really want Google to know my sleep habits and when I go to the bathroom. To be truthful, it sounds creepy.

A New Kind of Internet?

Hong-Kong-ProtestSomething very interesting happened over the last few weeks that provides a glimpse into a different future for the Internet. The students and others who were protesting in Hong Kong were able to set up a private network that bypassed the Chinese authorities using a blockchain. I will describe what that is below.

There are currently a lot of different people who don’t like the way the Internet is operated today. First, we have the NSA surveillance and everything that implies. Numerous countries around the world are in the process of setting up servers that will keep a lot of local data on their own servers so that it doesn’t leave the country. In effect we are looking at a world where each country may have its own Internet cloud and a firewall around their data

Probably even more intrusive, we have a few large companies controlling a large percentage of what happens on the web. Science Fiction writer and futurist Bruce Sterling calls these large companies the ‘Stacks’ and his current list includes Apple, Amazon, Google, Facebook and Microsoft. We know that these large companies each have their own agenda for tracking each of us, mostly for marketing purposes, but each of them also has cooperated to some degree with the NSA.

Finally, we have the possibility that in the US that the FCC is going to vote against net neutrality which will free the large companies and the ISPs to do what they wish. It may seem a bit nationalistic to think that what happens here is important to the world, but since the large companies that control the web today are all American, to some extent, as goes America so goes the web.

There are many who are disturbed by these trends and I have seen numerous articles asking if the Internet as we think of it is already dying. To counteract these trends we have seen numerous new browsers, email services and encryption programs introduced in the last year for people who are looking to opt out of the surveillance world.

And now we have Hong Kong which is behind the Great Firewall of China where the government micromanages Internet access. Had the Hong Kong protesters used any of the normal available services to communicate, such as email or the various social media sites in China they would have been quickly squashed. So instead they cleverly established a blockchain.

A blockchain is a software technology that was established as the basis for trading Bitcoins. A simplified explanation of a blockchain is that it’s a distributed consensus network that allows communications to be made securely and without any centralized authority. The Bitcoin world is entirely based upon trading currency, but the same technology can be used to exchange any other kind of communication such as emails, tweets, etc. Every transaction is encrypted including a unique encrypted code given to each user of the network. This means that even if intercepted by the Chines authorities, the blockchain communications were coded both in terms of content and identity of participants.

One of the interesting things about blockchain communications is the consensus required for it to work. Each user involved in the blockchain basically validates everybody else. If for some reason there is no communication on the blockchain, then after a set amount of time the whole chain collapses.

So the protesters in Hong Kong established a temporary encrypted peer-to-peer network that was impenetrable by the authorities. This let them communicate and coordinate their activities free from oversight or censorship. And when the protest ends the blockchain will collapse and disappear.

This concept could become the basis for establishing secure group communications in the future that falls outside of the NSA or large company tracking. It’s not hard to imagine those with similar interests of some sort being able to launch their own blockchain that would just look like indecipherable bits as it passed through any Internet hubs or monitoring points. Such networks need not be nefarious and a blockchain network could be used for any group like a college fraternity, a science fiction fan club, the fans of a sports team or band, or anything else. Such networks could spring up and disappear as needed and would only be available to those with some sort of in on how to join. But even those who are insiders in the network have no way to see what others are doing and they can only decode their own transactions.

It’s an interesting concept and is the first effective way that people are bypassing the surveillance world. The vast majority of people in the world have nothing to hide, but that doesn’t mean that they enjoy having large companies or governments track and record everything they do. Expect there to be numerous attempts to create alternatives to today’s Internet. And expect it to be a cat-and-mouse game where new strategies avoid surveillance for a while until cracked, but with new ideas strategies waiting to work next. We could be seeing the start here of a new Internet where people take back privacy by opting out of products offered by the mainstream companies.

Are We Ready for the New Digital Divide?

digital-divideYesterday I briefly discussed a few of the major predictions that have come out of a Pew Research survey of industry experts that ask what we’ll be seeing from broadband applications by 2025. They predicted such things as a major use of telepresence, greatly enhanced virtual reality and closer daily tie between us and our computers. Today I want to talk in more detail about one of the negative predictions where many of the experts predicted that we will see a new digital divide that will be more extreme than the current one.

The digital divide today is between people who have broadband and those that don’t. Those without broadband fall into a few categories – those that live in rural areas where broadband is not available, those who are too poor to afford broadband and those that don’t want it. I’ve talked about this before, but these experts are saying that the future digital divide will be more extreme because it will separate those who can participate in an all-digital world and those who cannot.

The future digital divide will matter because there are going to be essential services that require big bandwidth. Businesses without enough bandwidth will not be able to take part in telepresence, and this is going to cut them off from much of the world. Both their suppliers and customers are going to expect them to be able to communicate virtually. Homes are going to need big bandwidth for education, medical care and even shopping. Anybody without big bandwidth is going to be left out of the mainstream and will have to accept something less.

Those that have access to the bandwidth and the kinds of applications that are predicted for a decade from now will have a major advantage over those who do not have good enough broadband. This means people with big broadband will get the jobs, enjoy better health, be able to live in their homes to an older age and be better educated than those that don’t have big broadband. The gap today is not nearly this extreme, but with the future that the experts all foresee, broadband becomes a necessity and not something that is nice to have.

Big bandwidth services are going to require a landline broadband connection, be that fiber or an upgraded cable network. Wireless is going to have its place to keep you connected to the basic services while on the move, but telepresence, virtual reality and most IoT services are going to be landline-based.

It is almost certain that a lot more people will fall on the wrong side of the digital divide than today. Today there are tens of millions of households and businesses for which the broadband they have today will become totally inadequate in the future. Many of the technologies we use today that deliver okay bandwidth – DSL networks, older generation cable networks and WISP wireless networks – are not capable of delivering the kind of bandwidth that will be needed in the future. These technologies today can provide bandwidth speeds that most people find acceptable. But when we start using applications that are going to require speeds of a hundred megabits or maybe many hundreds of megabits, these technologies are all going to be inadequate.

The only two technologies that can deliver the kind of bandwidth needed in the future are fiber and updated cable networks. We all know that fiber is capable of incredible speeds and normally requires an upgrade in lasers and electronics to go faster. But there are upgrade paths for cable networks that ought to be able to provide gigabit speeds. The problem is that the cable network upgrades are complicated and costly. In many cases it’s not just electronics that needs to be changed for a cable network to go faster. It can mean building a lot more fiber into the cable network and sometimes even having to replace much of the coaxial cable. It means changing the cable headend, the settop boxes and the cable modems. It means almost a whole new network to get to gigabit speeds. But it can be done.

One has to realistically ask how many communities are going to get very fast, yet still affordable broadband. Certainly some of the major cities are getting gigabit fiber from Google and a handful of other providers. But even in those communities it looks like fiber isn’t going everywhere. Fiber is being put into neighborhoods willing to pay for the advanced services but it’s largely bypassing poorer neighborhoods and apartment buildings. In those same communities the cable companies are responding to fiber competition by upgrading speeds.

But what about all of the places that don’t get fiber over the next decade? Will the cable companies make the needed investments in smaller markets to get faster speeds? Much of small-town America that has broadband speeds today between 3 Mbps – 15 Mbps due to older technology and its not hard to bet that they are not going to upgraded.

One of the new industry buzzwords is that fiber is a utility, and is something that every community needs to be able to thrive. While this may be somewhat true today, within a decade fast data speeds will be essential for businesses to operate and for homes to partake in the services that come only with speed. The demands for faster broadband will become louder as more and more communities that have okay broadband today find that same broadband to be totally inadequate tomorrow.

What Does a Gigabit Get Us?

Alexander_Crystal_SeerPew Research did a survey of 1,464 industry experts and asked then what killer apps we can expect if the US is able to significantly increase customer bandwidth between now and 2025. About 86% of the experts thought that bandwidth would improve enough by then to provide a platform for supporting widespread new applications.

The question does not suppose that everybody will have a gigabit of download speed, although by then there will many homes and businesses with that much speed available. But one can also suppose that by then that there will be many people with download speeds of hundreds of megabits. The cable companies are on a path with DOCSIS 3.1 to be able to increase speeds significantly on their networks if they so choose. So the biggest chance for fast speeds for the masses is not having fiber built everywhere by 2025, but rather of having the cable companies stepping up over the next decade. Most experts are thinking that they will to some extent (and I agree).

There were a few applications that a number of the experts agreed would become prevalent if download speeds increase:

Telepresence. There was a feeling that telepresence will have come a long way over the next decade. We already see the beginning of this today. For example, Julian Assange from WikiLeaks recently appeared at a summit in Nantucket via hologram. That is the precursor for having routine meetings with people by hologram. This would not just be speakers at conferences (but it would make it easier to get more impressive speakers when they don’t have to travel). But it means having salesmen make calls by telepresence. It means having staff meeting and other business meetings by telepresence. This is going to have a huge impact on business and could represent huge cost savings by reducing travel and the wasted costs and hours due to travel.

But there is also going to be a huge market for residential telepresence. One of the most popular features today of an iPhone is Facetime that lets people easily see each other while they talk. And Skyping has become wildly popular. One can imagine that people will grab onto telepresence as soon as the associated hardware is affordable, as a way to spend time with family and friends.

The experts also think that telepresence will have a big impact on medicine and education. Telemedicine will have come a long way when if a patient can spend time in the ‘presence’ of a doctor. Telepresence also will be huge for shopping since you will be able to get 3D demos of products online. In fact, this might become the first most prominent use of the technology.

Virtual Reality. Somewhat related to telepresence will be greatly improved virtual reality. We have the start of this today with Oculus Rift, but over a decade, with more bandwidth and faster processors we can have improved virtual reality experiences that can be used for gaming or for blending the fantasy world with the real one. There was also news last week that Microsoft demonstrated a 3D hologram gaming platform they are calling GameAlive that brings something akin to a holodeck experience into your living room. Over a decade virtual reality is likely to move beyond the need for a special helmet and will instead move into our homes and businesses.

Imagine being in a gym room and playing a game of tennis or some other sport with a friend who is elsewhere or against an imaginary opponent. Imagine taking virtual tours of foreign tourist destinations or even of visiting imaginary places like other planets or fantasy worlds. It is likely that gaming and virtual reality will become so good that they will become nearly irresistible. So I guess if computers take all of our jobs at least we’ll have something fun to do.

Internet of Things. Within a decade the IoT will become a major factor in our daily lives and the interaction between people and machines will become more routine. We are already starting to see the beginning of this in that we spend a lot of our time connected to the web. But as we become more entwined with technology it means a big change in our daily lives. For example, experts all expect personal assistants like Siri to improve to the point where they become a constant part of our lives.

Just last week we saw IBM roll out their Watson supercomputer platform for the use in daily apps. That processing speed along with better conversational skills is quickly going to move the web and computer apps deeper into our lives. Many of the experts refer to this as a future of being ‘always-on”, where computers become such a routine part of life that we always are connected. Certainly wearables and other devices will make it easier to always have the web and your personal assistant with you.

Aside from the many benefits of the IoT which I won’t discuss here, the fact that computers will become omnipresent is perhaps the most important prediction about our future.

Not everything predicted by the experts was positive and tomorrow I am going to look at a few of those issues.

Mr. Watson . . . . come here.

Watson-supercomputer-635This week IBM cut the ribbon on a “Watson Client Experience Center” in New York City, where along with five other centers it will provide access to the Watson supercomputer. A few weeks ago IBM also announced the availability of what it calls Bluemix, a suite of several cognitive-based cloud services. Several of the articles I read about this announcement say that Watson is bringing artificial intelligence to the world. But it’s not. Watson is a pretty amazing computer system and can do a lot of great things, but the computer is still no smarter than your toaster. You may ask how I can say that since Watson was able to soundly beat the two best Jeopardy champs a few years ago.

Let’s look at how Watson works. First, Watson is a supercomputer, meaning that it has massive computational power and a fast input / output. Watson is configured as cluster of ninety IBM Power 750 servers each of which uses a 3.5 GHz POWER7 eight core processor, with four threads per core. In total, the system has 2,880 POWER7 processor cores and has 16 terabytes of RAM. Watson has a natural language interface meaning that it is designed to be queried by conversation, in the same manner as Apple’s Siri.

Watson uses a hypothesis generator. What this means is that when it is asked something, Watson searches its databases and compiles all of the answers that seem to answer the question posed to it. Through its sheer blazing computational speed Watson can search this entire database quickly. It then ranks the results according to the frequency that it encounters answers. For the Jeopardy challenge Watson was fed with multiple reference sources like encyclopedias, textbooks and all of Wikipedia.

Finally, Watson uses what IBM calls dynamic learning. This means that when Watson makes a mistake, which has to be often when working in something as imprecise as English, Watson can take feedback from the user when told that its answer is wrong. It stores this feedback and uses its ‘learning’ to influence the rankings when it next encounters the same question.

But under it all Watson is no smarter than your desktop computer because there is no actual intelligence in the system, artificial or otherwise. What Watson does to simulate intelligence is to present a friendly language interface and fast computational power to come up with answers to questions. But Watson is only as ‘smart’ as the databases underneath of it. For Jeopardy they did not allow Watson access to the Internet because the internet is full of incorrect facts. Watson has no way of distinguishing between what is true or not true, other than through feedback from users who correct its mistakes. But Watson would be like many of us and would fall for every Internet hoax that hits the web. For example, there was an Internet hoax earlier this year that said that Flo from the insurance commercials was killed, and if Watson was connected to the web it would believe such an untrue rumor based upon the sheer volume of claims made about the hoax.

This is not to say that Watson can’t do amazing things. Imagine Watson paired with Siri. Let’s face it, Siri is okay with driving directions but can quickly get flustered on almost anything else. With Watson’s database behind Siri it would become much more useful in a hurry. And even for driving directions Watson would help Siri be better. Siri is great at getting you between towns, but I’ve noticed that in crowded urban environments that Siri regularly wants you to pull into the wrong parking lot or driveway, and over time Watson would help Siri learn these little nuances of the map through user feedback.

Expect over the next few years to see a flood of new apps that do a better job of working through spoken interface. Already there are interesting new ventures that plan on incorporating Watson. For example, the founder of Travelocity wants to roll out a service called WayBlazer that will help you figure out things to do when you travel. The goal is to help you find activities that interest you rather than being steered to the normal tourist traps. A start-up called LifeLearn wants to build a tool to help veterinarians diagnose pet ailments better. A company called SparkCognition wants to offer a service to help security people spot security risks by having Watson ‘think like a security expert’. Expect all sorts of new programs and apps that take advantage of Watson’s language interface and the ability to quickly search databases.

This is a big breakthrough in that this is the first time that mass computational power will be brought into our daily lives through apps. Those apps are going to start doing things that we have always wanted computers to do. But let’s not forget how quickly computers are getting better. I reported last month on a company that expected to have a desktop supercomputer by 2017 that will be several magnitudes faster than Watson. Within a decade there will be computers everywhere with the power that Watson has today. And let’s also not forget that Watson is not smart and that there is zero cognition in the system. Watson doesn’t think, but rather just searches and compiles large databases quickly. That is incredibly useful and I will be glad to use Watson-based services – but this is not yet anything close to artificial intelligence.

What are We Downloading?

Sandvine has issued their latest report on the state of the Internet. Sandvine sells internet tools and is the leader in network policy control, selling products that help carriers with things like peer-to-peer caching and traffic redirection. As usual the Sandvine report is full of interesting facts.

I compared some of the statistics from the 2014 to the first half of 2012 just to see how things have changed. Consider the following chart that compares landline downloads in the US for the two periods.

Rank            1st Half 2012      Pct              1st Half 2014         Pct

1                   Netflix             32.9%            Netflix                34.2%

2                   YouTube         13.8%             YouTube            13.2%

3                   HTTP               12.1%             HTTP                   11.7%

4                   BitTorrent        6.3%              iTunes                  3.6%

5                   iTunes              3.8%              SSL                        3.4%

6                   Flash Video      2.6%              BitTorrent            3.4%

7                   MPEG               2.0%               MPEG                   2.9%

8                   RTMP               2.0%               Facebook              2.0%

9                   Hulu                 1.8%               Amazon Video     1.9%

10                 SSL                   1.6%               Hulu                      1.7%

To put this into perspective it’s also important to look at average Internet usage per Internet customer. Consider the following numbers for the two time period:

.                                                         1st Half 2012   1st Half 2014     Pct Increase

Average Monthly Download              26.3 GB              43.8 GB               67%

Average Monthly Upload                      5.7 GB                7.6 GB              33%

Aggregate Monthly Usage                   32.1 GB             51.4 GB               60%

This shows that average household downloads increased 67% over a two year period, or will stay on the historic pace to double every three years.

Combining these two sets of statistics tells the real story. Following is the average download in gigabits used by an average household for each of the top ten web uses in 2012 versus 2014.

Rank             1st Half 2012                       1st Half 2014

1                   Netflix             8.7 GB          Netflix             15.0 GB

2                   YouTube         3.6 GB          YouTube            5.8 GB

3                   HTTP               3.2 GB          HTTP                  5.1 GB

4                   BitTorrent       1.7 GB          iTunes               1.6 GB

5                   iTunes             1.0 GB          SSL                     1.5 GB

6                   Flash Video     0.7 GB          BitTorrent         1.5 GB

7                   MPEG               0.5 GB          MPEG                1.3 GB

8                   RTMP               0.5 GB          Facebook           0.9 GB

9                   Hulu                0.5 GB          Amazon Video   0.8 GB

10                 SSL                  0.4 GB          Hulu                    0.7 GB

If you considered only the percentage table above you might assume that Netflix has grown but that other web services are shrinking. Actually, most of the services in the top ten list have increased significantly from 2012 to 2014. Netflix download is up 73%. YouTube is up 59%. iTunes is up 58%. Hulu is up 58%. The commercial service that has dropped is BitTorrent which dropped 10% over the two years in download volume. And there are new players in the top ten now like Facebook and Amazon Video. What is quickly falling by the wayside is flash video which has fallen out of the top ten.

These statistics show that Americans are using more bandwidth across the board. It’s easy to believe the headlines and to think that the big increase in web traffic is all due to Netflix. Certainly they are the gorilla in the room, but we are using a lot more bandwidth for everything else we do like exchanging pictures (MPEG) and browsing the web (HTTP). The bandwidth we use for all of these functions is growing at about the same fast rate.

The other thing to consider is the sheer amount of increase. Our web usage has been doubling every three years and there is no reason to think this is going to slow down. This shows that in only a two year period that ISPs have had to handle a 67% increase in download volumes. One does not have to trend that out too many years into the future to realize that in a decade that the average home is going to be downloading nearly a terabit of data monthly. And long before that a lot of homes are going to be hitting up against the data caps that most large ISPs have in place today.

What Data Do You Store?

dumpsterI’ve read several blogs this year by Isaac Sacolik who has coined the phrase ‘dark data’ to describe anything retained by a company that is not a part of the day-to-day operations. In the old days this data would have consisted of the old paperwork that companies kept in file cabinets. But today, since everything is now electronic, companies can amass a mountain of emails, text messages, IMs, spreadsheets, word document, pictures and all other sorts of information.

I know in the old paper days that once a year I made all my employees take a few hours after New Year and toss out older paperwork. I refused to allow a proliferation of filing cabinets and we rarely kept anything more than a few years old. So each year I dealt with old data by tossing anything that was more than 3 – 4 years old.

But everything has changed today. It’s not unusual for companies to have computer systems or cloud systems that capture electronic information which might be kept forever if there isn’t a specific policy or plan to get rid of it. Storage is getting so cheap that it’s tempting to just keep everything.

But Mr. Sacolik points out that dark data can be a potential source of value to a company but is more likely to be a threat. Every day we read about companies getting hacked and records being copied and the big problem with dark data is that you probably don’t even know what it contains.

In the old days there was less risk of something really damaging being kept because the process of having to explicitly write and type memos meant that somebody had to go out of their way to put something damaging in writing. But it still happened. I remember being a witness in a lawsuit against one of the RBOCs where the normal legal discover process uncovered an internal memo where a VP said they wanted to put the plaintiff out of business. That smoking gun meant game over in the legal process.

But today, because everybody can write emails there is a much larger change that employees are saying things that you would not want to see the light of day. And it’s likely that unless you get into a lawsuit and you look at emails that you even know that damaging emails exist. The dark data you are keeping might detail illegal or unethical activities, embarrassing employee behavior or details about trade secrets that you would never want to see published.

While discover in a lawsuit could be damaging, the real danger comes from being hacked or from actions taken by a disgruntled employee. One would have to assume that somebody that hacks a company will have no compunction against using whatever they find for financial gain. There are likely many things in your email records that could either harm you or help a competitor.

The obvious answer to this issue is to have a policy to deal with data retention, and then to enforce it. That is not always as easy as it sounds because there are often many unofficial copies of dark data and you can’t assume that the copies on your corporate storage are the only copies of data that exist. Employees make copies of data on computers. And there are files stored with off-line storage devices and even on myriad thumbnail drives. The biggest problem with electronic data is that it’s so portable.

But there are steps you can take. Certainly there are tactics that can help protect you from being hacked. This starts with not putting really sensitive data onto public storage. But it also means taking basic precautions like always using encryption in both storing and transmitting data outside the company.

But perhaps the best precaution of all is to have a policy to get rid of old things. Every company has packrats who never want to get rid of anything. But if you believe there is value in your dark data one has to wonder why you aren’t mining that value. The chance are that you are going to be better off by getting rid of old data. There are restrictions of course. There are IRS rules for the retention of financial data and you may have contractual requirements, particularly if you work for government entities that require you to keep specific data. But the vast majority of the things you keep have little or no positive value and at least some of it can be damaging. So get out the electronic dumpster periodically and toss things away.

Is OTT Getting Closer?

Fatty_watching_himself_on_TVThe rumors were running around the industry last week that the FCC was considering an investigation to allow over-the-top video on the web by non-cable companies. FCC Chairman Wheeler dismissed this as something that is being discussed but that is not imminent. But it should be on the table along with a lot of other changes to the way we handle programming.

The rumor wasn’t very specific, as rumors tend to be. One thing that was mentioned was the possibility of requiring that networks owned by cable companies be made available to those who want to broadcast on the web. Specifically mentioned was Comcast which owns the NBC channels and AMC. So perhaps this rumor is tied into the possible merger of Comcast and Time Warner.

But the rumor also implied that the FCC is thinking of classifying OTT providers, on the web or using other medium as cable companies. This would give them the right to buy content and that is a big deal. It was lack of this classification that put Aereo out of business, and giving web-companies these rights would mean a proliferation of customer options for something other than the traditional cable packages.

There are currently several well-known attempts in the industry to break the cable monopoly. It seems to be a universal thought that once somebody is able to create a viable on-line alternative that the flood gates might open up and that all sorts of alternate programming would come to the market. One of the companies trying hard to come up with alternate programming is Dish networks. They own a lot of terrestrial spectrum and have said that they plan to bring an OTT line-up to major markets.

SONY is also working on an alternative OTT line-up and it was reported by Bloomberg News in September that SONY was getting close to a deal with Disney and Fox to go along with an arrangement they already have with Viacom. That would be enough channels to create a very attractive package for cord cutters and millennials who want to get away from the expensive cable packages.

To put this into perspective, the cable companies also want big changes. Consider the petition that Mediacom filed with the FCC last month that asked for an expedited rulemaking to consider various changes in programming rules. Mediacom said that the relationship between programmers and distributors is broken, and they are right. Back at the beginning of the cable industry the cable providers had most of the power since they could decide to carry or not carry any given programming. But over the years as some networks have become indispensable the power has shifted to the programmers.

In recent years the programmers have abused their power. They have increased rates significantly every year, much faster than the rate of inflation. And in doing so they are pushing the price of cable packages to the point where many households are considering alternatives. The programmers also have forced cable systems to take everything they offer if they want to get the better channels. Further, the more powerful programmers often force the cable companies to place a lot of their channels in the most expensive tiers. Recent programming contracts also have minimum penetration rules meaning that cable companies have to pay for a specified percentage of customers even if they don’t actually get those penetration rates.

A lot of this has come about due to industry consolidation and there are six programmers that now own 125 cable networks. And consolidation continues as the Comcast / Time Warner merger would bring more network channels to Comcast.

Mediacom asks for some very specific relief from the FCC. They want the ability to negotiate for channels on an a la carte basis so that they don’t have to take new channels offered or can opt out of the most expensive ones. They also ask for the ability to unbundle, meaning that they are not required to carry everything a programmer offers. Mediacom also asks that the programmers not be allowed to restrict access for putting programming on the Internet for their customers.

Mediacom says in their filing that they expect the programmers to fight vehemently against even opening such an investigation. They said that the programmers will argue that the FCC doesn’t have the authority to implement the changes that they are requesting. But the Mediacom filing lays forth why they think the FCC does have the authority.

At some point something has to give. Perhaps it will come through a generic FCC investigation implied by the rumors. Perhaps a crack will come as part of approving the Comcast / Time Warner merger. Or perhaps somebody like Dish Networks will put together an offering that attracts enough customers to threaten the big cable bundles. But one thing is certain. If nothing changes and cable prices keep soaring, at some point the public will vote with their pocketbooks and we will see a flood of cord cutters finding lower-cost alternatives. The programmers keep acting like that will never happen, but one doesn’t have to remember back very many years to a time when it became trendy to drop home phones and a huge chunk of the market dropped them over a few year period.

Is Service the End Game?

Filing_Cabinet_OverloadI have spent a lot of time lately reading and thinking about where technology is taking us. One of the most fascinating trends that is becoming apparent is that many aspects of our lives have already or will soon undergo a progression from physical to virtual to service. This phenomenon has been repeated so many times that I wonder if this is not a normal result of technology that we are only just starting to understand.

Let me give you one example (and there are hundreds of them). The music business underwent this transition rather quickly. For nearly a century the music business consisted of a physical medium – originally albums, then tapes and finally CDs. But in a rather short period of time that physical medium got subsumed by the virtual medium of locally-stored MP3s. Apple and a few others made a lot of money from people who downloaded songs rather than buy the physical medium. But now that virtual business is being decimated by streaming music services. So within a very short period of time the music business went from physical to virtual and now to service. And each of these changes was transformational. In both shifts the old industry got largely left behind and new players took over the bulk of the business. Like most such shifts, the changes weren’t all-encompassing and there are still some people buying physical medium and downloading songs. But almost everybody has accepted the new over the old.

You can see or will see this same kind of transformation in every industry where computers and software can perform a function once done in the physical world. Almost every aspect of our personal lives has already been transformed by computers. Remember the days when you had to carry cash or checks to shop? That has been replaced by the virtual transactions of credit cards. And we are within a decade where paying for things will become an automated service and your face or your handprint will suffice to identify you and pay for things.

This shift can happen to things that we think of as purely physical. Remember the day when many people could do basic maintenance on their car? It didn’t take too much knowledge to change spark plugs or check the timing. But as cars have become software driven they are now far beyond any backyard mechanic. That step of computerization is putting cars on a track to go from physical to service. It’s pretty obvious that Google and others will succeed in making self-driving cars, making driving virtual. And it’s not too hard to imagine soon after that when cars might become communal. Your tax dollars or your home-owners association dues will cover a fleet of self-driving cars that will be ready to take you anywhere, eliminating the need for car ownership and turning cars into a service.

The same thing can happen with people and jobs. Back in the 70s I worked for several large corporations including Southwestern Bell. In those days every group of 10 – 50 people had a group secretary who handled various clerical functions for the group. This would include typing memos, filing paperwork, answering the phones, etc. But over time most of those functions have gotten computerized and large companies have far fewer clerical positions than they did in the 1970s. The phone systems can now answer themselves. We all write our own emails and memos. For the most part we have gone paperless and you no longer see every hallway in a large business covered by filing cabinets. And we all have our smart phones to help with just about everything else. So we have taken clerical functions and made them virtual.

But that transition is not over. Everything I read about technology tells me that the move to replace people with computers is going to accelerate over the next few decades. Pretty much any job that does repetitive tasks – like accounts payable clerks, coffee baristas, shipping clerks, and maybe even truck drivers – are going to be replaced to some degree by software. And at that point these functions become a service. If you need an accounting function done you will just choose a program in the cloud to take care of it. You need something heavy delivered and a self-driving truck will appear at your door. Many jobs that require people today will eventually become services.

It’s human nature to think that these kinds of shifts are going to take a long time and that what I am talking about is some Star Trek future. But to the extent that computers and related technologies keep improving at the same pace that they historically have, then in twenty years computers and storage and processing speeds will be thousands of times faster and more powerful than the computers of today. That means everything we use today will be an obsolete relic. The iPhone 6 will be on display in a museum as an example of quaint technology, much the same way that we now think about the first Atari game console. This means that the devices we use daily will be as far advanced in capabilities over our smartphones as the smartphone is over the Atarti. That alone is mindboggling and it’s really hard to imagine what that is going to mean to our daily lives.