The Ethics of Big Data

SpyVsSpyBack in 2010 Eric Schmidt, the CEO of Google at the time said something really frames the issues associated with big data. He said, “One day we had a conversation where we figured we could use [use Google’s data] to predict the stock market. And then we decided it was illegal. So we stopped doing that.”

Google doesn’t say this, but when you look at all of their data-gathering efforts it is clear that their ultimate goal is to create a database in of all human knowledge. They are far away from that today, but they have already amassed the largest database in history of the human race. Google is not the only one tackling this task. For example, Facebook has mapped out the social connections between more than a billion people. Wikipedia has undertaken something much smaller but has accumulated over 4.5 million factual articles in English. The nerd’s favorite web site WolframAlpha has accumulated an amazing number of facts about the world and can display them in easy-to-understand presentations. Both Google Maps and the OpenStreetMap database are both trying to create a database of our physical world.

Google has the scariest data about each of us because they know what we are thinking about and looking for from the Google search engine. And when they pair this up with other web data that identifies each of us they know what we are doing individually, but also what we are doing collectively. It’s said that Google now has the ability to predict many things about you due to the profile they have built on you. As an example, they know who’s an insomniac and what behavior insomniacs engage in, and so they probably understand insomniacs better, at a macro level than anybody else in the world.

This is not to say that Google is analyzing their data in that specific way, but they could be. And certainly they are making their data available for sale to other large companies who do want to know that kind of thing about us. Perhaps there is not yet a company who wants to market to insomniacs (but there probably is), but there are certainly companies who want to pinpoint their marketing to the most likely people to respond.

If you don’t think that big data companies are watching you, spend fifteen minutes looking at new cars on the Internet and then watch how many times new car ads pop up in your web experience in the next week. At the marketing level big data is already manifesting itself. But marketing is only the beginning, but the one that is making Google so wealthy today.

One can only begin to imagine the possibilities for Google and others to profit from the data they are going to be gathering from the upcoming Internet of Things. That data will include a lot of detail about our personal lives, from such mundane things like when we turn lights on and off to very personal things they will gather from medical monitors. And it’s all relevant and tells them a little more about us and lets them categorize you. Because in the end they want to profile you in great detail so that they can sell your data to those who are most interested in people just like you.

The question about whether this is good or bad is going to depend upon how they use this data. If they will sell this data to anybody willing to pay the price, then it’s bad, because not everybody is going to do good things with the data. There is already talk of companies using big data to prey upon the most vulnerable among us. It’s a well-known fact that poorest among us spend the most money on mundane things like cashing checks or getting a car loan, and with big data companies can pinpoint advertising to the most vulnerable of us. It’s certainly also possible for big data to be sold to companies that will use it overtly to do us harm. For instance, it’s not hard to envision a group of private investigators using personal data about us in all sorts of unsavory ways.

And probably most scary to me is if the government or the press has access to this data. I’ve heard the old axiom that nobody’s life can survive total scrutiny and that we all have things we would like to keep private. If the government and the press have access to big data, everybody can be made to seem guilty of something. This is the premise of ‘1984’ and many other science fiction books. We are getting very close to the day when that is no longer fiction.

One Hundred Blog Entries

This blog is an introspective look at the blogging experience. Last Friday I posted my 100th blog post. I have written one every work day and they have been posted every day except once when my wife just forgot to post it.

The process of writing a blog every day is a really good exercise for somebody like me. We have so many different clients at CCG and we get pulled into so many different business situations with different issues that I find that I know a little bit about almost everything in the telecom industry. Some things I know a whole lot about, but on many topics I feel I am a thousand miles wide and an inch deep, meaning I know the basics about a topic, but there is much more that I can learn.

So writing this blog has been a learning opportunity for me. I have read books, read thousands of web articles, scoured Wikipedia and talked to people who I consider to be experts. When I write about most of the topics I end up knowing more than when I started, and for a consultant that is not a bad thing.

Not all of my blog posts are factual and some are just my opinion. When I first started the blog I tried to avoid opinion pieces, but then I came to realize that I have strong opinions about some of the things going on in the industry and I decided to let my opinions out of the box. This has also been a learning process for me, because before I can write an opinion piece I still need to think about the topic and do research, and I have found myself modifying my opinion a few times during this process. Certainly the act of thinking deeply about any topic is a healthy mental exercise and I actually look forward to the writing and thinking time.

After I had written 20 blogs I had a fear that I was going to run out of topics. I look around the industry and I don’t see anybody else writing daily blogs, and perhaps I am nuts to try to do so. A lot of industry blogs have posted a dozen entries in the time where I have done 100. I still don’t know exactly what that means, because in daily life I am not overly loquacious, but I seem to not lack words when I write. There may come a day where I struggle to find topics, and if that happens I will slow down. But there is so much going on in this industry that I just don’t see that happening for a while.

I am also a bit surprised by where the blog has taken me. I have spent a lot of time thinking about and writing about the future of the industry. The industry is undergoing more changes right now at a faster pace than ever. Companies are changing, technology is changing and products are changing. Trying to figure out where this is all headed is an interesting exercise. I am sure I going to be wrong about a lot of my predictions, but right about others. Certainly there will be new inventions that will change the landscape in ways we can’t imagine today and that will trump any current predictions. As an example, a decade ago it would have been nearly impossible to predict the impact of social media and there will be the new equivalents to Facebook coming.

I find the Internet of Things to be the most interesting of change of all. The idea of filling our environment with little sensors and tiny computers to give us a different interface with our environment is fascinating and exciting. I have read science fiction books since I was a pre-teen and it is amazing to see us on the cusp of so many of the things predicted in that genre.

I have found myself writing about the cable industry a lot more than I expected, but that is where all of the action is happening. I don’t think anybody expects a future where households buy the huge bundle of channels in the same way they do today. There are too many market forces pushing on the industry to change. But I’m not sure anybody really knows where it will end up in a decade.

I thank my readers for dipping in from time to time to see what I have to say. My volume of readers had grown continually and just watching the blog tools is an interesting thing to do. I can see how many visitors come to my blog each day, and I know how many articles they read (but not who they are). One thing I know now for sure is that the telecom industry shuts down on Friday afternoons and very few people read the blog then.

This is a blog for telecom carriers and it is meeting all of the goals I originally established. It’s making me stay current. It’s helping me understand the issues in the industry. It’s letting me think about the future of the industry and then turning around and making that process relevant to my friends and clients. So, at least for now, I see the blog continuing, so stay tuned.