Tag Archives: Big data

Farmers and Big Data

johndeereoutsideProbably the biggest change coming soon to crop farming is precision agriculture. This applies GPS and sensors to monitor field conditions like water, soil, nutrients, weeds, etc. to optimize the application of water, pesticides, and fertilizers in order to maximize the crop yields in different parts of the farm. Anybody who has ever farmed knows that fields are not uniform in nature and that the various factors that produce the best crops differ even within one field.

Precision agriculture is needed if we are to feed the growing world population, which is expected to reach almost 10 billion by 2050. As a planet we will need to get the best possible yield out of each field and farm. This might all have to happen against a back drop of climate change which is playing havoc with local weather conditions.

A number of farmers have started the process of gathering the raw data needed to understand their own farms and conditions. Farmers know the best and worst sections of their fields, but they do not understand the subtle differences between all of the acreage. In the past farmers haven’t known the specific yield differences between the various microcosms within their farm. But they are now able to gather the facts needed to know their land better. It’s a classic big data application that will recommend specific treatments for different parts of a field by sifting through and making sense of the large numbers of monitor readings.

In order to maximize precision agriculture new automated farm machinery will be needed to selectively treat different parts of the fields. The large farm equipment manufacturers expect that farming will be the first major application for drones of all types. They are developing both wheeled vehicles and aerial drone systems that can water or treat sections of the fields as needed.

This is a major challenge because farming has historically been a somewhat low technology business. While farms have employed expensive equipment, the thinking part of the business was always the responsibility of each farmer, and the farmers with the best knowledge and experience would typically out-produce their neighbors. But monitoring can level the playing field and dramatically increase yields for everybody.

There are several hurdles in implementing precision agriculture. First is access to the capital needed to buy the monitors and the equipment used to selectively treat fields. This need for capital is clearly going to favor large farms over small ones and will be yet one more factor leading to the consolidation of small farms into larger enterprises.

But the second need is broadband. Gathering all of the needed data, analyzing it, and turning it into workable solutions presupposes the ability to get data from the fields and sent to a supercomputer somewhere for analysis. And that process needs broadband. A farmer who is still stuck with dial-up or satellite broadband access is not going to have the bandwidth needed to properly gather and crunch the large amount of data needed to find the best solutions.

This doesn’t necessitate fiber to the fields because a lot of the data gathering can be done wirelessly. But it does require that farms are fed with high-speed Internet access and good wireless coverage, something that does require rural fiber. I published a blog a few weeks ago that outlined the availability of broadband on farms and it is not currently a pretty picture. Far too many farms are still stuck with dial-up, satellite, or very slow rural DSL.

Some farmers are lucky to live in areas where communications co-ops and rural telcos are bringing them good broadband, but most are in areas where there is neither broadband nor anybody currently planning on expanding broadband. At some point the need for farming broadband will percolate up as a national priority. Meanwhile, in every rural place I visit, the farmers are at the forefront of those asking for better broadband.

 

Big Data is Coming Your Way

The InternetI read all of the time that there is an explosion in the amount of data that companies have to process and that the data is getting more complex. Today I look at the data that big carriers have at their fingertips and that eventually will be available to smaller carriers.

Let me start by talking about small carriers, the folks who read this blog. For many years we have had a good grasp of what we need to know about our customers. Companies have had somewhat flat database files that captured things like the customer name and address, a list of the services they purchase and a list of the equipment associated with their account. Some small carriers have taken this further and captured things like a history of every service call at a given address. Overall, if you consider the amount of data that we keep on any one customer it’s a manageable pile in information.

And most of the world was just like the little carriers. If you went back ten or fifteen years it’s likely that the customer database at a large carrier like AT&T Wireless was not very much more complicated than the databases at the small carriers. But over the last decade, and particularly over the last few years a lot more data about customers is available to somebody like AT&T. It’s natural that they would start capturing new kinds of data, even if they aren’t entirely sure how to use it yet.

So what kind of data is available to AT&T that was not available ten years ago? Following are a few examples:

Geospatial Data. Once GPS apps were activated in smartphones we gained the ability to know at all times where the phone (and presumably the caller) is at. In the past AT&T could identify the cell site of a caller, but only during the time that they were using the phone. They didn’t have any way to otherwise routinely know the location of a customer.

But today through the GPS built into a smartphone they can know where a phone is at all of the time it’s powered. So naturally companies like AT&T gather this kind of data, as do many other companies such as Google or Apple who control the operating systems of the smartphone. This data is just now starting to be of use in targeted advertising and is expected to become more and more valuable over time.

Voice Recognition. In the old days somebody like AT&T would have noted that a given customer called them. They would know what the call was about to the extent that somebody at AT&T took notes that became part of the customer service record. But today natural language processing technology means that customer calls can be recorded and then saved as text. AT&T can have a stored transcript of every call they receive.

Text Analysis. Whether the source is transcribed voice call, a text message, a tweet or an email, there is now software that can make contextual sense out of correspondence with your customers. This means that you can not only save text from customers but that your software can take a stab data analyzing it and categorizing according to key words and phrases within the message.

Sentiment Analysis. To go one step further, there is now software that can make a judgment about the mood and intent of customers. This can be done with recorded voice looking for emotion or can be done with text looking positive or negative sentiments.

And these are just examples of the new source of customer data that any company might have access to. AT&T is now storing a vastly greater amount of data about each customer than they did in the past. They are keeping a record of where the customer is at all times and records on every interaction with the customer.

This doesn’t even consider the huge amount of data that AT&T Wireless can derive from being the ISP. As an ISP AT&T knows every website a customer visits, the text of every email, tweet or Facebook posting, ever topic they have looked at with search engines, every video watched over the web. The amount of data gathered by the ISP side of a large company is truly daunting.

We are at the very beginning of the big data analytics industry. It’s not hard to believe that within another decade that companies will be able to accurately profile their customers (or can buy the data from somebody else to do so). It’s already eerie how much somebody like Google can know about you. The next evolution of the industry is probably going to come from incorporating artificial intelligence from something like IBM’s Watson computer platform that will direct the analysis automatically.

We all know that what the big companies like AT&T do today that smaller carriers will be doing in ten years. For example, the software to analyze things like the sentiments of customers will get cheap enough or common enough that it will come built into customer service software. And this means that your company will be gathering vastly more data than you do today.

The Technology Revolution in Health Care

FantasticvoyageposterI’ve always kept an eye on how technology is improving health care. I think like a lot of baby boomers, I want to stay healthy and vigorous for as long as possible. It seems like I see a new article almost every day talking about how big data, or some new monitor that’s part of the Internet of Things is improving health care. Following are a few examples from very recent reading:

  • Enlitic is using the same sort of technology that can recognize faces on Facebook to make medical diagnosis. They are applying the technology to x-rays and other diagnostic tools to recognize cancer and other problems that might not always be obvious to a doctor. But the company is then using big data and taking the diagnosis process farther by looking at lab results, insurance claims and other records associated with a patient to try to refine a diagnosis. The software is only about a year old but it’s reported that it is already making big strides in providing accurate diagnoses. One use of this technology will be to identify cancers early. Possibly the biggest potential use of the software will be to provide diagnosis in third world countries where trained doctors are scarce.
  • Using a combination of wearables and big data, the Michael J. Fox Foundation is undertaking a large trial to gather data on people with Parkinson’s disease. It has always been difficult for doctors to understand the stage of the disease in a given patient due to the fact that the disease progresses in so many different ways. The trial is going to use smartwatches to monitor and report the movements of participants 24/7 and to gather other health data from thousands of Parkinson sufferers with the goal of building a database that will allow big data to identify trends and datapoints about the progression of the disease. The hope is that this will allow doctors to better understand the condition of any given patient. Big data ought to be able to spot trends in the progression of symptoms that have baffled doctors.
  • In a totally different use of technology, the combination of IBM’s Watson supercomputer and Siri are teaming up to bring the power of the supercomputer to the point of patient interaction. Apple and IBM are working together to create a platform that will allow doctors and first responders in making diagnosis and in gathering the needed data on a given patient, on site, anywhere. In essence they are bringing the diagnostic skills of a doctor into the field. The two companies believe that this same technology can be used in many other applications but are tackling mobile health first since they see it as a market in need of a solution. The two companies are providing the platform and have invited app writers to design programs that can take advantage of Watson’s computing power in a medical setting.
  • In news that many people might find disturbing, it’s been reported that medical insurance companies have been investigating the use of big databases that are commercially available from social media and other sources. These databases are already used to paint detailed pictures of us to use for advertising purposes, but some of this data also can tell a lot about our lifestyle. People don’t seem to much mind that this data is used to predict the next car they will buy, but are going to be a lot less comfortable if our insurance company looks to see how much alcohol they drink or if they watch a lot of TV. I doubt that the insurance companies even know how they might use this data yet, but one would think it might eventually be used in setting insurance rates that are specific by person based upon their health profile.
  • Finally, at the other end of the spectrum from big data is the whole trend of developing tiny devices that can be used to monitor us. There has been a lot of progress made in developing smart pills that can be swallowed and that can provide feedback from the intestinal track. There has also been a lot of progress in developing stick-on monitors with tiny chips that are allowing diagnostic tools to replace the maze of wires that have been connected to people in the past. A friend of mine just visited an emergency room and was connected to tiny wireless monitors rather than the typical array of wires. And there has been significant research into tiny blood monitors that swim around inside the blood stream. If anybody remembers the movie Fantastic Voyage you can see the potential for tiny medical devices inside the body that can find and fix problems at the cellular level. But we are probably still a long way from having tiny people inside of those devices!

The Downside of Big Data

DARPA_Big_DataBig tech companies have been crowing about some of the amazing things that can be done using big data. For example, in the area of interacting with people, retailers are working hard to create personalized shopping experiences aimed at individual shoppers. Specials will pop up on cell phones as someone walks by a display that are aimed at them specifically. While many will feel this is an invasion of privacy, others are looking forward to an enhanced shopping experience. Big data promises to also personalize things like health care so that every doctor you ever see will truly understand your health history and they can guard against conflicting medicines and other things detrimental to your health.

But there are already downsides to big data. Big data is being used to put together a detailed portrait of everybody. And that leads to various degrees of profiling. The very same data that can be used to make your shopping experience better can also be used for many negative purposes. Consider some of the following examples:

  • The Chicago Police department apparently used big data to create a list of the 400 people in the community that they think are most likely to commit a murder. But then they went so far as to contact these people to tell them they were watching them. If anybody remembers the movie Minority Report, this feels like we are already reaching that time where the police convict people for crimes they are going to commit in the future.
  • Big data contains a lot of information about us – our age, race, sexual orientation, religion, weight, general health, number of kids or pets, state of our finances, etc. That kind of data can be easily used to discriminate against people in a variety of settings. We start entering a scary societal place when we use this kind of data to profile people for consideration for housing, employment, etc. There is already an industry of firms who sell this kind of profiling data to anybody for a fee. Where a prospective landlord used to check your credit report they can now find out everything about you. Let’s face it – people are bigoted or just biased and the availability of this kind of data makes it easy to redline or discriminate.
  • There is a big uptick in scams against the elderly who are being found through big data. The scams themselves are as old as the hills, but it’s the use of big data to identify the most vulnerable among us that is disturbing.
  • It was reported in 2012 that Staples displays different on-line prices to different customers based upon where they live. For example, customers who live close to a competitor might get cheaper prices than somebody who does not. But this same ability makes it easy to price differently based upon other factors and again can lead to redlining.
  • I have read where it is fairly easy to buy databases of people who have something in common – such as having diabetes, having tried to quit smoking, or nameless other traits. These lists can be used to market products specific to an ailment, but they also have been used for scams, blackmail and other nefarious purposes. It’s not hard to picture being able to take advantage of people with a gambling addiction or some other such problem.
  • The FAA’s Do Not Fly list is another result of big data and is notorious for containing names of toddlers and others who are obviously not a threat to national security. The list even ended up including several US Congressmen.

This all points to the need for some sort of legal protection of people from the misuse of big data. This is a hot topic in Europe right now but is not yet commonly debated here. Several civil rights groups have identified big data as a big threat and a new source for discrimination. But misuse of big data can go far beyond discrimination based upon race, religion or sexual orientation. Unfortunately it’s now possible to discriminate based upon a whole lot of other reasons as well.

The Basics of Big Data

DARPA_Big_DataI read all of time how big data is going to transform our lives. Big data is supposed to make our lives better by sorting through the data that surrounds us to help us make sense out of the chaos. This will be accomplished using tools of the new science / technology called analytics.

The most commonly used tools that make some basic sense out of big data are called descriptive analytics. This is the process of screening big data sets to produce statistics that we can understand. In the simplest sense descriptive analytics is used to count and tally data into understandable pieces.

Descriptive analytics are used to do things like track hits on web sites, to track followers on social media sites and to track other statistics like page views or any other statistic that involves basic counting. One of the more well-known uses of descriptive analytics in our industry is when the cable and cellphone companies track the amount of data that customers have used during the month to apply against data caps. If you recall, some of the big companies like Comcast had a really difficult time getting this right and some people still say that they are not accurate. This illustrates that descriptive analytics does not necessarily mean simple counting and can involve tracking more complex pieces of the larger data set.

A more complicated type of analytics is predictive analytics. This is the process of not just analyzing the data, but then trying to make predictions about what might come next. The programs used to analyze the big data for this purpose use a number of statistical, modeling and data mining techniques to makes some sense out of the data. These techniques do not really predict the future, but rather look at existing and probable outcomes and calculate the percentage probability of different scenarios.

For example, you read all of the time how companies like Facebook or Google can figure out all sorts of things about you, such as whether you are an alcoholic or have insomnia or if you are just starting a new relationship. They do this by comparing data they have gathered on you to data from millions of other users. These companies look at your behavior, and when you start to resemble a known behavior pattern they used predictive analytics to start to fill in the gaps to paint a probable picture of you. For example, they will probably not know for sure that you are an alcoholic or have diabetes, but they can calculate the likelihood that you fit one of those known patterns.

This is the where the use of big data starts to concern many people. As the techniques used to analyze big data about people get better these companies might come to know more about you than you might know about yourself. For instance, I’ve read that Facebook is getting fairly good at predicting when relationships between couples are coming to an end. Most couples in this situation probably know this as well, but over time Facebook will probably get good at sensing this a lot sooner than the average person will be able to do. After all, people are sometimes very unaware of their own behavior patterns, but a company like Facebook, especially when combined with data gathered from other sources can paint a detailed and accurate picture of you.

The final kind of analytics is called prescriptive analytics and this takes the the trends and statistical possibilities found through predictive analytics are uses them to suggest solutions to problems. We are still a long way from trusting computers to use prescriptive analytics to solve specific problems. But already today we can uncover unsuspected trends in the analysis of big data and the computer can then suggest several solutions to fix those problems, and assign a statistical probability of the potential success of each solution. We are in the infancy of this process, but this is the hoped for end game from analyzing big data.

The Internet of Things is counting on success in the techniques of prescriptive analytics. In the near future there will be many more big data sets generated about each person from a number of sources like medical monitors, home security systems. location monitors and multiple other monitors in our lives. When these data sets are combined with the things we do such as write emails, search web sites, text our friends, there will be a detailed set of data created about each one of us.  For example, let’s say that we feel queasy one evening. Big data will be able to suggest that this might have been due to the fact that we walked close to glen full of oak trees in full pollen that afternoon or that it might have come from the shrimp we had for lunch and that a few other people who ate at that restaurant are experiencing the same feeling. Big data will be able to correlate the things that happen to us to what is happening in the wider world.

The average person is going to experience the results of big data by having something that seems like a self-aware assistant, or at least a set of programs that seem to be aware. These programs will track everything we do and will give us a whole new set of tools to understand ourselves and to control our personal world better. But these same big data sets could also be used by others to know things about us that we want to keep private. Probably the scariest thing about this kind of analytics is that everybody has secrets they would prefer to not reveal and these analytics tools can go a long way towards uncovering these little secrets we all keep.Today we are still exploring the techniques that will help us make sense of big data, but as that starts working we are also going to have to find ways to protect our privacy.

The Ethics of Big Data

SpyVsSpyBack in 2010 Eric Schmidt, the CEO of Google at the time said something really frames the issues associated with big data. He said, “One day we had a conversation where we figured we could use [use Google’s data] to predict the stock market. And then we decided it was illegal. So we stopped doing that.”

Google doesn’t say this, but when you look at all of their data-gathering efforts it is clear that their ultimate goal is to create a database in of all human knowledge. They are far away from that today, but they have already amassed the largest database in history of the human race. Google is not the only one tackling this task. For example, Facebook has mapped out the social connections between more than a billion people. Wikipedia has undertaken something much smaller but has accumulated over 4.5 million factual articles in English. The nerd’s favorite web site WolframAlpha has accumulated an amazing number of facts about the world and can display them in easy-to-understand presentations. Both Google Maps and the OpenStreetMap database are both trying to create a database of our physical world.

Google has the scariest data about each of us because they know what we are thinking about and looking for from the Google search engine. And when they pair this up with other web data that identifies each of us they know what we are doing individually, but also what we are doing collectively. It’s said that Google now has the ability to predict many things about you due to the profile they have built on you. As an example, they know who’s an insomniac and what behavior insomniacs engage in, and so they probably understand insomniacs better, at a macro level than anybody else in the world.

This is not to say that Google is analyzing their data in that specific way, but they could be. And certainly they are making their data available for sale to other large companies who do want to know that kind of thing about us. Perhaps there is not yet a company who wants to market to insomniacs (but there probably is), but there are certainly companies who want to pinpoint their marketing to the most likely people to respond.

If you don’t think that big data companies are watching you, spend fifteen minutes looking at new cars on the Internet and then watch how many times new car ads pop up in your web experience in the next week. At the marketing level big data is already manifesting itself. But marketing is only the beginning, but the one that is making Google so wealthy today.

One can only begin to imagine the possibilities for Google and others to profit from the data they are going to be gathering from the upcoming Internet of Things. That data will include a lot of detail about our personal lives, from such mundane things like when we turn lights on and off to very personal things they will gather from medical monitors. And it’s all relevant and tells them a little more about us and lets them categorize you. Because in the end they want to profile you in great detail so that they can sell your data to those who are most interested in people just like you.

The question about whether this is good or bad is going to depend upon how they use this data. If they will sell this data to anybody willing to pay the price, then it’s bad, because not everybody is going to do good things with the data. There is already talk of companies using big data to prey upon the most vulnerable among us. It’s a well-known fact that poorest among us spend the most money on mundane things like cashing checks or getting a car loan, and with big data companies can pinpoint advertising to the most vulnerable of us. It’s certainly also possible for big data to be sold to companies that will use it overtly to do us harm. For instance, it’s not hard to envision a group of private investigators using personal data about us in all sorts of unsavory ways.

And probably most scary to me is if the government or the press has access to this data. I’ve heard the old axiom that nobody’s life can survive total scrutiny and that we all have things we would like to keep private. If the government and the press have access to big data, everybody can be made to seem guilty of something. This is the premise of ‘1984’ and many other science fiction books. We are getting very close to the day when that is no longer fiction.

Do You Hadoop?

Hadoop clusterOne of the big area of IT growth is in providing solutions for big data. Big data is defined as any data set that is too large to conveniently compute with traditional methods. The breakeven of what constitutes big data gets larger each year as computers get more powerful and today traditional computing can handle up to a few exabytes of data.

However, many computing problems have much larger data sets and the field of big data was formalized as techniques and technologies to handle big data have been developed. Analyzing big data sets is now big business and a report by the McKinsey Global Institute estimates that the worldwide market for big data will be $32 billion by 2017.

What are some of the places that big data has a practical application today?  The most obvious are the social networking sites where companies like Facebook and Linked-In are faced making sense with monstrously large amounts of data. Apple’s Siri is based upon big data. Mining big data for marketing purposes is one of the hottest uses big data and there is a whole new field called consumer genomics is trying to understand consumer behavior through application of big data techniques.

Of course there are the more traditional sources of big data in such areas as weather analysis, astronomy, particle physics, oil exploration, medical diagnosis and genetics that require looking routinely at massive data sets. And just starting is probably the biggest future use of big data – the Internet of Things – where ubiquitous sensors in the environment are going to spit out huge amounts of constant data.

The business of big data has expanded so quickly that some large corporations now have a chief data officer. There are numerous industries that can benefit from manipulating and making sense of big data sets. The Obama administration announced the Big Data Research and Development Initiative in 2012 to explore how using big data could make the government more efficient. I suspect the NSA beat them to the punch a few years earlier.

There are a number of different techniques used to analyze big data. The most common one so far is to use a series of small servers to look at chunks of the data rather than to try to process it all on one large computer. There have been arrays of tens of thousands of individual blades used in the manner in a few applications.

In 2004 Google published a paper called MapReduce that discussed using a distributed architecture and parallel processors to handle large amounts of data faster. In this architecture the data queries are ‘mapped’ across multiple processors. The results are than gathered back from each processor in a ‘gather’ step. The Google effort was followed by an Apache open source project named Hadoop. Hadoop loops are still a fundamental piece of many of the big data techniques used today.

Probably the most visible result for most people of big data is going to be in targeted advertising and the use of personal assistants. Who has not looked at a product on a web site and then seen advertising for similar products pop up all over your web and social sites? Advertising is getting very personal and routine ads aimed at just you are right around the corner.

Some people are familiar with personal assistants from using Apple’s Siri. But future assistants are going to be far more sophisticated and will learn you over time. People will become personally integrated with their own assistant and having a constant computer companion will change the way that most people live.

Grasping the Internet of Things

Internet of Things IoT13 Forum June 2013 040

Internet of Things IoT13 Forum June 2013 040 (Photo credit: marklittlewood1)

I have written several blog entries about the Internet of Things. But I have not really defined it very well. I read as many articles about the topic as I can find since I find it personally fascinating. To me this is mankind finally using computer technology to affect everyday life and goes far beyond things you can do with a PC or tablet.

I recently saw an article that summarized the direction of the Internet of Things into three categories – and this is a good description of where this is all headed. These categories are:

Knowledge of Self. This part of the Internet of things is in its infancy. But the future holds the promise that the Internet can be used to help people with self-control, mindfulness, behavior modification and training.

Today there are gimmicky things people are doing with sensors, such as counting the number of times you have opened the refrigerator as a way to remind you to lose weight. But this can be taken much further. We are not far from a time when people can use computers to help them change their behavior effectively, be that in losing weight or in getting your work done on time. Personal sensors will get to know you intimately and will be able to tell when you are daydreaming or straying from your tasks and can bring you back to what you want to accomplish. Computers can become the good angel on your shoulder should you choose that.

Probably the biggest promise in this area is that computers can be used to train anybody in almost anything they want to know. The problem with the Internet today is that it is nearly impossible in a lot of cases to distinguish between facts and fiction. But it ought to be possible to have the needed facts at your fingertips in real-time. If you have never changed a tire your own personal computer assistant will lead you through the steps and even show you videos of what to do as you do it for the first time. But such training could bring universal education to everybody in the world, which would be a gigantic transformation of mankind – and would obviate the widespread ignorance and superstitions that still come today from lack of education.

Knowledge of Others. Perhaps the two most importance in this area will be virtual presence and remote health care.

With virtual presence you will be able to participate almost anywhere as if you were there. This takes the idea of video conferencing and makes it 3D and real-time. This is going to transform the way we do business, hire employees and seek help from others.

But perhaps the biggest change is going to come in health care. Personal medical sensors are going to be able to monitor your body continuously and will alert you to any negative change. For instance, you will know when you are getting the flu at the earliest possible time so that you can take medicine to mitigate the symptoms.

There is also great promise that medical sensors will make it possible for people to live in their own homes for longer as we all age, something just about everybody wants. Sensors might even change the way we die. Over 80% of people say they want to die at home, but in 2009 only 33% did so. Medical monitoring and treatment tied to sensors ought to let a lot more of us die in the peace of our own beds.

Perhaps the biggest promise of personal monitors is the ability to detect and treat big problems before they get started. Doctors are saying that it ought to be possible to monitor for pre-cancerous cells and kill them when they first get started. If so, cancer could become a disease of the past.

Knowledge of the World. The Internet of Things promises to eventually have sensors throughout the environment. More detailed knowledge of our surroundings will let us micromanage our environment. Those who want a different amount of humidity in the air will be able to have this done automatically in rooms where they are alone.

But remote sensors hold the most promise in areas of things like manufacturing and food production. For instance, sensors can monitor a crop closely and can make sure that each part of a field gets the right amount of water and nutrition and that pests are controlled before they get out of hand. Such techniques could greatly increase the production of food per acre.

And we can monitor anything. People living near to a volcano, for example, will know far ahead of time when there has been an increase in activity.

Monitoring the wide world is going to be the last part of the Internet of Things to be implemented because it is going to require drastic new technologies in terms of small sensors and the ability to interpret what they are telling us. But a monitored world is going to be a very different world – probably one that is far safer, but also one where there is far less personal freedom – at least the freedom to publicly misbehave.

My Take on the Internet of Things

I think there might be as many different predictions about the Internet of Things as there are bloggers and pundits. So I thought I would join the fray and give my take as well. The Internet of Things is that it is going to involve a new set of technologies that will enable us to get feedback from our local environment. That is going to allow for the introduction of a new set of tools and toys, some frivolous and some revolutionary.

I have read scores of articles talking about how this is going to change daily life for households. The day may come when our households resemble the Jetsons and where we have robots with more common sense than most of us running our households, but we are many years away from that.

There will be lots of new toys and gadgets that will sometimes make our daily lives easier. For instance food we buy may have little sensors put into packaging that will tell you when your produce is getting ready to go bad so that you won’t forget to eat it. There will be better robots that can vacuum the floors and maybe even do laundry and walk the dog. But I don’t see these as revolutionary and probably not affordable for the general populace for some time. For a long time the Internet of Things is going to create toys that wealthy people or tech geeks will play with, and it will take years to get these technologies to make it into everybody’s homes. Very little of what I have been reading for household use sounds revolutionary.

The biggest revolutionary change that will directly affect the average person is medical monitoring. Within a decade or two it will be routine to have sensors always tracking your vitals so that they will know there is something wrong with you before you do. There will be little sensors in your bloodstream looking for things like cancer cells, which is going to mean that we won’t have to worry about curing cancer, we’ll head it off before it gets started. This will revolutionize healthcare to be proactive and preventative and will eventually be affordable to all.

English: A technology roadmap of the Internet ...

English: A technology roadmap of the Internet of Things. (Photo credit: Wikipedia)

I think the most immediate big benefactor of the Internet of Things is going to be at the industrial level. For instance, it is not hard to envision soil sensors that will tell the farmer the conditions of each part of his fields so that his smart tractor can fertilize or weed each section only as appropriate. There is already work going on to produce mini-sensors that can be sent underground into oil fields to give oil geologists the most accurate picture they have ever had of the underground topology. This will make it possible to extract a lot more oil and to do so more efficiently.

Small sensors will also make it a lot easier to manufacturer complex objects or complicated molecules. This could lead to the production of new polymers and materials that will be cheaper stronger and biodegradable. It will mean that medicines can be modified to interact with your specific DNA to avoid side effects. It means 3D printing that will feel like Star Trek replicators that will be able to combine complex molecules to make food and other objects. NASA has already undertaken a project to be able to print pizza as the first step towards being able to print food in space to enable long flights to Mars.

And a lot of what the Internet of Things might mean is a bit scary. Some high-end department stores already track customers with active cell phones to see exactly how they shop. But this is going to get far more personal and with face recognition software stores are going to know everything about how you shop. They will not just know what you buy, but what you looked at and thought about buying. And they will offer you instant on-site specials to get you to buy – ads that are aimed just at you, right where you are standing.

I remember reading a science fiction book once where the ads on the street changed for each person who walked by, and we are not that far away from that reality. There are already billboards in Japan that look at the demographics in front of them and which change the ads appropriately. Add facial recognition into that equation and they will move beyond showing ads aimed at middle-aged men and instead show an ad aimed directly at you. The Internet of Things is going to create a whole new set of attacks on privacy and as a society we will need to develop strategies and policies to protect ourselves against the onslaught of billions of sensors.

Probably one of the biggest uses of new sensors will be in energy management. And this will be done on the demand end rather than the supply end. Today we all have devices that use electricity continuously even when we aren’t using them. It may not seem like a lot of power to have lights on in an empty room or to have the water warm all of the time in an automatic coffee pot, but multiply these energy uses by millions and billions and it adds up to a lot of wasted power. You read today about the smart grid, which is an effort to be more efficient with electricity mostly on the demand side. But the real efficiencies will be gained when the devices in our life can act independently to minimize power usage.

Sensor technologies will be the heart of the Internet of Things and will be able to work on tasks that nobody wants to do. For instance, small nanobots that can metabolize or bind oil could be dispatched to an oil spill to quickly minimize environmental damage. The thousands of toxic waste dumps we have created on the planet can be restored by nanobots. Harvard has been working on developing a robot bee and it is not hard to envision little flying robots that could be monitoring and protecting endangered species in the wild. We will eventually use these technologies to eat the excess carbon dioxide in our atmosphere and to terraform Mars with an oxygen atmosphere and water.

Many of the technologies involved will be revolutionary and they will spark new debates in areas like privacy and data security. Mistakes will be made and there will be horror stories of little sensors gone awry. Some of the security monitoring will be put to bad uses by repressive regimes. But the positive things that can come out of the Internet of Things make me very excited about the next few decades.

Of course there will be a lot of bandwidth needed. The amount of raw data we will be gathering will be swamp current bandwidth needs. We are going to need bandwidth everywhere from the City to the factory to the farm, and areas without bandwidth are going to be locked out of a lot more than just not being able to stream NetFlix. The kind of bandwidth we are going to need is going to require fiber and we need to keep pushing fiber out to where people play and work.