Telephone surveys have always been a staple of doing research in the business and political arenas. Surveys have been given to random samples of households to find out how the public as a whole feels about various topics. And surveys have been effective. The whole point of a survey is to sample a relatively small number of people and have good faith that the results of the survey represent the opinions of the public as a whole.
But there has been such a large drop in the number of households with landlines that one has to ask if it is possible to any longer do a valid telephone survey. The percentage of households with landlines has declined greatly and nationwide it is estimated to now be below 60%. We recently heard of a community in Colorado that has less than 45% of households with landlines.
The whole point of doing a survey is so that you can rely on the results to tell you something meaningful about the whole population. And there are several aspects to conducting a survey that are mandatory if the results are to be believable. In order to be valid, a survey must be delivered randomly to a sufficient proportion of the universe being sampled.
And therein lies the problem. I think it’s a valid question to ask if households who still use landlines are representative of the universe of all households. I think there is a lot of evidence that they are not representative. Telecom carriers everywhere are reporting that households that drop landlines are younger, more tech savvy and more innovative than households that keep landlines.
And so, in statistical terms, one must ask the hard question if a survey given only to households with landlines is any longer representative of the whole population. And the answer might be sometimes, based upon what is being asked. But for most of the purposes I see surveys used for, my gut tells me that landline households are no longer the same as all households.
For example, say that you wanted to ask how many people in a City wanted to get a gigabit of bandwidth. If you survey households with landlines you are most likely mostly talking to older households and households with kids. You are probably not going to be talking to younger households and tech savvy households who have a lifestyle that eschews landlines. And I think you are going to get a skewed answer that you cannot believe. One would think that a larger percentage of the landline houses would not be interested in gigabit speeds while you didn’t talk to many of the households who would be interested. And so, when you summarize your survey results you are not going to have a believable estimate of the number of people who would be interested in the gigabit speeds – which was the whole point of doing the survey.
There might be a way around this, but it is hard to pull off. If you can find a way to randomly call households in the town that includes landline and cellphone households, then you are again sampling the real universe of households. But this is a problem for several reasons:
- If you are already in business you are allowed to call any or all of your own customers. But as soon as you try to call in an area of people who are not your customers you must follow the Do Not Call rules, which says that it is illegal to call people who have registered to not get junk calls. You can obtain lists of such people, but it adds expense and cost to the survey.
- Then you must have access to a database that has a telephone number for everybody, and these rarely exist. Maybe some local government or utility might have such a list, but they can’t share these lists with anybody else due to privacy issues.
- Even if you have this kind of list it is against FCC rules to call cell phones to conduct a survey. The problem is that there are still plenty of customers on fixed-minute cellular plans and a lot of surveys require 20 minutes or more. If you are going to call cell phones you are strictly breaking the rules, so the first thing you should do is to tell cell phone users they can opt out of the call. But if enough cell phone callers refuse to take the survey, then you are back to having an invalid sample.
- You can’t solicit cell phone households to give their phone numbers for purposes of conducting a survey. As soon as you do that the sample is not random and we are back to square one.
A non-statistician might think, “As long as the results are close, I am okay with the survey not being entirely valid”. And they would be wrong. If a survey isn’t done properly, then there is no validity to the results. You do not want to make any important business decision based upon an invalid assumption. There are enough ways to fail in business and you shouldn’t add the sin of relying on false assumptions to the list of reasons why your business plan didn’t succeed.
There are other ways to do surveys such as going door-to-door, but other kinds of surveys are usually costlier and they have their own potential pitfalls. We might be soon be approaching the day when surveys are going to disappear from our lexicon of useful business tools.
