I write all of the time about the rural digital divide – about homes that have no broadband options or that have terrible options such as extremely slow DSL or wireless service. The COVID-19 crisis has reminded us that there are also a lot of homes in cities and towns that don’t have broadband.

John B. Horrigan published a paper earlier this year titled Measuring the Gap that makes the point that the reasons that homes don’t have broadband are complicated. There have been studies over the years that have tried to pin down the primary reason that homes don’t have broadband, but by doing so the studies have glossed over the fact that most homes have multiple reasons for not having broadband.

A good example of this is a Pew Research Center survey in 2019 that explored the issue. In that survey:

• 50% of respondents said that high prices is a reason for not having broadband, but only 21% said price is the primary reason.
• 45% of respondents said they relied on smartphones that could do everything they need, but only 23% said that was the primary reason for not buying broadband.
• 43% said they were able to get access to the Internet from a source outside the home, but only 11% gave that as the primary reason.
• 45% said that the cost of a computer is too expensive, but only 10% gave that as the primary reason.

As Horrigan points out, sometimes there is bias in the questions being asked in a survey. If the surveyor has pre-conceived ideas about why folks don’t have broadband they will miss some of the reasons. Consider a 2017 survey from the California Emerging Technology Fund. This survey showed different reasons than Pew for why homes don’t have broadband because the survey asked different questions. The survey showed:

• 69% said the cost of monthly access and of affording a computer or smartphone was too high. 34% listed this as the primary reason for not having broadband.
• 44% said it was too difficult to set up a computer and to learn how to use broadband, which 12% gave this as the primary reason.
• 42% said they were concerned about privacy and computer viruses, while 21% gave this as the primary reason for not having broadband.
• 41% said they had a lack of interest in being online, with 22% giving this as the primary reason for not having broadband.

The results of those two surveys are drastically different because the surveys asked different questions. If a survey doesn’t provide the option to say that privacy is a reason for not having broadband, then that gets missed. People can only respond to the questions asked in a survey as presented to them. For example, there were 12% of respondents in the second survey above that worried about privacy as their primary reason for not having broadband. There had to be people that felt the same way in the Pew survey, but since that question was never asked, respondents were forced to pick from among the choices they were given.

This highlights one of the issues of using surveys to find out why people do certain things. Surveys are best used when measuring what people do. For example, a well-designed survey can make a great and reliable estimate of the number of homes in a community that don’t have a home computer. But it’s a lot tougher to use a survey to find out why homes don’t have computers since there might be dozens of reasons for not having one.

Another issue to consider is that people might not tell a surveyor the truthful answer to a question if they think the response is personal. For example, people don’t like to admit that using a computer is too hard for them or that they are intimidated by technology. Many people are not going to tell a stranger that they can’t figure out how to use a computer. However, those same people might willingly share that they would be more likely to use a computer if they had better training. The manner of asking this sort of question can change the response.

This blog is not meant to bash surveys, because a survey is one of the best tools available for understanding broadband in a market. A survey can quantify how many people use different ISPs and can measure their happiness with the various ISPs already in the market. A survey can provide a decently reliable estimate of the percentage of the community that will consider switching to a new ISP. But surveys are a lot less reliable when they ask people to reveal personal reasons why they do or don’t do something – for the simple reason that people are often unwilling to share their shortcomings and fears with a stranger.

This is something to keep in mind if you want to use a survey to understand broadband in your community. Asking questions about sensitive subjects produce unreliable results. As an example, surveys do a lousy job of predicting what people are willing to pay for broadband. A survey can quantify what somebody would like to pay for broadband, but that is not the same question of what they will pay. I’ve seen surveys convince ISPs to set low broadband rates due to faulty survey questions. It’s somewhat meaningless when somebody who is already paying $75 per month for broadband tells you they would only change to a new ISP that charges $45. Such a respondent is likely somewhat embarrassed to admit they are paying too much for broadband today, and that bias makes their answer unreliable.

Writing good survey questions is an art. I’ve been doing that for twenty years and I still find situations where it’s nearly impossible to get the answers that clients are hoping for when the survey probes into questions that customers don’t necessarily want to answer.