Everybody I know in the telecom industry has been following the controversy surrounding the allegations that the NSA has been gathering information on everybody’s Internet usage. What I find somewhat amusing are the smaller ISPs who are telling people that they have not cooperated with the NSA, and that it is ‘safe’ for customers to use them. That is a great marketing ploy but it far from the truth. The Internet infrastructure in the country is very complex, but for the most part the data traffic in the country can be characterized in three ways: core Internet, peering and private traffic.
The private data traffic is just that. There are huge numbers of private data connections in the country that are not part of the ‘Internet’. For example, every banking consortium has a private network that connects branches and ATMs. Large corporations have private connections between different locations within the company. Oil companies have private data circuits between the oil fields and headquarters. And for the most part the data on these networks is private. Most corporations that use private networks do so for security purposes and many of them encrypt their data.
The FBI has always been able to get a ‘wiretap’ on private data circuits using a set of rules called CALEA (Communications Assistance for Law Enforcement Act). The CALEA rules proscribe the processes for the FBI to use to wiretap any data connection. But over the years I have asked hundreds of network technicians if they have ever seen a CALEA request and from what I can see this is not a widely used tool. It would require active assistance from telecom companies to tap into private data circuits, and there just does not seem to be much of that going on. Of course, there is also not a lot of likelihood in finding spy-worthy information in data dumps between oil pumps and many of the other sorts of transactions that happen on private networks.
But the NSA is not being accused of spying on private corporate data. The allegations are that they are monitoring routine Internet traffic and that they possess records of every web site visited and every email that is being sent over the Internet. And it seems plausible to me that the NSA could arrange this.
The Internet in the US works on a hub and spoke infrastructure. There are major Internet hubs in Los Angeles, New York, Atlanta, Chicago, Dallas and Washington DC. Most of ‘Internet’ traffic ends up at one of these hubs. There are smaller regional hubs, but all of the Internet traffic that comes from Albuquerque, Phoenix, San Francisco, Las Vegas and all of the other cities in that region will end up eventually in Los Angeles. You will hear ISP technicians talking about ‘hops’, meaning how many different regional hubs an Internet transmission must go through before it gets to one of these Internet hubs.
So when some smaller Internet provider says that the NSA does not have their data they are being hopeful, naive or they are just doing PR. I recall an article a few months back where Comcast, Time Warner and Cox all said that they had not cooperated with the NSA and that it was safer to use their networks than to use AT&T and Verizon, who supposedly have cooperated. But everything that comes from the Comcast and Cox networks ends up at one of these Internet hubs. If the NSA has figured out a way to collect data at these hubs then there would be no reason for them to come to the cable companies and ask for direct access. They would already be gathering the data on the customers of these companies.
But then there is the third piece of the Internet, the peering network. Peering is the process of carriers handing data directly to each other rather than sending it out over the general Internet. Companies do this to save money. There is a significant cost to send information to and from the Internet. Generally an ISP has to buy transport, meaning the right to send information through somebody’s fiber cable. And they have to buy ‘ports’ into the Internet, meaning bandwidth connection from the companies that own the Internet portals in those major hubs. If an ISP has enough data that goes to Google, for example, and if they also have a convenient place to meet Google that costs less than going to their normal Internet hub, then they will hand that data traffic directly to Google and avoid paying for the Internet ports.
And peering is also done locally. It is typical for the large ISPs in large cities to hand each other Internet traffic that is heading towards each other’s network. Peering used to be something that was done by the really large ISPs, but I now have ISP clients with as few as 10,000 customers who can justify some peering arrangements to save money. I doubt that anybody but the biggest ISPs understand what percentage of traffic is delivered through peering versus directly through the more traditional Internet connections.
But the peering traffic is growing all of the time, and to some extent peering traffic can bypass NSA scrutiny at the Internet hubs. But it sounds like the NSA probably has gotten their hands on a lot of the peering traffic too. For instance, a lot of peering traffic goes to Google, and so if the NSA has an arrangement with Google then that catches a lot of the peering traffic.
There certainly are smaller peering arrangements that the NSA could not intercept without direct help from the carriers involved. For now that would be the only ‘safe’ traffic on the Internet. But if the NSA is at the Internet hubs and also has arrangements with the larger companies in the peering chains, then they are getting most of the Internet traffic in the country. There really are no ‘safe’ ISPs in the US – just those who haven’t had the NSA knocking on their doors.