I’ve written several times in the past about how network consolidation and centralization of networks is putting our broadband and voice networks in increased jeopardy. Outages like the latest one that hit AT&T are evidence that we’ve probably gone too far with consolidation. Networks are always going to have problems and glitches. However, problems that would have affected only one city or region in the past can now affect huge parts of the country due to carriers having consolidated electronics hubs and network control at a small number of locations nationwide. It’s easy to understand why carriers are in favor of the savings that come from consolidation, but it’s vital that we recognize and acknowledge the increased risk that comes as a consequence of choosing efficiency over other factors.
There is one area of network consolidation that I think is of particular concern. Carriers have very quietly rearranged the 911 network configuration in the country in a way that I think puts 911 callers in jeopardy.
Historically, 911 was the ultimate local network arrangement. Each city or county had at least one 911 call center (PSAP). Over time, 911 call centers were consolidated regionally. For example, the 911 centers for a dozen cities in a metropolitan area might have been consolidated into one regional 911 center for efficiency. But the consolidated metropolitan 911 call centers were still local.
The 911 PSAP call centers were connected to the closest tandem switch. These were switches that didn’t serve customers but only switched calls between carriers. When a customer of a telephone company, cable company, or CLEC called 911, the call was routed to the closest tandem. The local tandem housed a database that could translate a calling telephone number into an address. Once the tandem identified the address of a caller, the 911 call was routed to the appropriate 911 PSAP. Cellular calls were more of a challenge, but over time the cellular carriers developed software that could triangulate the originating point of a cellular call to 911. Routing 911 calls got more complicated when Congress passed Kari’s Law and the Ray Baum Act, which requires multi-tenant buildings, including businesses and hotels, to identify the specific location within a building for each handset. In this historical network, 911 calls did not leave the region where the call was made.
Every carrier that had voice customers – telephone companies, cable companies, CLECs, VoIP companies, and cellular companies – had to buy a local dedicated connection between their switching center and each local 911 tandem. There was always a risk that a cable cut could stop a 911 call from reaching the 911 tandem, and most 911 rules required carriers to create a diverse path for reaching the 911 tandem. There were inevitably some carriers in every market where the voice switch was located outside of the market, and that carrier had a greater risk of being unable to reach 911 services than local carriers.
Over time, voice services have been consolidated as carriers moved voice functions to regional or national switching platforms. The biggest carriers gained a lot of efficiency by consolidating and reducing the number of voice switches, and a customer call might route calls to a switch in a distant state to initiate any voice call, including 911. In creating these efficiencies, carriers also increased the distance between customers and the electronics used to complete voice calls.
The next big change in 911 networks came when SIP-based trunks were allowed for 911. Historically, 911 trunks were placed on T1s or the equivalent. The T1s were directly routed on a dedicated path between a carrier and each local tandem close to its customers. The change to SIP trunks meant that voice routing to the 911 centers started to use the Internet (or IP-based transport) instead of a dedicated physical path – calls to 911 can be routed in multiple ways. The shift to SIP has resulted in the creation of SIP 911 tandems that can serve a huge footprint, even nationwide.
What does all of this mean? It’s now fairly routine when a customer calls 911 that the call is routed to a 911 tandem that is many states away before ultimately being routed back to the appropriate local 911 center. A call to 911 can travel great distances, and in the network world, greater distance means greater risk. A network outage in a distant city can stop or block a local call to 911. This awkward and unsafe system is why the first thing carriers announce in the middle of a big outage is that 911 calls are still safe. That statement was part of the first announcement made in the recent AT&T outage. Carriers say this because they fully understand how fragile the 911 system has become.
There is no doubt that the current 911 configuration is more efficient. However, I suspect the government folks in charge of 911 centers and public safety have little idea about the great distances covered by a routine 911 call. There would be a big improvement in 911 safety if the 911 tandems and databases were closer to customers, like the historical 911 networks. If I was a government 911 official, I’d make every carrier show me the route taken when a customer dials 911. I think they’ll be shocked, and maybe that can lead to putting the genie back in the bottle. Carriers will always choose consolidation and saving money over other factors. However, saving money is not as important as making sure that 911 calls get delivered when people are in trouble.
Doug,
Having just experienced 6 months of not being able to call 911 I know what you are speaking of. My LEC is Frontier and once I discovered quite by accident in January of this year that my 911 calls were going to other than my normal 911 PSAP. No address ;location info nor any 9111 Plus info were showing up there. The other PSAP had no idea who I was nor where I was located. My knowledge is that 911 worked correctly for me for the 24 years we have lived at this address so I opened a ticket with Frontier. The lady who worked with me from Frontier was very accommodating and we had several in depth conversations about where it was going as well as at least a dozen test calls , a couple with her conferenced on so she could here exactly what the foreign PSAP was saying. Once she had determined that Frontier was passing my ALI and ANI correctly and that it was with this other catier, I presume ATT, resolution came to almost a complete stop. My contact at Frontier sounded like she was being verbally abused by this other carrier and I finally took my writing a formal complaint with the MPSC to help her get it resolved with this other company. Even then it still took several weeks for a service which is expected to be there full time.
I have made several test calls and it all seems to be in place correctly once again some six months after I first became aware of the issue. I have to wonder just how long it was out of service for and how many of my neighbors if any might have been affected similarly. But what really galls one is the apparent lack of concern on the part of the other carrier and their condescending attitude toward the person at Frontier. Thanks for letting me vent a bit.
It shouldn’t be up to people to fix 911. There was a day in the industry when your problem would have meant all hands on deck to fix it. But with consolidation, things are obviously being lost in the shuffle.
Statewide 911 outage in Massachusetts going on now
Pingback: FCC to Ease Copper Retirement | POTs and PANs