Tag Archives: AI Traffic

Network Challenges from AI Traffic

Cisco recently published a report that looks at the impact of AI traffic on networks. It’s an interesting paper because Cisco found that AI traffic does not operate the same as most other web traffic. While the volume of AI traffic is small today, Cisco predicts that we’ll have to make changes to the web over time to accommodate growing AI traffic volumes. Cisco predicts that by 2035, one fourth of all web traffic will be AI agents and AI models in data centers.

Cisco notes that we’ve spent decades optimizing a web that delivers burst traffic, like video. When a video is viewed from the web, the data stream doesn’t have to be delivered evenly in real time. Instead, all that is needed is for the transmission of the video to reach the viewer before they are ready to watch it. Anybody who has watched a video can see that the streamed video is always working to to stay ahead of what you are watching.

AI traffic is very different. Cisco uses the term AI inference traffic to mean the real-time transfer of AI traffic between the AI models operating in data centers and users. AI inference traffic is delivered at what Cisco calls software speed, meaning the receiving end is ready to digest and use the data as it is delivered, quite different than streamed video that is only trying to stay ahead of a viewer.

The difference between AI traffic and normal web traffic is significant. The typical burst of AI traffic lasts twice as a typical burst of video data. While individual bursts of video data are smaller, the flow rate for video, which means the actual delivery time, lasts ten times longer than AI traffic, since video bursts are spread over time in multiple small bursts.

AI traffic is also two-way and requires a good upstream connection. In fact, Cisco found that 9% of AI traffic requires more upstream traffic than downstream traffic. Cisco believes the need for network upload speeds will increase as AI agents mature.

Current network latency is not a bottleneck for AI traffic, but Cisco says latency will become a problem as the volume of AI traffic increases. This will require a major rework of web architecture when latency becomes an issue.

Cisco found that tasks performed by AI generate 450% more traffic than the same task performed in a more traditional way. In Cisco’s vocabulary, AI agents act as power users and use a lot of network resources.

The bottom line is that AI traffic is different from current web traffic and will not only increase traffic volumes on networks, but it will also change the shape, symmetry, and needed priority of traffic.

There have already been discussions of creating a private web to connect between AI data centers. But that would only solve part of the problem, because AI traffic is eventually delivered to users throughout the web. AI traffic is going to create an interesting new set of challenges for network engineers, something that nobody envisioned just a few years ago.

Low Latency AI Networks

A partnership has been announced that has the goal of creating a low-latency private Internet for AI traffic. The three partners involved are Moonshot Energy, a manufacturer of electrical and modular infrastructure for AI data centers, QumulusAI, Inc., a provider of GPU-as-a Service, and Connected Nation Internet Exchange, which has been promoting the creation of more Internet Exchanges.

The group’s goal is to initially create 25 carrier-neutral interexchange points designed to handle only low-latency traffic. The goal is to scale to 125 locations, many which would be located at major research university campuses and municipalities. The coalition has labeled the new hubs as AI Pods.

The goal of this coalition is to create a network designed specifically for AI and other data traffic that requires low latency. The network will be designed with highly efficient switches at the hub sites that will move traffic quickly. This would essentially be a private network that would isolate low-latency traffic from the large volumes of general Internet traffic that can clog up Internet hubs at busy times.

The idea of creating private networks for data is an old one. Many universities in the country are connected to the Internet2 fiber network that allows for low-cost transfer of large amounts of research and other data between universities. Many corporations have created private networks between company sites to keep corporate data traffic out of normal Internet traffic flow and to provide a higher level of security.

Tackling this as a new venture makes a lot of sense. If the companies that run the large Internet hubs  decided to somehow give priority to AI or other traffic to reduce latency, they would awaken cries about violations of network neutrality, since such behavior is exactly what network neutrality is supposed to block. If the normal Internet hubs gave priority to bits from AI data centers, then all other traffic would get a lower priority and see more problems from delays. However, a private network for AI avoids such issues by isolating AI traffic from other traffic.

The first data site for the network is scheduled for activation in July 2026, located at the campus of Wichita State University. The coalition is working towards providing dual, geographically diverse fiber routes between the new AI hubs using 400 GB transport. Each AI site would house redundant 400 GB IX ports and switches. Data centers that want to connect to the network would acquire dark fiber to one of the AI hubs.

QumulusAI says the new network would result in moving GPU computing directly to the network edge, meaning the AI network could be expanded to reach large businesses and other users of large amounts of AI data.

Connected Nation has been touting the benefits of creating more Internet hubs for a number of years. These new hubs would also become carrier-neutral locations for the interexchange of normal Internet traffic, which would lower the cost to ISPs to reach the Internet.