For much of the AI era, the spotlight has been firmly fixed on models and compute. Bigger models, more GPUs and larger AI clusters have defined the race to build increasingly capable artificial intelligence.
But that race is changing.
We are moving from what might be called the model economy, where value is largely associated with creating and training AI models, towards a token economy, where value is generated every time those models are used. Every question posed to an AI assistant, every action taken by an AI agent and every interaction with an intelligent application creates computational work expressed through tokens.
And that changes the infrastructure equation.
The challenge is no longer simply how to build the biggest AI cluster. It is how to connect compute, data, memory and intelligence distributed across many locations and deliver AI services wherever they are needed.
That makes the network far more than connectivity. It becomes the infrastructure that unleashes the value of AI.
From the model economy to the token economy
Training built the foundations of generative AI. Inferencing is where those models are put to work.
A token is the basic unit of information processed by an AI model. But an AI interaction involves far more than simply converting a question into an answer. AI systems can process conversation history, retrieved documents, instructions, contextual information and state. A single interaction can therefore involve enormous amounts of information moving through the infrastructure.
This becomes even more significant as AI evolves towards agentic and increasingly distributed systems.
An AI request may involve prefill processing to interpret a request and build its context, decode engines generating a response token by token, KV caches maintaining working context, data platforms supplying information and distributed compute resources performing the processing. These components do not necessarily reside in the same place. They may be spread across cloud infrastructure, regional facilities and edge locations.
The network is what connects them.
So the future challenge of AI is not only about creating intelligence. It is about efficiently connecting and coordinating intelligence across a distributed infrastructure.
AI needs a new dimension of scale
AI infrastructure has traditionally been viewed through several dimensions of scale.
Scale-up connects processors within a server, node or rack, allowing tightly coupled compute and memory resources to operate together.
Scale-out connects GPUs across systems, enabling thousands of accelerators to operate as a large AI cluster.
Scale-across can extend that cluster across facilities when constraints such as power, cooling or space prevent further growth within one location. [DCD Webina…2 ppm.docx | Word]
Each solves an important problem. But fundamentally they remain focused on making a compute domain larger.
Distributed AI introduces a different challenge.
Rather than simply extending one cluster, we increasingly need to federate independent AI resources and services across different locations. A workload may need to access compute in one facility, data in another and contextual information maintained elsewhere.
This takes us beyond scaling the cluster towards scaling AI itself across the network.
The network becomes the mechanism through which distributed resources can exchange data, context and intelligence securely and efficiently, allowing geographically dispersed infrastructure to participate in a much larger AI system.
The best AI network may be the one nobody notices
There is an interesting paradox at the heart of networking for AI: the better the network works, the less visible it becomes.
Users do not care which GPU processed their request, where a KV cache was located or which data centre supplied additional context. They care about the experience.
Behind that experience, however, a considerable amount of information may be moving between locations, including prompts, cached data, model information, retrieval results, security policies and orchestration instructions.
If the network introduces latency, congestion, loss or unpredictable performance, the experience can suffer. Response times can increase and token generation can slow.
Conversely, a high-performance network can provide low latency, reliable transport, intelligent routing and capacity across packet and optical infrastructure. Its role is not simply to connect sites but to allow distributed resources to work together predictably as a cohesive AI system.
That is an important shift in thinking. For distributed AI, the network becomes part of the AI system.
Shared infrastructure without shared risk
As AI infrastructure becomes distributed, it must also increasingly support multiple customers, applications and AI services on common physical infrastructure.
That creates a fundamental requirement: sharing infrastructure cannot mean sharing risk.
IP routing and optical transport provide complementary mechanisms for creating isolation. Technologies including VPNs, VRFs, segment routing policies, traffic engineering and network slicing can establish logical separation, while the optical layer can add dedicated wavelengths, OTN channels, optical paths and reserved bandwidth.
This also applies within the context of an individual tenant.
Real-time inferencing, model updates, retrieval-augmented generation queries, agent coordination and data replication can have very different requirements. Isolating these services helps prevent a surge in one workload from adversely affecting another and supports predictable performance, security and service-level objectives.
The opportunity, therefore, is not simply multi-tenancy. It is an infrastructure capable of delivering assured AI services across shared resources.
Security has to move at network speed
The same principle applies to security.
In highly distributed environments, security cannot depend exclusively on people reacting after something has gone wrong. The source material, for example, describes the growing importance of automated DDoS defence as attacks become shorter and more intense. The goal moves beyond detection towards prevention, autonomous mitigation and resilience.
The broader principle is highly relevant to AI infrastructure.
As systems become faster and more distributed, people increasingly define the policies while the infrastructure responds automatically. AI operating at machine speed needs a network capable of protecting it at network speed.
Long-term protection matters as well. The threat posed by “harvest now, decrypt later” means organisations need to consider how long sensitive information must remain confidential, rather than waiting for some theoretical future “Q-Day”. Building a roadmap towards quantum-safe networking therefore becomes part of long-term infrastructure resilience.
The network is where AI becomes a service
Compute creates tremendous AI capability. Models create intelligence. Data provides context.
But none of those elements produces its full value in isolation.
As AI shifts towards distributed inferencing, agents and token-driven services, its value increasingly depends on the ability to connect the right compute with the right data and context, at the right location and at the right time.
That is the role the network is beginning to play.
It connects distributed AI resources. It provides the performance they depend upon. It creates isolation between tenants and services. It embeds resilience and security. And, critically, it lets all those individual components operate as a coherent system.
The next chapter of AI will therefore not be defined solely by who builds the largest model or deploys the most GPUs. It will be defined by how effectively we can connect intelligence and turn distributed compute into consumable AI services.
That is why the network does much more than connect AI.
It unleashes its value.
Table of Contents
