LAS VEGAS—VMware by Broadcom’s Explore event was headlined with talks on why private cloud using private data to run private AI is the path forward for enterprises. “It’s clear to me that the future of the enterprise is private,” as Broadcom CEO Hock Tan put it in a blog post. A sub-theme that played out, particularly in a lively session for analysts and media, was how to best network together the GPUs, and other data center infrastructure, needed to deliver AI. Broadcom’s Ram Velaga, SVP and GM of the Core Switching Group, was unequivocal: “Ethernet will be the technology to make this happen.”
Let’s take a step back. Velaga opened his comments by suggesting the audience “think about…what is machine learning and how is that different from cloud computing?” Cloud computing, he said, is about driving utilization of CPUs; with ML, it’s the opposite. “No one…machine learning workload can run on a single GPU…No single GPU can run an entire machine learning workload. You have to connect many GPUs together…so machine learning is a distributed computing problem. It’s actually the opposite of a cloud computing problem.”
For the Amazons, Microsofts, Metas and Tencents of the world, this means connecting tens of thousand or even hundreds of thousand GPUs together and, in some cases, across multiple facilities. In this problem space, “Network plays an extremely important role,” Velaga said. “We subscribe to this idea that the network is the computer.”
What about NVIDIA’s InfiniBand?
And to connect that computer, Ethernet is the way, Velaga said. The alternative here would be NVIDIA’s InfiniBand, a proprietary set of solutions the GPU giant describes as well-suited for “complex workloads [that] demand ultra-fast processing of high-resolution simulations, extreme-size datasets, and highly parallelized algorithms.” InfiniBand, they continue, “provides dramatic leaps in performance to achieve faster time to discovery with less cost and complexity.”
Not the case, Velaga said. InfiniBand is expensive, fragile and predicated on the faulty assumption that the physical infrastructure is lossless. As for Ethernet, which was standardized in the 1980s and has been the subject of ongoing innovation and advancement since, he delineated the following selling points:
- Pervasive deployment
- Open and standards-based
- Highest Remote Direct Access Memory (RDMA) performance for AI fabrics
- Lowest cost compared to proprietary tech
- Consistent across front-end, back-end, storage and management networks
- High availability, reliability and ease of use
- Broad silicon, hardware, software, automation, monitoring and debugging solutions from a large ecosystem
To that last point, Velaga said, “We steadfastly have been innovating in this world of Ethernet.” And, “When there’s so much competition, you have no choice but to innovate.” InfiniBand, he said, is “a road to nowhere.”
To support that position, he pointed to the nascent work Microsoft and OpenAI are engaged in to at some point build the $100 billion Stargate, essentially the data centers, or supercomputer if you like, that could ultimately run OpenAI’s large language models using millions of AI chips. There’s a lot of reporting out there but the gist seems to be that while Microsoft is currently using InfiniBand, OpenAI prefers Ethernet, so Ethernet will probably win out.
“We can today,” Velaga said, “deploy a million GPU cluster on Ethernet. InfiniBand cannot even scratch the surface of that.”
