For those consumers who waited until the last possible minute to upgrade their home Internet connections, finally moving up to high-speed broadband was likely a revelation. Everything got faster, certainly, but the changes went much deeper than speed. Late adopters hadn’t realized it, but their digital lives were essentially stuck in suspended animation. When they finally switched to higher-capacity connections, they found that all the digital content and services they use had become richer, more dynamic, more personalized. Capacity will do that.
Now, take that feeling and multiply it by many orders of magnitude, and you can start to approach what cloud providers, communication service providers (CSPs), and enterprises might expect with 400-gigabit Ethernet (400GbE) connectivity. By upgrading data centers and wide-area network (WAN) transport links to 400GbE—4x the capacity of the fattest pipes most organizations use today—they’ll be able to do things that weren’t possible before. Just as important, with the ability to deliver 400GbE with just 2.5x the power consumption of 100GbE, they’ll be able to serve their customers’ insatiable need for digital content and services far more efficiently.
But, while vendors and analysts have been talking up 400GbE for a while, we’ve yet to see widespread commercial 400GbE solutions deployed. Have the benefits of 400GbE—or the urgency to adopt it—been overhyped? The answer is a hard no. The huge increases in capacity, density, and power efficiency that come with 400GbE are in serious demand, and that demand will only grow. The holdup revolves around more prosaic factors. First, the industry must coalesce around standardized implementations. More importantly, 400GbE pluggable optics must become available at volume and at reasonable cost. Fortunately, the industry is making steady progress in both areas.
The drive to 400GbE
To understand the story of 400GbE, we have to start with the companies clamoring for it most urgently: cloud hyperscalers. Companies like Google, Amazon, Facebook and those like them are dealing with explosive traffic growth in their vast data centers. Facebook, for example, generates 4 petabytes of new data every day. Google’s requirements for its data center networks double every 12-15 months. Trying to meet those demands with 100GbE links is like trying to pump a firehose worth of water through a handful of drinking straws. You can do it, but you’d really prefer a better option.
Hyperscalers are just the tip of the spear. CSPs face similar growth trends in their own networks, and large enterprise data centers aren’t far behind. What’s driving this massive growth?
- New applications: Globally, we’re seeing explosive growth in cloud and digital applications, in particular, video. HD and ultra-HD 4K video content currently consumes 73 percent of the world’s bandwidth, projected to exceed 82 percent by 2021. Growing use of artificial intelligence (AI) and machine learning in cloud data centers also drive a need for substantial increases in capacity and speeds.
- Evolving data center backbones: As hyperscalers adopt new architectures to keep pace with these demands, they are upgrading from 50GbE to 100GbE server connections. The only way to carry this volume of traffic economically is with 400GbE optics.
- Emerging 5G networks: The rollout of 5G puts mobile operators on a fast track to network requirements approaching those of hyperscalers. A variety of 5G use cases will drive the need for faster links: high-speed broadband, telemedicine, immersive virtual reality experiences and many others. But, just connecting 5G cell sites will require huge capacity boosts. Massive MIMO (multiple input, multiple output) antenna technologies will mean a cell site that used to need four 10GbE interfaces to service a dense urban area may soon need 250 interfaces.
- Rapid growth in digital services to the home and remote work: The COVID-19 pandemic unleashed a massive, sudden spike in digital services in the home. As more people work remotely and shelter in place, the high-capacity services they use—video conferencing, remote workspaces, streaming video, gaming and more—are having a profound impact on global traffic patterns. Some of these changes will be short-lived, while others will have lingering effects. We can expect that timelines for upgrading capacity in residential networks will likely be accelerated.
Overcoming obstacles
These factors add up to huge new revenue opportunities for cloud providers, CSPs and enterprises alike. To capitalize though, they need more density and scale, with better power efficiency, at lower costs. 400GbE just makes sense—for data center networks, data center interconnect (DCI), and service provider core and transport networks as well. So why don’t we see more of it out there?
First, it just takes time for vendors to align around Multi-Source Agreements (MSAs) on optical standards, which then require exhaustive testing. It can be a long, complex process, but the end result is often hugely beneficial to the entire ecosystem. See, for example, the recent QSFP56-DD pluggable form factor, which increases the bandwidth and density of optical transceivers by many times that of previous-generation optics. This new form factor also enables DWDM and Ethernet in the same physical footprint and offers backward compatibility with other QSFP formats. These kinds of MSA processes around 400GbE are ongoing, and we can expect an increase in qualified optics released in the next six months.
The bigger holdup comes down to dollars and cents. Customers need 400GbE solutions that can be deployed at a manageable price. And, while network equipment vendors and ASIC manufacturers have continually driven down cost per bit, costs for optics have not kept pace. A decade ago, 10GbE optics made up about 10% of the cost of data center networking hardware. Today, optics make up more than half.
The bulk manufacturing processes, automation, and economies of scale that allow semiconductors to get more affordable with each new generation must come to optical networking. The industry is getting there, slowly. And, as hyperscalers begin to deploy 400GbE optics at scale (likely by the end of this year), prices for everyone should start declining.
Looking ahead
As we move closer to large-scale commercial viability of 400GbE, expect to see major optical advances this year. Ongoing development of silicon photonics, for example, will further the convergence of optical transport with routing and switching, delivering pluggable transceiver capabilities on fixed configuration platforms or directly on line cards for modular platforms. Also on the horizon: 400ZR, which can transform the economics of WAN, metro, and DCI use cases by extending the reach of high-speed Ethernet and coherent optics up to 120 kilometers.
As you prepare for these advances and wait for 400GbE economics to catch up with demand, here are some considerations to think about:
- Open and interoperable, with no lock-in. Pursue platforms based on open standards-based technologies, with multi-vendor interoperability. This saves you in the long run on capital expenditures and operational flexibility.
- Getting 400GbE-ready. As you pursue network refresh cycles, look for fixed configuration or modular solutions that support QSFP56-DD interfaces for 400GbE services, so you can make the transition quickly and easily—such as by swapping out one pluggable with another.
- Inline security. For many organizations, any traffic that leaves the data center must be encrypted. Look for 400GbE solutions that can provide MACsec encryption inline, so you don’t have to use separate components that increase power consumption and costs while sapping performance.
- Telemetry at scale. To manage and monitor your network as you scale up, you need networking equipment that can scale telemetry capabilities as well. That means supporting millions of counters and many millions of filter operations per second.
In all cases, if you’re bumping up against network capacity limitations, it’s time to think seriously about 400GbE. We are fast approaching the tipping point when commercial solutions become viable. If you’re going to deliver the next generation of digital services and applications—or just support current ones more efficiently—you’ll want to make sure you are positioned to thrive and compete.
