51.2 Tbps Switch ASICs to Drive the Migration to 800 Gbps and 1.6 Tbps Starting in 2H22
NVIDIA continues to drive products and solutions at a rapid pace. They continued to push the boundaries and accelerate the pace of innovation in the data center through the recent GTC announcements. With the launch of Spectrum-4, we got to see NVIDIA’s first Ethernet Switch announcement and what they were busy developing since acquiring Mellanox. Keeping with the accelerating theme, NVIDIA clearly accelerated the pace of Innovation and proudly showcased Spectrum-4 at GTC.
Over the next five years, 51.2 Tbps ASICs will be responsible for over $20B in Data Center Switching revenue. About 2X the size of the 100 Gbps upgrade cycle and 4X the size of the 400 Gbps upgrade cycle. This class of ASIC is critical to allowing data centers to scale as more workloads become hardware accelerated. Hardware acceleration can come via the server, NiC, or DPU and causes networking bandwidth to double every year compared to the more traditional growth of 30-40%.
NVIDIA’s 51.2 Tbps ASIC
NVIDIA’s Spectrum-4 announcement included many new and first-to-market capabilities. It was the first 51.2 Tbps ASIC announced in the market and the first switch that can do 64 ports of 800 Gbps from a single ASIC. It also supports 1.6 Tbps speeds, another first and the preferred speed for the two most prominent Hyperscalers.
In addition, NVIDIA announced Spectrum-4 as a fully integrated Switch that will sample later in 2022 and begin shipping in 2023. The chip will be produced at TSMC in the 4N process. NVIDIA did over $1B in Ethernet Switch and NIC revenue in 2021.
It's All About The SERDES
NVIDIA will be using its own, homegrown, 112 Gbps SERDES. SERDES has always been a challenge in switch development, with many ASIC companies having to integrate 3rd party SERDES. However, we view vertically integrated as a source of differentiation and will become more common in the future as ASICs increase in speed. SERDES will also be a key to adding additional IP blocks and photonics to the networking ASIC, as those markets evolve.
The Speeds And Feeds Race
NVIDIA’s Spectrum-4 introduced several features that help it distinguish and route traffic. These traffic classes are often described as elephant and mouse flows. The key at a market level is that AI and accelerated computing put a considerable burden on the network. So AI networking switches need to keep pace with this new class of ‘elephant flow’ traffic and not slow or drop packets. At the same time, accelerated traffic is growing as a percent of traffic. Multiple 100G ports are already standard at the RFP level for AI workloads. 12.8 Tbps and 25.6 Tbps ASICs can not keep pace with a fully loaded AI cluster.
While the announcement focused on NVIDIA examples, such as connecting Spectrum-4 to NVIDIA Certified OVX OEM servers, we view the market as benefiting from NVIDIA taking networking seriously. Cloud customers have always requested multi-vendor at the ASIC and system level, and the ability to collapse networking tiers with 51.2 Tbps will be a critical driver to adoption. We, therefore, expect Spectrum-4 to apply to the whole data center, not just focused on the high-end or NVIDIA-connected solutions. We are excited to see this switch show up in 2H22 in the customer’s hands.
Our supply chain checks across the technology and cloud sector continue to turn pessimistic during the early days of October. As a result, orders for semiconductors that were thought to be secured and guaranteed are being cut. The implications are clear that most segments we track are in for several bumpy months, with all efforts to bring 2022 back to normal.
Component suppliers are cutting or negotiating back existing commitments – Factory shutdowns, lack of fab capacity, and logistics gridlock continue to hamper the ability of system vendors to get adequate supplies. Vendors continue to burn through their stock, and no vendor can ship a 99% complete product. Vendor resources, which should be on 2022 product launches and securing capacity for future product cycles, are instead spent figuring out what to ship and adjusting components in existing product lines. In general, vendors are shipping fewer varieties of SKUs now than at any point in the past 20 years.
Prices are increasing for components and, in some cases, will never return to the previous price – While some price increases could be classified as temporary or transitionary like rush charges, air freight, and redesigns, many events are leading to more permanent price increases. At the very least, vendors will dual source suppliers, manufacturing across many geographic regions leading to lower volume with each supplier. As a result, we view some semiconductor components as never returning to pre-COVID price levels. So the big question remains when products will return to historic price erosion?
2022 lacks new product designs – In 2021, there were very few new product introductions and downsizing in SKU offerings. In some markets, we track there was minimal impact, but in other areas like 400/800G, the lack of clarity is causing many hyperscalers to reevaluate their speed transitions.
We expect a different tone in earnings season as vendors, component suppliers, and customers adjust to the new normal of not getting every type of product they want over the next several quarters. As a result, some projects will be delayed, others scaled back, and spare capacity at AWS/Azure/GCP will be put to the test as many enterprises embrace capacity to buffer shortfalls in premises-based hardware.