Google Cloud Splits Its AI Chip Strategy, But Nvidia's Grip Remains Firm

Google Cloud has unveiled a new, bifurcated generation of its custom AI processors. The company's eighth-generation Tensor Processing Units (TPUs) will come in two distinct flavors: the TPU 8t, designed for training complex models, and the TPU 8i, optimized for the ongoing task of running those models, known as inference.

Google claims significant leaps over its prior hardware. The new chips promise to train models up to three times faster while delivering an 80% improvement in performance per dollar. A key engineering feat allows over one million of these TPUs to be linked in a single cluster, aiming to provide more computational power at a lower energy and cost footprint for customers.

Despite these advances, this is not a wholesale replacement for Nvidia. Google, like its cloud rivals Amazon and Microsoft, is using its proprietary silicon to supplement, not supplant, the Nvidia GPUs that remain a cornerstone of its infrastructure. Google confirmed it will offer Nvidia's forthcoming Vera Rubin chips later this year. The relationship is, in some ways, collaborative; the companies are engineering improvements to Google's Falcon networking software to make Nvidia systems run more efficiently on its cloud.

This dynamic highlights a complex market reality. While cloud providers develop internal alternatives, Nvidia's dominance is not currently under threat. As chip analyst Patrick Moore noted, predictions of Nvidia's decline at the hands of Google's first TPU in 2016 proved spectacularly wrong. Nvidia's market value now approaches $5 trillion. For the foreseeable future, the expansion of AI in the cloud—even on custom chips—likely drives more business to Nvidia, as the ecosystem it anchors continues to grow.

Source: TechCrunch