AWS Launches TorchNeuron to Loosen Nvidia's Grip on AI Development

Amazon Web Services unveiled a new tool Monday called TorchNeuron that lets AI startups shift their applications onto Trainium, Amazon's own AI chip, with minimal effort.

The tool works with PyTorch, the free open-source framework developers use to build and train AI models. According to Amazon, moving a PyTorch app to Trainium requires changing just one line of configuration, with no code rewrite. Amazon says roughly 85% of AI apps are built with PyTorch, which Facebook originally developed and which has been part of the Linux Foundation since 2022.

Nvidia has spent years making its CUDA software work smoothly with PyTorch, which has helped make its hardware the default choice for AI work. TorchNeuron is adapted from PyTorch itself, so developers already familiar with the framework can use it. Amazon isn't first to try this: PyTorch users can also move apps to AMD GPUs or Apple chips with a simple configuration change.

Amazon argues that porting to Trainium also plugs startups into its broader cloud. Colin Brace, VP of Technology Software at AWS Annapurna Labs, said building on AWS means "you're not betting on a single-purpose compute provider and scrambling to build everything else around it." AWS says the same chip handles both training and inference, which it claims improves economics across a workload. Brace pointed to AI voice provider Deepgram, which he said moved to Trainium using native PyTorch and is processing 70% more audio per dollar at production-grade accuracy.

Amazon also said it is contributing code upstream to PyTorch to remove ties between the framework and specific chips, aiming to make it more hardware-agnostic for all developers. Brace framed the effort as "removing the lock-in that kept developers from having a real choice at all." Whether that delivers more choice and lower costs, or just more configuration headaches, remains to be seen.

Source: TechCrunch