TorchNeuron makes Hugging Face models run natively on AWS Trainium chips
A new PyTorch backend removes the compiler complexity that once kept Hugging Face models off Amazon's custom AI hardware.

A software plugin that lets AI models run directly on Amazon's custom chips using PyTorch, the same way they run on regular graphics cards.
A native path to Trainium
For years, running Hugging Face models on AWS Trainium chips required working through an indirect layer called optimum-neuron, which relied on XLA (Accelerated Linear Algebra), an open-source compiler that transforms machine-learning computation graphs into efficient code 1. That setup forced developers to think in terms of lazy execution, fixed tensor shapes, and constant recompilation whenever code changed. The experience was functional but fragile, and it kept many HF models off the platform.
TorchNeuron flips that approach. Released as a public beta on October 20, 2026, TorchNeuron is a PyTorch plugin that treats Neuron devices — AWS's custom AI accelerators — as first-class hardware, just like GPUs. It brings eager execution, torch.compile, and native distributed primitives such as FSDP and tensor parallelism directly into the PyTorch ecosystem.
What changes for Hugging Face users
The most visible shift is that HF libraries now run on Trainium without special wrappers. Transformers, Diffusers, Accelerate, PEFT and TRL all operate on Neuron devices using the same classes and scripts developers already use on NVIDIA hardware.
Inference has seen the largest jump in support. TorchNeuron now covers 89 percent of the top 100 most-used text-generation model architectures out of the box, compared with 46 percent under the older optimum-neuron path 1. To run a model, users simply pass a device-map argument pointing to neuron, and the model loads onto the accelerator.
For larger deployments, tensor parallelism lets a single model span multiple Trainium units, while a static KV-cache strategy avoids the frequent recompilations that previously hurt generation speed.
Training without custom code
Training on Trainium used to require Neuron-specific forks or dedicated trainers. TorchNeuron removes that barrier. A standard TRL supervised fine-tuning script runs on a Trainium instance with only minor hardware-specific settings, the same way a script would run on a GPU cluster.
The change matters because it closes a gap between GPU and Trainium workflows. Developers who already use Hugging Face libraries for fine-tuning and evaluation can now target AWS's cost-effective training hardware without learning a separate stack.
AWS and Hugging Face recommend that existing optimum-neuron users migrate to the native path going forward, citing the broader architecture support and simpler code as reasons to switch.
Key facts
- TorchNeuron public beta launched October 20, 2026
- Text-generation architecture support rose from 46% to 89% of the top 100 models
- TorchNeuron is installed via pip install torch-neuronx
- Hugging Face now recommends native libraries over optimum-neuron for Trainium
- Training uses standard TRL scripts instead of Neuron-specific forks
Sources
If this story concerns you or your company, or uses something that belongs to you, ask for a correction or for it to be taken down. How requests work
Keep reading
Amazon, Microsoft, and Google revealed very different AI plans in their latest earnings reports
Cloud providers are betting big on AI, but they're doing it in three completely different ways — and that matters for which services your company will use.
Apps and productsAlter gives AI agents their own keys and an audit trailHerald of AI
Alter gives AI agents their own keys and an audit trail
A new Y Combinator startup is building access control so every AI agent acts with its own identity instead of sharing a single overpowered credential.
Apps and productsllama.cpp v1.0.0 brings stability to the growing world of local AIHerald of AI
llama.cpp v1.0.0 brings stability to the growing world of local AI
The popular open-source tool that lets anyone run AI models on their own computer has reached version 1.0, making it more reliable for developers building private AI apps.