Google opens TorchTPU, a native PyTorch backend for cloud TPUs

Google has released TorchTPU as open source, giving developers a native way to run PyTorch models on Cloud TPU (tensor processing unit) chips. The project was created in partnership with Meta's PyTorch team and was revealed at the PyTorch Conference 2026 on October 20, 2026 ¹.

Existing PyTorch code already targeted TPUs using a standard command like torch.device("tpu"), but TorchTPU now makes TPUs a first-class device alongside GPUs and CPUs inside PyTorch itself ¹. Google recommends it for all new workloads and says it replaces the older PyTorch/XLA system .

How it works

TorchTPU connects through PyTorch's PrivateUse1 interface, which is an extension point for custom hardware backends. The result is that tensors stay as ordinary PyTorch tensors — no special subclasses or wrapper objects ¹. It also runs one process per device and keeps tensor views and in-place changes working correctly .

The old PyTorch/XLA system recorded what developers called "lazy" computation graphs, which often required extra code to mark regions that could be compiled. TorchTPU adds three eager execution modes instead: a default Strict mode for normal operation, a Debug mode with bounds checking, and a DeferAndFuse mode that groups operators together to cut memory traffic and can be over 50 percent faster than Strict ¹. For maximum speed, developers can use torch.compile with the TPU backend, which compiles code through StableHLO to OpenXLA and caches the results across restarts .

Performance and scale

All performance figures come from self-reported tests by Google and partners ¹. In training tests, Google's TorchTitan framework reached 52 percent machine flops utilization (MFU) while training the Qwen 3 8B language model on Ironwood (TPU7x) chips, and scaling was nearly linear up to 8,192 chips . In a fine-tuning example, step time dropped from 11.49 seconds in out-of-the-box eager mode to 2.78 seconds after full tuning, with torch.compile providing a 3.80x speed boost over the baseline .

Google also says it continuously tests around 5,000 Hugging Face models in both eager and compiled modes ¹. A partner called Zenteiq reported growing daily token output from 1.47 billion to 5.97 billion per 1,000 TPU v6e chips using llm-d routing on Google Kubernetes Engine .

Ecosystem integrations

The release ships with several companion projects. Hugging Face Transformers v5 and Diffusers now support TPU deployment through standard device mapping ¹. Two inference frameworks — vLLM-TorchTPU and SGLang-TorchTPU — carry forward their existing scheduling and attention features while adding custom kernels for TPU . Serving supports FP8, MXFP4 and NVFP4 checkpoint formats, along with speculative decoding . TorchTitan includes training recipes for models such as Llama 3 70B, Qwen 3 8B, Qwen 3 235B MoE and FLUX.1 12B .

The entire TorchTPU stack is open source under the Apache 2.0 license, with version 0.2.0 available on PyPI for Python 3.11 and later ¹.

Key facts

• TorchTPU is open source under Apache 2.0, version 0.2.0, co-developed by Google and Meta
• It replaces PyTorch/XLA as Google's recommended backend for new TPU workloads
• Three eager modes are available: Strict, Debug, and DeferAndFuse
• TorchTitan reached 52% MFU training Qwen 3 8B on up to 8,192 Ironwood TPUs
• Integrations include Hugging Face, vLLM-TorchTPU, SGLang-TorchTPU and TorchTitan

Sources

1. Google Open-Sources TorchTPU, a Native PyTorch Backend for Cloud TPUs — marktechpost.com