The short version
PyTorch code can now target Google’s TPUs natively, without a separate porting layer. Google released TorchTPU as open source on October 20, 2026 at PyTorch Conference, co-developed with Meta’s PyTorch team. Existing PyTorch training and serving code moves to Cloud TPUs with minimal changes, so teams can compare TPU, GPU and Trainium on cost and quality before they commit budget.
What did Google release?
TorchTPU registers the TPU as a first-class PyTorch device, so code that today calls .to("cuda") can call .to("tpu") and keep its training loop intact. Until now, PyTorch on TPUs ran through the separate PyTorch/XLA layer, which many teams found harder to debug than the GPU path. Google’s announcement, written by Mark Lohmeyer and Bill Jia, says TPUs now “join GPUs and CPUs as a native option for PyTorch.”
The backend supports eager execution for development and torch.compile for production performance. Alongside it, Google published an early preview of the TPU Agent Suite, an AI coding agent that helps with onboarding, error diagnosis and inference tuning, and moved LiteRT (formerly TensorFlow Lite) into the PyTorch ecosystem for exporting models to phones, laptops and embedded devices.
Which tools already work with TorchTPU?
The ecosystem is the real story. TechCrunch reports that Google worked directly with Meta, Hugging Face, Inferact (the company behind vLLM) and RadixArk (the maker of SGLang) before launch. According to Google, the supported stack at release looks like this:
- Training: TorchTitan recipes for Llama 3, Qwen 3 and FLUX.1; VeRL for reinforcement-learning post-training (preview); Ray for mixed CPU, GPU and TPU jobs.
- Serving: TorchTPU backends for vLLM and SGLang, plus llm-d 1.0, the CNCF Sandbox project for distributed inference backed by Red Hat and IBM.
- Models: Hugging Face Transformers v5 deploys to TPUs with
device_map="tpu"; Diffusers and TorchRec are also covered.
Google cites one customer result: Zenteiq reported 4x higher aggregate inference throughput with llm-d, raising daily capacity from 1.47 billion to 5.97 billion tokens per 1,000 TPU v6e chips.
Why does a second PyTorch port matter?
Nvidia’s lead rests heavily on CUDA being the default target for PyTorch. Amazon attacked that default on October 19 with TorchNeuron for Trainium; Google followed a day later. With two hyperscalers shipping native backends and upstreaming work into PyTorch itself, hardware-neutral PyTorch is becoming the expected baseline rather than a vendor experiment.
For buyers, the effect is leverage. When the same model code runs on three accelerator families, GPU reservations, TPU commitments and Trainium capacity become comparable line items in a negotiation.
What is still unproven?
The utilization and throughput figures come from Google and its partners, not independent benchmarks. In its April technical preview, Google itself noted that changing sequence lengths and batch sizes trigger recompilation, and that peak TPU performance can still require model refactoring. Custom CUDA kernels will not move with a device flag, and TPUs are available only in Google Cloud, so the portability gain is between chips, not between clouds.
What it means for US & EU software teams
Teams that train, fine-tune or serve models in plain PyTorch now have a realistic third option for steady, high-volume workloads such as batch inference, embeddings and fine-tuning runs. The deciding factors will be cost per token or per training run on your own models, the latency profile of serving through vLLM or SGLang, and capacity in the regions you are allowed to use.
The same discipline that made Trainium testable applies here: device selection in configuration, no hard-coded CUDA calls in the main path, and a reproducible evaluation set. EU teams should check TPU v6e and Ironwood availability in approved EU regions before planning a move and record any new processing location in GDPR documentation.
What ML teams should do this quarter
- Inventory CUDA lock-in. List custom kernels, CUDA-only libraries and hard-coded device calls across training and serving code.
- Make the device a setting. Route every pipeline’s accelerator choice through config so GPU, TPU and Trainium runs use identical code.
- Pilot serving first. Benchmark one production model through vLLM or SGLang on TPU against your current GPU setup, measuring latency, throughput and cost per million tokens.
- Test dynamic shapes. Replay real traffic with varied prompt lengths to see whether recompilation affects latency.
- Check regions and commitments. Confirm TPU capacity in your approved regions and how a shift would interact with existing GPU or AWS commitments.
Frequently asked questions
What is Google TorchTPU?
TorchTPU is an open-source PyTorch backend that runs PyTorch models natively on Google Cloud TPUs. Google released it publicly on October 20, 2026, after co-developing it with Meta’s PyTorch team.
How much code do I need to change to run PyTorch on TPUs?
Google says the change is minimal: code that targets the GPU device can target the TPU device instead, and Hugging Face Transformers v5 uses device_map="tpu". Custom CUDA kernels and CUDA-only libraries still need porting.
How is TorchTPU different from Amazon TorchNeuron?
Both make PyTorch run natively on a non-Nvidia chip. TorchTPU targets Google Cloud TPUs and ships with an AI porting agent and LiteRT for edge export; TorchNeuron, announced a day earlier, targets AWS Trainium. Each only runs inside its own cloud.
Is TorchTPU ready for production?
Google and Meta describe production use, and llm-d 1.0 is labeled production-ready, but the performance figures are vendor claims. Run a pilot on your own models and traffic before moving production workloads.
Sources
Google Cloud — TorchTPU is now open source: run native PyTorch on TPUs (October 20, 2026)
TechCrunch — Google has joined Amazon in attacking Nvidia’s stronghold with AI app developers (October 20, 2026)
Google Developers Blog — TorchTPU: Running PyTorch natively on TPUs at Google scale (April 7, 2026)
Techzine — Google and Meta are making AI chips more accessible with TorchTPU (October 21, 2026)