Hugging Face ❤️ TorchTPU: TPUs, Natively
Hugging Face + TorchTPU introduces a native PyTorch backend that makes TPUs a first‑class device, letting you load Transformers and Diffusers models with `device_map="tpu"` and run them in eager or compiled mode. The integration adds Pallas‑tuned kernels, distributed‑training support, and a `torch.compile` path for extra speed, while TensorParallelConfig enables sharding models across multiple TPU chips. Installing TorchTPU on a TPU VM is a single pip command that pulls the CPU‑only build of torch, avoiding the larger CUDA package. Engineers can now move PyTorch workloads to Google Cloud TPUs without rewriting code for XLA or JAX.
⚡ Key Takeaways
- Load any Hugging Face model on a TPU by passing `device_map="tpu"` to `from_pretrained`.
- TorchTPU ships with Pallas kernels tuned for TPUs and supports `torch.compile` for performance gains.
- Distributed training is enabled via a TensorParallelConfig that shards the model across chips at load time.
- Installation on a TPU VM requires a single pip command that uses an extra index to fetch the CPU‑only torch build.
- Optimum TPU is no longer required for running Transformers or Diffusers on TPUs.
This removes the friction of XLA‑or‑JAX pipelines, letting PyTorch teams scale training and inference on Cloud TPUs with minimal code changes and lower deployment overhead.
✅ Practical Steps
- Apply the concepts from this article to your own system design.
Want the full story? Read the original article.
Read on Hugging Face Blog ↗