← Back
Hugging Face Blog

Hugging Face ❤️ TorchTPU: TPUs, Natively

•5 min read•
#compute#deployment#llm
Level:Intermediate
For:AI Engineers
✦TL;DR

Hugging Face + TorchTPU introduces a native PyTorch backend that makes TPUs a first‑class device, letting you load Transformers and Diffusers models with `device_map="tpu"` and run them in eager or compiled mode. The integration adds Pallas‑tuned kernels, distributed‑training support, and a `torch.compile` path for extra speed, while TensorParallelConfig enables sharding models across multiple TPU chips. Installing TorchTPU on a TPU VM is a single pip command that pulls the CPU‑only build of torch, avoiding the larger CUDA package. Engineers can now move PyTorch workloads to Google Cloud TPUs without rewriting code for XLA or JAX.

⚡ Key Takeaways

  • Load any Hugging Face model on a TPU by passing `device_map="tpu"` to `from_pretrained`.
  • TorchTPU ships with Pallas kernels tuned for TPUs and supports `torch.compile` for performance gains.
  • Distributed training is enabled via a TensorParallelConfig that shards the model across chips at load time.
  • Installation on a TPU VM requires a single pip command that uses an extra index to fetch the CPU‑only torch build.
  • Optimum TPU is no longer required for running Transformers or Diffusers on TPUs.
💡 Why It Matters

This removes the friction of XLA‑or‑JAX pipelines, letting PyTorch teams scale training and inference on Cloud TPUs with minimal code changes and lower deployment overhead.

✅ Practical Steps

  1. Apply the concepts from this article to your own system design.

Want the full story? Read the original article.

Read on Hugging Face Blog ↗

More like this

Introducing workload recovery and expanded infrastructure checks on Amazon SageMaker HyperPod

AWS ML Blog•#deployment

Introducing Unity Gateway Insights

Databricks Blog•#deployment

Introducing Gemini 4 Neon

Google DeepMind•#llm

DaisyRose-27B: Pushing the Frontier of Post-Training Efficiency

Hugging Face Blog•#llm

EXPLORE AI NEWS

Daily hand-picked stories on LLMs, RAG, agents and production AI — curated for engineers who ship.

BROWSE NEWS

GET THE WEEKLY DIGEST

Join engineers getting the Monday signal-over-noise AI breakdown. No spam, unsubscribe anytime.

LEARN AI ENGINEERING

Curated courses, research papers, repos and tutorials built for engineers leveling up in AI.

START LEARNING