← Back
AWS ML Blog

Train Qwen3 on AWS Trainium with torchtitan

•11 min read•
#deployment#llm#compute
Level:Advanced
For:AI Engineers
✦TL;DR

This walkthrough demonstrates three ways to train the Qwen3‑4B‑Instruct‑2507 model on a single trn2.48xlarge Trainium instance using torchtitan‑neuron, including a Transformers v5 backend, torchtitan’s native Qwen3 spec, and a custom MoE kernel via the Neuron Kernel Interface. All methods share the same torchrun launch pattern and require specific pinned versions of torch 2.14.0, torch_neuronx 2.14.3, transformers 5.16.0, and torchtitan‑neuron release‑2.33 (0.2.2+neuron.1). Engineers can reproduce the results by cloning torchtitan‑neuron, installing dependencies with --no‑deps, and downloading the model from Hugging Face Hub.

⚡ Key Takeaways

  • Version table: torch 2.14.0, torch_neuronx 2.14.3, transformers 5.16.0, torchtitan‑neuron release‑2.33 (0.2.2+neuron.1).
  • Use TorchNeuron’s native device registration so torchtitan sees Trainium as a standard PyTorch device.
  • Custom MoE kernels are attached via torchtitan’s override registry and the NKI mixture‑of‑experts recipes.
  • Launch training with a standard torchrun command and a Python config function (e.g., transformers_modeling_backend_full()).
  • Requires a single trn2.48xlarge (16 Trainium2 accelerators, 128 physical NeuronCores, 64 logical cores at LNC=2).
💡 Why It Matters

By leveraging torchtitan‑neuron’s seamless PyTorch integration and custom kernel overrides, engineers can achieve high‑performance training of large LLMs on AWS Trainium without rewriting training loops, accelerating both pre‑training and fine‑tuning pipelines.

✅ Practical Steps

  1. Clone and install torchtitan‑neuron preserving the Neuron‑built torch:
  2. `git clone https://github.com/aws-neuron/torchtitan-neuron && cd torchtitan-neuron && pip install -r requirements.txt --no-deps`
  3. Download the Qwen3 model artifacts:
  4. `hf download Qwen/Qwen3-4B-Instruct-2507 --local-dir "$PWD/assets/hf/Qwen3-4B-Instruct-2507"`

Want the full story? Read the original article.

Read on AWS ML Blog ↗

More like this

Introducing workload recovery and expanded infrastructure checks on Amazon SageMaker HyperPod

AWS ML Blog•#deployment

Introducing Unity Gateway Insights

Databricks Blog•#deployment

Introducing Gemini 4 Neon

Google DeepMind•#llm

DaisyRose-27B: Pushing the Frontier of Post-Training Efficiency

Hugging Face Blog•#llm

EXPLORE AI NEWS

Daily hand-picked stories on LLMs, RAG, agents and production AI — curated for engineers who ship.

BROWSE NEWS

GET THE WEEKLY DIGEST

Join engineers getting the Monday signal-over-noise AI breakdown. No spam, unsubscribe anytime.

LEARN AI ENGINEERING

Curated courses, research papers, repos and tutorials built for engineers leveling up in AI.

START LEARNING