Train Qwen3 on AWS Trainium with torchtitan
This walkthrough demonstrates three ways to train the Qwen3‑4B‑Instruct‑2507 model on a single trn2.48xlarge Trainium instance using torchtitan‑neuron, including a Transformers v5 backend, torchtitan’s native Qwen3 spec, and a custom MoE kernel via the Neuron Kernel Interface. All methods share the same torchrun launch pattern and require specific pinned versions of torch 2.14.0, torch_neuronx 2.14.3, transformers 5.16.0, and torchtitan‑neuron release‑2.33 (0.2.2+neuron.1). Engineers can reproduce the results by cloning torchtitan‑neuron, installing dependencies with --no‑deps, and downloading the model from Hugging Face Hub.
⚡ Key Takeaways
- Version table: torch 2.14.0, torch_neuronx 2.14.3, transformers 5.16.0, torchtitan‑neuron release‑2.33 (0.2.2+neuron.1).
- Use TorchNeuron’s native device registration so torchtitan sees Trainium as a standard PyTorch device.
- Custom MoE kernels are attached via torchtitan’s override registry and the NKI mixture‑of‑experts recipes.
- Launch training with a standard torchrun command and a Python config function (e.g., transformers_modeling_backend_full()).
- Requires a single trn2.48xlarge (16 Trainium2 accelerators, 128 physical NeuronCores, 64 logical cores at LNC=2).
By leveraging torchtitan‑neuron’s seamless PyTorch integration and custom kernel overrides, engineers can achieve high‑performance training of large LLMs on AWS Trainium without rewriting training loops, accelerating both pre‑training and fine‑tuning pipelines.
✅ Practical Steps
- Clone and install torchtitan‑neuron preserving the Neuron‑built torch:
- `git clone https://github.com/aws-neuron/torchtitan-neuron && cd torchtitan-neuron && pip install -r requirements.txt --no-deps`
- Download the Qwen3 model artifacts:
- `hf download Qwen/Qwen3-4B-Instruct-2507 --local-dir "$PWD/assets/hf/Qwen3-4B-Instruct-2507"`
Want the full story? Read the original article.
Read on AWS ML Blog ↗