← Back
AWS ML Blog

Distributed training with Ray Train DLCs and EFA on EKS

•18 min read•
#deployment#compute
Level:Advanced
For:AI Engineers
✦TL;DR

The post demonstrates scaling PyTorch training on Amazon EKS using the Ray Train Deep Learning Container (DLC) — public.ecr.aws/deep-learning-containers/ray:train-ml-cuda-v1.1 — which bundles CUDA, PyTorch 2.13.0, torchvision 0.28.0, NCCL, EFA userspace libraries, and the AWS OFI NCCL plugin. By defining a RayCluster manifest with KubeRay, eight workers (one per NVIDIA L4 GPU) run Fully Sharded Data Parallel (FSDP) to fine‑tune the Qwen2.5‑1.5B model across two g6.12xlarge instances without building a custom image. The solution leverages EFA for low‑latency inter‑node collectives and keeps the stack version‑aligned via the pre‑built DLC.

⚡ Key Takeaways

  • Image tag used: public.ecr.aws/deep-learning-containers/ray:train-ml-cuda-v1.1
  • Ray version supported by KubeRay: 2.58.0 (with Train, Tune, Data)
  • PyTorch stack: PyTorch 2.13.0, torchvision 0.28.0 compiled for the CUDA version in the DLC
  • Hardware configuration: two g6.12xlarge nodes (EFA‑enabled) providing eight NVIDIA L4 GPUs
  • Training pattern: PyTorch Fully Sharded Data Parallel (FSDP) to fine‑tune Qwen2.5‑1.5B
💡 Why It Matters

Engineers can achieve multi‑node, GPU‑scaled PyTorch training on EKS instantly by pulling a vetted DLC, avoiding costly custom image builds and ensuring all low‑level communication stacks stay compatible.

✅ Practical Steps

  1. Pull the Ray Train DLC image `public.ecr.aws/deep-learning-containers/ray:train-ml-cuda-v1.1` and reference it in your RayCluster manifest.
  2. Deploy the KubeRay operator on your EKS cluster, then apply the provided RayCluster manifest (including the variables file) to launch a head pod and eight worker pods across the two g6.12xlarge nodes.
  3. Run your PyTorch FSDP training script as a Ray Train job; the DLC automatically enables NCCL over EFA for inter‑node collectives.

Want the full story? Read the original article.

Read on AWS ML Blog ↗

More like this

Introducing workload recovery and expanded infrastructure checks on Amazon SageMaker HyperPod

AWS ML Blog•#deployment

Introducing Unity Gateway Insights

Databricks Blog•#deployment

Introducing Gemini 4 Neon

Google DeepMind•#llm

DaisyRose-27B: Pushing the Frontier of Post-Training Efficiency

Hugging Face Blog•#llm

EXPLORE AI NEWS

Daily hand-picked stories on LLMs, RAG, agents and production AI — curated for engineers who ship.

BROWSE NEWS

GET THE WEEKLY DIGEST

Join engineers getting the Monday signal-over-noise AI breakdown. No spam, unsubscribe anytime.

LEARN AI ENGINEERING

Curated courses, research papers, repos and tutorials built for engineers leveling up in AI.

START LEARNING