Distributed training with Ray Train DLCs and EFA on EKS
The post demonstrates scaling PyTorch training on Amazon EKS using the Ray Train Deep Learning Container (DLC) — public.ecr.aws/deep-learning-containers/ray:train-ml-cuda-v1.1 — which bundles CUDA, PyTorch 2.13.0, torchvision 0.28.0, NCCL, EFA userspace libraries, and the AWS OFI NCCL plugin. By defining a RayCluster manifest with KubeRay, eight workers (one per NVIDIA L4 GPU) run Fully Sharded Data Parallel (FSDP) to fine‑tune the Qwen2.5‑1.5B model across two g6.12xlarge instances without building a custom image. The solution leverages EFA for low‑latency inter‑node collectives and keeps the stack version‑aligned via the pre‑built DLC.
⚡ Key Takeaways
- Image tag used: public.ecr.aws/deep-learning-containers/ray:train-ml-cuda-v1.1
- Ray version supported by KubeRay: 2.58.0 (with Train, Tune, Data)
- PyTorch stack: PyTorch 2.13.0, torchvision 0.28.0 compiled for the CUDA version in the DLC
- Hardware configuration: two g6.12xlarge nodes (EFA‑enabled) providing eight NVIDIA L4 GPUs
- Training pattern: PyTorch Fully Sharded Data Parallel (FSDP) to fine‑tune Qwen2.5‑1.5B
Engineers can achieve multi‑node, GPU‑scaled PyTorch training on EKS instantly by pulling a vetted DLC, avoiding costly custom image builds and ensuring all low‑level communication stacks stay compatible.
✅ Practical Steps
- Pull the Ray Train DLC image `public.ecr.aws/deep-learning-containers/ray:train-ml-cuda-v1.1` and reference it in your RayCluster manifest.
- Deploy the KubeRay operator on your EKS cluster, then apply the provided RayCluster manifest (including the variables file) to launch a head pod and eight worker pods across the two g6.12xlarge nodes.
- Run your PyTorch FSDP training script as a Ray Train job; the DLC automatically enables NCCL over EFA for inter‑node collectives.
Want the full story? Read the original article.
Read on AWS ML Blog ↗