← Back
AWS ML Blog

Announcing SOCI-indexed images for faster SageMaker AI jobs

•12 min read•
#deployment
Level:Intermediate
For:AI Engineers
✦TL;DR

Amazon SageMaker AI now supports Seekable OCI (SOCI)‑indexed container images, allowing training and processing jobs to lazy‑load only the files needed to start the container and stream the remainder in the background. Benchmarks show up to a 90 % reduction in GPU idle time during image pulls, cutting startup from minutes to seconds for deep‑learning images that often exceed 10 GB. Engineers can adopt SOCI by referencing a pre‑indexed AWS Deep Learning Container or adding an index to a custom image, with no changes required to job definitions or entry points. This reduces billable job time and accelerates iteration cycles for hyperparameter sweeps and short‑lived jobs.

⚡ Key Takeaways

  • Measured GPU idle time reduced by up to 90 % when using SOCI‑indexed images.
  • SOCI index provides metadata mapping every file in the image, stored as an OCI‑compliant artifact in Amazon ECR.
  • Lazy‑loading eliminates the full‑image download bottleneck, turning minutes‑long starts into seconds while preserving training performance.
  • Integration requires only pointing to a SOCI‑indexed AWS Deep Learning Container or adding an index to a custom image; no job definition changes are needed.
  • If the SOCI index cannot be read, SageMaker automatically falls back to the standard full‑image download, ensuring compatibility.
💡 Why It Matters

Faster container startup directly lowers cloud costs and speeds up model development cycles, especially for hyperparameter tuning and short‑duration processing jobs where image download overhead dominates runtime.

✅ Practical Steps

  1. Point your SageMaker training or processing job to a pre‑indexed AWS Deep Learning Container that includes a SOCI index.
  2. Add a SOCI index to your own Docker image and push it to Amazon ECR as an OCI‑compliant artifact linked to the original image.

Want the full story? Read the original article.

Read on AWS ML Blog ↗

More like this

New in AWS Quick: More capable flows, an updated web experience, and agents that work together

AWS ML Blog•#agents

Three insights you may have missed from theCUBE’s coverage of the Dell Technologies New York Symposium

SiliconANGLE AI•#deployment

Welcome Inkling 2 Mini, Cutting-edge Performance for your Local Setup

Hugging Face Blog•#llm

NVIDIA and Microsoft Expand Accelerated Computing From Azure to Windows PCs

NVIDIA Blog•#compute

EXPLORE AI NEWS

Daily hand-picked stories on LLMs, RAG, agents and production AI — curated for engineers who ship.

BROWSE NEWS

GET THE WEEKLY DIGEST

Join engineers getting the Monday signal-over-noise AI breakdown. No spam, unsubscribe anytime.

LEARN AI ENGINEERING

Curated courses, research papers, repos and tutorials built for engineers leveling up in AI.

START LEARNING