← Back
Amazon Science

Mitra-v2: Scaling synthetic pretraining for tabular foundation models

•5 min read•
#llm#deployment
Mitra-v2: Scaling synthetic pretraining for tabular foundation models
Level:Advanced
For:AI Engineers
✦TL;DR

Mitra‑v2 is the second‑generation tabular foundation model that scales synthetic pretraining to tables ten times larger in rows and three times wider in columns than its predecessor, while retaining the same Transformer architecture. On the TabArena benchmark of 51 real‑world datasets, Mitra‑v2 outperforms Mitra‑v1 on 32 of the 33 datasets where both run natively and completes all 51 datasets within the one‑hour budget that v1 missed on over a third of the tasks. Fine‑tuned Mitra‑v2 also ranks first on the ScoringBench regression benchmark, and the model, inference code, and fine‑tuning scripts are released under Apache‑2.0 on Hugging Face, enabling engineers to apply in‑context learning to new tables without additional training.

⚡ Key Takeaways

  • Mitra‑v2 is pretrained on synthetic tables with 10× more rows and ≈3× more columns than Mitra‑v1.
  • Architecture: unchanged Transformer that attends across rows and columns.
  • Performance: leads the single‑model field on TabArena, better on 32/33 comparable datasets and finishes all 51 datasets within the benchmark’s one‑hour limit.
  • Integration: fine‑tuned classification and regression versions, plus inference and fine‑tuning code, are available on Hugging Face under Apache‑2.0.
  • Prerequisite: requires providing labeled rows of a target table at inference time for in‑context learning; no real pretraining data is used.
💡 Why It Matters

Engineers can now deploy a single pretrained model to predict missing attributes across diverse, large‑scale business tables, cutting the need for per‑table model development and reducing latency by staying within strict inference budgets.

✅ Practical Steps

  1. Clone the Hugging Face repository for Mitra‑v2 and use the provided inference script to feed labeled rows of your target table for in‑context predictions.
  2. Fine‑tune the released classification or regression checkpoint on your specific task using the supplied fine‑tuning code before deployment.

Want the full story? Read the original article.

Read on Amazon Science ↗

More like this

Introducing workload recovery and expanded infrastructure checks on Amazon SageMaker HyperPod

AWS ML Blog•#deployment

Introducing Unity Gateway Insights

Databricks Blog•#deployment

Introducing Gemini 4 Neon

Google DeepMind•#llm

DaisyRose-27B: Pushing the Frontier of Post-Training Efficiency

Hugging Face Blog•#llm

EXPLORE AI NEWS

Daily hand-picked stories on LLMs, RAG, agents and production AI — curated for engineers who ship.

BROWSE NEWS

GET THE WEEKLY DIGEST

Join engineers getting the Monday signal-over-noise AI breakdown. No spam, unsubscribe anytime.

LEARN AI ENGINEERING

Curated courses, research papers, repos and tutorials built for engineers leveling up in AI.

START LEARNING