Mitra-v2: Scaling synthetic pretraining for tabular foundation models
Mitra‑v2 is the second‑generation tabular foundation model that scales synthetic pretraining to tables ten times larger in rows and three times wider in columns than its predecessor, while retaining the same Transformer architecture. On the TabArena benchmark of 51 real‑world datasets, Mitra‑v2 outperforms Mitra‑v1 on 32 of the 33 datasets where both run natively and completes all 51 datasets within the one‑hour budget that v1 missed on over a third of the tasks. Fine‑tuned Mitra‑v2 also ranks first on the ScoringBench regression benchmark, and the model, inference code, and fine‑tuning scripts are released under Apache‑2.0 on Hugging Face, enabling engineers to apply in‑context learning to new tables without additional training.
⚡ Key Takeaways
- Mitra‑v2 is pretrained on synthetic tables with 10× more rows and ≈3× more columns than Mitra‑v1.
- Architecture: unchanged Transformer that attends across rows and columns.
- Performance: leads the single‑model field on TabArena, better on 32/33 comparable datasets and finishes all 51 datasets within the benchmark’s one‑hour limit.
- Integration: fine‑tuned classification and regression versions, plus inference and fine‑tuning code, are available on Hugging Face under Apache‑2.0.
- Prerequisite: requires providing labeled rows of a target table at inference time for in‑context learning; no real pretraining data is used.
Engineers can now deploy a single pretrained model to predict missing attributes across diverse, large‑scale business tables, cutting the need for per‑table model development and reducing latency by staying within strict inference budgets.
✅ Practical Steps
- Clone the Hugging Face repository for Mitra‑v2 and use the provided inference script to feed labeled rows of your target table for in‑context predictions.
- Fine‑tune the released classification or regression checkpoint on your specific task using the supplied fine‑tuning code before deployment.
Want the full story? Read the original article.
Read on Amazon Science ↗