EmbeddingGemma 2: an open, lightweight multimodal embedding model
EmbeddingGemma 2 is a 740 M‑parameter, Apache‑2.0‑licensed multimodal embedding model that natively encodes text, code, images, audio, and video into a unified space, achieving best‑in‑class scores among sub‑1 B multimodal embedders on benchmarks such as MTEB Code (78.68, a 9.92‑point gain) and MAEB. It uses a modular architecture (270 M for text‑only, plus optional 170 M vision and 300 M audio encoders) and Matryoshka Representation Learning to truncate vectors from 768 → 512/256/128 dimensions, yielding up to 6× storage reduction. On‑device quantized inference fits within ~191 MB RAM for text‑only and ~567 MB for full multimodal on a Google Pixel 11 Pro, with an 8 K token context window that can process up to 5.5 min of audio, 29 images, or 58 video frames. Engineers can therefore add privacy‑first, low‑latency cross‑modal search and retrieval to edge applications without server calls.
⚡ Key Takeaways
- 740 M parameters; modular split: 270 M (text), 170 M (vision), 300 M (audio)
- Uses Matryoshka Representation Learning to truncate embeddings to 512, 256, or 128 dimensions for up to 6× storage savings
- Quantized model runs at ~191 MB RAM (text‑only) and ~567 MB RAM (full multimodal) on a Google Pixel 11 Pro
- 8 K token context window (4× larger than EmbeddingGemma 1) supports 5.5 min audio, 29 images, or 58 video frames in a single pass
- Improves code embedding score on MTEB Code from 68.76 to 78.68 (+9.92 points)
Enables developers to embed and search across text, code, images, audio, and video directly on consumer devices, delivering privacy, low latency, and reduced storage costs for on‑device retrieval‑augmented generation pipelines.
✅ Practical Steps
- Download the EmbeddingGemma 2 model from the provided model card and load it with the appropriate framework for on‑device inference.
- Apply Matryoshka Representation Learning during embedding generation to truncate vectors to 256 or 128 dimensions for storage‑constrained vector databases.
- Quantize the model and deploy it on edge hardware (e.g., Google Pixel 11 Pro) to stay within the ~191 MB (text‑only) or ~567 MB (full multimodal) RAM budget.
Want the full story? Read the original article.
Read on Google DeepMind ↗