← Back
Hugging Face Blog

Build specialized multimodal models with LFM3

•9 min read•
#llm#agents#deployment
Level:Advanced
For:AI Engineers
✦TL;DR

LFM3 introduces a new tri‑hybrid 1.7 B‑parameter backbone that outperforms the larger 2.5‑2.6 B LFM2 models while using 35 % fewer parameters, and extends the context window to 256 K tokens. Six 2 B‑class modality models (text, vision, audio, omni, and decision) are derived from this backbone, each fine‑tunable on a single GPU and deployable on‑device with a 1 GB 4‑bit text model. Architectural innovations include gated short convolutions, sliding‑window attention, head‑wise gated attention, partial RoPE, and sandwich norm, delivering a 25 % speed gain at 8 K token context. Engineers can now build specialized multimodal agents that run locally across edge hardware from phones to Raspberry Pi 5.

⚡ Key Takeaways

  • LFM3‑1.7B matches LFM2‑1.2B on short prompts per parameter and is ~25 % faster at 8 K token context.
  • Tri‑hybrid layer mix: 13 gated short convolutions (kernel 7), 5 sliding‑window attention layers (256‑token window), 6 global attention layers.
  • Head‑wise gated attention uses a sigmoid gate per head to suppress activation outliers, aiding quantization.
  • Vision encoder (164 M parameters) uses 3× less compute than SigLIP2 and is 2.2× more FLOP‑efficient than Qwen3.5‑2B’s encoder while matching its accuracy.
  • All models fine‑tune on a single GPU and the text model fits in ~1 GB at 4‑bit precision, enabling on‑device deployment.
💡 Why It Matters

The compact, high‑capability backbone lets production teams ship multimodal AI features to edge devices without cloud dependence, reducing latency and cost while preserving agentic reasoning over very long contexts.

✅ Practical Steps

  1. Fine‑tune the desired LFM3 modality model on your dataset using a single GPU as described.
  2. Deploy the 4‑bit quantized text model (~1 GB) on your target edge hardware (e.g., phone, Raspberry Pi 5) for on‑device inference.
  3. Utilize the vision encoder’s 3× compute reduction to process images/video locally, leveraging its temporal slicing for video frames.

Want the full story? Read the original article.

Read on Hugging Face Blog ↗

More like this

Censored vs. Truncated Regression in Python, Explained

Towards Data Science•#llm

New in AWS Quick: More capable flows, an updated web experience, and agents that work together

AWS ML Blog•#agents

Three insights you may have missed from theCUBE’s coverage of the Dell Technologies New York Symposium

SiliconANGLE AI•#deployment

How Google Duplex evolved to also call businesses for you in Gemini

Google Research•#agents

EXPLORE AI NEWS

Daily hand-picked stories on LLMs, RAG, agents and production AI — curated for engineers who ship.

BROWSE NEWS

GET THE WEEKLY DIGEST

Join engineers getting the Monday signal-over-noise AI breakdown. No spam, unsubscribe anytime.

LEARN AI ENGINEERING

Curated courses, research papers, repos and tutorials built for engineers leveling up in AI.

START LEARNING