← Back
Hugging Face Blog

Run fast, private agents on-device with LFM2.5-24B-A2B

•5 min read•
#agents#llm#deployment
Level:Advanced
For:AI Engineers
✦TL;DR

LFM2.5-24B-A2B is a 24‑billion‑parameter MoE reasoning model that activates only 2 B parameters per token, enabling cheap, continuous on‑device agent inference. It matches the 35 B‑parameter Qwen3.6‑35B‑A3B on tool‑use and agentic benchmarks (48.09 vs 38.19 on LiveClaw) while running at 184 tok/s on an Apple M5 Max and up to 21 K tok/s on a single NVIDIA B300 GPU. Engineers can deploy it today via llama.cpp, MLX, vLLM, SGLang, or ONNX for fast, private on‑device agents.

⚡ Key Takeaways

  • LiveClaw score: 48.09 (vs 38.19 for Qwen3.6‑35B‑A3B)
  • Architecture: 40‑layer MoE with 64 experts per block, top‑4 routing, gated short convolutions, and GQA; 2 B active parameters per token
  • Trade‑off: 2 B active parameters give cheap reasoning tokens but larger models still win knowledge‑heavy benchmarks (GPQA Diamond, HLE)
  • Integration: Supported out‑of‑the‑box by llama.cpp, MLX, vLLM, SGLang, and ONNX
  • Limitation: For knowledge‑intensive tasks, pair with retrieval or web search as the model lags larger peers
💡 Why It Matters

The model delivers agentic reasoning speed on consumer hardware, letting teams ship private, tool‑using assistants without cloud latency or cost overhead.

✅ Practical Steps

  1. Deploy LFM2.5-24B-A2B with vLLM (or llama.cpp, MLX, SGLang, ONNX) to run inference on your device.
  2. Use the model within existing agent harnesses such as OpenClaw, Hermes Agent, Pi, or OpenCode for tool‑use and multi‑step tasks.
  3. For knowledge‑heavy queries, augment the agent with a retrieval or web‑search component before invoking the model.

Want the full story? Read the original article.

Read on Hugging Face Blog ↗

More like this

Censored vs. Truncated Regression in Python, Explained

Towards Data Science•#llm

New in AWS Quick: More capable flows, an updated web experience, and agents that work together

AWS ML Blog•#agents

Three insights you may have missed from theCUBE’s coverage of the Dell Technologies New York Symposium

SiliconANGLE AI•#deployment

How Google Duplex evolved to also call businesses for you in Gemini

Google Research•#agents

EXPLORE AI NEWS

Daily hand-picked stories on LLMs, RAG, agents and production AI — curated for engineers who ship.

BROWSE NEWS

GET THE WEEKLY DIGEST

Join engineers getting the Monday signal-over-noise AI breakdown. No spam, unsubscribe anytime.

LEARN AI ENGINEERING

Curated courses, research papers, repos and tutorials built for engineers leveling up in AI.

START LEARNING