← Back
Amazon Science

Can we change an AI model’s architecture without retraining it from scratch?

•13 min read•
#llm#agents#deployment
Level:Advanced
For:AI Engineers
✦TL;DR

The paper “Priming: Hybrid state space models from pre‑trained Transformers” introduces priming, a technique that reuses pretrained Transformer weights to convert them into hybrid architectures with state‑space layers, cutting pre‑training cost by more than 99 %. Experiments show that a 32‑billion‑parameter model using Gated KalmanNet (GKA) matches its original Transformer on reasoning benchmarks while decoding up to 2 × faster and reaching correct answers in as little as 2/3 of the wall‑clock time. This enables engineers to replace attention layers with efficient alternatives without full retraining, dramatically lowering compute expense for long‑context models.

⚡ Key Takeaways

  • Pre‑training cost reduction of > 99 % when converting Transformers to hybrid models via priming.
  • Gated KalmaNet (GKA) outperforms Gated DeltaNet (GDN) and Mamba‑2, and uniquely supports variable‑length inference without retraining.
  • State‑space models consistently beat sliding‑window attention (SWA) even when SWA receives twice the memory budget, delivering the same quality at 2 × throughput.
  • Released zoo includes eight‑billion‑parameter and 32‑billion‑parameter primed models, training code, optimized kernels, and a vLLM‑serving plugin under Apache 2.0.
💡 Why It Matters

Engineers can now experiment with and deploy long‑context, high‑performance hybrid models at a fraction of the original compute cost, accelerating research cycles and production rollouts.

✅ Practical Steps

  1. Download the primed hybrid model zoo (8B and 32B) from the authors’ repository.
  2. Integrate the provided vLLM‑serving plugin to serve the hybrid model in production.
  3. Use the released training code and optimized kernels to fine‑tune or further adapt the primed models as needed.

Want the full story? Read the original article.

Read on Amazon Science ↗

More like this

Censored vs. Truncated Regression in Python, Explained

Towards Data Science•#llm

New in AWS Quick: More capable flows, an updated web experience, and agents that work together

AWS ML Blog•#agents

Three insights you may have missed from theCUBE’s coverage of the Dell Technologies New York Symposium

SiliconANGLE AI•#deployment

How Google Duplex evolved to also call businesses for you in Gemini

Google Research•#agents

EXPLORE AI NEWS

Daily hand-picked stories on LLMs, RAG, agents and production AI — curated for engineers who ship.

BROWSE NEWS

GET THE WEEKLY DIGEST

Join engineers getting the Monday signal-over-noise AI breakdown. No spam, unsubscribe anytime.

LEARN AI ENGINEERING

Curated courses, research papers, repos and tutorials built for engineers leveling up in AI.

START LEARNING