Welcome Beam: a 501B open model you can own, customize, and scale
Beam is a 501‑billion‑parameter mixture‑of‑experts (MoE) LLM released by Reflection, offering 23 B active parameters per token, a 1 M‑token context window, and a Rust‑based inference harness that delivers high performance. It ships with native Transformers support plus integrations for vLLM and SGLang, and includes day‑0 recipes for inference, supervised fine‑tuning (SFT), and reinforcement learning (RL). Engineers can run, fine‑tune, or quantize Beam on‑prem or in the cloud, leveraging its sparse MoE design to achieve large‑model capacity at smaller compute cost.
⚡ Key Takeaways
- 501 B total parameters, 23 B active per token; 52 layers, 256 experts per MoE layer, 8 experts routed per token.
- Native implementations in Transformers, vLLM, and SGLang; also a custom Rust harness for optimal inference.
- Context window of 1 M tokens with sliding‑window attention (512‑token windows) in 43 layers and full‑context attention in 9 layers, using scalable‑softmax query scaling for global layers.
- Supports BF16, FP8, and NVFP4 checkpoint formats; includes speculative decoding drafter model DFlash2.
- Provides SFT and full‑parameter RL recipes (full‑parameter RL with Miles).
Beam gives teams frontier‑scale model capacity on their own hardware while keeping inference cost comparable to much smaller models, enabling long‑context reasoning and agentic applications without relying on external APIs.
✅ Practical Steps
- Use the provided Rust harness or the Transformers/vLLM/SGLang libraries to load the Beam checkpoint (BF16, FP8, or NVFP4) and run inference with a 1 M‑token context.
- Follow the day‑0 SFT recipe to fine‑tune Beam on your own dataset, using the supplied supervised fine‑tuning scripts.
- Apply the full‑parameter RL recipe (Miles) to train reinforcement‑learning policies on top of Beam.
Want the full story? Read the original article.
Read on Hugging Face Blog ↗