US artificial intelligence lab Reflection released Beam, an open 501-billion-parameter mixture-of-experts model licensed under Apache 2.0, Hugging Face announced on Oct. 16, 2026. The release focuses on coding and autonomous agent workflows.
The text-only architecture activates 23 billion parameters per token across 52 decoder layers, operating within a context window of one million tokens. The network routes tokens across 49 mixture-of-experts layers containing 256 individual SwiGLU experts, following three initial dense layers. A group-limited routing system evaluates all 256 experts, splits them into eight groups of 32, selects the four best groups, and activates eight experts per token. Combined expert outputs scale by a factor of 2.5, with experts holding 98.4 percent of the model's total parameters.
Layer connections deviate from standard transformer designs. Beam weights residual connections so that step k scales sublayer additions by 1/sqrt(k) and retains existing streams by sqrt((k-1)/k), stabilizing hidden-state scale across all 104 additions. Final logits pass through an FP32 tanh softcap at ±30. Attention alternates between 43 sliding-window layers tracking 512 tokens with rotary position embeddings and nine global layers attending across the full context without positional encodings. This hybrid attention design curbs dynamic key-value cache memory to 36.08 GiB per sequence at the full one-million-token limit, compared to 208 GiB in a standard global setup.
Hardware and serving
Checkpoints ship in BF16, FP8, and NVFP4 formats. The BF16 files take up roughly one terabyte of disk space and require expert parallelism across 16 Nvidia H200 GPUs. The NVFP4 checkpoint compresses model weights to about 280 gigabytes for Nvidia Blackwell chips, fitting across four B200 accelerators, two B300s, or eight RTX Pro 6000 graphics cards. Older architectures such as Hopper and Ampere rely on the FP8 checkpoints.
Serving configurations include day-zero integrations for vLLM, SGLang, and Hugging Face transformers, alongside an open-source Rust harness. Speculative decoding runs through DFlash2, allowing a companion draft model to propose up to 15 tokens per forward pass. The tokenizer features a structured response template that parses reasoning and XML-style tool calls into separate output fields, supporting five reasoning effort settings from low to max.
Training recipes and testing
Reflection collaborated with RadixArk to build reinforcement learning recipes using group relative policy optimization in the Miles framework. A full-parameter BF16 configuration distributes workloads across 96 Nvidia H200 GPUs, split between 64 training units and 32 rollout workers. Supervised fine-tuning using LoRA adapters in TRL's SFTTrainer fits on a single eight-GPU B300 node, processing approximately 19,000 tokens per second.
Demonstration projects built around the release included BeamSpeak, a local macOS menu-bar dictation application using Apple Silicon MLX and the Parakeet v3 speech-to-text model. Hugging Face also evaluated multi-agent coordination setups across ProgramBench tasks, while Reflection introduced an incentive pool of $10,000 in Hub credits for developers sharing new Beam implementations.
