Step-3.5-Flash

Technical Overview

Step‑3.5‑Flash is an open‑source, high‑throughput foundation model designed for agentic and coding workloads. It belongs to the “Step‑3.5” family and pushes the frontier of reasoning speed while keeping inference costs comparable to a 11‑billion‑parameter dense model. The model is built on a sparse Mixture‑of‑Experts (MoE) architecture: a total of 196 B parameters are stored, but only 11 B “active” parameters are consulted per token, delivering a high that rivals proprietary LLMs.

Key capabilities

• Deep reasoning at speed: 3‑way Multi‑Token Prediction (MTP‑3) yields 100–300 tokens / second in typical chat/agent settings and peaks at 350 tok/s for single‑stream coding.
• Agent‑ready RL framework: Integrated reinforcement‑learning loops give the model self‑improvement abilities, reflected in 74.4 % SWE‑bench Verified and 51.0 % Terminal‑Bench 2.0 scores.
• Efficient long‑context handling: A 256 K token window is achieved via a 3:1 Sliding‑Window Attention (SWA) ratio, cutting the quadratic cost of full‑attention layers while preserving context fidelity.
• Local‑deployment friendly: Optimized for consumer‑grade hardware (e.g., Mac Studio M4 Max, NVIDIA DGX Spark) with a focus on data‑privacy and offline use.

Architecture highlights

• Sparse MoE with 196 B total parameters, 11 B active per token.
• Three‑way MTP‑3 decoder that predicts multiple tokens in parallel.
• Hybrid attention stack: every full‑attention layer is followed by three SWA layers, enabling a 256 K context window at roughly ⅓ the cost of a pure full‑attention model.
• RL‑augmented training pipeline that encourages self‑consistent planning and tool‑use.

Intended use cases

• Real‑time coding assistants and code‑generation agents.
• Multi‑step reasoning bots, autonomous agents, and tool‑augmented LLMs.
• Long‑document analysis, code‑base navigation, and extensive retrieval‑augmented generation.
• Any scenario where high throughput and low latency are critical (e.g., interactive IDE plugins, on‑device personal assistants).

Benchmark Performance

For a model that emphasizes speed and agentic competence, the most relevant benchmarks are those that stress multi‑step reasoning, code synthesis, and long‑context navigation. The README reports the following scores:

• SWE‑bench Verified: 74.4 % – top‑tier performance on real‑world software‑engineering tasks.
• Terminal‑Bench 2.0: 51.0 % – strong ability to execute long‑horizon terminal commands.
• τ²‑Bench: 88.2 % – a composite reasoning benchmark where Step‑3.5‑Flash outperforms many closed‑source rivals.
• BrowseComp: 51.6 % (69 % with Context Manager) – demonstrates effective web‑search‑augmented generation.

These benchmarks matter because they reflect the model’s capacity to plan, execute, and verify over many reasoning steps—precisely the skill set needed for autonomous agents and advanced coding assistants. Compared with other open‑source MoE models (e.g., DeepSeek V3.2, Kimi K2), Step‑3.5‑Flash achieves comparable or superior scores while consuming roughly 1‑2 × the decoding cost of a dense 11 B model, thanks to its MTP‑3 and SWA innovations.

Hardware Requirements

VRAM / GPU memory

• Baseline inference with 8‑bit quantization: ~24 GB VRAM (e.g., NVIDIA RTX 3090).
• Full‑precision (FP16) inference: 36–48 GB VRAM (e.g., RTX 4090, A100‑40 GB).
• For the 256 K context window, a GPU with at least 48 GB is recommended to keep attention buffers in memory.

Recommended GPU specifications

• CUDA compute capability ≥ 8.0 (Ampere or newer).
• High‑bandwidth memory (HBM2 or GDDR6X) to sustain 100‑300 tok/s throughput.
• Multi‑GPU setups (e.g., 2× A100‑40 GB) can be used for parallel inference or batch processing.

CPU & storage

• Modern 8‑core CPU (e.g., AMD Ryzen 7 5800X, Intel i9‑12900K) is sufficient for token preprocessing and I/O.
• Model file size (safetensors) ≈ 300 GB (196 B MoE with expert routing tables). SSD storage with ≥ 1 TB free space is recommended for fast loading.
• NVMe drives are strongly advised to avoid bottlenecks when using the 256 K context window.

In practice, a single high‑end consumer GPU (RTX 4090, 24 GB) can run the model in 8‑bit mode for most conversational workloads, while professional‑grade GPUs (A100, DGX Spark) unlock the full 256 K context and maximum token‑per‑second rates.

Use Cases

• Interactive coding assistants: Real‑time code completion, bug‑fix suggestions, and multi‑file refactoring thanks to the 256 K context.
• Autonomous agents: Planning‑and‑execution loops for web‑scraping, tool‑use, and terminal automation (validated by Terminal‑Bench 2.0).
• Long‑document summarisation: Legal contracts, scientific papers, or extensive codebases can be processed without truncation.
• RAG‑enhanced chatbots: The model’s fast MTP‑3 decoding makes it suitable for retrieval‑augmented generation where latency matters.
• Research prototyping: Researchers can experiment with MoE‑based scaling and sliding‑window attention without needing a multi‑billion‑parameter GPU cluster.

Integration is straightforward via the transformers library (pipeline tag text-generation) or through the provided Hugging Face Space and OpenRouter endpoints.

Training Details

While the exact training pipeline is not fully disclosed, the README and associated papers reveal the following high‑level methodology:

• Mixture‑of‑Experts pre‑training: A 196 B‑parameter MoE model was trained on a curated corpus of web text, code, and instruction data, with a routing network that activates 11 B parameters per token.
• Multi‑Token Prediction (MTP‑3): During pre‑training, the decoder learned to predict three tokens in parallel, dramatically increasing throughput.
• Reinforcement‑Learning for Agents: A specialized RL framework (similar to RLHF) was applied on agent‑centric benchmarks (SWE‑bench, Terminal‑Bench) to improve self‑consistency and tool‑use.
• Data sources: Large‑scale web crawls, public code repositories, and instruction‑following datasets (e.g., OpenAI‑Evals, Alpaca‑style instruction data).
• Compute footprint: Estimated to be on the order of several thousand GPU‑years on A100‑40 GB hardware, consistent with other 200 B‑scale MoE projects.
• Fine‑tuning: The model can be further adapted via LoRA, ‑tuning, or full‑parameter fine‑tuning using the transformers library, thanks to the standard Hugging Face safetensors format.

Related Papers

The README references three arXiv pre‑prints that form the scientific backbone of Step‑3.5‑Flash:

• arXiv:2602.10604 – Technical report describing the model’s MoE scaling laws and the “intelligence density” concept.
• arXiv:2601.05593 – Introduces Parallel Thinking and the 3‑way Multi‑Token Prediction (MTP‑3) mechanism that powers the high token‑throughput.
• arXiv:2507.19427 – Details the 3:1 Sliding‑Window Attention (SWA) architecture for ultra‑long context windows.

Together, these works explain how Step‑3.5‑Flash achieves a blend of speed, reasoning depth, and long‑context efficiency that is rare among open‑source LLMs.

Licensing Information

The model is released under the Apache‑2.0 license, as indicated in the README and the Hugging Face model card. Apache‑2.0 is a permissive open‑source license that grants:

• Free use, modification, and distribution of the model weights and code.
• Commercial deployment without royalty payments.
• Patent protection for contributors and downstream users.

Restrictions:

• Any redistribution must retain the original copyright notice and license text.
• Modifications must be clearly marked as such.
• Trademark use (e.g., “Step‑3.5‑Flash”) requires permission from the original author.

Because the license is permissive, you can embed Step‑3.5‑Flash in commercial products, SaaS platforms, or on‑device applications, provided you include the Apache‑2.0 notice and do not misrepresent the origin of the model.