Qwen3-Coder-30B-A3B-Instruct-FP8

Technical Overview

Qwen3‑Coder‑30B‑A3B‑Instruct‑FP8 is a 30‑billion‑parameter causal language model released by the Qwen team. It is built on the Qwen3‑Coder family and is specifically fine‑tuned for code‑centric instruction following, tool‑calling, and agentic workflows. The “A3B” suffix indicates a Mixture‑of‑Experts (MoE) configuration where only a subset of experts (≈3.3 B parameters) is activated per token, dramatically reducing compute while preserving the capacity of a 30 B model. The “FP8” suffix denotes a fine‑grained 8‑bit floating‑point quantization that enables inference on a single high‑end GPU with minimal loss of quality.

Key capabilities include:

• Agentic Coding – native support for tool‑calling and function‑call formats, making the model an excellent “coding assistant” that can invoke external utilities (e.g., Qwen Code, CLINE) during a session.
• Long‑Context Understanding – a native context window of 262 144 tokens (≈256 K) and optional Yarn‑based extension up to 1 M tokens, ideal for repository‑scale code review or multi‑file generation.
• High‑Performance Benchmarks – leading scores on open‑source “Agentic Coding” and “Agentic Browser‑Use” tasks, surpassing most publicly available models of comparable size.

Architecture highlights:

• 48 transformer layers with Grouped‑Query Attention (GQA): 32 query heads, 4 KV heads.
• 128 experts in the MoE layer; 8 experts are activated per token, yielding an effective parameter count of 3.3 B while retaining the expressive power of a 30.5 B model.
• Fine‑grained FP8 quantization (block size = 128) that preserves the original BF16 accuracy for most coding tasks.
• Non‑thinking mode only – the model does not emit <think></think> blocks, simplifying downstream parsing.

Intended use cases span interactive code generation, automated debugging, tool‑augmented software development, and any scenario that benefits from long‑context, agentic reasoning over source code.

Benchmark Performance

The most relevant benchmarks for Qwen3‑Coder are the “Agentic Coding” and “Agentic Browser‑Use” suites, which evaluate a model’s ability to generate correct code, invoke external tools, and navigate web‑based resources. According to the Qwen blog and accompanying GitHub repository, Qwen3‑Coder‑30B‑A3B‑Instruct‑FP8 achieves top‑tier scores among open‑source models, outperforming other 30 B‑class MoE and dense models on both accuracy and latency.

Why these benchmarks matter:

• Real‑world relevance – they simulate the full software‑development loop (write, test, debug, refactor).
• Tool‑calling proficiency – measuring how well a model can generate valid function‑call JSON and handle the resulting output.
• Scalability – long‑context tests verify that the model can retain information across large codebases.

Compared to peers such as Llama‑3‑Coder‑70B, DeepSeek‑Coder‑33B, and CodeLlama‑34B, Qwen3‑Coder‑30B‑A3B‑Instruct‑FP8 offers a superior trade‑off between speed (thanks to FP8) and code correctness, making it a compelling choice for production‑grade coding assistants.

Hardware Requirements

Running the FP8‑quantized checkpoint still demands substantial GPU resources. A single NVIDIA A100 40 GB (or 80 GB) can host the model in device_map="auto" mode, but for optimal throughput a multi‑GPU setup (e.g., 2 × A100 80 GB) is recommended to avoid out‑of‑memory errors when using the full 262 K context window.

• VRAM – ~30 GB for the FP8 model; ~45 GB when using the original BF16 checkpoint.
• GPU – NVIDIA Ampere or later (A100, H100, RTX 4090) with Tensor‑Core support for FP8.
• CPU – 16‑core Xeon or AMD EPYC for preprocessing and tokenization; minimal impact on inference speed.
• Storage – the model checkpoint (including safetensors) occupies roughly 30 GB; keep an additional 10 GB for tokenizer files and config.
• Performance – on a single A100, generation of 65 536 tokens (the maximum in the quick‑start example) completes in ~30 seconds for typical coding prompts; latency scales linearly with context length, so reducing the window to 32 K tokens can halve memory usage and improve speed.

Use Cases

Qwen3‑Coder‑30B‑A3B‑Instruct‑FP8 shines in any scenario that requires high‑fidelity code generation combined with tool‑calling. Below are representative applications:

• Interactive IDE assistants – embed the model in VS Code or JetBrains IDEs to suggest code snippets, refactorings, or unit tests on the fly.
• Automated code review bots – use the 256 K token window to ingest entire repository histories and produce actionable review comments.
• Agentic automation pipelines – combine the model with external tools (e.g., Docker, Git) via function calls to build, test, and deploy software autonomously.
• Educational platforms – generate step‑by‑step explanations, quizzes, and interactive coding exercises for students.
• Low‑code/no‑code platforms – translate natural‑language requirements into functional code blocks, accelerating prototype development.

Training Details

Training was performed in two stages: a massive pre‑training phase on a diverse multilingual and code‑heavy corpus, followed by instruction‑tuning with a focus on coding and tool‑calling data. The model uses a 48‑layer transformer stack with 32 query heads and 4 KV heads per layer, and a 128‑expert MoE layer where 8 experts are selected per token.

• Datasets – a blend of public code repositories (GitHub, StackOverflow), multilingual text corpora, and synthetic instruction data generated via self‑instruct pipelines.
• Compute – training leveraged a cluster of NVIDIA H100 GPUs (80 GB) for several weeks, accumulating on the order of 10 k GPU‑hours.
• Quantization – after the full‑precision BF16 checkpoint was obtained, a fine‑grained FP8 quantization pass (block size = 128) was applied, producing the final checkpoint distributed on Hugging Face.
• Fine‑tuning capability – the model can be further instruction‑tuned using the standard transformers API with LoRA or QLoRA, thanks to the retained BF16 weights in the underlying checkpoint.

Related Papers

The README references the arXiv preprint 2505.09388, which details the Qwen3‑Coder architecture, MoE design, and the fine‑grained FP8 quantization technique. The paper presents extensive benchmarks on agentic coding tasks and discusses the Yarn‑based context extension that enables up to 1 M tokens.

Key research foundations include:

• Mixture‑of‑Experts (MoE) scaling laws – enabling large parameter counts with sparse activation.
• Grouped‑Query Attention (GQA) – reducing KV memory bandwidth while preserving attention quality.
• Fine‑grained FP8 quantization – a novel block‑wise approach that retains BF16 accuracy for generative tasks.

Licensing Information

The model card lists the Apache‑2.0 license, which is a permissive open‑source license. Although the License field in the Hugging Face metadata shows “unknown”, the linked LICENSE file clarifies that the model is released under Apache‑2.0.

• Commercial use – allowed without royalty, provided you comply with the license terms.
• Modification & redistribution – you may modify the model, create derivative works, and distribute them, as long as you retain the original copyright notice and include a copy of the license.
• Patents – Apache‑2.0 grants a patent‑grant clause, protecting downstream users from patent litigation over contributions.
• Attribution – you must give appropriate credit to the Qwen team (e.g., “© Qwen, 2024”) and indicate any changes made.

There are no explicit restrictions on data usage, but you should verify that any downstream data complies with your own privacy and security policies.