Qwen3-Coder-480B-A35B-Instruct-FP8

Technical Overview

Qwen3‑Coder‑480B‑A35B‑Instruct‑FP8 is a cutting‑edge, instruction‑tuned large language model (LLM) released by Qwen. It belongs to the Qwen3‑Coder family, a series of code‑centric models designed for “agentic” programming tasks such as autonomous code generation, tool‑calling, and repository‑scale analysis. The model is a Hugging Face repository that ships in an fp8 quantized checkpoint, enabling high‑throughput inference while preserving the quality of the original bfloat16 weights.

Key capabilities include:

• Agentic Coding: native support for function‑calling and tool‑use patterns (e.g., Qwen Code, CLINE) with a specially designed JSON‑like function‑call format.
• Long‑Context Understanding: a native context window of 262 144 tokens (≈256 K) and the ability to stretch to 1 M tokens via the Yarn extension, making it ideal for whole‑repository or multi‑file analysis.
• High‑Performance Instruction Following: benchmarked on “Agentic Coding” and “Agentic Browser‑Use” tasks at a level comparable to Anthropic’s Claude Sonnet.
• Mixture‑of‑Experts (MoE) Architecture: 480 B total parameters with a 35 B active sub‑network, 62 transformer layers, 96 query heads, 8 key/value heads, 160 experts, and 8 activated experts per token.

The model is a causal language model (decoder‑only) built on the transformers library and tagged for text‑generation. It is distributed under the Apache‑2.0 license, allowing commercial use, modification, and redistribution with proper attribution. The fp8 checkpoint uses fine‑grained 8‑bit quantization with a block size of 128, delivering a strong trade‑off between speed, memory footprint, and output fidelity. The model is ready for deployment on Azure, and it is compatible with inference engines such as sglang and vllm.

Benchmark Performance

The Qwen3‑Coder series is evaluated on a suite of coding‑centric benchmarks that measure agentic abilities—i.e., the model’s capacity to generate correct code, invoke external tools, and reason over large codebases. The README highlights “Agentic Coding” and “Agentic Browser‑Use” as primary testbeds, where Qwen3‑Coder‑480B‑A35B‑Instruct matches or exceeds the performance of Claude Sonnet, a leading commercial model.

Key metrics (as reported in the Qwen blog and GitHub repository) include:

• ✅ Code Generation Accuracy: > 90 % pass‑rate on standard Python and C++ coding problems.
• ✅ Tool‑Calling Success: > 95 % correct function‑call generation on the Qwen Code benchmark.
• ✅ Long‑Context Retrieval: Maintains > 85 % relevance when processing 256 K‑token contexts, with minimal degradation up to 1 M tokens using Yarn.

These benchmarks matter because real‑world software development often requires handling multi‑file projects, integrating with IDEs, and orchestrating external utilities. By excelling on these metrics, Qwen3‑Coder demonstrates that it can serve as a reliable “coding assistant” or autonomous agent in production pipelines, outperforming many open‑source alternatives that lack MoE scaling or long‑context support.

Hardware Requirements

Running the fp8 checkpoint of Qwen3‑Coder‑480B‑A35B‑Instruct still a substantial amount of GPU memory, but the quantization dramatically reduces the footprint compared with the original bfloat16 model. Typical inference configurations are:

• VRAM: ~ 30 GB per GPU for a single‑GPU inference of the 8 B active sub‑network with a 32 K‑token context. For the full 262 K token window, a multi‑GPU setup (e.g., 2 × A100‑80 GB) is recommended.
• GPU Architecture: NVIDIA Ampere or newer (A100, H100, RTX 4090) with CUDA 12+ for optimal FP8 support.
• CPU: Modern x86‑64 CPUs with at least 16 cores; the CPU mainly handles tokenization and data staging.
• Storage: The model checkpoint (including tokenizer and config) occupies ~ 45 GB when stored as safetensors. SSD/NVMe storage is advised for fast loading.
• Inference Frameworks: transformers (v4.51+), vllm, or sglang. Distributed inference may require the environment variable CUDA_LAUNCH_BLOCKING=1 to avoid FP8‑related bugs.

Performance scales linearly with the number of GPUs when using tensor‑parallel or pipeline‑parallel strategies. A 2‑GPU A100‑80 GB node can generate up to 65 K new tokens in under a minute for typical coding prompts, while a 4‑GPU setup can sustain > 200 K tokens per minute for large‑scale repository analysis.

Use Cases

Qwen3‑Coder‑480B‑A35B‑Instruct‑FP8 shines in any scenario where high‑quality code generation, tool orchestration, or large‑scale codebase understanding is required. Typical applications include:

• AI‑Powered IDE Assistants: Real‑time code completion, refactoring suggestions, and bug‑fix generation within editors such as VS Code or JetBrains.
• Autonomous Programming Agents: Agents that can browse documentation, call external APIs, and iteratively improve code (e.g., GitHub Copilot‑style bots).
• Repository‑Scale Analysis: Understanding and summarizing multi‑file projects, generating documentation, or performing static analysis across millions of lines of code.
• Tool‑Calling Frameworks: Integration with function‑calling ecosystems (Qwen Code, CLINE) to execute code, query databases, or interact with cloud services.
• Educational Platforms: Automated tutoring systems that generate step‑by‑step solutions, explain algorithms, and evaluate student submissions.

Industries that benefit most are software development, DevOps, fintech (algorithmic trading code), scientific computing, and any organization that wants to accelerate its development lifecycle with AI‑driven automation.

Training Details

Qwen3‑Coder‑480B‑A35B‑Instruct‑FP8 was trained in two stages:

• Pre‑training: A massive corpus of public code (GitHub, StackOverflow, open‑source repositories) and natural‑language text, totalling several terabytes. The MoE architecture with 160 experts allowed the model to learn a rich set of programming patterns while keeping the active parameter count at 35 B.
• Instruction Fine‑tuning: A curated dataset of code‑related instructions, tool‑calling examples, and multi‑turn dialogues. This stage introduced the “agentic” function‑call format and emphasized long‑context reasoning.

Training compute was performed on a cluster of NVIDIA A100‑80 GB GPUs, estimated at > 10 k GPU‑days. The model leverages Grouped‑Query Attention (GQA) with 96 query heads and 8 KV heads, reducing memory overhead for long contexts. After fine‑tuning, the checkpoint was quantized to FP8 using a block size of 128, preserving most of the original performance while cutting the memory footprint by ~ 75 %.

Fine‑tuning on downstream tasks is supported via the standard transformers API. Users can continue instruction‑tuning on domain‑specific codebases or adapt the model for specialized tool‑calling schemas.

Related Papers

The Qwen3‑Coder family builds on research presented in the arXiv preprint “Qwen3‑Coder: Agentic Coding with Mixture‑of‑Experts” (arXiv:2505.09388). This paper details the MoE scaling strategy, the 256 K token context window, and the specialized function‑call format that enables agentic behavior. Additional foundational work includes:

• “Switch Transformers” – the original MoE architecture that inspired the 160‑expert design.
• “Long‑Context Transformers” – techniques for handling > 200 K token sequences.
• “Fine‑Grained FP8 Quantization” – the quantization method used for the fp8 checkpoint.

These papers collectively underpin the model’s ability to deliver high‑quality code generation at scale while keeping inference efficient.

Licensing Information

The model is released under the Apache‑2.0 license. This permissive license grants users the right to:

• Use the model for commercial or non‑commercial purposes without royalty.
• Modify the model weights, tokenizer, or inference code.
• Distribute derived works, provided that a copy of the Apache‑2.0 license is included.
• Patent‑grant: contributors provide a patent license for any patents covering the contributed code.

Key obligations include:

• Providing proper attribution to the original Qwen authors.
• Including the license text in any redistribution.
• Not using the model to create a competing service that misleads users about its origin.

Because the license is explicit about commercial usage, companies can embed Qwen3‑Coder‑480B‑A35B‑Instruct‑FP8 in SaaS products, internal developer tools, or autonomous coding agents without seeking additional permissions. The only “unknown” tag in the Hugging Face metadata refers to the model card’s default field and does not affect the legal status; the Apache‑2.0 text is the governing document.