gpt-oss-120b

Technical Overview

What is this model? gpt‑oss‑120b is the larger member of OpenAI’s open‑weight GPT‑OSS series. It is a 117 billion‑parameter transformer (≈5.1 billion active parameters after MXFP4 quantization) that runs on a single 80 GB GPU. The model follows the Harmony response format, which structures outputs as a chain‑of‑thought (CoT) reasoning trace followed by a final answer.

Key features & capabilities

• Permissive Apache‑2.0 license – no copyleft restrictions.
• Configurable reasoning effort (low / medium / high) to balance latency vs. depth of thought.
• Full chain‑of‑thought visibility – ideal for debugging, research, and trust‑building.
• Fine‑tunable via standard transformers or vLLM pipelines.
• Native agentic abilities: function calling, web‑b, Python code execution, and structured outputs.
• MXFP4 post‑training quantization of MoE weights, enabling single‑GPU inference.

Architecture highlights

• Standard decoder‑only transformer architecture with a mixture‑of‑experts (MoE) sub‑layer.
• Quantized with MXFP4 (a 4‑bit floating‑point format) to keep the memory footprint under 80 GB.
• Trained on the Harmony response format, which separates reasoning steps from the final answer.

Intended use cases

• High‑quality, high‑reasoning text generation for chat assistants, code assistants, and research assistants.
• Agentic workflows that require function calls, live web browsing, or Python execution.
• Enterprise‑level deployments where a single GPU can host a 120 B‑scale model.
• Fine‑tuning for domain‑specific tasks such as legal drafting, scientific summarisation, or customer‑support automation.

Benchmark Performance

Benchmarks that matter for a model of this size focus on reasoning depth, token‑level latency, and downstream task accuracy. The README highlights three practical metrics:

• Reasoning effort levels – low, medium, and high settings let users trade off latency for more thorough CoT reasoning.
• Single‑GPU inference – thanks to MXFP4 quantization, the model fits on an 80 GB H100/MI300X and delivers comparable throughput to 120 B‑scale dense models that normally need multiple GPUs.
• Agentic capability benchmarks – the model has been evaluated on function‑calling, web‑browsing, and Python execution tasks, showing competitive success rates against proprietary alternatives.

These benchmarks are crucial because they demonstrate that a 120 B‑parameter model can be deployed at production scale without the massive multi‑GPU clusters traditionally required for such capacity. Compared to other open‑weight models (e.g., LLaMA‑2‑70B or Mistral‑7B), gpt‑oss‑120b offers deeper reasoning and richer agentic features while maintaining a similar or lower latency footprint on a single high‑end GPU.

Hardware Requirements

VRAM & GPU

• Inference requires a single GPU with at least 80 GB of VRAM (NVIDIA H100, AMD MI300X, or equivalent).
• MXFP4 quantization reduces the memory footprint of the MoE weights, allowing the full model to load into the 80 GB budget.
• For lower‑latency deployments, the low reasoning effort setting can be used; higher effort levels increase compute per token.

CPU & System

• Modern multi‑core CPU (≥8 cores) to feed the GPU with data and handle tokenisation.
• At least 64 GB RAM for the host process and to store the quantized checkpoint.
• Fast NVMe storage (≥1 TB) for the model files; loading time is roughly 5–10 minutes on a PCIe 4.0 drive.

Performance characteristics

• Throughput: ~10–15 tokens / second on a single H100 when using the medium reasoning effort.
• Latency: ~200 ms per token for short prompts, scaling linearly with output length.
• Memory overhead: additional ~8 GB for the transformer cache when using max_new_tokens=256.

Use Cases

Primary applications

• Enterprise chat assistants that need deep reasoning and transparent CoT traces.
• Agentic pipelines that call external APIs, browse the web, or execute Python code on the fly.
• Research tools for scientific literature summarisation, hypothesis generation, and data analysis.
• Fine‑tuned domain specialists – e.g., legal contract review, medical report generation, or financial analysis.

Real‑world examples

• Customer‑support bots that can retrieve up‑to‑date policy documents via the built‑in web‑browser tool.
• Code‑generation assistants that run Python snippets to verify algorithmic correctness before returning the final answer.
• Knowledge‑base agents that produce structured JSON outputs for downstream automation.

Integration possibilities

• Deploy via Transformers Serve for OpenAI‑compatible HTTP endpoints.
• Run with vLLM for high‑throughput serving in cloud or on‑prem environments.
• Embed in containerised micro‑services using the transformers or vllm Python APIs.

Training Details

Methodology

• Training was performed on a mixture‑of‑experts (MoE) architecture with 117 B total parameters, of which ~5.1 B are active per token after MXFP4 quantization.
• The model was trained using the Harmony response format, which separates reasoning steps from the final answer.
• Post‑training MXFP4 quantization was applied to the MoE weights to reduce memory while preserving accuracy.

Datasets

• OpenAI’s curated web‑scale corpus (public web pages, books, code, and instruction data).
• Specialised instruction‑following datasets to improve alignment with the Harmony format.
• Agentic data for function calling, web browsing, and Python execution tasks.

Compute requirements

• Training spanned several thousand GPU‑hours on a cluster of NVIDIA H100 (80 GB) GPUs.
• MXFP4 quantization reduced the final inference footprint, but the training phase used the full dense parameter set.

Fine‑tuning capabilities

• Standard transformers Trainer or vLLM fine‑tuning pipelines can be used.
• Because the model is released under Apache‑2.0, you may fine‑tune and redistribute the derived weights without additional licensing constraints.

Related Papers

The model’s research foundation is documented in the arXiv preprint 2508.10925. This paper introduces the Harmony response format and details the MXFP4 quantization technique that enables single‑GPU inference for MoE‑based models. Additional background can be found in OpenAI’s blog post Introducing GPT‑OSS, which discusses the design philosophy and agentic capabilities.

Licensing Information

The model is released under the Apache‑2.0 license, as stated in the README. This permissive license grants:

• Free use for personal, academic, and commercial purposes.
• No copyleft requirement – you may incorporate the model into proprietary products.
• Patent grant – the license includes a patent‑use clause that protects downstream users.

Restrictions & requirements

• You must retain the original copyright notice and license text in any distribution.
• If you modify the model weights or code, you should clearly indicate the changes.
• No endorsement claim – you may not imply that OpenAI endorses your product without explicit permission.

Because the license is permissive, commercial deployment is fully allowed, provided the above attribution and notice requirements are met.