GPT-OSS-120B

OpenAI

Mixture-of-Experts (MoE) Transformer

Open Weights Commercial Use Multimodal BF16, U8 (MXFP4) GPT-OSS en

Parameters

116.8B / 5.1B active

MoE total / active

Context

131,072 tokens

native · extended 131,072

Modalities

text text

input → output

Benchmarks

#38/138

better than 72% of all scored models

Size class

#4/11

better than 64% of similar-size models

24h trend

+0.6%

Interest delta, 24h

License

Apache License 2.0

About

gpt-oss-120b (openai/gpt-oss-120b) is OpenAI's larger open-weight model, released August 4, 2025 under Apache 2.0 - a 116.83B-parameter Mixture-of-Experts transformer with 5.13B activated parameters per token, a 131,072-token context, and text-only input/output. The MoE weights are post-trained with MXFP4 quantization, so the model runs on a single 80GB GPU (NVIDIA H100 or AMD MI300X); all reported evals use that same MXFP4 quantization.

It offers configurable reasoning effort (low, medium, high) tuned to latency needs, exposes its full chain-of-thought (intended for debugging and trust, not for end users), and carries native agentic capabilities: function calling, web browsing, Python code execution and Structured Outputs. It was trained on a text-only dataset of trillions of tokens focused on STEM, coding and general knowledge (knowledge cutoff June 2024), with pre-training data filtered for harmful content using CBRN pre-training filters from GPT-4o. The models are fully fine-tunable. Model card paper: arXiv 2508.10925.

Training Data Trained on a text-only dataset with trillions of tokens, with a focus on STEM, coding, and general knowledge. Knowledge cutoff: June 2024. Pre-training data filtered for harmful content using CBRN pre-training filters from GPT-4o.

Benchmark Scores

Benchmark Score Date Standing
GPQA Diamond
stem_reasoning
73.15%
08.08.2025
#65/84
better than 23%
Humanity's Last Exam
stem_reasoning
24.24%
08.08.2025
#54/71
better than 24%
HLE (with tools)
stem_reasoning
9.09%
08.08.2025
#52/54
better than 4%
MMLU
knowledge
90.21%
08.08.2025
#10/19
better than 50%
SWE-bench Verified
coding_agent
71.10%
08.08.2025
#45/62
better than 28%
Tau-Bench Retail
general_agent
87.84%
08.08.2025
#2/3
better than 50%
Tau-Bench Airline
general_agent
100.00%
08.08.2025
#1/3
better than 100%
Aider Polyglot
coding_agent
100.00%
08.08.2025
#1/2
better than 100%
MMMLU
multilingual
56.50%
08.08.2025
#17/23
better than 27%
HealthBench Hard
general_capabilities
100.00%
08.08.2025
#1/2
better than 100%

Hardware Requirements

Estimated VRAM per quantization at 4,096 tokens context — MoE: all experts must fit in VRAM, only 5.1B are active per token

Quantization Weights KV Cache Total VRAM On your hardware
FP16 — 16-bit 233.7 GB 0.8 GB 246.1 GB Which GPUs can run it? Calculate on your GPU
INT8 — 8-bit 116.8 GB 0.4 GB 123.1 GB Which GPUs can run it? Calculate on your GPU
INT4 — 4-bit 58.4 GB 0.2 GB 61.5 GB Which GPUs can run it? Calculate on your GPU
FP4 — 4-bit 58.4 GB 0.2 GB 61.5 GB Which GPUs can run it? Calculate on your GPU

Only datacenter GPUs — FP16 needs 246.1 GB. For consumer hardware, choose the INT4 build with a multi-GPU setup.

Architecture

Decoder Block ×36 input Embedding vocab 201K · d 2880 Sliding Window Attn Hybrid 64:8 · dₕ 64 · win 128 ×18 Full Attention Hybrid 64:8 · dₕ 64 · win 128 ×18 MoE FFN 128 experts · top-4 · dᴻ 2880 Final Norm LM Head vocab 201K output

Mixture-of-Experts (MoE) Transformer

MoE Hybrid
Parameters
116.8B
Active params
5.1B
Layers
36 18× Sliding Window Attn + 18× Full Attention
Attention
Hybrid 64:8 Hybrid Attention
Experts
128 top-4
Hidden
2880
Head dim
64
Vocabulary
201K
Context
131K
silu BF16 RoPE θ 150K SWA 128 Linear router with top-4…

Source: Hugging Face config.json · GptOssForCausalLM · exact layer pattern · model repo

Training Pipeline

  1. 1
    pretraining

    Pretraining

    Trained on a text-only dataset with trillions of tokens, focusing on STEM, coding, and general knowledge. Knowledge cutoff: June 2024. Data filtered for harmful content using CBRN pre-training filters from GPT-4o. Training used NVIDIA H100 GPUs with PyTorch framework and expert-optimized Triton kernels. Required 2.1 million H100-hours. Uses Flash Attention algorithms. o200k_harmony tokenizer (BPE, 201,088 tokens).

  2. 2
    sft

    Post-Training: Distillation and SFT

    Post-training uses similar CoT RL techniques as OpenAI o3. Models trained on harmony chat format with System > Developer > User > Assistant > Tool instruction hierarchy. Training dataset covers coding, math, science, and more.

  3. 3
    rl

    Reinforcement Learning for Reasoning and Tool Use

    CoT RL techniques teach models to reason and solve problems using chain-of-thought. Variable effort reasoning training supports low, medium, high reasoning levels. Agentic tool use training includes browsing tool (search and open), Python tool (stateful Jupyter notebook), and arbitrary developer functions with structured outputs.

  4. 4
    other

    Safety Training: Deliberative Alignment

    Deliberative alignment training teaches models to refuse disallowed content, be robust to jailbreaks, and adhere to instruction hierarchy. CBRN pre-training filters applied. Safety training follows OpenAI safety policies by default.

Training & Evaluation Datasets

NameRoleSizeModalitiesCollection
Text-only pre-training corpus (STEM, coding, general knowledge) pretraining — —

Comments (0)

No comments yet

Be the first to share your thoughts!

What now?

You have read the full card for GPT-OSS-120B — here is where to go next.