Parameters
116.8B / 5.1B active
MoE total / active
Context
131,072 tokens
native · extended 131,072
Modalities
text text
input → output
Benchmarks
#38/138
better than 72% of all scored models
Size class
#4/11
better than 64% of similar-size models
24h trend
+0.6%
Interest delta, 24h
License
Apache License 2.0
About
gpt-oss-120b (openai/gpt-oss-120b) is OpenAI's larger open-weight model, released August 4, 2025 under Apache 2.0 - a 116.83B-parameter Mixture-of-Experts transformer with 5.13B activated parameters per token, a 131,072-token context, and text-only input/output. The MoE weights are post-trained with MXFP4 quantization, so the model runs on a single 80GB GPU (NVIDIA H100 or AMD MI300X); all reported evals use that same MXFP4 quantization.
It offers configurable reasoning effort (low, medium, high) tuned to latency needs, exposes its full chain-of-thought (intended for debugging and trust, not for end users), and carries native agentic capabilities: function calling, web browsing, Python code execution and Structured Outputs. It was trained on a text-only dataset of trillions of tokens focused on STEM, coding and general knowledge (knowledge cutoff June 2024), with pre-training data filtered for harmful content using CBRN pre-training filters from GPT-4o. The models are fully fine-tunable. Model card paper: arXiv 2508.10925.
Training Data Trained on a text-only dataset with trillions of tokens, with a focus on STEM, coding, and general knowledge. Knowledge cutoff: June 2024. Pre-training data filtered for harmful content using CBRN pre-training filters from GPT-4o.
Benchmark Scores
| Benchmark | Score | Date | Standing |
|---|---|---|---|
|
GPQA Diamond
stem_reasoning
|
73.15%
|
08.08.2025 |
#65/84
better than 23%
|
|
Humanity's Last Exam
stem_reasoning
|
24.24%
|
08.08.2025 |
#54/71
better than 24%
|
|
HLE (with tools)
stem_reasoning
|
9.09%
|
08.08.2025 |
#52/54
better than 4%
|
|
MMLU
knowledge
|
90.21%
|
08.08.2025 |
#10/19
better than 50%
|
|
SWE-bench Verified
coding_agent
|
71.10%
|
08.08.2025 |
#45/62
better than 28%
|
|
Tau-Bench Retail
general_agent
|
87.84%
|
08.08.2025 |
#2/3
better than 50%
|
|
Tau-Bench Airline
general_agent
|
100.00%
|
08.08.2025 |
#1/3
better than 100%
|
|
Aider Polyglot
coding_agent
|
100.00%
|
08.08.2025 |
#1/2
better than 100%
|
|
MMMLU
multilingual
|
56.50%
|
08.08.2025 |
#17/23
better than 27%
|
|
HealthBench Hard
general_capabilities
|
100.00%
|
08.08.2025 |
#1/2
better than 100%
|
|
HealthBench Consensus
general_capabilities
|
100.00%
|
08.08.2025 |
#1/2
better than 100%
|
|
CodeForces
stem_reasoning
|
70.01%
|
08.08.2025 |
#6/18
better than 71%
|
|
HealthBench
general_capabilities
|
85.31%
|
08.08.2025 |
#2/6
better than 80%
|
|
AIME 2024
stem_reasoning
|
63.79%
|
08.08.2025 |
#5/7
better than 33%
|
|
AIME 2025
stem_reasoning
|
89.35%
|
08.08.2025 |
#11/28
better than 63%
|
What is a good GPQA Diamond score? What is a good Humanity's Last Exam score? What is a good HLE (with tools) score?
Hardware Requirements
Estimated VRAM per quantization at 4,096 tokens context — MoE: all experts must fit in VRAM, only 5.1B are active per token
| Quantization | Weights | KV Cache | Total VRAM | On your hardware |
|---|---|---|---|---|
| FP16 — 16-bit | 233.7 GB | 0.8 GB | 246.1 GB | Which GPUs can run it? Calculate on your GPU |
| INT8 — 8-bit | 116.8 GB | 0.4 GB | 123.1 GB | Which GPUs can run it? Calculate on your GPU |
| INT4 — 4-bit | 58.4 GB | 0.2 GB | 61.5 GB | Which GPUs can run it? Calculate on your GPU |
| FP4 — 4-bit | 58.4 GB | 0.2 GB | 61.5 GB | Which GPUs can run it? Calculate on your GPU |
Only datacenter GPUs — FP16 needs 246.1 GB. For consumer hardware, choose the INT4 build with a multi-GPU setup.
Architecture
Mixture-of-Experts (MoE) Transformer
- Parameters
- 116.8B
- Active params
- 5.1B
- Layers
- 36 18× Sliding Window Attn + 18× Full Attention
- Attention
- Hybrid 64:8 Hybrid Attention
- Experts
- 128 top-4
- Hidden
- 2880
- Head dim
- 64
- Vocabulary
- 201K
- Context
- 131K
Source: Hugging Face config.json · GptOssForCausalLM · exact layer pattern · model repo
Training Pipeline
-
1
pretraining
Pretraining
Trained on a text-only dataset with trillions of tokens, focusing on STEM, coding, and general knowledge. Knowledge cutoff: June 2024. Data filtered for harmful content using CBRN pre-training filters from GPT-4o. Training used NVIDIA H100 GPUs with PyTorch framework and expert-optimized Triton kernels. Required 2.1 million H100-hours. Uses Flash Attention algorithms. o200k_harmony tokenizer (BPE, 201,088 tokens).
-
2
sft
Post-Training: Distillation and SFT
Post-training uses similar CoT RL techniques as OpenAI o3. Models trained on harmony chat format with System > Developer > User > Assistant > Tool instruction hierarchy. Training dataset covers coding, math, science, and more.
-
3
rl
Reinforcement Learning for Reasoning and Tool Use
CoT RL techniques teach models to reason and solve problems using chain-of-thought. Variable effort reasoning training supports low, medium, high reasoning levels. Agentic tool use training includes browsing tool (search and open), Python tool (stateful Jupyter notebook), and arbitrary developer functions with structured outputs.
-
4
other
Safety Training: Deliberative Alignment
Deliberative alignment training teaches models to refuse disallowed content, be robust to jailbreaks, and adhere to instruction hierarchy. CBRN pre-training filters applied. Safety training follows OpenAI safety policies by default.
Training & Evaluation Datasets
| Name | Role | Size | Modalities | Collection |
|---|---|---|---|---|
| Text-only pre-training corpus (STEM, coding, general knowledge) | pretraining | — | — |
Linked Resources
gpt-oss-120b & gpt-oss-20b Model Card
https://arxiv.org/abs/2508.10925
OpenAI gpt-oss - Reference implementations and inference tools
https://github.com/openai/gpt-oss
OpenAI harmony - Harmony chat format implementation
https://github.com/openai/harmony
OpenAI TikToken - o200k_harmony tokenizer
https://github.com/openai/tiktoken
Introducing gpt-oss - OpenAI Blog
https://openai.com/index/introducing-gpt-oss/
Try gpt-oss
https://gpt-oss.com/
OpenAI Cookbook - gpt-oss Guides
https://cookbook.openai.com/topic/gpt-oss
gpt-oss Collection on HuggingFace
https://huggingface.co/collections/openai/gpt-oss
OpenAI Open Models
https://openai.com/open-models
awesome-gpt-oss - Collection of gpt-oss resources and inference partners
https://github.com/openai/gpt-oss/blob/main/awesome-gpt-oss.md
Trend Analysis
24h Change
+0.6%
7d Change
+5.3%
30d Change
+12.7%
Current
15,900,000 pulls
followers
+0.1%
likes
+0.1%
downloads
-0.3%
downloads_all_time
+0.1%
No comments yet
Be the first to share your thoughts!