GPT-OSS-20B

OpenAI

Mixture-of-Experts (MoE) Transformer

Open Weights Commercial Use Multimodal BF16, U8 (MXFP4) GPT-OSS en

Parameters

20.9B / 3.6B active

MoE total / active

Context

131,072 tokens

native · extended 131,072

Modalities

text text

input → output

Benchmarks

#116/138

better than 16% of all scored models

Size class

#17/24

better than 29% of similar-size models

24h trend

+0.6%

Interest delta, 24h

License

Apache License 2.0

About

gpt-oss-20b (openai/gpt-oss-20b) is OpenAI's smaller open-weight model, released August 4, 2025 under Apache 2.0 alongside gpt-oss-120b - a 20.91B-parameter Mixture-of-Experts transformer with 3.61B activated parameters per token, a 131,072-token context, and text-only input/output. Its MoE weights are post-trained with MXFP4 quantization, letting the model run within 16GB of memory; all reported evals use that same quantization.

It shares the 120b feature set: configurable reasoning effort (low, medium, high) tuned to latency needs, fully exposed chain-of-thought (for debugging and trust, not for end users), native agentic capabilities (function calling, web browsing, Python code execution, Structured Outputs), and full fine-tunability. It was trained on a text-only dataset of trillions of tokens focused on STEM, coding and general knowledge (knowledge cutoff June 2024), with harmful content filtered via CBRN pre-training filters from GPT-4o; training required ~10x fewer H100-hours than gpt-oss-120b. Model card paper: arXiv 2508.10925.

Training Data Trained on a text-only dataset with trillions of tokens, with a focus on STEM, coding, and general knowledge. Knowledge cutoff: June 2024. Pre-training data filtered for harmful content using CBRN pre-training filters from GPT-4o. Training required ~10x fewer H100-hours than gpt-oss-120b.

Benchmark Scores

Benchmark Score Date Standing
AIME 2024 (with tools)
stem_reasoning
100.00%
05.08.2025
#1/1
only tracked model
AIME 2025
stem_reasoning
88.21%
05.08.2025
#12/28
better than 56%
AIME 2025 (with tools)
stem_reasoning
100.00%
05.08.2025
#1/1
only tracked model
GPQA Diamond
stem_reasoning
56.89%
05.08.2025
#70/84
better than 17%
GPQA Diamond (with tools)
stem_reasoning
100.00%
05.08.2025
#1/1
only tracked model
Humanity's Last Exam
stem_reasoning
16.67%
05.08.2025
#58/71
better than 19%
HLE (with tools)
stem_reasoning
6.06%
05.08.2025
#53/54
better than 2%
SWE-bench Verified
coding_agent
69.15%
05.08.2025
#46/62
better than 26%
Tau-Bench Retail
general_agent
54.80
05.08.2025
#3/3
better than 0%
Tau-Bench Airline
general_agent
38.00
05.08.2025
#3/3
better than 0%

Hardware Requirements

Estimated VRAM per quantization at 4,096 tokens context — MoE: all experts must fit in VRAM, only 3.6B are active per token

Quantization Weights KV Cache Total VRAM On your hardware
FP16 — 16-bit 41.8 GB 0.6 GB 44.5 GB Which GPUs can run it? Calculate on your GPU
INT8 — 8-bit 20.9 GB 0.3 GB 22.2 GB Which GPUs can run it? Calculate on your GPU
INT4 — 4-bit 10.5 GB 0.1 GB 11.1 GB Which GPUs can run it? Calculate on your GPU
FP4 — 4-bit 10.5 GB 0.1 GB 11.1 GB Which GPUs can run it? Calculate on your GPU

Consumer GPUs that fit this model with a single card (INT4):

Architecture

Decoder Block ×24 input Embedding vocab 201K · d 2880 Sliding Window Attn Hybrid 64:8 · dₕ 64 · win 128 ×12 Full Attention Hybrid 64:8 · dₕ 64 · win 128 ×12 MoE FFN 32 experts · top-4 · dᴻ 2880 Final Norm LM Head vocab 201K output

Mixture-of-Experts (MoE) Transformer

MoE Hybrid
Parameters
20.9B
Active params
3.6B
Layers
24 12× Sliding Window Attn + 12× Full Attention
Attention
Hybrid 64:8 Hybrid Attention
Experts
32 top-4
Hidden
2880
Head dim
64
Vocabulary
201K
Context
131K
silu BF16 RoPE θ 150K SWA 128 Linear router with top-4…

Source: Hugging Face config.json · GptOssForCausalLM · exact layer pattern · model repo

Training Pipeline

  1. 1
    pretraining

    Pretraining

  2. 2
    sft

    Post-Training: Distillation and SFT

  3. 3
    rl

    Reinforcement Learning for Reasoning and Tool Use

  4. 4
    other

    Safety Training: Deliberative Alignment

Training & Evaluation Datasets

NameRoleSizeModalitiesCollection
Text-only pre-training corpus (STEM, coding, general knowledge) pretraining — —

Comments (0)

No comments yet

Be the first to share your thoughts!

What now?

You have read the full card for GPT-OSS-20B — here is where to go next.