Gemma4-31B-it

Google

Parameters

30.7B

Architecture

Dense

Released

11.03.2026

License

Apache License 2.0

Open Weights Commercial Use Multimodal BF16 Gemma4 en multilingual

Input Modalities

text image

Output Modalities

text

Context (native)

262,144 tokens

Context (extended)

262,144 tokens

Openness Index Score 100.0/100

About

Gemma4-31B-it (google/gemma-4-31B-it) is the instruction-tuned flagship of Google DeepMind's Gemma 4 family - a dense multimodal model handling text and image input and generating text, with multilingual support in over 140 languages. Its 60-layer hybrid attention stack interleaves local Sliding-Window Attention (SWA) (50 layers, window 1024) with 10 full-attention layers (final layer global), with unified Keys and Values (K=V, 4 global KV heads, head dim 512) and Proportional RoPE (p-RoPE) on global layers, 32 Q / 16 KV GQA heads (head dim 256), hidden size 5376, FFN 21504, 262K vocabulary, soft logit capping at 30.0 and a 262,144-token context window. Multimodality comes from a ~550M vision encoder (27-layer ViT, patch 16, 280 soft tokens per image, variable aspect ratio and resolution); audio is not supported on this variant.

Gemma 4 capabilities include configurable thinking modes, native function calling for agentic workflows, native system role support, video understanding via frame sequences, interleaved multimodal input and enhanced coding. Pre-training spans web documents, code, mathematics and images (cutoff January 2025, 140+ languages) with CSAM and sensitive-data filtering; this checkpoint is the instruction-tuned variant. Released March 11, 2026 under Apache 2.0.

Training Data Large-scale diverse collection: web documents, code, images, audio. 140+ languages. Cutoff: January 2025.

Benchmark Scores

Benchmark Score Date Standing
SWE-bench Verified
coding_agent
59.17%
16.04.2026
#46/60
better than 22%
SWE-bench Multilingual
coding_agent
55.27%
16.04.2026
#40/48
better than 15%
Terminal-Bench 2.0
coding_agent
30.05%
16.04.2026
#15/23
better than 36%
Claw-Eval Avg
coding_agent
57.47%
16.04.2026
#33/40
better than 18%
Claw-Eval Pass^3
coding_agent
25.00
16.04.2026
#21/21
better than 0%
SkillsBench Avg5
coding_agent
30.82%
16.04.2026
#14/16
better than 13%
QwenClawBench
coding_agent
76.07%
16.04.2026
#8/17
better than 56%
NL2Repo
coding_agent
8.00%
16.04.2026
#38/41
better than 5%
QwenWebBench
coding_agent
39.25%
16.04.2026
#4/8
better than 57%
TAU3-Bench
general_agent
97.93%
16.04.2026
#4/21
better than 85%

Showing 10 of 81 results

What is a good SWE-bench Verified score? What is a good SWE-bench Multilingual score? What is a good Terminal-Bench 2.0 score?

Hardware Requirements

Estimated VRAM per quantization at 4,096 tokens context

Quantization Weights KV Cache Total VRAM On your hardware
FP16 — 16-bit 61.4 GB 4.8 GB 69.2 GB Which GPUs can run it? Calculate on your GPU
INT8 — 8-bit 30.7 GB 2.4 GB 34.6 GB Which GPUs can run it? Calculate on your GPU
INT4 — 4-bit 15.3 GB 1.2 GB 17.3 GB Which GPUs can run it? Calculate on your GPU
FP4 — 4-bit 15.3 GB 1.2 GB 17.3 GB Which GPUs can run it? Calculate on your GPU

Only datacenter GPUs — FP16 needs 69.2 GB. For consumer hardware, choose the INT4 build with a multi-GPU setup.

Consumer GPUs that fit this model with a single card (INT4):

GeForce RTX 3090 · 24GB GeForce RTX 4090 D · 24GB AMD Radeon RX 7900 XTX · 24GB GeForce RTX 3090 Ti · 24GB GeForce RTX 4090 · 24GB Apple M4 Pro · 24GB Apple M1 Max · 32GB Apple M2 Max · 32GB GeForce RTX 5090 · 32GB Apple M3 Max · 36GB Apple M4 Max · 36GB Apple M1 Ultra · 64GB

Architecture

Decoder Block ×60 input Embedding vocab 262K · d 5376 Sliding Window Attn Hybrid 32:16 · dₕ 256 · win 1024 ×50 Full Attention Hybrid 32:16 · dₕ 256 · win 1024 ×10 Dense FFN SwiGLU · d 22K Final RMSNorm LM Head tied with embedding output
Attention
Hybrid Attention (32:16)
Layers
60
Hidden size
5376
Context
262K tokens
Parameters
30700M

Source: Hugging Face config.json · Gemma4ForConditionalGeneration · exact layer pattern · model repo

Type: Dense
Attention: Hybrid: interleaved local sliding window + global attention, final layer always global. Global layers use unified Keys/Values and Proportional RoPE (p-RoPE).
Decoder: decoder-only transformer

Training Pipeline

  1. 1
    pretraining

    Multimodal pre-training (cutoff January 2025)

    Web documents, code, mathematics and images in 140+ languages; CSAM and sensitive-data filtering.

  2. 2
    sft

    Instruction tuning

    This checkpoint is the instruction-tuned (-it) variant of Gemma 4 31B.

Training & Evaluation Datasets

NameRoleSizeModalitiesCollection
Web documents (140+ languages) pretraining — —
Code pretraining — —
Mathematics pretraining — —
Images pretraining — —

Linked Resources

tech_report

Gemma 4 Technical Report

https://arxiv.org/abs/2607.02770

 blog

Gemma 4 Launch Blog

https://blog.google/innovation-and-ai/technology/developers-tools/gemma-4/

 website

Gemma Documentation

https://ai.google.dev/gemma/docs/core

 github

Google Gemma GitHub

https://github.com/google-gemma

 collection

Gemma 4 HuggingFace Collection

https://huggingface.co/collections/google/gemma-4

 website

Gemma 4 License (Apache 2.0)

https://ai.google.dev/gemma/docs/gemma_4_license

Comments (0)

No comments yet

Be the first to share your thoughts!

What now?

You have read the full card for Gemma4-31B-it — here is where to go next.

Compare with another model Test on your own GPU Related models Data as JSON