Parameters
2.3B
Architecture
Dense
Released
02.03.2026
License
Apache License 2.0
Input Modalities
Output Modalities
Context (native)
131,072 tokens
Context (extended)
131,072 tokens
About
Gemma4-E2B (google/gemma-4-E2B-it) is the smallest instruction-tuned model of Google DeepMind's Gemma 4 family - an edge-optimized multimodal model handling text, image and audio input and generating text, with multilingual support in over 140 languages and designed for efficient local execution on phones and laptops. The "E" stands for "effective": Per-Layer Embeddings (PLE) give each decoder layer its own small embedding table, so the E2B has 2.3B effective parameters (5.1B with embeddings) while keeping inference fast enough for on-device deployment.
Its 35-layer hybrid attention stack interleaves local Sliding-Window Attention (SWA) (28 layers, window 512) with 7 full-attention layers (final layer global), with cross-layer KV cache sharing on 20 layers to cut long-context memory, 8 Q / 1 KV heads (head dim 256), FFN 6144, hidden size 1536 with 256-dim per-layer inputs, 262K vocabulary, soft logit capping at 30.0 and a 131,072-token (128K) context window. Multimodality comes from dedicated encoders: a ~150M vision encoder (16-layer ViT, patch 16, 280 soft tokens per image, variable aspect ratio and resolution) and a ~300M audio encoder (12-layer chunked-attention network, 1536-dim projections).
Gemma 4 capabilities include configurable thinking modes, native function calling for agentic workflows, native system role support, video understanding via frame sequences, interleaved multimodal input and enhanced coding. Pre-training spans web documents, code, mathematics, images and audio (cutoff January 2025, 140+ languages) with CSAM and sensitive-data filtering; this checkpoint is the instruction-tuned variant. Released March 2, 2026 under Apache 2.0.
Training Data Large-scale diverse collection: web documents, code, images, audio. 140+ languages. Cutoff: January 2025.
Benchmark Scores
| Benchmark | Score | Date | Standing |
|---|---|---|---|
|
MMLU-Pro
knowledge
|
60.00
|
02.07.2026 |
#48/48
better than 0%
|
|
AIME 26
stem_reasoning
|
21.30%
|
02.07.2026 |
#43/45
better than 5%
|
|
LiveCodeBench v6
stem_reasoning
|
32.47%
|
02.07.2026 |
#44/49
better than 10%
|
|
GPQA Diamond
stem_reasoning
|
3.76%
|
02.07.2026 |
#78/80
better than 3%
|
|
TAU2-Bench
general_agent
|
10.37%
|
02.07.2026 |
#19/21
better than 10%
|
|
BigBench Extra Hard
stem_reasoning
|
4.72%
|
02.07.2026 |
#5/6
better than 20%
|
|
MMMLU
multilingual
|
67.40
|
02.07.2026 |
#20/20
better than 0%
|
|
MMMU-Pro
vision_language
|
30.85%
|
02.07.2026 |
#33/35
better than 6%
|
|
OmniDocBench 1.5
document_understanding
|
0.14%
|
02.07.2026 |
#24/26
better than 8%
|
|
MathVision
vision_language
|
12.36%
|
02.07.2026 |
#25/27
better than 8%
|
|
MedXpertQA-MM
vision_language
|
23.50
|
02.07.2026 |
#13/13
better than 0%
|
|
CoVoST
multilingual
|
33.47
|
02.07.2026 |
#3/3
better than 0%
|
|
FLEURS
multilingual
|
0.32%
|
02.07.2026 |
#3/5
better than 50%
|
|
MRCR v2 128K (8-needle)
long_context
|
10.59%
|
02.07.2026 |
#5/6
better than 20%
|
|
BFCL-V4
general_agent
|
30.20
|
— |
#27/27
better than 0%
|
|
TAU3-Bench
general_agent
|
11.23%
|
— |
#15/21
better than 30%
|
|
MCP-Atlas
general_agent
|
12.60
|
— |
#54/54
better than 0%
|
|
VITA-Bench
general_agent
|
4.40
|
— |
#16/16
better than 0%
|
|
BrowseComp
general_agent
|
0.67%
|
— |
#44/45
better than 2%
|
|
Gaokao 2026
|
81.80
|
— |
#8/8
better than 0%
|
|
HMMT Feb 26
stem_reasoning
|
20.50
|
— |
#44/44
better than 0%
|
|
IMOAnswerBench
stem_reasoning
|
22.60
|
— |
#30/30
better than 0%
|
|
IFBench
instruction_following
|
22.70
|
— |
#42/42
better than 0%
|
|
AA-LCR
long_context
|
21.79%
|
— |
#39/42
better than 7%
|
|
Humanity's Last Exam
stem_reasoning
|
0.76%
|
— |
#70/71
better than 1%
|
|
GPQA
reasoning
|
46.30%
|
— |
#10/14
better than 23%
|
|
MMLU-Redux
knowledge
|
71.80
|
— |
#30/30
better than 0%
|
|
SWE-bench Verified
coding_agent
|
1.83%
|
— |
#59/60
better than 2%
|
|
SWE-bench Pro
coding_agent
|
0.00
|
— |
#77/77
better than 0%
|
|
SuperGPQA
knowledge
|
9.23%
|
— |
#27/28
better than 4%
|
|
BrowseComp-zh
general_agent
|
2.10%
|
— |
#12/13
better than 8%
|
|
Gaia2
general_agent
|
5.79%
|
— |
#11/12
better than 9%
|
|
GDPVal-AA v2
general_agent
|
0.00
|
— |
#33/38
better than 0%
|
|
LongBenchPro
long_context
|
60.39%
|
— |
#5/10
better than 56%
|
|
LongBench v2
long_context
|
18.78%
|
— |
#18/22
better than 19%
|
|
WildClawBench
coding_agent
|
6.47%
|
— |
#18/20
better than 5%
|
|
QwenClawBench
coding_agent
|
20.45%
|
— |
#16/17
better than 6%
|
|
LCB-Pro 25Q2 (Easy)
stem_reasoning
|
29.12%
|
— |
#8/9
better than 12%
|
|
LCB-Pro 25Q2 (Medium)
stem_reasoning
|
0.00
|
— |
#6/9
better than 0%
|
|
MATH-500
math
|
85.40
|
— |
#11/11
better than 0%
|
|
Multi-IF
instruction_following
|
40.30
|
— |
#13/13
better than 0%
|
|
OJBench
stem_reasoning
|
13.22%
|
— |
#19/22
better than 14%
|
|
CodeForces
stem_reasoning
|
15.56%
|
02.07.2026 |
#17/18
better than 6%
|
|
Claw-Eval Avg
coding_agent
|
35.88%
|
— |
#37/40
better than 8%
|
|
NoLiMa
long_context
|
5.03%
|
— |
#5/9
better than 50%
|
|
Terminal Bench 2.1
coding_agent
|
0.40
|
— |
#44/44
better than 0%
|
|
IFEval
instruction_following
|
34.80
|
— |
#26/26
better than 0%
|
|
AIME 2025
stem_reasoning
|
2.98%
|
— |
#23/24
better than 4%
|
|
SciCode
|
27.62%
|
— |
#22/27
better than 19%
|
What is a good MMLU-Pro score? What is a good AIME 26 score? What is a good LiveCodeBench v6 score?
Hardware Requirements
Estimated VRAM per quantization at 4,096 tokens context
| Quantization | Weights | KV Cache | Total VRAM | On your hardware |
|---|---|---|---|---|
| FP16 — 16-bit | 4.6 GB | 0.4 GB | 5.2 GB | Which GPUs can run it? Calculate on your GPU |
| INT8 — 8-bit | 2.3 GB | 0.2 GB | 2.6 GB | Which GPUs can run it? Calculate on your GPU |
| INT4 — 4-bit | 1.1 GB | 0.1 GB | 1.3 GB | Which GPUs can run it? Calculate on your GPU |
| FP4 — 4-bit | 1.1 GB | 0.1 GB | 1.3 GB | Which GPUs can run it? Calculate on your GPU |
Consumer GPUs that fit this model with a single card (INT4):
Architecture
- Attention
- Hybrid Attention (8:1)
- Layers
- 35
- Hidden size
- 1536
- Context
- 131K tokens
- Parameters
- 2300M
Source: Hugging Face config.json · Gemma4ForConditionalGeneration · exact layer pattern · model repo
Training Pipeline
-
1
pretraining
Multimodal pre-training incl. audio (cutoff January 2025)
Web documents, code, mathematics, images and audio in 140+ languages; CSAM and sensitive-data filtering.
-
2
sft
Instruction tuning
This checkpoint is the instruction-tuned (-it) variant of Gemma 4 E2B.
Training & Evaluation Datasets
| Name | Role | Size | Modalities | Collection |
|---|---|---|---|---|
| Web documents (140+ languages) | pretraining | — | — | |
| Code | pretraining | — | — | |
| Mathematics | pretraining | — | — | |
| Images | pretraining | — | — | |
| Audio | pretraining | — | — |
Linked Resources
Gemma 4 Technical Report
https://arxiv.org/abs/2607.02770
Gemma 4 launch blog (Google)
https://blog.google/innovation-and-ai/technology/developers-tools/gemma-4/
Gemma documentation (Google AI for Developers)
https://ai.google.dev/gemma/docs/core
google-gemma (GitHub)
https://github.com/google-gemma
Gemma 4 (HuggingFace collection)
https://huggingface.co/collections/google/gemma-4
BibTeX citation (Google DeepMind, 2026)
https://huggingface.co/google/gemma-4-E2B-it
Trend Analysis
24h Change
+0.2%
7d Change
+0.9%
30d Change
+4.3%
Current
73,412
likes
+0.3%
downloads
-1.6%
downloads_all_time
+0.3%
downloads
+0.3%
No comments yet
Be the first to share your thoughts!