Parameters
4.5B
Architecture
Dense
Released
02.03.2026
License
Apache License 2.0
Input Modalities
Output Modalities
Context (native)
131,072 tokens
Context (extended)
131,072 tokens
About
Gemma4-E4B (google/gemma-4-E4B-it) is the instruction-tuned E4B model of Google DeepMind's Gemma 4 family - an edge-optimized multimodal model handling text, image and audio input and generating text, with multilingual support in over 140 languages. The "E" stands for "effective": Per-Layer Embeddings (PLE) give each decoder layer its own small embedding table, so the E4B has 4.5B effective parameters (8B with embeddings) while keeping inference fast enough for on-device deployment.
Its 42-layer hybrid attention stack interleaves local Sliding-Window Attention (SWA) (35 layers, window 512) with 7 full-attention layers (final layer global), with cross-layer KV cache sharing on 18 layers to cut long-context memory further, 8 Q / 2 KV heads (head dim 256), FFN 10240, 262K vocabulary, soft logit capping at 30.0 and a 131,072-token (128K) context window. Multimodality comes from dedicated encoders: a ~150M vision encoder (16-layer ViT, patch 16, 280 soft tokens per image, variable aspect ratio and resolution) and a ~300M audio encoder (12-layer chunked-attention network, 5-sample conv kernel, 1536-dim projections) - unlike the encoder-free 12B Unified sibling.
Gemma 4 capabilities include configurable thinking modes, native function calling for agentic workflows, native system role support, video understanding via frame sequences, interleaved multimodal input and enhanced coding. Pre-training spans web documents, code, mathematics, images and audio (cutoff January 2025, 140+ languages) with CSAM and sensitive-data filtering; this checkpoint is the instruction-tuned variant. Released March 2, 2026 under Apache 2.0.
Training Data Large-scale diverse collection: web documents, code, images, audio. 140+ languages. Cutoff: January 2025.
Benchmark Scores
| Benchmark | Score | Date | Standing |
|---|---|---|---|
|
MMMU-Pro
vision_language
|
45.41%
|
02.07.2026 |
#30/35
better than 15%
|
|
MMLU-Pro
knowledge
|
30.32%
|
02.07.2026 |
#38/48
better than 21%
|
|
AIME 26
stem_reasoning
|
27.68%
|
02.07.2026 |
#42/45
better than 7%
|
|
LiveCodeBench v6
stem_reasoning
|
43.38%
|
02.07.2026 |
#42/49
better than 15%
|
|
CodeForces
stem_reasoning
|
24.70%
|
02.07.2026 |
#16/18
better than 12%
|
|
GPQA Diamond
stem_reasoning
|
32.50%
|
02.07.2026 |
#70/80
better than 13%
|
|
TAU2-Bench
general_agent
|
32.22%
|
02.07.2026 |
#16/21
better than 25%
|
|
BigBench Extra Hard
stem_reasoning
|
25.05%
|
02.07.2026 |
#4/6
better than 40%
|
|
MMMLU
multilingual
|
40.17%
|
02.07.2026 |
#16/20
better than 21%
|
|
OmniDocBench 1.5
document_understanding
|
0.02%
|
02.07.2026 |
#25/26
better than 4%
|
|
MathVision
vision_language
|
26.06%
|
02.07.2026 |
#24/27
better than 12%
|
|
MedXpertQA-MM
vision_language
|
11.87%
|
02.07.2026 |
#12/13
better than 8%
|
|
CoVoST
multilingual
|
41.15%
|
02.07.2026 |
#2/3
better than 50%
|
|
FLEURS
multilingual
|
0.17%
|
02.07.2026 |
#4/5
better than 25%
|
|
MRCR v2 128K (8-needle)
long_context
|
22.50%
|
02.07.2026 |
#4/6
better than 40%
|
|
BFCL-V4
general_agent
|
15.88%
|
— |
#26/27
better than 4%
|
|
TAU3-Bench
general_agent
|
13.15%
|
— |
#13/21
better than 40%
|
|
BrowseComp
general_agent
|
5.84%
|
— |
#41/45
better than 7%
|
|
SWE-bench Pro
coding_agent
|
4.46%
|
— |
#72/77
better than 7%
|
|
SWE-bench Verified
coding_agent
|
15.60%
|
— |
#55/60
better than 8%
|
|
Gaokao 2026
|
38.36%
|
— |
#6/8
better than 29%
|
|
HMMT Feb 26
stem_reasoning
|
17.75%
|
— |
#41/44
better than 7%
|
|
IMOAnswerBench
stem_reasoning
|
6.11%
|
— |
#29/30
better than 3%
|
|
IFBench
instruction_following
|
34.63%
|
— |
#40/42
better than 5%
|
|
AA-LCR
long_context
|
41.31%
|
— |
#35/42
better than 17%
|
|
Humanity's Last Exam
stem_reasoning
|
3.41%
|
— |
#69/71
better than 3%
|
|
GPQA
reasoning
|
63.50%
|
— |
#8/14
better than 46%
|
|
MMLU-Redux
knowledge
|
50.00%
|
— |
#24/30
better than 21%
|
|
Terminal Bench 2.1
coding_agent
|
1.66%
|
— |
#42/44
better than 2%
|
|
SuperGPQA
knowledge
|
28.15%
|
— |
#23/28
better than 19%
|
|
BrowseComp-zh
general_agent
|
5.56%
|
— |
#11/13
better than 17%
|
|
NoLiMa
long_context
|
2.66%
|
— |
#6/9
better than 38%
|
|
GDPVal-AA v2
general_agent
|
0.00
|
— |
#33/38
better than 0%
|
|
LongBench v2
long_context
|
40.27%
|
— |
#15/22
better than 33%
|
|
Claw-Eval Avg
coding_agent
|
44.17%
|
— |
#34/40
better than 15%
|
|
WildClawBench
coding_agent
|
14.41%
|
— |
#15/20
better than 26%
|
|
QwenClawBench
coding_agent
|
24.95%
|
— |
#15/17
better than 12%
|
|
LCB-Pro 25Q2 (Easy)
stem_reasoning
|
61.53%
|
— |
#5/9
better than 50%
|
|
LCB-Pro 25Q2 (Medium)
stem_reasoning
|
10.29%
|
— |
#5/9
better than 50%
|
|
OJBench
stem_reasoning
|
24.08%
|
— |
#18/22
better than 19%
|
|
MATH-500
math
|
20.59%
|
— |
#9/11
better than 20%
|
|
VITA-Bench
general_agent
|
1.04%
|
— |
#15/16
better than 7%
|
|
Gaia2
general_agent
|
20.90%
|
— |
#9/12
better than 27%
|
|
LongBenchPro
long_context
|
80.46%
|
— |
#3/10
better than 78%
|
|
AIME 2025
stem_reasoning
|
10.65%
|
— |
#22/24
better than 9%
|
|
SciCode
|
40.95%
|
— |
#18/27
better than 35%
|
|
MCP-Atlas
general_agent
|
3.13%
|
— |
#52/54
better than 4%
|
|
IFEval
instruction_following
|
16.95%
|
— |
#25/26
better than 4%
|
|
Multi-IF
instruction_following
|
11.48%
|
— |
#12/13
better than 8%
|
What is a good MMMU-Pro score? What is a good MMLU-Pro score? What is a good AIME 26 score?
Hardware Requirements
Estimated VRAM per quantization at 4,096 tokens context
| Quantization | Weights | KV Cache | Total VRAM | On your hardware |
|---|---|---|---|---|
| FP16 — 16-bit | 9.0 GB | 0.7 GB | 10.1 GB | Which GPUs can run it? Calculate on your GPU |
| INT8 — 8-bit | 4.5 GB | 0.3 GB | 5.1 GB | Which GPUs can run it? Calculate on your GPU |
| INT4 — 4-bit | 2.2 GB | 0.2 GB | 2.5 GB | Which GPUs can run it? Calculate on your GPU |
| FP4 — 4-bit | 2.2 GB | 0.2 GB | 2.5 GB | Which GPUs can run it? Calculate on your GPU |
Consumer GPUs that fit this model with a single card (INT4):
Architecture
- Attention
- Hybrid Attention (8:2)
- Layers
- 42
- Hidden size
- 2560
- Context
- 131K tokens
- Parameters
- 4500M
Source: Hugging Face config.json · Gemma4ForConditionalGeneration · exact layer pattern · model repo
Training Pipeline
-
1
pretraining
Multimodal pre-training incl. audio (cutoff January 2025)
Web documents, code, mathematics, images and audio in 140+ languages; CSAM and sensitive-data filtering.
-
2
sft
Instruction tuning
This checkpoint is the instruction-tuned (-it) variant of Gemma 4 E4B.
Training & Evaluation Datasets
| Name | Role | Size | Modalities | Collection |
|---|---|---|---|---|
| Web documents (140+ languages) | pretraining | — | — | |
| Code | pretraining | — | — | |
| Mathematics | pretraining | — | — | |
| Images | pretraining | — | — | |
| Audio | pretraining | — | — |
Linked Resources
Gemma 4 Technical Report
https://arxiv.org/abs/2607.02770
Gemma 4 launch blog (Google)
https://blog.google/innovation-and-ai/technology/developers-tools/gemma-4/
Gemma documentation (Google AI for Developers)
https://ai.google.dev/gemma/docs/core
google-gemma (GitHub)
https://github.com/google-gemma
Gemma 4 (HuggingFace collection)
https://huggingface.co/collections/google/gemma-4
BibTeX citation (Google DeepMind, 2026)
https://huggingface.co/google/gemma-4-E4B-it
Trend Analysis
24h Change
+0.2%
7d Change
+0.9%
30d Change
+4.3%
Current
73,412
likes
+0.0%
downloads_all_time
+0.4%
downloads
+0.3%
downloads
+0.5%
No comments yet
Be the first to share your thoughts!