Parameters
27.0B
Architecture
Dense
Released
12.03.2025
License
Apache License 2.0
Input Modalities
Output Modalities
Context (native)
131,072 tokens
Context (extended)
131,072 tokens
About
Gemma3-27B (gemma-3-27b-it) is the largest instruction-tuned model of Google's Gemma 3 family - lightweight, state-of-the-art open models built from the same research and technology as the Gemini models. It is a dense decoder-only Transformer with Grouped-Query Attention (GQA), RMSNorm pre/post-norms and QK-Norm (replacing Gemma 2's soft-capping), using a 5:1 local-global attention interleaving - five local Sliding-Window Attention (SWA) layers followed by one global layer - to deliver a large 128K-token context window at low inference cost. The RoPE base frequency is raised to 1M on global attention layers (10k on local layers), with positional interpolation extending the global attention span.
Multimodality comes from a frozen 400M SigLIP vision encoder variant shared across the 4B/12B/27B models: images are resized to 896x896 and encoded into 256 soft tokens, with Pan & Scan adaptive cropping for non-square images. The 27B model was pretrained on 14T tokens (text + image mix, 140+ languages, knowledge distillation from a teacher model with 256 sampled logits per token) and instruction-tuned with SFT and RLHF; QAT-quantized variants are also released. It takes text and image input and generates text (output context 8192 tokens), runs on a single GPU or TPU, and suits question answering, summarization, reasoning and image understanding. Maintained by Google (DeepMind) under the Gemma license; released March 12, 2025.
Training Data Pre-trained on 14T tokens (text + image mix; 140+ languages; knowledge distillation from a teacher). Instruction-tuned with SFT and RLHF; QAT-quantized variants released.
Benchmark Scores
| Benchmark | Score | Date | Standing |
|---|---|---|---|
|
MMMU-Pro
vision_language
|
40.38%
|
02.07.2026 |
#31/35
better than 12%
|
|
MMLU-Pro
knowledge
|
24.52%
|
02.07.2026 |
#39/48
better than 19%
|
|
AIME 26
stem_reasoning
|
20.80
|
02.07.2026 |
#45/45
better than 0%
|
|
LiveCodeBench v6
stem_reasoning
|
12.14%
|
02.07.2026 |
#48/49
better than 2%
|
|
CodeForces
stem_reasoning
|
110.00
|
02.07.2026 |
#18/18
better than 0%
|
|
GPQA Diamond
stem_reasoning
|
1.87%
|
02.07.2026 |
#79/80
better than 1%
|
|
TAU2-Bench
general_agent
|
0.12%
|
02.07.2026 |
#20/21
better than 5%
|
|
BigBench Extra Hard
stem_reasoning
|
19.30
|
02.07.2026 |
#6/6
better than 0%
|
|
MMMLU
multilingual
|
14.41%
|
02.07.2026 |
#19/20
better than 5%
|
|
OmniDocBench 1.5
document_understanding
|
0.22%
|
02.07.2026 |
#23/26
better than 12%
|
|
MathVision
vision_language
|
46.00
|
02.07.2026 |
#27/27
better than 0%
|
|
MRCR v2 128K (8-needle)
long_context
|
13.50
|
02.07.2026 |
#6/6
better than 0%
|
What is a good MMMU-Pro score? What is a good MMLU-Pro score? What is a good AIME 26 score?
Hardware Requirements
Estimated VRAM per quantization at 4,096 tokens context
| Quantization | Weights | KV Cache | Total VRAM | On your hardware |
|---|---|---|---|---|
| FP16 — 16-bit | 54.0 GB | 4.2 GB | 60.9 GB | Which GPUs can run it? Calculate on your GPU |
| INT8 — 8-bit | 27.0 GB | 2.1 GB | 30.4 GB | Which GPUs can run it? Calculate on your GPU |
| INT4 — 4-bit | 13.5 GB | 1.0 GB | 15.2 GB | Which GPUs can run it? Calculate on your GPU |
| FP4 — 4-bit | 13.5 GB | 1.0 GB | 15.2 GB | Which GPUs can run it? Calculate on your GPU |
Consumer GPUs that fit this model with a single card (INT4):
Architecture
- Attention
- Grouped Query Attention
- Context
- 131K tokens
- RoPE θ
- 1M
- Parameters
- 27000M
Source: extracted model record · model repo
Training Pipeline
-
1
pretraining
Pre-training with knowledge distillation (14T tokens)
14T tokens of text + image mix in 140+ languages; teacher distillation with 256 sampled logits per token; quality reweighing, decontamination and CSAM/sensitive-data filtering.
-
2
sft
Supervised fine-tuning (IT)
Instruction tuning of the pre-trained model with SFT (IT variants).
-
3
rlhf
RLHF
Reinforcement learning from human feedback to steer the model away from undesirable behavior.
-
4
other
Quantization Aware Training (QAT)
~5,000-step finetune producing quantized variants in standard formats, using non-quantized checkpoint probabilities as targets.
Training & Evaluation Datasets
| Name | Role | Size | Modalities | Collection |
|---|---|---|---|---|
| Web documents (140+ languages) | pretraining | — | — | |
| Code | pretraining | — | — | |
| Mathematics | pretraining | — | — | |
| Images | pretraining | — | — |
Linked Resources
Gemma 3 Technical Report
https://arxiv.org/abs/2503.19786
Introducing Gemma 3 (Google blog)
https://blog.google/innovation-and-ai/technology/developers-tools/gemma-3/
Google's Gemma models family (HuggingFace collection)
https://huggingface.co/collections/google/googles-gemma-models-family
Gemma 3 Release (HuggingFace collection)
https://huggingface.co/collections/google/gemma-3-release
BibTeX citation (Google, 2025)
https://huggingface.co/google/gemma-3-27b-it
Trend Analysis
24h Change
+0.2%
7d Change
+0.9%
30d Change
+4.3%
Current
73,412
downloads
-2.0%
downloads_all_time
+0.0%
likes
+0.0%
downloads
+0.0%
No comments yet
Be the first to share your thoughts!