Parameters
27.0B
Architecture
Transformer (decoder-only)
Released
27.02.2025
License
Gemma Terms of Use
Input Modalities
Output Modalities
Context (native)
131,072 tokens
Context (extended)
131,072 tokens
About
Gemma 3 27B PT is the pre-trained-variant record for Google's largest Gemma 3 model (27B parameters, dense) - the base model behind gemma-3-27b-it, built from the same research and technology used to create the Gemini models, released February 27, 2025 under the Gemma Terms of Use. The underlying google/gemma-3-27b repository is access-gated; this record carries the pre-training (PT) benchmark scores for the 27B model.
Gemma 3 handles text and image input and generates text output, with a 128K context window and multilingual support in over 140 languages. The 27B model was trained on 14 trillion tokens of web documents, code, mathematics and images. Its architecture interleaves local sliding-window and global attention layers in a 5:1 local-global pattern, applies QK-Norm for stable attention, and uses a SigLIP-based vision encoder for image understanding.
Training Data 14 trillion tokens (web documents, code, mathematics, images) in 140+ languages
Benchmark Scores
| Benchmark | Score | Date | Standing |
|---|---|---|---|
|
BoolQ
reasoning
|
100.00%
|
27.02.2025 |
#1/4
better than 100%
|
|
PIQA
reasoning
|
100.00%
|
27.02.2025 |
#1/4
better than 100%
|
|
SocialIQA
reasoning
|
100.00%
|
27.02.2025 |
#1/4
better than 100%
|
|
TriviaQA
knowledge
|
100.00%
|
27.02.2025 |
#1/4
better than 100%
|
|
Natural Questions
knowledge
|
100.00%
|
27.02.2025 |
#1/4
better than 100%
|
|
WinoGrande
reasoning
|
52.42%
|
27.02.2025 |
#3/6
better than 60%
|
|
BIG-Bench Hard
reasoning
|
84.71%
|
27.02.2025 |
#2/8
better than 86%
|
|
DROP
reasoning
|
100.00%
|
27.02.2025 |
#1/4
better than 100%
|
|
MMLU Pro COT
knowledge
|
45.28%
|
27.02.2025 |
#2/4
better than 67%
|
|
AGIEval
reasoning
|
100.00%
|
27.02.2025 |
#1/3
better than 100%
|
|
MATH
math
|
100.00%
|
27.02.2025 |
#1/3
better than 100%
|
|
GSM8K
math
|
75.87%
|
27.02.2025 |
#2/6
better than 80%
|
|
GPQA
reasoning
|
14.95%
|
27.02.2025 |
#13/14
better than 8%
|
|
MBPP
code
|
100.00%
|
27.02.2025 |
#1/3
better than 100%
|
|
HumanEval
code
|
100.00%
|
27.02.2025 |
#1/3
better than 100%
|
|
Global-MMLU-Lite
multilingual
|
100.00%
|
27.02.2025 |
#1/4
better than 100%
|
|
WMT24++ (en→xx)
multilingual
|
37.92%
|
27.02.2025 |
#14/17
better than 19%
|
|
FloRes
multilingual
|
100.00%
|
27.02.2025 |
#1/4
better than 100%
|
|
XQuAD
multilingual
|
100.00%
|
27.02.2025 |
#1/4
better than 100%
|
|
ECLeKTic
multilingual
|
100.00%
|
27.02.2025 |
#1/4
better than 100%
|
|
IndicGenBench
multilingual
|
100.00%
|
27.02.2025 |
#1/4
better than 100%
|
|
COCOcap
multimodal
|
100.00%
|
27.02.2025 |
#1/3
better than 100%
|
|
MMMU
vision_language
|
36.90%
|
27.02.2025 |
#15/19
better than 22%
|
|
RealWorldQA
vision_language
|
19.53%
|
27.02.2025 |
#21/23
better than 9%
|
|
ReMI
multimodal
|
100.00%
|
27.02.2025 |
#1/3
better than 100%
|
|
AI2D_TEST
document_understanding
|
52.49%
|
27.02.2025 |
#12/15
better than 21%
|
|
ChartQA
multimodal
|
100.00%
|
27.02.2025 |
#1/3
better than 100%
|
|
BLINK
multimodal
|
10.51%
|
27.02.2025 |
#3/5
better than 50%
|
|
OKVQA
multimodal
|
100.00%
|
27.02.2025 |
#1/3
better than 100%
|
|
TallyQA
multimodal
|
100.00%
|
27.02.2025 |
#1/3
better than 100%
|
|
SpatialSense VQA
multimodal
|
93.41%
|
27.02.2025 |
#2/3
better than 50%
|
|
HellaSwag
reasoning
|
64.19%
|
27.02.2025 |
#4/7
better than 50%
|
|
ARC-c
reasoning
|
53.76%
|
27.02.2025 |
#3/6
better than 60%
|
|
ARC-e
reasoning
|
60.38%
|
27.02.2025 |
#3/6
better than 60%
|
|
MMLU
knowledge
|
56.38%
|
27.02.2025 |
#15/17
better than 12%
|
|
MGSM
multilingual
|
85.05%
|
27.02.2025 |
#2/5
better than 75%
|
|
DocVQA
multimodal
|
53.56%
|
27.02.2025 |
#3/5
better than 50%
|
|
TextVQA
multimodal
|
33.45%
|
27.02.2025 |
#2/4
better than 67%
|
|
VQAv2
multimodal
|
31.25%
|
27.02.2025 |
#2/4
better than 67%
|
|
CountBenchQA
multimodal
|
66.76%
|
27.02.2025 |
#2/4
better than 67%
|
|
InfoVQA
multimodal
|
36.34%
|
27.02.2025 |
#3/5
better than 50%
|
What is a good BoolQ score? What is a good PIQA score? What is a good SocialIQA score?
Hardware Requirements
Estimated VRAM per quantization at 4,096 tokens context
| Quantization | Weights | KV Cache | Total VRAM | On your hardware |
|---|---|---|---|---|
| FP16 — 16-bit | 54.0 GB | 4.2 GB | 60.9 GB | Which GPUs can run it? Calculate on your GPU |
| INT8 — 8-bit | 27.0 GB | 2.1 GB | 30.4 GB | Which GPUs can run it? Calculate on your GPU |
| INT4 — 4-bit | 13.5 GB | 1.0 GB | 15.2 GB | Which GPUs can run it? Calculate on your GPU |
| FP4 — 4-bit | 13.5 GB | 1.0 GB | 15.2 GB | Which GPUs can run it? Calculate on your GPU |
Consumer GPUs that fit this model with a single card (INT4):
Architecture
- Attention
- Grouped Query Attention
- Context
- 131K tokens
- Parameters
- 27000M
Source: extracted model record · model repo
Training Pipeline
-
1
pretraining
Pretraining on 14 trillion tokens
Pretrained on 14 trillion tokens of web documents, code, mathematics, and images in 140+ languages. Trained on TPU (TPUv4p, TPUv5p, TPUv5e) using JAX and ML Pathways.
Training & Evaluation Datasets
| Name | Role | Size | Modalities | Collection |
|---|---|---|---|---|
| Web Documents (Multilingual) | pretraining | 14T tokens (27B), 12T (12B), 4T (4B), 2T (1B) | text | automated |
| Code Training Data | pretraining | — | code | automated |
| Mathematics Training Data | pretraining | — | text | automated |
| Image Training Data | pretraining | — | image | automated |
| CSAM-Filtered Data | pretraining | — | automated | |
| Sensitive Data Filtered | pretraining | — | automated |
Linked Resources
Gemma 3 Technical Report
https://goo.gle/Gemma3Report
Gemma Model Page
https://ai.google.dev/gemma/docs/core
Responsible Generative AI Toolkit
https://ai.google.dev/responsible
Gemma on Kaggle
https://www.kaggle.com/models/google/gemma-3
Gemma on Vertex Model Garden
https://console.cloud.google.com/vertex-ai/publishers/google/model-garden/gemma3
Gemma Terms of Use
https://ai.google.dev/gemma/terms
Gemma Prohibited Use Policy
https://ai.google.dev/gemma/prohibited_use_policy
Gemini Family Technical Report
https://arxiv.org/abs/2312.11805
ECLeKTic Paper
https://huggingface.co/papers/2502.21228
WMT24++ Paper
https://huggingface.co/papers/2502.12404
IndicGenBench Paper
https://huggingface.co/papers/2404.16816
BLINK Paper
https://huggingface.co/papers/2404.12390
Google Sustainability Commitments
https://sustainability.google/operating-sustainably/
Google AI Responsibility Update
https://ai.google/static/documents/ai-responsibility-update-published-february-2025.pdf
JAX (training framework)
https://github.com/jax-ml/jax
ML Pathways
https://blog.google/technology/ai/introducing-pathways-next-generation-ai-architecture/
Google Cloud TPU
https://cloud.google.com/tpu/docs/intro-to-tpu
Google Gemma Models Family Collection
https://huggingface.co/collections/google/googles-gemma-models-family
Gemma 3 Release Collection
https://huggingface.co/collections/google/gemma-3-release
HuggingFace Inference Providers - Gemma 3 27B IT
https://huggingface.co/inference/models?model=google%2Fgemma-3-27b-it
Trend Analysis
24h Change
+0.2%
7d Change
+0.9%
30d Change
+4.3%
Current
73,412
downloads
-2.0%
downloads_all_time
+0.0%
likes
+0.0%
downloads
+0.0%
No comments yet
Be the first to share your thoughts!