Parameters
27.0B
Architecture
Transformer (decoder-only)
Released
27.02.2025
License
Gemma Terms of Use
Input Modalities
Output Modalities
Context (native)
131,072 tokens
Context (extended)
131,072 tokens
About
Gemma 3 27B IT (google/gemma-3-27b-it) is the instruction-tuned variant of Google's largest Gemma 3 model - a lightweight, state-of-the-art open multimodal transformer (27B parameters, dense) built from the same research and technology used to create the Gemini models, released February 27, 2025 under the Gemma Terms of Use. Gemma 3 models handle text and image input and generate text output, with a 128K context window, multilingual support in over 140 languages, and more sizes than previous versions (1B/4B/12B/27B; the 27B trained on 14 trillion tokens of web documents, code, mathematics and images).
Its architecture interleaves local sliding-window and global attention layers in a 5:1 local-global pattern, applies QK-Norm for stable attention, and uses a SigLIP-based vision encoder for image understanding. The instruction-tuned variant is well-suited for question answering, summarization and reasoning plus image understanding, and its relatively small size allows deployment on laptops, desktops or own cloud infrastructure - democratizing access to state-of-the-art AI.
Training Data 14 trillion tokens (web documents, code, mathematics, images) in 140+ languages
Benchmark Scores
| Benchmark | Score | Date | Standing |
|---|---|---|---|
|
GPQA Chain-of-Thought
reasoning
|
100.00%
|
27.02.2025 |
#1/1
only tracked model
|
|
GPQA Diamond
stem_reasoning
|
41.41
|
27.02.2025 |
#80/80
better than 0%
|
|
SWE-bench Pro
coding_agent
|
12.69%
|
27.02.2025 |
#70/77
better than 9%
|
What is a good GPQA Chain-of-Thought score? What is a good GPQA Diamond score? What is a good SWE-bench Pro score?
Hardware Requirements
Estimated VRAM per quantization at 4,096 tokens context
| Quantization | Weights | KV Cache | Total VRAM | On your hardware |
|---|---|---|---|---|
| FP16 — 16-bit | 54.0 GB | 4.2 GB | 60.9 GB | Which GPUs can run it? Calculate on your GPU |
| INT8 — 8-bit | 27.0 GB | 2.1 GB | 30.4 GB | Which GPUs can run it? Calculate on your GPU |
| INT4 — 4-bit | 13.5 GB | 1.0 GB | 15.2 GB | Which GPUs can run it? Calculate on your GPU |
| FP4 — 4-bit | 13.5 GB | 1.0 GB | 15.2 GB | Which GPUs can run it? Calculate on your GPU |
Consumer GPUs that fit this model with a single card (INT4):
Architecture
- Attention
- Grouped Query Attention
- Context
- 131K tokens
- Parameters
- 27000M
Source: extracted model record · model repo
Training Pipeline
-
1
pretraining
Pretraining on 14 trillion tokens
Pretrained on 14 trillion tokens of web documents, code, mathematics, and images in 140+ languages. Trained on TPU (TPUv4p, TPUv5p, TPUv5e) using JAX and ML Pathways.
-
2
sft
Instruction Tuning
Instruction-tuned variant fine-tuned for chat and instruction-following use cases. Includes safety filtering and responsible AI tuning.
Training & Evaluation Datasets
| Name | Role | Size | Modalities | Collection |
|---|---|---|---|---|
| Web Documents (Multilingual) | pretraining | 14T tokens (27B), 12T (12B), 4T (4B), 2T (1B) | text | automated |
| Code Training Data | pretraining | — | code | automated |
| Mathematics Training Data | pretraining | — | text | automated |
| Image Training Data | pretraining | — | image | automated |
| CSAM-Filtered Data | pretraining | — | automated | |
| Sensitive Data Filtered | pretraining | — | automated |
Linked Resources
Gemma 3 Technical Report
https://goo.gle/Gemma3Report
Gemma Model Page
https://ai.google.dev/gemma/docs/core
Responsible Generative AI Toolkit
https://ai.google.dev/responsible
Gemma on Kaggle
https://www.kaggle.com/models/google/gemma-3
Gemma on Vertex Model Garden
https://console.cloud.google.com/vertex-ai/publishers/google/model-garden/gemma3
Gemma Terms of Use
https://ai.google.dev/gemma/terms
Gemma Prohibited Use Policy
https://ai.google.dev/gemma/prohibited_use_policy
Gemini Family Technical Report
https://arxiv.org/abs/2312.11805
ECLeKTic Paper
https://huggingface.co/papers/2502.21228
WMT24++ Paper
https://huggingface.co/papers/2502.12404
IndicGenBench Paper
https://huggingface.co/papers/2404.16816
BLINK Paper
https://huggingface.co/papers/2404.12390
Google Sustainability Commitments
https://sustainability.google/operating-sustainably/
Google AI Responsibility Update
https://ai.google/static/documents/ai-responsibility-update-published-february-2025.pdf
JAX (training framework)
https://github.com/jax-ml/jax
ML Pathways
https://blog.google/technology/ai/introducing-pathways-next-generation-ai-architecture/
Google Cloud TPU
https://cloud.google.com/tpu/docs/intro-to-tpu
Google Gemma Models Family Collection
https://huggingface.co/collections/google/googles-gemma-models-family
Gemma 3 Release Collection
https://huggingface.co/collections/google/gemma-3-release
HuggingFace Inference Providers - Gemma 3 27B IT
https://huggingface.co/inference/models?model=google%2Fgemma-3-27b-it
Trend Analysis
24h Change
+0.2%
7d Change
+0.9%
30d Change
+4.3%
Current
73,412
downloads
-2.0%
downloads_all_time
+0.0%
likes
+0.0%
downloads
+0.0%
No comments yet
Be the first to share your thoughts!