Parameters
1.0B
Architecture
Transformer (decoder-only)
Released
27.02.2025
License
Gemma Terms of Use
Input Modalities
Output Modalities
Context (native)
32,768 tokens
Context (extended)
32,768 tokens
About
Gemma 3 1B PT is the pre-trained-variant record for Google's smallest Gemma 3 model - a lightweight, open transformer (1B parameters, dense) built from the same research and technology used to create the Gemini models, released February 27, 2025 under the Gemma Terms of Use. The underlying google/gemma-3-1b repository is access-gated; this record carries the pre-training (PT) benchmark scores for the 1B model.
The 1B model was trained on 2 trillion tokens of web documents, code, mathematics and images, with a smaller 32K context window and multilingual support in over 140 languages. Its architecture interleaves local sliding-window and global attention layers in a 5:1 local-global pattern and applies QK-Norm for stable attention.
Training Data 2 trillion tokens (web documents, code, mathematics, images) in 140+ languages
Benchmark Scores
| Benchmark | Score | Date | Standing |
|---|---|---|---|
|
BoolQ
reasoning
|
63.20
|
27.02.2025 |
#4/4
better than 0%
|
|
PIQA
reasoning
|
73.80
|
27.02.2025 |
#4/4
better than 0%
|
|
SocialIQA
reasoning
|
48.90
|
27.02.2025 |
#4/4
better than 0%
|
|
TriviaQA
knowledge
|
39.80
|
27.02.2025 |
#4/4
better than 0%
|
|
Natural Questions
knowledge
|
9.48
|
27.02.2025 |
#4/4
better than 0%
|
|
ARC-c
reasoning
|
38.40
|
27.02.2025 |
#6/6
better than 0%
|
|
ARC-e
reasoning
|
73.00
|
27.02.2025 |
#6/6
better than 0%
|
|
WinoGrande
reasoning
|
58.20
|
27.02.2025 |
#6/6
better than 0%
|
|
BIG-Bench Hard
reasoning
|
28.40
|
27.02.2025 |
#8/8
better than 0%
|
|
DROP
reasoning
|
42.40
|
27.02.2025 |
#4/4
better than 0%
|
|
MGSM
multilingual
|
2.04
|
27.02.2025 |
#5/5
better than 0%
|
|
Global-MMLU-Lite
multilingual
|
24.90
|
27.02.2025 |
#4/4
better than 0%
|
|
WMT24++ (en→xx)
multilingual
|
36.70
|
27.02.2025 |
#17/17
better than 0%
|
|
XQuAD
multilingual
|
43.90
|
27.02.2025 |
#4/4
better than 0%
|
|
ECLeKTic
multilingual
|
4.69
|
27.02.2025 |
#4/4
better than 0%
|
|
IndicGenBench
multilingual
|
41.40
|
27.02.2025 |
#4/4
better than 0%
|
|
HellaSwag
reasoning
|
62.30
|
27.02.2025 |
#7/7
better than 0%
|
|
FloRes
multilingual
|
29.50
|
27.02.2025 |
#4/4
better than 0%
|
What is a good BoolQ score? What is a good PIQA score? What is a good SocialIQA score?
Hardware Requirements
Estimated VRAM per quantization at 4,096 tokens context
| Quantization | Weights | KV Cache | Total VRAM | On your hardware |
|---|---|---|---|---|
| FP16 — 16-bit | 2.0 GB | 0.2 GB | 2.3 GB | Which GPUs can run it? Calculate on your GPU |
| INT8 — 8-bit | 1.0 GB | 0.1 GB | 1.1 GB | Which GPUs can run it? Calculate on your GPU |
| INT4 — 4-bit | 0.5 GB | 0.0 GB | 0.6 GB | Which GPUs can run it? Calculate on your GPU |
| FP4 — 4-bit | 0.5 GB | 0.0 GB | 0.6 GB | Which GPUs can run it? Calculate on your GPU |
Consumer GPUs that fit this model with a single card (INT4):
Architecture
- Attention
- Grouped Query Attention
- Context
- 33K tokens
- Parameters
- 1000M
Source: extracted model record · model repo
Training Pipeline
-
1
pretraining
Pretraining on 2 trillion tokens
Pretrained on 2 trillion tokens of web documents, code, mathematics, and images in 140+ languages. Trained on TPU (TPUv4p, TPUv5p, TPUv5e) using JAX and ML Pathways.
Training & Evaluation Datasets
| Name | Role | Size | Modalities | Collection |
|---|---|---|---|---|
| Web Documents (Multilingual) | pretraining | 14T tokens (27B), 12T (12B), 4T (4B), 2T (1B) | text | automated |
| Code Training Data | pretraining | — | code | automated |
| Mathematics Training Data | pretraining | — | text | automated |
| Image Training Data | pretraining | — | image | automated |
| CSAM-Filtered Data | pretraining | — | automated | |
| Sensitive Data Filtered | pretraining | — | automated |
Linked Resources
Gemma 3 Technical Report
https://goo.gle/Gemma3Report
Gemma Model Page
https://ai.google.dev/gemma/docs/core
Responsible Generative AI Toolkit
https://ai.google.dev/responsible
Gemma on Kaggle
https://www.kaggle.com/models/google/gemma-3
Gemma on Vertex Model Garden
https://console.cloud.google.com/vertex-ai/publishers/google/model-garden/gemma3
Gemma Terms of Use
https://ai.google.dev/gemma/terms
Gemma Prohibited Use Policy
https://ai.google.dev/gemma/prohibited_use_policy
Gemini Family Technical Report
https://arxiv.org/abs/2312.11805
ECLeKTic Paper
https://huggingface.co/papers/2502.21228
WMT24++ Paper
https://huggingface.co/papers/2502.12404
IndicGenBench Paper
https://huggingface.co/papers/2404.16816
BLINK Paper
https://huggingface.co/papers/2404.12390
Google Sustainability Commitments
https://sustainability.google/operating-sustainably/
Google AI Responsibility Update
https://ai.google/static/documents/ai-responsibility-update-published-february-2025.pdf
JAX (training framework)
https://github.com/jax-ml/jax
ML Pathways
https://blog.google/technology/ai/introducing-pathways-next-generation-ai-architecture/
Google Cloud TPU
https://cloud.google.com/tpu/docs/intro-to-tpu
Google Gemma Models Family Collection
https://huggingface.co/collections/google/googles-gemma-models-family
Gemma 3 Release Collection
https://huggingface.co/collections/google/gemma-3-release
HuggingFace Inference Providers - Gemma 3 27B IT
https://huggingface.co/inference/models?model=google%2Fgemma-3-27b-it
Trend Analysis
24h Change
-1.3%
7d Change
+8.0%
30d Change
-85.0%
Current
208,127
followers
+0.2%
likes
+0.0%
downloads_all_time
+0.0%
downloads
+0.0%
No comments yet
Be the first to share your thoughts!