MMBench EN-DEV v1.1
MMBench-EN-DEV v1.1 - multimodal benchmark English dev set.
Category
Vision & Language
Score type
percent
Scale
100%
higher is better
Models tested
12
What is MMBench EN-DEV v1.1?
MMBench EN-DEV v1.1 is a benchmark for understanding images combined with text. The official description says: "MMBench-EN-DEV v1.1 - multimodal benchmark English dev set". Below we describe in simple words what benchmarks of this type generally examine.
What does it test?
MMBench EN-DEV v1.1 measures understanding images combined with text. In detail, it checks:
- answering questions about images (figures, photos, charts, memes)
- reading text inside images and connecting it to the question
- visual reasoning such as counting, locating and comparing objects
- combining what is seen with general knowledge
How to read the score
The score is accuracy: the percentage of tasks answered or solved correctly. 100% means every task was solved perfectly; higher is better.
Official methodology note: MMBench-EN-DEV v1.1 - multimodal benchmark English dev set.
Frequently asked questions
What is the MMBench EN-DEV v1.1 benchmark?
MMBench EN-DEV v1.1 is a benchmark for understanding images combined with text. The official description says: 'MMBench-EN-DEV v1.1 - multimodal benchmark English dev set'. Below we describe in simple words what benchmarks of this type generally examine.
What does MMBench EN-DEV v1.1 test?
MMBench EN-DEV v1.1 mainly measures understanding images combined with text. Concretely, it checks: answering questions about images (figures, photos, charts, memes); reading text inside images and connecting it to the question; visual reasoning such as counting, locating and comparing objects.
How is the MMBench EN-DEV v1.1 score calculated?
The score is accuracy: the percentage of tasks answered or solved correctly. 100% means every task was solved perfectly; higher is better. The declared maximum score of this benchmark is 100.0.
Where do I get MMBench EN-DEV v1.1 data and scores?
The official benchmark page/reference is linked on this page. Every score on Open LLM Radar is also available as open data: download the scores JSON (http://open-llm-radar-api:8000/api/v1/benchmarks/mmbench-en-dev-v1-1/scores) or pull it directly from the Open LLM Radar API with any HTTP client.
Benchmark data & downloads
Where to get, inspect and reuse the data behind this benchmark.
Benchmark dataset
The question/task data itself — download and evaluate locally
Benchmark metadata (JSON)
This page's benchmark record from the Open LLM Radar API
All model scores (JSON download)
Full score table as JSON; paginate with limit/offset params
Open LLM Radar API docs
Swagger UI to explore and reuse all radar API endpoints
Model scores 12 models tested
| # | Model | Score | Date | Source | Verified |
|---|---|---|---|---|---|
| 1 |
Qwen3.6-35B-A3B
Qwen Team (Alibaba Cloud)
|
92.8%
|
16.04.2026 | Qwen Team (official model card) | Verified |
| 2 |
Qwen3.5-122B-A10B
Qwen Team (Alibaba Cloud)
|
92.8%
|
01.02.2026 | Report | Verified |
| 3 |
Qwen3.5-27B
Qwen Team (Alibaba Cloud)
|
92.6%
|
16.04.2026 | Qwen Team (official model card - comparison) | Unverified |
| 4 |
Qwen3.6-27B
Qwen Team (Alibaba Cloud)
|
92.3%
|
22.04.2026 | Qwen Team (official model card) | Verified |
| 5 |
Qwen3.5-35B-A3B
Qwen Team (Alibaba Cloud)
|
91.5%
|
16.04.2026 | Qwen Team (official model card - comparison) | Unverified |
| 6 |
Gemma4-31B-it
Google
|
90.9%
|
16.04.2026 | Qwen Team (official model card - comparison) | Unverified |
| 7 |
Qwen3-235B-A22B
Qwen Team (Alibaba Cloud)
|
89.7%
|
01.02.2026 | Report | Unverified |
| 8 |
Qwen3.5-4B
Qwen
|
89.4%
|
01.02.2026 | Report | Verified |
| 9 |
Gemma4-26B-A4B
Google
|
89%
|
16.04.2026 | Qwen Team (official model card - comparison) | Unverified |
| 10 |
d1-2B
LiquidAI
|
87.9%
|
— | LiquidAI/d1-2B model card (performance table, "MMBench") | Unverified |
| 11 |
LFM3-VL-2B
LiquidAI
|
86.1%
|
— | Report | Verified |
| 12 |
LFM3-Omni-2B
LiquidAI
|
73.68%
|
— | Report | Verified |