Vision & Language Benchmark

OmniCompass VL

Vision-language metric of the OmniCompass suite (VCLab, PolyU). Scores are accuracy percent over the VCLab-PolyU/OmniCompass dataset (https://huggingface.co/datasets/VCLab-PolyU/OmniCompass); official leaderboard: https://omnicompass.netlify.app/#leaderboard.

Category

Vision & Language

Score type

percent

Scale

100%

higher is better

Models tested

1

What is OmniCompass VL?

OmniCompass VL is a benchmark for understanding images combined with text. The official description says: "Vision-language metric of the OmniCompass suite (VCLab, PolyU)". Below we describe in simple words what benchmarks of this type generally examine.

What does it test?

OmniCompass VL measures understanding images combined with text. In detail, it checks:

  • answering questions about images (figures, photos, charts, memes)
  • reading text inside images and connecting it to the question
  • visual reasoning such as counting, locating and comparing objects
  • combining what is seen with general knowledge

How to read the score

The score is accuracy: the percentage of tasks answered or solved correctly. 100% means every task was solved perfectly; higher is better.

Official methodology note: Vision-language metric of the OmniCompass suite (VCLab, PolyU). Scores are accuracy percent over the VCLab-PolyU/OmniCompass dataset (https://huggingface.co/datasets/VCLab-PolyU/OmniCompass); official leaderboard: https://omnicompass.netlify.app/#leaderboard.

Frequently asked questions

What is the OmniCompass VL benchmark?

OmniCompass VL is a benchmark for understanding images combined with text. The official description says: 'Vision-language metric of the OmniCompass suite (VCLab, PolyU)'. Below we describe in simple words what benchmarks of this type generally examine.

What does OmniCompass VL test?

OmniCompass VL mainly measures understanding images combined with text. Concretely, it checks: answering questions about images (figures, photos, charts, memes); reading text inside images and connecting it to the question; visual reasoning such as counting, locating and comparing objects.

How is the OmniCompass VL score calculated?

The score is accuracy: the percentage of tasks answered or solved correctly. 100% means every task was solved perfectly; higher is better. The declared maximum score of this benchmark is 100.0.

Where do I get OmniCompass VL data and scores?

The official source is https://huggingface.co/datasets/VCLab-PolyU/OmniCompass. The official benchmark page/reference is linked on this page. Every score on Open LLM Radar is also available as open data: download the scores JSON (http://open-llm-radar-api:8000/api/v1/benchmarks/omnicompass-vl/scores) or pull it directly from the Open LLM Radar API with any HTTP client.

Model scores 1 model tested

Reset
# Model Score Date Source Verified
1 DeepSeek-V4.1-Flash-1017
DeepSeek
28%
17.10.2026 Report Verified

What now?

You've seen how models score on OmniCompass VL — here's what to do next.