NOVA-63

NOVA-63 is a benchmark for performance in languages other than English. The official description says: "NOVA-63 - multilingual benchmark covering 63 languages". Below we describe in simple words what benchmarks of this type generally examine.

NOVA-63 measures performance in languages other than english. In detail, it checks:

• understanding and answering in many different languages
• translation quality between language pairs
• keeping math and reasoning quality when the question is not in English
• treating low-resource languages fairly instead of defaulting to English

The score is accuracy: the percentage of tasks answered or solved correctly. 100% means every task was solved perfectly; higher is better.

Official methodology note: NOVA-63 - multilingual benchmark covering 63 languages.

What is the NOVA-63 benchmark?

NOVA-63 is a benchmark for performance in languages other than English. The official description says: 'NOVA-63 - multilingual benchmark covering 63 languages'. Below we describe in simple words what benchmarks of this type generally examine.

What does NOVA-63 test?

NOVA-63 mainly measures performance in languages other than English. Concretely, it checks: understanding and answering in many different languages; translation quality between language pairs; keeping math and reasoning quality when the question is not in English.

How is the NOVA-63 score calculated?

The score is accuracy: the percentage of tasks answered or solved correctly. 100% means every task was solved perfectly; higher is better.

Where do I get NOVA-63 data and scores?

The official benchmark page/reference is linked on this page. Every score on Open LLM Radar is also available as open data: download the scores JSON (http://open-llm-radar-api:8000/api/v1/benchmarks/nova-63/scores) or pull it directly from the Open LLM Radar API with any HTTP client.