XQuAD

XQuAD is a benchmark for performance in languages other than English. The official description says: "Cross-lingual Question Answering Dataset - QA in 10+ languages". Below we describe in simple words what benchmarks of this type generally examine.

XQuAD measures performance in languages other than english. In detail, it checks:

• understanding and answering in many different languages
• translation quality between language pairs
• keeping math and reasoning quality when the question is not in English
• treating low-resource languages fairly instead of defaulting to English

The score is accuracy: the percentage of tasks answered or solved correctly. 100% means every task was solved perfectly; higher is better.

Official methodology note: Cross-lingual Question Answering Dataset - QA in 10+ languages.

What is the XQuAD benchmark?

XQuAD is a benchmark for performance in languages other than English. The official description says: 'Cross-lingual Question Answering Dataset - QA in 10+ languages'. Below we describe in simple words what benchmarks of this type generally examine.

What does XQuAD test?

XQuAD mainly measures performance in languages other than English. Concretely, it checks: understanding and answering in many different languages; translation quality between language pairs; keeping math and reasoning quality when the question is not in English.

How is the XQuAD score calculated?

The score is accuracy: the percentage of tasks answered or solved correctly. 100% means every task was solved perfectly; higher is better. The declared maximum score of this benchmark is 100.0.

Where do I get XQuAD data and scores?

The official source is https://arxiv.org/abs/1910.11856. The official benchmark page/reference is linked on this page. Every score on Open LLM Radar is also available as open data: download the scores JSON (http://open-llm-radar-api:8000/api/v1/benchmarks/xquad/scores) or pull it directly from the Open LLM Radar API with any HTTP client.