LOCA-Bench (256k)

LOCA-Bench (256k) is a benchmark for reading and reasoning over very long inputs. It was built to compare language models on a clearly defined set of tasks under the same grading rules, so scores are directly comparable between models.

LOCA-Bench (256k) measures reading and reasoning over very long inputs. In detail, it checks:

• finding relevant facts buried in documents hundreds of thousands of tokens long
• retrieving multiple needles hidden across the full context
• reasoning over whole books, codebases or conversation histories
• staying accurate near the context window limit (no 'lost in the middle' falloff)

The score is accuracy: the percentage of tasks answered or solved correctly. 100% means every task was solved perfectly; higher is better.

What is the LOCA-Bench (256k) benchmark?

LOCA-Bench (256k) is a benchmark for reading and reasoning over very long inputs. It was built to compare language models on a clearly defined set of tasks under the same grading rules, so scores are directly comparable between models.

What does LOCA-Bench (256k) test?

LOCA-Bench (256k) mainly measures reading and reasoning over very long inputs. Concretely, it checks: finding relevant facts buried in documents hundreds of thousands of tokens long; retrieving multiple needles hidden across the full context; reasoning over whole books, codebases or conversation histories.

How is the LOCA-Bench (256k) score calculated?

The score is accuracy: the percentage of tasks answered or solved correctly. 100% means every task was solved perfectly; higher is better. The declared maximum score of this benchmark is 100.0.

Where do I get LOCA-Bench (256k) data and scores?

The official benchmark page/reference is linked on this page. Every score on Open LLM Radar is also available as open data: download the scores JSON (http://open-llm-radar-api:8000/api/v1/benchmarks/loca-bench-256k/scores) or pull it directly from the Open LLM Radar API with any HTTP client.