IFBench

IFBench measures how precisely a model follows verifiable instructions: constraints like 'exactly three paragraphs, no commas, answer only in the form of a JSON object'. Each constraint is machine-checkable, which removes all subjectivity from grading.

IFBench measures how precisely a model obeys the instructions it was given. In detail, it checks:

• exact compliance with machine-checkable constraints
• multiple constraints at the same time
• instruction honesty when the constraints conflict
• precision that general chat quality scores easily miss

The score is accuracy: the percentage of tasks answered or solved correctly. 100% means every task was solved perfectly; higher is better.

What is the IFBench benchmark?

IFBench measures how precisely a model follows verifiable instructions: constraints like 'exactly three paragraphs, no commas, answer only in the form of a JSON object'. Each constraint is machine-checkable, which removes all subjectivity from grading.

What does IFBench test?

IFBench mainly measures how precisely a model obeys the instructions it was given. Concretely, it checks: exact compliance with machine-checkable constraints; multiple constraints at the same time; instruction honesty when the constraints conflict.

How is the IFBench score calculated?

The score is accuracy: the percentage of tasks answered or solved correctly. 100% means every task was solved perfectly; higher is better.

Where do I get IFBench data and scores?

The official benchmark page/reference is linked on this page. Every score on Open LLM Radar is also available as open data: download the scores JSON (http://open-llm-radar-api:8000/api/v1/benchmarks/ifbench/scores) or pull it directly from the Open LLM Radar API with any HTTP client.