IFBench (prompt loose)

IFBench (prompt loose) is a benchmark for how precisely a model obeys the instructions it was given. The official description says: "IFBench (prompt loose) - instruction following benchmark, loose prompt evaluation". Below we describe in simple words what benchmarks of this type generally examine.

IFBench (prompt loose) measures how precisely a model obeys the instructions it was given. In detail, it checks:

• meeting verifiable constraints (length, format, language, layout)
• following several constraints correctly at once
• keeping the constraints over long, multi-turn conversations
• not ignoring parts of the instruction while sounding plausible

The score is accuracy: the percentage of tasks answered or solved correctly. 100% means every task was solved perfectly; higher is better.

Official methodology note: IFBench (prompt loose) - instruction following benchmark, loose prompt evaluation.

What is the IFBench (prompt loose) benchmark?

IFBench (prompt loose) is a benchmark for how precisely a model obeys the instructions it was given. The official description says: 'IFBench (prompt loose) - instruction following benchmark, loose prompt evaluation'. Below we describe in simple words what benchmarks of this type generally examine.

What does IFBench (prompt loose) test?

IFBench (prompt loose) mainly measures how precisely a model obeys the instructions it was given. Concretely, it checks: meeting verifiable constraints (length, format, language, layout); following several constraints correctly at once; keeping the constraints over long, multi-turn conversations.

How is the IFBench (prompt loose) score calculated?

The score is accuracy: the percentage of tasks answered or solved correctly. 100% means every task was solved perfectly; higher is better. The declared maximum score of this benchmark is 100.0.

Where do I get IFBench (prompt loose) data and scores?

The official benchmark page/reference is linked on this page. Every score on Open LLM Radar is also available as open data: download the scores JSON (http://open-llm-radar-api:8000/api/v1/benchmarks/ifbench-prompt-loose/scores) or pull it directly from the Open LLM Radar API with any HTTP client.