IFEval

IFEval (Instruction Following Evaluation) is Google's benchmark for verifiable instructions. The prompts contain machine-checkable rules — 'more than 400 words', 'no bullet points', 'mention the keyword twice' — and the score is how often every rule is actually followed.

IFEval measures how precisely a model obeys the instructions it was given. In detail, it checks:

• following layout, length and format rules exactly
• several rules fulfilled simultaneously in one answer
• objectively verifiable rules rather than human taste
• keeping precision in longer, natural answers

The score is accuracy: the percentage of tasks answered or solved correctly. 100% means every task was solved perfectly; higher is better.

What is the IFEval benchmark?

IFEval (Instruction Following Evaluation) is Google's benchmark for verifiable instructions. The prompts contain machine-checkable rules — 'more than 400 words', 'no bullet points', 'mention the keyword twice' — and the score is how often every rule is actually followed.

What does IFEval test?

IFEval mainly measures how precisely a model obeys the instructions it was given. Concretely, it checks: following layout, length and format rules exactly; several rules fulfilled simultaneously in one answer; objectively verifiable rules rather than human taste.

How is the IFEval score calculated?

The score is accuracy: the percentage of tasks answered or solved correctly. 100% means every task was solved perfectly; higher is better.

Where do I get IFEval data and scores?

The official benchmark page/reference is linked on this page. Every score on Open LLM Radar is also available as open data: download the scores JSON (http://open-llm-radar-api:8000/api/v1/benchmarks/ifeval/scores) or pull it directly from the Open LLM Radar API with any HTTP client.