Ifstruct V1

Ifstruct V1 is a benchmark for how precisely a model obeys the instructions it was given. The official description says: "Ifstruct V1 - instruction following structured evaluation benchmark by LiquidAI". Below we describe in simple words what benchmarks of this type generally examine.

Ifstruct V1 measures how precisely a model obeys the instructions it was given. In detail, it checks:

• meeting verifiable constraints (length, format, language, layout)
• following several constraints correctly at once
• keeping the constraints over long, multi-turn conversations
• not ignoring parts of the instruction while sounding plausible

The score is accuracy: the percentage of tasks answered or solved correctly. 100% means every task was solved perfectly; higher is better.

Official methodology note: Ifstruct V1 - instruction following structured evaluation benchmark by LiquidAI.

What is the Ifstruct V1 benchmark?

Ifstruct V1 is a benchmark for how precisely a model obeys the instructions it was given. The official description says: 'Ifstruct V1 - instruction following structured evaluation benchmark by LiquidAI'. Below we describe in simple words what benchmarks of this type generally examine.

What does Ifstruct V1 test?

Ifstruct V1 mainly measures how precisely a model obeys the instructions it was given. Concretely, it checks: meeting verifiable constraints (length, format, language, layout); following several constraints correctly at once; keeping the constraints over long, multi-turn conversations.

How is the Ifstruct V1 score calculated?

The score is accuracy: the percentage of tasks answered or solved correctly. 100% means every task was solved perfectly; higher is better.

Where do I get Ifstruct V1 data and scores?

The official benchmark page/reference is linked on this page. Every score on Open LLM Radar is also available as open data: download the scores JSON (http://open-llm-radar-api:8000/api/v1/benchmarks/ifstruct-v1/scores) or pull it directly from the Open LLM Radar API with any HTTP client.