Creative Writing v3

Creative Writing v3 is a benchmark for how precisely a model obeys the instructions it was given. The official description says: "Creative Writing v3 - Qwen's creative writing quality evaluation benchmark". Below we describe in simple words what benchmarks of this type generally examine.

Creative Writing v3 measures how precisely a model obeys the instructions it was given. In detail, it checks:

• meeting verifiable constraints (length, format, language, layout)
• following several constraints correctly at once
• keeping the constraints over long, multi-turn conversations
• not ignoring parts of the instruction while sounding plausible

The score is accuracy: the percentage of tasks answered or solved correctly. 100% means every task was solved perfectly; higher is better.

Official methodology note: Creative Writing v3 - Qwen's creative writing quality evaluation benchmark.

What is the Creative Writing v3 benchmark?

Creative Writing v3 is a benchmark for how precisely a model obeys the instructions it was given. The official description says: 'Creative Writing v3 - Qwen's creative writing quality evaluation benchmark'. Below we describe in simple words what benchmarks of this type generally examine.

What does Creative Writing v3 test?

Creative Writing v3 mainly measures how precisely a model obeys the instructions it was given. Concretely, it checks: meeting verifiable constraints (length, format, language, layout); following several constraints correctly at once; keeping the constraints over long, multi-turn conversations.

How is the Creative Writing v3 score calculated?

The score is accuracy: the percentage of tasks answered or solved correctly. 100% means every task was solved perfectly; higher is better. The declared maximum score of this benchmark is 100.0.

Where do I get Creative Writing v3 data and scores?

The official source is https://arxiv.org/abs/2511.21631. The official benchmark page/reference is linked on this page. Every score on Open LLM Radar is also available as open data: download the scores JSON (http://open-llm-radar-api:8000/api/v1/benchmarks/creative-writing-v3/scores) or pull it directly from the Open LLM Radar API with any HTTP client.