Aider Polyglot

The Aider Polyglot benchmark measures how well a model performs as the 'brain' of the Aider coding assistant. The model gets 225 exercism-style coding exercises in different languages and must produce code that passes the tests. It shows how good a model is for real code-editing workflows.

Aider Polyglot measures how well a model works as an autonomous coding agent. In detail, it checks:

• precise code edits that satisfy unit tests
• several programming languages, not only Python
• following the format the coding tool expects
• correctness under a large number of automated tests

The score is accuracy: the percentage of tasks answered or solved correctly. 100% means every task was solved perfectly; higher is better.

What is the Aider Polyglot benchmark?

The Aider Polyglot benchmark measures how well a model performs as the 'brain' of the Aider coding assistant. The model gets 225 exercism-style coding exercises in different languages and must produce code that passes the tests. It shows how good a model is for real code-editing workflows.

What does Aider Polyglot test?

Aider Polyglot mainly measures how well a model works as an autonomous coding agent. Concretely, it checks: precise code edits that satisfy unit tests; several programming languages, not only Python; following the format the coding tool expects.

How is the Aider Polyglot score calculated?

The score is accuracy: the percentage of tasks answered or solved correctly. 100% means every task was solved perfectly; higher is better. The declared maximum score of this benchmark is 100.0.

Where do I get Aider Polyglot data and scores?

The official source is https://github.com/Aider-AI/aider. The official benchmark page/reference is linked on this page. Every score on Open LLM Radar is also available as open data: download the scores JSON (http://open-llm-radar-api:8000/api/v1/benchmarks/aider-polyglot/scores) or pull it directly from the Open LLM Radar API with any HTTP client.