ActivityNet-QA (test)

ActivityNet-QA (test) is a benchmark for understanding videos instead of static images. The official description says: "ActivityNet-QA (test) as evaluated on the LFM3-VL-2B card (vision benchmark, with-thinking setting): Video category, with-thinking column". Below we describe in simple words what benchmarks of this type generally examine.

ActivityNet-QA (test) measures understanding videos instead of static images. In detail, it checks:

• following events, actions and their order across time
• answering questions about long videos (with and without subtitles)
• reasoning about what happened and why, not just single frames
• integrating audio/subtitle context with the visual stream

The score is accuracy: the percentage of tasks answered or solved correctly. 100% means every task was solved perfectly; higher is better.

Official methodology note: ActivityNet-QA (test) as evaluated on the LFM3-VL-2B card (vision benchmark, with-thinking setting): Video category, with-thinking column. ActivityNet-QA test split; no inventory match.

What is the ActivityNet-QA (test) benchmark?

ActivityNet-QA (test) is a benchmark for understanding videos instead of static images. The official description says: 'ActivityNet-QA (test) as evaluated on the LFM3-VL-2B card (vision benchmark, with-thinking setting): Video category, with-thinking column'. Below we describe in simple words what benchmarks of this type generally examine.

What does ActivityNet-QA (test) test?

ActivityNet-QA (test) mainly measures understanding videos instead of static images. Concretely, it checks: following events, actions and their order across time; answering questions about long videos (with and without subtitles); reasoning about what happened and why, not just single frames.

How is the ActivityNet-QA (test) score calculated?

The score is accuracy: the percentage of tasks answered or solved correctly. 100% means every task was solved perfectly; higher is better. The declared maximum score of this benchmark is 100.0.

Where do I get ActivityNet-QA (test) data and scores?

The official benchmark page/reference is linked on this page. Every score on Open LLM Radar is also available as open data: download the scores JSON (http://open-llm-radar-api:8000/api/v1/benchmarks/activitynet-qa-test/scores) or pull it directly from the Open LLM Radar API with any HTTP client.