CountQA

Dataset Summary CountQA is the new benchmark designed to stress-test the Achilles’ heel of even the most advanced Multimodal Large Language Models (MLLMs): object counting. While modern AI demonstrates stunning visual fluency, it often fails at this fundamental cognitive skill, a critical blind spot limiting its real-world reliability. This dataset directly confronts that weakness with over 1,500 challenging question-answer pairs built on real-world images, hand-captured to feature… See the full description on the dataset page:

Source: Hugging Face Hub (Jayant-Sravan/CountQA). Metadata imported from the dataset’s Hub tags.