AudioSet is a large-scale dataset of labeled audio events released by Google Research. It consists of roughly 2 million human-labeled 10-second sound clips sourced from YouTube videos, annotated against a structured ontology of over 600 audio event categories. The ontology is a hierarchical graph covering human and animal sounds, musical instruments and genres, and everyday environmental and mechanical sounds.
What’s in it
Because YouTube content cannot be redistributed directly, AudioSet is published as metadata: video IDs, start and end timestamps, and multi-label annotations, alongside precomputed 128-dimensional frame-level audio embeddings from a VGG-style network. Clips typically carry multiple labels, making it a weakly labeled, multi-label collection rather than a set of clean single-event recordings, and it is split into balanced and much larger unbalanced training sets plus an evaluation set.
Tasks, license & access
It is the standard benchmark for audio event recognition, sound tagging, and audio-visual learning, and it underpins models such as VGGish, PANNs, and later audio transformers. The labels and ontology are released under a CC BY 4.0 license and are freely downloadable, though users must retrieve the audio from YouTube themselves, so availability of individual clips varies over time as videos are removed.
- Modality: audio (as YouTube clip references plus embeddings and labels)
- Scale: ~2M 10-second clips, 600+ sound classes
- Tasks: audio event tagging, sound classification, audio-visual learning
- License: CC BY 4.0 labels/ontology; audio via YouTube
- Creator: Google Research
AudioSet’s scale and broad ontology made it central to training general-purpose audio recognition, an important non-visual modality for multimodal AI.