Ego4D

Ego4D is a large-scale collection of egocentric (first-person) video capturing everyday human activity as seen through wearable cameras. It comprises on the order of 3,600+ hours of unscripted footage recorded by roughly 900 unique camera wearers across dozens of locations in nine countries, spanning activities from cooking and shopping to crafting, socializing, and outdoor work. Many clips are accompanied by dense, time-stamped natural-language narrations describing what the wearer is doing.

What’s in it

Beyond RGB video, selected subsets add synchronized modalities including audio, 3D environment scans, eye gaze, IMU, stereo, and multi-camera views, enabling research that fuses vision with other signals. The collection was assembled by Meta AI (FAIR) together with a consortium of 13+ universities and labs worldwide, with a deliberate emphasis on geographic and demographic diversity rather than curated internet video. Data is distributed as video files plus structured JSON annotations, retrieved through a command-line downloader after credential setup.

Tasks & benchmarks

Ego4D ships with a benchmark suite organized into five tracks: episodic memory, hands-and-objects interaction, audio-visual diarization, social understanding, and future forecasting, each with its own tasks, splits, and metrics. It is widely used to pretrain and evaluate models for video understanding, action recognition, temporal localization, and embodied perception, and it seeded follow-on efforts such as Ego-Exo4D.

• Modality: egocentric video; audio, 3D, gaze, IMU and multi-view in subsets
• Scale: ~3,600+ hours, ~900 camera wearers, 74 locations, 9 countries
• Tasks: episodic memory, forecasting, hand-object interaction, AV diarization
• License: Ego4D License Agreement (signed data-use terms)
• Creator: Meta AI with a 13+ university consortium

Because it records the world from a human vantage point, Ego4D is a foundational resource for training multimodal and physical-AI systems (assistants, AR devices, and robots) that must reason about actions, objects, and intent from a first-person perspective.