Ego-Exo4D
Synchronized first-person and external-camera recordings of skilled human activities with rich annotations.
About
Ego-Exo4D captures skilled activities from both wearable and external cameras. The synchronized perspectives support research into how actions look to the person performing them and to observers in the surrounding space.
Start by checking temporal synchronization and camera coverage for one activity. A shared timeline helps reveal occlusion, hand-object interaction, and viewpoint gaps before model design begins.
Suggested agent workflows
Copy a prompt into your agent of choice. Each workflow starts small, respects the source terms, and makes the result inspectable in Rerun.
Audit a training slice
Fetch Ego-Exo4D from its official source at https://ego-exo4d-data.org/. Read and preserve its source terms and citation requirements. Build a reproducible manifest for a representative slice, then load it into Rerun. Inspect the Egocentric video, External video, Audio, Poses streams for timestamp gaps, missing samples, calibration problems, and annotation inconsistencies. Summarize the findings and save both the validation script and a Rerun recording that makes every issue easy to review.
Build a baseline pipeline
Use Ego-Exo4D to scaffold a baseline pipeline for Activity understanding, Pose, Tracking. Start with a small deterministic cohort and document the selection criteria. Create the dataset adapter, preprocessing steps, train and validation splits, and one lightweight baseline. Log inputs, labels, predictions, losses, and representative failures to Rerun so I can inspect the full experiment timeline. Include exact setup and reproduction commands.
Find useful failure cohorts
Analyze Ego-Exo4D and propose a practical cohort taxonomy for Activity understanding, Pose, Tracking. Use available metadata and the Egocentric video, External video, Audio, Poses signals to identify hard cases, rare conditions, coverage gaps, and possible leakage between splits. Materialize the cohorts as queryable metadata, log examples from each cohort to Rerun, and produce a short report explaining which cohorts should become regression tests. Do not infer labels that are not present in the source data.