EgoActionBench introduces 1,407 egocentric video episodes and 41,132 human‑refined atomic action labels. Each label indicates the left hand, right hand, or both hands that perform the action (f1). The dataset breaks down continuous manipulation into brief, temporally precise events, a step forward from prior benchmarks that lack atomicity or hand–action links.

To evaluate models, EgoActionBench uses a three‑stage protocol. First, predictions that overlap in time with a ground‑truth action are gathered. Next, the prediction whose semantic description best matches the ground truth is chosen. Finally, scores are computed separately for semantic fidelity, temporal overlap, and hand association (f2). This method keeps segmentation errors distinct from semantic mistakes, allowing clearer insight into model performance.

Large multimodal models were fine‑tuned and reinforced on this benchmark. After training, the Qwen3‑VL‑4B variant improved its overall EgoActionScore by 15.9 points, and the LLaVA‑OneVision‑2‑8B variant by 23.2 points, outperforming all open‑source baselines that were tested (f3). Gains appeared across temporal localization, hand assignment, and most semantic fields.

Despite these advances, current VLMs still struggle with very short actions. Models routinely achieve low temporal‑IoU for actions shorter than 0.5 seconds and have trouble identifying motion types such as lifting and rotating. The benchmark tables show that even the best open‑source models lag behind human annotators on hand, motion, and overall accuracy (f4).

EgoActionBench therefore offers a new, fine‑grained standard for egocentric action understanding and highlights the remaining gaps that future research must address.