arXiv News

First-hand information for everyone

Site

AboutPrivacy PolicyTerms of Use

© 2026 arXiv News

New Cross‑View Dense Video Object Captioning Task Links Multiple Cameras into Unified Descriptions | arXiv News

arXiv News

EnglishJapanese

Switch language

EnglishJapanese
Loading account…

First-hand information for everyone

Back to front page
Computer VisionEnglishPublished

New Cross‑View Dense Video Object Captioning Task Links Multiple Cameras into Unified Descriptions

November 10, 2026arXiv: 2611.09617v1

Researchers have created a new task called Cross‑View Dense Video Object Captioning (CVDVOC). The goal is to combine object tracks from synchronized cameras into one global trajectory and produce a single caption for that trajectory.

A diagram showing multiple camera views feeding into a unified caption for a single object trajectory
Figure 1. Illustration of the Cross‑View Dense Video Object Captioning (CVDVOC) task: synchronized camera views are linked into global trajectories, each assigned a single unified caption.Zhang, Xiangqun, Wang, Likai, Qian, Zekun, Zhou, Rui, Han, Ruize, Feng, Wei · arXiv:2611.09617v1 · Figure 1: · CC BY 4.0

To support the task, a new dataset named CRTrack‑Cap was built. It contains 425 global trajectories and 838 camera‑local tracklets across 13 scenes, with reference descriptions for 335 identities.

The authors also propose a reference framework. First, View‑Exclusive Cross‑Camera Association (VECA) groups tracks from different cameras into global identities while ensuring only one track from each camera per group. Second, Conflict‑Evidence Gating (CEG) selects which camera views provide reliable visual evidence for each appearance attribute and gates their influence on the caption. Finally, a CaptionFormer‑based decoder generates the final unified caption.

Flowchart displaying the steps of VECA, CEG, and CaptionFormer in the proposed framework
Figure 2. Reference framework for cross‑view DVOC: local tracker outputs tracklets and appearance descriptors; VECA forms global identity groups; CEG gates visual evidence; CaptionFormer‑based decoder generates the final caption.Zhang, Xiangqun, Wang, Likai, Qian, Zekun, Zhou, Rui, Han, Ruize, Feng, Wei · arXiv:2611.09617v1 · Figure 2: · CC BY 4.0

When tested on CRTrack‑Cap with predicted tracklets, the framework achieved a CV‑CHOTA score of 83.43, a Global‑CapA of 88.19, and a CIDEr score of 157.04 on the in‑domain test split—outperforming existing methods DenseVOC, SMOTer, and a single‑view CaptionFormer.

On the cross‑domain test split the results were lower: a CV‑CHOTA of 33.48, Global‑CapA of 68.34 and CIDEr of 112.62. This shows the model’s performance drops when applied to scenes outside its training distribution.

In summary, CVDVOC and CRTrack‑Cap provide a benchmark for multi‑camera dense video object captioning, and the new framework demonstrates that careful cross‑view association and selective evidence integration improve both tracking and caption quality, though challenges remain in cross‑domain generalization.

Source paper

CRTrack-Cap: A New Benchmark for Cross-View Dense Video Object Captioning

Computer Vision
View on arXiv