First-hand information for everyone
Coda‑DaVinci is a new agentic framework that decides when to let a text‑to‑video model generate a scene from scratch and when to guide it with an explicit 3‑D proxy. The system adapts the proxy’s granularity—coarse models for human bodies and medium‑grained models for objects—and runs physics simulations when the scene needs realistic interactions.

The framework was evaluated on three public benchmarks (WorldMemBench, VGIF‑Bench, and Principia). Across all tests, the configuration G1/P1/PE2—medium‑grained proxies, physics simulation enabled, and enhanced prompting—achieves the highest average human‑judged score for consistency, instruction following, physical plausibility and naturalness. The data show that agents using coarse proxies tend to produce more natural motion, while medium‑grained proxies improve overall consistency.
| WorldMemBench | VGIF-Bench | Principia | Average | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Configuration | C↑ | I↑ | P↑ | N↑ | C↑ | I↑ | P↑ | N↑ | C↑ | I↑ | P↑ | N↑ | C↑ | I↑ | P↑ | N↑ |
| Direct generation | 2.182 | 1.242 | 2.485 | 2.333 | 2.483 | 1.828 | 2.207 | 2.000 | 2.562 | 2.406 | 2.000 | 2.156 | 2.404 | 1.819 | 2.234 | 2.170 |
| G0/P0/PE0 | 2.758 | 3.000 | 2.606 | 2.455 | 2.429 | 2.276 | 1.897 | 1.414 | 2.969 | 2.938 | 2.562 | 2.594 | 2.731 | 2.755 | 2.372 | 2.181 |
| G1/P0/PE0 | 2.606 | 3.000 | 2.576 | 2.455 | 2.690 | 2.207 | 1.448 | 1.034 | 2.969 | 3.000 | 2.656 | 2.719 | 2.755 | 2.755 | 2.255 | 2.106 |
| G2/P0/PE0 | 2.788 | 3.000 | 2.606 | 2.455 | 2.552 | 2.069 | 1.448 | 1.000 | 2.906 | 2.875 | 2.531 | 2.562 | 2.755 | 2.670 | 2.223 | 2.043 |
| G0/P1/PE0 | 2.606 | 2.970 | 2.758 | 2.697 | 2.448 | 2.276 | 1.759 | 1.345 | 2.969 | 2.969 | 2.531 | 2.625 | 2.681 | 2.755 | 2.372 | 2.255 |
| G1/P1/PE0 | 2.818 | 3.000 | 2.788 | 2.606 | 2.586 | 2.276 | 1.690 | 1.207 | 2.969 | 2.938 | 2.688 | 2.781 | 2.798 | 2.755 | 2.415 | 2.234 |
| G2/P1/PE0 | 2.727 | 3.000 | 2.727 | 2.545 | 2.414 | 2.241 | 1.379 | 0.931 | 2.969 | 2.938 | 2.750 | 2.562 | 2.713 | 2.745 | 2.319 | 2.053 |
| G1/P0/PE1 | 2.455 | 2.970 | 2.636 | 2.606 | 2.414 | 2.172 | 1.345 | 0.897 | 2.062 | 2.469 | 2.438 | 2.281 | 2.309 | 2.553 | 2.170 | 1.968 |
| G1/P0/PE2 | 2.788 | 3.000 | 2.576 | 2.545 | 2.586 | 2.310 | 1.448 | 1.034 | 2.938 | 2.969 | 2.531 | 2.656 | 2.777 | 2.777 | 2.213 | 2.117 |
| G1/P1/PE1 | 2.182 | 2.697 | 2.515 | 2.273 | 2.724 | 2.897 | 1.828 | 1.379 | 3.000 | 3.000 | 2.111 | 2.278 | 2.562 | 2.837 | 2.175 | 1.950 |
| G1/P1/PE2 | 2.818 | 2.970 | 2.727 | 2.515 | 2.966 | 2.931 | 2.034 | 1.448 | 3.000 | 3.000 | 2.500 | 2.594 | 2.926 | 2.968 | 2.436 | 2.213 |
While the proxy‑guided approach outperforms straight generation on these metrics, it has limits. In long‑video experiments, the system sometimes fails to maintain scene stability or return to the prescribed setting after an instruction, and it struggles to keep objects correctly placed through successive interactions. Direct generation, on the other hand, often loses scene state fidelity between steps, leading to inconsistent object placements and properties.
Overall, Coda‑DaVinci demonstrates that a task‑adaptive mix of proxy modeling and direct generation can improve visual realism and physical plausibility, but further work is needed to handle long‑duration consistency and strict scene adherence.