First-hand information for everyone
AdaPixFlow discovers a continuous‑time dynamics operator that works across different scenes, given only a few RGB‑D views. It fuses those views into a common voxel grid, uses a 3‑D U‑Net to predict a velocity field, and then integrates object points through a differentiable ODE solver to estimate motion between and beyond observation times.
Evaluated on held‑out PhysInOne scenes, AdaPixFlow increases the F1 score by 2.5 % over the best competing baseline for in‑between prediction and by 15 % for future prediction. Relative to per‑scene velocity‑field methods it outperforms them by 30 % and 85 % in the same tasks.
| In-between prediction | Future prediction | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Method | |||||||||||||
| Input |
|---|
| Elastic Fall |
|---|
| DropIn Liquid |
|---|
| Crank Push |
|---|
| Panel Spin |
|---|
| Double Pendulum |
|---|
| Balloon Lift |
|---|
| Mean |
|---|
| Elastic Fall |
|---|
| Panel Spin |
|---|
| Double Pendulum |
|---|
| Balloon Lift |
|---|
| Mean |
|---|
| FreeGave | RGB, D0, M | 77.1 | 21.7 | 85.2 | 56.3 | 91.5 | 82.8 | 69.1 | 0.7 | 18.6 | 33.6 | 73.3 | 31.5 |
|---|
| TRACE | RGB, D0, M | 0.0 | 15.1 | 70.2 | 32.5 | 15.7 | 45.8 | 29.9 | 2.5 | 32.8 | 34.4 | 44.0 | 28.4 |
|---|
| SoM | RGB-D, M, 1 view | 77.5 | 49.1 | 73.5 | 72.2 | 82.2 | 49.3 | 67.3 | 13.5 | 23.8 | 44.0 | 43.5 | 31.2 |
|---|
| MoSca | RGB-D, M, 1 view | 73.5 | 19.5 | 40.6 | 33.6 | 52.2 | 40.3 | 43.3 | 5.9 | 31.0 | 27.1 | 50.6 | 28.6 |
|---|
| STv2 | RGB-D, M | 96.4 | 65.1 | 84.7 | 89.2 | 96.1 | 95.5 | 87.8 | 4.2 | 57.4 | 50.4 | 90.8 | 50.7 |
|---|
| OpenD4RT | RGB-D, M | 93.2 | 61.6 | 82.2 | 77.3 | 94.5 | 95.3 | 84.0 | 16.5 | 30.7 | 63.4 | 73.1 | 45.9 |
|---|
| AdaPixFlow | RGB-D, M | 97.3 | 77.9 | 87.2 | 86.3 | 95.4 | 95.8 | 90.0 | 22.0 | 47.6 | 80.7 | 83.2 | 58.4 |
|---|
The improvement is especially large for future steps that extend up to ten frames after the last input. Accuracy naturally degrades when observation spacing grows, and the model is less accurate on validation scenes featuring ground‑contact events compared to test scenes.
The performance gains are illustrated in Table 1, which reports per‑scene F1 scores for methods such as FreeGave, TRACE, and AdaPixFlow, and in Figure 3, which visualises the predicted 3‑D point clouds for in‑between and future prediction alongside ground truth and baseline methods.
