First-hand information for everyone
Researchers examined 11 public pretrained encoders on five 2‑D segmentation tasks. Each encoder’s frozen features were paired with independently trained 1×1 readouts, run with four random seeds. The study measured per‑case Dice scores, a common accuracy metric, and compared how often different model pairs failed on the same cases.
The pairings that used the same checkpoint—meaning they shared the exact pretrained weights—had the highest mean Dice correlation across all cases. On average, these shared‑checkpoint pairs reached a correlation 0.30 points higher than cross‑family pairs (Δ = 0.265–0.640), a difference that a 95 % confidence interval shows is statistically significant.
Fine‑tuning the encoders or adding richer readouts narrowed the correlation gap but never consistently increased the effective size of a failure‑correlated pool at the 33rd‑percentile Dice threshold used in the analysis. Therefore, simply fine‑tuning does not guarantee greater diversity of failures.
The audit used a controlled sample of 11 public checkpoints; it is not a census of all available encoder models, so the findings may not hold for every possible encoder.
All tasks were slice‑based 2‑D evaluations. In the BraTS dataset, foreground target labels were merged, which might influence dependence estimates for full 3‑D segmentation tasks.
These results emphasize that shared pretrained checkpoints produce highly correlated failure patterns, a key consideration when building robust segmentation pipelines.