First-hand information for everyone
This paper checks how large, pre-trained medical “foundation” models behave when asked to find traumatic bowel injury on CT scans. Foundation models are trained on many kinds of medical images and text, so they can sometimes work without extra training. The authors found that these models could identify injured cases about as well as models trained for the task, but they made many more false alarms when other injuries were present.
The team used a multi-institutional CT dataset collected by RSNA from 23 centers (2019–2023). They trained and evaluated five approaches on a training set of 3,147 patients, where bowel injury was rare (2.3% prevalence). Two foundation models were tested: MedCLIP used in zero‑shot mode (no task-specific training) and RadDINO used as a frozen feature extractor with a simple classifier. They compared these to three task-specific systems (a CNN, a transformer, and an ensemble). To isolate why models failed, the authors compared specificity — the ability to correctly say “no injury” — in two groups that both had zero bowel injuries: patients with other solid organ injuries (liver, spleen, kidney; n=58) and patients with no abdominal pathology (n=50).
Overall discrimination, measured by area under the curve (AUC), was similar: foundation models scored 0.64–0.68 versus 0.58–0.64 for task‑specific models. Foundation models had higher sensitivity (they caught more true positives): 79–91% versus 41–74% for the task‑specific approaches. But foundation models had lower specificity overall (more false alarms): 33–50% versus 50–88% for task‑specific systems. All models kept high specificity (84–100%) in the no‑pathology group. The big problem appeared when solid organ injuries were present: specificity fell by about 50–51 percentage points for foundation models, while task‑specific models showed smaller drops of 12–41 points.
Why this matters: traumatic bowel injury is uncommon but can be deadly when missed, so tools that increase detection could help. Foundation models are attractive because they can work without collecting many labelled examples of a rare disease. But the large drop in specificity when other injuries are present means these models would produce many false positives in typical trauma patients. False positives can lead to extra tests, delays, and unnecessary interventions.
Important caveats: this was a retrospective study on a public dataset and the main test set was intentionally enriched with positive cases to enable measurement. That enrichment changes the apparent prevalence compared with real clinical practice. The authors controlled for prevalence in their stratified analysis, which supports their conclusion that the specificity drop is driven by heterogeneity in the negative class (different kinds of non‑bowel pathology), not just by prevalence mismatch. They also report that susceptibility to this problem decreases with more labelled, task‑specific training, suggesting that foundation models need targeted adaptation before use in clinics.
The clear takeaway is cautious. Foundation models can match overall accuracy for this rare, varied injury without task training, but they are vulnerable to confusion when other injuries are present. The paper proposes a simple testing approach — compare specificity across negative subgroups with the same true prevalence — as a way to diagnose whether false alarms come from prevalence mismatch or from confounding pathology. Before these models are used for real patients, developers should adapt and test them specifically on the kinds of mixed injuries common in trauma care.