Researchers evaluated two biomedical language models—BioBERT, with 108 million parameters, and BioMistral, with 7 billion parameters—on the TOP benchmark, which contains 12 477 registered trials. Each trial was represented by ten ClinicalTrials.gov fields. For each model, two training strategies were tested: full fine‑tuning for BioBERT, low‑rank adaptation (LoRA) for BioMistral, and a probing approach that froze the pretrained weights and trained only a lightweight classifier on the resulting embeddings.

On held‑out test sets for every phase of the benchmark, all four model variants achieved higher receiver‑operating‑curve area‑under‑curve (ROC‑AUC) and precision‑recall area‑under‑curve (PR‑AUC) than the previously published HINT system. The relative PR‑AUC gains ranged from 10 % to 20 %. Differences in F1 score were small, and none of the methods beat a trivial classifier that simply predicts success for every trial.

While the results suggest that domain‑specific language models can improve predictive performance on trial outcome data, the study did not report confidence intervals or significance tests. The reported differences are likely within the sampling uncertainty of the test sets.

Overall, these findings show that fine‑tuning or probing transformer‑based models tailored to biomedical text outperforms a multimodal baseline on a large trial‑prediction benchmark, though statistical certainty remains limited.