First-hand information for everyone
A retrieval system that splits questions into subquestions and selects documents for maximal coverage improves the α‑nDCG@10 score from 0.625 to 0.702 on the TREC RAG 2024 benchmark. The method first trains a bi‑encoder to score documents for each subquestion, then predicts a document‑×‑subquestion coverage matrix with a cross‑encoder, and finally picks documents by greedy maximum‑marginal coverage. In the benchmark, adding coverage training raises α‑nDCG@10 from 0.625 to 0.656, and the subsequent selection stage adds another 0.046, reaching 0.702. A gold‑oracle selector that feeds the exact coverage matrix further improves the score to 0.887, showing the upper bound.
| Method | α-nDCG@10 | StRecall@10 | P@10 | nDCG@10 |
|---|
| Relevance-trained bi-encoder | 0.625 | 0.744 | 0.754 | 0.784 |
| + coverage training | 0.656 | 0.762 | 0.773 | 0.798 |
| + coverage estimator and greedy selection | 0.702 | 0.770 | 0.775 | 0.800 |
| Gold nuggets (decomposition upper bound) | 0.718 | 0.781 | 0.771 | 0.802 |
| Oracle greedy (selection upper bound) | 0.887 | 0.858 | 0.800 | 0.857 |
Compared to a relevance‑only baseline, the coverage‑aware method achieves a higher transfer rate (+0.050) and a higher sentence precision (+0.070) on TREC RAG 2024, though none of the five metrics remain statistically significant after Holm–Bonferroni correction.
| A | B | O | |||
|---|---|---|---|---|---|
| Metric | n | bi-encoder | ours | oracle | Δ(B−O) |
| StRecall@10 (retrieval) | 54 | 0.791 | 0.799 | 0.888 | −0.089‡ |
| Transfer rate | 54 | 0.294 | 0.326 | 0.275 | +0.050† |
| Answer StRecall | 54 | 0.239 | 0.266 | 0.247 | +0.020 |
| Vstrict | 51 | 0.350 | 0.351 | 0.322 | +0.029 |
| Sentence precision | 53 | 0.715 | 0.711 | 0.642 | +0.070† |
Despite higher retrieved coverage, an oracle selector that covers more nuggets does not deliver more of them to the answer. The relationship between retrieved coverage and answer coverage is not monotonic, suggesting that simply adding more evidence does not automatically improve the final answer.
These findings highlight how explicitly modeling coverage can sharpen the breadth of retrieved evidence while maintaining relevance, but also reveal that more retrieved coverage does not always translate to more complete answers.