First-hand information for everyone
Conformal prediction is a popular way to wrap any black‑box model with a guarantee that prediction sets cover the true answer at a stated rate on average. But “on average” can hide big errors at particular covariate values. Randomly localized conformal prediction (RLCP) tries to fix that by giving more weight to calibration data near the test point. This paper gives the first finite‑sample, high‑probability guarantees for the actual localized prediction set that RLCP produces. The results control both the local coverage error (how often the set misses a true label near the point) and the set’s length compared with an ideal oracle interval.
To get these guarantees the authors study two cases. First they fix a scoring rule (a way to rank how surprising a candidate label is) and assume a mild smoothness condition on how the score distribution changes with the covariates (a Hölder condition), together with standard assumptions on the covariate density and the kernel used for localization. Under those assumptions they prove that, with high probability over the calibration sample and conditional on the realized auxiliary localization centre, the coverage error and the excess length are uniformly small for every test point inside the realized neighbourhood. The error splits into two parts: a localization bias that shrinks like a power of the bandwidth h (written O(h^β)), and a calibration term that shrinks as the effective local calibration sample grows. Balancing these two terms recovers the usual nonparametric rate (roughly n^{-β/(2β+d)} up to log factors), which makes explicit the familiar bandwidth bias–variance tradeoff in this setting.
Second, the paper treats learned scores that are fitted on an independent training sample. When the learned score targets a pivotal quantity (a score whose relevant quantile is the same everywhere, as in conformalized quantile regression), the localization bias vanishes. In that case the local guarantees decompose cleanly into a calibration term and a score‑estimation term. The analysis shows that better score learning directly sharpens the localized RLCP guarantees, and that estimation error affects the threshold comparison in a controlled, linear way.
These results matter because they connect a practical localized conformal procedure to an oracle benchmark in finite samples. Practitioners gain guidance on how to pick the localization bandwidth and how to trade off bias and effective calibration sample size. The bounds are uniform over the actual neighbourhood selected by the method, so they describe the realized prediction set rather than only an average property.
Important caveats apply. The guarantees require specific regularity assumptions: Hölder continuity of the conditional score distribution, a local lower bound on covariate density, and standard kernel and design conditions. The finite‑sample statements are conditional on the realized localization centre and calibration sample (they are sample‑specific), and the results are proved for RLCP rather than for every localized or conditional conformal approach. Also, exact distribution‑free conditional coverage at every covariate value is impossible in finite samples without extra assumptions, so the paper gives controlled approximations rather than a full solution. The authors illustrate the bandwidth trade‑off and the way score estimation error propagates in numerical experiments reported in the paper.