New spline‐based Additive Model Secures Top Third Rank on Large Tabular Benchmarks and Shines Under Distribution Shifts
arXiv: 2611.05012v1
CoGA2M, a multi‑resolution, higher‑order composite Generalised Additive Model, fills two gaps in spline‑based GAM benchmarking: it is evaluated on the large‑scale TALENT tabular learning suite, and its robustness to out‑of‑distribution shifts is assessed. The model builds a differentiable spline layer that concatenates univariate linear and cubic Hermite splines at 4, 16 and 64 knot resolutions and their bivariate products. Training is end‑to‑end by back‑propagation, with an L1 sparsity penalty and a smoothness penalty that encourages gentle spline curves.
On TALENT, CoGA2M ranks in the top third of all competitors: 11th of 29 for regression tasks and 7th of 32 for classification. In the newly created OOD benchmark, consisting of 22 regression datasets with purely numerical features, CoGA2M outperforms LightGBM, the most successful baseline on TALENT. For each dataset the drop in $R^2$ from in‑distribution to OOD data is smaller for CoGA2M than for LightGBM, and 13 of the 22 comparisons show a stronger advantage for the spline model. The table below summarises these differences.
Table summarising R² differences for 22 regression datasets, comparing CoGA2M and LightGBM; bold entries show superior CoGA2M performance.
TABLE I. Figure shows the differences in R² between in‑distribution and out‑of‑distribution data for each of the 22 datasets, comparing CoGA2M and LightGBM; the highlights (bold) indicate where CoGA2M outperforms LightGBM.Dimitriadis, George, Machlanski, Damian, Clarke, Paul, Samothrakis, Spyridon · arXiv:2611.05012v1 · TABLE I: · CC BY 4.0