First-hand information for everyone
A new benchmark study finds that the latest probabilistic AI weather models can improve where they think tropical cyclones will go, even while continuing to struggle with how strong those storms will be.
Background and approach
Numerical weather prediction — the physics-based computer models run by national weather services — has long been the gold standard for tropical cyclone forecasting. Over the past few years, however, machine‑learning models trained on large historical datasets have closed much of the gap for many routine weather forecasts, often at far lower computational cost. Whether those data‑driven systems can handle rare, high‑impact events such as tropical cyclones is a central, unresolved question.
To probe that question for the newest probabilistic AI systems, researchers benchmarked three operational or near–operational learned ensemble models — GenCast, FourCastNet 3 (FCN3), and FGN — against ECMWF’s operational physics‑based ensemble (IFS‑ENS). The study examined 178 tropical cyclones across two evaluation settings (a reanalysis setting using ERA5 as reference, and an operational setting using ECMWF’s high‑resolution analysis HRES‑fc0), and developed probabilistic, spatial and dependence diagnostics targeted at cyclone tracking, intensity, ensemble calibration, and the wind–pressure relationship that characterizes storm structure.
Key findings
- Substantial tracking gains. On cyclone location, the best AI model (FGN) assigned higher probability density to the observed storm positions than IFS‑ENS for roughly 90% of the storms retained in the 2025 operational comparison. Both FGN and GenCast improved on IFS‑ENS more often than not, and these gains extended beyond the storm center to the placement of strong winds and low pressures.
- Persistent intensity bias. Despite better placement, AI models tended to underestimate peak 10‑m wind speeds compared with high‑resolution operational analyses. GenCast and FGN showed systematic negative biases in maximum wind, especially at short and intermediate forecast leads. These intensity errors coexisted with improved probabilistic track scores.
- Calibration and spread issues. The AI ensembles still exhibited calibration problems and imperfect ensemble spread. Rank‑histogram diagnostics and other spread measures signalled that uncertainty quantification remains an area needing work.
- Physical dependence preserved. When the researchers removed uniform marginal differences (i.e., adjusted away intensity biases) and compared the joint dependence structure of lowest central pressure versus maximum wind, the AI models reproduced the expected wind–pressure relationship seen in analysis products. That is, although AI forecasts were often weaker in magnitude, the co‑variation between wind and pressure — a key physical signature of cyclones — was largely preserved.
Why these results matter
The study separates three elements often conflated in previous work: spatial placement (where the storm is), magnitude (how strong it is), and dependence (how variables like wind and pressure relate). The results suggest that learned models can capture spatial patterns and some joint physical relationships that remain predictive even for extreme weather. At the same time, systematic intensity underestimation and imperfect uncertainty remain important limitations.
Caveats and data limitations
The authors emphasize caution in interpreting the results. The evaluation is retrospective and focused on a selected set of storms; it does not test unconditional cyclone occurrence (cases where a cyclone did not form). The gridded reference products used for verification differ in their own biases: ERA5 is known to underestimate peak cyclone winds, and IBTrACS observational wind measures are not directly comparable to instantaneous gridded winds. Models also had different training histories and architectures (GenCast uses conditional diffusion, FCN3 uses spherical convolutions and CRPS training, FGN forms ensembles from multiple networks and learned perturbations), which complicates an apples‑to‑apples interpretation. Finally, operational fine‑tuning and available forecast archives differed across models.
Implications and next steps
The findings point to practical routes for improving learned forecasts: targeted intensity correction, loss functions that place more weight on extreme winds, and better uncertainty quantification. They also motivate more diagnostic work to disentangle errors inherited from training targets (for example, biases in ERA5) from limitations of model architectures and training objectives. Comparative experiments that control for architecture and compute — for example, comparing marginal (per‑variable) versus joint (multivariate) training objectives on otherwise matched models — could clarify how and why physical dependencies emerge in AI forecasts.
Policy and operational caution
The authors and reviewers stress that this research is not operational guidance. Tropical cyclone warnings drive life‑critical decisions; any operational use of learned models would require thorough evaluation under real‑world conditions, including seasons with no cyclones and rigorous calibration of occurrence probabilities.
Bottom line
Probabilistic AI weather models are showing meaningful progress at predicting cyclone location and the spatial footprint of strong winds and low pressures, but they continue to underestimate peak intensity and need better uncertainty handling. The coexistence of improved spatial skill and preserved physical dependence alongside magnitude and calibration errors suggests that targeted fixes could yield practical benefits for extreme‑weather forecasting — provided evaluations remain careful, probabilistic, and focused on the right metrics.