AI ensemble models track tropical cyclones better than IFS-ENS but still miss intensity

Why it matters

Doubts about AI weather models centre on rare, damaging events. These results suggest the picture is mixed: location and spatial structure look strong, while intensity and uncertainty remain weak. The authors argue that extreme-weather skill should be judged by separate parts (location, intensity, spread, dependence), not by global averages alone.

What they did

The authors compared GenCast, FourCastNet 3 and FGN with ECMWF’s operational ensemble, IFS-ENS. They used 178 historical cyclones with winds of at least 64 knots, in two settings (125 storms in 2021–2023 against ERA5, and 53 in 2025 against HRES-fc0). Each model ran 15-day forecasts with about 50 members. They scored track density, field errors near storms, peak-wind bias, placement of the strongest winds, and the wind–pressure relationship.

Key findings

• FGN gave more probability density to the observed storm position than IFS-ENS for about 90% of the 41 retained storms in 2025.
• In 2021–2023, GenCast beat IFS-ENS on track density for 68% of retained storms, and FCN3 did so for 58%.
• Both operational AI models gained more over IFS-ENS in field error near cyclones than globally, and FGN placed the strongest winds and lowest pressures best.
• FGN and GenCast underestimated peak 10m winds, and ensemble spread was miscalibrated: too narrow for the AI models, too wide for IFS-ENS.
• FGN and GenCast reproduced a wind–pressure rank correlation of −0.83 and −0.84, against −0.88 for HRES-fc0, despite weak intensities.

Limitations

• Only storms that reached at least 64 knots were studied, with forecasts started before the first wind record. Cases with no cyclone were not tested, so occurrence probabilities are not assessed.
• The two settings use different storms, models and baselines, so the effect of fine-tuning cannot be isolated. The 2025 sample is small (41 retained storms).
• ERA5 itself underestimates peak winds, and the study does not identify what the models have learned. This is a preprint that has not been peer reviewed.

Glossary

• IFS-ENS: ECMWF’s operational physics-based ensemble forecast, used here as the baseline.
• CRPS: A score that compares a forecast distribution with the observed value; lower is better.
• Negative log-likelihood (NLL): A score for how little probability density an ensemble puts on the observed storm position; lower is better.
• Copula: The description of how two variables move together, once each variable’s own distribution is removed.