Setting the Standard: Recommended Practices for Data Preprocessing in Data-Driven Climate Prediction

Abstract

Artificial intelligence (AI)—and specifically machine learning (ML)—applications for climate prediction across time scales are proliferating quickly. The emergence of these methods prompts a revisit to the impact of data preprocessing, a topic familiar to the climate community, as more traditional statistical models work with relatively small sample sizes. Indeed, the skill and confidence in the forecasts produced by data-driven models are directly influenced by the quality of the datasets and how they are treated during model development, yielding the colloquialism, “garbage in, garbage out.” As such, this article establishes protocols for the proper preprocessing of input data for AI/ML models designed for climate prediction (i.e., subseasonal-to-decadal and longer time scales). The three aims are to 1) educate researchers, developers, and end users on the effects that data preprocessing has on climate prediction; 2) provide recommended practices for data preprocessing for such applications; and 3) empower end users to decipher whether the models they are using are properly designed for their objectives. Specific topics covered include creating (standardized) anomalies, dealing with nonstationarity and the spatiotemporally correlated nature of climate data, and handling of extreme values and variables with potentially complex distributions. Case studies will illustrate how using different preprocessing techniques can produce different predictions from the same model, which can create confusion and decrease confidence in the overall process. Ultimately, implementing the recommended practices set forth in this article will enhance the robustness and transparency of AI/ML in climate prediction studies.