Real-Time Automated Quality Control of Extreme Precipitation Data from Automatic Weather Stations in Peru Using Deep Learning and Geostationary Satellite Images

Abstract

Accurate and timely extreme precipitation data are crucial for effectively predicting and mitigating the impacts of natural phenomena. In Peru, automatic weather stations operated by the National Meteorological and Hydrological Service of Peru (SENAMHI) collected approximately 3.5 million precipitation data points between 2020 and 2021. The automated phase of the quality control (QC) system at SENAMHI flagged 4% of the data as suspect due to extreme values, but only 53% of these suspect data were validated in the manual QC phase in a timely fashion, even though 98.8% of these were ultimately classified as correct. To address this, we propose a deep learning model using satellite images and auxiliary inputs to validate extreme precipitation data more efficiently in real time, trained with human flags from the manual QC phase. We utilized a convolutional neural network (CNN)–recurrent neural network (RNN) architecture and satellite images to determine whether an extreme precipitation value is correct. The model yields a true positive rate of 95.9% considering the default threshold probability (0.5), so these suspect data could be automatically approved and published with a low false positive (error) rate of 0.329%, which would strongly reduce the workload of the human meteorologists in the manual QC. This could be optimized further by lowering the threshold and increasing the automatic approval rate while keeping the error rate at an acceptable level for SENAMHI. Additionally, an out-of-time validation using data for 2023–24, obtained after the original development and testing, showed a relatively good generalization to new climatological conditions, including the 2023–24 El Niño, albeit with a somewhat reduced performance, highlighting the need for continuous monitoring and readjusting the model.