Neural networks in weather radar nowcasting
Exploring the effect of passing other meteorological data to the neural networks for radar echo extrapolation.
Everyone has been in a situation, when clouds appeared on the horizon, wondering whether it will rain. While it is uncomfortable to get wet and maybe even dangerous for individuals in case of storms, these questions about incoming clouds can become in some scenarios crucial. One example for all is from the air traffic field, where severe weather affects transportation and can endanger many people at the same time.
In such cases, weather radars can be of great assistance, thanks to their ability to complexly display precipitation in real-time. While describing the current situation helps, ability to see the development in the near future enhance the planning and prevention tasks even more. These short-term weather forecasts got name nowcasting, and there is active research ongoing in this field in recent years.
In weather radar nowcasting (also called weather radar echo extrapolation) the first methods were based on optical flow — vectors describing the motion of individual pixels in subsequent frames. COTREC represents one of these methods, which is still widely used, despite its 30 years of history. The only nowcasting system currently available in the Czech Republic is based on it.
However, researchers design recent approaches with various neural network architectures. These machine learning algorithms can utilize the vast amount of data measured in the last decades and cope well with uncertainty. The uncertainty of future precipitation is significant due to the fact, that precipitation is influenced not just by the recent development of it but rather by several physical quantities of the atmosphere as pressure, temperature, direction and speed of the wind or humidity. Still, most of the research looks at the radar echo extrapolation task as on the prediction of frames of a video with a static camera and dynamic motion in front of it.
My research has two objectives. The first one is to explore the effect of adding more information about the atmosphere and other influencing aspects to the prediction model on its performance. Secondly, the lack of a proper nowcasting system in the Czech Republic creates an opportunity to publish the real-world predictions of the created model and qualitatively evaluate them by the broad public in the country. This evaluation can reveal if the used metrics correlate with the way how people use weather radar data.
Background
Weather radars measure an immediate precipitation intensity from a distance by sending electromagnetic beams in every location and capturing reflections. Every object with the electrically leading surface, which applies to the drips of water too, reflects back these electromagnetic beams. The radar reads all of these reflections, which are slightly different for every type of object. After filtering out signals from birds, planes, ground and everything else, an image similar to the following can be displayed.
The most commonly, weather radar measurements are shown in dBZ. It stands for decibels relative to Z, and it is a logarithmic dimensionless quantity. dBZ is obtained by taking the logarithm of the radar reflectivity of the object to the reflectivity of a rain droplet with 1mm diameter.
There are two radar networks in the Czech Republic. The one of the Czech Hydrometeorological Institute can produce these images once in 10 minutes. The other one, built by the company Meteopress, consists of a higher number of different less powerful radars and creates images in 1-minute intervals.
Brief specifications of the used networks.
Neural networks used for radar the echo extrapolation primarily utilize two machine learning concepts.
Convolutional neural networks (CNN) are in general build with the layers that slide learned filter performing some arithmetic operation over each region of the input. This course notes from Stanford can serve as a tutorial.
Recurrent neural networks (RNN) process the input sequentially. In our case, it is a sequence of subsequent weather radar images. The most commonly used RNN is called LSTM and is well explained in the colah’s blog.
Evaluation metric
The most basic metrics used in machine learning are the mean absolute error (MAE) and mean squared error (MSE). Respectively, they are based on the Minkowski distance of the first and second order. However, this pixel by pixel comparison does not tell the expected information, whether it rains at the predicted locations.
Therefore, the critical success index (CSI) is often used across the nowcasting literature. Both the prediction and ground truth images are first converted to binary matrices indicating if it rains at a particular pixel, according to some minimal rainfall intensity. Afterwards, the number of hits, misses and false alarms is calculated. The CSI has a form of
CSI = hits / (hits + misses + false alarms).
Related work
My current work is primarily based on my bachelor’s thesis [1], which I wrote at the Faculty of Information Technology CTU in Prague. In the thesis, I look on the radar echo extrapolation purely as on the video frame prediction. I propose a simple, fully convolutional network (every layer has a form of a filter slid over every position of the input). It takes as input three consecutive weather radar images concatenated in the colour channel dimension and was trained to predict the next frame. I chose the structural similarity SSIM index as the evaluation metric and evaluated the model on a dataset created from the weather radar images above the Czech Republic. The chose of the SSIM is questionable, in the light of the later research, but the results show that even a simple CNN can outperform the COTREC.
Convolutional LSTM
Authors in [2] formulate the problem as a spatiotemporal sequence (related to both space and time) prediction problem. In a traditional LSTM network, the trainable parts are fully-connected (FC) layers. While encoder-decoder models based on LSTMs perform well in many sequence modelling tasks (e. g. speech recognition), the FC layers struggle to capture the spatiotemporal correlations from images, due to a large number of unstructured parameters. As a solution, authors introduce a ConvLSTM network that happens to become a framework for many later nowcasting models. Generally speaking, they replaced the FC layers in the LSTM cells for convolutional ones with 3D tensors passing through the network. On their radar echo dataset, ConvLSTM performed better than both LSTM and ROVER algorithm [5] (a modern optical flow nowcasting algorithm).
State-of-the-art model
Authors built the current state-of-the-art architecture on ConvLSTM cells in [3]. The essence is in rewiring the information flow between particular ConvLSTM cells in the encoder-decoder architecture to form a star-shape information bridge.
Moreover, the authors replace the often-used batch normalization (BN) for group normalization [4] (GN). In the image processing tasks, the size of the data passed to the input sharply limits the possible number of samples in the mini-batch. As a consequence, it bounds the performance of BN as well. On the other hand, the GN groups channels of the input tensor to groups and computes needed mean and variance over them. The reported performance difference of these normalization techniques is remarkable.
Finally, for training, authors use multi-sigmoid loss, which simulates the undifferentiable CSI score. The created model achieves state-of-the-art on their East China radar echo dataset.
First results
As I mentioned above, future development of precipitation depends on many aspects. Revealing some information about these aspects to the ML model for radar echo nowcasting could theoretically lead to performance improvement. The most accessible data addable to the network is the altitude profile of the area scanned by the radar. I use it in the form of images with the same dimensions as the input weather radar ones. Each pixel denotes the elevation in that location, which is the height in meters above the sea level scaled with division by 2000.
The neural network used for the initial testing is a fully convolutional network derived from the one proposed in my bachelor’s thesis. The only difference is in the format of the input and output. The depth of the input 3D tensor is 4 — the last three consecutive weather radar images and the altitude profile of the scanned area. Using MAE as the loss function, the network is trained to predict the three following frames at once.
Dataset
The dataset used for training and evaluation of the models is created from the weather radar data capturing precipitation above the Czech Republic, which was provided by the weather forecasting company Meteopress. The RGB images were decoded into dBZ and scaled to [0,1]. Afterwards, I have cropped sequences of the 320x320 input images from the series of the full frames and removed the ones with less than 10% of the area covered with rain. The rest were split into rows with 24 frames. Data augmentation during training has two steps. First, the starting point of the shorter sequence used for training is chosen randomly, and than it is also randomly rotated.
Effect of adding elevation to the input
I have compared the network with and without altitude information with the CSI score. Both models were trained for the same 15 epochs, guided by the MAE loss function.
The graph displays the CSI value for various thresholds of rain intensity. It indicates that adding elevation data always improves performance. There is a phenomenon of decreasing index with the rising threshold. This behaviour is caused by the fact that the areas with stronger precipitation are smaller and therefore harder to predict.
Next steps
Achieving good results in quantitative evaluation is just one side of the coin. The other one is the qualitative evaluation by the public who judge by the real usability of the created solution. With this in mind, the current version of the predictions is going to be shortly published on the Meteopress web pages. Measured score from the user feedback will serve as a baseline for testing the following models in the same way.
The model described in this blog is quite simple. The following step is to reimplement the state-of-the-art StarBriNet and trained it on the created dataset. Again, predictions of the finished reimplementation will be published and publicly evaluated. Finally, I will explore the effect of adding more data describing the state of the atmosphere and location to the StarBriNet and try to improve the performance of this state-of-the-art network on my dataset.
When successful, the created model will be available online and will serve nowcasting predictions for the Czech Republic.
References
[1] Choma, Matej. Interpolation and Extrapolation of Subsequent Weather Radar Images. Bachelor’s thesis. Czech Technical University in Prague, Faculty of Information Technology, 2019.
[2] Shi, X., Chen, Z., Wang, H., Yeung, D., Wong, W., & Woo, W. Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting. ArXiv, abs/1506.04214. 2015.
[3] Cao, Y., Li, Q., Chen, L., Zhang, J., & Ma, L. Video Prediction for Precipitation Nowcasting. ArXiv, abs/1907.08069. 2019.
[4] Wu, Y., He, K.: Group normalization. ECCV. 2018.
[5] Woo, W.C., Wong., W.K. Application of optical flow techniques to rainfall nowcasting. In the 27th Conference on Severe Local Storms. 2014.
See more FIT CTU research blogposts.
