HTML
--> --> --> -->3.1. MMCR
The MMCR used in this study is located in the southern suburb observatory in Beijing. It operates at 8.6 mm wavelength and 35 GHz frequency. Its antenna scans the sky in the vertical direction and its maximum height of observation is 15 km. The vertical spatial resolution is 30 m and the temporal resolution is 60 s. The major technical and subsystem parameters are listed in Table 1. The MMCR uses three vertically pointing observation modes to detect simultaneously, and each observation mode corresponds to a different pulse width. Table 2 shows the specific parameters of the three observation modes. Figure 4 is a schematic diagram of the MMCR’s system configuration (Nashashibi and Ulaby, 2001), in which both indoor and outdoor parts are included.Item | Technical specifications |
Radar system | Coherent, pulsed doppler, solid-statetransmitter, pulse compression |
Radar frequency | 35 GHz (Ka band) |
Detecting parameters | Reflectivity factor (Z), vertical velocity (V), spectral width (W) |
Transmitter peak power | Pt ≥ 50 W |
Antenna subsystem | Antenna type: Cassegrain |
Antenna diameter: 2.4 m | |
Beam width: 0.25° | |
Range of detection | Height: 0.120–15 km |
Temporal resolution: 60 s (adjustable) | |
Height resolution: 30 m |
Table1. Major technical parameters for the MMCR.
Pulse width | 2 μs | 5 μs | 20 μs |
Minimum range | 360 m | 960 m | 3060 m |
Maximum range | 2460 m | 5010 m | 18 510 m |
Dwell time | 4 s | 4 s | 4 s |
Number of range bins | 30–100 | 65–200 | 210–617 |
Range sample volume spacing | 30 m | 30 m | 30 m |
Table2. Specific parameters of three observation modes for the MMCR.
Figure4. Schematic diagram of the MMCR system configuration.
The base data and power-spectrum data from the MMCR are used in this study. The base data comprise the reflectivity factor, velocity, and spectral width. The power-spectrum data are obtained from Fast Fourier Transform of the time-domain signal, and reflect the echo power distribution of different Doppler velocity. Each radar range corresponds to a power spectrum, which is composed of 256 spectral points, and each spectral point corresponds to a Doppler velocity. The power-spectrum data are closely related to the microphysics and dynamics in the cloud, and are critical for extracting the features of radar echoes.
2
3.2. Laser ceilometer
The laser ceilometer used in this study is Vaisala CL51, which is manufactured by Vaisala in Finland and certified by the CMA. Its maximum detection height is 15 km and its repeat frequency is 10 kHz, with a spatial resolution of about 5 m and temporal resolution of 60 s.The laser ceilometer emits continuous laser pulses vertically upwards. When the laser pulses enter and pass through the clouds from the cloud base, strong scattering at the cloud border will take place and the backscattered signal is received (He et al., 2011). Thus, the laser ceilometer can be used to assist in determining the cloud base height.
2
3.3. All-sky camera
The all-sky camera is designed to automatically monitor cloud conditions during daytime. Its camera above the instrument looks down at a hemispherical mirror with a heating device and takes the image of the sky reflected in the mirror. The average operation time of the camera used in this study is 0600–2000 Local standard time (LST hereafter). The all-sky images, with a resolution of 2848 × 4288 and 24 true colors, are taken every 10 min.2
3.4. L-band radiosonde
In meteorological observation, the radiosonde system is important for upper-air observation. The maximum detectable height of the L-band radiosonde system is 30 km. The radiosonde sounding data used in this study are from the southern suburb observatory in Beijing, which are observed twice a day at 0715 and 1915 LST.2
3.5. Detectability comparison between the MMCR and laser ceilometer
Discriminating clear-air echoes from cloud echoes becomes extremely challenging when the echoes mix in the lower level of the atmosphere, meaning it is necessary to first check the cloud detection ratios (the ratio between the time when one instrument detects the cloud and the time when both instruments are in operation) of the MMCR and the laser ceilometer for detecting clouds at different heights. The observed clouds are divided into three categories according to their base height. Clouds with a cloud base height of less than 2.5 km are low clouds, 2.5–4.5 km are middle clouds, and higher than 4.5 km are high clouds. Table 3 shows the cloud detection ratios of the MMCR and the laser ceilometer during selected observation times when both instruments were in operation.Cloud type | Selected observation time when both instruments are in operation (min) | Total time when MMCR detects cloud (min) | Total time when laser-ceilometer detects cloud (min) | Cloud detection ratio of MMCR (%) | Cloud detection ratio of laser-ceilometer (%) |
Low cloud | 420 | 265 | 402 | 63.1 | 95.7 |
Middle cloud | 4394 | 3234 | 3086 | 73.6 | 70.2 |
High cloud | 7808 | 6502 | 4115 | 83.3 | 52.7 |
Table3. Cloud detection ratio of the MMCR and the laser ceilometer in selected observation periods for clouds of different heights.
It can be seen from Table 3 that the detection of high cloud, i.e., the cloud detection ratio of the MMCR, is larger than that of the laser ceilometer, which may be caused by the thin cloud particles of high clouds when backscatter to the laser ceilometer becomes weak and less detectable. For the detection of middle clouds, the MMCR and the laser ceilometer perform similarly, with only a slightly higher detection ratio for the MMCR. For low-cloud detection, the cloud detection ratio of the laser ceilometer is significantly higher than that of the cloud radar. Therefore, it is necessary to combine both instruments to determine the actual cloud base height accurately.
In accordance with commonly used indicators of statistical significance, we employ the t-test method for significance testing, with the level of significance (α) set to 0.05. The cloud base height detected by the MMCR and the laser ceilometer from 1 December 2015 to 31 January 2016 are compared, as shown in the scatterplot in Fig. 5, and a clear linear correlation is shown, with a significant correlation coefficient of R = 92%, which is greater than the t-test critical correlation coefficient and thus is significant. Therefore, the results show that the cloud base height observed by the MMCR and the laser ceilometer is consistent.
Figure5. Cloud base height consistency of the MMCR and the laser ceilometer.
2
3.6. Comparison of the MMCR and L-band radiosonde
During 1–18 January 2016 and 1 April to 31 June 2016, a total of 109 days, there were 248 sounding observations from the L-band radiosonde. However, during the observation periods we selected, only 42 observations matched the MMCR observations. Similar as above, comparison of the performances of the MMCR and the L-band radiosonde on the detection of the cloud base indicated that they have consistent cloud base height observations, which is shown in Fig. 6. An example case analysis is presented below.Figure6. Cloud base height consistency of the MMCR and the radiosonde.
Figure 7a shows the reflectivity factor profile of the MMCR observed at 0701 LST 3 June 2016. Figure 7b shows vertical profiles of temperature and relative humidity observed by the corresponding radiosonde. As shown in Fig. 7a, the MMCR detected a cloud base at 6150 m, while the radiosonde detected a cloud base at 6517 m. They agree with each other reasonably well.
Figure7. (a) Reflectivity factor profile measured by cloud radar at 0701 LST 3 June 2016. (b) Temperature and relative humidity curves with altitude obtained from sounding at 0701 LST 3 June 2016.
2
3.7. Benchmark for clear-air echoes
According to section 3.5 and the linear correlation shown in Fig. 5, the threshold of difference between the heights of the laser ceilometer and the MMCR is selected within 1000 m, which is also consistent with the results of Oh et al. (2016) and Zhao et al. (2017). With reference to the above consistent analysis among the instruments, as well as the study of Clothiaux et al. (2000), the benchmark samples for clear-air echoes can be derived: For a given MMCR observation, the cloud base height measured by the laser ceilometer at the same time is marked on the MMCR echoes. If the difference between the cloud base heights measured by the two instruments is within 1000 m, the cloud base height obtained by the MMCR is used as an actual cloud base height. Then, by analyzing the cloud information from the radiosonde data and examining the pictures from the all-sky camera, the reliability of the previously determined cloud base is further verified. If the result is reliable, the obtained cloud base height is used as the benchmark height. Echoes that appear above the benchmark height are considered as cloud echoes, and those appearing below the benchmark height are considered as clear-air echoes.-->
4.1. Procedures of the algorithm
The flowchart in Fig. 8 shows the procedures of the clear-air-echo recognition algorithm. The procedures include feature extraction, feature selection, and clear-air-echo recognition.Figure8. General framework of the algorithm for clear-air echo recognition.
Spectral features, including the power spectral kurtosis and spectral width, are effective in distinguishing cloud and clear-air echoes (Luke et al., 2008). The extraction of the spectral features is accomplished in three steps: (1) collection of clear-air and cloud echo samples; (2) extraction of spectral features of the samples; and (3) establishment of a set of features.
As introduced in section 3, the data from September 2015 to August 2016 are used in clear-air-echo recognition. After initial data screening, 54 834 radar echo samples can be obtained to train the neural network. Among them, there are 11 706 clear-air echoes and 43 128 cloud-echo echoes.
From the analysis of the above established dataset, it is found that most of the clear-air echoes display a concentrated area of high-power spectral density along the velocity spectrum. They are multi-peaked with sharp fluctuations and narrow spectral width, compared to that of cloud echoes. Due to the existence of noises, the main spectrum, which might contain meteorological signals, must be extracted first, and then the number of extreme points contained in this main spectrum is calculated. Assuming the radar noise is Gaussian white noise, the piecewise averaging method is used to remove the unwanted noises (Fabry et al., 1997). The specific steps of feature extraction are as follows:
(1) The entire spectrum is divided into 16 segments and the average value of each segment is calculated;
(2) The minimum mean value is used as the noise level;
(3) The signal-to-noise ratio of each spectral point is calculated;
(4) The threshold value of the signal-to-noise ratio is set, the points below this threshold are removed, and the main spectrum is retained.
Figures 9 and 10 show an example of the extracted main spectrum from clear-air echoes and cloud echoes. In order to better distinguish the clear-air echoes and cloud echoes, as many features as possible are extracted for the main spectrum. Presently, 28 features are extracted and are listed in Table 4 in section 4.2.
Feature | Weight | |
1 | Reflectivity factor Z | 0.213 |
2 | Doppler velocity V | 0.028 |
3 | Spectral width W | 0.026 |
4 | Spectrum width (SW) | 0.049 |
5 | Kurtosis (K) | 0.048 |
6 | Skewness (S) | 0.035 |
7 | Number of spectral peaks (NSP) | 0.007 |
8 | Location of spectral peak (LSP) | 0.009 |
9 | Mean of main spectrum (MMS) | 0.123 |
10 | Horizontal difference of reflectivity factor (Zx) | 0.018 |
11 | Horizontal difference of doppler velocity (Vx) | 0.004 |
12 | Horizontal difference of spectral width (Wx) | 0.003 |
13 | Horizontal difference of spectrum width (SWx) | 0.001 |
14 | Horizontal difference of kurtosis (Kx) | 0.007 |
15 | Horizontal difference of skewness (Sx) | 0.023 |
16 | Horizontal difference of number of spectral peaks (NSPx) | 0.026 |
17 | Horizontal difference of location of spectral peak (LSPx) | 0.011 |
18 | Horizontal difference of mean of main spectrum (MMSx) | 0.007 |
19 | Vertical difference of reflectivity factor (Zy) | 0.010 |
20 | Vertical difference of doppler velocity (Vy) | 0.015 |
21 | Vertical difference of spectral width (Wy) | 0.001 |
22 | Vertical difference of spectrum width (SWy) | 0.001 |
23 | Vertical difference of kurtosis (Ky) | 0.004 |
24 | Vertical difference of skewness (Sy) | 0.013 |
25 | Vertical difference of number of spectral peaks (NSPy) | 0.019 |
26 | Vertical difference of location of spectral peak (LSPy) | 0.007 |
27 | Vertical difference of mean of main spectrum (MMSy) | 0.005 |
28 | Height (H) | 0.296 |
Table4. The weight of each feature.
Figure9. Extracted main spectrum (red) from clear-air echo at 450 m height at 1414 LST 19 April 2016.
Figure10. Extracted main spectrum (red) from cloud echo at 4.8 km height at 1030 LST 19 April 2016.
For all the extracted features, histograms are made and analyzed. For example, the reflectivity factor histogram is shown in Fig. 11, which shows that separating the cloud echoes from the clear-air echoes by the histogram of the reflectivity factor is possible. In most cases the reflectivity factor values of the two objects are obviously different, while in fewer cases the reflectivity factor values of the two objects are mixed, which makes it difficult to separate them with a high degree of accuracy. Therefore, by analyzing the degree of separation along the histograms of the features, the capability of each feature for distinguishing the two types of echoes can be evaluated.
Figure11. Reflectivity factor including 11 706 clear-air echo sample points and 43 128 cloud echo sample points.
2
4.2. Feature selection
In this study, there are 28 features (Table 4) extracted from clear-air echoes and cloud echoes. The purpose of constructing the sample feature set is to send the features of the clear-air echoes and cloud echoes to the neural network model for training, and let the algorithm model automatically learn the corresponding characteristics of the clear-air echoes and cloud echoes so that more accurate classification of clear-air echoes and cloud echoes can be achieved. The number of features is large and there is often redundancy between them, whose mutual information value is large. At the same time, the correlation between some features of the sample is not high. Therefore, if all the features are used for classification, the effect is not good and it also increases the complexity of subsequent learning algorithms. Therefore, it is necessary to optimize the feature set and to form a feature subset. In this study, the Relief algorithm (Amjady et al., 2010) is used to achieve the feature selection.The Relief algorithm is a nonlinear instance-based feature selection technique. In this study, the problem is to distinguish which class each sample belongs to—clear-air echoes or non-clear-air (or cloud in this case) echoes. Based on the correlation between the features and the classes, weights are calculated and assigned to the features of each sample. The weight reflects the ability of each feature to distinguish between the two classes.
Let
After a given number of iterations, the weight of all candidate features will be updated based on the selected samples. The weight of a feature represents its capability in classifying the samples based on that feature.
Using the above-mentioned Relief algorithm to perform a validity analysis on all the features in the sample feature set, the weight value of each feature can be obtained, which is shown in Table 4. In order to see the difference between each feature weight more visually, the results are shown in a bar chart in Fig. 12. Obviously, there are three features with larger weights than others, and their corresponding features are height
Figure12. Weight of each feature (feature 1: height; feature 9: spectral mean; feature 28: reflectivity factor).
2
4.3. Identification algorithm
In this study, we use a feed-forward neural network architecture and the back propagation of error training algorithm (Bai et al., 2016). Figure 13 shows the framework of the neural network algorithm. The algorithm inputs the data from the input layer, processes it in the hidden layer, outputs the current result from the output layer, then enters the backward propagation, calculates the error between the predicted value and the true value, and sequentially derives the forward direction for each neuron. Then, the eigenvalues are updated to minimize the cost function by means of gradient descent, and the weights and thresholds of the connected neurons are updated according to the errors of the hidden layer neurons. This iterative process will be carried out in a cycle until the pre-set stopping conditions are reached. The details of the classification of cloud echoes and clear-air echoes using this algorithm are as follows.Figure13. Framework of the neural network algorithm.
The neural network includes one input layer, two hidden layers, and one output layer. The first hidden layer contains eight neuron units, the second hidden layer contains six neuron units, and the output layer has two neuron units. We take the three optimal features extracted earlier as inputs for the neural network, and the output is a vector of continuous values with a component for the possible classification. The output ranges from 0.0 to 1.0, representing in the meantime the confidence of a membership classification in each class, and will be treated as the input for the decision criterion. The decision criterion is a method of interpreting the neural network output vector and converting it into a discrete decision state. In this study, we use a “winner-takes-all” function to choose the output with the highest value as the classification result. The two neural network outputs are either clear-air echoes or cloud echoes.
Table 5 shows the statistics of the clear-air-echo recognition rate when the pixel points on the radar echoes are counted. For the simple radar-echo scenario, there are 513 409 points being actual clear-air echoes and 4 821 474 points actual cloud echoes. After applying the recognition algorithm, the number of clear-air echoes being recognized is 492 165 and the number of cloud echoes is 4 842 718 with 21 244 clear-air echo points being recognized as cloud. Therefore, for these simple weather conditions, the clear-air-echo recognition rate is 95.86%. Similarly, in the complex weather scenarios, there are 420 896 points being actual clear-air echoes and 3 321 780 points actual cloud echoes. From the algorithm recognition, 371 998 points are recognized as clear-air echoes and 3 370 678 points as cloud echoes. Thus, the clear-air-echo recognition rate is 88.38% in the complex weather conditions.
Number of actual points before recognition | Number of points after recognition | |||||
Radar echo scenarios | Clear-air echo | Cloud echo | Clear-air echo | Cloud echo | Clear-air echo recognition rate | |
Simple | 513 409 | 4 821 474 | 492 165 | 4 842 718 | 95.86% | |
Complex | 420 896 | 3 321 780 | 371 998 | 3 370 678 | 88.38% |
Table5. The clear-air-echo recognition rate.
For further evaluation of the new recognition algorithm, we used three cases that include simple and complex weather conditions. The data of 3 April 2016 were representative of a simple weather condition, while the data of 19 April and 27 April 2016 were representative of complex weather conditions. The three cases were excluded from the algorithm training, and are investigated in detail to test the validity and efficiency of recognition. The clear-air echoes and cloud echoes of the two days have already been verified with the data from the MMCR, the laser ceilometer, and the all-sky camera.
2
5.1. Simple weather condition: 3 April 2016
Figure 14a shows the observed reflectivity factor on 3 April 2016. The day was a cloudy day, and the regions of cloud and clear-air echoes were separate from each other. The echoes at heights above 3 km are cloud echoes while the echoes at altitudes below 3 km are clear-air echoes.Figure14. (a) Reflectivity factor of 3 April 2016. (b) Result of reflectivity factor after filtering clear-air echoes.
Because of the radar self-interference, the echo image displays some noises, such as the thin strips of echoes shown in Fig. 14a. Thus, a 3 × 3 convolution kernel of mean is used to smooth the radar echo images after filtering the clear-air echoes, and the outcome of the final filtered clear-air echoes is shown in Fig. 14b.
Overall, Fig. 14b illustrates a very satisfactory identification result, whereas the clear-air echoes are filtered out completely. By statistical analysis, there are 98 162 echoes in the original radar returns, and the number of clear-air echoes is 8931. The number of clear-air echoes in the original image that was correctly identified as clear-air echoes by the recognition algorithm is 8845, indicating that about 99.14% of clear-air echoes are successfully identified and eliminated.
2
5.2. Complex weather condition: 27 April 2016
Figure 15a shows the reflectivity factor of stratiform clouds and clear-air echoes on 27 April 2016. We saved the reflectivity factor data samples of Fig. 15a and sent them to the trained BP neural network system. The BP neural network system recognizes the samples and divides all the samples into clear-air echoes and cloud echoes. Finally, we filtered out the clear-air echoes and showed only the samples of cloud echoes as shown in Fig. 15b. In this experiment, the new recognition method is very effective in terms of the overall recognition effect. The total number of samples of the echo points is 110 348 and the number of clear-air echo points is 22 376. Through the new algorithm in this paper, 91.25% of clear-air echoes are successfully identified. Comparing Figs. 14b and 15b, the size and thickness of clouds have no effect on the accuracy of the algorithm of identifying clear-air echoes.Figure15. (a) Reflectivity factor of 27 April 2016. (b) Result of reflectivity factor after filtering clear-air echoes.
2
5.3. Complex weather condition: 19 April 2016
Figure 16a shows a complicated distribution of reflectivity factor on 19 April 2016. In Fig. 16a, echoes of heights above 3 km are cloud echoes while echoes of heights below 3 km are mixed with precipitation echoes, low cloud echoes, and clear-air echoes. It is known that the echoes from the elliptical circle in the area below the figure are cloud echoes or precipitation echoes by combining the laser ceilometer, the L-band radiosonde, and the all-sky camera. From the statistics, there are 118 635 echoes in the original radar images, and the number of clear-air echoes is 20 219. Like the processes of the filtering and recognition algorithm of clear echoes used for the previous case, the result after filtering out the clear-air echoes is shown in Fig. 16b. After recognition, there are 1887 clear-air echoes left, indicating a 90.67% recognition accuracy. Comparing Figs. 16a and b, we can see that some of the three echo regions of the marker are filtered out while the cloud echoes and precipitation echoes are retained. The part enclosed by the ellipse in Fig. 16b is a clear-air echo that is mistaken for cloud echo. The reason is that the reflectivity factor of the echo is too close to the reflectivity factor of the low cloud echo. Therefore, it is necessary to extract more features that can effectively distinguish cloud echoes from clear-air echoes.Figure16. (a) Reflectivity factor of 19 April 2016, in which the circled echoes are confirmed clear-air echoes. (b) Result of reflectivity factor after filtering clear-air echoes.
Through the above case studies, our automatic recognition algorithm can effectively identify the clear-air echoes whether in simple cases or in the complicated situation of cloud-water mixing.