关键词: 灾害调查, 山洪灾害, 数据质量, 异常检测, 空间聚类

Abstract:

"Natural disaster" is the phenomenon of the losses of life and property, which is caused by the interaction of human society and natural environment. It’s also the product of the disaster environment, disaster-causing factors and disaster-bearing body. In order to study the processes, mechanisms and impacts of natural disasters as well as the reduction of the losses caused by natural disasters, it is necessary to conduct surveys of basic data and natural disaster events on a large scale of which the authenticity and consistency are much significant for ensuring the reliability and validity of the research results. However, the large number of organizations and investigators participating in the survey and evaluation process, large regional differences and large spatial scale create challenges in data quality control and validating the consistency of data from various survey units. To ensure the correctness and consistency of the data, it is necessary to carry out manual inspection. However, for the massive survey data, it is unrealistic to totally rely on manual work to effectively identify the abnormities. As a result, we design a multi-scale anomaly detection method for natural disaster survey data by using the single-element detection method of outliers based on normal distribution and spatial clustering method of Anselin Local Moran's I to detect the abnormal values and abnormal spatial distribution patterns of the massive survey data. It can effectively extracts the abnormalities and abnormal investigation units at all levels of scale and gains the reasons for abnormal data. It provides the support for the manual checking of survey data. In this paper, taking the project of flash flood disaster investigation and evaluation in mainland of China as an example, this method is used to audit the events of historical flash flood disaster and the areas of the towns which are in the prevention zones. Also, it quickly extract the anomaly units of flash flood disaster point density and township units with exceptional area values. Further analysis found that the reasons for these abnormalities were due to the inconsistency of filling methods, unit errors, and repetition of records and so on. The method resolved the inconsistency in massive amounts of flash flood survey data. This method is an effective approach of checking the quality of various other large-scale disaster datasets. Although the data validation approach used in this study is very effective, there are still some problems, i.e. the outlier checking only considers the outliers between survey units based on the administrative divisions. Regions are not divided according to their economic development and natural conditions. Finally, we analyze the applicable conditions of this method in the large-scale natural disaster investigations.

Key words: disaster investigation, flash flood disasters, data quality, anomaly detection, spatial clustering