Automatic Building Damage Detection Method Using High-Resolution Remote Sensing Images and 3 D GIS Model

In this paper, a novel approach of building damaged detection is proposed using high resolution remote sensing images and 3D GIS-Model data. Traditional building damage detection method considers to detect damaged building due to earthquake, but little attention has been paid to analyze various building damaged types(e.g., trivial damaged, severely damaged and totally collapsed.) Therefore, we want to detect the different building damaged type using 2D and 3D feature of scenes because the real world we live in is a 3D space. The proposed method generalizes that the image geometric correction method firstly corrects the post-disasters remote sensing image using the 3D GIS model or RPC parameters, then detects the different building damaged types using the change of the height and area between the preand post-disasters and the texture feature of post-disasters. The results, evaluated on a selected study site of the Beichuan earthquake ruins, Sichuan, show that this method is feasible and effective in building damage detection. It has also shown that the proposed method is easily applicable and well suited for rapid damage assessment after natural disasters.


Introduction 
Earthquake-induced building damage or collapse is an extremely grave threat to human life and property, particularly in high-density urban area.After an earthquake, rapid damage assessment is vital for emergency response actions, rescue work and post-disaster reconstruction.In recent decades, various kinds of remote sensing data, such as aerial or satellite images, Lidar and SAR, have been widely used to identify, detect and assess for natural disasters [1] [2].
Therefore, the accuracy classification of individual building damaged types using the remote sensing images has become a critical issue.
Many approaches for detection and estimation earthquake-building damage have been proposed.These approaches can be categorized into four types: (1) The visual interpretation method [3][4] [5] is employed to detect and assess the building damage using various remote sensing images and GIS data, but  *Corresponding author depends on many auxiliary tools(e.g., ArcGIS).
Unfortunately, it is time-consuming and requires trained operators.(2)The single post-disaster data method is another method to detect building damage using only one set of remote sensing data collected after a seismic event [6][7] [8].The approach mainly uses feature extraction to detect the building damage supported by auxiliary pre-earthquake GIS vector data.
Pre-earthquake data may not always be available, so this method is more suitable for detecting the extent of building damage in real time.(3)The change detection method [9][10] [11], which compares various pre-and post-earthquake features, can lead to more accurate and reliable results, including the height feature change from Lidar and stereo images, the texture feature change from optical images and the backscattering, correlation coefficients and coherence from SAR images.Owing to its speed and automation, this approach is the most commonly employed technique for damaged building detection.However, little attention has been paid to analyze the different building damaged types, especially the trivial damaged building.3D GIS model data provides the accurate geographic coordinates and height information for the pre-disasters building.But 3D GIS model data do not provide good texture information and is not easy to be got in time after natural disasters, meanwhile post-disasters remote sensing images are easy to get and have rich texture information, which makes up the shortage of 3D GIS.Based on these, Hence, the purpose of this paper is to present an approach to detect the different building damaged types using pre-disaster 3D GIS model and post-disaster remote sensing images.The approach was implemented on a selected study site of the Beichuan earthquake ruins, one of these area most strongly hit by the 2008 Beichuan, Wenchuan earthquake, the experimental results are showed the superiority of the proposed approach to the other existing ones.
The rest of this paper is organized as follows.Section 2 describes the proposed method.Section 3 analyses and compares the experimental results, followed by the conclusion in Section 4.

2．Methodology
Fig. 1 shows the overall workflow of the study.In the figure, there are three main parts.The first part is the pre-processing, including the extraction of the building vector from the 3D GIS model, the geometric correction for high resolution remote sensing image.
The second part is the building-height estimation using the shadow detection.The third part is to detect the different building damaged types by utilizing the changes detection of the building area, the height between the pre-and post-disaster, and the building rooftops texture information.

High-Resolution Remote
Sensing

Data Preprocessing
This step obtains the building vector information and the georeferenced remote sensing images, then the building vector information is used to extract the building from the georeferenced remote sensing images.The required processing for the building vector information and the georeferenced remote sensing images is described below.
First, 3D GIS model provide us with comprehensive information, such as 3D model, 3D GIS, object vector information, thematic information, etc.Therefore, the building vector information is extracted from the 3D GIS model, which can be simply carried out by any GIS package such as ArcGIS.
Second, the georeferencing and geometric correction of the remote sensing image.The raw remote sensing image does correspond the real geometry of the area, so the aim of geometric correction make the raw image's pixels correspond to a given place on Earth, namely, the remote sensing image have the real world coordinate.In this paper, we use the method of polynomial correction to correct the remote sensing image with the PRC parameter.After the geometric correction, the vector-to-image registration is using the georeference information.

The building-height estimation
Height feature is the important feature of the building, but post-disaster remote sensing images have only rich texture rather than height feature.In order to obtain the post-disaster building-height, this paper use shadow to estimate the building-height for the following change detection.
3D GIS model in the pre-disaster include the building vector information and the building height information.
We use these information to number for the building object and construct a set of the building objects in the pre-disaster.Set the building object be , where V i is a building object in the set, i=1,2,3,… m, m is the total of the building object.
In the following, the building vectors are superimposed on the post-disaster remote sensing image, which builds the buffer of the building in the image.Within the buffer area, we further extract the building using level set segmentation algorithm.There are the detail descriptions in the following step.
1) After the building vectors are superimposed on the image, the building buffer area is created.And then it is applied to gray-scale transformation and binarization in the buffer area.
2) To segment the building using level set method [15].
3) In order to obtain the building height, the proposed method includes two steps: shadow segmentation and extraction, building-height estimation.With increased spatial resolution, shadows from buildings or trees become an apparent class that may not be ignored.It is closely related to building shape, sun solar, altitude angles, the sun azimuth angle, the luminous intensity, and so on.On flat area, when the building height is constant, the shadow length L is mainly related to solar altitude angles а and building height H , have the following relation: tan

The building rooftops damaged detection based on Visual BoW model
Building rooftop damaged detection is a key work for the whole building damaged detection.Gap is a marked characteristic of the building rooftops damage.
Only when we calculated the gap area size can it determine that whether the building rooftop is damaged.However, these gaps are uneven distribution as the form of a block of pixels on the rooftop, hence it (1) Image segmentation.
In the proposed method, rooftop is segmented by utilizing the simple linear iterative clustering(SLIC) algorithm [16].SLIC can clusters pixels in the combined five-dimensional color and image plane space to efficiently generate compact, nearly uniform (2) BOW representation of superpixel region features.
The first step is to extract the feature descriptors in each superpixel region.In the proposed approach, color and HOG is extracted for each superpixel as the feature descriptor.The damaged areas intensity of the building rooftops is darker than that of the undamaged areas, HSV color space provide better intuition for humans than RGB color space, and it is more sensitive to the brightness, which is more advantage to distinguish the color feature.Hence the HSV space is nonuniformly quantized by using H, S and V in the Due to the different effect on the image representation in the different visual words, it will cause bad effect if using the histogram vector directly for recognition.
Hence we employ the weight scheme of [17] and normalize with the L2 norm [18] to generate a different weight histogram vector.Support the weight vector is represented by a K-vector T = [t 1 ,t where n id is the number of occurrences of word i in the superpixel region, n d is the total number of words in the superpixel region, n i is the number of occurrences of term i in the whole database and N is the number of documents in the whole database.The result is a linear classification rule that can be used to classify new test examples.

Accessing the building damage type
The characteristics of a damaged building are strongly dependent on many factors, such height change, area change and texture feature.So only using a certain characteristics to determine the damage building is not accurate.This paper extracts the height, the area and texture characteristics of building to determine the building damage.The detail processing are as follows. ( . (2) Set building area of pre-disaster was S iBefore , building area of post-disaster was S iAfter , area difference was ∆S i = S iBefore -S iAfter .When the area ∆S i is less than the given minimum threshold Min∆S i , the membership degree R i∆S (Area) of building damage is 0;When the area ∆S i is greater than the given maximum threshold Min∆S i , the membership degree R i∆S (Area)of building damage is 1;When Min∆S i < ∆S i < Max∆S i , the membership degree of building damage According to the principles of evidence theory, this paper will give evidence to the various features of buildings damaged confidence in the probability distribution by the way of statistical experts, on the basis of evidence we build the theory model to assess whether buildings damaged or not through using various confidence coefficient.Building damaged confidence formula is as follows: Where R j is the membership degree of the evidence j, ρ j is the prediction setting confidence degree of the evidence j. n is the total number of evidence.Finally calculate the building damage of confidence P Judge .
3．Experiment and Result

Experiments overall description
In order to evaluate the effectiveness of the proposed method in this paper, author has investigated and measured on the Beichuan earthquake ruins in April, 2014.The data collection described as follows: the foundation geography information data of the building in the pre-disaster, the high resolution remote sensing image in the post-disaster, the ground image of the building and the building height in the post-disaster.
In this experimental part, the 3D GIS model of the Beichuan is simulated by using the GIS tool "ArcGIS" according to the foundation geography information data in the pre-disaster(shown in Fig. 3

The result of building-height estimate
Building vector is overlaid with the remote sensing image to obtain the building's buffer area, then we obtain the building boundaries by using the level set method of section 2.1(2) in the buffer area.In the building boundaries we estimate the building height using the shadow feature.To assess the accuracy of building-height estimation, actual building heights were obtained by the laser range finder.Building heights were obtained for a total of 30 randomly selected from the Beichuan earthquake ruins.
The actual heights for these 30 buildings were in the   For validation, the performance of the proposed approach is evaluated through comparison with the approach presented by reference [7].The approach presented by reference [7] detects building damage by calculating the area of overlap between the theoretical ground shadow area and the actual ground shadow area.
In this paper, we use the presented approach by Tong et al(2013) to detect the building damage.From the experimental result of this approach (shown Table 2), we found that the overall accuracy and overall kappa for reference [7] method are only 58.75% and 40.69%, respectively.The reason for this result is that this approach determines the building damage by using the building shadow, namely, the building height, but many building's rooftops or facade happened to broke or collapse, but reference [7] method think this damaged building as non-damaged building.Therefore, the overall accuracy and overall kappa of this method is much lower than the proposed method in this paper.

Conclusions
This paper has presented a novel building damage detection method from high resolution remote sensing

Fig. 1 .
Fig.1.Framework of the proposed method

Fig. 2
Fig.2 Flowchart of the proposed method for detecting rooftops damaged areas.
superpixel areas, this includes 7 levels in H, 3 levels in S, and 3 levels in V, then the three color components are combined into one one-dimensional vector: L = 9H +3S+V(L∈[0,71]); Because there are some different shape in the superpixel areas, a rectangular area is determined according to the length of the major axis and centroid of the super-pixel which computes HOG feature.Experimental statistics show that the major axis length is 30 pixels, so we choice 30×30 rectangular area to compute HOG feature, HOG's cell is 6×6,gradient direction is 9 orientation bins, which generate 225 HOG features.The second step is to generate a codebook for the BoW model.Let F=[f 1 , f 2 , f 3 ,…, f N ] be feature descriptors extracted in each superpixel region as calculated above, where each f i is M dimensional vectors and N is the number of feature points in each superpixel region.We cluster the feature descriptors F with k-means algorithm, the number of k-means clusters is the size of our codebook.For the convenience of the following development, we denote a generated codebook by W = [w 1 ,w 2 ,w 3 ,…,w k ]∈R D×K , where K is the size of the codebook.The third step is to represent the damage and non-damage region respectively using the codebook.In BOW model, image is represented by different frequencies codebooks.Therefore, the histogram of the damage non-damage region can be obtained.Let N = [n 1 , n 2, n 3, .. n i ,…, n K ] be a histogram statistical vector.

( 3 )
Classification by SVM.Based on the features, an SVM classifier is trained to process the samples that were manually collected as damage and non-damage region.Given a set of labeled training samples, SVMs can learn a linear decision boundary to discriminate between the two classes.For a test image, image segmentation by SLICO, extraction of the color and HOG descriptors for each object, and classification by SVMs are consequently performed.

)
The texture feature is to detect the damaged area size of rooftop using Visual BoW model.When the size of the rooftop damaged area is greater than or equal to the given maximum threshold MaxFeatureArea i , the membership degree R (Area) of the building damage is 1; when the damaged area size is less than or equal to the minimum threshold i MinFeatureArea , the membership degree R ifeature (Area) of the building damage is 1; when MinFeatureArea i < FeatureArea i < MaxFeatureArea i ,

Fig. 3
Fig.3 Beichuan data.(a)3DGIS model (b)Remote Sensing Image; Fig.4 (a) Building vector (b)The overlaying result of vector overlapping with the image

range of 15
to 40 m, which is considered to be representative in the Beichuan earthquake ruins.The actual building heights and the automatic detected values are plotted in Fig.5.The correlation coefficient was 0.98.The root-mean-square error (RMSE) was 0.51 m.

Fig. 5 . 3 . 3 Fig. 6
Fig.5.Accuracy evaluation of the building-height.3.3 The result of rooftop damaged area detection In this experiment, 50 building rooftops are segmented by utilizing the SLIC algorithm, then we manually classify the damaged/non-damaged areas as training samples, there are 8921 damaged areas and 4367 non-damaged areas, respectively.Because the key step for the Visual BoW model is feature descriptors, which determined the quality of the detection result, different local feature descriptors (Surf and Gabor) will be compared with the proposed method in the Visual BoW model.Fig.6 shows some example of the testing rooftops damaged areas detection from datum.The results show that the proposed method is better than the other two methods from the visual effect, which obtains the satisfactory results of the detection.
imagery and 3D GIS model.Towards that end, we have proposed and tested techniques for image rectification, building vector and feature extraction, change detection and damage detection.This new one fully combines the 2D and 3D information of the real world to detect the building damage.The results, evaluated on real data, demonstrate that this method is feasible and effective in building damage detection.In the future we intend to use multiply view oblique aerial imagery and public-source imagery on the ground to detection the building damage, which is better to assess the level of the building damage.03-Y20A10-9001-15/16), National High Technology R esearch and Development Programme of China (863 Pr ogramme) [Project NO. 2013AA122301] 2, t 3, …, t i,…, t K ].The Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume III-8, 2016 XXIII ISPRS Congress, 12-19 July 2016, Prague, Czech Republic i =H iBefore -H iAfter .When the height ∆H i is less than the given minimum threshold Min∆H i , the membership degree R i∆H (Height) of building damage is 0;When the height ∆H i is greater than the given maximum ISPRS This contribution has been peer-reviewed.The double-blind peer-review was conducted on the basis of the full paper.doi:10.5194/isprsannals-III-8-43-2016thresholdMin∆H i , the membership degree

Table 1
, there are 50 testing rooftops average values of precision and recall in the variant combination of modules, the results show that the method produces better precision and recall.

Table 2
[7]ult of the confusion matrix for the detection of building damage using the proposed approach and reference[7]