基于MAEU-CNN的高分辨率遥感影像建筑物提取

doi:10.12082/dqxxkx.2022.210727

地球信息科学学报 ›› 2022, Vol. 24 ›› Issue (6): 1189-1203.doi: 10.12082/dqxxkx.2022.210727

基于MAEU-CNN的高分辨率遥感影像建筑物提取

张华¹(), 郑祥成¹, 郑南山¹^,^*(), 史文中²

1.中国矿业大学环境与测绘学院,徐州 22116
2.香港理工大学土地测量及地理资讯学系,香港 999077

收稿日期:2021-11-15 修回日期:2021-12-01 出版日期:2022-06-25 发布日期:2022-08-25
通讯作者: *郑南山（1974— ）,男,安徽安庆人,博士,教授,主要从事遥感数据处理与应用。E-mail: znshcumt@163.com
作者简介:张华（1979— ）,男,安徽合肥人,博士,副教授,主要从事遥感数据智能解译及GIS理论与应用研究。E-mail: zhhua_79@163.com
基金资助:
国家自然科学基金项目(41971400);国家自然科学基金项目(41974039)

Building Extraction from High Spatial Resolution Imagery based on MAEU-CNN

ZHANG Hua¹(), ZHENG Xiangcheng¹, ZHENG Nanshan¹^,^*(), SHI Wenzhong²

1. School of Environment and Spatial Informatics, China University of Mining and Technology, Xuzhou 22116, China
2. Department of Land Surveying and Geo-informatics, The Hong Kong Polytechnic University, Hongkong 999077, China

Received:2021-11-15 Revised:2021-12-01 Online:2022-06-25 Published:2022-08-25
Supported by:
National Natural Science Foundation of China(41971400);National Natural Science Foundation of China(41974039)

摘要/Abstract

摘要：

从高空间分辨率图像（HSRI）中提取建筑物信息在遥感应用领域具有重要意义。然而,由于遥感影像中的建筑物尺度变化大、背景复杂和外观变化大等因素,从HSRI中自动提取建筑物仍然是一项具有挑战性的任务。特别是从影像中同时提取小型建筑物群和具有精确边界的大型建筑物时,难度更大。为解决这些问题,本文提出了一种端到端的编码器-解码器神经网络模型,用于从HSRI中自动提取建筑物。所设计的网络称为MAEU-CNN（Multiscale Feature Enhanced U-shaped CNN with Attention Block and Edge Constraint）。首先,在设计的网络编码部分加入多尺度特征融合（MFF）模块,使网络能够更好地聚集多个尺度特征。然后,在编码器和解码器部分之间添加了多尺度特征增强模块（MFEF）,以获得不同尺寸的感受野,用于获取更多的多尺度上下文信息。在跳跃连接部分引入双重注意机制,自适应地选择具有代表性的特征图用于提取建筑物。最后,为了进一步解决MAEU-CNN中由于池化及卷积操作导致的分割结果边界模糊的问题,引入多任务学习机制,将建筑物的边界几何信息融入网络中以优化提取的建筑物边界,最终获得精确边界的建筑物信息。MAEU-CNN在ISPRS Vaihingen语义标记数据集和WHU航空影像数据集2种不同尺度建筑物数据集上进行了试验分析,在ISPRS Vaihingen语义标记数据集上,MAEU-CNN在精度、F1分数和IoU指标中获得了最高精度,分别达到了93.4%、93.62%和88.01%;在WHU航空影像数据集上,召回率、F1分数和IoU指标中也获得了最高精度,分别达到了95.45%、95.58%和91.54%。结果表明,本文所提出的MAEU-CNN从遥感图像中提取建筑物信息精度较高,并且对于不同尺度具有较强的鲁棒性。

关键词: 建筑物提取, 深度学习, 注意力机制, 多任务学习, 多尺度特征增强, 感受野, 边界约束, 高分辨率遥感影像

Abstract:

Extraction of buildings from High Spatial Resolution Imagery (HSRI) plays an important role in remotely sensed imagery application. However, automatically extracting buildings from HSRI is still a challenging task due to factors such as large-scale variation of buildings, background complexity, and variation in appearance, etc. Especially, it is difficult in extracting both crowded small buildings and large buildings with accurate boundaries. To address these challenges, this paper presents an end-to-end encoder-decoder model to automatically extract buildings from HSRI. The designed network is called multiscale feature enhanced U-Shaped CNN with attention block and edge constraint (MAEU-CNN). Firstly, a Multiscale Feature Fusion (MFF) module is adopted in the encoder part of the network, which enables the network to aggregate features from multiple scales. Then, a Multi-scale Feature Enhancement module (MFEF) is added between the encoder and decoder parts to obtain multiscale receptive fields for obtaining multiscale context information. Thirdly, a dual attention mechanism is introduced to adaptively select representative feature maps for extraction of buildings instead of direct skipping connections. Lastly, in order to further solve the problem of segmentation result with poor boundaries aroused by the pooling operations in the MAEU-CNN, the geometric information of building boundary is introduced into the proposed MAEU-CNN by multi-task learning using the distance class map to produce fine-grained segmentations with precise boundaries. The performance of MAEU-CNN is examined through two different data sets at different building scales. The results show that MAEU-CNN obtains the greatest accuracy in each data set. The Precision, F1, and IoU is 93.4%, 93.62%, and 88.01%, respectively using the ISPRS Vaihingen semantic labeling contest data set. The Recall, F1, and IoU reach 95.45%, 95.58%, and 91.54%, respectively, using the WHU aerial image data set. Experimental results demonstrate that our proposed MAEU-CNN can achieve high accuracy for the extraction of building from remotely sensed imagery and show great robustness at different scales.

Key words: building extraction, deep learning, attention mechanism, multi-task learning, multiscale feature enhancement, receptive field, edge constraint, high spatial resolution imagery

张华, 郑祥成, 郑南山, 史文中. 基于MAEU-CNN的高分辨率遥感影像建筑物提取[J]. 地球信息科学学报, 2022, 24(6): 1189-1203.DOI:10.12082/dqxxkx.2022.210727

ZHANG Hua, ZHENG Xiangcheng, ZHENG Nanshan, SHI Wenzhong. Building Extraction from High Spatial Resolution Imagery based on MAEU-CNN[J]. Journal of Geo-information Science, 2022, 24(6): 1189-1203.DOI:10.12082/dqxxkx.2022.210727

图/表 15

图1

图2

图3

图4

图5

图6

图7

图8

表1

图9

表2

图10

表3

表4

表5

参考文献 28

[1]	Wang J, Yang X C, Qin X B, et al. An efficient approach for automatic rectangular building extraction from very high-resolution optical satellite imagery[J]. IEEE Geoscience and Remote Sensing Letters, 2015, 12(3):487-491. DOI: 10.1109/LGRS.2014.2347332 doi: 10.1109/LGRS.2014.2347332
[2]	Sirmacek B, Unsalan C. Urban-area and building detection using SIFT key points and graph theory[J]. IEEE Transactions on Geoscience and Remote Sensing, 2009, 47(4):1156-1167. DOI: 10.1109/TGRS.2008.2008440 doi: 10.1109/TGRS.2008.2008440
[3]	Huang X, Zhang L P. Morphological building/shadow index for building extraction from high-resolution imagery over urban areas[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2012, 5(1):161-172. DOI: 10.1109/JSTARS.2011.2168195 doi: 10.1109/JSTARS.2011.2168195
[4]	Li E, Xu S B, Meng W L, et al. Building extraction from remotely sensed images by integrating saliency cue[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2017, 10(3):906-919. DOI: 10.1109/JSTARS.2016.2603184 doi: 10.1109/JSTARS.2016.2603184
[5]	Du S H, Zhang F L, Zhang X Y. Semantic classification of urban buildings combining VHR image and GIS data: An improved random forest approach[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2015, 105:107-119. DOI: 10.1016/j.isprsjprs.2015.03.011 doi: 10.1016/j.isprsjprs.2015.03.011
[6]	Shi Y L, Li Q Y, Zhu X X. Building segmentation through a gated graph convolutional neural network with deep structured feature embedding[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 159:184-197. DOI: 10.1016/j.isprsjprs.2019.11.004 doi: 10.1016/j.isprsjprs.2019.11.004
[7]	Zhu X X, Tuia D, Mou L C, et al. Deep learning in remote sensing: A comprehensive review and list of resources[J]. IEEE Geoscience and Remote Sensing Magazine, 2017, 5(4):8-36. DOI: 10.1109/MGRS.2017.2762307 doi: 10.1109/MGRS.2017.2762307
[8]	Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, 1:3431-3440. DOI: 10.1109/CVPR.2015.7298965 doi: 10.1109/CVPR.2015.7298965
[9]	Ji S P, Wei S Q, Lu M. Fully convolutional networks for multisource building extraction from an open aerial and satellite imagery data set[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(1):574-596. DOI: 10.1109/TGRS.2018.2858817 doi: 10.1109/TGRS.2018.2858817
[10]	Yi Y N, Zhang Z J, Zhang W C, et al. Semantic segmentation of urban buildings from VHR remote sensing imagery using a deep convolutional neural network[J]. Remote Sensing, 2019, 10(15):1774-1792. DOI: 10.3390/rs11151774 doi: 10.3390/rs11151774
[11]	Xia L G, Zhang X B, Zhang J X, et al. Building extraction from very-high-resolution remote sensing images using semi-supervised semantic edge detection[J]. Remote Sensing, 2021, 13(11):2187-2206. DOI: 10.3390/rs13112187 doi: 10.3390/rs13112187
[12]	Liu Y Y, Chen D Y, Ma A L, et al. Multiscale U-shaped CNN building instance extraction framework with edge constraint for high-spatial-resolution remote sensing imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 59(7):6106-6120. DOI: 10.1109/TGRS.2020.3022410 doi: 10.1109/TGRS.2020.3022410
[13]	Ronneberger O, Fischer P, Brox T. U-Net: Convolutional networks for biomedical image segmentation[C]// International Conference on Medical image computing and computer assisted intervention. Springer, Cham, 2015:234-241. DOI: 10.1007/978-3-319-24574-4_28 doi: 10.1007/978-3-319-24574-4_28
[14]	Liu J Y, Wang S S, Hou X W, et al. A deep residual learning serial segmentation network for extracting buildings from remote sensing imagery[J]. International Journal of Remote Sensing, 2020, 41(14):5573-5587. DOI: 10.1080/01431161.2020.1734251 doi: 10.1080/01431161.2020.1734251
[15]	Diakogiannis F I, Waldner F, Caccetta P, et al. ResUNet-a: A deep learning framework for semantic segmentation of remotely sensed data[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 162:94-114. DOI: 10.1016/j.isprsjprs.2020.01.013 doi: 10.1016/j.isprsjprs.2020.01.013
[16]	Zhang X Q, Xiao Z H, Li D Y, et al. Semantic segmentation of remote sensing images using multiscale decoding network[J]. IEEE Geoscience and Remote Sensing Letters, 2019, 16(9):1492-1496. DOI: 10.1109/LGRS.2019.2901592 doi: 10.1109/LGRS.2019.2901592
[17]	徐佳伟, 刘伟, 单浩宇,等. 基于PRCUnet的高分遥感影像建筑物提取[J]. 地球信息科学学报, 2021, 23(10):1838-1849. doi: 10.12082/dqxxkx.2021.210283
	[ Xu J W, Liu W, Shan H Y, et al. High-resolution remote sensing image building extraction based on PRCUnet[J]. Journal of Geo-information Science, 2021, 23(10):1838-1849. ] DOI: 10.12082/dqxxkx.2021.210283 doi: 10.12082/dqxxkx.2021.210283
[18]	张玉鑫, 颜青松, 邓非. 高分辨率遥感影像建筑物提取多路径RSU网络法[J]. 测绘学报, 2021, 50(10):1-10.
	[ Zhang Y X, Yan Q S, Deng F. Multi-path RSUnetwork method for high-resolution remote sensing image building extraction[J]. Acta Geodaetica Sinica, 2021, 50(10):1-10. ] DOI: 10.11947/j.AGCS.2021.20200508 doi: 10.11947/j.AGCS.2021.20200508
[19]	Rastogi K, Bodani B, Sharma S A. Automatic building footprint extraction from very high-resolution imagery using deep learning techniques[J]. Geocarto International [online]. DOI: 10.1080/10106049.2020.1778100 doi: 10.1080/10106049.2020.1778100
[20]	唐璎, 刘正军, 杨懿,等. 基于特征增强和ELU的神经网络建筑物提取研究[J]. 地球信息科学学报, 2021, 23(4):692-709. doi: 10.12082/dqxxkx.2021.200130
	[ Tang Y, Liu Z J, Yang Y, et al. Research on building extraction based on neural network with feature enhancement and ELU activation function[J]. Journal of Geo-information Science, 2021, 23(4):692-709. ] DOI: 1 0.12082/dqxxkx.2021.200130 doi: 1 0.12082/dqxxkx.2021.200130
[21]	Liu Y H, Gross L, Li Z Q, et al. Automatic building extraction on high-resolution remote sensing imagery using deep convolutional encoder-decoder with spatial pyramid pooling[J]. IEEE Access, 2019, 7:128774-128786. DOI: 10.1109/ACCESS.2019.2940527 doi: 10.1109/ACCESS.2019.2940527
[22]	Wei S Q, Ji S P, Lu M. Toward automatic building footprint delineation from aerial images using CNN and regularization[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 58(3):2178-2189. DOI: 10.1109/TG RS.2019.2954461 doi: 10.1109/TG RS.2019.2954461
[23]	Shrestha S, Vanneschi L. Improved fully convolutional network with conditional random fields for building extraction[J]. Remote Sensing, 2018, 10(7):1135-1155. DOI: 10.3390/rs10071135 doi: 10.3390/rs10071135
[24]	Yuan J Y. Learning building extraction in aerial scenes with convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(11):2793-2798. DOI: 10.1109/TPAMI.2017.2750680 doi: 10.1109/TPAMI.2017.2750680
[25]	Pan S M, Tao Y L, Nie C C, et al. PEGNet: Progressive edge guidance network for semantic segmentation of remote sensing images[J]. IEEE Geoscience and Remote Sensing Letters, 2021, 18(4):637-641. DOI: 10.1109/LGRS.2020.2983464 doi: 10.1109/LGRS.2020.2983464
[26]	Xia L G, Zhang J X, Zhang X B, et al. Precise extraction of buildings from high-resolution remote sensing images based on semantic edges and segmentation[J]. Remote Sensing, 2021, 13(16):3083-3104. DOI: 10.3390/rs13163083 doi: 10.3390/rs13163083
[27]	Sun Y, Zhang X C, Zhao X Y, et al. Extracting building boundaries from high resolution optical images and LiDAR data by integrating the convolutional neural network and the active contour model[J]. Remote Sensing, 2018, 10(9):1459. DOI: 10.3390/rs10091459 doi: 10.3390/rs10091459
[28]	Vaihingen 2D Semantic Labeling-ISPRS. [Online]. Available:http://www2.isprs.org/commissions/comm3/wg4/semantic-labeling.html .

参数	WHU数据集	Vaihingen数据集
优化器	Adam	Adam
损失函数	标准交叉熵损失函数	标准交叉熵损失函数
训练次数	100	300
批次大小	5	5
初始学习率	0.0001	0.0001
学习率衰减	0.5倍/50次	0.5倍/50次

方法	精度/%	召回率/%	F1分数/%	IoU/%
UNet	93.12	91.31	92.21	85.54
UNet++	92.12	93.83	92.97	86.86
PSPNet	93.25	92.98	93.12	87.13
HRNetV2	93.07	94.07	93.56	87.91
MAEU-CNN	93.44	93.81	93.62	88.01

方法	精度	召回率	F1分数	IoU
UNet	92.72	94.71	93.70	88.15
UNet++	95.23	94.64	94.94	90.36
PSPNet	94.94	94.23	94.58	89.72
HRNetV2	96.53	94.01	95.26	90.94
MAEU-CNN	95.73	95.45	95.58	91.54

方法	精度	召回率	F1分数	IoU
UNet	93.12	91.31	92.21	85.54
UNet + Attention	92.42	93.76	93.08	87.06
UNet + Attention + MF	93.15	93.43	93.29	87.42
UNet + Attention + MF+ MFEM	92.80	94.29	93.53	87.85
UNet + Attention+ MF + MFEM + ECM（MAEU-CNN）	93.44	93.81	93.62	88.01

方法	精度	召回率	F1分数	IoU
UNet	92.71	94.71	93.70	88.15
UNet + Attention	94.30	94.92	94.61	89.77
UNet + Attention + MF	95.55	94.78	95.18	90.78
UNet + Attention + MF+ MFEM	95.80	95.25	95.52	91.42
UNet + Attention + MF + MFEM + ECM（MAEU-CNN）	95.73	95.45	95.58	91.54

基于MAEU-CNN的高分辨率遥感影像建筑物提取

Building Extraction from High Spatial Resolution Imagery based on MAEU-CNN

RichHTML

PDF (PC)

赞

可视化

摘要/Abstract

引用本文

使用本文

图/表 15

参考文献 28

相关文章 15

Metrics

本文评价

推荐阅读 0

[1]	陈振, 陈芸芝, 吴婷, 李佳优. 面向高分遥感影像道路提取的轻量级双注意力和特征补偿残差网络模型[J]. 地球信息科学学报, 2022, 24(5): 949-961.
[2]	蒯宇, 王彪, 吴艳兰, 陈搏涛, 陈兴迪, 薛维宝. 基于多尺度特征感知网络的城市植被无人机遥感分类[J]. 地球信息科学学报, 2022, 24(5): 962-980.
[3]	吴新辉, 毛政元, 翁谦, 施文灶. 利用基于残差多注意力和ACON激活函数的神经网络提取建筑物[J]. 地球信息科学学报, 2022, 24(4): 792-801.
[4]	杨先增, 周亚男, 张新, 李睿, 杨丹. 融合边缘特征与语义信息的人工坑塘精准提取方法[J]. 地球信息科学学报, 2022, 24(4): 766-779.
[5]	龙怡灿, 雷蓉, 董杨, 李东子, 赵琛琛. 基于YOLOv5算法的飞机类型光学遥感识别[J]. 地球信息科学学报, 2022, 24(3): 572-582.
[6]	周欣昕, 吴艳兰, 李梦雅, 郑智腾. 基于特征分离机制的深度学习植被自动提取方法[J]. 地球信息科学学报, 2021, 23(9): 1675-1689.
[7]	李国清, 柏永青, 杨轩, 陈正超, 余海坤. 基于深度学习的高分辨率遥感影像土地覆盖自动分类方法[J]. 地球信息科学学报, 2021, 23(9): 1690-1704.
[8]	叶凡, 孙玉, 陈崇成, 于大宇. 基于地理标记照片的个性化景点推荐方法[J]. 地球信息科学学报, 2021, 23(8): 1391-1400.
[9]	刘戈, 姜小光, 唐伯惠. 特征优选与卷积神经网络在农作物精细分类中的应用研究[J]. 地球信息科学学报, 2021, 23(6): 1071-1081.
[10]	许泽宇, 沈占锋, 李杨, 柯映明, 李硕, 王浩宇, 焦淑慧. 结合模糊度和形态学指数约束的深度学习建筑物提取[J]. 地球信息科学学报, 2021, 23(5): 918-927.
[11]	唐璎, 刘正军, 杨懿, 顾海燕, 杨树文. 基于特征增强和ELU的神经网络建筑物提取研究[J]. 地球信息科学学报, 2021, 23(4): 692-709.
[12]	朱盼盼, 李帅朋, 张立强, 李洋. 基于多任务学习的高分辨率遥感影像建筑提取[J]. 地球信息科学学报, 2021, 23(3): 514-523.
[13]	郭紫甜, 王春梅, 刘欣, 庞国伟, 朱梦阳, 王晋卿. 基于小流域抽样单元的中国FROM-GLC30数据精度评价[J]. 地球信息科学学报, 2021, 23(3): 524-535.
[14]	李传林, 黄风华, 胡威, 曾江超. 基于Res_AttentionUnet的高分辨率遥感影像建筑物提取方法[J]. 地球信息科学学报, 2021, 23(12): 2232-2243.
[15]	冯叶涵, 陈亮, 贺晓冬. 基于百度街景的SVF计算及其在城市热岛研究中的应用[J]. 地球信息科学学报, 2021, 23(11): 1998-2012.