结合模糊度和形态学指数约束的深度学习建筑物提取

doi:10.12082/dqxxkx.2021.200397

地球信息科学学报 ›› 2021, Vol. 23 ›› Issue (5): 918-927.doi: 10.12082/dqxxkx.2021.200397

结合模糊度和形态学指数约束的深度学习建筑物提取

许泽宇¹^,²(), 沈占锋¹^,⁴^,^*(), 李杨¹^,³, 柯映明¹^,⁴, 李硕¹^,⁴, 王浩宇¹^,², 焦淑慧¹^,²

1.中国科学院空天信息创新研究院国家遥感应用工程技术研究中心,北京 100101
2.中国科学院大学电子电气与通信工程学院,北京 100049
3.中国科学院空天信息创新研究院遥感科学国家重点实验室,北京 100101
4.中国科学院大学资源与环境学院,北京 100049

收稿日期:2020-07-26 修回日期:2020-11-23 出版日期:2021-05-25 发布日期:2021-07-25
通讯作者: 沈占锋
作者简介:许泽宇（1995— ）,男,河北保定人,硕士生,研究方向为遥感信息提取。E-mail:xuzeyu@aircas.ac.cn
基金资助:
国家重点研发计划项目(2018YFB0505000);国家重点研发计划项目(2017YFB0504204);国家重点研发计划项目(2016YFC0803109);国家自然科学基金项目(41971375);国家自然科学基金项目(41871283)

Building Extraction by Deep Learning Method Combined with Ambiguity and Morphological Index Constraints

XU Zeyu¹^,²(), SHEN Zhanfeng¹^,⁴^,^*(), LI Yang¹^,³, KE Yingming¹^,⁴, LI Shuo¹^,⁴, WANG Haoyu¹^,², JIAO Shuhui¹^,²

1. National Engineering Research Center for Geomatics, Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100101, China
2. School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences, Beijing 100049, China
3. State Key Laboratory of Remote Sensing Science, Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100101, China
4. College of Resources and Environment, University of Chinese Academy of Sciences, Beijing 100049, China

Received:2020-07-26 Revised:2020-11-23 Online:2021-05-25 Published:2021-07-25
Contact: SHEN Zhanfeng
Supported by:
National Key Research and Development Project(2018YFB0505000);National Key Research and Development Project(2017YFB0504204);National Key Research and Development Project(2016YFC0803109);National Natural Science Foundation of China(41971375);National Natural Science Foundation of China(41871283)

摘要/Abstract

摘要：

基于高分辨率遥感影像的建筑物提取一直是研究的热点问题,深度学习的深层次特征提取方法,非常适合高分辨率影像中建筑物的提取,但使用深度学习提取建筑物时,大多以改变网络结构为主进行算法优化,很少与其他方法结合。本文研究在改进深度学习网络结构的基础上,结合影像模糊度约束增强、形态学建筑指数约束增强等方法,对建筑物提取方法进行更全面更有针对性的改进。本文主要改进内容为：① 提出PwDeepLab网络,该网络基于DeepLab v3+网络结构,在特征融合方式和损失函数等方面进行了改进。② 提出模糊度约束方法,在固定影像块大小的情况下,通过影像模糊度约束对影像进行上采样增强。③ 提出形态学指数约束方法,通过形态学建筑物指数（MBI）约束范围拉伸增强的方法,在较少改变原始影像特征的情况下,突出建筑信息。本文在Massachusetts数据集和武汉大学的Satellite Dataset Ⅱ(East Asia) 数据集上进行验证, 2个数据集的主要建筑类型存在较大区别。本文提出的方法在2个数据集上的精度相对于DeepLab v3+分别提高了10.9%和3.8%,相对于U-Net分别提高了10.0%和9.6%。实验结果表明本文提出的方法对建筑物提取效果有明显提升,且具有很好的鲁棒性和通用性。

关键词: 深度学习, 建筑提取, 高分辨率影像, PwDeepLab模型, 特征融合, 模糊度, 形态学指数, 窗口增强

Abstract:

Extraction of buildings from high-resolution remote sensing images has been a hot topic. It is important to quickly and accurately extract the location and outline of buildings from high-resolution remote sensing images for earthquake disaster assessment, urban and rural planning management, smart city construction, and other fields. However, it is challenging to extract buildings accurately from high-resolution remote sensing images due to the complexity of ground features and the occlusion phenomenon. The classic building extraction algorithms usually have incomplete results with many wrong detections and missed detections. The deep-level feature extraction method of deep learning is very suitable for the extraction of buildings from high-resolution images. However, for the existing researches using deep learning algorithms, most algorithms are optimized by changing the network structure and are rarely combined with other methods. The pertinence of building extraction is not strong. This paper not only studies the influence of the internal structure of the convolutional neural network in deep learning on the extraction results, but also studies the combination of ambiguity, Morphological Building Index (MBI) with deep learning. The main improvements in this paper are as follows: (1) Based on the DeepLab v3+ network structure, we propose the PwDeepLab network which improves feature fusion methods and loss functions; (2) We propose a blur degree constraint method. We define a new blur degree formula to evaluate the image blur degree. In the case of a fixed image block size, the image is upsampled and enhanced by the image blur degree constraint; (3) We propose a morphological index constraint method. The pixels of the original image are stretched and enhanced where the Morphological Building Index (MBI) is above the threshold. Therefore, the building information can be highlighted with less changes to the original image characteristics. Our method is verified using the Massachusetts dataset and the satellite dataset II (East Asia) of Wuhan University. The main building types of the two datasets are quite different. As a result, the accuracy of the proposed method on the two datasets increases by 10.9% and 3.8%, respectively, compared with DeepLab v3+, and increases by 10.0% and 9.6%, respectively, compared with U-Net. The higher accuracy reflects the superiority and robustness of our method. Moreover, the extracted results match the real labels very well in details. The experimental results show that the method proposed in this paper can significantly improve the building extraction results.

Key words: deep learning, building extraction, high resolution image, PwDeepLab, feature fusion, ambiguity, morphological index, window enhancement

许泽宇, 沈占锋, 李杨, 柯映明, 李硕, 王浩宇, 焦淑慧. 结合模糊度和形态学指数约束的深度学习建筑物提取[J]. 地球信息科学学报, 2021, 23(5): 918-927.DOI:10.12082/dqxxkx.2021.200397

XU Zeyu, SHEN Zhanfeng, LI Yang, KE Yingming, LI Shuo, WANG Haoyu, JIAO Shuhui. Building Extraction by Deep Learning Method Combined with Ambiguity and Morphological Index Constraints[J]. Journal of Geo-information Science, 2021, 23(5): 918-927.DOI:10.12082/dqxxkx.2021.200397

图/表 11

图1

图2

图3

图4

图5

表1

图6

表2

图7

图8

图9

参考文献 26

[1]	龚健雅, 季顺平. 摄影测量与深度学习[J]. 测绘学报, 2018,47(6):693-704.
	[ Gong J Y, Ji S P. Photogrammetry and deep learning[J]. Acta Geodaetica et Cartographica Sinica, 2018,47(6):693-704. ]
[2]	李军军, 曹建农, 朱莹莹等. 高分辨率遥感影像建筑区域局部几何特征提取[J]. 遥感学报, 2020,24(3):233-244.
	[ Li J J, Cao J N, Zhu Y Y, et al. Built-up area detection from high resolution remote sensing images using geometric features[J]. Journal of Remote Sensing, 2020,24(3):233-244. ]
[3]	Wang J, Yang X C, Qin X B, et al. An efficient approach for automatic rectangular building extraction from very high resolution optical satellite imagery[J]. IEEE Geoscience and Remote Sensing Letters, 2015,12(3):487-491. doi: 10.1109/LGRS.2014.2347332
[4]	张浩, 赵云胜, 陈冠宇, 等. 基于支持向量机的遥感图像建筑物识别与分类方法研究[J]. 地质科技情报, 2016,35(6):194-199.
	[ Zhang H, Zhao Y S, Chen G Y, et al. Remote sensing image building recognition and classification based on the support vector machine[J]. Geological Science and Technology Information, 2016,35(6):194-199. ]
[5]	杨山. 发达地区城乡聚落形态的信息提取与分形研究——以无锡市为例[J]. 地理学报, 2000,55(6):671-678. doi: 10.11821/xb200006004
	[ Yang S. On extraction and fractal of urban and rural residential spatial pattern in developed area[J]. Acta Geographica Sinica, 2000,55(6):671-678. ]
[6]	查勇, 倪绍祥, 杨山. 一种利用TM图像自动提取城镇用地信息的有效方法[J]. 遥感学报, 2003,7(1):37-40.
	[ Zha Y, Ni S X, Yang S. An effective approach to automatically extract urban land-use from TM imagery[J]. Journal of Remote Sensing, 2003,7(1):37-40. ]
[7]	Huang X, Zhang L. A Multidirectional and multiscale morphological index for automatic building extraction from multispectral GeoEye-1 imagery[J]. Photogrammetric Engineering and Remote Sensing, 2011,77(7):721-32. doi: 10.14358/PERS.77.7.721
[8]	陈志强, 陈健飞. 基于NDBI指数法的城镇用地影像识别分析与制图[J]. 地球信息科学, 2006,8(2):137-140.
	[ Chen Z Q, Chen J F. Investigation on extracting the space information of urban land-use from high spectrum resolution image of ASTER by NDBI method[J]. Journal of Geo-information Science, 2006,8(2):137-140. ]
[9]	杨智翔, 何秀凤. 基于改进的NDBI指数法的遥感影像城镇用地信息自动提取[J]. 河海大学学报(自然科学版), 2010,38(2):181-184.
	[ Yang Z X, He X F. Automatic extraction of urban land-use information from remote sensing hnages based on improved NDBI method[J]. Journal of Hohai University (Natural Sciences), 2010,38(2):181-184. ]
[10]	陈行, 卓莉, 陶海燕. 基于MMBI的高分辨率影像建筑物提取研究[J]. 遥感技术与应用, 2016,31(5):930-938.
	[ Chen H, Zhuo L, Tao H Y. Study on building extraction from high spatial resolution images using MMBI[J]. Remote Sensing Technology and Application, 2016,31(5):930-938. ]
[11]	贺晓璐, 刘振华, 胡月明. 基于面向对象的建筑物信息提取方法研究[J]. 河南理工大学学报(自然科学版), 2020,39(2):53-61.
	[ He X L, Liu Z H, Hu Y M. Research on object-oriented building information extraction method[J]. Journal of Henan Polytechnic University (Natural Science), 2020,39(2):53-61. ]
[12]	游永发, 王思远, 王斌等. 高分辨率遥感影像建筑物分级提取[J]. 遥感学报, 2019,23(1):125-136.
	[ You Y F, Wagn S Y, Wang B, et al. Study on hierarchical building extraction from high resolution remote sensing imagery[J]. Journal of Remote Sensing, 2019,23(1):125-136. ]
[13]	Guo M Q, Liu H, Xu Y Y, et al. Building extraction based on U-Net with an attention block and multiple losses[J]. Remote Sensing, 2020,12(9):1400. doi: 10.3390/rs12091400
[14]	Ronneberger O, Fischer P, Brox T. U-Net: Convolutional networks for biomedical image segmentation[C]. Proceeding of 18^th International Conference on Medical Image Computing and Computer-Assisted Intervention, 2015: 234-241.
[15]	Lin T Y, Goyal P, Girshick R, et al. Focal loss for dense object detection[C]. Proceeding of 16^th IEEE International Conference on Computer Vision. 2017: 2999-3007.
[16]	Kang W C, Xiang Y M, Wang F, et al. EU-Net: An efficient fully convolutional network for building extraction from optical remote sensing images[J]. Remote Sensing, 2019,11(23):2813. doi: 10.3390/rs11232813
[17]	Liu H, Luo J C, Huang B, et al. DE-Net: Deep encoding network for building extraction from high-resolution remote sensing imagery[J]. Remote Sensing, 2019,11(20):2380. doi: 10.3390/rs11202380
[18]	朱光亚. 基于深度学习的遥感影像建筑物提取方法研究[D]. 杭州:浙江大学, 2019.
	[ Zhu G Y. Research on building extraction from remote sensing images based on deep learning[D]. Hangzhou: Zhejiang University, 2019. ]
[19]	Chen L C, Zhu Y, Papandreou G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]. Proceeding of 15^th European Conference on Computer Vision, 2018, 833-851.
[20]	Chollet F. Xception: Deep learning with depthwise separable convolutions[C]. Proceeding of 30^th IEEE Conference on Computer Vision and Pattern Recognition, 2017: 1800-1807.
[21]	Chaurasia A, Culurciello E. LinkNet: exploiting encoder representations for efficient semantic segmentation[C]. Proceeding of 2017 IEEE Visual Communications and Image Processing, 2017.
[22]	Zhou L, Zhang C, Wu M. D-LinkNet: LinkNet with pretrained encoder and dilated convolution for high resolution satellite imagery road extraction[C]. Proceeding of 31^st Meeting of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2018: 192-196.
[23]	Pech-pacheco J L, Cristobal G, Chamorro-martinez J, et al. Diatom autofocusing in brightfield microscopy: A comparative study[C]. Proceeding of 15th International Conference on Pattern Recognition, 2000, 314-317.
[24]	Mnih V. Machine learning for aerial image labeling[D]. Toronto: University of Toronto, 2013.
[25]	Ji S, Weu S, Lu M. Fully convolutional networks for multisource building extraction from an open aerial and satellite imagery data set[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019,57(1):574-586. doi: 10.1109/TGRS.2018.2858817
[26]	Karen S, Andrew Z. Very deep convolutional networks for large-scale image recognition[C]. Proceeding of 3rd International Conference on Learning Representations, 2019.

	精确度	召回率	F1	等值点
U-Net	0.7455	0.7988	0.7712	0.7712
DeepLab v3+	0.7826	0.7417	0.7616	0.7649
PwDeepLab	0.7853	0.7783	0.7818	0.7821
PwDeepLab.A	0.8271	0.8414	0.8342	0.8339
PwDeepLab.A.M	0.8459	0.8513	0.8486	0.8483

	精确度	召回率	F1	等值点
U-Net	0.7584	0.7377	0.7479	0.7490
DeepLab v3+	0.7848	0.7969	0.7908	0.7907
PwDeepLab	0.8247	0.8072	0.8159	0.8168
PwDeepLab.A	0.8285	0.8062	0.8172	0.8175
PwDeepLab.A.M	0.8224	0.8198	0.8211	0.8209

结合模糊度和形态学指数约束的深度学习建筑物提取

Building Extraction by Deep Learning Method Combined with Ambiguity and Morphological Index Constraints

RichHTML

PDF (PC)

赞

可视化

摘要/Abstract

引用本文

使用本文

图/表 11

参考文献 26

相关文章 15

Metrics

本文评价

推荐阅读 0

[1]	刘戈, 姜小光, 唐伯惠. 特征优选与卷积神经网络在农作物精细分类中的应用研究[J]. 地球信息科学学报, 2021, 23(6): 1071-1081.
[2]	唐璎, 刘正军, 杨懿, 顾海燕, 杨树文. 基于特征增强和ELU的神经网络建筑物提取研究[J]. 地球信息科学学报, 2021, 23(4): 692-709.
[3]	李广洋, 寇卫利, 陈帮乾, 代飞, 强振平, 吴超. 多核学习算法及其在高光谱图像分类中的应用研究进展[J]. 地球信息科学学报, 2021, 23(3): 492-504.
[4]	朱盼盼, 李帅朋, 张立强, 李洋. 基于多任务学习的高分辨率遥感影像建筑提取[J]. 地球信息科学学报, 2021, 23(3): 514-523.
[5]	陈昂, 杨秀春, 徐斌, 金云翔, 张文博, 郭剑, 邢晓语, 杨东. 基于面向对象与深度学习的榆树疏林识别方法研究[J]. 地球信息科学学报, 2020, 22(9): 1897-1909.
[6]	郭峰, 毛政元, 邹为彬, 翁谦. 融合LiDAR数据与高分影像特征信息的建筑物提取方法[J]. 地球信息科学学报, 2020, 22(8): 1654-1665.
[7]	崔成, 任红艳, 赵璐, 庄大方. 基于街景影像多特征融合的广州市越秀区街道空间品质评估[J]. 地球信息科学学报, 2020, 22(6): 1330-1338.
[8]	毛文山, 赵红莉, 孙凤娇, 蒋云钟, 姜倩, 朱彦儒. 基于Item2Vec负采样优化的专题地图产品个性化推荐方法研究[J]. 地球信息科学学报, 2020, 22(11): 2128-2139.
[9]	郭子慧, 刘伟. 深度学习和遥感影像支持的矢量图斑地类解译真实性检查方法[J]. 地球信息科学学报, 2020, 22(10): 2051-2061.
[10]	方志祥, 倪雅倩, 黄守倩. 顾及上网行为特征的手机用户停留行为预测方法[J]. 地球信息科学学报, 2020, 22(1): 136-144.
[11]	宋关福, 卢浩, 王晨亮, 胡辰璞, 黄科佳. 人工智能GIS软件技术体系初探[J]. 地球信息科学学报, 2020, 22(1): 76-87.
[12]	蔡博文,王树根,王磊,邵振峰. 基于深度学习模型的城市高分辨率遥感影像不透水面提取[J]. 地球信息科学学报, 2019, 21(9): 1420-1429.
[13]	蔡惠慧, 朱伟, 李孜轩, 刘园园, 李龙斌, 刘畅. 基于深度学习的钨钼找矿靶区预测方法研究[J]. 地球信息科学学报, 2019, 21(6): 928-936.
[14]	方颖, 李连发. 基于机器学习的高精度高分辨率气象因子时空估计[J]. 地球信息科学学报, 2019, 21(6): 799-813.
[15]	张刚, 刘文彬, 张男. 基于区域特征分割的密集匹配点云渐进形态学滤波[J]. 地球信息科学学报, 2019, 21(4): 615-622.