基于Res_AttentionUnet的高分辨率遥感影像建筑物提取方法

doi:10.12082/dqxxkx.2021.210008

地球信息科学学报 ›› 2021, Vol. 23 ›› Issue (12): 2232-2243.doi: 10.12082/dqxxkx.2021.210008

基于Res_AttentionUnet的高分辨率遥感影像建筑物提取方法

李传林¹^,³^,⁴(), 黄风华²^,^*(), 胡威¹^,³^,⁴, 曾江超¹^,³^,⁴

1.福州大学空间数据挖掘与信息共享教育部重点实验室,福州350108
2.阳光学院空间数据挖掘与应用福建省高校工程研究中心,福州 350015
3.卫星空间信息技术综合应用国家地方联合工程研究中心,福州350108
4.福州大学数字中国研究院（福建）,福州350108

收稿日期:2021-01-07 修回日期:2021-04-03 出版日期:2021-12-25 发布日期:2022-02-25
通讯作者: *黄风华（1982— ）,男,福建莆田人,博士,教授,主要从事机器学习与遥感影像处理研究。E-mail: fenghuait@sina.com
作者简介:李传林（1996— ）,男,安徽安庆人,硕士生,主要从事深度学习与遥感影像处理研究。E-mail: 1179180471@qq.com
基金资助:
福建省自然科学基金项目(2019J01088)

Building Extraction from High-Resolution Remote Sensing Image based on Res_AttentionUnet

LI Chuanlin¹^,³^,⁴(), HUANG Fenghua²^,^*(), HU Wei¹^,³^,⁴, ZENG Jiangchao¹^,³^,⁴

1. Key Laboratory of Spatial Data Mining &Information Sharing of Ministry of Education, Fuzhou University, Fuzhou 350108, China
2. Fujian University Engineering Research Center of Spatial Data Mining and Application, Yango University, Fuzhou 350015, China
3. National Engineering Research Centre of Geospatial Information Technology, Fuzhou University, Fuzhou 350108, China
4. The Academy of Digital China, Fuzhou University, Fuzhou 350108, China

Received:2021-01-07 Revised:2021-04-03 Online:2021-12-25 Published:2022-02-25
Supported by:
Fujian Natural Science Foundation(2019J01088)

摘要/Abstract

摘要：

针对目前基于深度学习与高分辨率遥感影像的建筑物提取研究现状,本文提出了一种综合ResNet中的ResBlock残差模块和Attention注意力机制的改进型Unet网络（Res_AttentionUnet）,并将其应用于高分辨率遥感影像建筑物提取,有效地提高了建筑物的提取精度。具体优化方法为：在传统的Unet语义分割网络卷积层中加入针对初高级特征加强提取的ResBlock残差模块,并在网络阶跃连接部分加入Attention注意力机制模块。其中,ResBlock残差模块使卷积后的特征图获取更多的底层信息,增强卷积结构的鲁棒性,从而防止欠拟合;Attention注意力机制可增强对建筑物区域像素的特征学习,使特征提取更完善,从而提高建筑物提取的准确率。本研究采用武汉大学季顺平团队提供的开放数据集（WHU Building Dataset）作为实验数据,并从中选取3个具有不同建筑物特征和代表性的实验区域,然后分别对不同实验区域进行预处理（包括滑动裁剪和图像增强等）,最后分别使用Unet、ResUnet、AttentionUnet和Res_AttentionUnet 4种不同的网络模型对3个不同实验区进行建筑物提取实验,并对实验结果进行交叉对比分析。实验结果表明,与其他3种网络相比,本文所提出的Res_AttentionUnet在基于高分辨率遥感影像的建筑物提取中具有更高的精度,平均提取精度达到95.81%,相较于原始Unet网络提升17.94%,同时相较于仅加入残差模块的Unet网络（ResUnet）提升2.19%,能够显著地提升高分辨率遥感影像中建筑物提取的效果。

关键词: 深度学习, 高分辨率遥感影像, 建筑物提取, 残差模块, 注意力模块, 卷积神经网络, Unet网络, Res_AttentionUnet

Abstract:

To contribute to the current research of building extraction based on deep learning and high-resolution remote sensing images, we propose an improved Unet network (Res_AttentionUnet), which combines the Residual module of ResNet and Attention mechanism. We apply the Unet network to the extraction of buildings from high-resolution remote sensing images, which effectively improves the extraction accuracy of buildings. The specific optimization method can be divided into three parts. Firstly, in the traditional Unet semantic segmentation network convolution layer, the ResBlock module is added to enhance the extraction of low-level and high-level features. Meanwhile, the Attention mechanism module is added to the network step connection part. Secondly, in the whole net, the ResBlock module enables the convoluted feature map to obtain more bottom information and enhance the robustness of the convolution structure, so as to prevent underfitting. Thirdly, the Attention mechanism can enhance the feature learning of building area pixels, making feature extraction more complete, so as to improve the accuracy of building extraction. In this study, we use the open data set (WHU Building Dataset), provided by Ji Shunping team of Wuhan University, as the experimental data and select three experimental areas with different building characteristics and representativeness. Then, we preprocess the different experimental areas (including sliding, cropping, and image enhancement, etc.). Finally, we use four different network models of Unet, ResUnet, AttentionUnet, and Res_AttentionUnet to extract buildings from three different experimental areas. The experimental results are cross-compared and analyzed. The experimental results show that, compared with the other three networks, the Res_AttentionUnet proposed in this paper has higher accuracy in the building extraction from high-resolution remote sensing images. The average extraction accuracy of Res_AttentionUnet is 95.81%, which is 17.94% higher than the original Unet network, and 2.19% higher than ResUnet (the Unet with only residual module). The results demonstrate that Res_AttentionUnet can significantly improve the effectiveness of building extraction in high-resolution remote sensing images.

Key words: deep learning, remote sensing image, building extraction, residual module, attention module, convolution neural network, unet network, Res_AttentionUnet

李传林, 黄风华, 胡威, 曾江超. 基于Res_AttentionUnet的高分辨率遥感影像建筑物提取方法[J]. 地球信息科学学报, 2021, 23(12): 2232-2243.DOI:10.12082/dqxxkx.2021.210008

LI Chuanlin, HUANG Fenghua, HU Wei, ZENG Jiangchao. Building Extraction from High-Resolution Remote Sensing Image based on Res_AttentionUnet[J]. Journal of Geo-information Science, 2021, 23(12): 2232-2243.DOI:10.12082/dqxxkx.2021.210008

图/表 14

图1

图2

图3

表1

图4

表2

表3

表4

表5

图5

表6

表7

表8

表9

参考文献 20

[1]	杨州, 慕晓冬, 王舒洋, 等. 基于多尺度特征融合的遥感图像场景分类[J]. 光学精密工程, 2018, 26(12):3099-3107.
	[ Yang Z, Mu X D, Wang S Y, et al. Scene classification of remote sensing images based on multiscale features fusion[J]. Optics and Precision Engineering, 2018, 26(12):3099-3107. ]
[2]	付秀丽, 黎玲萍, 毛克彪, 等. 基于卷积神经网络模型的遥感图像分类[J]. 高技术通讯, 2017, 27(3):203-212.
	[ Fu X L, Li L P, Mao K B, et al. Remote sensing image classification based on CNN model[J]. Chinese High Technology Letters, 2017, 27(3):203-212. ]
[3]	苏健民, 杨岚心, 景维鹏. 基于U-Net的高分辨率遥感图像语义分割方法[J]. 计算机工程与应用, 2019, 55(7):207-213.
	[ Su J M, Yang L X, Jing W P. U-net based semantic segmentation method for high resolution remote sensing image[J]. Computer Engineering and Applications, 2019, 55(7):207-213. ]
[4]	Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6):84-90. doi: 10.1145/3065386
[5]	Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation[C]// 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2015:3431-3440.
[6]	顾炼, 许诗起, 竺乐庆. 基于FlowS-Unet的遥感图像建筑物变化检测[J]. 自动化学报, 2020, 46(6):1291-1300.
	[ Gu L A, Xu S Q, Zhu L Q. Detection of building changes in remote sensing images via FlowS-unet[J]. Acta Automatica Sinica, 2020, 46(6):1291-1300. ]
[7]	Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation[C]// Medical Image Computing and Computer-Assisted Intervention - MICCAI 2015, DOI: 10.1007/978-3-319-24574-4_28. doi: 10.1007/978-3-319-24574-4_28
[8]	Zhou Z W, Siddiquee M M R, Tajbakhsh N, et al. UNet: Redesigning skip connections to exploit multiscale features in image segmentation[J]. IEEE Transactions on Medical Imaging, 2020, 39(6):1856-1867. doi: 10.1109/TMI.42
[9]	Tang P, Liang Q K, Yan X T, et al. Efficient skin lesion segmentation using separable-Unet with stochastic weight averaging[J]. Computer Methods and Programs in Biomedicine, 2019, 178:289-301. doi: 10.1016/j.cmpb.2019.07.005
[10]	He N J, Fang L Y, Plaza A. Hybrid first and second order attention Unet for building segmentation in remote sensing images[J]. Science China Information Sciences, 2020, 63(4):1-12.
[11]	刘文涛, 李世华, 覃驭楚. 基于全卷积神经网络的建筑物屋顶自动提取[J]. 地球信息科学学报, 2018, 20(11):1562-1570. doi: 10.12082/dqxxkx.2018.180159
	[ Liu W T, Li S H, Qin Y C. Automatic building roof extraction with fully convolutional neural network[J]. Journal of Geo-information Science, 2018, 20(11):1562-1570. ]
[12]	张华博. 基于深度学习的图像分割研究与应用[D]. 成都:电子科技大学, 2018.
	[ Zhang H B. Research and application of image segmentation by deep learning[D]. Chengdu: University of Electronic Science and Technology of China, 2018. ]
[13]	唐文博. 基于卷积神经网络的高分辨率多光谱遥感图像上的城区建筑物变化检测技术[D]. 杭州:浙江大学, 2019.
	[ Tang W B. Building change detection technology based on convention neural network in high resolution multispectral remote sensing images[D]. Hangzhou: Zhejiang University, 2019. ]
[14]	Gu J X, Wang Z H, Kuen J, et al. Recent advances in convolutional neural networks[J]. Pattern Recognition, 2018, 77:354-377. doi: 10.1016/j.patcog.2017.10.013
[15]	Liu W B, Wang Z D, Liu X H, et al. A survey of deep neural network architectures and their applications[J]. Neurocomputing, 2017, 234:11-26. doi: 10.1016/j.neucom.2016.12.038
[16]	He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2016:770-778.
[17]	Mnih V, Heess N, Graves A, et al. Recurrent models of visual attention[EB/OL]. 2014: arXiv: 1406.6247[G]. https://arxiv.org/abs/1406.6247.
[18]	Vaswani A, Shazeer N, Parmar N, et al. Attention Is All You Need[C]. Advances in Neural Information Processing Systems, Long Beach, 2017:6000-6010.
[19]	Oktay O, Schlemper J, Folgoc L L, et al. Attention u-net: Learning where to look for the pancreas[J]. arXiv preprint arXiv: 1804. 03999, 2018.
[20]	季顺平, 魏世清. 遥感影像建筑物提取的卷积神经元网络与开源数据集方法[J]. 测绘学报, 2019, 48(4):448-459.
	[ Ji S P, Wei S Q. Building extraction via convolutional neural networks from an open remote sensing building dataset[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(4):448-459. ]

	区域方框颜色	大小/像素×像素	特征
区域1	Green	2683×2580	小面积规则建筑物较多
区域2	Blue	4591×2455	大面积不规则建筑物较多
区域3	Red	2641×3958	大小面积建筑物,规则与不规则混合

	检测建筑像元数	检测非建筑像元数
实际建筑像元数	TP	FN
实际非建筑像元数	FP	TN

	区域影像		实际参考建筑物Label
区域1
区域2
区域3

	原始影像	水平翻转	垂直翻转	对角镜像
影像
参考建筑物标签

参数	具体设定
Epoch/轮	60
Batchsize/张	2
学习率	0.001
优化器	Adam
迭代次数/次	区域1: 20 640 区域2: 31 560 区域3: 30 960

基于Res_AttentionUnet的高分辨率遥感影像建筑物提取方法

Building Extraction from High-Resolution Remote Sensing Image based on Res_AttentionUnet

RichHTML

PDF (PC)

赞

可视化

摘要/Abstract

引用本文

使用本文

图/表 14

参考文献 20

相关文章 15

Metrics

本文评价

推荐阅读 0

	Unet	ResUnet	AttentionUnet	Res_AttentionUnet
Precision	0.7780	0.8787	0.9214	0.8541
Recall	0.3743	0.8980	0.3158	0.9173
F1	0.5055	0.8883	0.4704	0.8846
Total Precision	0.6629	0.9510	0.5407	0.9506
IoU	0.3382	0.7990	0.3075	0.7931
mIoU	0.4655	0.8691	0.3652	0.9068

	Unet	ResUnet	AttentionUnet	Res_AttentionUnet
Precision	0.6344	0.9118	0.8597	0.9600
Recall	0.8961	0.9753	0.7209	0.9636
F1	0.7429	0.9425	0.7842	0.9618
Total Precision	0.8508	0.9621	0.8392	0.9740
IOU	0.5910	0.8912	0.6450	0.9264
mIOU	0.7003	0.9183	0.7090	0.9440

	Unet	ResUnet	AttentionUnet	Res_AttentionUnet
Precision	0.7781	0.6203	0.9403	0.8734
Recall	0.6341	0.9770	0.3921	0.9327
F1	0.6988	0.7588	0.5534	0.9021
Total Precision	0.8223	0.8956	0.5981	0.9498
IOU	0.5370	0.6114	0.3826	0.8216
mIOU	0.6566	0.7433	0.4237	0.8782

	Unet	ResUnet	AttentionUnet	Res_AttentionUnet
Precision	0.7302	0.8036	0.9071	0.8958
Recall	0.6348	0.9501	0.4763	0.9379
F1	0.6491	0.8632	0.6027	0.9162
Total Precision	0.7787	0.9362	0.6593	0.9581
IOU	0.4887	0.7672	0.4450	0.8470
mIOU	0.6075	0.8436	0.4993	0.9097

[1]	周欣昕, 吴艳兰, 李梦雅, 郑智腾. 基于特征分离机制的深度学习植被自动提取方法[J]. 地球信息科学学报, 2021, 23(9): 1675-1689.
[2]	李国清, 柏永青, 杨轩, 陈正超, 余海坤. 基于深度学习的高分辨率遥感影像土地覆盖自动分类方法[J]. 地球信息科学学报, 2021, 23(9): 1690-1704.
[3]	刘戈, 姜小光, 唐伯惠. 特征优选与卷积神经网络在农作物精细分类中的应用研究[J]. 地球信息科学学报, 2021, 23(6): 1071-1081.
[4]	许泽宇, 沈占锋, 李杨, 柯映明, 李硕, 王浩宇, 焦淑慧. 结合模糊度和形态学指数约束的深度学习建筑物提取[J]. 地球信息科学学报, 2021, 23(5): 918-927.
[5]	唐璎, 刘正军, 杨懿, 顾海燕, 杨树文. 基于特征增强和ELU的神经网络建筑物提取研究[J]. 地球信息科学学报, 2021, 23(4): 692-709.
[6]	朱盼盼, 李帅朋, 张立强, 李洋. 基于多任务学习的高分辨率遥感影像建筑提取[J]. 地球信息科学学报, 2021, 23(3): 514-523.
[7]	郭紫甜, 王春梅, 刘欣, 庞国伟, 朱梦阳, 王晋卿. 基于小流域抽样单元的中国FROM-GLC30数据精度评价[J]. 地球信息科学学报, 2021, 23(3): 524-535.
[8]	王毅, 方志策, 牛瑞卿, 彭令. 基于深度学习的滑坡灾害易发性分析[J]. 地球信息科学学报, 2021, 23(12): 2244-2260.
[9]	储国中, 李蒙蒙, 汪小钦. 融合高度特征的高分遥感影像多尺度城市建筑类型分类[J]. 地球信息科学学报, 2021, 23(11): 2073-2085.
[10]	陆大进, 黎东, 朱笑笑, 聂胜, 周国清, 张兴忆, 杨超. 基于卷积神经网络的ICESat-2光子点云去噪分类[J]. 地球信息科学学报, 2021, 23(11): 2086-2095.
[11]	冯叶涵, 陈亮, 贺晓冬. 基于百度街景的SVF计算及其在城市热岛研究中的应用[J]. 地球信息科学学报, 2021, 23(11): 1998-2012.
[12]	王海起, 孔浩然, 李学伟. 基于过滤文本和社交网络的用户常驻位置预测[J]. 地球信息科学学报, 2021, 23(10): 1778-1786.
[13]	杨佳宾, 范大昭, 杨幸彬, 纪松, 雷蓉. 面向倾斜摄影的深度学习航空影像匹配方法[J]. 地球信息科学学报, 2021, 23(10): 1823-1837.
[14]	徐佳伟, 刘伟, 单浩宇, 史嘉诚, 李二珠, 张连蓬, 李行. 基于PRCUnet的高分遥感影像建筑物提取[J]. 地球信息科学学报, 2021, 23(10): 1838-1849.
[15]	陈昂, 杨秀春, 徐斌, 金云翔, 张文博, 郭剑, 邢晓语, 杨东. 基于面向对象与深度学习的榆树疏林识别方法研究[J]. 地球信息科学学报, 2020, 22(9): 1897-1909.