利用基于残差多注意力和ACON激活函数的神经网络提取建筑物

doi:10.12082/dqxxkx.2022.210530

地球信息科学学报 ›› 2022, Vol. 24 ›› Issue (4): 792-801.doi: 10.12082/dqxxkx.2022.210530

利用基于残差多注意力和ACON激活函数的神经网络提取建筑物

吴新辉¹^,²(), 毛政元¹^,²^,^*(), 翁谦³^,⁴, 施文灶⁵^,⁶

1.福州大学数字中国研究院（福建）,福州 350108
2.福州大学空间数据挖掘与信息共享教育部重点实验室,福州 350108
3.福州大学计算机与大数据学院,福州 350108
4.福建省网络计算与智能信息处理省重点实验室（福州大学）,福州 350108
5.福建师范大学光电与信息工程学院,福州 350007
6.福建师范大学福建省光电传感应用工程技术研究中心,福州 350007

收稿日期:2021-09-02 修回日期:2021-10-10 出版日期:2022-04-25 发布日期:2022-06-25
通讯作者: ^*毛政元（1964— ）,男,湖南邵阳人,博士,教授,博士生导师,主要从事时空系统认知与测度、高分影像信息提取与地表变化检测、地理空间数据不确定性分析及其应用、土地资源信息化管理与决策服务研究。 E-mail: zymao@fzu.edu.cn
作者简介:吴新辉（1995— ）,男,福建莆田人,硕士生,主要从事深度学习、遥感影像的分析与应用研究。E-mail: wdlan4869@163.com
基金资助:
国家自然科学基金项目(41801324);国家自然科学基金项目(41701491);福建省自然科学基金面上项目(2019J01244);福建省自然科学基金面上项目(2019J01791)

A Neural Network based on Residual Multi-attention and ACON Activation Function for Extract Buildings

WU Xinhui¹^,²(), MAO Zhengyuan¹^,²^,^*(), WENG Qian³^,⁴, SHI Wenzao⁵^,⁶

1. Academy of Digital China, Fuzhou University, Fuzhou 350108, China
2. Key Laboratory of Spatial Data Mining & Information Sharing of Ministry of Education, Fuzhou University, Fuzhou 350108, China
3. College of Computer and Data Science, Fuzhou University, Fuzhou 350108, China
4. Fujian Provincial Key Laboratory of Network Computing and Intelligent Information Processing (Fuzhou University), Fuzhou 350108, China
5. College of Opto-Electronic and Information Engineering, Fujian Normal University, Fuzhou 350007, China
6. Fujian Engineering Technology Research Center of Photoelectric Sensing Application, Fujian Normal University, Fuzhou 350007, China

Received:2021-09-02 Revised:2021-10-10 Online:2022-04-25 Published:2022-06-25
Contact: MAO Zhengyuan
Supported by:
Youth Project of National Natural Science Foundation of China(41801324);Youth Project of National Natural Science Foundation of China(41701491);General project of Natural Science Foundation of Fujian Province(2019J01244);General project of Natural Science Foundation of Fujian Province(2019J01791)

摘要/Abstract

摘要：

针对目前主流深度学习网络模型应用于高空间分辩率遥感影像建筑物提取存在的内部空洞、不连续以及边缘缺失与边界不规则等问题,本文在U-Net模型结构的基础上通过设计新的激活函数（ACON）、集成残差以及通道-空间与十字注意力模块,提出RMAU-Net模型。该模型中的ACON激活函数允许每个神经元自适应地激活或不激活,有利于提高模型的泛化能力和传输性能;残差模块用于拓宽网络深度并降低训练和学习的难度,获取深层次语义特征信息;通道-空间注意力模块用于增强编码段与解码段信息的关联、抑制无关背景区域的影响,提高模型的灵敏度;十字注意力模块聚合交叉路径上所有像素的上下文信息,通过循环操作捕获全局上下文信息,提高像素间的全局相关性。以Massachusetts数据集为样本的建筑物提取实验表明,在所有参与比对的7个模型中,本文提出的RMAU-Net模型交并比与F1分数2项指标最优、查准率和查全率两项指标接近最优, RMA-UNet总体效果优于同类模型。通过逐步添加每个模块来进一步验证各模块的有效性以及本文所提方法的可靠性。

关键词: 高分影像, 建筑物提取, 卷积神经网络, ACON激活函数, 残差块, 空间注意力, 通道注意力模块, 十字注意力

Abstract:

Current mainstream deep learning network models have many problems such as inner cavity, discontinuity, missed periphery, and irregular boundaries when applied to building extraction from high spatial resolution remote sensing images. This paper proposed the RMAU-Net model by designing a new activation function (Activate Customized or Not, ACON) and integrating residuals block with channel-space and criss-cross attention module based on the U-Net model structure. The ACON activation function in the model allows each neuron to be activated or not activated adaptively, which helps improve the generalization ability and transmission performance of the model. The residual module is used to broaden the depth of the network, reduce the difficulty in training and learning, and obtain deep semantic feature information. The channel-spatial attention module is used to enhance the correlation between encoding and decoding information, suppress the influence of irrelevant background region, and improve the sensitivity of the model. The cross attention module aggregates the context information of all pixels on the cross path and captures the global context information by circular operation to improve the global correlation between pixels. The building extraction experiment using the Massachusetts dataset as samples shows that among all the 7 comparison models, the proposed RMA-UNET model is optimal in terms of intersection of union and F1-score, as well as indexes of precision and recall, and the overall performance of RMAU-Net is better than similar models. Each module is added step by step to further verify the validity of each module and the reliability of the proposed method.

Key words: High resolution image, Building extraction, Convolutional neural network, ACON activation function, Residual Block, Spatial Attention, Channel Attention, criss-cross attention

吴新辉, 毛政元, 翁谦, 施文灶. 利用基于残差多注意力和ACON激活函数的神经网络提取建筑物[J]. 地球信息科学学报, 2022, 24(4): 792-801.DOI:10.12082/dqxxkx.2022.210530

WU Xinhui, MAO Zhengyuan, WENG Qian, SHI Wenzao. A Neural Network based on Residual Multi-attention and ACON Activation Function for Extract Buildings[J]. Journal of Geo-information Science, 2022, 24(4): 792-801.DOI:10.12082/dqxxkx.2022.210530

图/表 13

图1

图2

图3

图4

图5

图6

图7

表1

图8

表2

表3

表4

表5

参考文献 24

[1]	施文灶, 毛政元. 基于图割与阴影邻接关系的高分辨率遥感影像建筑物提取方法[J]. 电子学报, 2016,44(12):2849-2854. doi: 10.3969/j.issn.0372-2112.2016.12.006
	[ Shi W Z, Mao Z Y. Building extraction from high resolution remotely sensed imagery based on shadows and graph-cut segmentation[J]. Acta Electronica Sinica, 2016,44(12):2849-2854. ] DOI: 10.3969/j.issn.0372-2112.2016.12.006 doi: 10.3969/j.issn.0372-2112.2016.12.006
[2]	Kim T, Muller J. Development of a graph-based approach for building detection[J]. Image and Vision Computing, 1999,17(1):3-14. DOI: 10.1016/S0262-8856(98)00092-4 doi: 10.1016/S0262-8856(98)00092-4
[3]	Jung C R, Schramm R. Rectangle detection based on a windowed Hough transform[C]//17th Brazilian Symposium on Computer Graphics and Image Processing. IEEE, 2004:113-120. DOI: 10.1109/SIBGRA.2004.1352951 doi: 10.1109/SIBGRA.2004.1352951
[4]	Huang X, Zhang L. A multidirectional and multiscale morphological index for automatic building extraction from multispectral GeoEye-1 imagery[J]. Photogrammetric Engineering & Remote Sensing, 2011,77(7):721-732. DOI: 10.14358/PERS.77.7.721 doi: 10.14358/PERS.77.7.721
[5]	冯凡, 王双亭, 张津, 等. 基于尺度自适应全卷积网络的遥感影像建筑物提取[J]. 激光与光电子学进展, 2021:1-20.
	[ Feng F, Wang S T, Zhang J, et al. Building extraction from remote sensing imagery based on scale-adaptive fully convolutional network[J]. Laser & Optoelectronics Progress, 2021:1-20. ]
[6]	Shao Z, Tang P, Wang Z, et al. BRRNet: A fully convolutional neural network for automatic building extraction from high-resolution remote sensing images[J]. Remote Sensing, 2020,12(6):1050. DOI: 10.3390/rs12061050 doi: 10.3390/rs12061050
[7]	Zhao W, Persello C, Stein A. Building outline delineation: From aerial images to polygons with an improved end-to-end learning framework[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2021,175:119-131. DOI: 10.1016/j.isprsjprs.2021.02.014 doi: 10.1016/j.isprsjprs.2021.02.014
[8]	崔卫红, 熊宝玉, 张丽瑶. 多尺度全卷积神经网络建筑物提取[J]. 测绘学报, 2019,48(5):597-608.
	[ Cui W H, Xiong B Y, Zhang L Y. Multi-scale fully convolutional neural network for building extraction[J]. Acta Geodaetica et Cartographica Sinica, 2019,48(5):597-608. ] DOI: CNKI:SUN:CHXB.0.2019-05-008 doi: CNKI:SUN:CHXB.0.2019-05-008
[9]	Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation[C]//International Conference on Medical image computing and computer-assisted intervention. Springer, Cham, 2015:234-241. DOI: 10.1007/978-3-319-24574-4_28 doi: 10.1007/978-3-319-24574-4_28
[10]	Jha D, Smedsrud P H, Riegler M A, et al. Resunet++: An advanced architecture for medical image segmentation[C]//2019 IEEE International Symposium on Multimedia (ISM). IEEE, 2019:225-2255. DOI: 10.1109/ISM46123.2019.00049 doi: 10.1109/ISM46123.2019.00049
[11]	Oktay O, Schlemper J, Folgoc L L, et al. Attention u-net: Learning where to look for the pancreas[J]. arXiv preprint arXiv:1804.03999, 2018.
[12]	Pan X, Yang F, Gao L, et al. Building extraction from high-resolution aerial imagery using a generative adversarial network with spatial and channel attention mechanisms[J]. Remote Sensing, 2019,11(8):917. DOI: doi.org/10.3390/rs11080917 doi: doi.org/10.3390/rs11080917
[13]	季顺平, 魏世清. 遥感影像建筑物提取的卷积神经元网络与开源数据集方法[J]. 测绘学报, 2019,48(4):448-459.
	[ Ji S P, Wei S Q. Building extraction via convolution neural networks from an open remote sensing building dataset[J]. Acta Geodaetica et Cartographica Sinica, 2019,48(4):448-459. ] DOI: 10.11947/j.AGCS.2019.20180206. doi: 10.11947/j.AGCS.2019.20180206
[14]	Guo M, Liu H, Xu Y, et al. Building extraction based on U-Net with an attention block and multiple losses[J]. Remote Sensing, 2020,12(9):1400. DOI: 10.3390/rs12091400 doi: 10.3390/rs12091400
[15]	Huang Z L, Wang X G, Huang L C, et al. Ccnet: Criss-cross attention for semantic segmentation[C]//The IEEE/CVF International Conference on Computer Vision. 2019:603-612. DOI: 10.1109/tpami.2020.3007032 doi: 10.1109/tpami.2020.3007032
[16]	Zhang Z, Wang Y. JointNet: A common neural network for road and building extraction[J]. Remote Sensing, 2019,11(6):696. DOI: 10.3390/rs11060696 doi: 10.3390/rs11060696
[17]	Zhang Z, Liu Q, Wang Y. Road extraction by deep residual u-net[J]. IEEE Geoscience and Remote Sensing Letters, 2018,15(5):749-753. DOI: 10.1109/LGRS.2018.2802944 doi: 10.1109/LGRS.2018.2802944
[18]	Alom M Z, Yakopcic C, Hasan M, et al. Recurrent residual U-Net for medical image segmentation[J]. Journal of Medical Imaging, 2019,6(1):14006. DOI: 10.1117/1.JMI.6.1.014006 doi: 10.1117/1.JMI.6.1.014006
[19]	陈凯强, 高鑫, 闫梦龙, 等. 基于编解码网络的航空影像像素级建筑物提取[J]. 遥感学报, 2020,24(9):1134-1142.
	[ Chen K Q, Gao X, Yan M L, et al. Building extraction in pixel level from aerial imagery with a deep encoder-decoder network[J]. Journal of Remote Sensing (Chinese), 2020,24(9):1134-1142. ] DOI: CNKI:SUN:YGXB.0.2020-09-008 doi: CNKI:SUN:YGXB.0.2020-09-008
[20]	Ramachandran P, Zoph B, Le Q V. Searching for activation functions[J]. arXiv preprint arXiv:1710.05941, 2017.
[21]	He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition[C]//The IEEE conference on computer vision and pattern recognition. 2016:770-778. DOI: 10.1109/CVPR.2016.90 doi: 10.1109/CVPR.2016.90
[22]	Wang X L, Girshick R, Gupta A, et al. Non-local neural networks[C]//The IEEE conference on computer vision and pattern recognition. 2018:7794-7803. DOI: 10.1109/CVPR.2018.00813 doi: 10.1109/CVPR.2018.00813
[23]	Mnih V. Machine learning for aerial image labeling[M]. University of Toronto (Canada), 2013.
[24]	Wang F, Jiang M Q, Qian C, et al. Residual attention network for image classification[C]//The IEEE conference on computer vision and pattern recognition. 2017:3156-3164. DOI: 10.1109/CVPR.2017.683 doi: 10.1109/CVPR.2017.683

项目	参数
中央处理器	Intel(R) Core(TM) i5-10600KF CPU @4.10 GHz
显存	8 G
硬盘	2 T
显卡	NVIDIA GeForce RTX 3070
操作系统	Window10
开发语言	Python
深度学习框架	PyTorch

模型	U-Net	Res-Unet	Attention-Unet	Res-Attention-Unet	RMAU-Net
训练时间/h	30.6	32.6	38.3	40.1	39.2
测试时间/s	5.3	5.6	6.8	7.3	6.9

模型	IOU	F1-score	Precision	Recall
U-Net	70.69	82.48	85.99	79.61
Res-Unet	70.51	82.06	85.98	79.04
Attention-Unet	71.34	83.11	87.09	79.69
Res-Attention-Unet	71.41	83.12	87.63	79.37
SA-Net^[5]	73.45	84.69	86.78	82.70
RMAU-Net	73.68	84.75	87.10	82.69

模型	IOU	F1-score	Precision	Recall
U-Net+ReLu	70.69	82.48	85.99	79.61
U-Net+ACON-C	71.33	83.09	86.65	80.09
RMAU-Net+ReLu	73.16	84.40	86.89	82.21
RMAU-Net+ACON-C	73.68	84.75	87.10	82.69

模型	IOU	F1-score	Precision	Recall
U-Net(基准)	71.33	83.09	86.65	80.09
U-Net(基准)+RB	71.61	83.33	85.78	81.22
U-Net(基准)+RB+CSA	72.76	84.11	86.73	81.78
U-Net(基准)+RB+CSA+CCA	73.68	84.75	87.10	82.69

利用基于残差多注意力和ACON激活函数的神经网络提取建筑物

A Neural Network based on Residual Multi-attention and ACON Activation Function for Extract Buildings

RichHTML

PDF (PC)

赞

可视化

摘要/Abstract

引用本文

使用本文

图/表 13

参考文献 24

相关文章 15

Metrics

本文评价

推荐阅读 0

[1]	令振飞, 刘涛, 杜萍, 赵丹, 陈朴一, 马天恩. 一种支持建筑群组相似模式检索的变分图卷积自编码模型[J]. 地球信息科学学报, 2023, 25(7): 1405-1417.
[2]	陈晖萱, 郭旦怀, 葛世寅, 王婧, 王彦棡, 陈峰, 杨微石. M2T多源知识图谱融合的空间场景描述文本自动生成框架[J]. 地球信息科学学报, 2023, 25(6): 1176-1185.
[3]	刘潇, 刘智, 林雨准, 王淑香, 左溪冰. 面向遥感影像场景分类的类中心知识蒸馏方法[J]. 地球信息科学学报, 2023, 25(5): 1050-1063.
[4]	侯慧太, 蓝朝桢, 徐青. 基于卫星影像全局和局部深度学习特征检索的无人机绝对定位方法[J]. 地球信息科学学报, 2023, 25(5): 1064-1074.
[5]	陈科, 管海燕, 雷相达, 曹爽. 基于特征增强核点卷积网络的多光谱LiDAR点云分类方法[J]. 地球信息科学学报, 2023, 25(5): 1075-1087.
[6]	衡雪彪, 许捍卫, 唐璐, 汤恒, 许怡蕾. 基于改进全卷积神经网络模型的土地覆盖分类方法研究[J]. 地球信息科学学报, 2023, 25(3): 495-509.
[7]	徐繁树, 王保云, 韩俊. 一种沟谷型潜在泥石流危险性评价方法：基于多源数据融合的卷积神经网络[J]. 地球信息科学学报, 2023, 25(3): 588-605.
[8]	高鹏飞, 曹雪峰, 李科, 游雄. 融合多元稀疏特征与阶层深度特征的遥感影像目标检测[J]. 地球信息科学学报, 2023, 25(3): 638-653.
[9]	黄帅元, 董有福, 李海鹏. 黄土高原区SRTM1 DEM高程误差校正模型构建及对比分析[J]. 地球信息科学学报, 2023, 25(3): 669-681.
[10]	黄昕, 毛政元. 基于时空多图卷积网络的网约车乘客需求预测[J]. 地球信息科学学报, 2023, 25(2): 311-323.
[11]	刘洋, 康健, 管海燕, 汪汉云. 基于双注意力残差网络的高分遥感影像道路提取模型[J]. 地球信息科学学报, 2023, 25(2): 396-408.
[12]	何海清, 周福阳, 陈敏, 陈婷, 官云兰, 曾怀恩, 魏燕. 耦合卷积神经网络与注意力机制的无人机摄影测量果树树冠分割方法[J]. 地球信息科学学报, 2023, 25(12): 2387-2401.
[13]	贺日兴, 唐宗棣, 姜超, 林艳, 陆宇梅, 李欣然, 龙伟, 邓悦. 顾及道路权重的图卷积犯罪时空预测模型[J]. 地球信息科学学报, 2023, 25(10): 1986-1999.
[14]	于明洋, 陈肖娴, 张文焯, 刘耀辉. 融合网格注意力阀门和特征金字塔结构的高分辨率遥感影像建筑物提取[J]. 地球信息科学学报, 2022, 24(9): 1785-1802.
[15]	张华, 郑祥成, 郑南山, 史文中. 基于MAEU-CNN的高分辨率遥感影像建筑物提取[J]. 地球信息科学学报, 2022, 24(6): 1189-1203.