基于双注意力残差网络的高分遥感影像道路提取模型

doi:10.12082/dqxxkx.2023.220513

地球信息科学学报 ›› 2023, Vol. 25 ›› Issue (2): 396-408.doi: 10.12082/dqxxkx.2023.220513

基于双注意力残差网络的高分遥感影像道路提取模型

刘洋¹(), 康健¹, 管海燕¹^,^*(), 汪汉云²

1.南京信息工程大学遥感与测绘工程学院，南京 210044
2.信息工程大学测绘工程学院，郑州 450052

收稿日期:2022-07-14 修回日期:2022-09-08 出版日期:2023-02-25 发布日期:2023-04-19
通讯作者: *管海燕（1976— ），女，江苏南京人，博士，教授，主要从事遥感数据智能解译。E-mail: guanhy.nj@nuist.edu.cn
作者简介:刘洋（1996— ），男，安徽淮南人，硕士，主要从事高分辨率遥感影像处理。E-mail: 20201248058@nuist.edu.cn
基金资助:
国家自然科学基金项目(41971414)

Road Extraction Model of High-resolution Remote Sensing Images based on Dual-attention Residual Network

LIU Yang¹(), KANG Jian¹, GUAN Haiyan¹^,^*(), WANG Hanyun²

1. School of Remote Sensing & Geomatics Engineering,Nanjing University of Information Science & Technology, Nanjing 210044, China
2. School of Geomatics Engineering, Information Engineering University, Zhengzhou 450052, China

Received:2022-07-14 Revised:2022-09-08 Online:2023-02-25 Published:2023-04-19
Contact: GUAN Haiyan
Supported by:
National Natural Science Foundation of China(41971414)

摘要/Abstract

摘要：

高分辨率遥感影像中，道路光谱信息丰富，且空间几何结构更清晰。但是，基于高分遥感影像的道路提取面临道路尺寸变化大、容易受树木、建筑物及阴影遮挡等因素影响，导致提取结果不完整。此外，高分遥感影像中同物异谱和异物同谱现象较为严重，从而影响道路提取结果连续性及细小道路信息完整性，而且难以区分道路和非道路不透水层。因此，本文提出基于双注意力残差网络的道路提取模型DARNet，利用深度编码网络，获取细粒度高阶语义信息，增强网络对细小道路的提取能力，通过嵌入串联式通道-空间双重注意力模块，获取道路特征图逐通道的全局语义信息，实现道路特征的高效表达及多尺度道路信息的深层融合，增强阴影和遮挡环境下网络模型的鲁棒性，改善道路提取细节缺失现象，实现复杂环境下高效、准确的道路自动化提取。本文在3个实验数据集对DARNet和DLinkNet、DeepLabV3+等5个对比模型进行对比试验和定量评估，结果表明，本文DARNet模型的F1分别为77.92%、67.88%和80.37%，高于对比模型。此外，定性比较表明，本文提出模型可以有效克服由于物体阴影、遮挡和高分影像光谱变化导致道路提取不准确与不完整问题，改善细小道路漏提、错提等现象，提高道路网提取的完整性和连续性。

关键词: 高分辨率影像, 道路提取, 深度学习, 语义分割, 卷积神经网络, 残差, DARNet, 注意力机制

Abstract:

In high-resolution remote sensing images, the spectral information of road is rich, and the spatial geometric structure is clear. However, the road extraction is still faced with challenges such as changes in road size and influences from trees, buildings, and occlusion shadow, which often leads to incomplete extraction results. In addition, the phenomenon of the same object with different spectrum and the foreign body with the same spectrum is more serious, which affects the continuity of road extraction and the integrity of small road information, and it is difficult to distinguish road and non-road impervious layer. Therefore, a road extraction model, DARNet, is proposed in this study to address the above limitations. It uses a deep learning network to obtain fine-grained high-level semantic information and enhance the network's ability to extract fine roads. By embedding the serial channel-space dual attention module, the global semantic information of road feature map is obtained, and the robustness of the network model in shadow and occlusion environment is enhanced. The efficient expression of road features and the deep fusion of multi-scale road information are achieved, the phenomenon of missing details in road extraction is improved, and the efficient and accurate automatic road extraction in complex environment is realized. In this paper, a quantitative comparison is carried out based on three experimental datasets, using DARNet, DLinkNet, and DeepLabV3+ etc. The results show that the F1 of the proposed model is 77.92%, 67.88% and 80.37% for three datasets, respectively, which is higher than that of the comparison models. In addition, the qualitative comparison shows that the proposed model can effectively overcome the problem of inaccurate and incomplete road extraction caused by object shadow, occlusion, and spectral changes of high-resolution images, avoid the phenomenon of missing and miscarrying of small roads, and improve the integrity and continuity of road network extraction.

Key words: high resolution imagery, road extraction, deep learning, semantic segmentation, convolutional neural network, residual, DARNet, attention mechanism

刘洋, 康健, 管海燕, 汪汉云. 基于双注意力残差网络的高分遥感影像道路提取模型[J]. 地球信息科学学报, 2023, 25(2): 396-408.DOI:10.12082/dqxxkx.2023.220513

LIU Yang, KANG Jian, GUAN Haiyan, WANG Hanyun. Road Extraction Model of High-resolution Remote Sensing Images based on Dual-attention Residual Network[J]. Journal of Geo-information Science, 2023, 25(2): 396-408.DOI:10.12082/dqxxkx.2023.220513

图/表 12

图1

图2

图3

图4

表1

图5

表2

图6

图7

图8

表3

图9

参考文献 38

[1]	Cheng G L, Zhu F Y, Xiang S M, et al. Road centerline extraction via semisupervised segmentation and multidirection nonmaximum suppression[J]. IEEE Geoscience and Remote Sensing Letters, 2016, 13(4):545-549. DOI:10.1109/LGRS.2016.2524025 doi: 10.1109/LGRS.2016.2524025
[2]	张永宏, 何静, 阚希, 等. 遥感图像道路提取方法综述[J]. 计算机工程与应用, 2018, 54(13):1-10,51. doi: 10.3778/j.issn.1002-8331.1804-0271
	[ Zhang Y H, He J, Kan X, et al. Summary of road extraction methods for remote sensing images[J]. Computer Engineering and Applications, 2018, 54(13):1-10,51. ] DOI:10.3778/j.issn.1002-8331.1804-0271 doi: 10.3778/j.issn.1002-8331.1804-0271
[3]	曹敏. 基于频谱的高分辨率遥感影像纹理尺度分析及选择[D]. 北京: 中国地质大学(北京), 2020.
	[ Cao M. Frequency spectrum based optimal texture window size selection for high spatial resolution remote sensing image analysis[D]. Beijing: China University of Geosciences, 2020. ]
[4]	Lin X G, Zhang J X, Liu Z J, et al. Semi-automatic road tracking by template matching and distance transform[C]// Joint Urban Remote Sensing Event. IEEE, 2009:1-7. DOI:10.1109/URS.2009.5137485 doi: 10.1109/URS.2009.5137485
[5]	Udomhunsakul S, Kozaitis S P, Sritheeravirojana U. Semi-automatic road extraction from aerial images[C]// Remote Sensing. Proc SPIE 5239, Remote Sensing for Environmental Monitoring, GIS Applications, and Geology III, Barcelona, Spain. 2004, 5239:26-32. DOI:10.1117/12.508365 doi: 10.1117/12.508365
[6]	Mayer H, Laptev I, Baumgartner A, et al. Automatic road road extraction based on multi-scale modeling, context, and snakes[J]. International Archives of Photogrammetry and Remote Sensing, 1997, XXXII(3-2W3):106-113. DOI:doi:http://dx.doi.org/ doi: doi:http://dx.doi.org/
[7]	Baumgartner A, Steger C, Mayer H, et al. Automatic road extraction based on multi-scale, grouping, and context[J]. Photogrammetric Engineering and Remote Sensing, 1999, 65(7):777-785.
[8]	Treash K, Amaratunga K. Automatic road detection in grayscale aerial images[J]. Journal of Computing in Civil Engineering, 2000,14( 1):60-69. DOI:10.1061/(asce)0887-3801(2000)14:1(60). doi: 10.1061/(asce)0887-3801(2000)14:1(60
[9]	Gaetano R, Zerubia J, Scarpa G, et al. Morphological road segmentation in urban areas from high resolution satellite images[C]// 17th International Conference on Digital Signal Processing(DSP). IEEE, 2011:1-8. DOI:10.1109/ICDSP.2011.6005015 doi: 10.1109/ICDSP.2011.6005015
[10]	韩洁, 郭擎, 李安. 结合非监督分类和几何—纹理—光谱特征的高分影像道路提取[J]. 中国图象图形学报, 2017, 22(12):1788-1797.
	[ Han J, Guo Q, Li A. Road extraction based on unsupervised classification and geometric-texture-spectral features for high-resolution remote sensing images[J]. Journal of Image and Graphics, 2017, 22(12):1788-1797. ] DOI:10.11834/jig.170222 doi: 10.11834/jig.170222
[11]	曹云刚, 王志盼, 杨磊. 高分辨率遥感影像道路提取方法研究进展[J]. 遥感技术与应用, 2017, 32(1):20-26.
	[ Cao Y G, Wang Z P, Yang L. Advances in method on road extraction from high resolution remote sensing images[J]. Remote Sensing Technology and Application, 2017, 32(1):20-26. ] DOI:10.11873/j.issn.1004-0323.2017.1.0020 doi: 10.11873/j.issn.1004-0323.2017.1.0020
[12]	Alshehhi R, Marpu P R. Hierarchical graph-based segmentation for extracting road networks from high-resolution satellite images[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2017,126 (APR.):245-260. DOI:10.1016/j.isprsjprs.2017.02.008 doi: 10.1016/j.isprsjprs.2017.02.008
[13]	Song M J, Civco D. Road extraction using SVM and image segmentation[J]. Photogrammetric Engineering and Remote Sensing, 2004, 70(12):1365-1371. DOI:10.14358/PERS.70.12.1365 doi: 10.14358/PERS.70.12.1365
[14]	Xiao L, Dai B, Liu D X, et al. CRF based road detection with multi-sensor fusion[C]// IEEE Intelligent Vehicles Symposium. IEEE, 2015:192-198. DOI:10.1109/IVS.2015.7225685 doi: 10.1109/IVS.2015.7225685
[15]	戴激光, 王杨, 杜阳, 等. 光学遥感影像道路提取的方法综述[J]. 遥感学报, 2020, 24(7):804-823.
	[ Dai J G, Wang Y, Du Y, et al. Development and prospect of road extraction method for optical remote sensing image[J]. Journal of Remote Sensing, 2020, 24(7):804-823. ] DOI:10.11834/jrs.20208360 doi: 10.11834/jrs.20208360
[16]	Guo M Q, Liu H, Xu Y Y, et al. Building extraction based on U-net with an attention block and multiple losses[J]. Remote Sensing, 2020, 12(9):1400. DOI:10.3390/rs12091400 doi: 10.3390/rs12091400
[17]	LeCun Y, Bengio Y, Hinton G. Deep learning[J]. Nature, 2015, 521(7553):436-444. DOI:10.1038/nature14539 doi: 10.1038/nature14539
[18]	LeCun Y, Boser B, Denker J S, et al. Backpropagation applied to handwritten zip code recognition[J]. Neural Computation, 1989, 1(4):541-551. DOI:10.1162/neco.1989.1.4.541 doi: 10.1162/neco.1989.1.4.541
[19]	Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation[C]. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015:3431-3440. DOI:10.1109/CVPR.2015.72 98965 doi: 10.1109/CVPR.2015.72 98965
[20]	He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition[C]// Proceedings of the IEEE conference on computer vision and pattern recognition, 2016:770-778. DOI: 10.1109/CVPR.2016.90 doi: 10.1109/CVPR.2016.90
[21]	Badrinarayanan V, Kendall A, Cipolla R. SegNet:A deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017: 39(12):2481-2495. DOI:10.1109/TPAMI.2016.2644615. doi: 10.1109/TPAMI.2016.2644615 pmid: 28060704
[22]	Chen L C, Papandreou G, Kokkinos I, et al. Semantic image segmentation with deep convolutional nets and fully connected CRFs[J]. Computer Science, 2014(4):357-361. DOI:10.1080/17476938708814211 doi: 10.1080/17476938708814211
[23]	Chen L C, Papandreou G, Schroff F, et al. Rethinking atrous convolution for semantic image segmentation[C]// Conference on Computer Vision and Pattern Recognition (CVPR). IEEE/CVF. 2017. DOI:10.48550/arXiv.1706.05587 doi: 10.48550/arXiv.1706.05587
[24]	Chen L C, Papandreou G, Kokkinos I, et al. DeepLab: Semantic image segmentation with deep convolutional nets,atrous convolution,and fully connected CRFs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(4):834-848. DOI:10.1109/TPAMI.2017.2699184 doi: 10.1109/TPAMI.2017.2699184
[25]	Chen L C, Zhu Y, Papandreou G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]// Proceedings of the European conference on computer vision (ECCV). 2018:801-818. DOI:10.1007/978-3-030-01234-2_49 doi: 10.1007/978-3-030-01234-2_49
[26]	Mendes C C T, Frémont V, Wolf D F. Exploiting fully convolutional neural networks for fast road detection[C]// IEEE International Conference on Robotics and Automation. IEEE, 2016:3174-3179. DOI:10.1109/ICRA.2016.7487486 doi: 10.1109/ICRA.2016.7487486
[27]	Cheng G L, Wang Y, Xu S B, et al. Automatic road detection and centerline extraction via cascaded end-to-end convolutional neural network[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(6):3322-3337. DOI:10.1109/TGRS.2017.2669341. doi: 10.1109/TGRS.2017.2669341
[28]	Almeida T, Lourenco B, Santos V. Road detection based on simultaneous deep learning approaches[J]. Robotics and Autonomous Systems, 2020, 133:103605. DOI:10.1016/j.robot.2020.103605 doi: 10.1016/j.robot.2020.103605
[29]	Chaurasia A, Culurciello E. Linknet: Exploiting encoder representations for efficient semantic segmentation[C]// 2017 IEEE Visual Communications and Image Processing (VCIP). IEEE, 2017:1-4. DOI:10.1109/VCIP.2017.8305148 doi: 10.1109/VCIP.2017.8305148
[30]	Zhou L C, Zhang C, Wu M. D-LinkNet: LinkNet with pretrained encoder and dilated convolution for high resolution satellite imagery road extraction[C]// IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). IEEE, 2018:192- 1924. DOI:10.1109/CVPRW.2018.00034 doi: 10.1109/CVPRW.2018.00034
[31]	宋廷强, 刘童心, 宗达, 等. 改进U-Net网络的遥感影像道路提取方法研究[J]. 计算机工程与应用, 2021, 57(14):209-216. doi: 10.3778/j.issn.1002-8331.2007-0392
	[ Song T Q, Liu T X, Zong D, et al. Research on road extraction method from remote sensing images based on improved U-net network[J]. Computer Engineering and Applications, 2021, 57(14):209-216. ] DOI:10.3778/j.issn.1002-8331.2007-0392 doi: 10.3778/j.issn.1002-8331.2007-0392
[32]	Mnih V, Heess N, Graves A, et al. Recurrent models of visual attention[J]. Advances in Neural Information Processing Systems, 2014, 3. DOI:10.48550/arXiv.1406.6247 doi: 10.48550/arXiv.1406.6247
[33]	刘航, 汪西莉. 基于注意力机制的遥感图像分割模型[J]. 激光与光电子学进展, 2020, 57(4):11.
	[ Liu H, Wang X L. Remote sensing image segmentation model based on attention mechanism[J]. Advances in laser and optoelectronics, 2020, 57(4):11. ] DOI:10.3788/LOP57.041015 doi: 10.3788/LOP57.041015
[34]	Woo S, Park J, Lee J Y, et al. CBAM: convolutional block attention module[C]// Proceedings of the European conference on computer vision (ECCV). 2018:3-19. DOI:10.1007/978-3-030-01234-2_1 doi: 10.1007/978-3-030-01234-2_1
[35]	Jie H, Li S, Gang S, et al. Squeeze- and- Excitation Networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 99. DOI:10.1109/TPAMI.2019.2913372 doi: 10.1109/TPAMI.2019.2913372
[36]	Mnih V. Machine learning for aerial image labeling.[D]. Toronto: University of Toronto (Canada). 2013.
[37]	Demir I, Koperski K, Lindenbaum D, et al. DeepGlobe 2018: A challenge to parse the earth through satellite images[C]. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2018:172-17209. DOI:10.1109/CVPRW.2018.00031 doi: 10.1109/CVPRW.2018.00031
[38]	Zhu Q Q, Zhang Y, Wang L, et al. A global context-aware and batch-independent network for road extraction from VHR satellite imagery[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2021, 175(12):353-365. DOI:10.1016/j.isprsjprs.2021.03.016 doi: 10.1016/j.isprsjprs.2021.03.016

数据集	年份	影像大小/像素	空间分辨率/m	影像数量/张
Massachusetts	2013	1500×1500	1	1171
DeepGlobe	2018	1024×1024	0.5	6226
CHN6-CUG	2021	512×512	0.5	4511

方法	Massachusetts			DeepGlobe			CHN6-CUG
方法	Pre	F1	mIoU	Pre	F1	mIoU	Pre	F1	mIoU
DARNet	78.81	77.92	80.43	98.56	67.88	73.91	81.82	80.37	81.25
DLinkNet	77.25	74.48	78.22	97.16	64.66	71.39	80.48	77.43	79.02
DeepLabv3+	77.89	72.79	77.09	97.07	64.42	71.24	76.82	72.38	75.30
FCN8s	72.30	68.80	57.96	96.14	65.17	71.68	73.97	70.32	73.81
SegNet	78.52	73.25	57.79	96.85	61.66	69.63	76.99	71.96	75.02
UNet	78.78	74.50	78.24	96.94	67.09	72.90	77.57	74.72	76.95

	Massachusetts			DeepGlobe			CHN6-CUG
Method	Pre	F1	mIoU	Pre	F1	mIoU	Pre	F1	mIoU
DARNet	78.81	77.92	80.43	98.56	67.88	73.91	81.82	80.37	81.25
DR-DLinkNet	77.42	74.68	78.36	97.33	65.03	71.54	80.57	77.70	79.22
DA-DLinkNet	77.89	74.66	78.57	96.93	65.32	71.78	80.82	77.58	79.14
DLinkNet	77.25	74.48	78.22	97.16	64.66	71.39	80.48	77.43	79.02

基于双注意力残差网络的高分遥感影像道路提取模型

Road Extraction Model of High-resolution Remote Sensing Images based on Dual-attention Residual Network

RichHTML

PDF (PC)

赞

可视化

摘要/Abstract

引用本文

使用本文

图/表 12

参考文献 38

相关文章 15

Metrics

本文评价

推荐阅读 0

[1]	左溪冰, 刘智, 金飞, 林雨准, 王淑香, 刘潇, 李美霖. 面向高光谱影像小样本分类的全局-局部特征自适应融合方法[J]. 地球信息科学学报, 2023, 25(8): 1699-1716.
[2]	林娜, 何静, 王斌, 唐菲菲, 周俊宇, 郭江. 结合植被光谱特征与Sep-UNet的城市植被信息智能提取方法[J]. 地球信息科学学报, 2023, 25(8): 1717-1729.
[3]	张春菊, 刘文聪, 张雪英, 叶鹏, 汪陈, 朱少楠, 张达玉. 基于本体的金矿知识图谱构建方法[J]. 地球信息科学学报, 2023, 25(7): 1269-1281.
[4]	张彤, 刘仁宇, 王培晓, 高楚林, 刘杰, 王望舒. 感知物理先验的机器学习及其在地理空间智能中的研究前景[J]. 地球信息科学学报, 2023, 25(7): 1297-1311.
[5]	令振飞, 刘涛, 杜萍, 赵丹, 陈朴一, 马天恩. 一种支持建筑群组相似模式检索的变分图卷积自编码模型[J]. 地球信息科学学报, 2023, 25(7): 1405-1417.
[6]	曹兴文, 吴孟泉, 郑雪婷, 郑宏伟, 李映祥, 张安安. 室内空间布局约束下的在线跟踪注册学习方法[J]. 地球信息科学学报, 2023, 25(7): 1418-1431.
[7]	柳林, 谢华芳, 岳瀚. 不同街景影像微环境提取方法对街面财产犯罪解释的差异分析[J]. 地球信息科学学报, 2023, 25(7): 1432-1447.
[8]	赵丹, 杜萍, 刘涛, 令振飞. 融合图自编码器与GRU的城市盗窃犯罪时空分布预测模型[J]. 地球信息科学学报, 2023, 25(7): 1448-1463.
[9]	金广垠, 沙恒宇, 张金雷, 黄金才. 基于路段-路口联合建模的对偶图卷积网络的行程时间估计方法[J]. 地球信息科学学报, 2023, 25(7): 1500-1513.
[10]	刘潇, 刘智, 林雨准, 王淑香, 左溪冰. 面向遥感影像场景分类的类中心知识蒸馏方法[J]. 地球信息科学学报, 2023, 25(5): 1050-1063.
[11]	侯慧太, 蓝朝桢, 徐青. 基于卫星影像全局和局部深度学习特征检索的无人机绝对定位方法[J]. 地球信息科学学报, 2023, 25(5): 1064-1074.
[12]	陈科, 管海燕, 雷相达, 曹爽. 基于特征增强核点卷积网络的多光谱LiDAR点云分类方法[J]. 地球信息科学学报, 2023, 25(5): 1075-1087.
[13]	张金雷, 陈奕洁, Panchamy Krishnakumari, 金广垠, 王骋程, 杨立兴. 基于注意力机制的城市轨道交通网络级多步短时客流时空综合预测模型[J]. 地球信息科学学报, 2023, 25(4): 698-713.
[14]	李心雨, 闫浩文, 王卓, 王炳瑄. 街景图像与机器学习相结合的道路环境安全感知评价与影响因素分析[J]. 地球信息科学学报, 2023, 25(4): 852-865.
[15]	衡雪彪, 许捍卫, 唐璐, 汤恒, 许怡蕾. 基于改进全卷积神经网络模型的土地覆盖分类方法研究[J]. 地球信息科学学报, 2023, 25(3): 495-509.