基于卷积神经网络的遥感影像建筑物提取方法综述
杨明旺(2000— ),男,河南新乡人,硕士生,主要从事遥感影像智能信息提取方面的研究。E-mail: ymw@stu.haut.edu.cn |
收稿日期: 2024-01-24
修回日期: 2024-03-05
网络出版日期: 2024-05-24
基金资助
国家自然科学基金项目(41901276)
国家自然科学基金项目(41901265)
河南省科技攻关项目(232102320348)
河南省科技攻关项目(232102321057)
河南工业大学自科创新基金(2021ZKCJ18)
河南工业大学青年骨干教师培育计划(21420198)
A Review of Convolutional Neural Networks Related Methods for Building Extraction from Remote Sensing Images
Received date: 2024-01-24
Revised date: 2024-03-05
Online published: 2024-05-24
Supported by
National Natural Science Foundation of China(41901276)
National Natural Science Foundation of China(41901265)
The Science and Technology Research Project of Henan Province(232102320348)
The Science and Technology Research Project of Henan Province(232102321057)
The Self-Science Innovation Fund of Henan University of Technology(2021ZKCJ18)
The Cultivation Programme for Young Backbone Teachers in Henan University of Technology(21420198)
建筑物提取作为遥感影像处理领域备受关注的研究方向之一,对于城市规划、灾害管理、智慧城市建设等方面具有重要意义。近年来,随着遥感技术的不断突破和深度学习算法的迅速发展,卷积神经网络凭借强大的特征提取能力成为从遥感影像中提取建筑物的新兴解决方案。本文对基于卷积神经网络的建筑物提取方法进行系统总结,并将相关文献的方法针对模型结构、多尺度特征差异性、边界信息缺失以及模型复杂度的优化策略进行归纳分析。随后,我们阐述了典型的建筑物数据集以及当前数据集存在的问题,并根据数据集上的实验结果对相关方法的精度及参数量进行详细分析,旨在帮助读者更好地理解各种方法的性能和适用范围。最后,立足于领域的研究现状,面向人工智能高质量发展的新时代,从Transformer与CNN的结合、深度学习与强化学习的结合、跨模态数据融合、无监督或半监督学习方法、基于大规模遥感模型的实时提取、建筑物实例分割和建筑物轮廓矢量提取等方面对建筑物提取的未来研究方向进行了展望。
杨明旺 , 赵丽科 , 叶林峰 , 蒋华伟 , 杨震 . 基于卷积神经网络的遥感影像建筑物提取方法综述[J]. 地球信息科学学报, 2024 , 26(6) : 1500 -1516 . DOI: 10.12082/dqxxkx.2024.240057
Building extraction is one of the important research directions that has attracted great attention in the field of remote sensing image processing. It refers to the process of accurately extracting building information such as the location and shape of buildings by analyzing and processing remote sensing images. This technology plays an irreplaceable and important role in urban planning, disaster management, map production, smart city construction, and other fields. In recent years, with the advancement of science and technology, especially the continuous evolution of earth observation technology and the rapid development of deep learning algorithms, Convolutional Neural Networks (CNNs) have become an emerging solution for extracting buildings from remote sensing images because of their powerful feature extraction capability. The aim of this paper is to provide a comprehensive and systematic overview and analysis of building extraction methods based on convolutional neural networks. We conduct a comprehensive literature review to summarize the building extraction methods from perspectives of model structure, multi-scale feature differences, lack of boundary information, and model complexity. This will help researchers to better understand the advantages and disadvantages of different methods and the applicable scenarios. In addition, several typical building datasets in this field are described in detail, as well as the potential issues associated with these datasets. Subsequently, by collecting experimental results of relevant algorithms on these typical datasets, a detailed discussion on the accuracy and parameter quantities of various methods is conducted, aiming to provide a comprehensive assessment of performance and applicability of these methods. Finally, based on the current research status of this field and looking forward to the new era of high-quality development in artificial intelligence, the future directions for building extraction are prospected. Specifically, this paper discusses the combination of Transformers and CNNs, the combination of deep learning and reinforcement learning, multi-modal data fusion, unsupervised or semi-supervised learning methods, real-time extraction based on large-scale remote sensing model, building instance segmentation, and building contour vector extraction. In conclusion, our review can provide some valuable references and inspirations for future related research, so as to promote the practical application and innovation of building extraction from remote sensing images. This will fulfill the demand for efficient and precise map information in remote sensing technology and other related fields, contributing to the sustainable and high-quality development of human society.
表2 马萨诸塞州数据集实验结果Tab. 2 Experimental results of the Massachusetts dataset (%) |
类别 | 方法 | 准确率 | 精确率 | 召回率 | F1-score | IoU |
---|---|---|---|---|---|---|
A | CFENet[44] | 96.26 | 82.77 | - | 83.04 | 74.86 |
SCGFA-Net[45] | - | 83.20 | 87.10 | 85.00 | 74.10 | |
B | SA-Net[52] | - | 86.78 | 82.70 | 84.69 | 73.45 |
BRRNet[54] | - | - | - | 85.36 | 74.46 | |
C3Net[55] | - | 82.45 | 83.27 | 82.86 | 70.74 | |
C | VAF-Net[59] | - | 82.10 | 82.50 | 82.30 | - |
BOMSC-Net[62] | 94.71 | 86.64 | 83.68 | 85.13 | 74.71 | |
D | IRU-Net[73] | 96.61 | 93.16 | 91.41 | 92.34 | - |
MSL-Net[74] | 93.60 | 81.90 | 84.10 | 83.00 | 70.90 |
表3 WHU数据集实验结果Tab. 3 Experimental results of WHU dataset (%) |
类别 | 方法 | 准确率 | 精确率 | 召回率 | F1-score | IoU |
---|---|---|---|---|---|---|
A | MPRSU-Net[41] | - | 95.65 | 95.11 | 95.38 | 91.17 |
MAP-Net[42] | - | 95.62 | 94.81 | 95.21 | 90.86 | |
SER-UNet[43] | - | 95.67 | 95.87 | 95.65 | 91.46 | |
CFENet[44] | 98.71 | 93.70 | - | 92.62 | 87.22 | |
SCGFA-Net[45] | 96.00 | 94.60 | 95.30 | 90.90 | ||
B | SU-Net[15] | - | 95.20 | 93.00 | - | 88.80 |
ResUNet+[50] | - | 96.13 | 95.14 | 95.63 | - | |
MAEU-CNN[51] | - | 95.73 | 95.45 | 95.58 | 91.54 | |
SA-Net[52] | - | 95.27 | 93.80 | 94.53 | 89.62 | |
AFP-Net[53] | 96.68 | 94.90 | - | - | 87.02 | |
C | BOMSC-Net[62] | 98.20 | 95.14 | 94.50 | 94.80 | 90.15 |
MMB-Net[63] | - | 95.75 | 94.98 | 95.37 | 91.14 | |
D | ESFNet[71] | - | - | - | - | 85.34 |
ARC-Net[72] | 97.50 | 96.40 | 95.10 | 95.70 | 91.80 | |
MSL-Net[74] | 98.90 | 95.10 | 94.80 | 95.00 | 90.40 |
表4 Inria数据集实验结果Tab. 4 Experimental results of Inria dataset (%) |
类别 | 方法 | 准确率 | 精确率 | 召回率 | F1-score | IoU |
---|---|---|---|---|---|---|
A | MPRSU-Net[41] | - | 88.63 | 88.29 | 88.46 | 79.31 |
SER-UNet[43] | - | 92.75 | 91.68 | 92.42 | 82.61 | |
B | SU-Net[15] | - | 84.30 | 84.90 | - | 73.30 |
ResUNet+[50] | - | 90.81 | 90.11 | 90.46 | - | |
C3Net[55] | - | 86.94 | 85.95 | 86.42 | 76.21 | |
C | ICT-Net[58] | - | 87.30 | 86.90 | 87.10 | 77.20 |
BOMSC-Net[62] | 95.31 | 87.93 | 87.58 | 87.75 | 78.18 | |
D | ARC-Net[72] | 92.50 | 89.60 | 86.80 | 87.50 | 77.90 |
MSL-Net[74] | 96.80 | 89.30 | 89.90 | 89.60 | 81.10 |
表5 模型复杂度对比结果Tab. 5 Comparison results of model complexity |
类别 | 方法 | 影像尺寸/(像元×像元) | FLOPs/G | Parameters/M | IoU/% |
---|---|---|---|---|---|
A | MPRSU-Net[41] | 512×512 | 81.60 | 13.80 | 91.17 |
MAP-Net[42] | 512×512 | 48.09 | 24.00 | 90.86 | |
SER-UNet[43] | 512×512 | 135.00 | 72.05 | 91.46 | |
B | SA-Net[52] | 256×256 | - | 7.13 | 89.62 |
AFP-Net[53] | 256×256 | 27.95 | 48.76 | 87.02 | |
BRRNet[54] | 256×256 | - | 17.30 | - | |
C | BOMSC-Net[62] | 256×256 | - | 129.32 | 90.15 |
MMB-Net[63] | 512×512 | - | 54.46 | 91.14 | |
D | ESFNet[71] | 512×512 | 2.51 | 0.18 | 85.34 |
IRU-Net[73] | 256×256 | 11.11 | 6.01 | - | |
MSL-Net[74] | 512×512 | - | 6.00 | 90.40 |
注:G和M分别代表计算机的计量单位,G=109,M=106。 |
表6 不同方法优缺点比较Tab. 6 Comparison of advantages and disadvantages of different methods |
[1] |
王振庆, 周艺, 王世新, 等. IEU-Net高分辨率遥感影像房屋建筑物提取[J]. 遥感学报, 2021, 25(11):2245-2254.
[
|
[2] |
吴炜, 骆剑承, 沈占锋, 等. 光谱和形状特征相结合的高分辨率遥感图像的建筑物提取方法[J]. 武汉大学学报(信息科学版), 2012, 37(7):800-805.
[
|
[3] |
吕凤华, 舒宁, 龚龑, 等. 利用多特征进行航空影像建筑物提取[J]. 武汉大学学报(信息科学版), 2017, 42(5):656-660.
[
|
[4] |
高贤君, 郑学东, 沈大江, 等. 城郊高分影像中利用阴影的建筑物自动提取[J]. 武汉大学学报(信息科学版), 2017, 42(10):1350-1357.
[
|
[5] |
|
[6] |
|
[7] |
邓睿哲. 基于边缘感知GCN-FCN的高分遥感影像建筑物提取方法[D]. 武汉: 中国地质大学, 2022.
[
|
[8] |
庞世燕, 郝京京, 邢立宁, 等. 融合域自适应和弱监督策略的遥感影像建筑物提取算法[J]. 系统工程与电子技术, 2023, 45(6):1616-1623.
[
|
[9] |
王钰涵. 基于深度学习的高分辨率遥感影像建筑物提取研究[D]. 长春: 吉林大学, 2020.
[
|
[10] |
|
[11] |
|
[12] |
|
[13] |
|
[14] |
|
[15] |
季顺平, 魏世清. 遥感影像建筑物提取的卷积神经元网络与开源数据集方法[J]. 测绘学报, 2019, 48(4):448-459.
[
|
[16] |
|
[17] |
|
[18] |
|
[19] |
吕少云, 李佳田, 阿晓荟, 等. Res_ASPP_UNet++:结合分离卷积与空洞金字塔的遥感影像建筑物提取网络[J]. 遥感学报, 2023, 27(2):502-519.
[
|
[20] |
|
[21] |
|
[22] |
江宝得, 黄威, 许少芬, 等. 融合分散自适应注意力机制的多尺度遥感影像建筑物实例细化提取[J]. 测绘学报, 2023, 52(9):1504-1514.
[
|
[23] |
|
[24] |
|
[25] |
李小祥, 黄亮, 朱娟娟, 等. 边缘增强的EDU-Net遥感影像建筑物提取[J]. 遥感信息, 2023, 38(2):134-141.
[
|
[26] |
宋佳, 徐慧窈, 高少华, 等. 轻量化卷积神经网络遥感影像建筑物提取模型[J]. 遥感技术与应用, 2023, 38(1):190-199.
[
|
[27] |
|
[28] |
|
[29] |
|
[30] |
|
[31] |
|
[32] |
|
[33] |
|
[34] |
|
[35] |
|
[36] |
|
[37] |
|
[38] |
|
[39] |
崔卫红, 熊宝玉, 张丽瑶. 多尺度全卷积神经网络建筑物提取[J]. 测绘学报, 2019, 48(5):597-608.
[
|
[40] |
|
[41] |
张玉鑫, 颜青松, 邓非. 高分辨率遥感影像建筑物提取多路径RSU网络法[J]. 测绘学报, 2022, 51(1):135-144.
[
|
[42] |
|
[43] |
胡明洪, 李佳田, 姚彦吉, 等. 结合多路径的高分辨率遥感影像建筑物提取SER-UNet算法[J]. 测绘学报, 2023, 52(5):808-817.
[
|
[44] |
|
[45] |
|
[46] |
|
[47] |
|
[48] |
|
[49] |
|
[50] |
罗松强, 李浩, 陈仁喜. 多尺度特征增强的ResUNet+遥感影像建筑物提取[J]. 激光与光电子学进展, 2022, 59(8):0828007.
[
|
[51] |
张华, 郑祥成, 郑南山, 等. 基于MAEU-CNN的高分辨率遥感影像建筑物提取[J]. 地球信息科学学报, 2022, 24(6):1189-1203.
[
|
[52] |
冯凡, 王双亭, 张津, 等. 基于尺度自适应全卷积网络的遥感影像建筑物提取[J]. 激光与光电子学进展, 2021, 58(24):459-470.
[
|
[53] |
于明洋, 陈肖娴, 张文焯, 等. 融合网格注意力阀门和特征金字塔结构的高分辨率遥感影像建筑物提取[J]. 地球信息科学学报, 2022, 24(9):1785-1802.
[
|
[54] |
|
[55] |
|
[56] |
|
[57] |
|
[58] |
罗壮, 李明, 张德朝. 结合边界约束网络和分水岭分割算法的建筑物提取[J]. 遥感学报, 2022, 26(7):1459-1468.
[
|
[59] |
赫晓慧, 邱芳冰, 程淅杰, 等. 基于边缘特征融合的高分影像建筑物目标检测[J]. 计算机科学, 2021, 48(9):140-145.
[
|
[60] |
高贤君, 冉树浩, 张广斌, 等. 基于多特征融合与对象边界联合约束网络的建筑物提取[J]. 武汉大学学报(信息科学版):1-19.
[
|
[61] |
|
[62] |
|
[63] |
|
[64] |
|
[65] |
|
[66] |
|
[67] |
何家峰, 陈宏伟, 骆德汉. 深度学习实时语义分割算法研究综述[J]. 计算机工程与应用, 2023, 59(8):13-27.
[
|
[68] |
|
[69] |
|
[70] |
|
[71] |
|
[72] |
|
[73] |
|
[74] |
|
[75] |
|
[76] |
吴开顺, 郑道远, 陈妍伶, 等. 中国典型城市建筑物实例数据集[J]. 中国科学数据, 2021, 6(1):182-190.
[
|
[77] |
刘耀辉, 杨新月, 李嘉禾, 等. 中国农村地区建筑物样本及标注无人机影像数据集[J]. 中国科学数据, 2022, 7(2):182-194.
[
|
[78] |
|
[79] |
|
[80] |
|
[81] |
|
[82] |
|
/
〈 |
|
〉 |