地球信息科学学报 ›› 2021, Vol. 23 ›› Issue (3): 514-523.doi: 10.12082/dqxxkx.2021.190805
朱盼盼1,2(), 李帅朋1,2, 张立强1,2,*(
), 李洋1,2
收稿日期:
2019-12-26
修回日期:
2020-04-23
出版日期:
2021-03-25
发布日期:
2021-05-25
通讯作者:
张立强
E-mail:zlyxbmsl@163.com;zhanglq@bnu.edu.cn
作者简介:
朱盼盼(1989- ),女,河南周口人,博士生,主要从高分辨率光学遥感影像信息提取研究。E-mail: zlyxbmsl@163.com
基金资助:
ZHU Panpan1,2(), LI Shuaipeng1,2, ZHANG Liqiang1,2,*(
), LI Yang1,2
Received:
2019-12-26
Revised:
2020-04-23
Online:
2021-03-25
Published:
2021-05-25
Contact:
ZHANG Liqiang
E-mail:zlyxbmsl@163.com;zhanglq@bnu.edu.cn
Supported by:
摘要:
建筑物的自动提取对城市发展与规划、防灾预警等意义重大。当前的建筑物提取研究取得了很好的成果,但现有研究多把建筑提取当成语义分割问题来处理,不能区分不同的建筑个体,且在提取精度方面仍然存在提升的空间。近年来,基于多任务学习的深度学习方法已在计算机视觉领域得到广泛应用,但其在高分辨率遥感影像自动解译任务上的应用还有待进一步发展。本研究借鉴经典的实例分割算法Mask R-CNN和语义分割算法U-Net的思想,设计了一种将语义分割模块植入实例分割框架的深度神经网络结构,利用多种任务之间的信息互补性来提升模型的泛化性能。自底向上的路径增强结构缩短了低层细节信息向上传递的路径。自适应的特征池化使得实例分割网络可以充分利用多尺度信息。在多任务训练模式下完成了对遥感影像中建筑物的自动分割,并在经典的遥感影像数据集SpaceNet上对该方法进行验证。结果表明,本文提出的基于多任务学习的建筑提取方法在巴黎数据集上建筑实例分割精度达到58.8%,在喀土穆数据集上建筑实例分割精度达到60.7%,相比Mask R-CNN和U-Net提升1%~2%。
朱盼盼, 李帅朋, 张立强, 李洋. 基于多任务学习的高分辨率遥感影像建筑提取[J]. 地球信息科学学报, 2021, 23(3): 514-523.DOI:10.12082/dqxxkx.2021.190805
ZHU Panpan, LI Shuaipeng, ZHANG Liqiang, LI Yang. Multitask Learning-based Building Extraction from High-Resolution Remote Sensing Images[J]. Journal of Geo-information Science, 2021, 23(3): 514-523.DOI:10.12082/dqxxkx.2021.190805
[1] | 王俊, 秦其明, 叶昕, 等. 高分辨率光学遥感图像建筑物提取研究进展[J]. 遥感技术与应用, 2016,31(4):653-662. |
[ Wang J, Qin Q M, Ye X, et al. A survey of building extraction methods from optical high resolution remote sensing imagery[J]. Remote Sensing Technology and Application, 2016,31(4):653-662. ] | |
[2] | 刘帆. 一种高分辨率遥感影像建筑物提取方法研究[J]. 中国新技术新产品, 2018,13(7):18-19. |
[ Liu F. A method of building extraction from high resolution remote sensing image[J]. New Technology & New Products of China, 2018,13(7):18-19. ] | |
[3] | 杨安妮, 许亚辉, 苏红军. 结合建筑指数的城市建筑用地提取与变化检测分析[J]. 测绘与空间地理信息, 2014,37(8):30-34. |
[ Yang A N, Xu Y H, Su H J. Urban built-up land extraction and change detection analysis using built -up indexes[J]. Geomatics & Spatial Information Technology, 2014,37(8):30-34. ] | |
[4] | Boonpook W, Tan Y M, Ye Y H, et al. A deep learning approach on building detection from unmanned aerial vehicle-based images in riverbank monitoring[J]. Sensors, 2018,18(11):3921. |
[5] | Gavankar N L, Ghosh S K. Automatic building footprint extraction from high-resolution satellite image using mathematical morphology[J]. European Journal of Remote Sensing, 2018,51(1):182-193. |
[6] | Huang X, Zhang L P. Morphological building/shadow index for building extraction from high-resolution imagery over urban areas[J]. IEEE Journal of Selected Topics in Applied Earth Observations & Remote Sensing, 2011,5:161-172. |
[7] | Jabri S, Zhang Y, Alaeldin S. Stereo-based building detection in very high resolution satellite imagery using IHS color system[J]. Geoscience and Remote Sensing Symposium (IGARSS), 2014: 2301-2304. |
[8] | Sirmacek B, Unsalan C. Urban-area and building detection using sift keypoints and graph theory[J]. IEEE Transactions on Geoscience & Remote Sensing, 2009,47(4):1156-1167. |
[9] | Tournaire O, Brédif M, Boldo D, et al. An efficient stochastic approach for building footprint extraction from digital elevation models[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2010,65(4):317-327. |
[10] | 李青, 李玉, 王玉, 等. 利用格式塔的高分辨率遥感影像建筑物提取[J]. 中国图象图形学, 2017,22(8):1162-1174. |
[ Li Q, Li Y, Wang Y, et al. Building extraction from high resolution remote sensing image by using Gestalt[J]. Journal of Imageand Graphics, 2017,22(8):1162-1174. ] | |
[11] | Yuan J Y.Automatic building extraction in aerial scenes using convolutional networks[J]. arXiv Preprint arXiv: 1602. 06564, 2016. |
[12] | Sohn G, Dowman I. Data fusion of high-resolution satellite imagery and LiDAR data for automatic building extraction[J]. ISPRS Journal of Photogram metry and Remote Sensing, 2007,62(1):43-63. |
[13] | Zhai W, Shen H F, Huang C L, et al. Fusion of polarimetric and texture information for urban building extraction from fully polarimetric SAR imagery[J]. Remote Sensing Letters, 2016,7(1):31-40. |
[14] | Maruyama Y, Tashiro A, Yamazaki F. Use of digital surface model constructed from digital aerial images to detect collapsed buildings during earthquake[J]. Procedia Engineering, 2011,14:552-558. |
[15] | Pan X R, Gao L R, Marinoni A, et al. Semantic labeling of high resolution aerial imagery and Lidar data with fine segmentation network[J]. Remote Sensing, 2018,10(5):743. |
[16] | Alshehhi R, Marpu P R, Woon W L, et al. Simultaneous extraction of roads and buildings in remote sensing imagery with convolutional neural networks[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2017,130:139-149. |
[17] | Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation [C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015: 3431-3440. |
[18] | Maggiori E, Tarabalka Y, Charpiat G, et al. High-resolution aerial image labeling with convolutional neural networks[J]. IEEE Transactions on Geoscience and Remote Sensing. 2017,55(12):7092-7103. |
[19] | Kaiser P, Wegner J D, Lucchi A, et al. Learning aerial image segmentation from online maps[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017,55(11):6054-6068. |
[20] | Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation [C]//International Conference on Medical Image Computing and Computer-assisted Intervention. Springer, Cham, 2015: 234-241. |
[21] | Wu G, Shao X, Guo Z, et al. Automatic building segmentation of aerial imagery using multi-constraint fully convolutional networks[J]. Remote Sensing, 2018,10(3):407. |
[22] | 惠健, 秦其明, 许伟, 等. 基于多任务学习的高分辨率遥感影像建筑实例分割[J]. 北京大学学报(自然科学版), 2019,55(6):1-12. |
[ Hui J, Qin Q M, Xu W, et al. Instance segmentation of buildings from high-resolution remote sensing images with multitask learning[J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2019,55(6):1-12. ] | |
[23] | Bischke B, Helber P, Folz J, et al. Multi-task learning for segmentation of building footprints with deep neural networks [C]//2019 IEEE International Conference on Image Processing (ICIP). IEEE, 2019: 1480-1484. |
[24] | Lounici K, Pontil M, Tsybakov A B, et al. Taking advantage of sparsity in multi-task learning[J]. arXiv preprint arXiv: 0903. 1468, 2009. |
[25] | Zhang Y, Yeung D Y. A convex formulation for learning task relationships in multi-task learning[J]. arXiv preprint arXiv: 1203. 3536, 2012. |
[26] | Mou L C, Zhu X X. Vehicle instance segmentation from aerial image and video using a multitask learning residual fully convolutional network[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018,56(11):6699-6711. |
[27] | He K M, Gkioxari G, Dollár P, et al. Mask r-cnn [C]//Proceedings of the IEEE International Conference on Computer Vision, 2017: 2961-2969. |
[28] | Ren S Q, He K M, Girshick R, et al. Faster r-cnn: Towards real-time object detection with region proposal networks [C]//Advances in Neural Information Processing Systems, 2015: 91-99. |
[29] | Liu S, Qi L, Qin H F, et al. Path aggregation network for instance segmentation [C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 8759-8768. |
[30] | Lin T Y, Dollár P, Girshick R, et al. Feature pyramid networks for object detection [C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 2117-2125. |
[31] | He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition [C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 770-778. |
[32] | Sergey I, Christian S. Batch normalization: Accelerating deep network training by reducing internal covariate shift [C]// International Conference on International Conference on Machine Learning. JMLR.org, 2015. |
[33] | Deng J, Dong W, Socher R, et al. Imagenet: A large-scale hierarchical image database [C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2009: 248-255. |
[34] | Glorot X, Bengio Y. Understanding the difficulty of training deep feedforward neural networks [C]//Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, 2010: 249-256. |
[35] | Lin T Y, Maire M, Belongie S, et al. Microsoft COCO: Common Objects in Context[J]. Lecture Notes in Computer Science, 2014: 740-755. |
[36] | Adam V E, Dave L, Bacastow T. Spacenet: A remote sensing dataset and challenge series. 2019. arXiv preprint arXiv:1807.01232. http://explore.digitalglobe.com/spacenet. |
[37] | Everingham M, Van G L, Williams C K I, et al. The pascal visual object classes (voc) challenge[J]. International Journal of Computer Vision, 2010,88(2):303-338. |
[1] | 郭紫甜, 王春梅, 刘欣, 庞国伟, 朱梦阳, 王晋卿. 基于小流域抽样单元的中国FROM-GLC30数据精度评价[J]. 地球信息科学学报, 2021, 23(3): 524-535. |
[2] | 陈昂, 杨秀春, 徐斌, 金云翔, 张文博, 郭剑, 邢晓语, 杨东. 基于面向对象与深度学习的榆树疏林识别方法研究[J]. 地球信息科学学报, 2020, 22(9): 1897-1909. |
[3] | 王学文, 赵庆展, 韩峰, 马永建, 龙翔, 江萍. 机载多光谱影像语义分割模型在农田防护林提取中的应用[J]. 地球信息科学学报, 2020, 22(8): 1702-1713. |
[4] | 赵青松, 兰安军, 范泽孟, 杨青. 贵州省不同地貌形态类型土壤侵蚀强度变化的定量分析[J]. 地球信息科学学报, 2020, 22(7): 1555-1566. |
[5] | 崔成, 任红艳, 赵璐, 庄大方. 基于街景影像多特征融合的广州市越秀区街道空间品质评估[J]. 地球信息科学学报, 2020, 22(6): 1330-1338. |
[6] | 毛文山, 赵红莉, 孙凤娇, 蒋云钟, 姜倩, 朱彦儒. 基于Item2Vec负采样优化的专题地图产品个性化推荐方法研究[J]. 地球信息科学学报, 2020, 22(11): 2128-2139. |
[7] | 何红术, 黄晓霞, 李红旮, 倪凌佳, 王新歌, 陈崇, 柳泽. 基于改进U-Net网络的高分遥感影像水体提取[J]. 地球信息科学学报, 2020, 22(10): 2010-2022. |
[8] | 郭子慧, 刘伟. 深度学习和遥感影像支持的矢量图斑地类解译真实性检查方法[J]. 地球信息科学学报, 2020, 22(10): 2051-2061. |
[9] | 宋关福, 卢浩, 王晨亮, 胡辰璞, 黄科佳. 人工智能GIS软件技术体系初探[J]. 地球信息科学学报, 2020, 22(1): 76-87. |
[10] | 蔡博文,王树根,王磊,邵振峰. 基于深度学习模型的城市高分辨率遥感影像 不透水面提取[J]. 地球信息科学学报, 2019, 21(9): 1420-1429. |
[11] | 王兴,康俊锋,刘学军,王美珍,张超. 设施农业典型地物改进Faster R-CNN识别方法[J]. 地球信息科学学报, 2019, 21(9): 1444-1454. |
[12] | 李泽宇,明冬萍,范莹琳,赵林峰,刘思民. 遥感影像监督分割评价指标比较与分析[J]. 地球信息科学学报, 2019, 21(8): 1265-1274. |
[13] | 蔡惠慧, 朱伟, 李孜轩, 刘园园, 李龙斌, 刘畅. 基于深度学习的钨钼找矿靶区预测方法研究[J]. 地球信息科学学报, 2019, 21(6): 928-936. |
[14] | 方颖, 李连发. 基于机器学习的高精度高分辨率气象因子时空估计[J]. 地球信息科学学报, 2019, 21(6): 799-813. |
[15] | 张刚, 刘文彬, 张男. 基于区域特征分割的密集匹配点云渐进形态学滤波[J]. 地球信息科学学报, 2019, 21(4): 615-622. |
|