融合边缘特征与语义信息的人工坑塘精准提取方法

doi:10.12082/dqxxkx.2022.210489

地球信息科学学报 ›› 2022, Vol. 24 ›› Issue (4): 766-779.doi: 10.12082/dqxxkx.2022.210489

融合边缘特征与语义信息的人工坑塘精准提取方法

杨先增¹(), 周亚男¹^,^*(), 张新², 李睿¹, 杨丹¹

1.河海大学水文水资源学院,南京 211100
2.中国科学院空天信息创新研究院遥感科学国家重点实验室,北京 100101

收稿日期:2021-08-20 修回日期:2021-09-30 出版日期:2022-04-25 发布日期:2022-04-13
通讯作者: ^*周亚男（1987— ）,男,河南漯河人,博士,副教授,主要从事基于深度学习的遥感信息提取、遥感时序分析与农业遥感。E-mail: zhouyn@hhu.edu.cn
^*周亚男（1987— ）,男,河南漯河人,博士,副教授,主要从事基于深度学习的遥感信息提取、遥感时序分析与农业遥感。E-mail: zhouyn@hhu.edu.cn
^*周亚男（1987— ）,男,河南漯河人,博士,副教授,主要从事基于深度学习的遥感信息提取、遥感时序分析与农业遥感。E-mail: zhouyn@hhu.edu.cn
作者简介:杨先增（1997— ）,男,安徽六安人,硕士生,主要从事深度学习遥感应用研究。E-mail: yangxz19970909@163.com
基金资助:
国家自然科学基金项目(42071316);中央高校业务费项目(B200202008);自然资源部地理国情监测重点实验室开放基金项目(2020NGCM03);农业产业数字化地图(21C00346)

Accurate Extraction of Artificial Pit-pond Integrating Edge Features and Semantic Information

YANG Xianzeng¹(), ZHOU Ya'nan¹^,^*(), ZHANG Xin², LI Rui¹, YANG Dan¹

1. College of Hydrology and Water Resources, Hohai University, Nanjing 211100, China
2. State Key Laboratory of Remote Sensing Science, Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100101, China

Received:2021-08-20 Revised:2021-09-30 Online:2022-04-25 Published:2022-04-13
Contact: ZHOU Ya'nan
Supported by:
National Natural Science Foundation of China(42071316);Fundamental Research Funds for the Central Universities(B200202008);Open Foundation of Key Laboratory of National Geographic Census and Monitoring, Ministry of Natural Resources(2020NGCM03);Chongqing agricultural industry digital map projec(21C00346)

摘要/Abstract

摘要：

针对高空间分辨率遥感影像目标提取中定位精度低、边缘粗糙等问题,提出一种融合目标边缘特征与语义信息的人工坑塘提取网络模型。方法首先利用改进的U-Net语义分割网络模块来提取遥感影像中丰富的目标语义信息,然后拓展上述语义分割网络构建边缘提取子网络来获取遥感影像的多尺度边缘特征,最后借助于编码-解码子网络融合边缘特征与语义信息,实现遥感影像目标的精准提取。将该方法运用到雷州半岛复杂背景条件下人工坑塘提取实验中,实验结果中本文提出的方法在F分数以及边界F分数等评价指标上表现最优,达到97.61%与83.01%,验证了融合高层语义信息结合低层的边缘特征在提升遥感目标提取精确度上的有效性。

关键词: 高分遥感, 深度学习, 语义分割, 边缘提取, 目标提取, 人工坑塘, 特征融合, 多任务学习

Abstract:

High-resolution remote sensing images have more detailed spatial, geometric, and textural features, which provides useful visual description features such as spot position, shape, and texture, and reliable and abundant data sources for accurate extraction of spatial elements. However, traditional methods require the researchers to extract these features manually and have some limitation such as low positioning accuracy and rough edges. With the development of deep learning, it can extract typical elements such as water bodies, buildings, and roads from remote sensing images with higher accuracy and without the support of prior knowledge. The extracted element information can provide a data basis for innovative applications in urban and rural land resource actuarial calculation and planning, disaster risk assessment, and industrial output evaluation and estimation. However, traditional deep learning semantic segmentation methods focus more on the improvement of semantic segmentation accuracy in the extraction process of remote sensing elements and pay less attention to boundary accuracy. In view of the existing problems of deep learning methods in target extraction from high resolution remote sensing images, such as rough edge and much noise, a network model combined with edge and semantic features of targets was proposed to extract the artificial pit-pond. The improved U-Net semantic segmentation network was used to extract rich semantic information of targets in remote sensing images, which could be developed in edge structure and sub-network extraction, thus acquiring multi-scale edge features in remote sensing image. In this case, an encoding-decoding subnetwork combined with edge features and semantic information were applied to extract remote sensing image objects accurately. Meanwhile, the synchronous extraction of boundary information was also realized, and feature fusion and noise screening were realized through the encoding-decoding subnetwork. The proposed method was used to extract artificial pit-pond in a complicated background condition in Leizhou Peninsula. First, we designed labeled training and testing images for the experiment and performed data augmentation to increase the number of samples. Second, we provided a series of evaluation indicators for the extraction effect. Finally, we evaluated the performance of the model from multiple perspectives including semantic accuracy and boundary. Results show that the method proposed in this paper had the best performance in the evaluation, the F score and boundary F score reached 97.61% and 83.01%, respectively, which demonstrated the effectiveness of the fusion of high-level semantic information and low-level edge features in improving the accuracy of remote sensing target extraction.

Key words: high-resolution remote sensing, deep learning, semantic segmentation, edge extraction, target extraction, artificial pit-pond, feature fusion, multi-task learning

杨先增, 周亚男, 张新, 李睿, 杨丹. 融合边缘特征与语义信息的人工坑塘精准提取方法[J]. 地球信息科学学报, 2022, 24(4): 766-779.DOI:10.12082/dqxxkx.2022.210489

YANG Xianzeng, ZHOU Ya'nan, ZHANG Xin, LI Rui, YANG Dan. Accurate Extraction of Artificial Pit-pond Integrating Edge Features and Semantic Information[J]. Journal of Geo-information Science, 2022, 24(4): 766-779.DOI:10.12082/dqxxkx.2022.210489

图/表 14

图1

图2

图3

图4

图5

表1

图6

图7

图8

图9

图10

表2

表3

图11

参考文献 33

[1]	齐永菊, 裴亮, 雷济升. 基于GF-1的坑塘信息精确提取方法研究[J]. 测绘与空间地理信息, 2017, 40(3):145-148.
	[ QI Y J, PEI L, LEI J S. Study on extraction of pond information accurately based on GF-1[J]. Geomatics & Spatial Information Technology, 2017, 40(3):145-148. ] DOI: 10.3969/j.issn.1672-5867.2017.03.042 doi: 10.3969/j.issn.1672-5867.2017.03.042
[2]	Julien B, Huber C, Studer M, Lei C, Kunpeng Y, Yésou H. Water resource monitoring exploiting sentinel-2 satellite and sentinel-2 satellite like time series; application in yangtze river water bodies[J]. Journal of Geodesy and Geoinformation Science, 2020, 3(4):41-49. DOI: 10.11947/j.JGGS.2020.0404 doi: 10.11947/j.JGGS.2020.0404
[3]	Ayehu G, Tadesse T, Gessesse B, et al. Combined use of sentinel-1 SAR and landsat sensors products for residual soil moisture retrieval over agricultural fields in the upper blue nile basin, ethiopia[J]. Sensors, 2020, 20(11):3282. DOI: 10.3390/s20113282 doi: 10.3390/s20113282
[4]	杜培军, 王欣, 蒙亚平, 等. 面向地理国情监测的变化检测与地表覆盖信息更新方法[J]. 地球信息科学学报, 2020, 22(4):857-866. doi: 10.12082/dqxxkx.2020.190747
	[ Du P J, Wang X, Meng Y P, et al. Effective change detection approaches for geographic national condition monitoring and land cover map updating[J]. Journal of Geo-information Science, 2020, 22(4):857-866. ] DOI: CNKI:SUN:DQXX.0.2020-04-022 doi: CNKI:SUN:DQXX.0.2020-04-022
[5]	Zhou Y N, Chen Y H, Feng L, et al. Supervised and adaptive feature weighting for object-based classification on satellite Images[J]. IEEE Journal of Selected Topics in Applied Earth Observations & Remote Sensing, 2018:1-11. DOI: 10.1109/JSTARS.2018.2851753 doi: 10.1109/JSTARS.2018.2851753
[6]	Zhang X, MING D P, Zhou W, et al. Cropland extraction based on OBIA and adaptive scale pre-estimation[J]. Photogrammetric Engineering & Remote Sensing, 2016, 82(8):635-644. DOI: 10.14358/PERS.82.8.635 doi: 10.14358/PERS.82.8.635
[7]	郭峰, 毛政元, 邹为彬, 等. 融合LiDAR数据与高分影像特征信息的建筑物提取方法[J]. 地球信息科学学报, 2020, 22(8):1654-1665.
	[ Guo F, Mao Z Y, Bing W B, et al. A method for building extraction by fusing feature information from LIDAR data and high-resolution imagery[J] Journal of Geo-information Science, 2020, 22(8):1654-1665. ] DOI: 10.12082/dqxxkx.2020.190459 doi: 10.12082/dqxxkx.2020.190459
[8]	曹云刚, 王志盼, 慎利, 等. 像元与对象特征融合的高分辨率遥感影像道路中心线提取[J]. 测绘学报, 2016, 45(10):1231-1240+1249
	[ Cao Y G, Wang Z P, Shen L, et al. Fusion of pixel-based and object-based features for road centerline extraction from high-resolution satellite imagery[J] Acta Geodaetica et Cartographica Sinica, 2016, 45(10):1231-1240+1249.] DOI: 10.11947/j.AGCS.2016.20160158 doi: 10.11947/j.AGCS.2016.20160158
[9]	王猛, 张新长, 王家耀, 等. 结合随机森林面向对象的森林资源分类[J]. 测绘学报, 2020, 49(2):235-244.
	[ Wang M, Zhang X C, Wang J Y, et al. Forest resource classification based on random forest and object oriented method[J]. Acta Geodaetica et Cartographica Sinica, 2020, 49(2):235-244. ] DOI: CNKI:SUN:CHXB.0.2020-02-011 doi: CNKI:SUN:CHXB.0.2020-02-011
[10]	陈生, 王宏, 沈占锋, 等. 面向对象的高分辨率遥感影像桥梁提取研究[J]. 中国图象图形学报, 2009, 14(4):585-590.
	[ Chen S, Wang H, Shen Z F, et al. Study on object-oriented extracting bridges from high resolution remote sensing image[J]. Journal of Image and Graphics, 2009, 14(4):585-590. ] DOI: 10.11834/jig.20090404 doi: 10.11834/jig.20090404
[11]	娄艺涵, 张力小, 潘骁骏, 等. 1984年以来8个时期杭州主城区西部湿地格局研究[J]. 湿地科学, 2021, 19(2):247-254.
	[ Lou Y H, Zhang L X, Pan X J, et al. Pattern of wetlands in the west of main city zone of hangzhou for 8 periods since 1984[J]. Wetland Science, 2021, 19(2):247-254. ] DOI: 10.13248/j.cnki.wetlandsci.2021.02.013 doi: 10.13248/j.cnki.wetlandsci.2021.02.013
[12]	张寅丹, 王苗苗, 陆海霞, 等. 基于监督与非监督分割评价方法提取高分辨率遥感影像特定目标地物的对比研究[J]. 地球信息科学学报, 2019, 21(9):1430-1443. doi: 10.12082/dqxxkx.2019.180628
	[ Zhang Y D, Wang M M, Lu H X, et al. Comparing supervised and unsupervised segmentation evaluation methods for extracting specific land cover from high-resolution remote sensing imagery[J]. 2019, 21(9):1430-1443. ] DOI: CNKI:SUN:DQXX.0.2019-09-014 doi: CNKI:SUN:DQXX.0.2019-09-014
[13]	刘扬, 付征叶, 郑逢斌. 高分辨率遥感影像目标分类与识别研究进展[J]. 地球信息科学学报, 2015, 17(9):1080-1091. doi: 10.3724/SP.J.1047.2015.01080
	[ Liu Y, Fu Z Y, Zhen F B. Review on high resolution remote sensing image classification and recognition[J]. Journal of Geo-information Science, 2015, 17(9):1080-1091. ] DOI: 10.3724/SP.J.1047.2015.01080 doi: 10.3724/SP.J.1047.2015.01080
[14]	Zhou L C, Zhang C, Wu M. D-linknet: Linknet with pretrained encoder and dilated convolution for high resolution satellite imagery road extraction[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Salt Lake City, UT, 2018. DOI: 10.1109/CVPRW.2018.00034 doi: 10.1109/CVPRW.2018.00034
[15]	He H, Wang S, Wang S C, et al. A road extraction method for remote sensing image based on encoder decoder network[J]. Journal of Geodesy and Geoinformation Science, 2020, 3(2):16-25. DOI: 10.11947/j.JGGS.2020.0202 doi: 10.11947/j.JGGS.2020.0202
[16]	Liu H, Luo J C, Huang B, et al. DE-Net: deep encoding network for building extraction from high-resolution remote sensing imagery[J]. Remote Sensing, 2019, 11(20):2380. DOI: 10.3390/rs11202380 doi: 10.3390/rs11202380
[17]	刘浩, 骆剑承, 黄波, 等. 基于特征压缩激活SE-Net网络的建筑物提取[J]. 地球信息科学学报, 2019, 21(11):1779-1789. doi: 10.12082/dqxxkx.2019.190285
	[ Liu H, Luo J C, Huang B, et al. Building extraction based on se-unet[J]. Journal of Geo-information Science, 2019, 21(11):1779-1789. ] DOI: CNKI:SUN:DQXX.0.2019-11-012 doi: CNKI:SUN:DQXX.0.2019-11-012
[18]	Cheng B, Liang C B, Liu Y M, et al. Research on a novel extraction method using deep learning based on GF-2 images for aquaculture areas[J]. International Journal of Remote Sensing, 2020, 41(9):3575-3591. DOI: 10.1080/01431161.2019.1706009 doi: 10.1080/01431161.2019.1706009
[19]	Li R R, Liu W J, Yang L, et al. Deepunet: A deep fully convolutional network for pixel-level sea-land segmentation[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2018, 11(11):3954-3962. DOI: 10.1109/JSTARS.2018.2833382 doi: 10.1109/JSTARS.2018.2833382
[20]	Liu Y, Cheng M M, Hu X W, et al. Richer convolutional features for edge detection[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Hunolulu, HI, 2017. DOI: 10.1109/TPAMI.2018.2878849 doi: 10.1109/TPAMI.2018.2878849
[21]	He J Z, Zhang S L, Yang M, et al. Bi-directional cascade network for perceptual edge detection[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR), Long Beach, USA, 2019. DOI: 10.1109/TPAMI.2020.3007074 doi: 10.1109/TPAMI.2020.3007074
[22]	李森, 彭玲, 胡媛, 等. 基于FD-RCF的高分辨率遥感影像耕地边缘检测[J]. 中国科学院大学学报, 2020, 37(4):483-489
	[ Li S, Peng L, Hu Y, et al. FD-RCF-based boundary delineation of agricultural fields in high resolution remote sensing images[J]. Journal of University of Chinese Academy of Sciences, 2020, 37(4):483-489. ] DOI: 10.7523/j.issn.2095-6134.2020.04.007 doi: 10.7523/j.issn.2095-6134.2020.04.007
[23]	Diakogiannis F I, Waldner F, Caccetta P, et al. Resunet-a: a deep learning framework for semantic segmentation of remotely sensed data[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 162:94-114. DOI: 10.1016/j.isprsjprs.2020.01.013 doi: 10.1016/j.isprsjprs.2020.01.013
[24]	Yuan K, Meng G F, Cheng D C, et al. Efficient cloud detection in remote sensing images using edge-aware segmentation network and easy-to-hard training strategy[C]. 2017 IEEE International Conference on Image Processing (ICIP), Beijing, China, 2017. DOI: 10.1109/ICIP.2017.8296243 doi: 10.1109/ICIP.2017.8296243
[25]	Cheng D C, Meng G F, Xiang S M, et al. Fusionnet: Edge aware deep convolutional networks for semantic segmentation of remote sensing harbor images[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2017, 10(12):5769-5783. DOI: 10.1109/JSTARS.2017.2747599 doi: 10.1109/JSTARS.2017.2747599
[26]	刘巍, 吴志峰, 骆剑承等. 深度学习支持下的丘陵山区耕地高分辨率遥感信息分区分层提取方法[J]. 测绘学报, 2021, 50(1):105-116.
	[ Liu W, Wu Z F, Luo J C, et al. A divided and stratified extraction method of high-resolution remote sensing information for cropland in hilly and mountainous areas based on deep learning[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(1):105-116. ] DOI: 10.11947/j.AGCS.2021.20190448. doi: 10.11947/j.AGCS.2021.20190448
[27]	Zuo T C, Feng J T, Chen X J. HF-FCN: Hierarchically fused fully convolutional network for robust building extraction[C]. Asian Conference on Computer Vision, Taipei, China, 2016. DOI: 10.1007/978-3-319-54181-5_19 doi: 10.1007/978-3-319-54181-5_19
[28]	Liu W, Dong J, Xiang K L, et al. A sub-pixel method for estimating planting fraction of paddy rice in Northeast China[J]. Remote Sensing of Environment, 2018, 205:305-314. DOI: 10.1016/j.rse.2017.12.001 doi: 10.1016/j.rse.2017.12.001
[29]	Takikawa T, Acuna D, Jampani V, et al. Gated-scnn: Gated shape cnns for semantic segmentation[C]. Proceedings of the IEEE/CVF International Conference on Computer Vision(ICCV), Seoul, Korea, 2019. DOI: 10.1109/ICCV.2019.00533 doi: 10.1109/ICCV.2019.00533
[30]	Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation[C]. International Conference on Medical Image Computing and Computer-assisted Intervention, Munich, Germany, 2015:234-241. DOI: 10.1007/978-3-319-24574-4_28 doi: 10.1007/978-3-319-24574-4_28
[31]	Li X Y, Sun X F, Meng Y X, et al. Dice Loss for Data-imbalanced NLP Tasks[J]. arXiv preprint arXiv:1911.02855, 2019
[32]	Chen L C, Zhu Y, Papandreou G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany, 2018. DOI: 10.1007/978-3-030-01234-2_49 doi: 10.1007/978-3-030-01234-2_49
[33]	He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Las Vegas, CA, 2016. DOI: 10.1016/j.patcog.2021.107817 doi: 10.1016/j.patcog.2021.107817

语义精度	边缘精度
语义精度	非松弛边界	松弛边界
正确率Precision(P)	边界正确率Boundary Precision (BP)	松弛边界正确率Relax Boundary Precision (RBP)
召回率Recall(R)	边界召回率Boundary Recall (BR)	松弛边界召回率Relax Boundary Recall (RBR)
F分数F-Score(F₁)	边界F分数Boundary F-Score (F_b)	松弛边界F分数Relax Boundary F-Score (RF_b)
交并比Intersection over Union (IoU)	-	-

网络模型	精确度(P)	召回率(R)	F分数(F₁)	交并比(IoU)
U-Net	0.9501	0.9516	0.9507	0.9334
DeepLabV3+	0.9632	0.9640	0.9635	0.9360
D-LinkNet	0.9725	0.9530	0.9626	0.9371
ES-Net*	0.9707	0.9652	0.9679	0.9379
ES-Net	0.9775	0.9748	0.9761	0.9534

网络模型	边界正确率		边界召回率		边界F分数
网络模型	BP	RBP	BR	RBR	F_b	RF_b
U-Net	0.8040	0.8440	0.8077	0.8477	0.8058	0.8459
DeepLabV3+	0.8079	0.8501	0.8011	0.8491	0.8044	0.8495
D-LinkNet	0.8071	0.8469	0.8051	0.8462	0.8061	0.8466
ES-Net^*	0.8170	0.8552	0.7978	0.8377	0.8073	0.8464
ES-Net	0.8300	0.8646	0.8301	0.8649	0.8301	0.8647

融合边缘特征与语义信息的人工坑塘精准提取方法

Accurate Extraction of Artificial Pit-pond Integrating Edge Features and Semantic Information

RichHTML

PDF (PC)

赞

可视化

摘要/Abstract

引用本文

使用本文

图/表 14

参考文献 33

相关文章 15

Metrics

本文评价

推荐阅读 0

[1]	刘潇, 刘智, 林雨准, 王淑香, 左溪冰. 面向遥感影像场景分类的类中心知识蒸馏方法[J]. 地球信息科学学报, 2023, 25(5): 1050-1063.
[2]	侯慧太, 蓝朝桢, 徐青. 基于卫星影像全局和局部深度学习特征检索的无人机绝对定位方法[J]. 地球信息科学学报, 2023, 25(5): 1064-1074.
[3]	张金雷, 陈奕洁, Panchamy Krishnakumari, 金广垠, 王骋程, 杨立兴. 基于注意力机制的城市轨道交通网络级多步短时客流时空综合预测模型[J]. 地球信息科学学报, 2023, 25(4): 698-713.
[4]	李心雨, 闫浩文, 王卓, 王炳瑄. 街景图像与机器学习相结合的道路环境安全感知评价与影响因素分析[J]. 地球信息科学学报, 2023, 25(4): 852-865.
[5]	衡雪彪, 许捍卫, 唐璐, 汤恒, 许怡蕾. 基于改进全卷积神经网络模型的土地覆盖分类方法研究[J]. 地球信息科学学报, 2023, 25(3): 495-509.
[6]	于方圆, 曹家玮, 李发源, 李思进. 顾及对象特征的地面式光伏电站提取及减碳效益评估[J]. 地球信息科学学报, 2023, 25(3): 529-545.
[7]	高建文, 管海燕, 彭代锋, 许正森, 康健, 季雅婷, 翟若雪. 基于局部-全局语义特征增强的遥感影像变化检测网络模型[J]. 地球信息科学学报, 2023, 25(3): 625-637.
[8]	江宝得, 许少芬, 王进, 王淼. 基于条件生成对抗网络的电子地图可视水印去除方法[J]. 地球信息科学学报, 2023, 25(2): 288-297.
[9]	谢谦, 陆明, 谢春山. 基于LBS和深度学习的旅游景区客流量的高时频预测[J]. 地球信息科学学报, 2023, 25(2): 298-310.
[10]	黄昕, 毛政元. 基于时空多图卷积网络的网约车乘客需求预测[J]. 地球信息科学学报, 2023, 25(2): 311-323.
[11]	饶子昱, 卢俊, 郭海涛, 余东行, 侯青峰. 利用视角转换的跨视角影像匹配方法[J]. 地球信息科学学报, 2023, 25(2): 368-379.
[12]	王龙号, 蓝朝桢, 姚富山, 侯慧太, 武蓓蓓. 多源遥感影像深度特征融合匹配算法[J]. 地球信息科学学报, 2023, 25(2): 380-395.
[13]	刘洋, 康健, 管海燕, 汪汉云. 基于双注意力残差网络的高分遥感影像道路提取模型[J]. 地球信息科学学报, 2023, 25(2): 396-408.
[14]	于明洋, 陈肖娴, 张文焯, 刘耀辉. 融合网格注意力阀门和特征金字塔结构的高分辨率遥感影像建筑物提取[J]. 地球信息科学学报, 2022, 24(9): 1785-1802.
[15]	赵祥, 王涛, 张艳, 郑迎辉, 张昆, 王龙辉. 基于改进DeepLabv3+孪生网络的遥感影像变化检测方法[J]. 地球信息科学学报, 2022, 24(8): 1604-1616.