遥感科学与应用技术

联合混合CNN和相似性评估的异源影像无监督变化检测方法

  • 庄会富 , 1 ,
  • 郭建林 , 1, 2, * ,
  • 薛倩 1 ,
  • 张宇 1
展开
  • 1.中国矿业大学 自然资源部国土环境与灾害监测重点实验室,徐州 221110
  • 2.宁夏理工学院建筑与环境学院,石嘴山 753000
*郭建林(1997— ),男,宁夏固原人,硕士,助教,主要从事多源遥感影像智能化变化检测研究。 E-mail:

庄会富(1990— ),男,山东费县人,博士,讲师,硕士生导师,主要研究方向为遥感影像智能化处理及变化检测。E-mail:

Copy editor: 黄光玉 , 蒋树芳

收稿日期: 2024-09-26

  修回日期: 2024-12-27

  网络出版日期: 2025-01-24

基金资助

国家自然科学基金项目(42401539)

国家自然科学基金项目(U23A20598)

自然资源部国土卫星遥感应用重点实验室开放基金(KLSMNR-G202205)

Unsupervised Change Detection Method for Heterogeneous Image Combining Hybrid CNN and Similarity Assessment

  • ZHUANG Huifu , 1 ,
  • GUO Jianlin , 1, 2 ,
  • XUE Qian 1 ,
  • ZHANG Yu 1
Expand
  • 1. Key Laboratory of Resources and Environmental Information Engineering, China University of Mining and Technology, Xuzhou 221116, China
  • 2. School of Architectural Environment, Ningxia Institute of Science and Technology, Shizuishan 753000, China
*GUO Jianlin, E-mail:

Received date: 2024-09-26

  Revised date: 2024-12-27

  Online published: 2025-01-24

Supported by

National Natural Science Foundation of China(42401539)

National Natural Science Foundation of China(U23A20598)

Key Laboratory of Land Satellite Remote Sensing Application, Ministry of Natural Resources of the People's Republic of China(KLSMNR-G202205)

摘要

【目的】 异源影像变化检测是遥感领域的研究热点,现有研究通常先把异源影像通过域迁移方法得到伪同源影像,再从伪同源影像中提取变化信息,存在异源影像中地物特征的互补信息利用不充分问题。本文围绕域迁移得到的多时相异源影像对(每个时相都有一对异源影像)开展变化检测,为解决多时相异源影像对的时相-空间-频谱联合特征融合提取,并在无标注样本情况下进行无监督变化检测等极具挑战性的问题,提出了一种联合混合卷积网络和相似性评估的异源影像无监督变化检测方法。【方法】 首先利用本文提出的基于斑块相似性评估的伪标签样本数据集生成方法,克服深度学习变化检测模型对人工标注样本数据的依赖。然后,构建了融合异源影像的2维、3维混合卷积神经网络模型,用于提取变化信息。【结果】 该模型不仅可充分利用异源影像之间的特征互补优势,还可有效提取多时相异源影像对的时相-空间-频谱联合特征,可在伪标签样本数据驱动下实现无监督变化检测。为了验证本文提出方法的有效性,在基于循环一致生成对抗网络获取的多时相异源影像对数据上进行了实验,包括曙光和格洛斯特数据集,并与传统方法、深度学习方法和本文方法的消融方法进行了定性和定量比较。【结论】 本文方法通过样本数据的无监督选取,提高了异源影像变化检测的自动化程度;同时,与前述对比方法中最好的变化检测结果相比,本文方法在2个数据集上的平均F1-Score提高了2.54%,有效提高了变化检测结果的可靠性。

本文引用格式

庄会富 , 郭建林 , 薛倩 , 张宇 . 联合混合CNN和相似性评估的异源影像无监督变化检测方法[J]. 地球信息科学学报, 2025 , 27(2) : 507 -521 . DOI: 10.12082/dqxxkx.2025.240535

Abstract

[Objectives] Change detection in heterogeneous images is a research hotspot in the field of remote sensing. Existing studies typically obtain pseudo-homogeneous images from heterogeneous images through domain adaptation methods and then extract change information from these pseudo-homogeneous images, which often leads to insufficient utilization of complementary information from features in heterogeneous images. This study focuses on change detection using multi-temporal heterogeneous image pairs (each time having a pair of heterogeneous images) obtained through domain adaptation. To address the challenging issues including the temporal-spatial-spectral joint features fusion extraction from multi-temporal heterogeneous image pairs and perform unsupervised change detection without labeled samples. This study proposes an unsupervised change detection method for heterogeneous images that combines a hybrid convolutional network with similarity assessment. [Methods] To overcome the dependence of deep learning change detection models on manually labeled sample data, this method first utilizes a pseudo-labeled samples generation method based on patch similarity assessment proposed in this study. Then, a 2D-3D hybrid convolutional neural network model that integrates heterogeneous images is constructed to extract change information. [Results] This model can not only fully utilize the complementary advantages of features between heterogeneous images but also effectively extract the temporal-spatial-spectral joint features from multi-temporal heterogeneous image pairs, enabling unsupervised change detection driven by pseudo-labeled sample data. To validate the effectiveness of the proposed method, experiments were conducted on Shuguang and Gloucester datasets. Qualitative and quantitative comparisons were made with traditional methods, deep learning methods, and ablation methods derived from the proposed method. [Conclusions] The experimental results show that the proposed method enhances the automation level of change detection in heterogeneous images through unsupervised selection of sample data. Meanwhile, compared to the best change detection results among the aforementioned comparison methods, the average F1 Score of the proposed method is improved by 2.54% on both datasets, effectively enhancing the reliability of the change detection results.

[1]
Han W, Zhang X H, Wang Y, et al. A survey of machine learning and deep learning in remote sensing of geological environment: Challenges, advances, and opportunities[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2023, 202:87-113. DOI:10.1016/j.isprsjprs.2023.05.032

[2]
Liu S L, Chen L W, Zhang L, et al. A large-scale climate-aware satellite image dataset for domain adaptive land-cover semantic segmentation[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2023, 205:98-114. DOI:10.1016/j.isprsjprs.2023.09.007

[3]
Ji Y Y, Zhan W F, Du H L, et al. Urban-rural gradient in vegetation phenology changes of over 1500 cities across China jointly regulated by urbanization and climate change[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2023, 205:367-384. DOI:10.1016/j.isprsjprs.2023.10.015

[4]
Duan K K, Vrieling A, Schlund M, et al. Detection and attribution of cereal yield losses using Sentinel-2 and weather data: A case study in South Australia[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2024, 213:33-52. DOI:10.1016/j.isprsjprs.2024.05.021

[5]
Sarkar A, Chowdhury T, Murphy R R, et al. SAM-VQA: Supervised attention-based visual question answering model for post-disaster damage assessment on remote sensing imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61:4702716. DOI:10.1109/TGRS.2023.3276293

[6]
Wang X M, Chen W X, Yin J, et al. Risk assessment of flood disasters in the Poyang lake area[J]. International Journal of Disaster Risk Reduction, 2024, 100:104208. DOI:10.1016/j.ijdrr.2023.104208

[7]
Khelifi L, Mignotte M. Deep learning for change detection in remote sensing images: Comprehensive review and meta-analysis[J]. IEEE Access, 2020, 8:126385-126400. DOI:10.1109/ACCESS.2020.3008036

[8]
张祖勋, 姜慧伟, 庞世燕, 等. 多时相遥感影像的变化检测研究现状与展望[J]. 测绘学报, 2022, 51(7):1091-1107.

DOI

[ Zhang Z X, Jiang H W, Pang S Y, et al. Review and prospect in change detection of multi-temporal remote sensing images[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(7):1091-1107. ] DOI:10.11947/j.AGCS.2022.20220070

[9]
眭海刚, 冯文卿, 李文卓, 等. 多时相遥感影像变化检测方法综述[J]. 武汉大学学报(信息科学版), 2018, 43(12):1885-1898.

[ Sui H G, Feng W Q, Li W Z, et al. Review of change detection methods for multi-temporal remote sensing imagery[J]. Geomatics and Information Science of Wuhan University, 2018, 43(12):1885-1898. ] DOI:10.13203/j.whugis20180251

[10]
赵忠明, 高连如, 陈东, 等. 卫星遥感及图像处理平台发展[J]. 中国图象图形学报, 2019, 24(12): 2098-2110.

[ Zhao Z M, Gao L R, Chen D, et al. Development of satellite remote sensing and image processing platform[J]. Journal of Image and Graphics, 2019, 24(12):2098-2110. ] DOI:10.11834/jig.190450

[11]
Zhuang H F, Hao M, Deng K Z, et al. Change detection in SAR images via ratio-based Gaussian kernel and nonlocal theory[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60:5210215. DOI:10.1109/TGRS.2021.3083364

[12]
Zhang Z C, Vosselman G, Gerke M, et al. Change detection between multimodal remote sensing data using Siamese CNN[Z]. ArXiv, 2018. DOI:10.48550/arxiv.1807.09562

[13]
Orsomando F, Lombardo P, Zavagli M, et al. SAR and optical data fusion for change detection[C]// 2007 Urban Remote Sensing Joint Event. IEEE, 2007:1-9. DOI:10.1109/URS.2007.371770

[14]
Ebel P, Saha S, Zhu X X. Fusing multi-modal data for supervised change detection[J]. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2021,XLIII-B3-2021:243-249. DOI:10.5194/isprs-archives-XLIII-B3-2021-243-2021

[15]
Luppino L T, Hansen M A, Kampffmeyer M, et al. Code-aligned autoencoders for unsupervised change detection in multimodal remote sensing images[J]. IEEE Transactions on Neural Networks and Learning Systems, 2024, 35(1):60-72. DOI:10.1109/TNNLS.2022.3172183

[16]
Sun Y L, Lei L, Guan D D, et al. Iterative robust graph for unsupervised change detection of heterogeneous remote sensing images[J]. IEEE Transactions on Image Processing, 2021, 30:6277-6291. DOI:10.1109/TIP.2021.3093766

[17]
Niu X D, Gong M G, Zhan T, et al. A conditional adversarial network for change detection in heterogeneous images[J]. IEEE Geoscience and Remote Sensing Letters, 2019, 16(1):45-49. DOI:10.1109/LGRS.2018.2868704

[18]
Sun Y L, Lei L, Li X, et al. Nonlocal patch similarity based heterogeneous remote sensing change detection[J]. Pattern Recognition, 2021, 109:107598. DOI:10.1016/j.patcog.2020.107598

[19]
Sun Y L, Lei L, Li X, et al. Structure consistency-based graph for unsupervised change detection with homogeneous and heterogeneous remote sensing images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60:4700221. DOI:10.1109/TGRS.2021.3053571

[20]
Jiang X, Li G, Liu Y, et al. Change detection in heterogeneous optical and SAR remote sensing images via deep homogeneous feature fusion[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020, 13:1551-1566. DOI:10.1109/JSTARS.2020.2983993

[21]
韩特, 汤玉奇, 邹滨, 等. 基于分层极限学习机影像转换的多源影像变化检测方法[J]. 地球信息科学学报, 2022, 24(11):2212-2224.

DOI

[ Han T, Tang Y Q, Zou B, et al. Heterogeneous images change detection method based on hierarchical extreme learning machine image transformation[J]. Journal of Geo-information Science, 2022, 24(11):2212-2224. ] DOI:10.12082/dqxxkx.2022.220089

[22]
Yang X, Zhao J Y, Wei Z Y, et al. SAR-to-optical image translation based on improved CGAN[J]. Pattern Recognition, 2022, 121:108208. DOI:10.1016/j.patcog.2021.108208

[23]
Zhu J Y, Park T, Isola P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]// 2017 IEEE International Conference on Computer Vision (ICCV). IEEE, 2017:2242-2251. DOI:10.1109/ICCV.2017.244

[24]
Li X H, Du Z S, Huang Y Y, et al. A deep translation (GAN) based change detection network for optical and SAR remote sensing images[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2021, 179:14-34. DOI:10.1016/j.isprsjprs.2021.07.007

[25]
Liu Z G, Zhang Z W, Pan Q, et al. Unsupervised change detection from heterogeneous data based on image translation[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60:4403413. DOI:10.1109/TGRS.2021.3097717

[26]
Luppino L T, Kampffmeyer M, Bianchi F M, et al. Deep image translation with an affinity-based change prior for unsupervised multimodal change detection[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60:4700422. DOI:10.1109/TGRS.2021.3056196

[27]
Liu J, Gong M G, Qin K, et al. A deep convolutional coupling network for change detection based on heterogeneous optical and radar images[J]. IEEE Transactions on Neural Networks and Learning Systems, 2018, 29(3):545-559. DOI:10.1109/TNNLS.2016.2636227

PMID

[28]
Girod B. Psychovisual aspects of image processing: What's wrong with mean squared error?[C]// IEEE, 1991:2. DOI:10.1109/MDSP.1991.639240

[29]
Liu G H, Yang J Y. Content-based image retrieval using color difference histogram[J]. Pattern Recognition, 2013, 46(1):188-198. DOI:10.1016/j.patcog.2012.06.001

[30]
Fuhrmann D R, Baro J A, Cox Jr J R. Experimental evaluation of psychophysical distortion metrics for JPEG-encoded images[J]. Journal of Electronic Imaging, 1995, 4(4):397-406. DOI:10.1117/12.220346

[31]
Lv X D, Wang Z J. Reduced-reference image quality assessment based on perceptual image hashing[C]// 2009 16th IEEE International Conference on Image Processing (ICIP). IEEE, 2009:4361-4364. DOI: 10.1109/ICIP.2009.5413652.[LinkOut]

[32]
Wang Z, Bovik A C, Sheikh H R, et al. Image quality assessment: From error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4):600-612. DOI:10.1109/TIP.2003.819861

PMID

[33]
Girshick R, Donahue J, Darrell T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]// 2014 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2014:580-587. DOI:10.1109/CVPR.2014.81

[34]
Ren S Q, He K M, Girshick R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6):1137-1149. DOI:10.1109/TPAMI.2016.2577031

PMID

[35]
Caye Daudt R, Le Saux B, Boulch A. Fully convolutional Siamese networks for change detection[C]// 2018 25th IEEE International Conference on Image Processing (ICIP). IEEE, 2018:4063-4067. DOI: 10.1109/ICIP.2018.8451652.[LinkOut]

[36]
Khaireddin Y, Chen Z F. Facial emotion recognition: State of the art performance on FER2013[Z]. ArXiv, 2021. DOI:10.48550/arXiv.2105.03588

[37]
Qiao M J, He X H, Cheng X J, et al. Crop yield prediction from multi-spectral, multi-temporal remotely sensed imagery using recurrent 3D convolutional neural networks[J]. International Journal of Applied Earth Observation and Geoinformation, 2021, 102:102436. DOI:10.1016/j.jag.2021.102436

[38]
卢元兵, 李华朋, 张树清. 基于混合3D-2D CNN的多时相遥感农作物分类[J]. 农业工程学报, 2021, 37(13):142-151.

[ Lu Y B, Li H P, Zhang S Q. Multi-temporal remote sensing based crop classification using a hybrid 3D-2D CNN model[J]. Transactions of the Chinese Society of Agricultural Engineering, 2021, 37(13):142-151. ] DOI:10.11975/j.issn.1002-6819.2021.13.017

[39]
Xu L C, Lu C H, Zhou T, et al. A 3D-2DCNN-CA approach for enhanced classification of hickory tree species using UAV-based hyperspectral imaging[J]. Microchemical Journal, 2024, 199:109981. DOI:10.1016/j.microc.2024.109981

[40]
Roy S K, Krishna G, Dubey S R, et al. HybridSN: Exploring 3-D-2-D CNN feature hierarchy for hyperspectral image classification[J]. IEEE Geoscience and Remote Sensing Letters, 2020, 17(2):277-281. DOI:10.1109/LGRS.2019.2918719

[41]
Lyu Z Y, Huang H T, Gao L P, et al. Simple multiscale UNet for change detection with heterogeneous remote sensing images[J]. IEEE Geoscience and Remote Sensing Letters, 2022, 19:2504905. DOI:10.1109/LGRS.2022.3173300

文章导航

/