地球信息科学学报 ›› 2021, Vol. 23 ›› Issue (10): 1823-1837.doi: 10.12082/dqxxkx.2021.210305
杨佳宾1(), 范大昭1,*(
), 杨幸彬2, 纪松1, 雷蓉1
收稿日期:
2021-05-31
修回日期:
2021-07-02
出版日期:
2021-10-25
发布日期:
2021-12-25
通讯作者:
* 范大昭(1973— ),男,河南开封人,博士后,教授,研究方向为数字摄影测量理论与应用。E-mail: fdzcehui@163.com作者简介:
杨佳宾(1995— ),男,河南郑州人,硕士,主要从事数字摄影测量与立体视觉三维重建方向研究。E-mail: yangbin187513@163.com
基金资助:
YANG Jiabin1(), FAN Dazhao1,*(
), YANG Xingbin2, JI Song1, LEI Rong1
Received:
2021-05-31
Revised:
2021-07-02
Online:
2021-10-25
Published:
2021-12-25
Supported by:
摘要:
针对传统方法和深度学习匹配方法在倾斜影像上获取匹配点少、复现率低以及精度不高等问题,本文提出一种面向倾斜摄影的深度学习航空影像匹配方法。首先,利用POS信息计算影像重叠区域,并对倾斜影像进行透视变换改正,减弱几何变形对匹配过程的影响;其次,在变换后的重叠区域影像上利用训练的多尺度特征点检测网络推理其对应的高斯热力图,在高斯热力图尺度空间检测极值点作为稳定特征点,基于自监督主方向检测网络获取特征点主方向;接着,在特征点描述阶段,结合网络学习得到的特征点位置和主方向获取尺度旋转不变GeoDesc基础描述子,并考虑图像的几何、视觉上下文信息对描述子进行增强处理;最后,通过双向比值提纯法获取初始匹配点,利用RANSAC和图约束方法剔除误匹配后获得最终匹配点结果。使用ISPRS提供的2组典型区域倾斜影像进行匹配实验,结果表明,相比于SIFT、ASIFT、SuperPoint、GeoDesc及ContextDesc等算法,本文方法能够在大视角变化和纹理信息贫乏的倾斜影像对上获取更多均匀分布的匹配点,同时复现率也要优于其他方法。
杨佳宾, 范大昭, 杨幸彬, 纪松, 雷蓉. 面向倾斜摄影的深度学习航空影像匹配方法[J]. 地球信息科学学报, 2021, 23(10): 1823-1837.DOI:10.12082/dqxxkx.2021.210305
YANG Jiabin, FAN Dazhao, YANG Xingbin, JI Song, LEI Rong. Deep Learning based on Image Matching Method for Oblique Photogrammetry[J]. Journal of Geo-information Science, 2021, 23(10): 1823-1837.DOI:10.12082/dqxxkx.2021.210305
表1
相机检校参数
相机 | 焦距/mm | x0/mm | y0/mm | 影像大小/pixel | Roll/(°) | Pitch/(°) | Yaw/(°) |
---|---|---|---|---|---|---|---|
163 | 50.193 | 50.193 | 18.345 | 6132×8176 | -0.110 | 0.119 | 0.276 |
148 | 81.938 | 81.938 | 24.186 | 8176×6132 | -0.243 | 45.134 | -0.035 |
147 | 82.045 | 82.045 | 24.335 | 8176×6132 | -0.506 | -44.944 | 0.692 |
159 | 81.860 | 81.860 | 24.348 | 8176×6132 | 44.926 | 0.210 | 0.009 |
145 | 82.037 | 82.037 | 24.419 | 8176×6132 | -45.198 | -0.025 | -0.085 |
[1] | 闫利, 费亮, 陈长海, 等. 利用网络图进行高分辨率航空多视影像密集匹配[J]. 测绘学报, 2016, 45(10):1171-1181. |
[ Yan L, Fei L, Chen C H, et al. A muti-view dense matching algorithm of high resolution aerial images based on graph network[J]. Acta Geodaetica et Cartographica Sinica, 2016, 45(10):1171-1181.] | |
[2] | 陈敏, 朱庆, 何海清, 等. 面向城区宽基线立体像对视角变化的结构自适应特征点匹配[J]. 测绘学报, 2019, 48(9):1129-1140. |
[ Chen M, Zhu Q, He H Q, et al. Structureadaptive feature point matching for urban area wide-baseline images with viewpoint variation[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(9):1129-1140. ] | |
[3] |
Lowe D G, Lowe D G. Distinctive Image Features from Scale-Invariant Keypoints[J]. International Journal of Computer Vision, 2004, 60(2):91-110.
doi: 10.1023/B:VISI.0000029664.99615.94 |
[4] | Bay H, Tuytelaars T, Gool L V. SURF: Speeded up robust features[C]. Proceedings of the 9th European conference on Computer Vision - Volume Part I, 2006. |
[5] | 张力, 艾海滨, 许彪, 等. 基于多视影像匹配模型的倾斜航空影像自动连接点提取及区域网平差方法[J]. 测绘学报, 2017, 46(5):554-564. |
[ Zhang L, Ai H B, Xu B A, et al. Automatic Tie-point extraction based on multiple-image matching and bundle adjustment of large block of oblique aerial images[J]. Acta Geodaetica et Cartographica Sinica, 2017, 46(5):554-564. ] | |
[6] | 赵霞, 朱庆, 肖雄武, 等. 基于同形变换的航空倾斜影像自动匹配方法[J]. 计算机应用, 2015, 35(6):1720-1725. |
[ Zhao X, Zhu Q, Xiao X W, et al. Automatic matching method for aviation oblique images based on homography transformation[J]. Journal of Computer Applications, 2015, 35(6):1720-1725. ] | |
[7] | Han X, Leung T, Jia Y, et al. MatchNet: Unifying feature and metric learning for patch-based matching[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015. |
[8] | Balntas V, Johns E, Tang L L, et al. PN-net: Conjoined triple deep network for learning local image descriptors[EB/OL]. 2016: arXiv: 1601.05030[cs.CV], https://arxiv.org/abs/1601.05030 |
[9] | Simo-Serra E, Trulls E, Ferraz L, et al. Discriminative learning of deep convolutional feature point descriptors[C]. IEEE International Conference on Computer Vision (ICCV), 2016. |
[10] | Tian Y, Fan B, Wu F. L2-Net: Deep learning of discriminative patch descriptor in euclidean space[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017. |
[11] | Luo Z, Shen T, Zhou L, et al. GeoDesc: Learning local descriptors by integrating geometry constraints[C]. European Conference on Computer Vision (ECCV), 2018. |
[12] | Luo Z, Shen T, Zhou L, et al. ContextDesc: Local descriptor augmentation with cross-modality context[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019. |
[13] | Yi K M, Trulls E, Lepetit V, et al. LIFT: Learned invariant feature transform[C]. European Conference on Computer Vision (ECCV), 2016. |
[14] | Verdie Y, Yi K M, Fua P, et al. TILDE: A temporally invariant learned DEtector[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015. |
[15] | Ono Y, Trulls E, Fua P, et al. LF-Net: Learning local features from images[J]. Advances in Neural Information Processing Systems, 2018:6237-6247. |
[16] | DeTone D, Malisiewicz T, Rabinovich A. Toward geometric deep SLAM[EB/OL]. 2017: arXiv: 1707.07410[cs.CV]. https://arxiv.org/abs/1707.07410 . |
[17] | DeTone D, Malisiewicz T. SuperPoint: self-supervised interest point detection and description[EB/OL]. 2017: arXiv: 1712.07629[cs.CV]. https://arxiv.org/abs/1712.07629 . |
[18] | Fischler M A. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography[J]. Readings in Computer Vision, 1987:726-740. |
[19] | Dániel Baráth, Noskova J, Matas J. MAGSAC: Marginalizing sample consensus[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019. |
[20] | Dusmanu M, Schönberger J L, Pollefeys M. Multi-view optimization of local feature geometry[EB/OL]. 2020: arXiv: 2003.08348[cs.CV]. https://arxiv.org/abs/2003.08348 |
[21] |
Ma J, Zhao J, Tian J, et al. Robust Point Matching via Vector Field Consensus[J]. IEEE Transaction on Image Process, 2014, 23(4):1706-1721.
doi: 10.1109/TIP.83 |
[22] |
Bian J W, Lin W Y, Liu Y, et al. GMS: Grid-based motion statistics for fast, ultra-robust feature correspondence[J]. International Journal of Computer Vision, 2020, 128(6):1580-1593.
doi: 10.1007/s11263-019-01280-3 |
[23] | Sun W W, Jiang W, Trulls E, et al. ACNe: attentive context normalization for robust permutation-equivariant learning[EB/OL]. 2019: arXiv: 1907.02545[cs.CV]. https://arxiv.org/abs/1907.02545 . |
[24] | Sarlin P E, DeTone D, Malisiewicz T, et al. SuperGlue: learning feature matching with graph neural networks[EB/OL]. 2019: arXiv: 1911.11763[cs.CV]. https://arxiv.org/abs/1911.11763 |
[25] | 龚健雅, 季顺平. 摄影测量与深度学习[J]. 测绘学报, 2018, 47(6):693-704. |
[ Gong J Y, Ji S P. Photogrammetry and deep learning[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(6):693-704. ] | |
[26] | Ling Y G, Wang K X, Shen S J. Probabilistic dense reconstruction from a moving camera[EB/OL]. 2019: arXiv: 1903.10673[cs.RO]. https://arxiv.org/abs/1903.10673 . |
[27] | 肖雄武, 李德仁, 郭丙轩, 等. 一种具有视点不变性的倾斜影像快速匹配方法[J]. 武汉大学学报·信息科学版, 2016, 41(9):1151-1159. |
[ Xiao X W, Li D R, Guo B X, et al. A robust and rapid viewpoint-invariant matching method for oblique images[J]. Geomatics and Information Science of Wuhan University, 2016, 41(9):1151-1159. ] | |
[28] | Dai J, Zhang J, Nguyen T. A robust and rapid viewpoint-invariant matching method for oblique images[J]. Geomatics and Information Science of Wuhan University, 2019. |
[29] | Shen T, Luo Z, Zhou L, et al. Matchable Image Retrieval by Learning from Surface Reconstruction[C]. The Asian Conference on Computer Vision (ACCV), 2018. |
[30] | He K, Zhang X, Ren S, et al. Deep Residual Learning for Image Recognition[C]. IEEE Conference on Computer Vision and Pattern Recognition, 2016. |
[31] | Nex F, Gerke M, Remondino F, et al. ISPRS Benchmark for Multi-Platform Photogrammetry[C]. Pia15+hrigi15-Joint Isprs Conference. ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences, 2015. |
[32] |
Morel J M, Yu G. ASIFT: A new framework for fully affine invariant image comparison[J]. SIAM Journal on Imaging Sciences, 2009, 2(2):438-469.
doi: 10.1137/080732730 |
[1] | 周欣昕, 吴艳兰, 李梦雅, 郑智腾. 基于特征分离机制的深度学习植被自动提取方法[J]. 地球信息科学学报, 2021, 23(9): 1675-1689. |
[2] | 李国清, 柏永青, 杨轩, 陈正超, 余海坤. 基于深度学习的高分辨率遥感影像土地覆盖自动分类方法[J]. 地球信息科学学报, 2021, 23(9): 1690-1704. |
[3] | 刘戈, 姜小光, 唐伯惠. 特征优选与卷积神经网络在农作物精细分类中的应用研究[J]. 地球信息科学学报, 2021, 23(6): 1071-1081. |
[4] | 许泽宇, 沈占锋, 李杨, 柯映明, 李硕, 王浩宇, 焦淑慧. 结合模糊度和形态学指数约束的深度学习建筑物提取[J]. 地球信息科学学报, 2021, 23(5): 918-927. |
[5] | 唐璎, 刘正军, 杨懿, 顾海燕, 杨树文. 基于特征增强和ELU的神经网络建筑物提取研究[J]. 地球信息科学学报, 2021, 23(4): 692-709. |
[6] | 许承权, 柳庆威. 基于线段检测算法的倾斜摄影直角像控点目标检测方法[J]. 地球信息科学学报, 2021, 23(3): 505-513. |
[7] | 朱盼盼, 李帅朋, 张立强, 李洋. 基于多任务学习的高分辨率遥感影像建筑提取[J]. 地球信息科学学报, 2021, 23(3): 514-523. |
[8] | 陈昂, 杨秀春, 徐斌, 金云翔, 张文博, 郭剑, 邢晓语, 杨东. 基于面向对象与深度学习的榆树疏林识别方法研究[J]. 地球信息科学学报, 2020, 22(9): 1897-1909. |
[9] | 崔成, 任红艳, 赵璐, 庄大方. 基于街景影像多特征融合的广州市越秀区街道空间品质评估[J]. 地球信息科学学报, 2020, 22(6): 1330-1338. |
[10] | 毛文山, 赵红莉, 孙凤娇, 蒋云钟, 姜倩, 朱彦儒. 基于Item2Vec负采样优化的专题地图产品个性化推荐方法研究[J]. 地球信息科学学报, 2020, 22(11): 2128-2139. |
[11] | 郭子慧, 刘伟. 深度学习和遥感影像支持的矢量图斑地类解译真实性检查方法[J]. 地球信息科学学报, 2020, 22(10): 2051-2061. |
[12] | 宋关福, 卢浩, 王晨亮, 胡辰璞, 黄科佳. 人工智能GIS软件技术体系初探[J]. 地球信息科学学报, 2020, 22(1): 76-87. |
[13] | 蔡博文,王树根,王磊,邵振峰. 基于深度学习模型的城市高分辨率遥感影像 不透水面提取[J]. 地球信息科学学报, 2019, 21(9): 1420-1429. |
[14] | 蔡惠慧, 朱伟, 李孜轩, 刘园园, 李龙斌, 刘畅. 基于深度学习的钨钼找矿靶区预测方法研究[J]. 地球信息科学学报, 2019, 21(6): 928-936. |
[15] | 方颖, 李连发. 基于机器学习的高精度高分辨率气象因子时空估计[J]. 地球信息科学学报, 2019, 21(6): 799-813. |
|