基于Dense SIFT特征的无人机影像快速拼接方法

doi:10.12082/dqxxkx.2019.190001

地球信息科学学报 ›› 2019, Vol. 21 ›› Issue (4): 588-599.doi: 10.12082/dqxxkx.2019.190001

基于Dense SIFT特征的无人机影像快速拼接方法

杨佳宾¹(), 姜永涛¹, 杨幸彬², 郭广猛¹

1. 南阳师范学院河南省自然灾害遥感监测重点实验室,南阳 473061
2. 北京建筑大学测绘与城市空间信息学院,北京 100044

收稿日期:2019-01-01 修回日期:2019-03-14 出版日期:2019-04-24 发布日期:2019-04-24
作者简介:
作者简介：杨佳宾（1995-）,男,河南郑州人,本科生,主要从事数字摄影测量与计算机视觉方向研究。 E-mail:yangbin187513@163.com
基金资助:
国家自然科学基金项目（41604009、41071280）;南阳师范学院青年项目（18060）

A Fast Mosaic Algorithm of UAV Images based on Dense SIFT Feature Matching

Jiabin YANG^1,^*(), Yongtao JIANG¹, Xingbin YANG², Guangmeng GUO¹

1. Key laboratory of Natural disaster and remote sensing of Henan province, Nanyang Normal University, Nanyang 473061, China
2. College of Surveying and Urban Spatial Information, Beijing University of Civil Engineering and Architecture, Beijing 100044, China

Received:2019-01-01 Revised:2019-03-14 Online:2019-04-24 Published:2019-04-24
Contact: Jiabin YANG
Supported by:
National Natural Science Foundation of China, No.41604009, 41071280;Nanyang Normal University Youth Project, No.18060

摘要/Abstract

摘要：

特征匹配是无人机影像拼接过程的关键步骤,针对传统的特征匹配方法在影像拼接过程中获取匹配点少、特征点分布不均匀、匹配耗时长等问题,本文提出一种基于Dense SIFT特征的无人机影像快速拼接算法。首先,利用影像POS信息构建连接矩阵以引导匹配过程;然后在降采样影像上进行影像分块,利用Dense SIFT算子获取初始匹配点,并采用两次NCC方法分别实现降采样影像和原始影像上匹配点的精化;最后,基于共线方程将影像投影至物方面上,完成影像的快速拼接。本文选取2组无人机影像进行拼接实验,将本文算法与SIFT和SURF匹配拼接方法进行对比,结果表明：在影像特征点匹配方面,本文方法获取匹配点数量是SIFT和SURF算法的5倍以上,且匹配点分布更加均匀;在影像拼接结果方面,本文方法不仅能够较快完成影像拼接,而且有效避免了拼接影像中的“重影”现象,保证了较好的拼接质量。

关键词: 影像拼接, Dense SIFT, 连接矩阵, NCC, 共线方程, 光束法平差, 无人机

Abstract:

The UAV (Unmanned Aerial Vehicle) photography is a new remote sensing system emerging in recent years. It plays an important role in the rapid emergency response of natural disasters. However, due to the large amount of UAV image data, the traditional method for image matching and mosaic is low accuracy and time-consuming. Feature matching is one of key steps in UAV image mosaic. Traditional matching algorithms have several problems, including less feature points, feature maldistribution, and time-consuming. To solve these problems, a fast image mosaic algorithm based on Dense SIFT feature matching is proposed. Firstly, the connection matrix is build based on POS (Position and Orientation System) data to conduct the matching process. The UAV images are then down-sampled. Secondly, image segmentation is performed on the down-sampled images. Then the Dense SIFT operator is used in overlap area of down-sampled images to obtain the initial matching points which are eliminated through matching by the RANSAC (Random Sample Consensus) algorithm and refined by the NCC algorithm on the original and down-sampled images, respectively. Finally, processed images are projected to the object coordinate system based on collinear equation which is calculated by the bundle adjustment method. By contrast, the SIFT (Scale-Invariant Feature Transform) and SURF (Speeded Up Robust Feature) algorithms and the Pix4Dmapper Photogrammetry software are used to test the quality and efficiency of the Dense SIFT algorithm. Two groups of UAV images mosaic experiment results indicate: (1) The Dense SIFT algorithm can be used to obtain about five times more evenly distributed matching points than the SIFT and SURF algorithm at the same time; (2) The Dense SIFT algorithm can be used to effectively improve the quality of the images mosaic by removing the phenomenon of ghosting; (3) It takes about half the time of Pix4D mapper software to complete the same image mosaic test using the Dense SIFT algorithm. This indicates that the presented algorithm has a high image mosaic quality and fast processing speed, which can play an important role in the rapid emergency response of natural disasters.

Key words: UAV image mosaic, Dense SIFT feature matching, connection matrix, NCC algorithm, collinear equation, bundle adjustment

杨佳宾, 姜永涛, 杨幸彬, 郭广猛. 基于Dense SIFT特征的无人机影像快速拼接方法[J]. 地球信息科学学报, 2019, 21(4): 588-599.DOI:10.12082/dqxxkx.2019.190001

Jiabin YANG, Yongtao JIANG, Xingbin YANG, Guangmeng GUO. A Fast Mosaic Algorithm of UAV Images based on Dense SIFT Feature Matching[J]. Journal of Geo-information Science, 2019, 21(4): 588-599.DOI:10.12082/dqxxkx.2019.190001

图/表 20

图1

图2

图3

图4

图5

图6

图7

图8

表1

图9

表2

表3

表4

像点平均反投影误差结果

	加密点数量/个	$σ ? x$ /像元	$σ ? y$ /像元	$σ ?$ /像元
实验区1	35 719	0.6215	0.5773	0.8482
实验区2	151 351	0.5935	0.5517	0.8103

表4

图10

图11

图12

图13

图14

图15

表5

参考文献 29

[1]	崔红霞,孙杰,林宗坚.无人机遥感设备的自动化控制系统[J].测绘科学,2004(1):47-49,6. doi: 10.3771/j.issn.1009-2307.2004.01.016
	[Cui H X, Sun J, Lin Z J.The auto-control system of remote sensing devices for unmanned air vehicle[J]. Science of Surveying and Mapping, 2004(1):47-49,6. ] doi: 10.3771/j.issn.1009-2307.2004.01.016
[2]	Lowe D G.Distinctive image features from scale-invariant keypoints[C]// International [J].urnal of Computer Vision, 2004:91-110.
[3]	倪希亮,丁琳,江涛,等.应用不变特征获取均匀控制点的遥感影像配准算法[J].测绘科学,2011,36(2):70-72.
	[Ni X L, Ding L, Jiang T, et al.A remote sensing image registration algorithm by obtaining uniform control points based on invariant feature[J]. Science of Surveying and Mapping, 2011,36(2):70-72. ]
[4]	Moussa A, El-Sheimy N. A fast approach for stitching of aerial images[J].International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, 2016,XLI-B3:769-774.
[5]	刘如飞,卢秀山,刘冰,等.一种改进的无人机航摄影像快速拼接方法[J].测绘通报,2014(2):46-49. doi: 10.13474/j.cnki.11-2246.2014.0051
	[Liu R F, Lu X S, Liu B et al. An improved UAV aerial image quick mosaic method[J]. Bulletin of Surveying and Mapping, 2014(2):46-49. ] doi: 10.13474/j.cnki.11-2246.2014.0051
[6]	Mehradad S, Satari M, Safdary M, et al. Toward real time UAVS' image mosaicking[J]. International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, 2016, XLI-B1:941-946.
[7]	董梅,苏建东,杨举田,等.大区域无人机影像快速无缝拼接方法[J].测绘科学,2014,39(11):129-132.
	[Dong M, Su J D, Yang J T, et al. A fast seamless image mosaic method for UAV images in large areas[J]. Science of Surveying and Mapping, 2014,39(11):129-132. ]
[8]	Bay H, Tuytelaars T, Van G L.Surf: Speeded up robust features[C]//European conference on computer vision. Springer, Berlin, Heidelberg, 2006:404-417.
[9]	胡同喜,牛雪峰,谭洋,等.基于SURF算法的无人机遥感影像拼接技术[J].测绘通报,2015(1):55-58,74.
	[Hu T X, Niu X F, Tan Y, et al.Unmanned aerial vehicle images mosaic based on SURF algorithm[J]. Bulletin of Surveying and Mapping, 2015(1):55-58,74. ]
[10]	王勃,龚志辉,顾广杰,等.一种快速的无人机影像无缝拼接方法[J].测绘科学,2012,37(5):23-26.
	[Wang B, Gong Z H, Gu G J, et al.A fast method for image registration and seamless mosaicking of UAV[J]. Science of Surveying and Mapping, 2012,37(5):23-26. ]
[11]	Rublee E, Rabaud V, Konolige K, et al.ORB: An efficient alternative to SIFT or SURF[C]//Computer Vision (ICCV),2011 IEEE international conference on. IEEE, 2011:2564-2571.
[12]	佘建国,徐仁桐,陈宁.基于ORB和改进RANSAC算法的图像拼接技术[J].江苏科技大学学报(自然科学版),2015,29(2):164-169.
	[She J G, Xu R T, ChenN. Image stitching technology based on ORB and improved RANSAC algorithm[J]. Journal of [J].angsu University of Science and Technology (Natural Science Edition), 2015,29(2):164-169. ]
[13]	Mur-Artal R, Tardos J D.ORB-SLAM2: An open-source SLAM system for monocular, stereo, and RGB-D cameras[J]. IEEE Transactions on Robotics, 2017:1-8.
[14]	龚健雅,季顺平.摄影测量与深度学习[J].测绘学报,2018,47(6):693-704.
	[Gong J Y, Ji S P.Photogrammetry and deep Learning[J]. Acta Geodaetica et Cartographica Sinica, 2018,47(6):693-704. ]
[15]	袁修孝,明洋.大重叠度面阵航空数码影像的自动转点[J].武汉大学学报·信息科学版,2009,34(5):505-508.
	[Yuan X X, Ming Y.Automatic extraction of image tie points for aerial digital frame imagery with large overlap[J]. Geomatics and Information Science of Wuhan University, 2009,34(5):505-508. ]
[16]	Li M, Li D, Fan D. A study on automatic UAV image mosaic method for paroxysmal disaster[J].ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2012, XXXIX-B6:123-128. doi: 10.5194/isprsarchives-XXXIX-B6-123-2012
[17]	Bosch A, Zisserman A, Munoz X.Image classification using random forests and ferns[C]//Computer Vision, 2007. ICCV 2007. IEEE 11th International Conference on. IEEE, 2007:1-8.
[18]	杨幸彬,吕京国,张丹璐,等. Dense SIFT与改进最小二乘匹配结合的倾斜航空影像匹配方法[J].测绘通报,2018(10):32-36,70.
	[Yang X B, Lu J G, Zhang D L, et al.Oblique aerial image matching method based on Dense SIFT and improved least squares matching algorithm[J]. Bulletin of Surveying and Mapping, 2018(10):32-36,70. ]
[19]	宫阿都,何孝莹,雷添杰,等.无控制点数据的无人机影像快速处理[J].地球信息科学学报,2010,12(2):2254-2260.
	[Gong A D, He X Y, Lei T J, et al.Fast image processing method of UAV without control data[J]. Journal of Geo-Information Science, 2010,12(2):2254-2260. ]
[20]	Fischler M A, Bolles R C.Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography[J]. Communications of the ACM, 1981,24(6):381-395.
[21]	姜文聪,张继贤,程春泉,等. SIFT与粗差剔除算法相结合的机载SAR影像匹配研究[J].地球信息科学学报,2013,15(3):440-445. doi: 10.3724/SP.J.1047.2013.00440
	[Jiang W C, Zhang J X, Cheng C Q, et al.Matching of airborne SAR images based on a combination of SIFT algorithm with mismatching points eliminated algorithm[J]. Journal of Geo-information Science, 2013,15(3):440-445. ] doi: 10.3724/SP.J.1047.2013.00440
[22]	Li G.Stereo Matching using normalized cross-correlation in logRGB space[C]// International Conference on Computer Vision in Remote Sensing. IEEE, 2013.
[23]	Toni Schenk.数字摄影测量学[M].武汉:武汉大学出版社,2009.
	[Toni Schenk.Digital photogrammetry[M]. Wuhan: Wuhan University Press, 2009. ]
[24]	王佩军,徐亚明.摄影测量学[M].武汉:武汉大学出版社,2005.
	[Wang P J, Xu Y M.Photogrammetry[M]. Wuhan: Wuhan University Press, 2005. ]
[25]	赵健,郭忠磊,滕惠忠,等.基于LM方法的单位四元数光束法平差模型解算[J].海洋测绘, 2017,37(5):56-59.
	[Zhao J, Guo Z L, Teng H Z, et al. A solution to unit quaternion bundle adjustment model based on LM method[J].Hydrographic Surveying and Charting, 2017,37(5):56-59. ]
[26]	袁修孝. POS辅助光束法区域网平差[J].测绘学报,2008,37(3):342-348.
	[Yuan X X.POS-supported bundle block adjustment[J]. Acta Geodaetica et Cartographica Sinica, 2008,37(3):342-348. ]
[27]	袁修孝,段梦梦,曹金山.正射影像镶嵌线自动搜索的视差图算法[J].测绘学报,2015,44(8):877-883. doi: 10.11947/j.AGCS.2015.20140421
	[Yuan X X, Duan M M, Cao J S.A seam line detection algorithm for orthophoto mosaicking based on disparity image[J]. Acta Geodaetica et Cartographica Sinica, 2015,44(8):877-883. ] doi: 10.11947/j.AGCS.2015.20140421
[28]	周清华,潘俊,李德仁.遥感图像镶嵌接缝线自动生成方法综述[J].国土资源遥感,2013,25(2):1-7.
	[Zhou Q H, Pan J, Li D R.Overview of automatic generation of mosaicking seamlines for remote sensing images[J]. Remote Sensing for Land & Resources, 2013,25(2):1-7. ]
[29]	王竞雪,崔昊.局部点、线仿射不变性约束的近景影像直线段匹配[J].地球信息科学学报,2019,21(2):137-146.
	[Wang J X, Cui H.Line segment matching based on local point-line affine invariance constraints for close-range image[J]. Journal of Geo-information Science, 2019,21(2):137-146. ]

	影像分辨率/像元	焦距/mm	GSD/cm	影像数	相对航高/m	重叠度/%
	影像分辨率/像元	焦距/mm	GSD/cm	影像数	相对航高/m	航向	旁向
实验区1	4896×3672	33	13.0	17	449.0	75	60
实验区2	4896×3672	33	12.0	69	423.7	75	60

算法类别	左影像特征点/个	右影像特征点/个	初始匹配点数量/个	RANSAC剔除粗差后数量/个	最终匹配点数/个
SIFT算法	1742	1764	421	390	387
SURF算法	1749	1 825	199	195	195
本文匹配算法	7275	7275	2774	2568	2055

算法类别	降采样影像特征提取用时/s	降采样影像特征匹配用时/s	NCC精化用时/s	总时间/s
SIFT算法	0.839	0.349	0.257	1.445
SURF算法	0.259	0.351	0.131	0.741
本文匹配算法	0.131	0.201	1.127	1.459

	影像数量	影像匹配总次数/次	影像匹配用时/s	光束法平差用时/s	影像纠正用时/s	影像拼接总用时/s
实验区1	17	92	103.356	13.900	402.100	522.758
实验区2	69	376	421.861	20.100	1521.200	2152.360

基于Dense SIFT特征的无人机影像快速拼接方法

A Fast Mosaic Algorithm of UAV Images based on Dense SIFT Feature Matching

RichHTML

PDF (PC)

赞

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 20

参考文献 29

相关文章 15

Metrics

本文评价

推荐阅读 0

[1]	侯慧太, 蓝朝桢, 徐青. 基于卫星影像全局和局部深度学习特征检索的无人机绝对定位方法[J]. 地球信息科学学报, 2023, 25(5): 1064-1074.
[2]	蒯宇, 王彪, 吴艳兰, 陈搏涛, 陈兴迪, 薛维宝. 基于多尺度特征感知网络的城市植被无人机遥感分类[J]. 地球信息科学学报, 2022, 24(5): 962-980.
[3]	郝明, 林惠晶, 高彦彦. 基于改进主动轮廓模型的无人机影像矿区地裂缝提取[J]. 地球信息科学学报, 2022, 24(12): 2448-2457.
[4]	刘树超, 邵全琴, 杨帆, 郭兴健, 王东亮, 黄海波, 汪阳春, 刘纪远, 樊江文, 李愈哲. 黄河源区放牧家畜数量及空间分布无人机遥感调查[J]. 地球信息科学学报, 2021, 23(7): 1286-1295.
[5]	李芹, 刘旭林, 李荣昊, 冯朝晖, 李子锦, 赵红颖. 基于野外台站无人机组网遥感仿真航迹规划[J]. 地球信息科学学报, 2021, 23(5): 948-957.
[6]	朱婉雪, 孙志刚, 李彬彬, 杨婷, 刘振, 彭金榜, 朱康莹, 李仕冀, 娄金勇, 侯瑞星, 李静, 于武江, 王永利, 张峰, 刘向冶, 胡华浪, 欧阳竹. 基于无人机遥感的滨海盐碱地土壤空间异质性分析与作物光谱指数响应胁迫诊断[J]. 地球信息科学学报, 2021, 23(3): 536-549.
[7]	刘见礼, 廖小罕, 倪文俭, 王勇, 叶虎平, 岳焕印. 顾及单木三维形态的无人机立体影像单木识别算法[J]. 地球信息科学学报, 2021, 23(10): 1861-1872.
[8]	陈昂, 杨秀春, 徐斌, 金云翔, 张文博, 郭剑, 邢晓语, 杨东. 基于面向对象与深度学习的榆树疏林识别方法研究[J]. 地球信息科学学报, 2020, 22(9): 1897-1909.
[9]	王学文, 赵庆展, 韩峰, 马永建, 龙翔, 江萍. 机载多光谱影像语义分割模型在农田防护林提取中的应用[J]. 地球信息科学学报, 2020, 22(8): 1702-1713.
[10]	蔡顺, 耿豪鹏, 郑炜珊, 潘保田. 基于傅里叶变换的谷间距特征信息提取及其影响因素研究[J]. 地球信息科学学报, 2020, 22(3): 399-409.
[11]	张志慧, 刘雯, 李笑含, 朱靖轩, 张洪涛, 杨东, 徐超昊, 徐宪立. 基于无人机航摄影像的喀斯特地区裸岩信息提取及景观格局分析[J]. 地球信息科学学报, 2020, 22(12): 2436-2444.
[12]	许佳峰, 李云梅, 徐杰, 雷少华, 毕顺, 周玲. 黑臭水体水面阴影提取的自适应阈值算法研究[J]. 地球信息科学学报, 2020, 22(10): 1959-1970.
[13]	耿仁方,付波霖,蔡江涛,陈晓雨,蓝斐芜,余杭洺,李青逊. 基于无人机影像和面向对象随机森林算法的岩溶湿地植被识别方法研究[J]. 地球信息科学学报, 2019, 21(8): 1295-1306.
[14]	鹿明, 廖小罕, 岳焕印, 黄诗峰, 徐晨晨, 卢海英, 柏艺琴. 面向中国洪涝灾害应急监测的无人机空港布局[J]. 地球信息科学学报, 2019, 21(6): 854-864.
[15]	吴方明, 张淼, 吴炳方. 无人机影像的面向对象水稻种植面积快速提取[J]. 地球信息科学学报, 2019, 21(5): 789-798.