关键词: 倾斜摄影, 深度学习, 透视变换, 高斯热力图, 特征描述

Abstract:

To solve the problem of few points, low recall rate, and low accuracy of feature matching points in oblique image matching by traditional and deep learning methods, we propose a deep learning-based oblique photogrammetry image matching method. Firstly, the oblique image overlapping areas are computed using Position and Orientation System (POS) information. The geometric deformation of the image overlapping areas, caused by large angle change and inconsistent depth of scene, is compensated using perspective transformation. After removing geometric deformation, the transformed images only have small scale rotation changes. Secondly, we trained the feature point detection neural network in two stages to get the multi-scale feature detection network. The pre-trained multi-scale feature detection network is used to infer the Gaussian heat map on the transformed images. The robust sub-pixel feature points are detected in the extreme scale space of the Gauss heat maps, which effectively avoids the influence of image-scale changes. In order to assist feature points description, the feature points scale and direction are obtained based on the pre-trained self-supervised principal feature direction network. In the feature description stage, the scale and rotation invariant GeoDesc descriptor information is obtained by self-supervised feature detection and principal feature direction network. The feature descriptor is enhanced by considering the image geometric and visual context information, which is useful to describe oblique images with large angle change and those with less texture information. Finally, the initial matching points are obtained by a two-stage ratio purification method, which ensures that not many gross errors in the initial points. The mismatches of initial matching points are further removed by fundamental-based Random Sample Consensus (RANSAC) algorithm and geometry-based graph constraint method, which guarantees that the final obtained matching points accuracy is reliable for bundle block adjustment. In order to verify the matching effect of the proposed method, the two typical rural and city area oblique images in ISPRS oblique photogrammetry datasets are selected to qualitatively and quantitatively analyze the matching results of all methods. The experimental results show that our proposed method can obtain lots of uniformly distributed matching points in large scale perspective and poor-texture oblique images. Compared with SIFT, Affine SIFT (ASIFT), SuperPoint, GeoDesc, and ContextDesc algorithms, our proposed method can acquire more robust feature points in scale space of the Gauss heat maps, which is helpful to increase the matching recall rate and accuracy.

Key words: oblique photogrammetry, deep learning, perspective transformation, gaussian heat map, feature description