地球信息科学学报 ›› 2023, Vol. 25 ›› Issue (5): 883-895.doi: 10.12082/dqxxkx.2023.220544

• 地球信息科学理论与方法 •    下一篇

结构语义辅助约束的数字孪生场景虚实配准方法

胡校飞(), 周杨(), 蓝朝桢, 黄高爽, 赵璐颖   

  1. 中国人民解放军战略支援部队信息工程大学地理空间信息学院,郑州 450001
  • 收稿日期:2022-07-25 修回日期:2022-09-20 出版日期:2023-05-25 发布日期:2023-04-27
  • 通讯作者: *周 杨(1974—),男,贵州安顺人,博士,教授,博士生导师,主要研究方向为网络空间测绘。E-mail: zhouyang3d@163.com
  • 作者简介:胡校飞(1990—),男,河南平顶山人,博士,主要从事视觉定位、互联网图像大数据挖掘研究。E-mail: huxiaofeicn@163.com

Virtual Real Registration Assisted by Structural Semantic Constraint for Digital City Scene

HU Xiaofei(), ZHOU Yang(), LAN Chaozhen, HUANG Gaoshuang, ZHAO Luying   

  1. PLA Strategic Support Force Information Engineering University, Zhengzhou 450001, China
  • Received:2022-07-25 Revised:2022-09-20 Online:2023-05-25 Published:2023-04-27
  • Contact: ZHOU Yang

摘要:

数字孪生城市通常以实景三维模型为空间数据体,并集成各类传感器的物联感知数据,实现虚实融合目的。作为最常用的传感器类型之一,将城市中的视觉传感器与虚拟的实景三维模型准确地配准是数字孪生城市应用的关键,但是现有方法获取的户外6DOF位姿精度通常不足,致使视觉传感器配准精度不高。本文提出了一种结构语义辅助约束的数字孪生场景虚实配准方法,从图像中提取出包含结构语义信息的铅垂线,并生成虚拟场景的透视图和逆深度图,在逆深度和结构语义的约束下,准确地恢复传感器位姿,实现视觉传感器图像在虚拟场景空间中的融合配准。通过试验证明,该方法可以有效对初始位姿进行优化,进而提高数字孪生城市中视觉传感器的配准精度。

关键词: 数字孪生城市, 实景三维, 虚实融合, 视觉传感器, 地理配准, 视觉地理定位

Abstract:

Digital city is one of the main requirements of three-dimensional (3D) real scene and leads the direction of future smart city construction. Digital city usually uses the 3D model of the real scene as the spatial data volume and integrates the object-linked data of various sensors to achieve virtual-real fusion. The integration of spatial data volume and object-linked perception data is the key to digital city applications. Visual sensor is an important sensor type which is widely used in urban life, such as surveillance cameras, vehicles, and other devices. The key to digital city application is registering the visual sensors with virtual 3D model accurately. The purpose of spatial registration for visual sensor is to estimate or optimize the position and orientation of the visual sensor and to get the accurate spatial position of any object in the image. It is one of the key technologies for applications such as Augmented Reality and Video GIS. Currently, the spatial registration methods for visual sensors can be divided into hardware-based and vision-based methods. Due to the popularity of vision sensors, vision-based registration methods have been widely used. However, in digital city applications, seasons and weather always change, there are often large differences in appearance between the real image taken by visual sensor and the image of virtual scene. Therefore, the accuracy of outdoor 6 Degree of Freedom (DOF) position obtained by existing methods is usually insufficient, resulting in low registration accuracy of the visual sensor. In order to improve the accuracy of visual sensor spatial registration in digital city scene, this paper presents a method of virtual-real registration for digital city scene with structural semantic information in urban area. Firstly, the virtual perspective image of digital city scene is obtained, the plumb line which contains structural semantic information is extracted from the target image, and the properties of global constraints of the plumb line is used to restore the camera's position accurately and achieve the registration of monocular image in the virtual digital scene. Experiments show that this method achieves accurate registration of virtual and real images with large differences in appearance. Compared with the existing methods, the position and orientation errors are reduced by 35.9% and 39.3%. This method can effectively optimize the initial pose and improve the registration accuracy of visual sensors in digital city scene. A lightweight cloud-edge registration framework is designed and can be used in image geolocation tasks based on portable devices.

Key words: digital city, 3D real sScene, virtual real fusion, visual sensor, geographic registration, visual geographic location