虚拟地理环境中沉浸式多人协同交互技术研究及实现

doi:10.12082/dqxxkx.2018.180082

地球信息科学学报 ›› 2018, Vol. 20 ›› Issue (8): 1055-1063.doi: 10.12082/dqxxkx.2018.180082

• 地球信息科学理论与方法 • 上一篇下一篇

虚拟地理环境中沉浸式多人协同交互技术研究及实现

袁帅^1,²(), 陈斌^1,^*(), 易超¹, 徐丙立³

1. 北京大学遥感与地理信息系统研究所,北京 100871
2. 中国人民武装警察部队警种学院,北京 102202
3. 陆军装甲兵工程学院,北京 100072

收稿日期:2018-01-31 修回日期:2018-04-18 出版日期:2018-08-25 发布日期:2018-08-24
作者简介:
作者简介：袁帅（1987-）,男,硕士生,主要从事虚拟地理环境研究。E-mail: gisyuanshuai@pku.edu.cn
基金资助:
装备发展部预研项目（315050501）;国家自然科学基金项目（41771442、41271402）

Research and Implementation of Immersive Multi-User Collaborative Interaction Technology in Virtual Geographic Environment

YUAN Shuai^1,²(), CHEN Bin^1,^*(), YI Chao¹, XU Bingli³

1. Institute of Remote Sensing and GIS, Peking University, Beijing 100871, China
2. Specialized Forces College of Chinese People's Armed Police Force, Beijing 102202, China
3. Academy of Armored Force Engineering, Beijing 100072, China

Received:2018-01-31 Revised:2018-04-18 Online:2018-08-25 Published:2018-08-24
Contact: CHEN Bin
Supported by:
Pre-research Project of Equipment Development Department, No.315050501;National Natural Science Foundation of China, No.41771442, 41271402.

摘要/Abstract

摘要：

在VGE的发展过程中,自然高效的人机交互一直是其研究的核心内容之一,但在当前众多VGE应用案例中,用户仍然是基于传统的人机交互界面与VGE进行交流,这导致了：① 用户的沉浸感不高;② 多人在VGE中协同交互的需求无法得到有效满足。因此,为了实现VGE中的沉浸式多人协同交互,本文首先综述了VGE中人机交互技术研究和应用现状,并简要阐述了动作捕捉技术发展历程、基本原理及其在提高VGE用户交互沉浸感上的优势。在此基础上,对VGE中沉浸式多人协同交互系统进行了重点设计,主要包括分析了基于动作捕捉技术的沉浸式多人协同交互模式及其对应的交互动作语义,提出了多人协同冲突控制机制,以及着重介绍了VGE中沉浸式多人协同交互系统的结构和功能。最后,以北京大学为例开发了虚拟燕园辅助规划设计原型系统,并进行了实验。研究结果表明,沉浸式多人协同交互技术的交互效果受到用户认可,它在提高化身之间及化身与VGE之间的协同交互方面具有明显的优势,可应用于地学教育、城市规划、应急救援和军事指挥等领域,具有良好的应用前景。

关键词: 虚拟地理环境, 动作捕捉, 人机交互, 多人协同, 沉浸感

Abstract:

In the development of virtual geographic environment (VGE), natural and efficient human-computer interactions have always been one of the core contents of its research. However, users still communicate with VGE based on the traditional human-computer interaction interface in many current VGE application cases. This leads to two problems: (1) the immersive feeling of the users is poor; (2) the requirements for multi-users to interact in VGE cannot be effectively satisfied. It is crucial to take advantage of the latest achievements of human-computer interaction technology in order to solve these problems to meet the immersive multi-user collaborative interaction requirements in VGE. In this paper, the status of research and application of the human-computer interaction technology in VGE was reviewed. The development process and basic principle of motion capture technology and its advantages in improving the interactive immersion of VGE users were briefly described. Next, focusing on the design of immersive multi-user collaborative interaction system in VGE, the immersive multi-user collaborative interaction model based on motion capture technology and the corresponding interactive semantics of its interactive actions were analyzed. A multi-user collaborative conflict control mechanism was proposed. The structure and function of the immersive multi-user collaborative interaction system in VGE were introduced emphatically. Moreover, the Virtual Yanyuan Auxiliary Planning Prototype System (VYAPPS) adopting Peking University campus as a case study was developed by using Javascript and Python language. The experiments based on VYAPPS were carried out. Questionnaire statistical results show that the interactive effects of the immersive multi-user collaborative interaction technology in VGE are recognized by users. The results also showed that the advantages of the immersive multi-user collaborative interaction technology in improving multi-user immersion make it have a good application prospect. For example, it can be applied to geography education, urban planning, emergency rescue, military command fields and so on.

Key words: VGE, motion capture, human-computer interaction, multi-user collaboration, immersion

袁帅, 陈斌, 易超, 徐丙立. 虚拟地理环境中沉浸式多人协同交互技术研究及实现[J]. 地球信息科学学报, 2018, 20(8): 1055-1063.DOI:10.12082/dqxxkx.2018.180082

YUAN Shuai,CHEN Bin,YI Chao,XU Bingli. Research and Implementation of Immersive Multi-User Collaborative Interaction Technology in Virtual Geographic Environment[J]. Journal of Geo-information Science, 2018, 20(8): 1055-1063.DOI:10.12082/dqxxkx.2018.180082

图/表 9

图1

表1

图2

表2

图3

图4

图5

图6

表3

参考文献 37

[1]	Batty M.Virtual geography[J]. Futures, 1997,29(45):337-352. doi: 10.1016/S0016-3287(97)00018-9
[2]	龚建华,林珲.虚拟地理环境:在线虚拟现实的地理学透视[M].北京:高等教育出版社,2001.
	[ Gong J H, Lin H.VGE: Geographical perspective of online virtual reality[M]. Beijing: Higher Education Press, 2001. ]
[3]	林珲,龚建华,施晶晶.从地图到地理信息系统与虚拟地理环境——试论地理学语言的演变[J].地理与地理信息科学,2003,19(4):18-23.
	[ Lin H, Gong J H, Shi J J.From maps to GIS and VGE: A discussion on the evolution of the geographic language[J]. Geography and Geo-information Science, 2003,19(4):18-23. ]
[4]	龚建华,周洁萍,张利辉.虚拟地理环境研究进展与理论框架[J].地球科学进展,2010,25(9):915-926. doi: 10.11867/j.issn.1001-8166.2010.09.0915
	[ Gong J H, Zhou J P, Zhang L H.Study progress and theorectical framework of virtual geographic environments[J]. Advances in Earth Science, 2010,25(9):915-926. ] doi: 10.11867/j.issn.1001-8166.2010.09.0915
[5]	万刚,曹雪峰.地理空间信息网格的历史演变与思考[J].测绘学报,2016,45(s1):15-22. doi: 10.11947/j.AGCS.2016.F002
	[ Wan G, Cao X F.The historical evolution and reflection of geospatial information grid[J]. Acta Geodaetica et Cartographica Sinica, 2016,45(s1):15-22. ] doi: 10.11947/j.AGCS.2016.F002
[6]	Voinov A, Çöltekin A, Chen M, et al.Virtual geographic environments in socio-environmental modeling: A fancy distraction or a key to communication?[J]. International Journal of Digital Earth, 2018,11(4):408-419. doi: 10.1080/17538947.2017.1365961
[7]	林珲,朱庆.虚拟地理环境的地理学语言特征[J].遥感学报,2005,9(2):158-165. doi: 10.3321/j.issn:1007-4619.2005.02.008
	[ Lin H, Zhu Q.The linguistic characteristics of virtual geographic environments[J]. Journal of Remote Sensing, 2005,9(2):158-165. ] doi: 10.3321/j.issn:1007-4619.2005.02.008
[8]	Lin H, Chen M, Lu G N.Virtual geographic environment: A workspace for computer-aided geographic experiments[J]. Annals of the Association of American Geographers, 2013,103(3):465-482. doi: 10.1080/00045608.2012.689234
[9]	林珲,陈旻.利用虚拟地理环境的实验地理学方法[J].武汉大学学报·信息科学版,2014,39(6):689-694. doi: 10.13203/j.whugis20140153
	[ Lin H, Chen M.Experimental geography based on virtual geographic environments[J]. Geomatics and Information Science of Wuhan University, 2014,39(6):689-694. ] doi: 10.13203/j.whugis20140153
[10]	贾奋励,张威巍,游雄.虚拟地理环境的认知研究框架初探[J].遥感学报,2015,19(2):179-187. doi: 10.11834/jrs.20154013
	[ Jia F L, Zhang W W, You X.Cognitive research framework of virtual geographic environment[J]. Journal of Remote Sensing, 2015,19(2):179-187. ] doi: 10.11834/jrs.20154013
[11]	林天鹏,林珲,胡明远,等.基于虚拟地理环境平台的地理设计研究[J].中国园林,2014(10):18-21.
	[ Lin T P, Lin H, Hu M Y, et al.Research on geodesign based on virtual geographic environment platform[J]. Chinese Landscape Architecture, 2014(10):18-21. ]
[12]	徐丙立,荆涛,林珲,等.利用CryEngine构建虚拟地理环境[J].武汉大学学报·信息科学版,2017,42(1):28-34. doi: 10.13203/j.whugis20140768
	[ Xu B L, Jing T, Lin H, et al.CryEngine based virtual geographic environments construction[J]. Geomatics and Information Science of Wuhan University, 2017,42(1):28-34. ] doi: 10.13203/j.whugis20140768
[13]	胡香,巩保胜,胡建磊,等.面向军事指挥人员空间认知规律的虚拟地理环境设计研究[J].测绘与空间地理信息,2017,40(10):129-131.
	[ Hu X, Gong B S, Hu J L, et al.Research on the design of virtual geographical environment for military commanders[J]. Geomatics & Spatial Information Technology, 2017,40(10):129-131. ]
[14]	Mekni M.Spatial simulation using abstraction of virtual geographic environments[J]. International Journal of Digital Earth, 2018,11(4):334-355. doi: 10.1080/17538947.2017.1381190
[15]	周洁萍,龚建华,陈铮,等.协同虚拟地理环境中多用户交流交互模式及实现[J].地理与地理信息科学,2005,21(5):33-37. doi: 10.3969/j.issn.1672-0504.2005.05.008
	[ Zhou J P, Gong J H, Chen Z, et al.Study on collaborative virtual geographic environment system for multi-user interaction and communication[J]. Geography and Geo-information Science, 2005,21(5):33-37. ] doi: 10.3969/j.issn.1672-0504.2005.05.008
[16]	郑炼功. 协同虚拟战场研讨环境构建理论及技术研究[D].郑州:信息工程大学,2007.
	[ Zheng L G.Research on theory and technology of collaborative virtual battlefield research environment construction[D]. Zhengzhou: Information Engineering University, 2007. ]
[17]	陈斌,黄凤茹,方裕,等.基于OpenSimulator的分布式虚拟地理环境应用研究[J].高技术通讯,2011,21(12):1272-1278. doi: 10.3772/j.issn.1002-0470.2011.12.009
	[ Chen B, Huang F R, Fang Y, et al. Distributed virtual geographic environments applications based on OpenSimulator[J]. Chinese High Technology Letters, 2011,21(12):1272-1278. ] doi: 10.3772/j.issn.1002-0470.2011.12.009
[18]	Xu B L, Lin H, Chiu L S, et a1. Collaborative virtual geographic environments: A case study of air pollution simulation[J]. Information Sciences, 2011,181(11):2231-2246. doi: 10.1016/j.ins.2011.01.017
[19]	龚建华. 论虚拟地理实验思想与方法[J].测绘科学技术学报,2013,30(4):399-408.
	[ Gong J H.On thought and methodology of virtual geographic experiment[J]. Journal of Geomatics Science and Technology, 2013,30(4):399-408. ]
[20]	江辉仙. 地理环境虚拟实验系统[J].实验室研究与探索,2014,33(11):62-66. doi: 10.3969/j.issn.1006-7167.2014.11.016
	[ Jiang H X.Virtual experiment system of geographical environment[J]. Research and Exploration in Laboratory, 2014,33(11):62-66. ] doi: 10.3969/j.issn.1006-7167.2014.11.016
[21]	易思蓉,聂良涛.基于虚拟地理环境的铁路数字化选线设计系统[J].西南交通大学学报,2016,51(2):373-380. doi: 10.3969/j.issn.0258-2724.2016.02.016
	[ Yi S R, Nie L T.Digital railway location system based on virtual geographic environment[J]. Journal of Southwest Jiaotong University, 2016,51(2):373-380. ] doi: 10.3969/j.issn.0258-2724.2016.02.016
[22]	Menache A.Understanding motion capture for computer animation and video games[M]. San Francisco: Morgan Kaufmann Publishers Incorporated, 1999.
[23]	Srurma D J.A brief history of motion capture for computer character animation[C]. Proceding of ACM Annual Conference on Computer Graphics. New York, USA, 1994.
[24]	黄波士,陈福民.人体运动捕捉及运动控制的研究[J].计算机工程与应用,2005,41(7):60-63. doi: 10.3321/j.issn:1002-8331.2005.07.022
	[ Huang B S, Chen F M.Research on human body motion capture and motion control[J]. Computer Engineering and Applications, 2005,41(7):60-63. ] doi: 10.3321/j.issn:1002-8331.2005.07.022
[25]	李晓丹,肖明,曾莉.人体动作捕捉技术综述以及一种新的动作捕捉方案陈述[J].中国西部科技,2011,10(15):35-37.
	[ Li X D, Xiao M, Zeng L.Review on the motion capture technologies[J]. Science and Technology of West China, 2011,10(15):35-37. ]
[26]	李明宇,赵亮,姜军.动作捕捉技术应用研究调查报告[J].科技信息,2011(36):471. doi: 10.3969/j.issn.1001-9960.2011.36.413
	[ Li M Y, Zhao L, Jiang J.Investigation report on application research of motion capture technology[J]. Science & Technology Information, 2011(36):471. ] doi: 10.3969/j.issn.1001-9960.2011.36.413
[27]	向泽锐,支锦亦,徐伯初,等.运动捕捉技术及其应用研究综述[J].计算机应用研究,2013,30(8):2241-2245. doi: 10.3969/j.issn.1001-3695.2013.08.001
	[ Xiang Z R, Zhi J Y, Xu B C, et al.Survey on motion capture technique and its applications[J]. Application Research of Computers, 2013,30(8):2241-2245. ] doi: 10.3969/j.issn.1001-3695.2013.08.001
[28]	黄玉飞. 动作捕捉技术在体育运动领域的发展现状[J].当代体育科技,2017,7(27):210-211. doi: 10.16655/j.cnki.2095-2813.2017.27.210
	[ Huang Y F.The development status of motion capture technology in sports field[J]. Contemporary Sports Technology, 2017,7(27):210-211. ] doi: 10.16655/j.cnki.2095-2813.2017.27.210
[29]	张凤军,戴国忠,彭晓兰.虚拟现实的人机交互综述[J].中国科学:信息科学,2016,46(12):1711-1736.
	[ Zhang F J, Dai G Z, Peng X L.A survey on human-computer interaction in virtual reality[J]. SCIENTIA SINICA Informationis, 2016,46(12):1711-1736. ]
[30]	赵永惠. 人机交互研究综述[J].信息与电脑(理论版),2017(23):24-28.
	[ Zhao Y H.Survey of human-computer interaction research[J]. China Computer & Communication, 2017(23):24-28. ]
[31]	李小杰,李响.手势识别技术在虚拟地理环境中的应用初探[J].测绘与空间地理信息,2010,33(5):41-44.
	[ Li X J, Li X.Preliminary application of gesture recognition to virtual geographical environment[J]. Geomatics & Spatial Information Technology, 2010,33(5):41-44. ]
[32]	Lin H, Chen M, Lu G N, et al.Virtual geographic environments (VGEs): A new generation of geographic analysis tool[J]. Earth-science Reviews, 2013,126(11):74-84. doi: 10.1016/j.earscirev.2013.08.001
[33]	张晓丽. 三维运动捕捉技术在消防模拟训练中的应用[J].武警学院学报,2011,27(8):88-90. doi: 10.3969/j.issn.1008-2077.2011.08.034
	[ Zhang X L.A research on the application of three-dimensional motion capturing technology in fire simulation training[J]. Journal of Chinese People's Armed Police Force Academy, 2011,27(8):88-90. ] doi: 10.3969/j.issn.1008-2077.2011.08.034
[34]	陈楠楠,朱东鸣.基于视觉交互的虚拟校园研究[J].中国医学教育技术,2011,25(5):510-513.
	[ Chen N N, Zhu D M.Virtual campus research based on visual interaction[J]. China Medical Education Technology, 2011,25(5):510-513. ]
[35]	陈帼鸾,陆雷敏,何灵辉,等.基于HTC VIVE虚拟校园漫步系统——以中山职业技术学院为例[J].中国科技信息,2017(10):63-64.
	[ Chen G L, Lu L M, He L H, et al.Virtual campus ramble system based on HTC VIVE-taking Zhongshan Polytechnic as an example[J]. China Science and Technology Information, 2017(10):63-64. ]
[36]	杨光辉,马纯永,韩勇,等.基于Kinect的虚拟展馆游览系统的设计与实现[J].计算机技术与发展,2014(6):174-178. doi: 10.3969/j.issn.1673-629X.2014.06.043
	[ Yang G H, Ma C Y, Han Y, et al.Design and implementation of virtual pavilion tour system based on Kinect technology[J]. Computer Technology and Development, 2014(6):174-178. ] doi: 10.3969/j.issn.1673-629X.2014.06.043
[37]	霍宇平,张秀娥,李兵,等.运动捕捉技术在变电站虚拟环境中的应用研究[J].系统仿真学报,2016,28(10):2632-2637.
	[ Huo Y P, Zhang X E, Li B, et al.Research on motion capture in substation virtual environment[J]. Journal of System Simulation, 2016,28(10):2632-2637. ]

交互模式		动作类型	交互动作语义
用户与用户之间交互		转向	用户化身的朝向发生改变
		蹲下	用户化身下蹲,以便观察
		起立	用户化身起立,以便观察
		对话	用户通过自然语音进行沟通
		传递	虚拟对象从一个化身手中传递到另一个化身手中
		研讨	用户化身通过手势和语音进行讨论,相互交流
用户与VGE 之间交互	与虚拟对象交互	指点	用户化身通过手势或工具来使感兴趣对象高亮,准备交互
		拾取	用户化身指点目标虚拟对象后,用手抓住该对象
		移动	用户化身拾取虚拟对象后,移动手部位置,该对象位置改变
		放置	用户化身移动虚拟对象后手部松开,该对象置于新位置
		增加	用户化身拾取虚拟对象后移动手部位置到虚拟地理场景内,该虚拟对象增添
		删除	用户化身拾取虚拟对象后移动手部位置到虚拟地理场景外或指点物体后点击菜单删除按钮,该虚拟对象删除
	与虚拟地理景观交互	放大	用户化身左右手从中间向两侧分开,局部场景放大
	与虚拟地理景观交互	缩小漫游	用户化身左右手从两侧向中间合拢,局部场景缩小用户化身在虚拟地理景观中的位置发现变化

系统角色		功能模块	功能描述
服务器端	Http服务器		完成通信协议解析等服务功能
	虚拟世界服务器	实体服务器	提供动态内容同步服务
		资源服务器	提供动态LOD资源服务,以优化渲染的资源分配和显示效果
		化身服务器	把捕捉到的用户位置、动作和姿态数据合成为化身动作,分析并实现动作语义,并将操作结果实时发送给其他用户
		语音服务器	打包并根据声场量级来传输实时语音数据
	数据管理服务器	空间数据库	对三维地形数据和遥感影像数据等地学数据进行管理,构成基础VGE
	数据管理服务器	模型数据库	对三维模型信息和化身模型信息进行管理,构成VGE
客户端		资源管理模块	对本地的各类数据资源进行管理
		通信模块	负责对服务器端的请求与响应
		渲染模块	对场景进行渲染,完成PC显示屏、头盔显示器等设备上的显示
动作捕捉端		动作捕捉插件	完成对各种类型动作捕捉设备的驱动
动作捕捉端		动作捕捉设备	获取用户位置、动作和姿态数据

项目	具体内容	很好		一般		很差
项目	具体内容	数目	比率/%	数目	比率/%	数目	比率/%
系统性能	系统流畅性	32	91	2	6	1	3
系统性能	捕捉准确率	31	88	2	6	2	6
用户体验	交互自由度	34	97	1	3	0	0
	操作适应度	33	94	1	3	1	3
	多人沉浸度	33	94	2	6	0	0

虚拟地理环境中沉浸式多人协同交互技术研究及实现

Research and Implementation of Immersive Multi-User Collaborative Interaction Technology in Virtual Geographic Environment

RichHTML

PDF (PC)

赞

可视化

被引次数

摘要/Abstract

引用本文

使用本文

图/表 9

参考文献 37

相关文章 15

Metrics

本文评价

推荐阅读 0

[1]	曾梦熊, 华一新, 张政, 张江水, 杨振凯, 韦原原. 面向大规模空间Agent建模的分布式地理模拟框架[J]. 地球信息科学学报, 2022, 24(5): 815-826.
[2]	江锋, 唐丽玉, 林定, 陈晓玲, 冯先超, 陈崇成. 基于城市园林树木景观三维模拟的绿视率估算方法[J]. 地球信息科学学报, 2021, 23(12): 2151-2162.
[3]	林珲, 胡明远, 陈旻, 张帆, 游兰, 陈宇婷. 从地理信息系统到虚拟地理环境的认知转变[J]. 地球信息科学学报, 2020, 22(4): 662-672.
[4]	马晓辉,周洁萍,龚建华,黄琳,李文航,邹宇玲. 面向室内应急疏散标识的VR眼动感知实验与布局评估[J]. 地球信息科学学报, 2019, 21(8): 1170-1182.
[5]	易超, 陈斌, 袁帅, 徐丙立. 虚拟地理环境中基于多模式的多人动作捕捉方法[J]. 地球信息科学学报, 2019, 21(3): 305-314.
[6]	刘君妍, 陈雅茜, 高亦远, 李创, 佘江峰. 一种基于屏幕的三维地图线状符号渲染方法[J]. 地球信息科学学报, 2018, 20(8): 1047-1054.
[7]	唐丽玉, 王灵霞, 陈崇成, 陈琪. 生长模型驱动的单株杉木三维动态模拟[J]. 地球信息科学学报, 2015, 17(6): 668-674.
[8]	林珲, 游兰. 虚拟地理环境知识工程初探[J]. 地球信息科学学报, 2015, 17(12): 1423-1430.
[9]	朱军, 张恒. 虚拟高速铁路环境快速建模服务与案例[J]. 地球信息科学学报, 2013, 15(6): 895-901.
[10]	杜云虎, 陈崇成, 唐丽玉, 李界光. 一种单机多屏的时序三维地理场景同步可视化系统[J]. 地球信息科学学报, 2013, 15(2): 200-208.
[11]	唐丽玉, 林定, 黄洪宇, 邹杰, 陈崇成, 杜云虎. 基于虚拟植物的幼龄杉木生长模拟[J]. 地球信息科学学报, 2012, 14(5): 569-575.
[12]	苏红军, 盛业华, 温永宁, 陈旻. 面向虚拟地理环境的多源异构数据集成方法[J]. 地球信息科学学报, 2009, 11(3): 292-298.
[13]	宋仁波, 江南. 矿体线框模型编辑器的设计与开发[J]. 地球信息科学学报, 2008, 10(2): 195-199.
[14]	杨慧, 温永宁, 闾国年, 韦程. 面向虚拟地理环境的空间信息共享研究[J]. 地球信息科学学报, 2007, 9(4): 74-79.
[15]	唐卫, 闾国年, 温永宁, 陶虹. 面向虚拟地理环境的图形化地理建模与应用[J]. 地球信息科学学报, 2007, 9(2): 78-84.