Journal of Geo-information Science >
Research and Implementation of Immersive Multi-User Collaborative Interaction Technology in Virtual Geographic Environment
Received date: 2018-01-31
Request revised date: 2018-04-18
Online published: 2018-08-24
Supported by
Pre-research Project of Equipment Development Department, No.315050501
National Natural Science Foundation of China, No.41771442, 41271402.
Copyright
In the development of virtual geographic environment (VGE), natural and efficient human-computer interactions have always been one of the core contents of its research. However, users still communicate with VGE based on the traditional human-computer interaction interface in many current VGE application cases. This leads to two problems: (1) the immersive feeling of the users is poor; (2) the requirements for multi-users to interact in VGE cannot be effectively satisfied. It is crucial to take advantage of the latest achievements of human-computer interaction technology in order to solve these problems to meet the immersive multi-user collaborative interaction requirements in VGE. In this paper, the status of research and application of the human-computer interaction technology in VGE was reviewed. The development process and basic principle of motion capture technology and its advantages in improving the interactive immersion of VGE users were briefly described. Next, focusing on the design of immersive multi-user collaborative interaction system in VGE, the immersive multi-user collaborative interaction model based on motion capture technology and the corresponding interactive semantics of its interactive actions were analyzed. A multi-user collaborative conflict control mechanism was proposed. The structure and function of the immersive multi-user collaborative interaction system in VGE were introduced emphatically. Moreover, the Virtual Yanyuan Auxiliary Planning Prototype System (VYAPPS) adopting Peking University campus as a case study was developed by using Javascript and Python language. The experiments based on VYAPPS were carried out. Questionnaire statistical results show that the interactive effects of the immersive multi-user collaborative interaction technology in VGE are recognized by users. The results also showed that the advantages of the immersive multi-user collaborative interaction technology in improving multi-user immersion make it have a good application prospect. For example, it can be applied to geography education, urban planning, emergency rescue, military command fields and so on.
Key words: VGE; motion capture; human-computer interaction; multi-user collaboration; immersion
YUAN Shuai , CHEN Bin , YI Chao , XU Bingli . Research and Implementation of Immersive Multi-User Collaborative Interaction Technology in Virtual Geographic Environment[J]. Journal of Geo-information Science, 2018 , 20(8) : 1055 -1063 . DOI: 10.12082/dqxxkx.2018.180082
Fig. 1 Avatar bone structure图1 化身骨骼结构 |
Tab. 1 Immersive multi-user collaborative interaction model and interactive actions' semantics in VGE表1 VGE中沉浸式多人协同交互模式及交互动作语义 |
交互模式 | 动作类型 | 交互动作语义 | |
---|---|---|---|
用户与用户 之间交互 | 转向 | 用户化身的朝向发生改变 | |
蹲下 | 用户化身下蹲,以便观察 | ||
起立 | 用户化身起立,以便观察 | ||
对话 | 用户通过自然语音进行沟通 | ||
传递 | 虚拟对象从一个化身手中传递到另一个化身手中 | ||
研讨 | 用户化身通过手势和语音进行讨论,相互交流 | ||
用户与VGE 之间交互 | 与虚拟对象交互 | 指点 | 用户化身通过手势或工具来使感兴趣对象高亮,准备交互 |
拾取 | 用户化身指点目标虚拟对象后,用手抓住该对象 | ||
移动 | 用户化身拾取虚拟对象后,移动手部位置,该对象位置改变 | ||
放置 | 用户化身移动虚拟对象后手部松开,该对象置于新位置 | ||
增加 | 用户化身拾取虚拟对象后移动手部位置到虚拟地理场景内,该虚拟对象增添 | ||
删除 | 用户化身拾取虚拟对象后移动手部位置到虚拟地理场景外或指点物体后点击菜单删除按钮,该虚拟对象删除 | ||
与虚拟地理景观交互 | 放大 | 用户化身左右手从中间向两侧分开,局部场景放大 | |
缩小 漫游 | 用户化身左右手从两侧向中间合拢,局部场景缩小 用户化身在虚拟地理景观中的位置发现变化 |
Fig. 2 Structure of immersive multi-user collaborative interaction system in VGE图2 VGE中沉浸式多人协同交互系统结构 |
Tab. 2 Module function design of immersive multi-user collaborative interaction system in VGE表2 VGE中沉浸式多人协同交互系统模块功能设计 |
系统角色 | 功能模块 | 功能描述 | |
---|---|---|---|
服务器端 | Http服务器 | 完成通信协议解析等服务功能 | |
虚拟世界服务器 | 实体服务器 | 提供动态内容同步服务 | |
资源服务器 | 提供动态LOD资源服务,以优化渲染的资源分配和显示效果 | ||
化身服务器 | 把捕捉到的用户位置、动作和姿态数据合成为化身动作,分析并实现动作语义,并将操作结果实时发送给其他用户 | ||
语音服务器 | 打包并根据声场量级来传输实时语音数据 | ||
数据管理服务器 | 空间数据库 | 对三维地形数据和遥感影像数据等地学数据进行管理,构成基础VGE | |
模型数据库 | 对三维模型信息和化身模型信息进行管理,构成VGE | ||
客户端 | 资源管理模块 | 对本地的各类数据资源进行管理 | |
通信模块 | 负责对服务器端的请求与响应 | ||
渲染模块 | 对场景进行渲染,完成PC显示屏、头盔显示器等设备上的显示 | ||
动作捕捉端 | 动作捕捉插件 | 完成对各种类型动作捕捉设备的驱动 | |
动作捕捉设备 | 获取用户位置、动作和姿态数据 |
Fig. 3 The operation scene of VYAPPS图3 VYAPPS运行实景 |
Fig. 4 The structure of VYAPPS图4 VYAPPS结构 |
Fig. 5 VGE in VYAPPS图5 VYAPPS中的VGE |
Fig. 6 Experimental results of VYAPPS图6 VYAPPS实验结果 |
Tab. 3 Questionnaire statistics of VYAPPS表3 VYAPPS问卷统计结果 |
项目 | 具体内容 | 很好 | 一般 | 很差 | |||||
---|---|---|---|---|---|---|---|---|---|
数目 | 比率/% | 数目 | 比率/% | 数目 | 比率/% | ||||
系统性能 | 系统流畅性 | 32 | 91 | 2 | 6 | 1 | 3 | ||
捕捉准确率 | 31 | 88 | 2 | 6 | 2 | 6 | |||
用户体验 | 交互自由度 | 34 | 97 | 1 | 3 | 0 | 0 | ||
操作适应度 | 33 | 94 | 1 | 3 | 1 | 3 | |||
多人沉浸度 | 33 | 94 | 2 | 6 | 0 | 0 |
The authors have declared that no competing interests exist.
[1] |
|
[2] |
[
|
[3] |
[
|
[4] |
[
|
[5] |
[
|
[6] |
|
[7] |
[
|
[8] |
|
[9] |
[
|
[10] |
[
|
[11] |
[
|
[12] |
[
|
[13] |
[
|
[14] |
|
[15] |
[
|
[16] |
[
|
[17] |
[
|
[18] |
|
[19] |
[
|
[20] |
[
|
[21] |
[
|
[22] |
|
[23] |
|
[24] |
[
|
[25] |
[
|
[26] |
[
|
[27] |
[
|
[28] |
[
|
[29] |
[
|
[30] |
[
|
[31] |
[
|
[32] |
|
[33] |
[
|
[34] |
[
|
[35] |
[
|
[36] |
[
|
[37] |
[
|
/
〈 |
|
〉 |