Journal of Geo-information Science >
Model Design of Integrated Vector and Raster Data Organization Under the Distributed Environment
Received date: 2016-04-28
Request revised date: 2016-10-18
Online published: 2016-12-20
Copyright
With the combination and development of geographic information technology and computer network technology, geographic information services based on global framework demand for more efficient massive data management, the traditional single center relational database management mode is unable to meet the requirements. Since the distributed file systems, semi-structured databases and relational database technology have complementary advantages to each other, a new technical method for efficient management of massive data is developed. In order to achieve high effective geospatial data management, this paper presented an integrated architecture oriented to the storage and management of geospatial data in distributed environment, designed user - oriented massive geospatial data integration model and distributed storage organization model . In this model, a technical route combining the NoSQL database and relational database is adopted, and a layered + partitioned data model and multi-level index mechanism for the rapid access of massive data is designed, so it can realizes the integrated management of vector and raster data in distributed environment. Because the model has taken advantages of relational database and semi-structured database, structured geographic information, spatial index and entity data can be managed separately and the efficiency of data processing and access is improved effectively. Vector data and raster data is the largest and most widely used geospatial data, In this paper, an experiment system is set up in the experiment environment, which realizes vector data and raster data management model. TB-level data are used to conduct experiments of data loading, index (pyramid) creation and concurrent data access efficiency, compared with the traditional data model, the model in the data management capacity, processing speed and access efficiency have greatly improved. The results show that the model can support the parallel operation in distributed environment, with a higher data management capability will offer an effective solution to massive data management in distributed environment.
XU Daozhu , LUO Bin , ZHOU Yan , JIN Cheng . Model Design of Integrated Vector and Raster Data Organization Under the Distributed Environment[J]. Journal of Geo-information Science, 2016 , 18(12) : 1588 -1596 . DOI: 10.3724/SP.J.1047.2016.01588
Fig.1 The integrated geographic data storage architecture based on distributed environment图1 分布式环境下空间数据一体化存储架构 |
Fig.2 The vector data model图2 矢量数据模型 |
Fig.3 The layered and partitioned structure of vector data图3 矢量数据分层和分块结构 |
Fig.4 The logical organization of raster data图4 栅格数据逻辑组织 |
Fig.5 The physical storage form of vector and raster data图5 矢栅数据物理存储方式 |
Fig.6 The physical storage model of vector and raster data图6 矢栅物理存储模型 |
Tab.1 The vector data storage andspatial indexing efficiency表1 矢量数据入库和空间索引建立效率 |
入库数据量/GB | 并行客户端数/台 | 入库时间/h | 入库速度/(GB/(小时×单台)) |
---|---|---|---|
1648 | 25 | 29.76 | 2.22 |
120 | 25 | 2.13 | 2.25 |
Tab.2 The image data storage and pyramid building efficiency表2 影像数据入库和金字塔建立效率 |
入库数据量/GB | 并行客户端数/台 | 入库时间/h | 入库速度/(GB/(小时×单台)) |
---|---|---|---|
6656 | 10 | 30.05 | 22.15 |
878 | 10 | 3.92 | 22.40 |
Tab.3 The DEM data storage and pyramid building efficiency表3 DEM数据入库和金字塔建立效率 |
入库数据量/GB | 并行客户端数/台 | 入库时间/h | 入库速度/(GB/(小时×单台)) |
---|---|---|---|
88 | 10 | 1.97 | 4.47 |
37 | 10 | 1.60 | 2.31 |
Tab.4 The storage efficiency of traditional data management system表4 传统数据管理系统入库效率 |
试验内容 | 入库数据/GB | 客户端数/台 | 入库时间/h | 入库速度/(GB/(小时×单台)) |
---|---|---|---|---|
矢量数据入库、索引建立 | 300 | 1 | 85 | 3.45 |
影像数据入库、金字塔创建 | 500 | 1 | 50 | 10 |
Tab.5 The test record of vector data concurrent access表5 矢量数据并发访问测试记录表 |
并发访问 用户个数 | 访问时间 /min | 发送请求 个数 | 系统成功处理 请求个数 | 系统处理请求 成功率/(%) | 成功处理的请求的响应速度/(要素/秒) | ||
---|---|---|---|---|---|---|---|
最小速度 | 最大速度 | 平均速度 | |||||
25×20 | 5 | 624 | 621 | 99.51 | 1231.85 | 578000 | 44152.99 |
25×24 | 5 | 608 | 607 | 99.83 | 1189.83 | 578000 | 46118.62 |
25×28 | 5 | 700 | 697 | 99.58 | 4244.28 | 578000 | 31001.02 |
25×32 | 5 | 805 | 805 | 100 | 785.82 | 578000 | 43859.84 |
25×36 | 5 | 900 | 893 | 99.22 | 766.60 | 578000 | 39124.65 |
25×40 | 5 | 1000 | 990 | 99 | 2918.80 | 578000 | 26832.16 |
25×45 | 5 | 1125 | 1116 | 99.2 | 568.63 | 578000 | 9292.78 |
Tab.6 The test record of image data concurrent access表6 影像数据并发访问测试记录表 |
并发访问 用户个数 | 访问时间 /min | 发送请求 个数 | 系统成功处理 请求个数 | 系统处理 请求成功率/(%) | 成功处理的请求的响应速度/(MB/s) | ||
---|---|---|---|---|---|---|---|
最小速度 | 最大速度 | 平均速度 | |||||
25×20 | 5 | 601 | 601 | 100 | 12 | 300 | 57 |
25×24 | 5 | 622 | 622 | 100 | 18 | 520 | 34 |
25×28 | 5 | 700 | 688 | 98.3 | 23.7 | 430 | 33 |
25×32 | 5 | 805 | 805 | 100 | 40 | 446 | 52.9 |
25×36 | 5 | 900 | 872 | 96.9 | 36 | 270 | 59 |
25×40 | 5 | 1000 | 987 | 98.7 | 10.1 | 280 | 90.5 |
25×44 | 5 | 1100 | 1096 | 99.6 | 6.77 | 275 | 116.59 |
The authors have declared that no competing interests exist.
[1] |
[
|
[2] |
[
|
[3] |
[
|
[4] |
|
[5] |
|
[6] |
|
[7] |
[
|
[8] |
|
[9] |
[
|
[10] |
[
|
[11] |
[
|
[12] |
[
|
[13] |
[
|
[14] |
[
|
[15] |
[
|
[16] |
[
|
[17] |
[
|
/
〈 |
|
〉 |