基于ODC的国产卫星影像存储与应用研究
李俊杰(1996— ),男,湖北宜昌人,硕士,主要从事水利遥感监测研究。E-mail:junjieli@whu.edu.cn |
收稿日期: 2019-07-03
要求修回日期: 2019-10-16
网络出版日期: 2020-11-25
基金资助
国家重点研发计划项目(2017YFC0405806)
版权
Management and Application of Domestic Satellite Imagery based on Open Data Cube
Received date: 2019-07-03
Request revised date: 2019-10-16
Online published: 2020-11-25
Supported by
National Key Research and Development Pro gram of China(2017YFC0405806)
Copyright
随着地球观测进入大数据时代,传统的数据管理技术已经难以适应大数据需求, Open Data Cube (ODC)作为新型的开源的地球观测数据管理与分析平台,适合进行时间序列数据的高性能计算和探索性数据分析,是亚大区域综合地球观测系统AOGEOSS的重要技术支撑平台。但当前ODC对国产卫星影像支持不友好,缺乏自动化管理和数据组织工具,使用ODC进行国产卫星影像数据管理的技术不成熟。因此,本文以高分一号卫星影像为例,通过开发ODC_GFTool中间件和自定义高分卫星影像产品格式探索ODC框架下国产影像数据自动化管理流程,最后以鄱阳湖为试验区,进行了基于ODC框架的水体提取应用实验,论证了ODC框架下国产卫星数据存取的可行性,研究结果表明相较于传统的数据处理工具ODC具有明显的效率优势,能够为AOGEOSS基础设施建设和国产卫星影像数据管理提供一定的参考。
李俊杰 , 陈舒博 , 张文 , 余长慧 , 张志远 , 孟令奎 . 基于ODC的国产卫星影像存储与应用研究[J]. 地球信息科学学报, 2020 , 22(9) : 1860 -1867 . DOI: 10.12082/dqxxkx.2020.190351
As earth observation enters the big data era, the traditional data management technology has often become insufficient in meeting research needs. As a new open source earth observation data management and analysis platform, Open Data Cube (ODC) is suitable for high-performance computing and exploratory data analysis of time series data. It's an important technical support platform for Asia-Oceania Global Earth Observation System of Systems (AOGEOSS). However, few attempts have been made on domestic satellite imagery data management using ODC. The current ODC lacks automated data organization tools for China's satellite imagery. Therefore, this paper took the domestic GF-1 imagery as an example to explore the automated management of domestic imagery data under the ODC framework by developing middleware called ODC_GFTool. The middleware's development environment is .NET Framework 4.0, which supports the current mainstream Windows 10 operating system. Based on GDAL, YamlDotNet and other components, ODC_GFTool achieves spectral information, band information, spatial reference, and other contents from high-resolution image files. In addition, with the strong matrix computing ability of the Matlab software, ODC_GFTool realizes high score image processing and visualization through mixed programming. The main processes of importing domestic satellite data into ODC using ODC_GFTool include defining GF-1 image products, building indexes, and ingesting data. For demenstration, the paper took Poyang Lake as the test area and did an experimental water extraction. Results show that ODC has obvious efficiency advantage compared with traditional data processing tools. The use of ODC for domestic satellite imagery management increased the storage space by 7.2% and reduced the processing time to one third of the traditional methods. In addition, compared to Google Earth Engine and EarthServer, ODC has the following advantages: (1) ODC supports localized database storage, considering the security of some satellite imagery, data storage and computing need to be local, not in the cloud. (b) As an open source earth observation data management framework, ODC facilitates integration and secondary development according to actual project requirements. Research on ODC data management and application for domestic satellite imagery not only can enrich new ways of domestic image data management in China and increase the practical application value of ODC in the world, but also promote international cooperation and respond to the national earth observation infrastructure construction.
Key words: Open Data Cube; ODC; domestic satellite images; data management; image storage; GF-1; Poyang Lake
表2 不同工具处理效率对比Tab. 2 Comparison of processing efficiency of different tools |
处理时间/s | 占用空间/GB | 空间冗余率/% | |
---|---|---|---|
ODC | 79.06 | 15.05 | 7.2 |
IDL | 227.55 | 14.04 | 0.0 |
ArcPy | 245.41 | 14.04 | 0.0 |
[1] |
李国庆, 庞禄申. 公众化驱动的地球观测发展新时代[J]. 中国科学:信息科学, 2017,47(2):193-206.
[
|
[2] |
|
[3] |
李国庆, 张红月, 张连翀, 等. 地球观测数据共享的发展和趋势[J]. 遥感学报, 2016,20(5):979-990.
[
|
[4] |
Committee on Earth Observation Satellites (CEOS). The Open Data Cube Initiative[EB/OL]. [2019-04-30]. https://docs.wixstatic.com/ugd/f9d4ea_1aea90c5bb7149c8a7308 90c0f791496.pdf.
|
[5] |
|
[6] |
|
[7] |
|
[8] |
|
[9] |
何昭欣, 张淼, 吴炳方, 等. Google Earth Engine支持下的江苏省夏收作物遥感提取[J]. 地球信息科学学报, 2019,21(5):752-766.
[
|
[10] |
|
[11] |
|
[12] |
|
[13] |
顾行发, 周翔, 张松梅, 等. 亚洲大洋洲区域综合地球观测系统计划进展[J]. 遥感学报, 2018,22(4):658-671.
[
|
[14] |
|
[15] |
|
[16] |
|
[17] |
|
[18] |
|
[19] |
|
[20] |
|
[21] |
况润元, 谢佳, 张萌, 等. 基于实测数据的鄱阳湖悬浮泥沙粒度遥感反演模式分析[J]. 长江科学院院报, 2017,34(11):132-137.
[
|
[22] |
万松贤, 兰志春, 刘以珍, 等. 基于NDVI指数的鄱阳湖丰水期湿地植被覆盖对水文情势变化的响应[J]. 南昌大学学报(理科版), 2017,41(4):341-348.
[
|
[23] |
方朝阳, 邬浩, 陶长华, 等. 鄱阳湖南矶湿地景观信息高分辨率遥感提取[J]. 地球信息科学学报, 2016,18(6):847-856.
[
|
[24] |
|
[25] |
朱小强, 丁建丽, 夏楠, 等. 一种稳定阈值的湖泊水体信息提取方法[J]. 资源科学, 2019,41(4):790-802.
[
|
[26] |
李国庆, 黄震春. 遥感大数据的基础设施:集成、管理与按需服务[J]. 计算机研究与发展, 2017,54(2):267-283.
[
|
/
〈 |
|
〉 |