海量遥感数据的高性能地学计算应用与发展分析

doi:10.3724/SP.J.1047.2013.00128

摘要/Abstract

摘要：

航空及航天遥感器的快速发展,使得多源、多时空分辨率的遥感数据成TB级增长,对海量遥感数据的高性能计算与处理提出了更高的要求。据此,当前的遥感应用已经吸收了新型硬件架构计算、集群计算和分布式计算等高性能计算领域的最新技术。本文针对高性能计算处理海量遥感数据的效率问题,分别从分布式并行遥感文件系统和高性能遥感地学计算模式两个方面来论述该问题的研究进展;在此基础上,列举了当前具有代表性的集群和分布式遥感计算平台/系统,并结合具体实验工作,详细阐述了遥感高性能计算平台gDos-IPM(Geospatial Data Operation System-Image Processing Machine)的设计思路;最后总结了高性能遥感地学计算的发展趋势。

关键词: 分布式计算, 并行文件系统, 多核, 集群, 图形处理器

Abstract:

As the amount of remote sensing data is sharply increasing within the continuing development in remote sensors, the exploitation of massive amount of remote sensing data is booming in recent years. Therefore, the computational problems in the applications that involve the large collection of remotely sensed imagery processing, such as global climate change and hazard assessment, arise inevitably. On this point, high performance computing (HPC)-based patterns, including cluster computing, grid computing, cloud computing and computing with hardware such as field-programmable gate arrays (FPGA) and graphic processing units (GPU), are introduced to the applications that concern a huge amount of remote sensing data processing. This paper focuses on the state of the art coping with the challenges that emerge when the massive remote sensing data are processed by the HPC-based platforms. In particular, we review recent developments in the parallel file systems for storing the remote sensing data and high performance geocomputation. Specifically, the HPC-based paradigms delivered in this paper involve cluster-based platform, grid and cloud based environments. Further, the typical examples of the HPC-based platforms that process the massive remote sensing data, comprising the Pixel Factory, the Grid Processing on Demand (G-POD) and the Geospatial Data Operation System-Image Processing Machine (gDos-IPM), are discussed. And the gDos-IPM, a solution for the platform of high performance computing for remote sensing, is described in detail. The gDos-IPM, which integrates the computation and storage resources and involves GPUs, multicore processors and clusters, provides the remote sensing tools about preprocessing and information extraction for massive remote sensing data in the heterogeneous computing environment. Also, it supports a dynamic model for spatio-temporal data and multi-level parallel computing. At the end of this paper, we present a thoughtful view on the challenges of HPC for further remote sensing applications.

Key words: multicore, parallel file system, graphic processing unit (GPU), distributed computing, cluster

杨海平, 沈占锋, 骆剑承, 吴炜. 海量遥感数据的高性能地学计算应用与发展分析[J]. 地球信息科学学报, 2013, 15(1): 128-136.DOI:10.3724/SP.J.1047.2013.00128

YANG Hai-Beng, CHEN Tie-Feng, JIA Jian-Cheng, TUN Wei. Recent Developments in High Performance GeoComputation for Massive Remote Sensing Data[J]. , 2013, 15(1): 128-136.DOI:10.3724/SP.J.1047.2013.00128

参考文献

[1] 骆剑承,周成虎,沈占锋,等.遥感信息图谱计算的理论方法研究[J].地球信息科学学报,2009,11(5):664-669.

[2] 李德仁.论空天地一体化对地观测网络[J].地球信息科学学报,2012,14(4):419-425.

[3] Guo H. Understanding global natural disasters and the role of earth observation[J]. International Journal of Digital Earth, 2010,3(3):221-230.

[4] 徐冠华. 全球变化研究与高性能计算 [R].济南:中国计算机学会,2011.

[5] Goodchild M F, Guo H, Annoni A, et al. Next-generation digital earth[J]. Proceedings of the National Academy of Sciences, 2012,109(28):11088-11094.

[6] 霍严梅,杨可新,胡亮,等.并行文件系统研究综述[J].小型微型计算机系统,2008,29(9):1631-1636.

[7] 康俊锋.云计算环境下高分辨率遥感影像存储与高效管理技术研究 [D].杭州:浙江大学,2011.

[8] Park S, Kim T, Park J, et al. Parallel skyline computation on multicore architectures [C]. Proceedings of the 25th International Conference on Data Engineering, ICDE 2009, March 29-April 2 2009, Shanghai, China, 2009.

[9] ERDAS IMAGINE Brochure. http://geospatial.intergraph.com/Libraries/Brochures/ERDAS_IMAGINE_Brochure.sflb.ashx.

[10] OpenMP.org. http://openmp.org/wp/.

[11] Phillips R D, Watson L T, Wynne R H. Hybrid image classification and parameter selection using a shared memory parallel algorithm[J]. Computers & Geosciences, 2007,33(7):875-897.

[12] 朱志文,沈占锋,骆剑承.改进SIFT点特征的并行遥感影像配准[J].遥感学报,2011,15(5):1024-1039.

[13] 邹贤才,李建成,汪海洪,等. OpenMP 并行计算在卫星重力数据处理中的应用[J].测绘学报,2010(6):636-641.

[14] Lambers M, Kolb A, Nies H, et al. GPU-based framework for interactive visualization of sar data [C]. Proceedings of IEEE International Geoscience and Remote Sensing Symposium 2007, July 23-28, 2007, Barcelona, Spain, 2007.

[15] 靳海亮,卢小平,刘慧杰.利用可编程GPU硬件进行大规模真实感地形绘制[J].武汉大学学报:信息科学版,2010(2):143-146.

[16] 罗岱,谢茂金,曹卫群,等.基于GPU编程的地形可视化[J].中国图象图形学报,2008,13(11):2244-2249.

[17] Setoain J, Prieto M, Tenllado C, et al. Parallel morphological endmember extraction using commodity graphics hardware[J]. Geoscience and Remote Sensing Letters, IEEE, 2007,4(3):441-445.

[18] Sanchez S, Paz A, Martin G, et al. Parallel unmixing of remotely sensed hyperspectral images on commodity graphics processing units[J]. Concurrency and Computation: Practice and Experience, 2011,23(13):1538-1557.

[19] Qin C Z, Zhan L J. Parallelizing flow-accumulation calculations on graphics processing units-From iterative DEM preprocessing algorithm to recursive multiple-flow-direction algorithm[J]. Computers & Geosciences, 2012(43):7-16.

[20] 陈帅,李刚,张颢,等.SAR图像压缩采样恢复的GPU并行实现[J].电子与信息学报,2011,33(3):610-615.

[21] 肖汉,张祖勋.基于GPGPU的并行影像匹配算法[J].测绘学报,2010(1):46-51.

[22] Song C, Li Y, Huang B. A GPU-accelerated wavelet decompression system with SPIHT and Reed-Solomon decoding for satellite images[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing,, 2011,4(3):683-690.

[23] 张兵.智能遥感卫星系统[J].遥感学报,2011,15(3):415-431.

[24] Gonzalez C, Nchez S A, Paz A, et al. Use of FPGA or GPU-based architectures for remotely sensed hyperspectral image processing[J]. Integration, the VLSI Journal, 2012, 46(2):89-103.

[25] Gonzalez C, Resano J, Mozos D, et al. FPGA implementation of the pixel purity index algorithm for remotely sensed hyperspectral image analysis[J]. EURASIP Journal on Advances in Signal Processing, 2010(969806):1-13.

[26] 刘虹,孙春燕,丁雷.一种基于RICE算法的遥感图像数据的实时无损压缩与解压缩算法的研究及FPGA实现[J].量子电子学报,2005,22(1):19-24.

[27] Besiris D, Tsagaris V, Fragoulis N, et al. An FPGA-based hardware implementation of configurable pixel-level color image fusion[J]. IEEE Transactions on Geoscience and Remote Sensing, 2012,50(2):362-373.

[28] 史园莉.遥感数据集群处理系统架构设计与实现. 北京:中国测绘科学研究院,2010.

[29] Lee C A, Gasster S D, Plaza A, et al. Recent developments in high performance computing for remote sensing: A review[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2011,4(3):508-527.

[30] Columbia Supercomputer. http://www.nas.nasa.gov/hecc/resources/columbia.html.

[31] Pleiades Supercomputer. http://www.nas.nasa.gov/hecc/resources/pleiades.html.

[32] Collaborative Supercomputing for Global Change Science. http://ecocast.arc.nasa.gov/pubs/pdfs/2011/NEX_Eos_2011.pdf.

[33] 李文亮.GPU集群调度管理系统关键技术的研究 [D].武汉:华中科技大学,2011.

[34] 肖汉.基于CPU+ GPU的影像匹配高效能异构并行计算研究 [D].武汉大学,2011.

[35] 沈占锋,骆剑承,吴炜,等.采用并行均值漂移算法实现农林地块边界的精确提取[J].中国图象图形学报,2011,16(9):1689-1695.

[36] 吴炜,沈占锋,骆剑承,等.均值漂移高分辨率遥感影像多尺度分割的集群实现[J].计算机工程与应用,2009,45(34):7-9,24.

[37] Plaza A, Du Q, Chang Y L, et al. High performance computing for hyperspectral remote sensing[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing,2011,4(3):528-544.

[38] Plaza A, Valencia D, Plaza J, et al. Commodity cluster-based parallel processing of hyperspectral imagery[J]. Journal of Parallel and Distributed Computing, 2006,66(3):345-358.

[39] Hawick K A, Coddington P D, James H A. Distributed frameworks and parallel algorithms for processing large-scale geographic data[J]. Parallel Computing,2003,29(10):1297-1333.

[40] Aloisio G, Cafaro M. A dynamic earth observation system[J]. Parallel Computing,2003,29(10):1357-1362.

[41] 周成虎,骆剑承.高分辨率卫星遥感影像地学计算[M].北京:科学出版社,2009.

[42] Shen Z F, Luo J C, Zhou C H, et al. System design and implementation of digital-image processing using computational grids[J]. Computers & Geosciences, 2005,31(5):619-630.

[43] Shen Z, Luo J, Zhou C, et al. Architecture design of grid GIS and its applications on image processing based on LAN[J]. Information Sciences, 2004,166(1):1-17.

[44] Xue Y, Chen Z, Xu H, et al. A high throughput geocomputing system for remote sensing quantitative retrieval and a case study[J]. International Journal of Applied Earth Observation and Geoinformation, 2011,13(6):902-911.

[45] grid processing on demand. http://gpod.eo.esa.int/.

[46] GEO Grid. http://www.geogrid.org/en/index.html.

[47] Ghemawat S, Gobioff H, Shun-Tak L. The Google File System [C]. 19th Symposium on Operating Systems Principles (conference), Lake George, NY, 2003.

[48] Dean J, Ghemawat S. MapReduce: Simplified data processing on large clusters[J]. Communications of the ACM, 2008,51(1):107-113.

[49] Chang F, Dean J, Ghemawat S, et al. Bigtable: A distributed storage system for structured data[J]. ACM Transactions on Computer Systems (TOCS), 2008,26(2):4.

[50] 霍树民.基于Hadoop的海量影像数据管理关键技术研究 [D].长沙:国防科学技术大学,2010.

[51] 遥感云服务. http://www.casc.ac.cn/templet/default/ShowArticle.jsp?id=147&pagen=distrucenter

[52] 杨靖宇.遥感影像GPU并行化处理技术与实现方法 [D].郑州:解放军信息工程大学,2008.

[53] 胡晓东,骆剑承,沈占锋,等.高分辨率遥感影像并行分割结果缝合算法[J].遥感学报,2010,14(5):922-927.

[54] Bignone F. Processing of stereo scanner: from stereo plotter to pixel factory [C]. the 49th Photogrammetric Week in Stuttgart, 1-5 September, 2003.

[55] 曹敏,史照良.新一代海量影像自动处理系统"像素工厂"初探[J].测绘通报,2006(10):55-58.

[56] PIXEL FACTORY. http://www.astrium-geo.com/files/pmedia/public/r367_9_geo_0014_pixelfactory_en_2012_03.pdf.

[57] 欧洲宇航防务集团—INFOTERRA及其中国业务. http://www.eads.com/eads/china/cn/our-company/EADS-China/astrium/TO-DELETE-.html

[58] Huang S, Siegert F. ENVISAT multisensor data for fire monitoring and impact assessment[J]. International Journal of Remote Sensing, 2004, 25(20):4411-4416.

[59] Pritchard M, Maddison B, Abolins J, et al. (A) ATSR Series Archive at the NEODC [C]. Proceedings of the MERIS (A)ATSR Workshop 2005 (ESA SP-597), 26-30 September 2005 ESRIN, Frascati, Italy, 2005.