地球信息科学学报 ›› 2015, Vol. 17 ›› Issue (5): 515-522.doi: 10.3724/SP.J.1047.2015.00515

• 本期要文(可全文下载) • 上一篇    下一篇

基于MPI的大规模遥感影像金字塔并行构建方法

赫高进, 熊伟, 陈荦, 吴秋云, 景宁   

  1. 国防科学技术大学电子科学与工程学院, 长沙410073
  • 收稿日期:2015-01-04 修回日期:2015-02-22 出版日期:2015-05-10 发布日期:2015-05-10
  • 通讯作者: 熊伟(1976-),男,副教授,研究方向为地理信息系统与空间数据库。E-mail:xiongwei@nudt.edul.cn E-mail:xiongwei@nudt.edu.cn
  • 作者简介:赫高进(1990-),男,硕士生,研究方向为高性能地理计算、内存空间数据库。E-mail:gaojinhejs@126.com
  • 基金资助:

    国家高技术研究发展计划"( 863"计划)项目"面向新型硬件架构的复杂地理计算平台"(2011AA120300);国家自然科学基金项目"集群环境下内存空间数据库管理与查询技术研究"(41471321)。

An MPI-based Parallel Pyramid Building Algorithm for Large-scale RS Image

HE Gaojin, XIONG Wei, CHEN Luo, WU Qiuyun, JING Ning   

  1. College of Electronic and Engineering, National University of Defense Technology, Changsha 410073, China
  • Received:2015-01-04 Revised:2015-02-22 Online:2015-05-10 Published:2015-05-10

摘要:

影像金字塔是实现影像数据多分辨率组织的重要方式,是提高影像可视化性能的有效手段。传统串行金字塔构建算法,对大规模影像数据的构建性能已无法满足遥感影像快速浏览的预处理需求。故此,其成为一个亟待解决的问题,而利用多核、多节点的高性能集群计算环境和并行机制是一个重要的技术途径。本文在共享外存的高性能集群环境下,提出使用消息传递接口(MPI)的金字塔并行构建算法,对构建遥感影像金字塔过程中的重采样与I/O 过程进行并行处理,大大缩短了遥感影像金字塔构建时间。实验结果表明:(1)该算法比传统串行构建方法的加速效果明显,对于单波段遥感影像,其加速效果可达到GDAL的5 倍以上,而对于多波段遥感影像,加速效果可达到GDAL的2 倍以上;(2)遥感影像数据量越大,并行构建算法加速效果越显著,对于大规模的遥感影像,本文提出的金字塔并行构建算法的速度可达到GDAL的10 倍左右。

关键词: 集群, 遥感影像金字塔, 消息传递接口, 并行

Abstract:

With the rapid development of remote sensing (RS) image acquisition and processing technology, the spatial resolution and temporal resolution of RS image have been greatly improved. For large-scale RS image, traditional sequential pyramid building algorithms have been found difficult to meet the quick browsing requirements. Technologies and facilities of high performance computing have become more and more feasible to researchers. Taking the advantages of multi-core, multi-node cluster computing environments and parallel processing mechanism is turning to be an inevitable trend. Some of recent works explore the efficiency and flexibility of parallel pyramid building methods. However, these methods all have deficiencies. For example, GPU-based parallel method is hardware-aware, the improvement of its performance is limited on a single node, and the system architecture will be too complicated when applied in a cluster environment. Whereas the distributed clusterbased method requires the data to be distributed to be stored in different nodes, and the complete pyramid file needs to be merged, which is excessive time consuming. Therefore, using the high-performance disk-shared cluster is an alternative mechanism for achieving the parallel building pyramid of large-scale RS images. In this paper, we proposed a parallel algorithm based on Message Passing Interface (MPI). Based on this, the whole pyramid building task is decomposed into several subtasks. The result of each subtask can be written to the same pyramid file simultaneously by incorporating MPI/IO. The algorithm can greatly improve the performance of pyramid building through parallel resampling and parallel I/O. Specially, with regard to the multi-band pyramid file stored in BIP format, a parallel I/O strategy using file view was proposed to improve the performance of parallel writing. Experimental results show that our algorithm has better acceleration effect compared to the sequential method, and there is a positive correlation between the acceleration effect and the image size. For large remote sensing images (in our case it is 46G), the performance of our parallel algorithm can be approximately 10 times faster than GDAL.

Key words: remote sensing image pyramid, MPI (Message Passing Interface), parallel, cluster