关键词: 集群, 遥感影像金字塔, 消息传递接口, 并行

Abstract:

With the rapid development of remote sensing (RS) image acquisition and processing technology, the spatial resolution and temporal resolution of RS image have been greatly improved. For large-scale RS image, traditional sequential pyramid building algorithms have been found difficult to meet the quick browsing requirements. Technologies and facilities of high performance computing have become more and more feasible to researchers. Taking the advantages of multi-core, multi-node cluster computing environments and parallel processing mechanism is turning to be an inevitable trend. Some of recent works explore the efficiency and flexibility of parallel pyramid building methods. However, these methods all have deficiencies. For example, GPU-based parallel method is hardware-aware, the improvement of its performance is limited on a single node, and the system architecture will be too complicated when applied in a cluster environment. Whereas the distributed clusterbased method requires the data to be distributed to be stored in different nodes, and the complete pyramid file needs to be merged, which is excessive time consuming. Therefore, using the high-performance disk-shared cluster is an alternative mechanism for achieving the parallel building pyramid of large-scale RS images. In this paper, we proposed a parallel algorithm based on Message Passing Interface (MPI). Based on this, the whole pyramid building task is decomposed into several subtasks. The result of each subtask can be written to the same pyramid file simultaneously by incorporating MPI/IO. The algorithm can greatly improve the performance of pyramid building through parallel resampling and parallel I/O. Specially, with regard to the multi-band pyramid file stored in BIP format, a parallel I/O strategy using file view was proposed to improve the performance of parallel writing. Experimental results show that our algorithm has better acceleration effect compared to the sequential method, and there is a positive correlation between the acceleration effect and the image size. For large remote sensing images (in our case it is 46G), the performance of our parallel algorithm can be approximately 10 times faster than GDAL.

Key words: remote sensing image pyramid, MPI (Message Passing Interface), parallel, cluster