Design and implementation of a file prefetching model oriented to Distributed file system

Design and implementation of a file prefetching model oriented to Distributed file system

Shi Ming Liu Yi Tangge Real

How to provide stable and efficient file I/O performance for upper application and computation is the hotspot of distributed File system performance research. This paper analyzes the common characteristics of the Distributed file system in the design mechanism, and proposes a general heuristic document Prefetching model based on this, and selects the HDFs platform for system implementation. The heuristic file prefetching is transparent to the upper layer, adopts the method of establishing the prefetch thread pool in the file system, and realizes the data storage file of the file block as the prefetch unit in the Distributed File system. This kind of design idea has certain universality, it is suitable to popularize in many kinds of distributed file system. Experimental results show that the heuristic file prefetching can effectively improve the I/O performance of Distributed File system.

Design and implementation of a file prefetching model oriented to Distributed file system