Research and implementation of HDFs ha under cloud Platform

Research and implementation of HDFs ha under cloud Platform

Wang Gang, Northwestern University

In this paper, the existing problems are studied in depth. This paper analyzes the reason of Namenode single point fault, studies the design idea of the existing scheme, analyzes the feasibility of porting Hadoop to the existing cloud platform, and puts forward the high availability scheme of Namenode based on the virtualization platform. The scheme guarantees the high availability of namenode from two aspects. On the one hand, the implementation of a Namenode host load migration strategy, when the host load is too large, this strategy will be namenode real-time migration to a suitable host, to ensure that the Namenode provide uninterrupted service. On the other hand, for the namenode problem, an automatic switching mechanism is implemented by adding a standby Namenode (Standby Namenode), which realizes the automatic switching of the standby node when the Namenode is down, and the node continues to provide services externally. Finally, by deploying the experimental environment, simulating the overload operation of the host and Namenode single point fault, the feasibility of the scheme is validated and the performance is evaluated.

Research and implementation of HDFs ha under cloud Platform