Data processing technology of cloud Platform image

Data processing technology of cloud Platform image

Liu Tangzheng Yao Hu Chengyu Liang

Aiming at the inefficiency of the MapReduce computing model under the Hadoop cloud platform in the process of graph data, this paper puts forward a kind of graph data processing calculation frame--mybsp similar to Google Pregel. Firstly, it analyzes the operation mechanism and deficiencies of mapreduce; The structure, workflow and main interface of mybsp frame are expounded, and the PageRank algorithm based on MYBSP framework is designed and implemented on the basis of analyzing the principle of PageRank graph processing algorithm. The experimental results show that Compared with mapreduce algorithm, the algorithm of Graph data processing based on mybsp frame improves the performance of iterative processing 1.9~3 times. The execution time of mybsp algorithm is reduced by 67%, which can satisfy the application foreground of efficient processing of graph data.

Data processing technology of cloud Platform image