Research on distributed polar Graph construction algorithm based on MapReduce

Research on distributed polar Graph construction algorithm based on MapReduce

Beijing Jiaotong University Nannan

With the rapid development of cloud computing technology, many research and applications related to large-scale data processing have been migrated to cloud computing environment, such as data mining, network search, image processing and bioinformatics analysis. The large scale data processing technology is also the research hotspot in the field of high performance computing. In the graph theory research, the polar graph constructs the algorithm as the Polar graph theory important research content, receives the people's attention more and more. The polar graph is a graph which satisfies certain contract conditions and has the most number of edges, and its construction algorithm produces large-scale critical Atlas cooperation as intermediate data. The traditional algorithm of serial polar graph construction will become inefficient because of the significant increase in the number of critical graphs to be processed. The MapReduce model is a programming model that is often used to study cloud computing-related issues, and the Hadoop project is an open source implementation of the model, which can greatly simplify the difficulty of distributed programming. A distributed polar graph construction algorithm based on MapReduce is presented in this paper. Based on the in-depth analysis of the algorithm of the serial polar graph, a parallel implementation scheme is proposed, and then a distributed parallel algorithm for the polar graph construction is implemented on the Hadoop Distributed Foundation platform. Each map task processes the input data of critical graphs that have been divided into blocks, and the reduce task is responsible for merging the intermediate critical graph data generated by all map tasks and obtaining the final graph set. In this paper, a series of comparative tests are carried out to verify the effectiveness and efficiency of the distributed polar Graph construction algorithm. Experimental results show that the algorithm can construct a set of polar graphs with no more than 28 vertices and no hexagonal. Compared with the serial algorithm, the overall speedup and execution efficiency of the algorithm are 2.2645 and 75.48% respectively. In particular, when constructing a set of critical graphs with 19 vertices without hexagonal and the number of edges not less than 35, the acceleration ratio and execution efficiency reach 2.7423 and 91.41% respectively.

Research on distributed polar Graph construction algorithm based on MapReduce