Research and implementation of clustering and convex package algorithm under MapReduce framework

Research and implementation of clustering and convex package algorithm under MapReduce framework

Chengdu University of Technology Zhaoju

First of all, this paper studies the generation and value growth of large data, explains the necessity of improving the execution efficiency of data mining algorithm, and gives a general introduction to the technology and tools that support large-data processing nowadays. Then the paper studies the running mechanism of Hadoop file system, the stored procedure and the programming model of MapReduce framework, and the operation principle. Secondly, the data are distributed and processed on a certain scale Hadoop cluster to evaluate the performance of the whole cluster to see if it is suitable for the standard data Mining task. In the MapReduce framework, the search and sequencing tasks are performed to analyze the effects of different system configurations. At the same time, the K clustering algorithm is provided for iterative implementation in MapReduce framework. Finally, the traditional computer graphics convex package algorithm is implemented in parallel with the MapReduce frame, and the experimental data is simulated with the K algorithm, which shows that the convex packet algorithm can be applied to the research of the data mining algorithm in the MapReduce framework, The result of data mining algorithm is introduced in data compression.

Research and implementation of clustering and convex package algorithm under MapReduce framework