Research on the cloud computing framework of plagiarism detection based on spark

Research on the cloud computing framework of plagiarism detection based on spark

Hu Haiho

Plagiarism detection is essentially a computational problem of text similarity, which needs to be quickly and accurately detected in the mass anthology, which consumes a lot of time and resources, and is a complicated process of computation and data-intensive. Using distributed computing is one of the effective methods to improve the detection efficiency. This paper puts forward a set of distributed plagiarism detection cloud computing framework based on spark the framework uses the MapReduce computing framework, which is supported by the cluster explorer Apache Mesos, which supports memory-resident, and the distributed computing cluster, which is composed of distributing Hadooop file systems. The test results show that this frame is more traditional than hadooop
The layout calculation frame has a great improvement in efficiency.

Research on the cloud computing framework of plagiarism detection based on spark