Compound similarity analysis system based on cloud computing technology

Compound similarity analysis system based on cloud computing technology

Fudan University Li Jiehui

This paper studies the theory, characteristics and key technologies of cloud computing, explores the distributed data processing programming model MapReduce and its operating mechanism and principle of open source to realize Hadoop, and puts forward a distributed solution for the molecular structure comparison problem. Mainly did the following research: 1 The cloud computing technology and the molecular similarity comparison algorithm, combined with the advantages of Hadoop cloud computing technology, for Hadoop can not be directly applied to the molecular similarity of the problem, proposed a processing method based on index files, Apply Hadoop cloud computing technology to the field of molecular similarity comparison. 2 The feasibility of the method is verified by experiments. The experimental results show that the system has 16 CPU parallel processing, the average of this scheme is 14.4 times times faster than the single machine processing system, which satisfies the demand of large scale computation of the molecular similarity comparison system. 3 in view of the possible data skew problem of Hadoop's partition algorithm, a uniform polling partitioning algorithm is proposed, which reduces the effect of data skew, and experiments show that the algorithm can improve the efficiency of the whole system by more than 5%. 4 The MapReduce frame has fault-tolerant function, that is, when a compute node fails, the master node automatically assigns the node computing task to another idle machine, but the data block of the compute node is processed anew.

Http://bbs.chinacloud.cn/showtopic-16942.aspx