Parallel computing of distributed computing

1 , Parallel computing

Parallel computing or parallel computing is relative to serial computing. It is an algorithm that can execute multiple instructions at once to improve the computational speed and solve large and complex computational problems by enlarging the problem solving scale the so-called parallel computing is divided into time parallel and spatially parallel Parallel in time refers to the pipelining technology and the parallel in space refers to the execution computation with multiple processors concurrently. This is a multi-core processor in the context of the definition of parallel computing, from the macro point of view, this is a computer application of the vertical extension of the server, here we do not open the discussion.

In the parallel computation of distributed computing, we must first split the task, separate the different tasks, divide and compute, and summarize the final calculation data.

or to the end of the year, to do a lot of reports, the leadership of the need to arrange the report, I and my team to do statistics. Asynchronous computing does not reduce the time the leader gets the final result; we have to think of a way, I as a dispatch system, the need to split the report, found that need 8 Sets of data, I arrange 8 handsome guys to get the data, which 8 Siege Lions to give me the data, I follow the rules to complete the report, and provide to the leadership. This is probably a simple parallel computing model.

splitting and merging is a great idea in distributed computing, and this is also Map/reduce of thought.

2 , Map/reduce

Map/reduce , it thinks that all computing tasks can go through two processes from split to summary, that is, as long as the Map and the Reduce can describe all of the computational tasks, like using 0 and the 1 will be able to store all the data.

MapReduce The reliability of the data is achieved by dividing the big work into different small jobs and distributing them to different workers. Each individual worker handles the work of the distribution and then returns the results and status to the parent node. If a node fails, the primary node discovers and logs the failed node and distributes the work assigned to the node to another node.

3 , Map/reduce Execution Process:

1.Master divide the input file into M part, usually 16M – 64M per copy

2.master select in idle State worker assign worker The role of map worker assign The process of task piece transfer to worker on

3.map worker read after getting the task task key/value pairs worker map worker map worker partition function r keykey/value

4. when map completes the task , the location of these partition files on the local disk will be notified Master.

5.Masterafter receiving the task completion message, look forIdleState ofworker, inform him to goReduce, this time theworkeris thatreduce worker(It's worth noting that the task has just been completedmap workermay assumereduce worker's role),reduce workerthroughRPCReadmap workerThe intermediate files on the disk to the local disk, after reading,reduce workerto sort it,If the intermediate file is too large, it needs to be sorted externally.

Master not a random one . worker to do Reduce , try to get a reduce worker do contain the same Key of the Partition file.

For example:Map Work1includeRaPartitionfiles,Keyto beK1,K2...... KR,Masterwill put allMap Worksproduced byPartitionfile containsK1as far as possible to the samereduce worker, and of course there is a load balancing problem.

6.Reduce Work writes the merged result to a file because a reduce worker try to deal with one Key of the Partition file, so when reduce worker after the merge is complete, all of the included Key the results are in this file.

7.Reduce Work transfer the results of the merger back to Master

8. When all the reduce workers are complete,master gets the R A result file (one per reduce worker ) that wakes up the user process and gives the R result file to the client program.

4 , Map/reduce Optimized

1. Merging Functions

Mapoften produce a lot ofKey/valueYes, like1million of{a,1}, these are to beReduceto merge, you canReducebefore work there are merge functions onMapin theKey/valueto merge. It seems that merging functions andReducefunction, in fact, it's different, and the merge function mergesMapof localKey/valueYes, the resulting result is writtenMapthe localBuffer;Reducethe functions are merged from differentMapof thePartitionfile, the resulting results are passed backMaster.

2. Fault- tolerant processing

Clustera large cluster, machine failure is inevitable, soMasterto perform fault-tolerant processing. In theMasterWithin or within subgroups of all clusters.workerthe state of being, constantly getting workworkerthe heartbeat, ifworkerdead, then will thisworkermarked as Downand reassign the tasks he performed above to anotherworker. But ifMasterWhat if it dies? Because there is only one in the clusterMaster, you need to recover from the logs (refer to the Distributed File System).

Parallel computing of distributed computing