Multi-core era: Parallel Programming (2) -- Parallel Implementation Technology

Parallel Program Design (2)) -- Parallel Implementation Technology

Through the introduction of "Parallel Program Design (1)", I believe everyone has a preliminary understanding of the concepts related to parallel systems. But it looks quite scary. A bunch of terms and concepts are a little too big to remember !!

Fortunately, there are not many systems that really need parallel programming. Let's take a look:

Sisd: essentially, it determines that it cannot be parallel;

SIMD: proprietary systems. I believe most people have never touched it;

Misd: there is no such system;

MIMD: essentially to support parallelism.

Therefore, when dealing with parallel program design, we only need to consider systems like MIMD! Thank God, orz !! :)

 

But don't be happy too early. There are many MIMD products. What should I do? MIMD is divided into two types: Shared Memory and message-driven (in fact, distributed memory). We will analyze them from classification to see how to handle them.

Shared Memory:All CPUs share memory, and all CPUs are controlled by one operating system, such as Windows and Linux/Unix. This is the case for popular multi-core and multi-CPU machines;

Message-driven:It is actually distributed memory. The CPU is controlled by different operating systems, and different CPUs communicate through the network. For example, the grid communicates over the Internet, the cluster communicates over the LAN, and the MPP communicates over the proprietary high-speed network.

Through the comparison above, intelligent readers quickly think of the differences between the two parallel program implementation methods:

Shared Memory:Through the Operating SystemMulti-process and multi-threadTo complete parallel tasksInter-process communicationTo complete collaboration;

Message-driven:PassMultiple machinesTo complete parallel tasksMessageTo complete collaboration. (MPP is a physical machine, logically multiple machines ).

Of course, since each processing unit in the message-driven system is an independent machine, the independent machine itself can also achieve parallel processing through shared memory.

 

Well, it was very good. After our analysis and screening, we finally summarized two parallel implementation technologies:Multi-process, multi-thread, multi-Machine Collaboration.

Apply a sentence from Master Hou Jie to fund yourself !!

======================================== Not complete to be continued, next we will introduce Windows and Linux multi-process and multi-thread =======================================