Building high performance distributed parallel computing environment based on MPICH2 in Linux system

MPICH2 provides a platform for parallel and distributed processing of large data under existing hardware and software architectures. This article will describe how to build a high-performance distributed parallel computing environment based on MPICH2 in Linux systems.

MPI (Message passing Interface) is a messaging standard developed by the MPI Committee, which defines a series of programming interfaces for interprocess communication in a distributed environment, currently with MPI-1 and MPI-2 two versions. MPICH2 is a concrete implementation of MPI, MPICH2 maintains high levels of portability while ensuring high performance, including AIX, Linux (IA32 and were), Mac OS (PowerPC and Intel), Solaris (32-bit and 64 And Windows, and many other platforms. MPICH2 is an open source project.

Prepare system and http://www.aliyun.com/zixun/aggregation/16327.html "> network environment

The hardware and software system in this paper is as follows:

Three 64-bit Red Hat Enterprise Linux 5.5 machines are installed in the LAN. The hostname of the three machines are S1, S2 and S3 respectively. We will use S1 as the SERVER,S2 and S3 in the MPICH2 environment as the worker in the MPICH2 environment. Python2.6, SSH Services, and NFS services are already installed on the Linux system.

Step 1 Configure SSH

MPICH2 provides a series of commands for managing the processes in a distributed computing environment, which may be running on different machines, which operate in the background with SSH, so it is convenient to configure the Server and Worker to have a password-free SSH login.

Execute the following command on the S1:

[Root@s1 ~]# ssh-keygen-t RSA [root@s1 ~]# ssh-copy-id-i ~/.ssh/id_rsa.pub root@s2] [root@s1 ~]# ssh-copy-id-i ~/.ssh/id_ Rsa.pub ROOT@S3

Step 2 Configuring NFS

MPICH2 does not provide a unified file system, so the shared file system provided by the operating system needs to share data between the Server and multiple Worker. Here is an example of NFS.

First set up NFS shared directories on Server S1:

Create a shared directory: [root@s1 ~]# Mkdir/share adds the directory to the NFS shared directory list, adding the following configuration in/etc/exports:/share * (Sync,rw,no_root_squash) to start the NFS service on S1: [ROOT@S1 ~]# Service NFS Start

Then mount the S1 shared NFS directory on Worker S2 and S3:

[Root@s2 ~]# Mkdir/share [root@s2 ~]# Mount S1:/share/share [root@s3 ~]# mkdir/share] [root@s3 ~]# Mount S1:/share/share