Rack server cluster solution for LINUX

LINUX system rack server cluster solution-Linux Enterprise Application-Linux server application information. The following is a detailed description. More SD/hd nle workstation NLE cluster 3DMX rendering-3D animation, rendering, video animation, broadcast-level NLE workstation. Hosting, streaming media, games, movies, video servers

1. What is a server cluster?

With the explosive growth of the Internet, the Internet is increasingly closely related to people's lives, and transactions over the Internet are becoming more and more important. In recent years, the annual growth of e-commerce has exceeded 100%. The workload of servers also increases rapidly, so servers (especially a popular WEB server) are easily overloaded during peak hours.

On the other hand, from the purely scientific computing tasks of the 1946 s to a large number of complicated information processing tasks, the workload is getting bigger and bigger, and the processing capability is getting faster and faster. Therefore, the computer industry has to constantly study faster processors and memories to meet this demand.

However, scientists realized that the speed and development of a single processor is limited. Why can't we make computers work together like humans! At this moment, we started the research of "Parallel Computing.

For example, we have set up a WWW server with an e-commerce website built on it. However, over time, the website becomes more and more famous, and the click rate is getting higher and higher, WWW server load is getting higher and higher. In this case, we must improve the capability of the WWW server to meet the increasing service requests. At this time, we are faced with two options:

1) Upgrade the WWW server to use a faster CPU and more memory to make it more powerful. However, the increasing number of service requests will overload the server and need to be upgraded again, in this way, it is in a strange circle of upgrading. Also, you have to consider how to continue the service during the upgrade. Can you stop it!

2) Add WWW servers to allow multiple servers to complete the same service.

This method is a server cluster, which greatly improves system performance through parallel technology. That is, the main content of this chapter. This method has good scalability and can maximize the use of existing investment.

1. cluster concept

A cluster is a group of independent computer systems that constitute a loosely coupled multi-processor system. They communicate with each other through a network. Applications can transmit messages through the network shared memory to implement distributed computers.

In recent years, the microprocessor, memory, bus technology, and network technology have been greatly improved, and the parallel technology of software has also been greatly improved, this makes it possible for a group of cheap pcs to work with workstations, and even compete with super computers with powerful chip processing capabilities.

For example, a cluster system composed of 16 general microprocessors can provide hundreds of millions of sub-floating point computers, and the total cost is less than 0.4 million yuan.

Unfortunately, building a cluster is not a simple event. The components of the cluster must be adjusted according to the main application to be run to make it run in the optimal state. The result of this factor is that there is not only one standard solution for building a cluster system. In this chapter, we will see many construction solutions.

Cluster technology is a cutting-edge field of computer system structure. I only have a little understanding about it. I hope this article can be used as a reference.

1. 2. Parallel technology

This is a very simple example of building a four-node small cluster system. It is built on a Linux operating system and implemented through the MPICH software package, I hope this small example will give you a basic understanding of cluster system construction.

2. Use MPICH to build a four-node cluster system

This is a very simple example of building a four-node small cluster system. It is built on a Linux operating system and implemented through the MPICH software package, I hope this small example will give you a basic understanding of cluster system construction.

2.1 required devices

1) Four PCs using the Pentium II processor, each with 64 MB memory, a hard disk larger than 2 GB, and a disc drive with an EIDE interface.

2 ). five 9332 m fast Ethernet NICs, such as SMC 10/100 EtherPower (four cards are used to connect nodes in the cluster, and the other one is used to connect one node in the cluster to other networks .)

3). Five RJ45 cables that are sufficient to connect each node in the cluster system using five unshielded twisted pair wires

4). 1 Fast Ethernet (100BASE-Tx) hub or switch

5). One Linux installation disk

2.2 build description

If you are not familiar with computer hardware, it is difficult to perform the following build steps. If so, contact experienced professionals for help.

1. Prepare a PC using the Pentium II processor. Make sure that all PCs are not powered on. Open the PC's chassis and install two fast Ethernet cards on the PC that is prepared to connect to other devices on the network, install a Fast Ethernet card on other PCs. Of course, don't forget to add additional memory. Confirm to complete the chassis on the back cover and connect to the power supply.

2. Use four RJ45 cables to connect four PCs to the hub or switch of the Fast Ethernet. Use the remaining RJ45 line to connect the extra Ethernet Card (the one connected to other networks, so that the Organization can use the cluster) connect to the LAN of the organization (assuming that your LAN is also fast Ethernet) and turn on the power.

3. Use the LINUX installation disk to install on each PC. Be sure to install the C compiler and c lib library in LINUX. When configuring TCP/IP, we recommend that you specify 192.168.1.1, 192.168.1.2, 192.168.1.3, and 192.168.1.4 for each of the four PCs. The first PC is your server node (the one with two NICS ). On this server node, you must specify an IP address that matches the LAN.

4. After Linux is installed on all PCs, edit the/etc/hosts file of each machine to include the following lines:

192.168.1.1 node1 server

192.168.1.2 node2

192.168.1.3 node3

192.168.1.4 node4

Edit the/etc/hosts. equiv file of each machine to include the following lines:

Node1

Node2

Node3

Node4

The following configurations enable the MPICH's p4 policy to execute distributed parallel processing applications.

1. Create a/mirror directory on the server node, configure it as an NFS server, and add a line in the/etc/exports file:

/Mirror node1 (rw) node2 (rw) node3 (rw) node4 (rw)

2. Create a/mirror directory on other nodes and add a line to the/etc/fstab file:

Server:/mirror nfs rw, bg, soft 0 0

3. The/mirror directory is output from the server and loaded on each client to distribute software tasks among nodes.

4. Install MPICH on the server node. MPICH documentation can be found in

Http://www.mcs.anl.gov/mpi/mpich/docs.html.

5. any cluster user (you must create an identical user on each node), You must create a subdirectory under the/mirror directory, such as/mirror/username, it is used to store MPI programs and shared data files. In this case, you only need to compile the MPI program on the server node, and then copy the compiled program to its subdirectory under the/mirror directory, then run the MPI program using the p4 MPI policy under the/mirror directory that belongs to it.

2.3 MPICH Installation Guide

1. If you have gunzips, download mpich.tar.gzin in d, and then download mpich.tar. Z. You can get it in the http: // logs directory. (If you think this is too big, you can go to pub/mpi/mpisplit to take several packets that are separated into blocks, and then use the cat command to merge them)

2. decompress: gunzip ?; C mpich.tar.gz | tar xovf-(or zcat mpich.tar. Z | tar xovf -)

3. Enter the mpich directory

4. run :. /configure selects a set of parameter groups suitable for your actual software and hardware environment for MPICH. If you are not satisfied with these default parameters, you can configure them by yourself (For details, refer to the MPICH Configuration documentation ). It is best to select a specified directory to install and configure MPICH, for example:

../Configure-prefix =/usr/local/mpich-1.2.0

5. run: make> & make. log takes a long time, and the time spent on different hardware environments may vary, from 10 minutes to 1 hour, or even more.

6. (Optional) on the workstation network or a separate workstation, edit mpich/util/machines. xxx (xxx is the name of MPICH for your machine architecture, which you can easily recognize) to reflect the local host name of your workstation. You can skip this step. This step is not required in the cluster.

7. (optional) Compile and run a simple test program:

Cd examples/basic

Make cpi

Ln ?; S ../bin/mpirun

./Mpirun ?; Np 4 cpi

At this point, you run an MPI program on your system.

8. (optional) build other MPICH environments. Using a secure service for the ch_p4 policy will speed up any startup. You can execute the following command to build the environment:

Make serv_p4

(Serv_p4 is a newer version of P4 Security Service, which is included in MPICH 1.2.0). nupshot is a faster version of upshot, but he needs the source code of tk 3.6. If you have this package, you can use the following command to build it:

Make nupshot

9. (optional) If you want to install MPICH in a public place for others to use it, you can execute:

Make install or bin/mpiinstall

You can use the-prefix option to specify the MPICH installation directory. After installation, the include, lib, bin, sbin, www, and man directories and a small example directory are generated,

(Optional) You can notify all users about how to compile and execute An MPI program.

2.4 fee statistics

The total cost of this small cluster solution is roughly as follows:

Device fee:

4 Intel XOEN 5110/1 GB memory, GB hard drive with EIDE CD-ROM machines

Five Enis,

One set machine for fast Ethernet,

Five twisted pair wires,

Software fee,

LINUX installation CD,

MPICH,

Such a cluster system can be used for a large amount of scientific computing and information processing, and its cost effectiveness is really great.