Linux high-performance computing cluster-Beowulf Cluster

Linux high-performance computing cluster-Beowulf cluster-Linux general technology-Linux programming and kernel information, the following is a detailed description. Author: Jin Ge
　　
This article is the second part of the High-Performance cluster series. In this article, the author first introduces the history and classification of Beowulf clusters, and then focuses on the system architecture and components of Beowulf clusters: hardware, network, software and applications.

　　 1. What is a Beowulf cluster?
　　
Beowulf is the oldest English epic:
　　
Famed was this Beowulf: far ew the boast of him, son of Scyld, in the Scandian lands. so becomes it a youth to quit him well with his father's friends, by week and gift, that to aid him, aged, in after days, come warriors willing, shocould war draw nigh, liegemen loyal: by lauded deeds shall an earl have honor in every clan.
　　
It praises a hero, who has a strong body and unparalleled courage. He finally defeated the devil Grendel. You can find this historical poem at http://legends.dm.net/beowulf/index.html.
　　
Beowulf is a cluster system that is as powerful as a hero in the epic. In The summer of 1994, Thomas Sterling and Don Becker formed a Computer Cluster System with 16 nodes and Ethernet in CESDIS (The Center of Excellence in Space Data and Information Sciences, and name the system Beowulf. Beowulf cluster. Beowulf cluster provides a method to construct a cluster system using the Commodity off the shelf hardware to meet special computing requirements. Here, COTS is a standard device that is widely used, such as PC and Ethernet. It can be provided by multiple vendors, so it is usually very cost-effective. Beowulf clusters quickly spread across scientific research institutions and communities from NASA. In fact, Beowulf clusters are now seen as a branch or genre in high-performance computing.
　　
Because almost every Beowulf cluster designer has its own Beowulf cluster definition, it is difficult to give a definite definition to the Beowulf cluster. Some people think that only those systems built in the same way as the original Beowulf cluster system are called Beowulf clusters. Others think that all systems that can run parallel code on multiple workstations are called Beowulf clusters. Here we only list the features of most Beowulf clusters as the definition of Beowulf clusters:
　　
Beowulf is a system structure that enables a system composed of multiple computers to be used for parallel computing.
Beowulf systems generally have one management node and multiple computing nodes. They are connected over Ethernet (or other networks. The management node monitors the computing node, which is usually the gateway and control terminal of the computing node. Of course, it is usually also a cluster system file server. In a large cluster system, the functions of these management nodes may be shared by multiple nodes due to special requirements.
Beowulf systems generally consist of the most common hardware devices, such as PCs, Ethernet cards, and Ethernet switches. Beowulf systems rarely contain special custom devices.
Beowulf systems generally use inexpensive and widely spread software, such as Linux operating systems, parallel virtual machines (PVM), and message passing interfaces (MPI ).

　　 2 Beowulf cluster classification
　　
Some Beowulf cluster systems also use custom devices (which are usually provided by a vendor) for special purposes such as system performance ). Beowulf is usually divided into two categories to distinguish these special systems:
　　
2.1 first-CLASS Beowulf cluster (class I Beowulf)
　　
This type of Beowulf cluster is composed of all COTS devices. The advantages of the first Beowulf system are:
　　
Hardware devices come from multiple sources and are usually cheap and easy to manage and maintain.
Independent from a single hardware vendor
All device drivers are provided by the Linux development community.
Usually standard devices, such as SCSI and Ethernet
Of course, the shortcomings of the first Beowulf cluster are also very obvious. Because the hardware used has not been optimized by performance, it is difficult to achieve good performance. For example, due to the high latency and low bandwidth of Ethernet, message transmission in the cluster system is difficult to meet the requirements of MIMD applications, thus greatly reducing the computing capacity of the entire cluster system.
　　
2.2 second-CLASS Beowulf cluster (class ii Beowulf)
　　
The second type of Beowulf cluster refers to the Beowulf cluster that uses custom devices. This type of cluster system has good performance. For example, using Myrinet as the cluster system's IPC network can greatly provide inter-process message transmission latency and speed. Of course, its disadvantage is that it relies on a single hardware provider and is expensive.
　　
It cannot be said, which type of cluster is superior to the other type of cluster. This depends on the requirements and budget of your cluster system.
　　
　　 3 Beowulf cluster architecture
　　
As mentioned above, there are various Beowulf clusters in reality. Although they are all derivatives of the original Beowulf cluster, there are various minor differences in their architecture. This article takes IBM eServer Cluster 1300 as an example to describe the Beowulf Cluster architecture and system composition.
　　
　　　


(400) {this. resized = true; this. width = 400; this. alt = 'click here to open new window';} "onmouseover =" if (this. resized) this. style. cursor = 'hand'; "onclick =" window. open ('HTTP: // www.stcore.com/Images/uploadimg/20051019/1129748602_1662524769.gif'); ">

Is the System View of the Beowulf Cluster on Cluster 1300.
　　
Is the System View of the Beowulf Cluster on Cluster 1300. Both management nodes and computing nodes are xSeries PC servers in Intel IA32 architecture. They are connected over the network (Ethernet and Myrinet. All nodes run the Linux operating system. Parallel applications running on computing nodes use MPI for communication between parallel processes. The compute node is connected to an external LAN through the Management node. The entire cluster system is monitored by a set of cluster management tools.
　　
　　　


(400) {this. resized = true; this. width = 400; this. alt = 'click here to open new window';} "onmouseover =" if (this. resized) this. style. cursor = 'hand'; "onclick =" window. open ('HTTP: // www.stcore.com/Images/uploadimg/20051019/1129748604_1158233415.gif'); ">

　　
Figure 2 Beowlf cluster components on Cluster1300
　　
Figure 2 shows the component view of the Beowulf Cluster on Cluster 1300. It reveals the components of the Beowulf cluster. Generally, Beowulf clusters are composed of four layers:
　　
Hardware: mainly refers to the Intel IA32 architecture PC server.
Network: refers to the LAN (common Ethernet) used for inter-node communication and the high-speed network (Myrinet and other high-speed networks) for inter-process communication ).
Software: mainly refers to the Linux operating system and parallel programming libraries (such as MPI and PVM) used for parallel communication ).
Parallel Applications
The following three sections describe these four layers. The last two parts of this series will introduce them in more detail.
　　
3.1 Beowulf cluster hardware and network
　　
Beowulf cluster hardware and network layers need to solve the problem is how to organize hardware to make it the most cost-effective. Beowulf usually uses cheap COTS hardware to achieve good cost effectiveness. Of course, sometimes some special devices are used to provide certain key performance.
　　
From the perspective of hardware organization, all nodes in the Beowulf cluster are non-memory-sharing computers, and they communicate through message transmission. In fact, we have other ways to organize hardware to complete parallel computing.
　　
In short, there are two ways to organize hardware to complete parallel computing:
　　
Local Memory (non-shared memory) Computer System (Beowulf cluster) for communication through message transmission)
Shared Memory computer system (SMP computer) that accesses communication through memory)
Of course, there is also the possibility of connecting multiple local or shared memory computers and creating a mixed shared memory computer system. But in the end user's opinion, such a computer system is like a large computer with shared memory. This technology is called Non-consistent Memory Access NUMA (Non Uniform Memory Access ). However, at the underlying layer, a NUMA computer system must transmit messages between nodes.
　　
Of course, the shared memory computer can also be connected to a Beowulf cluster system as a local memory computer. Because Linux supports SMP computers, the Linux system can schedule jobs on multiple CPUs in SMP. Therefore, the Beowulf cluster itself does not need to identify whether the nodes in the cluster are shared memory computers.
　　
Compared with the SMP system, the cluster system has obvious advantages. For more information, see the first article in this series.
　　
Because Beowulf clusters use message transmission to complete parallel inter-program communication, network transmission becomes a bottleneck of the system. In actual systems, two sets of independent network devices are usually used. A set of common Ethernet interfaces are used for common network communications such as system management and file services. Another network is a high-speed network for inter-process communication, such as Myrinet and Giganet. Compared with M Ethernet, this type of network features low latency and high bandwidth.
　　
Three other types of devices are not required, but are very important for cluster management:
　　
KVM Swither: KVM refers to the Keyboard, Video, and Mouse. This device allows the system administrator to manage all nodes in a KVM system.
Remote Power Management devices (such as ASM): these devices allow administrators to Power on/off other nodes on the Management node.
Terminal Server: This device connects nodes through a serial port. With this device, the administrator can virtualize the control terminal on other nodes on the Management node. Compared with KVM, this method does not require hardware switching.
3.2 Beowulf cluster Software
　　
The problem Beowulf clusters face at the software level is how to achieve maximum performance at the hardware level. Generally, Beowulf clusters support parallel computing in Linux + MPI mode. MPI implements parallel inter-program communication by means of message transmission. Although other communication methods can also be used, the message transmission mode is most suitable for cluster systems. To put it simply, there are two ways to transfer concurrency between parallel programs:
　　
Message Passing between processors (MPI)
Use the Thread of the Operating System)
Other methods exist, but these two methods are the most widely used. Although there are efficiency and porting problems, both methods can be implemented on SMP, NUMA, and Cluster.
　　
Message transmission needs to copy data between CPUs, but threads can share data between CPUs. The speed and delay of data copying are the most critical factors affecting message transmission efficiency. PVM and MPI are the two most common message passing APIs. Message transmission can also be implemented on the thread, and message transmission can be used in the cluster system or SMP system. Compared with threads, the advantage of Message Passing in SMP systems is that it is convenient to port SMP applications to cluster systems.
　　
The biggest feature of a thread is that data is shared among threads. Therefore, it works better on SMP systems. Linux also supports Posix Threads. However, the biggest drawback of using threads is that it is difficult to extend threads beyond the SMP system. Although NUMA technology can efficiently achieve this, it is very expensive and not supported by Linux itself.
　　
The following table summarizes the comparison between thread and message passing on SMP and Cluster Systems:
　　
SMP system performance Cluster System Performance scalability
Good message delivery
Thread is good * poor
　　
* Requires expensive NUMA Technology
3.3 Beowulf cluster application
　　
The Application Layer of the Beowulf cluster is located at the hardware and software layers. The problem to be solved is how to parallelize the CONCURRENT part of the PARALLEL application in the actual cluster system so that the application on the cluster can achieve the best performance.
　　
Before proceeding to this section, we need to distinguish two concepts: PARALLEL and CONCURRENT. The CONCURRENT part of a program is the part that can be independently calculated in the program. The PARALLEL part of a program is the CONCURRENT part of the program that is executed separately at the same time.
　　
Their differences are very important. Concurrency is the property of the program, while parallelism is the property of the computer system. The purpose of parallelism is to achieve good performance. The factors limiting parallel performance are the speed and delay of Inter-computing node communication. Most benchmarks have high concurrency and communication and latency are not bottlenecks. However, other applications are not that simple. For these applications, sometimes concurrent partial parallel execution may lead to lower application performance. To put it simply, parallel execution reduces efficiency unless the additional communication time cost is less than the computing time saved by concurrency.
　　
　　


(400) {this. resized = true; this. width = 400; this. alt = 'click here to open new window';} "onmouseover =" if (this. resized) this. style. cursor = 'hand'; "onclick =" window. open ('HTTP: // www.stcore.com/Images/uploadimg/20051019/1129748605_1852007105.gif'); ">

The programmer's task is to determine which concurrent parts need to be executed in parallel, and which ones should not be executed in parallel. These decisions will ultimately affect the application execution efficiency. Describes the relationship between the concurrent parts of the program and the computing time ratio of the communication in the actual application.
　　
In an ideal system, the communication computing time ratio should be constant. Any concurrent part should be implemented in parallel. Unfortunately, the actual systems, including SMP systems, are all shown in that way. Therefore, when designing Beowulf cluster applications, we must realize that the efficiency of concurrency depends on the communication processing time ratio on the system. For this reason, although the application can be transplanted to another concurrent system, it cannot be guaranteed that the application is still efficient on the transplanted system. Generally, there are no portable and efficient applications.
　　
In the same way, using a faster CPU may also slow down your applications.
　　
　　 4 references
　　
Linux HPC Cluster Installation, IBM Redbooks, http://www.redbooks.ibm.com/
IBM eServer xSeries Clustering Planning Guide, IBM Redbooks, http://www.redbooks.ibm.com/
Linux Clustering with CSM & GPFS, IBM Redbooks, http://www.redbooks.ibm.com/
Cluster Computing White Paper, Mark Baker, University of Portsmouth, UK
Beowulf HOW-TO, http://www.beowulf-underground.org
Beowulf Introduction and Overview, http://www.beowulf.org
Http://www.mosix.org, The Mosix Howto
OSCAR: A packaged Cluster software stack for High Performance Computing, http://www.openclutergroup.org/
Linux-HA Heartbeat System Design, http://www.linux-ha.org
HOW-TO, http://www.x-CAT.org
MPICH, http://www.mcs.anl.gov/mpi/mpich.
PVM http://www.epm.ornl.gov/pvm/pvm_home.html
OpenPBS, http://www.openpbs.org/
Maui, http://www.supercluster.org/
Condor Manual, Condor Team, University of Wisconsin-Madison
Http://globalfilesystem.org/GFS/
Http://inter-mezzo.org, Intermezzo/
Coda, http://www.coda.cs.cmu.edu/