Key Technical Analysis of Linux cluster system

In the cluster system, the process arrival time and the amount of resources required for the new arrival process are unpredictable. Therefore, process placement and migration are very important. Due to the unpredictability in the cluster system, processes are sometimes placed on unsuitable machines, and Process Migration gives the system a chance to make up for such errors. Using better algorithms to place new processes on appropriate nodes for execution, and migrating some processes can shorten the average task execution time, therefore, the overall performance of the system is improved.

The placement of processes is very complicated because the resources in the cluster are heterogeneous, such as memory, CPU, and inter-process communication. The memory unit is byte, And the CPU unit is loop and Communication Resource Unit is bandwidth.

Process placement policies include static placement policies and dynamic placement policies. A static placement policy allocates a newly created process based on predefined rules. It does not use runtime information. The dynamic placement policy places processes on the most appropriate node based on changes in the system status.

There are three common static placement policies: Round RR, Best-Fit (BF), and Round Robin Next-Fit (NF ).

Round Robin places the newly created process in the form of rotation to each node in the cluster. The defect of this method is that if the memory required by the newly created process is larger than the available memory size of the node to be allocated to it, the algorithm will fail.

An improved method is to use the Best-Fit method to place processes on nodes with the maximum available memory.

Round Robin Next-fit scans nodes in the form of Round Robin and sends the process to the first node with sufficient memory. Its disadvantage is that the load may be unevenly distributed to each node.

The performance of the three process placement policies is 1-1. The average process size is 16 MB)

The figure shows that the NF algorithm can fully utilize memory resources. When the number of nodes in the cluster increases, the performance of BF algorithm and RR algorithm also decreases significantly. This is because when the number of nodes increases, the total memory in the Cluster also increases proportionally, and new processes will be created for the newly added nodes, which means that the number of large processes will also increase, these large processes are difficult to place for BF and RR algorithms, resulting in a reduction in their performance.

A Dynamic process placement policy is called MSMigrate the Smallest process). It scans all nodes in the form of Round Robin and places the new process on the next node. Unlike Round Robin, if the memory of the node to be placed is insufficient for use by new processes, the MS algorithm migrates to a process. The process to be migrated is the smallest of all processes on the node, but it just meets the memory required by the new process, other nodes can also accommodate the node to be migrated. This method has a low network overhead. If such nodes do not exist, if all other nodes do not have enough memory space, the algorithm fails. Shows the comparison between the MS algorithm and the NF algorithm. When the average process size is 1 MB, both algorithms achieve nearly 100% memory utilization, but 1-2 shows that when the average process size is 16 MB, the MS algorithm is more than 20 percentage points higher than the NF algorithm.

All of the above algorithms are centralized process placement policies. Global information is required to determine placement policies, which is not conducive to scalability and cannot be effectively executed on clusters with multiple nodes. A ms-based distributed process placement algorithm (mongowed MS) is implemented as follows: it places the migrated process to a node with the maximum available memory selected from the Information Window. The Information Window refers to a buffer that stores the available memory information of other nodes. The memory information of other nodes is collected to the information window at a certain time and updated.

Figure 1-1 Performance Comparison of process placement policies
Figure 1-2 Performance Comparison of process placement policies

Migration of processes
As early as 1980s, people began to study process migration. Most of the research focuses on how to use better methods to transfer the state of processes between machines. Homogeneous Process Migration refers to the same architecture of the original and target machines for Process Migration, while heterogeneous Process Migration refers to Process Migration between machines with different architecture. Examples of homogeneous process Migration systems include V charlote, DEMOS/MP, Sprite, Condor, and Accent. Heterogeneous process Migration systems include Tui, Emerald, and HMF (Heterogeneous Migration Facility). Process Migration is mainly used in the following situations.

1. When an error is fixed on an invalid machine and re-enters the cluster system, you need to migrate the original processes on the machine back.
2. Share the load in the cluster system. To allow a process to spend as much CPU time as possible, it needs to be migrated to a machine that provides most of the commands and I/O operations. However, sometimes load sharing is flawed, because most processes only need a small portion of the CPU time, considering the overhead of Process Migration, migration of simple processes that can run locally is not worth the candle, but the migration process is very effective for those programs that require a large amount of processing time, such as simulation programs.
3. Improve communication performance. If a process needs to communicate frequently with other processes, placing these processes closer will reduce the communication overhead. The specific migration method is to migrate a process to the CPU where other processes are located.
4. Availability. When a machine on the network fails, the process can be migrated to another machine for further execution through Process Migration, which ensures the availability of the system in the event of a disaster.
5. Reconfigure. When managing a cluster, you sometimes need to move services from one node to another. transparent process migration can migrate services without stopping services.
6. Use the special capabilities of some machines in the cluster. If a process can benefit from a specific machine in the cluster, it should be executed on that machine. For example, a program for numerical computation can greatly shorten the program execution time by using a mathematical coprocessor or multiple processors in a supercomputer.

Although Process Migration has been successfully implemented in the experimental environment, it is not widely accepted yet. One reason is that MSDOS, Microsoft Windows, and many types of UNIX operating systems do not support process migration. Another reason is that the Migration overhead of a process may be greater than that of a non-migration process. However, at present, two new computing fields have promoted the development of Process Migration. One is mobile computing, and the other is wide area computing. Mobile Computing refers to the computation of portable small computers. Wide Area Computing refers to the computing problem of machines in the wide area network.

Process Migration migrates an ongoing process from one node to another node connected through the network, that is, the local shared memory mechanism is not used ). The operating system on the original node where the process is located should pack all the states of the process so that the target machine can continue to execute the process.

To complete the process migration, You need to migrate the status of the process, especially the address space of the Process, access to other processes, such as interfaces and pipelines). The code can be part of the address space) and execution status registers, stacks, etc ). In addition, all accesses to the original process must be re-linked to the new process copy. Otherwise, the migration will not be seamless and errors will occur. The entire process migration operation must be an atomic operation to avoid process loss or two copies.

The following modifications are required for Process Migration:

1. You must modify the file system so that each machine can see the same namespace.
2. Sufficient status must be transmitted to ensure that normal core calls can be executed on the remote machine.
3. Some special core system calls such as gettimeofday and getpgrp should be sent back to the original node for execution.

The following is an example of migration from a heterogeneous process to describe the entire process of migration. Figure 1-3 illustrates how a process is migrated from one machine to another in the Tui process migration system.

1. First, compile a program. For the four architectures supported by Tui, compile the program four times.
2. The program is executed in normal mode on the original machine. Such as the command line method)
3. When a migration process is selected, the migrout program first sets a checkpoint for the process, then suspends the process, then performs a memory image, and then scans global variables, stacks, and stacks to locate all the data. Then all of these are converted into an intermediary format and sent to the target machine. Finally, the process on the original machine is killed.
4. On the target machine, the migrin program obtains the mediation value and creates a new process. Because the program has been compiled according to the architecture of the target machine, therefore, the information of the body segment and the type information of the datagram are available. Then, by re-creating global variables, stacks, and stacks, the program continues to run from the checkpoint.

After statistics, it takes 0.1 seconds to select an idle host and start a new process. The average migration time is 330 milliseconds. Process Migration can increase the performance by nearly five times.

Figure 1-3 Process Migration