Linux process management for Linux Performance and Tuning Guide (translation)

The translation is as follows:
1.1 Linux Process Management

Process management is one of the most important functions of the operating system. Efficient process management can ensure a smooth and efficient operation of a program.

Linux process management is similar to UNIX's process management. It includes process scheduling, interrupt processing, signaling, process prioritization, context switching, process status, progress memory, and so on.

In this section, we will describe the implementation of the fundamentals of Linux process management. It will better help you understand how the Linux kernel handles processes and their impact on system performance.

1.1.1 What is a process?
A process is an instance of a program that runs on a processor. The process uses whatever resources the Linux kernel can handle to accomplish its tasks.

All processes running on the Linux operating system are managed by the TASK_STRUCT structure, which is called a process description at the same time. A process description contains all the necessary information for a running process, such as process identity, process properties, and resources for the build process. If you understand the process constructs, you can understand what is important for the running and performance of the process. Figure 1-2 shows an overview of process information related to process architecture.

Linux process management for Linux Performance and Tuning Guide (translation)


Figure 1-2 Task_struct Structural Body


The life cycle of the 1.1.2 process

Each process has its life cycle, such as Create, run, terminate, and eliminate. These phases are repeated countless times during system startup and operation. Therefore, the life cycle of a process is important for its performance analysis.

Figure 1-3 shows the classic process life cycle.

Linux process management for Linux Performance and Tuning Guide (translation)


Figure 1-3 The classic process life cycle


When a process creates a new process, the process's creation process (the parent process) calls a fork () system call. When the fork () system call is called, it gets the process description of the new creation process (child process) and invokes a new process ID. It copies the value to the parent process that the process describes into the child process. The address space of the entire parent process is not replicated at this time, and the parent-child process shares the same address space.


The exec () system calls to copy the new program to the child process's address space. Because the parent-child process shares the address space, writing a new program's data causes a paging error. In this scenario, the memory allocates a new physical memory page to the child process.


This deferred operation is called copy-on-write. Child processes typically run their own programs instead of running the same program as the parent process. This operation avoids unnecessary overhead because replicating the entire address space is a very slow and inefficient operation that requires a lot of processor time and resources.


When the program has finished executing, the child process terminates by calling the exit () system call. The exit () system call frees most of the data for the process and notifies its parent process by sending a signal. At this point, the child process is a process called a zombie process (see page 7, "Zombie processes").


A child process is not completely removed until its parent process knows that its child process calls the wait () system call and terminates. When the parent process is notified that the child process terminates, it removes all the data structures of the child process and releases its process description.


1.1.3 Threads
A thread is a single execution unit that is generated by a separate process. It runs in parallel with other threads in the same process. Each thread can share the resources of the process, such as memory, address space, open files, and so on. They can access the same program data set. Threads are also called lightweight processes (light Weight process,lwp). Because they share resources, each thread should not change the resources they share at the same time. Mutex implementation, lock, serialization, etc. are the responsibility of the user program.


From a performance standpoint, the overhead of creating a thread is less than the creation process, and it is not necessary to copy resources when creating a thread. On the other hand, processes and threads have similar characteristics on scheduling algorithms. The kernel handles them in a similar way.

Linux process management for Linux Performance and Tuning Guide (translation)

Figure 1-4 Processes and threads

In today's Linux implementation, threads support the Portable Operating System interface (POSIX) standard library for UNIX. There are several thread implementations available in the Linux operating system. The following are the widely used line libraries:
Linuxthreads
Linuxthreads has been implemented as the default thread since Linux kernel 2.0. Some implementations of linuxthreads do not conform to the POSIX standard. The Native POSIX Thread Library (NPTL) is replacing Linuxthreads. Linuxthreads will not be supported in future Linux Enterprise releases.


Native POSIX Thread libary (NPTL)
NPTL was originally developed by Red Hat Corporation. NPTL is more compatible with POSIX. With the advanced features of Linux kernel 2.6, such as the New Clone () system call, signal processing implementation, and so on, it has higher performance and scalability than linuxthreads.
NPTL and Linuxthreads have some incompatibilities. An application that relies on linuxthreads may not work in the NPTL implementation.


Next Generation POSIX Thread (NGPT)
NGPT is a POSIX line libraries developed by IBM. It is now in the maintenance phase and there is no development plan in the future.
Using the LD_ASSUME_KERNEL environment variable, you can choose which line libraries to use in your app.


1.1.4 process priority and nice values
A process priority is a numeric value that determines the order in which processes are processed by the CPU through dynamic precedence and static precedence. A process with a higher process priority has a greater chance of getting processor-level processing.


The kernel uses heuristic algorithms based on the behavior and characteristics of the process to dynamically adjust or lower the dynamic priority level. A user process can indirectly change the static priority by using the Nice value of the process. A process with a higher static priority will have a longer time slice (how long the process can run on the process).


Linux supports nice values from 19 (lowest priority) to 20 (highest priority). The default value is 0. Change the nice value of the program to a negative number (higher priority for the process) and need to log in as root or execute as root using the SU command.


1.1.5 Context Switch
As the process runs, the running information for the process is stored in the processor's register and its cache. The data set that is being executed by the process loaded into the register is called the context. In order to switch processes, the context of the running process will be saved and the context of the next running process will be restored to the register. The region of the process description and kernel-mode stack will be used to save the context. This switchover is called context switching. Excessive context switching is undesirable because each time the processor must flush the flush register and cache, creating space for the new process. It may cause performance problems.


Figure 1-5 illustrates how the context switch works.

Linux process management for Linux Performance and Tuning Guide (translation)

Figure 1-5 Context Switches


1.1.6 Interrupt Handling
Interrupt processing is one of the highest-priority tasks. Interrupts are typically generated by I/O devices, such as network interface cards, keyboards, disk controllers, serial adapters, and so on. The interrupt processor notifies the kernel via an event (for example, keyboard input, Ethernet frame arrival, and so on). It allows the kernel to interrupt the execution of the process and perform interrupt processing as quickly as possible because some devices require a quick response. It is the key to the stability of the system. When an interrupt signal reaches the kernel, the kernel must switch the current process to a new interrupt processing process. This means that the interrupt causes a context switch, so a large number of interrupts can cause performance degradation.


There are two types of interrupts in the implementation of Linux. A hard interrupt is made by the device requesting the response (disk I/O interrupts, network adapter interrupts, keyboard interrupts, mouse interrupts). Soft interrupts are used to handle tasks that can be deferred (TCP/IP operations, SCSI protocol operations, and so on). You can view information about hard interrupts in the/proc/interrupts file.


In multiprocessor environments, interrupts are handled by each processor. The performance of the system can be improved by breaking the binding into a single physical process. For more details, see 4.4.2, "CPU Interrupt handling Affinity".


1.1.7 Process Status
Each process has its own state, and the status represents what the process is currently taking place.


The status of the process changes during the execution of the process. Some processes have the following status:
Task_running
In this state, it indicates that the process is running in the CPU or waiting in the queue (running the queue).


task_stopped
Processes in this state are paused by certain signals (such as sigint,sigstop). The process is waiting for a signal to resume running, such as Sigcont.


Task_interruptible
In this state, the process is paused and waits for a certain conditional state to arrive. If a process is in the task_interruptible state and receives a stopped signal, the status of the process is changed and the operation is interrupted. An example of a typical task_interruptible state process is a process waiting for the keyboard to break.


Task_uninterruptible
Similar to task_interruptible. When a process is in? The task_uninterruptible state can be interrupted to send a signal to a process in the task_uninterruptible state without any action occurring. A typical example of a task_uninterruptible process is waiting for disk I/O operations.


Task_zombie
When a process calls exit () the system call exits, its parent process should know the termination of the process. A process in the Task_zombie state waits for its parent process to notify it to release all data structures.

Linux process management for Linux Performance and Tuning Guide (translation)

Figure 1-6 Process Status


Zombie Process
When a process receives a signal and terminates, it usually takes some time to end all tasks (such as closing open files) before ending itself. In this usually very short period of time, the process is a zombie process.


After the process has completed all the shutdown tasks, it reports to the parent process that it is about to terminate. There are times when a zombie process cannot terminate itself, which will guide its status to display as Z (Zombie).


It is not possible to use the KILL command to close such a process because the process is already considered dead. If you can't clear the zombie process, you can end its parent process, and then the zombie process disappears. However, if the parent process is the INIT process, you cannot end it. The init process is a very important process, so it may be necessary to restart the system to clear the zombie process.


1.1.8 Process Memory Segment
The process uses its own area of memory to perform its work. The changes in work are determined by the circumstances and the use of the process. A process can have different workload characteristics and different data size requirements. The process must handle a variety of data sizes. To meet the requirements, the Linux kernel uses the mechanism of dynamic request memory for each process. The process memory allocation is shown in data structure 1-7.

Linux process management for Linux Performance and Tuning Guide (translation)

Figure 1-7 Process address space


The process memory area consists of the following parts:
Text segment
This zone is used to store running code.

Data segment
The data segment consists of three regions.
-Data: The zone stores the initialized information, such as a static variable.
-BSS: The region stores data initialized to 0. The data is initialized to 0.
-Heap: This area is used to use malloc () dynamically requested memory on demand. The heap grows to the high address direction.

Stack segment
This area is used to store local variables, function parameters, and the address of the return function. The stack is growing in the low address direction.


The address space memory distribution of the user process can be viewed using the PMAP command. You can use the PS command to see the size of the memory segment. You can refer to the 2.3.10 "Pmap", "Ps and Pstree".


1.1.9 Linux CPU Scheduling
The basic functions of any computer are very simple, that is, computing. To be able to calculate, it means that you must manage compute resources or processor and compute tasks, which is the thread or process we know. Thanks to the great contribution of Ingo Molnar, the Linux kernel uses an O (1) algorithm instead of the previous O (n) CPU scheduling algorithm. O (1) refers to a static algorithm, which means that the time it takes to select a process and execute it is a constant, regardless of the number of processes.


The new scheduling algorithm is very extensible, regardless of the number of processes or the number of processors, and the overhead of the system is very low. The algorithm uses two process priority series groups:
Active (Active)
Expired (expired)


The scheduler allocates the time slices for the process based on the priority and priority interception rate of the process, and the process is placed in the active array in order of precedence. When the process time slice is exhausted, the process requests a new time slice and places it inside the expired array. When the time slice of all the processes in the active array is exhausted, the two arrays are switched and rerun the algorithm. For a general interactive process (as opposed to a real-time process), a process with a high priority usually gets a longer time slice and more compute time than a low-priority process, but this does not mean that low-priority processes are completely ignored (starved). The advantage of this algorithm is to increase the scalability of the Linux kernel for enterprise-class environments with a large number of threads and processes and multi-processors. The new CPU Scheduler for O (1) was designed for memory 2.6, but has now been ported to the 2.4 series. Figure 1-8 illustrates how the Linux CPU is scheduled to work.

Linux process management for Linux Performance and Tuning Guide (translation)

Figure 1-8 Linux Kernel 2.6 O (1) Scheduler


Another notable improvement to the new scheduler is the support for non-conforming Memory Architecture (NUMA) and symmetric multithreaded processors, such as Intel Hyper-Threading technology.


Improved NUMA support ensures that load balancing spans a NUMA node when only one node is overloaded. This mechanism ensures the minimization of extended link traffic that is relatively slow in NUMA systems. Although load balancing iterates through the processors in the dispatch domain group each time the tick is scheduled, the load is transferred across the dispatch domain only when the node is overloaded and requests load balancing.

Linux process management for Linux Performance and Tuning Guide (translation)

Manuscripts: Small program development www1.qixoo.com

Linux process management for Linux Performance and Tuning Guide (translation)