Linux thread, process classic article __linux

I. Basic knowledge: Threads and processes

According to the textbook definition, the process is the smallest unit of resource management, and the thread is the smallest unit of program execution. In operating system design, the main purpose of evolving threads from process is to better support SMP and reduce (process/thread) context switching overhead.

Regardless of the division, a process requires at least one thread as its execution body, processes managing resources (such as CPUs, memory, files, and so on), and assigning threads to a CPU for execution. A process can, of course, have multiple threads, at which point, if the process is running on an SMP machine, it can use multiple CPUs at the same time to execute each thread to achieve maximum parallelism to improve efficiency, and even on a single CPU machine, using a multithreaded model to design the program, Just as using a multiple process model instead of a single process model, it makes the design more concise, more functional and more efficient, such as multiple threads responding to multiple inputs, while the functionality implemented by the multithreaded model can actually be implemented using a multiple process model, compared to the latter The thread's context-switching overhead is much smaller than the process, and semantically, the ability to respond to multiple inputs at the same time is actually sharing all the resources except the CPU.

Two thread models of core-level thread and user-level thread are developed, which are mainly based on the thread dispatcher in the core or the kernel. The former is more conducive to concurrent use of multiprocessor resources, while the latter is more about context switching overhead. In current commercial systems, the two are often combined to provide core threads to meet the needs of SMP systems, as well as a thread-threading way to implement another threading mechanism in user mode, at which point a core thread becomes a dispatcher of multiple user-state threads. As with many technologies, "mixing" usually brings more efficiency, but it also brings greater implementation difficulty, for "simple" design ideas, Linux from the outset did not implement a hybrid model of the plan, but it is implemented on the implementation of a different idea of the "hybrid."

On the implementation of the thread mechanism, threads can be implemented on the operating system kernel or out of the core, which clearly requires that the process be implemented at least in the kernel, while the former generally requires that the process also be supported at the same time in the kernel. The core-level threading model obviously requires the support of the former, while the user-level threading model is not necessarily based on the latter implementation. This difference, as mentioned earlier, is the result of different criteria for the two classifications.

When the kernel supports both processes and threads, you can implement the "Many-to-many" model of the thread-process, where a thread of a process is scheduled by the kernel, while it can also act as a dispatcher of the user-level thread pool and select the appropriate user-level threads to run in its space. This is the "mixed" threading model mentioned earlier, which can meet the needs of multiprocessor and minimize scheduling overhead. The vast majority of commercial operating systems (such as digital Unix, Solaris, Irix) Use this threading model to fully implement the POSIX1003.1C standard. The threads that are implemented outside the core can be divided into "one-to-one", "more than one" two models, the former with a core process (perhaps a lightweight process) corresponding to a thread, the thread scheduling equivalent to the process scheduling, to the core to complete, while the latter is completely outside the core implementation of multithreading, scheduling also completed in the user state. The latter is the simple implementation of the user-level threading model mentioned earlier, obviously, such an external thread scheduler actually only needs to complete the switch of the line Cheng Broker's storehouses, the dispatch overhead is very small, but also because the core signal (whether synchronous or asynchronous) is in the process unit, therefore cannot navigate to the thread, So this implementation cannot be used in multiprocessor systems, and this requirement is becoming more and more large, so in reality, the implementation of pure user-level threads has almost disappeared, except for algorithmic research purposes.

The Linux kernel provides only lightweight process support and limits the implementation of more efficient threading models, but Linux focuses on optimizing the scheduling overhead of the process to some extent compensating for this shortcoming. At present, the most popular thread mechanism linuxthreads is the thread-process "one-to-one" model, the dispatch to the core, and at the user level to implement a thread management mechanism including signal processing. The operation mechanism of Linux-linuxthreads is the focus of this paper.

Two. Lightweight process implementation in the Linux 2.4 kernel

The initial process definition includes the program, the resource and its execution three parts, in which the program usually refers to the code, the resource at the operating system level usually includes memory resources, IO resources, signal processing, and so on, and the execution of the program is usually understood as the execution context, including the CPU occupancy, and later developed Before the thread concept appeared, to reduce the overhead of process switching, the operating system designer gradually corrects the concept of the process, gradually allowing the process to be stripped of resources from its main body, allowing some processes to share a portion of resources, such as files, signals, data memory, and even code, which develops the concept of lightweight processes. The Linux kernel has implemented a lightweight process in the 2.0.x version, and the application can specify whether to create a lightweight process or a normal process with different parameters, using a unified clone () system to invoke the interface. In the kernel, the clone () call is called Do_fork () after the parameter is passed and interpreted, which is also the final implementation of the fork (), vfork () system call:

<linux-2.4.20/kernel/fork.c>
int do_fork (unsigned long clone_flags, unsigned long stack_start,
struct Pt_regs *regs, unsigned long stack_size)

The clone_flags is taken from the "or" value of the following macro:

<linux-2.4.20/include/linux/sched.h>
#define CSIGNAL 0X000000FF/* Signal mask to is sent at exit */
#define CLONE_VM 0x00000100/* Set if VM shared between processes * *
#define CLONE_FS 0x00000200/* Set if FS info shared between processes * *
#define Clone_files 0x00000400/* Set if open FILES shared between processes * *
#define Clone_sighand 0x00000800/* Set if signal handlers and blocked shared */
#define CLONE_PID 0x00001000/* Set if PID shared */
#define CLONE_PTRACE 0x00002000/* Set if we want to let tracing continue on the child too * *
#define CLONE_VFORK 0x00004000/* Set if the parent wants to wake it up on Mm_release * *
#define Clone_parent 0x00008000/* Set if we want to have the same PARENT as the Cloner * *
#define Clone_thread 0x00010000/* Same THREAD Group? */
#define CLONE_NEWNS 0x00020000/* New namespace group? */
#define Clone_signal (Clone_sighand | Clone_thread)

In Do_fork (), different clone_flags will lead to different behaviors, for linuxthreads, it is used (CLONE_VM | Clone_fs | Clone_files | Clone_sighand) parameter to invoke Clone () to create a "thread" that represents shared memory, shared file system access count, shared file descriptor, and shared signal processing. This section focuses on these parameters to see how the Linux kernel implements the sharing of these resources.

1.clone_vm

Do_fork () needs to invoke COPY_MM () to set the MM and active_mm entries in Task_struct, which correspond to the memory space associated with the process. Two mm_struct data. If the CLONE_VM switch is specified when Do_fork (), copy_mm () sets the MM and active_mm in the new task_struct to be the same as current, while increasing the number of users of the Mm_struct (mm_struct: : mm_users). In other words, the lightweight process shares the memory address space with the parent process, as shown in the following diagram to see the status of Mm_struct in the process:

2.clone_fs

FS (struct fs_struct *) is used in task_struct to record the root directory and current directory information of the filesystem in which the process resides, Do_fork () when Copy_fs () is replicated, and the lightweight process only increases fs-> Count count, which shares the same fs_struct as the parent process. In other words, lightweight processes do not have independent file system-related information, and any thread in the process that changes the current directory, root directory, and so on will directly affect other threads.

3.clone_files

A process may have opened some files, using the files (struct files_struct *) in the process structure task_struct to hold the file structure (struct file) information open by the process, Do_fork () Copy_files ( To handle this process property; The lightweight process shares the structure with the parent process, adding only the Files->count count when Copy_files (). This share makes it possible for any thread to access open files maintained by the process, and their operations are reflected directly to other threads in the process.

4.clone_sighand

Each Linux process can define its own way of handling the signal, using an array of struct k_sigaction structures in the sig (struct signal_struct) in task_struct to save this configuration information do_fork () The Copy_sighand () in is responsible for replicating the information, and the lightweight process does not replicate, but only increases the signal_struct::count count and shares the structure with the parent process. That is, the child processes are in exactly the same way as the parent process, and can be changed from one to the other.

Many of the work done in do_fork () are not described in detail here. For SMP systems, when all processes are fork out, they are assigned to the same CPU as the parent process until the process is scheduled for CPU selection.

Although Linux supports lightweight processes, it cannot be said to support core-level threads because Linux's "threads" and "processes" are actually at a scheduling level, sharing a process identifier space that makes it impossible to implement a full POSIX thread mechanism on Linux, So many Linux line threading implementations try to achieve most of POSIX semantics as much as possible and function as close as possible.

Three. Linuxthread Threading mechanism

Linuxthreads is the most widely used line threading on the Linux platform, which is developed by Xavier Leroy (Xavier.Leroy@inria.fr) and is bound to be released in GLIBC. What it does is a "one-to-one" threading model based on the core lightweight process, one thread entity corresponding to a core lightweight process, and the management between threads implemented in an external function library.

1. Thread description data structure and implementation restrictions

Linuxthreads defines a struct _PTHREAD_DESCR_STRUCT data structure to describe the thread and uses the global array variable __pthread_handles to describe and reference the process-managed threads. In the first two items in __pthread_handles, Linuxthreads defines two global system threads: __pthread_initial_thread and __pthread_manager_thread, and __pthread The _main_thread represents the parent thread of the __pthread_manager_thread (initially __pthread_initial_thread).

Struct _pthread_descr_struct is a double loop linked list structure, where the __pthread_manager_thread contains only one element, in fact, __pthread_manager_thread is a special thread , Linuxthreads uses only three domains, such as errno, P_pid, p_priority, and so on. The chain in which __pthread_main_thread is located will string together all the user threads in the process. The __pthread_handles array formed after a series of Pthread_create () is shown in the following illustration:

Figure 2 __pthread_handles Array structure

The newly created thread will first occupy an item in the __pthread_handles array, and then the chain pointer in the data structure is connected to the linked list in the __pthread_main_thread-led pointer. The use of this list will be mentioned when introducing the creation and release of threads.

Linuxthreads follows the POSIX1003.1C standard, which limits the implementation of the line threading, such as the maximum number of threads in the process, the size of the thread private data area, and so on. In the implementation of the Linuxthreads, the basic adherence to these restrictions, but also made a certain change, the trend is to relax or expand these restrictions to make programming more convenient. These qualifying macros are mainly concentrated in sysdeps/unix/sysv/linux/bits/local_lim.h (different file locations used by different platforms), including the following:

The number of private data keys per process, POSIX-defined _posix_thread_keys_max uses pthread_keys_max,1024 for 128,linuxthreads, the number of operations allowed when private data is released, Linuxthreads is consistent with POSIX, defining pthread_destructor_iterations as 4, number of threads per process, POSIX defined as 64,linuxthreads to 1024 (pthread_threads_ MAX) Line Cheng Broker's storehouses minimum space size, POSIX unspecified, linuxthreads using pthread_stack_min,16384 (bytes).

2. Managing Threads

One of the benefits of a "one-to-one" model is that thread scheduling is done by the core, while other tasks, such as thread cancellation, synchronization between threads, are done in the kernel perimeter library. In Linuxthreads, a management thread is constructed specifically for each process, responsible for handling thread-related management work. The first time a process calls Pthread_create () creates a thread (__clone ()) and starts the management thread.

Within a process space, the management thread communicates with other threads through a pair of "management pipes (Manager_pipe[2]"), which is created before the management thread is created, and the read and write ends of the management pipe are assigned to two global variables after the managed thread has successfully started __pthread_ Manager_reader and __pthread_manager_request, after which each user thread Cheng the request to the management line, but the management thread itself is not directly using __pthread _manager_reader, the channel's Read End (Manager_pipe[0]) is passed to the management thread as one of the parameters of the __clone (), and the main task of the management thread is to listen to the pipe read-side and react to requests taken from it.

The process of creating an administrative thread is as follows:
(global variable Pthread_manager_request initial value is-1)

Figure 3 Creating a process for managing threads

After initialization, the lightweight process number is recorded in __pthread_manager_thread and the id,2*pthread_threads_max+1 of the assigned and managed thread is not conflicting with any regular user thread IDs. The administrative thread runs as a child of the caller thread of Pthread_create (), and the user thread created by Pthread_create () is created by the administrative thread to invoke clone (), and therefore is actually the child thread that manages the thread. (The concept of a child thread here should be understood as a subprocess.) ）

__pthread_manager () is the main loop where the thread is managed, and after a series of initialization work, it enters the while (1) loop. In a loop, the thread manages the read end of the pipe in 2 seconds for the timeout query (__poll ()). Before processing the request, check whether its parent thread (that is, the main thread that created the manager) has exited, and exits the entire process if it has exited. Call Pthread_reap_children () cleanup If there is an exit child thread that needs to be cleaned.

The request is then read in the pipeline, and the corresponding action is performed according to the type of request (Switch-case). Specific request processing, the source code is more clear, here will not repeat.

3. Thread Stacks

In Linuxthreads, the stack of the management thread and the stack of the user's threads are separate, and the management thread allocates a thread_manager_stack_size byte area as its running stack in the process heap through malloc ().

The stack allocation method for user threads varies with the architecture, mainly based on two macro definitions, one is Need_separate_register_stack, the property is used only on IA64 platforms, and the other is Floating_stack macros, Used on a few platforms such as I386, where the user thread stack is determined by the system and provides protection. At the same time, users can also specify a user-defined stack by using the thread property structure. Due to space limitations, only the two types of stack organizations used by the I386 platform can be analyzed: floating_stack mode and user customization.

In the Floating_stack mode, Linuxthreads uses mmap () to allocate 8MB space from the kernel space (I386 system default maximum stack space size, if there is a run limit (rlimit), set according to the run limit), use Mprotect () Sets the first page to be a non-access area. The function of the 8M space is assigned the following figure:

Fig. 4 schematic diagram of stack structure

Low address protected pages are used to monitor stack overflows.

For the user-specified stack, in accordance with the pointer to the bounds, set the line stacks top, and calculate the bottom of the stack, do not protect, the correctness of the user to ensure their own.

Regardless of the organization, the thread description structure is always positioned on the top of the stack.

4. Thread ID and Process ID

Each linuxthreads thread has both a thread ID and a process ID, where the process ID is the process number maintained by the kernel, while the thread ID is allocated and maintained by Linuxthreads.

__pthread_initial_thread's thread ID is pthread_threads_max,__pthread_manager_thread is 2*pthread_threads_max+ 1, the thread ID of the first user thread is pthread_threads_max+2, and the thread ID of the nth user thread follows the following formula:

Tid=n*pthread_threads_max+n+1

This allocation ensures that all threads in the process, including those that have exited, do not have the same thread ID, and that the type pthread_t of the thread ID is defined as an unsigned long integer (unsigned long int), which also guarantees that a rational run-time thread ID does not repeat.

Finding the thread data structure from the thread ID is done in the Pthread_handle () function, and actually just takes the thread number Pthread_threads_max, and gets the index of the thread in __pthread_handles.

5. Creation of threads

After Pthread_create () sends a REQ_CREATE request to the administrative thread, the management thread invokes Pthread_handle_create () to create a new thread. After the stack is allocated, the thread attribute is set, the __clone () is called pthread_start_thread () to create and start a new thread for the function entry. Pthread_start_thread () reads its own process ID number into the thread description structure and configures the schedule according to the scheduling method in which it is recorded. When everything is ready, call the real thread execution function and call Pthread_exit () to clean the scene after this function returns.

6.LinuxThreads of deficiency

Due to the limitations of the Linux kernel and the difficulty of implementation, linuxthreads is not completely POSIX compliant, as described in its release readme.

1 Process ID problem

This deficiency is the most critical of the deficiencies, caused by the linuxthreads of the "one-to-one" model.

The Linux kernel does not support threads in real sense, linuxthreads is implemented with a lightweight process that has the same kernel schedule view as the normal process. These lightweight processes have separate process IDs and have the same capabilities as normal processes in process scheduling, signal processing, IO, and so on. In the view of the source reader, the Linux kernel clone () does not implement the support for the Clone_pid parameter.

The processing of clone_pid in kernel do_fork () is as follows:

if (Clone_flags & Clone_pid) {
if (current->pid)
Goto Fork_out;
}

This code shows that the current Linux kernel recognizes the Clone_pid parameter only when the PID is 0, and actually uses the Clone_pid parameter only when the SMP is initialized and the process is created manually.

By POSIX definition, all threads in the same process should share a process ID and a parent process ID, which is not possible under the current "one-to-one" model.

2) Signal processing problem

Since asynchronous signals are distributed by the kernel in processes, and each thread of the linuxthreads is a process for the kernel, and no thread group is implemented, some semantics do not conform to the POSIX standard, such as not implementing a signal to all threads in the process, as illustrated by the Readme.

If the core does not provide real-time signals, Linuxthreads will use SIGUSR1 and SIGUSR2 as internal restart and cancel signals, so that the application cannot use the two signals that were originally reserved for the user. Later versions of Linux kernel 2.1.60 support extended real-time signals (from _sigrtmin to _sigrtmax), so this problem does not exist.

The default actions of some signals are difficult to implement on the current system, such as Sigstop and Sigcont,linuxthreads can suspend only one thread and cannot suspend the entire process.

3 Total number of threads

Linuxthreads defines the maximum number of threads per process as 1024, but in practice this number is limited by the total number of processes in the entire system, which is because the thread is actually the core process.

In kernel 2.4.x, a new set of total process number method is used to make the total process count limited to the physical memory, and the formula is in the Fork_init () function of kernel/fork.c:

Max_threads = mempages/(thread_size/page_size)/8

On i386, thread_size=2*page_size,page_size=2^12 (4KB), mempages= physical memory size/page_size, for 256M memory machines, mempages=256*2^20/2^ 12=256*2^8, at which point the maximum number of threads is 4096.

However, to ensure that the total number of processes per user (except root) does not consume more than half of the physical memory, Fork_init () continues to specify:

Init_task.rlim[rlimit_nproc].rlim_cur = MAX_THREADS/2;
Init_task.rlim[rlimit_nproc].rlim_max = MAX_THREADS/2;

The number of these processes is checked in do_fork (), so for linuxthreads, the total number of threads is also limited by these three factors.

4 Manage Threading Issues

Managing threads can easily be a bottleneck, this is a common fault of this structure, while the management thread is responsible for the cleanup of the user's thread, so that although the management thread has blocked most of the signal, once the management thread dies, the user thread has to be cleaned up manually, and the user thread is unaware of the state of the management thread. Requests for subsequent thread creation will not be processed.

5) Synchronization problem

The thread synchronization in Linuxthreads is largely based on the signal, and the efficiency is always a problem through the synchronization of the complex signal processing mechanism of the kernel.

6 other POSIX compatibility issues

Many system calls in Linux are related to processes in terms of semantics, such as Nice, setuid, Setrlimit, and so on, and in the current linuxthreads, these calls affect only the caller thread.

7) Real-time problems

The introduction of the

  thread has some real time considerations, but Linuxthreads is temporarily unsupported, such as scheduling options, which are not yet implemented. Not only is linuxthreads so, Standard Linux is considered very little in real time.