Analysis on the creation of Linux threads

Analysis on the creation of Linux threads

This article first uses the interface pthread_create to create a thread, and uses the strace command to track the steps for the interface pthread_create to create a thread and the system calls involved, and then discusses the relationship between threads and processes in Linux, finally, the Linux kernel modifications are outlined to implement POSIX Threads.

 

Use pthread_create to create a thread

 

In Linux, you can use pthread_create to create a thread. The interface declaration is as follows:

#include <pthread.h>int pthread_create(phtread_t *thread, const pthread_attr_t *attr,                                    void *(*start_routine) (void *), void *arg);

We can see that when creating a thread, we can specify the thread property pthread_attr_t, for example, attributes such as the separation state attribute of the thread and the size of the thread stack (of course, The pthread_attr_init interface is required to operate on this attribute struct). Alternatively, when creating a thread, pass the arg parameter to the thread entry function. Note that when you use the modified interface to create a new thread, the newly created thread may run before the pthread_create function returns. The following is a simple example:

 

#include <stdio.h>#include <stdlib.h>#include <unistd.h>#include <pthread.h>void* thread(void* arg){    printf("This is a pthread.\n");    sleep(5);    return((void *)0);}int main(int argc,char **argv){    pthread_t id;     int ret;    printf("pthread_start\n");    ret = pthread_create(&id,NULL,thread,NULL);    printf("pthread_end\n");    if(ret != 0)    {           printf("Create pthread error!\n");        exit(1);    }       printf("This is the main process.\n");    pthread_join(id,NULL);    return 0;}

 

Compile the program to obtain the executable file:

$gcc -g -lpthread -Wall -o hack_pthread_create hack_pthread_create.c

We can use the strace command to track the thread creation process:

$strace  ./hack_pthread_create

The output of the pthread_create interface is as follows:

mmap(NULL, 8392704, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0x7f70b6c2f000brk(0)                                  = 0x1fe3000brk(0x2004000)                          = 0x2004000mprotect(0x7f70b6c2f000, 4096, PROT_NONE) = 0clone(child_stack=0x7f70b742eff0, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x7f70b742f9d0, tls=0x7f70b742f700, child_tidptr=0x7f70b742f9d0) = 64861

As you can see from the above output, the steps for creating a thread using the pthread_create interface are as follows:

1) Call mmap to allocate memory on the stack. The size is 8392704 bytes, that is, 8196KB, that is, 8 M + 4 K, which is 4 K larger than the stack space, the 4 K size is the alert buffer size of the stack.

2) Call mprotect () to set the protection zone for a memory page (4 K in size). The starting address of the page is 0x7f70b6c2f000. This page is used to monitor stack overflow. If you have read/write operations on this memory, a SIGSEGV signal is triggered.

3) Call clone () to create a thread. In Linux, this interface is used to create processes. In essence, threads in Linux are implemented by processes. For details, see the following. The first parameter of the call is the address at the bottom of the stack. The memory usage of stack space starts from high memory. The description of flags is as follows:

CLONE_VM indicates that the parent process and the child process share the memory space. That is to say, any modification of a process in the memory will also affect other processes, including the execution of mmap or munmap operations in the process, other processes are also affected. It is worth mentioning that fork also calls clone to create sub-processes, and it does not set CLONE_VM flag.

CLONE_FS indicates that the parent and child processes share the file system information, including the file system root directory, the current working directory, and umask. If you call chroot in a parent or child process, chdir and umask may also affect other processes.

CLONE_FILES indicates that the parent and child processes share the same file descriptor table. Opening a new file in the parent process or child process, closing a file, or using fcntl to modify the relevant file flag may also affect other processes.

CLONE_SIGHAND indicates that the parent process and the child process share the same signal processing program table, that is, the parent process or child process modifies the signal processing method through sigaction, and also affects other processes. However, the parent process and child process have independent masks. Therefore, a process blocks or does not block a signal through sigprocmask, which does not affect other processes.

CLONE_THREAD indicates that the child process and the parent process are in the same thread group. Simply put, the created sub-process is to create a thread for the user space.

CLONE_SYSVSEM is used to indicate that the child process shares the same semaphore list with the parent process.

The "sub-process" mentioned above is actually the thread we created. From these identifiers, we can also see that various threads in the process share those resources.

Relationship between threads and processes in Linux

In Linux, although a process is defined as an execution instance of a program, it does not execute anything. It only maintains various resources required by the application, and the thread is the real execution entity. In order for a process to complete certain work, the process must contain at least one thread. The Process maintains resources (static resources) in the program, such as virtual address space, set of opened file descriptors, file system status, and signal processing programs; resources (Dynamic Resources) related to running maintained by threads, such as running stacks, control information related to scheduling, and signal sets to be processed. There is no thread concept in the Linux kernel. Every execution entity is a task_struct structure, usually called a process. A process is an execution unit that maintains execution-related dynamic resources. At the same time, it references the static resources required by the program. When the system calls clone to create a child process, the child process can selectively share resources referenced by the parent process. Such a sub-process is usually called a lightweight process. Linux threads are implemented by the user-state pthread Library Based on lightweight processes.

When pthread is used, in the user's opinion, each task_struct corresponds to a thread, and a group of threads and a group of resources referenced by them are a process, however, a group of threads is not only enough to reference the same group of resources, but must also be considered as a whole, that is, a so-called thread group.

In short, a process can contain multiple threads, which are automatically scheduled by the kernel and each thread has its own thread context ), including thread ID, stack, stack pointer, program counter, general purpose register, and condition code. Other Resources in the process are shared by all threads, including virtual address space (Code, Data, heap, and shared library), file system information, file descriptor table, and signal processing program.

Implementation of threads in Linux

From the function called by the pthread_create interface, in Linux, threads are implemented through processes. The Linux Kernel provides a clone () system call for process creation. The clone parameters include CLONE_VM, CLONE_FILES, CLONE_SIGHAND, CLONE_THREAD. When creating a thread, the newly created process (also known as LWP (Lightweight process) shares memory space with the parent process through the clone () parameter, file descriptor table and signal processing program to achieve the same purpose of creating a thread.

Before Linux 2.4, the implementation of the phtread thread library is a lib named linuxthreads, but the library does not meet the requirements of POSIX. It is implemented in the user space, there are some problems in signal processing, process scheduling (each process requires an additional scheduling thread), and synchronizing and sharing resources among multiple threads.

After Linux2.4, the implementation of the phtread Thread Library is NPTL (Native POSIX Thread Library), which meets POSIX requirements. NPTL is a 1 × 1 thread model, that is, a thread schedules processes for an operating system. The implementation of NPTL depends on the modification of the Linux kernel. The kernel has the following modifications:

1) added futex (fast userspace mutex) in the kernel to support processing sleep and wake between threads. Futex is an efficient algorithm for mutually exclusive access to shared resources. Kernel plays an arbitration role in it, but it is usually completed by the process itself.

2) in Linux 2.4, the kernel has the thread group concept. All threads in the thread group share a PID, this PID is the so-called thread group identifier (TGID), and a field is added to the task_struct structure to store this value. If the newly created thread is the first thread in the thread group, that is, the main thread, the TGID value is the thread PID value, otherwise, the value of TGID is equal to the PID of the process (that is, the PID of the main thread ).

If getpid is called in the newly created thread, the returned value is the TGID, that is, the PID of the main thread (also known as the process PID ), to obtain the ID of the thread itself in the kernel, that is, the PID in tast_struct, you can call gettid to obtain it.

In the clone system call, pass the CLONE_THREAD parameter (that is, when the thread is created) you can set the TGID of the new process to the TGID of the parent process (otherwise, the TGID of the new process will be set to its own PID ). There are two similar IDs in task_struct: task> signal> pgid: The PID of the headers in the process group, task> signal> session: the pid of the session headers, these two IDs are used to associate the process group and session. With TGID, the kernel or related shell program will know whether a tast_struct represents a process or a thread, and when to display them, when should it not be displayed (for example, in ps, the thread is usually not displayed, but it will be displayed with option-L added ).

Execve functions similar to execve in any thread in the process, except the main thread, other threads will terminate and new programs will be executed in the main thread.

Note that the id discussed above is completely unrelated to the pthread_t of the thread, and in most system calls on processes that use PID as a parameter or function, the PID will be treated as the TGID and the operation will be applied to the entire thread group (process ).

3) to cope with "signals sent to processes" and "signals sent to threads", task_struct maintains two sets of signal_pending, one of which is shared by thread groups, one set is unique to the thread. The signal sent by kill is placed in the signal_pending shared by the thread group and can be processed by any thread. The signal sent by pthread_kill is the interface of the pthread library, corresponding System Call tkill) is placed in the unique signal_pending of the thread, which can only be processed by the current thread. When the thread stops/continues, or receives a fatal signal, the kernel will apply the processing action to the entire thread group.