Process categories under Linux (kernel threads, lightweight processes, and user processes) and how they are created--linux process management and scheduling (iv)

This document declares

Date	Kernel version	Architecture	author	GitHub	CSDN
2016-05-12	Linux-4.5	X86 & Arm	Gatieme	Linuxdevicedrivers	Linux process management and scheduling-the creation of processes

The concepts that appear in this article, kernel threads, lightweight processes, user processes, user threads, and so on, can be found if you are not familiar with

Kernel thread, lightweight process, user thread three thread concept (thread ≠ lightweight process)

Linux Process Categories

While we are differentiating the Linux process categories, I would like to say that there is only one type of process under Linux, and that is task_struct, and of course I also want to say that Linux does not actually have a thread concept, just that the processes that share resources with other processes are called threads.

1. A process because of its different running space, so that there are kernel threads and user processes , kernel threads run in kernel space, the reason is called a thread because it has no virtual address space, only access to the kernel code and data, and user processes run in user space, However, you can get into the kernel state from the user state by means of interrupts, system calls, etc.

2. The user process runs on the user space, and some of the processes that are implemented through shared resources are called thread groups, and the Linux kernel actually has no threading concept, and Linux threads are actually the process of sharing certain resources with other processes. But we're used to calling them threading or lightweight processes .

As a result, there are 3 processes on Linux, kernel threads (or core processes), user processes, user threads, and, of course, if more rigorous, you can assume that both user processes and user threads are user processes.

The concept of lightweight processes is not actually equivalent to threading

Depending on the implementation of different operating systems, the lightweight process is actually a different concept.

For more information, see the Wikipedia-LWP lightweight process

or my other blog core thread, lightweight process, user thread three thread concept (thread ≠ lightweight process)

In computer operating systems, a light-weight process (LWP) is a means of achieving multitasking. In the traditional meaning of the term, as used in Unix System V and Solaris, a LWP runs in user space on top of a single Kernel thread and shares its address space and system resources with other lwps within the same process. Multiple user level threads, managed by a thread library, can is placed on top of one or many lwps-allowing multitasking To being done at the user level, which can has some performance benefits. [1]

In some operating systems there is no separate LWP layer between kernel threads and user threads. This means, user threads is implemented directly on top of kernel threads. In those contexts, the term ' light-weight process ' typically refers to kernel threads and the term ' threads ' can refer to User threads. [2] on Linux, user threads is implemented by allowing certain processes to share resources, which sometimes leads to thes E processes to is called "Light weight Processes". [3] [4] Similarly, in SunOS version 4 onwards (prior to Solaris) ' light weight process ' referred to user threads.

Processes and Threads

A process is a run-time activity of a program with independent functionality about a data collection. It can apply and own system resources, is a dynamic concept, is an active entity. It is not just the code of the program, it also includes the current activity, which is represented by the value of the program counter and the contents of the processing register. A process is an "executing program". A program is an inanimate entity that can become an active entity when the processor is given the program's life, which we call a process.

Typically, you can include several threads in a process that can take advantage of the resources owned by the process. In the operating system in which threading is introduced, the process is usually used as the basic unit for allocating resources, while threads are used as the basic unit of independent operation and independent Dispatch. Because the thread is smaller than the process, and basically does not have the system resources, the cost of its scheduling will be much smaller, can be more efficient to improve the degree of concurrent execution among the multiple programs within the system.

The difference between a thread and a process is that the child process and the parent process have different code and data spaces, while multiple threads share the data space, and each thread has its own execution stack and program counter for its execution context. Multithreading is mainly to save CPU time, play use, depending on the situation. The running of a thread requires the use of the computer's memory resources and CPU.

A process is a program with a certain independent function about a single run activity on a data set, a process that is an independent unit of the system's resource allocation and scheduling. A thread is an entity of a process that is the basic unit of CPU dispatch and dispatch, which is a smaller unit that can run independently than a process. The thread itself does not own the system resources, it has only a few resources (such as program counters, a set of registers and stacks) that are essential in the run, but it can share all the resources owned by the process with other threads belonging to one process.

The difference between threads and processes is summarized:

• Address space and other resources: processes are independent of each other and are shared among threads of the same process. Threads within a process are not visible in other processes.

• Communication: Inter-process communication IPC, between threads can directly read and write process data segments (such as global variables) to communicate--requires process synchronization and mutual exclusion means of support to ensure data consistency.

• Scheduling and switching: Thread context switches are much faster than process context switches.

• In a multithreaded OS, a process is not an executable entity.

Kernel threads

A kernel thread is a clone of the kernel, and a single clone can handle a specific thing. This is particularly useful when dealing with asynchronous events such as asynchronous IO. The use of kernel threads is inexpensive, and the only resource used is the space for storing registers when the kernel stack and context switches. Multithreading-enabled cores are called multithreaded cores (multi-threads kernel).

• Kernel threads run only in the kernel state and are not affected by the user-state context.

• Processor competition: Can compete for processor resources on a system-wide scale;

• Use resources: The only resources that are used are the space that holds the registers when the kernel stack and context switches

• Scheduling: The overhead of scheduling can be almost as expensive as the process itself

• Synchronization efficiency: Synchronization of resources and data sharing are lower than the data synchronization and sharing of the entire process.

The process of creating a Linux process

A kind of abstract concept commonly used in modern programming technology of threading mechanism. This mechanism provides a set of threads that share memory address space within the same program, opening files and resources.

Replication of the process fork and load Execve

We are programming under Linux, often by fork out a new program, fork Conghua literally understood to mean "fork", which in fact means that our fork process is not really created from scratch.

A process, which consists of code, data, and resources assigned to a process, is actually a copy of the existing process (the parent process) that is copied out (the subprocess), and the fork () function creates a process that is almost identical to the original process through a system call, that is, two processes can do exactly the same thing, Then if we load the new application through EXECVE for the child process, then the new process will start executing the new application

In short, the new process is created by fork and Execve, first fork out a basic consistent copy from the parent process, and then load the new application image via Execve

• Fork generates an identical copy of the current process, which becomes a child process

All the resources of the original process (the parent process) are copied to the new process (child process) in the appropriate way. Therefore, after the system call, the original process has two separate instances, the two instances of the connection includes: the same set of open files, the same working directory, process virtual space (memory) of the same data (of course, two processes each have a copy, that is, their virtual address is the same, But the corresponding physical address is different) and so on.

• Execve loading an application from an executable binary image to replace the currently running process

In other words, a new application is loaded. So EXECV is not creating a new process

So when we're working on Linux to create an application, the thing that actually does is

1. First Use fork to copy an old process

2. Then call Execve to load a new application for the new process

Write-time Replication technology

Some people think that such a large volume of replication can lead to inefficient execution. In fact, during the replication process, Linux uses a copy-on-write strategy.

Write-time replication (copy-on-write) is an optimization strategy that is used in the field of program design. The underlying idea is that if multiple callers (callers) require the same resources simultaneously, they will collectively get the same metric pointing to the same resource until a caller (caller) attempts to modify the resource, and the system will actually replicate a copy to the caller. To avoid being directly aware of the modified resources, this process is only transparent to the other calls (transparently). The main advantage of this approach is that if the caller does not modify the resource, no replica (private copy) is created.

The first-generation Unix system implements a fool-like process creation: when a fork () system call is made, the kernel copies the entire address space of the parent process as-is and assigns the copied copy to the child process. This behavior is time-consuming, and this method of creating address space involves many memory accesses, consumes many CPU cycles, and completely destroys the content in the cache. In most cases, this is often meaningless because many child processes start their execution by loading a new program, which completely discards the inherited address space.

The Linux kernel now uses a more efficient method called write-time replication (copy on Write,cow). The idea is quite simple: the parent process and the child process share the page frame instead of copying the page frame. However, as long as the page frames are shared, they cannot be modified, that is, page frames are protected. When a parent or child process attempts to write a shared page frame, an exception occurs, and the kernel copies the page to a new page frame and marks it as writable. The original page frame is still write-protected: When another process attempts to write, the kernel checks whether the write process is the only owner of the page frame, and if so, marks the page frame as writable for the process.

When process a uses system call fork to create a child process B, because child process B is actually a copy of parent process A,

Therefore, it will have the same physical page as the parent process. To save memory and speed up creation, the fork () function causes child process B to share the physical page of parent process A in a read-only manner. The access rights of parent process A to these physical pages are also set to read-only.

Thus, when either parent process A or child process B performs a write operation on these shared physical pages, a page error exception (Page_fault int14) interrupt occurs, and the CPU executes the system-supplied exception handler Do_wp_page () to resolve the exception.

Do_wp_page () cancels the shared operation on the physical page that caused the write exception to be interrupted, copying a new physical page for the write process, and having the parent process A and child process B each own a physical page of the same content. Finally, when returned from the exception handler, The CPU will re-execute the write instructions that led to the exception, and the process continues.

After a process calls the fork () function, the system first assigns resources to the new process, such as space for storing data and code. Then all the values of the original process are copied into the new process, and only a few values are different from the original process values (such as PID). The equivalent of cloning a self.

About Process-created

See Fork,vfork and Clone in Linux (differences and contact)

Implementation mechanisms for different operating system threads specialized thread-supported system-LWP mechanism

Threads better support concurrent programming techniques, and on multiprocessor systems, he can guarantee real parallel processing. Threads are supported by operating systems such as Microsoft windows or Sun Solaris.

All of these systems provide a dedicated thread-support mechanism in the kernel, and Unix System V and Sun Solaris use threading as a lightweight process (lwp-light-weight processes), in which, compared to heavyweight processes, Threads are abstracted into an execution unit that consumes less resources and runs quickly.

Implementation mechanism of Linux offline thread

But the Linux implementation threading mechanism is very unique. From the kernel point of view, he does not have the concept of threading. Linux implements all processes as processes. The kernel does not prepare special scheduling algorithms or define special data structures to represent threads. Instead, a thread is only considered a process that shares certain resources with other processes. Each thread has its own unique task_struct, so in the kernel it looks like a normal process (just a thread sharing some resources with other processes in the same group)

Before the Linux process descriptor TASK_STRUCT The structure of the –linux process management and scheduling (a) and the Linux process ID number –linux process management and scheduling (iii) in the process of the PID number explained, we mentioned that the process task_struct The PID store is the kernel to the process of the unique flag , that is, the process is marked with the process number, for the thread is its thread number, then for the thread to say that all the thread group threads with the lead thread has the same process number, stored in the Tgid field

So getpid () returns the process number of the current process, which should return the Tgid value instead of the PID value, and for the user space the same group of threads have the same process number as tpid, and for the kernel, there is no concept of threading in Chengdu. Then the PID is the kernel that uniquely distinguishes each process from its mark.

It's the smart way to implement Linux under Group Management, write-time replication, etc.

• The cost of creating a process or thread under Linux is minimal

• Since either the thread or the process kernel is indistinguishable, a group of threads that share the address space or resources can form a group of threads, so that other processes can form a group of processes even if they do not share resources, and even a group of process groups can make up a conversation group, and the process group simplifies the operation of signaling to all groups of processes. A set of sessions is also more adaptable to multi-channel programming environments

Differences in implementation mechanisms

In summary, Linux threads are completely different from specialized threading support systems.

Unix System V and Sun Solaris Use user threads as lightweight processes (lwp-light-weight process), compared to heavyweight processes, where threads are abstracted into an execution unit that consumes less resources and runs quickly.

For Linux, the user thread is just a means of sharing resources between processes, and the process itself is lightweight compared to other system processes.

For example, if we have a process that includes four threads,

In systems that provide specialized threading support, there is usually a process descriptor that contains a pointer to only four different threads. This descriptor copies a shared resource that describes a file such as an address space, open. The thread itself goes on to describe its exclusive resources.

Instead, Linux simply creates four processes and allocates four ordinary task_struct structures, and then establishes these four processes when they share certain resources.

Kernel threads

The Linux kernel can be viewed as a service process (managing hardware and software resources, responding to the various reasonable and unreasonable requests of user processes). The kernel requires multiple execution streams to be parallelized, and multithreading is necessary to prevent possible blocking.

A kernel thread is a clone of the kernel, and a single clone can handle a specific thing. The Linux kernel uses kernel threads to divide the kernel into several functional modules, such as KSWAPD, KFLUSHD, etc., which are particularly useful when dealing with asynchronous events such as asynchronous IO. The use of kernel threads is inexpensive, and the only resource used is the space for storing registers when the kernel stack and context switches. Multithreading-enabled cores are called multithreaded cores (multi-threads kernel). Kernel thread Scheduling is the responsibility of the kernel, while one kernel thread is in a blocking state without affecting other kernel threads because it is the basic unit of dispatch. This is not the same as the user thread.

Kernel threads run only in the kernel state and are not affected by the user-state context.

• Processor competition: Can compete for processor resources on a system-wide scale;

• Use resources: The only resources that are used are the space that holds the registers when the kernel stack and context switches

• Scheduling: The overhead of scheduling can be almost as expensive as the process itself

• Synchronization efficiency: Synchronization of resources and data sharing are lower than the data synchronization and sharing of the entire process.

The similarities and differences between kernel threads and ordinary processes

1. Like normal processes, kernel threads also have priority and are scheduled.
   Kernel threads have the opportunity to get more CPU resources when they have the same static_prio as the user process

2. Kernel thread bugs directly affect the kernel, it is easy to kill the entire system, but the user process is under the management of the kernel, the most serious of its bugs will only be the whole crash

3. Kernel threads do not have their own address space, so their "current->mm" are empty;

4. Kernel threads can only operate in kernel space and cannot interact with user space;

The kernel thread does not need to access the user space memory, which is no better. So the task_struct mm field of the kernel thread is empty

But just now, kernel threads also have a core stack, and no mm how to access its core stack? This core stack shares 8k of space with Task_struct's thread_info, so no mm description is used.

But kernel threads always have to access the kernel space of other kernels ah, no mm domain after all, it is not possible.
So when the kernel thread is called, the kernel will point its task_strcut active_mm to the MM domain of the previous scheduled process, and the kernel thread can use the memory descriptor of the previous process when needed.

Because kernel threads do not access user space, only kernel space memory is manipulated, and all processes have the same kernel space. This saves the memory of a mm field.

Kernel thread creation

In the kernel, there are two ways to generate kernel threads, one is to use the Kernel_thread () interface, and the other is to use the kthread_create () interface

Kernel_thread

First of all, the Kernel_thread interface, the thread created with the interface, must call the Daemonize () function in that thread, because the thread is only considered a kernel thread when the parent process of the thread points to "Kthreadd", and exactly daemonize () The main task of the function is to change the thread's parent process to "kthreadd" kernel thread; By default, after calling Deamonize (), all signals are blocked and if you want to manipulate a signal you can call the Allow_signal () function.

int kernel_thread(int (*fn)(voidvoidunsignedlong flags);             // fn为线程函数，arg为线程函数参数，flags为标记void daemonize(constchar// name为内核线程的名称

Kthread_create

While the Kthread_create interface is the standard kernel thread creation interface, the kernel thread can be created only by invoking the interface, and the thread created by default is stored in a non-operational state, so the thread needs to be started in the parent process by calling the Wake_up_process () function.

struct task_struct *kthread_create(int (*threadfn)(void *data),void *data,                                  constchar//threadfn为线程函数;data为线程函数参数;namefmt为线程名称，可被格式化的, 类似printk一样传入某种格式的线程名

After the thread is created, it does not run immediately, but it needs to pass the Task_struct pointer returned by Kthread_create () to Wake_up_process () and then run the thread through this function.

Kthread_run

Of course, there is also a function to create and start a thread: Kthread_run

struct task_struct *kthread_run(int (*threadfn)(void *data),                                    void *data,                                    constchar *namefmt, ...);

Once the thread is started, it runs until the thread is actively called by the Do_exit function, or another process calls the Kthread_stop function to end the thread's run.

int  kthread_stop ( Span class= "Hljs-keyword" >struct  task_struct *thread); Kthread_stop () sends a signal to the thread. If the thread function is working on a very important task, it will not be interrupted. Of course, if the thread function never returns and does not check the signal, it will never stop. "' Cint  wake_up_process (struct  task_struct *p ); //wake-up thread  struct  task_struct *kthread_run (int  (*                                THREADFN) (void  *data), void  *data, const  char  namefmt[], ...); //is the sum of the functions of the above two functions   

Because the thread is also a process, its structure is also the struct task_struct that uses the process.

Kernel thread exit

When the thread executes to the end of the function, it automatically calls the kernel in the Do_exit () function to exit or another thread calls Kthread_stop () to specify that the thread exits.

Summarize

Linux uses task_struct to describe processes and threads

1. A process because of its different running space, so that there are kernel threads and user processes , kernel threads run in kernel space, the reason is called a thread because it has no virtual address space, only access to the kernel code and data, and user processes run in user space, Cannot directly access the data of the kernel but can be trapped in the kernel state from the user state by means of interrupts, system calls, etc., but the kernel state is just a status of the process, which is fundamentally different from the kernel thread.

2. The user process runs on the user space, and some of the processes that are implemented through shared resources are called thread groups, and the Linux kernel actually has no threading concept, and Linux threads are actually the process of sharing certain resources with other processes. But we're used to calling them threading or lightweight processes .

As a result, there are 3 processes on Linux, kernel threads (or core processes), user processes, user threads, and, of course, if more rigorous, you can assume that both user processes and user threads are user processes.

Kernel threads have process descriptors, PID, process body segments, core stacks

User processes have process descriptors, PID, process body segments, core stacks, user-space data segments, and stacks

User threads have process descriptors, PID, process body segments, core stacks, data segments and stacks for the half-process to share user space

User threads can also be user processes through the EXEC function family, which has its own user-space data segments and stacks.

Process categories under Linux (kernel threads, lightweight processes, and user processes) and how they are created--linux process management and scheduling (iv)