Linuxthreads and nptl

The linuxthreads project initially introduced the concept of multithreading to Linux ?, However, linuxthreads does not comply with POSIX thread standards. Although the updated native POSIX thread Library (nptl) Library fills in some gaps, there are still some problems. This article describes the differences between the two Linux thread models for developers who need to port their applications from linuxthreads to nptl or who just want to understand the differences.

When Linux was initially developed, threads were not actually supported in the kernel. But it can indeed use the clone () system call to take the process as a schedulable entity. This call creates a copy of The Calling process, which shares the same address space with the calling process. The linuxthreads project uses this call to simulate thread support in user space completely. Unfortunately, this method has some shortcomings, especially in signal processing, scheduling, and inter-process synchronization primitives. In addition, this thread model does not comply with POSIX requirements.

To improve linuxthreads, we need Kernel support and rewrite the thread library. Two competing projects begin to meet these requirements. A team of IBM developers started the ngpt (next-generation POSIX Threads) project. At the same time, some Red Hat developers launched the nptl project. Ngpt was abandoned in the middle of 2003, leaving this field completely to nptl.

Although it seems to be an inevitable process from linuxthreads to nptl, if you are maintaining some applications for a long-history Linux release and plan to upgrade soon, so how to migrate to nptl will become an important part of the entire migration process. In addition, we may want to know the differences between the two, so that we can design our own applications so that they can better utilize these two technologies.

This article describes in detail the releases on which these thread models are implemented.

Linuxthreads design details

A thread divides an application into one or more tasks that run simultaneously. The difference between a thread and a traditional multi-task process is that a thread shares the status information of a single process and directly shares memory and other resources. Context switching between threads in a process is usually faster than Context switching between processes. Therefore, a multithreaded program can run faster than a multi-process application. In addition, we can implement parallel processing using threads. These advantages over process-based methods promote the implementation of linuxthreads.

The original design of linuxthreads believed that context switching between processes was fast, so each kernel thread was sufficient to process many related user-level threads. This leads to a one-to-one thread model revolution.

Let's review some basic concepts of the linuxthreads design details:

• A famous feature of linuxthreads is the management thread ). Management threads can meet the following requirements:

  • The system must be able to respond to the termination signal and kill the entire process.
  • The memory used in the form of a stack must be recycled after the thread is completed. Therefore, the thread cannot complete this process on its own.
  • The terminating threads must wait so that they do not enter the botnet state.
  • The collection of local data of a thread needs to traverse all threads; this must be done by the management thread.
  • If the main thread needs to call pthread_exit (), this thread cannot end. The main thread needs to enter the sleep state, and the Management thread is used to wake up the main thread after all threads are killed.
• To maintain the local data and memory of the thread, linuxthreads uses the high memory of the process address space (under the stack address ).

  The synchronization of primitives is implemented by signals. For example, the thread is blocked until it is awakened by a signal.

• Under the initial design of the clone system, linuxthreads implements each thread as a process with a unique process ID.

  The termination signal can kill all threads. After linuxthreads receives the termination signal, the management thread uses the same signal to kill all other threads (processes ).

• According to the design of linuxthreads, if an asynchronous signal is sent, the management thread will send the signal to a thread. If this thread is blocking this signal, it will be suspended. This is because the management thread cannot send this signal to the process. On the contrary, every thread is executed as a process.

  Scheduling between threads is handled by the kernel scheduler.

Linuxthreads and its limitations

The Design of linuxthreads can usually work well. However, the performance, scalability, and availability of applications that are under great pressure may be problematic. Let's take a look at some of the limitations of the linuxthreads design:

• It uses management threads to create threads and coordinates all threads owned by each process. This increases the overhead required to create and destroy threads.
• Because it is designed around a management thread, it will lead to a lot of context switching overhead, which may hinder the scalability and performance of the system.
• Since the management thread can only run on one CPU, the synchronization operations executed on SMP or NUMA systems may cause scalability problems.
• Because the thread management method and each thread uses a different process ID, linuxthreads is not compatible with other POSIX-related thread libraries.

The signal is used to implement the synchronization primitive, which affects the operation response time. In addition, the concept of sending signals to the main process does not exist. Therefore, this does not follow the POSIX Method for signal processing.

In linuxthreads, signal processing is based on the principle of each thread, rather than the principle of each process. This is because each thread has an independent process ID. Because the signal is sent to a dedicated thread, the signal is serialized-that is, the signal is transmitted to other threads through this thread. This is in stark contrast to the requirements of POSIX standards for concurrent processing of threads. For example, in linuxthreads, the signal sent through kill () is transferred to some separate threads, rather than the whole process. This means that if a thread blocks this signal, linuxthreads will only be able to queue this thread and process it when the thread opens this signal, instead of processing the signal immediately as in other threads without blocking signals.

Since every thread in linuxthreads is a process, the user and group ID information may not be common to all threads in a single process. For example, a multithreaded setuid ()/Setgid () process may be different for different threads.

In some cases, the multi-thread core dump does not contain all thread information. Similarly, this behavior is also the result of the fact that every thread is a process. If any thread has a problem, we can only see the thread information in the core file of the system. However, this behavior is mainly applicable to the implementation of linuxthreads in earlier versions.

Since each thread is a separate process, the/proc directory will be filled with many process items, which should actually be a thread.

Since each thread is a process, only a limited number of threads can be created for each application. For example, on the ia32 system, the total number of available processes-that is, the total number of threads that can be created-Is 4,090.

Because the local data method of the computing thread is based on the location of the stack address, the access speed to the data is very slow. Another drawback is that you cannot specify the size of the stack in a trusted manner, because you may map the stack address to the region that would have been used for other purposes. The concept of on-demand growth (grow on demand) is implemented in the Linux Kernel 2.4.10. Previously, linuxthreads used a fixed stack.

About nptl

Nptl, or native POSIX thread Library, is a new implementation of Linux threads. It overcomes the shortcomings of linuxthreads and also meets POSIX requirements. Compared with linuxthreads, it provides significant improvements in performance and stability. Like linuxthreads, nptl also implements a one-to-one model.

Ulrich drepper and Ingo Molnar are two employees in Red Hat's nptl design. Their overall design goals are as follows:

• This new thread library should be compatible with the POSIX standard.

  This thread implementation should also work well on systems with many processors.

  Creating a New thread for a small task requires a low startup cost.

• The nptl thread library should be Binary compatible with linuxthreads. Note that ld_assume_kernel can be used for this purpose, which will be discussed later in this article.
• This new thread library should be able to take advantage of NUMA support.

Advantages of nptl

Compared with linuxthreads, nptl has many advantages:

• Nptl does not use a management thread. Some requirements of Management threads, such as sending termination signals to all threads as part of the process, are not required because the kernel itself can implement these functions. The kernel also processes the memory used by each thread stack for recycling. It even waits before the parent thread is cleared to manage the end of all threads, so as to avoid zombie processes.
• Because nptl does not use management threads, its thread model has better scalability and synchronization mechanisms on NUMA and SMP systems.
• By using the nptl thread Library and the new kernel implementation, we can avoid using signals to synchronize threads. For this purpose, nptl introduced a new mechanism named futex. Futex works in the shared memory area, so it can be shared among processes. This provides the POSIX synchronization mechanism between processes. We can also share a futex between processes. This behavior makes synchronization between processes possible. In fact, nptl contains a pthread_process_shared macro, allowing developers to share mutex locks between user-level processes in different threads.
• Because nptl is POSIX compatible, it processes signals according to the principles of each process. getpid () returns the same process ID for all threads. For example, if a sigstop signal is sent, the entire process will stop. With linuxthreads, only the thread that receives the signal will stop. In this way, nptl-based applications can better utilize the debugger, such as GDB.
• Because all threads in nptl have a parent process, the resource usage (such as CPU and memory percentage) reported by the parent process is counted, instead of counting a thread.

An Implementation feature introduced by the nptl thread library is the support for Abi (Application binary interface. This helps achieve backward compatibility with linuxthreads. This feature is implemented by using ld_assume_kernel. The following describes this feature.

Ld_assume_kernel environment variable

As described above, the introduction of ABI enables both nptl and linuxthreads models to be supported. Basically, this is handled through LD (a dynamic linker/loader), and it will decide which runtime thread library to dynamically link.

For example, the following are some general settings used by WebSphere Application Server for this variable. You can set them as needed:

• Ld_assume_kernel = 2.4.19: This overwrites the nptl implementation. Generally, this implementation uses the standard linuxthreads model and enables the floating stack feature.
• Ld_assume_kernel = 2.2.5: This overwrites the nptl implementation. This implementation usually indicates that the linuxthreads model is used, and the fixed stack size is used at the same time.

We can use the following command to set this variable:

Export ld_assume_kernel = 2.4.19

Note that the support for any ld_assume_kernel settings depends on the ABI version of the currently supported thread library. For example, if the thread library does not support Abi of version 2.2.5, you cannot set ld_assume_kernel to 2.2.5. Generally, nptl requires 2.4.20, while linuxthreads requires 2.4.1.

If you are running a Linux release with nptl enabled, but the application is designed based on the linuxthreads model, all these settings are generally usable.

Gnu_libpthread_version macro

Most modern Linux distributions are pre-installed with linuxthreads and nptl, so they provide a mechanism to switch between them. To view which thread library is being used on your system, run the following command:

$ Getconf gnu_libpthread_version

This produces output results similar to the following:

Nptl 0.34

Or:

Linuxthreads-0.10

The thread model, glibc version, and kernel version used by the Linux release

Table 1 lists some popular Linux distributions, as well as their thread implementation types, glibc libraries, and kernel versions.

Table 1. Linux release and thread implementation

Thread implementation	C library	Release	Kernel
Linuxthreads 0.7, 0.71 (for libc5)	Libc 5.x	Red Hat 4.2
Linuxthreads 0.7, 0.71 (for glibc 2)	Glibc 2.0.x	Red Hat 5.x
Linuxthreads 0.8	Glibc 2.1.1	Red Hat 6.0
Linuxthreads 0.8	Glibc 2.1.2	Red Hat 6.1 and 6.2
Linuxthreads 0.9		Red Hat 7.2	2.4.7
Linuxthreads 0.9	Glibc 2.2.4	Red Hat 2.1	2.4.9
Linuxthreads 0.10	Glibc 2.2.93	Red Hat 8.0	2.4.18
Nptl 0.6	Glibc 2.3	Red Hat 9.0	2.4.20
Nptl 0.61	Glibc 2.3.2	Red Hat 3.0 el	2.4.21
Nptl 2.3.4	Glibc 2.3.4	Red Hat 4.0	2.6.9
Linuxthreads 0.9	Glibc 2.2	SuSE Linux Enterprise Server 7.1	2.4.18
Linuxthreads 0.9	Glibc 2.2.5	SuSE Linux Enterprise Server 8	2.4.21
Linuxthreads 0.9	Glibc 2.2.5	United Linux	2.4.21
Nptl 2.3.5	Glibc 2.3.3	SuSE Linux Enterprise Server 9	2.6.5

Note: from kernel 2.6.x and glibc 2.3.3, the version naming conventions used by nptl have changed: the Library is now numbered based on the version of glibc used.

Java Virtual Machine (JVM) support may be slightly different. Ibm jvm supports most releases with glibc Versions later than 2.1 in table 1.

Conclusion

The limitations of linuxthreads have been overcome in nptl and later versions of linuxthreads. For example, the latest linuxthreads implementation uses thread registration to locate local thread data. For example, on an Intel processor, it uses the % FS and % GS segment registers to locate the virtual address used to access the local data of the thread. Although this result shows the improvement results of some modifications adopted by linuxthreads, it still has many problems in higher load and stress testing, because it relies too much on a management thread, it is used for signal processing and other operations.

Remember to use the-d_reentrant compile flag when using linuxthreads to build a library. This makes the library thread safe.

Finally, it may be the most important thing. Remember that the creators of the linuxthreads project no longer actively update it. They think that nptl will replace linuxthreads.

The disadvantage of linuxthreads does not mean that nptl has no error. As an SMP-oriented design, nptl also has some disadvantages. I have seen this problem in the recent Red Hat kernel: A simple thread runs well on a single processor machine, but hangs on an SMP machine. I believe that there is more work to be done on Linux to make it more scalable and meet the needs of high-end applications.