Avoid Memory leakage in POSIX Thread Programming

POSIX thread (pthread) programming defines a set of standard C programming language types, functions, and constants-And pthreads provides a powerful thread management tool. To make full use of pthreads, you must avoid common errors. A common error is that you forget to join a thread that can be joined, resulting in Memory leakage and increased workload. In this skillful article, learn POSIX thread basics, learn how to identify and detect thread memory leaks, and obtain reliable suggestions to avoid this situation.

 

 

Introduction to POSIX Threads

The main reason for using threads is to improve program performance. Thread creation and management only requires a small operating system overhead and a small amount of system resources. All threads in a process share the same address space, making inter-thread communication more efficient and easier to implement than inter-process communication. For example, if a thread is waiting for an input/output system call to complete, other threads can process CPU-intensive tasks. Through threads, you can give priority to important tasks-or even interrupt-low-priority tasks. You can place occasional tasks between scheduled tasks to create scheduling flexibility. Finally, pthreads is an ideal option for parallel programming on multiple CPU computers.

The main reason for using POSIX Threads or pthreads is simpler: as part of the standard C language thread programming interface, they can be highly portable.

POSIX thread programming has many advantages, but if you do not specify some basic rules, you may write some code that is difficult to debug and cause memory leakage. First, let's review POSIX Threads, which are dividedConnectable threadsOrSeparating threads.

Connectable threads

If you want to generate a new thread and know how it is terminated, you need a thread that can be joined. For connectable threads, the system allocates dedicated memory to store the thread termination status. The status is updated after the thread is terminated. To obtain the thread termination status, callpthread_join(pthread_t thread, void** value_ptr).

The system allocates underlying storage for each thread, including the stack, thread ID, and thread termination status. This underlying storage will remain in the process space (and cannot be recycled) until the thread ends and is connected by other threads.

Separating threads

Most of the time, you only need to create a thread, allocate some tasks to it, and then continue to process other transactions. In these cases, you do not pay attention to how the thread is terminated. Using a separate thread is a good choice.

For a detached thread, the system automatically recycles its underlying resources after the thread is terminated.

Identify leaks

If you create a thread that can be joined but forget to join it, its resources or private memory will be stored in the process space and never be recycled.Be sure to join a thread that can be joined; otherwise, it may cause serious memory leakage.

For example, a thread on Red Hat Enterprise Linux (RHEL4) needs a 10 MB stack, which means that if it is not connected, there will be at least 10 MB memory leakage. Suppose you design a program in Manager-worker thread mode to process incoming requests. Then, you need to create more and more worker threads to execute various tasks and finally terminate these threads. If they are connectable threads and you have not calledpthread_join()After the thread is terminated, each generated thread will leak a large amount of memory (at least 10 MB per stack ). As more and more worker threads are created and terminated without connection, the amount of memory leaked continues to increase. In addition, the process cannot create a new thread because there is no memory available for creating a new thread.

Listing 1 shows the serious memory leakage that occurs when you forget to join a thread that can be joined. You can also use this code to check the maximum number of threads that can coexist in a process space.

Listing 1. Memory leakage

#include<stdio.h>#include<pthread.h>void run() { pthread_exit(0);}int main () { pthread_t thread; int rc; long count = 0; while(1) { if(rc = pthread_create(&thread, 0, run, 0) ) { printf("ERROR, rc is %d, so far %ld threads created/n", rc, count); perror("Fail:"); return -1; } count++; } return 0;}

In list 1pthread_create()Create a new thread with the default thread attributes. By default, new threads can be joined. It creates new connectable threads until a fault occurs. Then output the error code and cause.

When you use the following command to compile the code in Listing 1 on Red Hat Enterprise Linux Server 5.4:[root@server ~]# cc -lpthread thread.c -o thread, You will get the results shown in Listing 2.

List 2. Memory leakage results

[root@server ~]# ./threadERROR, rc is 12, so far 304 threads createdFail:: Cannot allocate memory

It cannot create more threads after the Code creates 304 threads. The error code is12, Which means no more memory is available.

As shown in lists 1 and 2, although a thread can be joined is generated, it is not connected. Therefore, each terminated thread can still occupy process space and leak process memory.

A posix thread on RHEL has a private stack of 10 MB. In other words, the system allocates at least 10 MB of dedicated storage for each pthread. In our example, 304 threads are created before the process is stopped. These threads occupy 304*10 MB of memory, totaling about 3 GB. The virtual memory size of a process is 4 GB, of which 1/4 of the process space is reserved for the Linux kernel. In this way, 3 GB of memory space is available for users. Therefore, 3 GB memory is consumed by dead threads. This is a serious memory leak. And it's easy to understand why it happened so quickly.

To fix the leakage, you can add code to callpthread_join()This method can be used to connect each thread that can be joined.

Leakage Detection

As with other memory leaks, the problem may not be so obvious during process startup. This section describes how to detect such problems without accessing the source code:

1. Calculates the number of thread stacks in a process.This includes the number of active and terminated threads.
2. Calculates the number of active threads running in the process.
3. Compare the two.If the number of existing thread Stacks is greater than the number of active threads that are running, and the difference between the two numbers is increasing while the program is running, there is a leak in the number.

This kind of memory leakage is probably caused by failure to connect to the thread that can be joined.

Use pmap to calculate the number of thread stacks

In a running process, the number of thread stacks equals the number of thread bodies in the process. The thread body includes the running active thread and the dead thread that can be joined.

pmapIs a Linux tool used to report process memory. Use the following command to obtain the number of thread stacks:

[root@server ~]# pmap PID | grep 10240 | wc -l

(5.4 kb is the default stack size on Red Hat Enterprise Linux server .)

Use/proc/Pid/task to calculate the number of active threads

Each time you create a thread and the thread is running, an entry is filled in/proc/Pid/task. When a thread is terminated, the entry is deleted from/proc/Pid/task regardless of whether the thread can be joined or detached. Therefore, the number of active threads can be obtained by running the following command:

[root@server ~]# ls /proc/PID/task | wc -l.

Comparison output

Checkpmap PID | grep 10240 | wc -lAndls /proc/PID/task | wc -l. If the number of all thread Stacks is greater than the number of active threads and the difference between the two continues to grow when the program is running, you can determine that the memory leakage problem exists.

Leakage Prevention

The connectable thread should be connected during programming. If you create a thread that can be joined in the program, do not forget to callpthread_join(pthread_t, void**)To recycle the dedicated storage allocated to the thread. Otherwise, it will cause serious memory leakage.

You can usepmapAnd/proc/PID/taskCheck whether the leakage exists. If it does exist, check the source code to see if all the joining threads are connected.

That's all. With only a small amount of prevention work, you can save a lot of subsequent work to avoid the headache of Memory leakage.