Multi-thread synchronization mutex on Windows platform: kernel source code analysis-Sub-thread (3)

Author: Chen Xi

Date: 13:05:34

Environment: [win7 32-bit OS intel I3 support 64-bit command vs2010; wrk-v1.2; source insight]

Reprinted please indicate the source

Q1: A simple thread example for Windows.

A: Save As thread_test.c as follows:

#include <windows.h>#include <stdio.h> #define PRINT_U(ulongValue)       printf(#ulongValue" is %lu\n", ((unsigned long)ulongValue));int main()  {      HANDLE thread = GetCurrentThread();    HANDLE process = GetCurrentProcess();    PRINT_U(GetThreadId(thread))    PRINT_U(GetProcessId(process))        getchar();    return 0;  } 

Compile thread_test.exe and run:

We can see that the thread ID and process ID are obtained. We can also view the thread ID in the task manager:

We can also see that the PID is indeed output, and the number of threads is 1, which indicates that there is only one main thread. If you want more information, you can use procexp.exe (provided by sysinternals) of Microsoft to view:

You can see where the thread_text.exe process is located. Double-click to enter:

Here we can see that the thread ID is indeed the 7416 output above; at the same time, we can also see that this thread runs from _ maincrtstartup. Click the stack button to view the stack information:

The figure above describes the stack information of the thread running, and the position of the thread running in different modules (Note: running .). here we can also see the relationship between threads running in Kernel Status calls.

Q2: What is the difference between createthread and _ beginthread? Why do people often say that using createthread may cause memory leakage?

A: From the Perspective of purpose, they all aim to create a thread, but the details are different: the former is the system API, this means that it is not bound to the C Library and other libraries normally used by the program. The latter is the C Runtime function provided by Microsoft. Therefore, _ beginthread may do something to maintain the normal operation of the C library, while the createthread function is simple. You can easily find the differences between them by checking their source code. No code is posted here.

If you already know that they belong to different levels, it is easy to understand why the createthread thread may cause memory leakage.

However, on platforms such as win7 or 2003 Server, even if createthread is used to create a subthread, the strtok function of the C library is called in the subthread, and no leakage will occur, the reason is that the thread exits and releases fiber local storage, which correctly releases the data stored locally by the thread.

The following code:

#include <windows.h>#include <stdio.h> #include <process.h>#define PRINT_U(ulongValue)       printf(#ulongValue" is %lu\n", ((unsigned long)ulongValue));DWORD thread_func(LPVOID *arg){    char str[] = "111,222,333";    char *p;    printf("thread_func begins...\n");    // print all tokens seperated by ','    p = strtok(str, ",");    while(p)    {printf("token:%s \n", p);p = strtok(NULL, ",");    }        printf("thread_func ends...\n");    return 0;}int main()  {      HANDLE thread;    thread = CreateThread(NULL, 0, (LPTHREAD_START_ROUTINE)thread_func, NULL, 0, NULL);    if(!thread)    {        perror("CreateThread error");        return -1;    }    WaitForSingleObject(thread, INFINITE);  // wait for son thread to exit    CloseHandle(thread);    printf("main thread will end...\n");    return 0;  }  

Add a breakpoint to the _ freefls function in the tidtable. c file and debug and run it:

We can see that the _ freefls function is executed when the sub-thread exits from the thread, and the TLS structure PTD is released internally.

Of course, you still need to note that the method for linking the program to the C library is static link, that is, using/Mt or/MTD, instead of dynamically linking DLL to/MD or/MDD. Because the TLS data is automatically released during DLL initialization and exit in the form of dynamic link to the C library, it cannot be verified whether exitthread releases TLS.

In addition, as mentioned above, running programs on win7 and Windows Server 2003 virtual machines conforms to the above analysis, that is to say, after createthread creates a thread, the thread internally calls a function using the TLS structure. For example, after strtok is created, memory leakage will not occur. However, when I run this program on XP, I find Memory leakage. We can test it on our own (it is best to use the while loop to constantly create threads so that we can clearly observe the memory leakage process. On Windows Server 2003 or Windows server, the memory will fluctuate, however, as the thread ends, the corresponding structure is released, the memory occupied by the process is always in a small fluctuation range, and the memory usage increases rapidly on XP ).

However, no matter how much memory a process exposes, the memory will be released at the end of the process, so after the end, the memory will be recycled, you don't have to be afraid that your machine will run out of memory several times.

In addition, I checked NTDLL. the source of the _ rtlprocessflsdata function in the DLL module found that it was introduced from the Vista system. So I guess the Vista system is similar to the running status of the above win7 and Server 2003. This is not tested, if anyone has this system or virtual machine, you can test it easily.

Q3: What is the difference between the event object created by createevent and the mutex created by createmutex?

A: In fact, the intuitive feeling of event is more inclined to synchronize, while mutex is more inclined to mutually exclusive. However, synchronization mutex is not a contradiction, and synchronization sometimes means mutual exclusion. Mutual exclusion means synchronization, most of the time they are used together. Mutex is no longer used as an example. The following is an example of event stored as test_event.c:

#include <windows.h>#include <stdio.h> #include <process.h>#include <tchar.h>#define PRINT_U(ulongValue)       printf(#ulongValue" is %lu\n", ((unsigned long)ulongValue));HANDLE waitDataEvent;HANDLE waitThreadEndEvent;static int  data = 0;DWORD thread_func(LPVOID *arg){    printf("thread_func begins...\n");    WaitForSingleObject(waitDataEvent, INFINITE);   // wait for the dataEvent's be signaled    Sleep(1000);    printf("son thread update:main thread has set data:%d...\n", data);        printf("thread_func ends...\n");    SetEvent(waitThreadEndEvent);   // tell main thread that it will exit    return 0;}int main()  {      HANDLE thread;    thread = CreateThread(NULL, 0, (LPTHREAD_START_ROUTINE)thread_func, NULL, 0, NULL);    if(!thread)    {perror("CreateThread error");return -1;    }    waitDataEvent = CreateEvent(NULL, TRUE, FALSE, _T("dataEvent"));   // init to FALSE    if(!waitDataEvent)    {perror("CreateEvent waitDataEvent error");CloseHandle(thread);return -1;    }    waitThreadEndEvent = CreateEvent(NULL, TRUE, FALSE, _T("threadEvent"));   // init to FALSE    if(!waitThreadEndEvent)    {perror("CreateEvent waitThreadEndEvent error");CloseHandle(thread);CloseHandle(waitDataEvent);return -1;    }    Sleep(2000);    // set data and let son thread go on...    data = 1;    SetEvent(waitDataEvent);        // wait the son thread end    WaitForSingleObject(waitThreadEndEvent, INFINITE);    Sleep(1000);    CloseHandle(thread);    CloseHandle(waitDataEvent);    CloseHandle(waitThreadEndEvent);    printf("main thread will end...\n");    return 0;  }  

As you can see clearly above, the subthread first waits for the main thread to modify the data value, then outputs it, And then prepares to end, notifying the main thread that it is about to end;

After the main thread modifies the data, it sends the data modification event and waits for the subthread to send the end event and then ends.

In this way, the main thread and sub-thread can be executed according to the predefined steps without errors in the execution sequence. The running result is as follows:

Q4: in the preceding example, what is the event created by createevent in the kernel?

A: To better understand what it is, first check the kernel source code (wrkv1.2, NT kernel source code, Windows XP, and Windows Server 2003 kernel source code ).

NTSTATUSNtCreateEvent (    __out PHANDLE EventHandle,    __in ACCESS_MASK DesiredAccess,    __in_opt POBJECT_ATTRIBUTES ObjectAttributes,    __in EVENT_TYPE EventType,    __in BOOLEAN InitialState    )

First, we can see that the above declaration is the createevent kernel implementation function declaration. The specific implementation is as follows:

NTSTATUSNtCreateEvent (    __out PHANDLE EventHandle,    __in ACCESS_MASK DesiredAccess,    __in_opt POBJECT_ATTRIBUTES ObjectAttributes,    __in EVENT_TYPE EventType,    __in BOOLEAN InitialState    )/*++Routine Description:    This function creates an event object, sets it initial state to the    specified value, and opens a handle to the object with the specified    desired access.Arguments:    EventHandle - Supplies a pointer to a variable that will receive the        event object handle.    DesiredAccess - Supplies the desired types of access for the event object.    ObjectAttributes - Supplies a pointer to an object attributes structure.    EventType - Supplies the type of the event (autoclearing or notification).    InitialState - Supplies the initial state of the event object.Return Value:    NTSTATUS.--*/{    PVOID Event;    HANDLE Handle;    KPROCESSOR_MODE PreviousMode;    NTSTATUS Status;    //    // Get previous processor mode and probe output handle address if    // necessary.    //    PreviousMode = KeGetPreviousMode();    if (PreviousMode != KernelMode) {        try {            ProbeForWriteHandle(EventHandle);        } except(EXCEPTION_EXECUTE_HANDLER) {            return GetExceptionCode();        }    }    //    // Check argument validity.    //    if ((EventType != NotificationEvent) && (EventType != SynchronizationEvent)) {        return STATUS_INVALID_PARAMETER;    }    //    // Allocate event object.    //    Status = ObCreateObject(PreviousMode,                            ExEventObjectType,                            ObjectAttributes,                            PreviousMode,                            NULL,                            sizeof(KEVENT),                            0,                            0,                            &Event);    //    // If the event object was successfully allocated, then initialize the    // event object and attempt to insert the event object in the current    // process' handle table.    //    if (NT_SUCCESS(Status)) {        KeInitializeEvent((PKEVENT)Event, EventType, InitialState);        Status = ObInsertObject(Event,                                NULL,                                DesiredAccess,                                0,                                NULL,                                &Handle);        //        // If the event object was successfully inserted in the current        // process' handle table, then attempt to write the event object        // handle value. If the write attempt fails, then do not report        // an error. When the caller attempts to access the handle value,        // an access violation will occur.        //        if (NT_SUCCESS(Status)) {            if (PreviousMode != KernelMode) {                try {                    *EventHandle = Handle;                } except(EXCEPTION_EXECUTE_HANDLER) {                    NOTHING;                }            } else {                *EventHandle = Handle;            }        }    }    //    // Return service status.    //    return Status;}

We can find that it mainly calls three functions: obcreateobject, keinitializeevent and obinsertobject.

The first is to create an object, the second is the kernel initialization event object, and the third is to insert this object into the process handle table.

The implementation of the keinitializeevent function is as follows:

VOIDKeInitializeEvent (    __out PRKEVENT Event,    __in EVENT_TYPE Type,    __in BOOLEAN State    )/*++Routine Description:    This function initializes a kernel event object. The initial signal    state of the object is set to the specified value.Arguments:    Event - Supplies a pointer to a dispatcher object of type event.    Type - Supplies the type of event; NotificationEvent or        SynchronizationEvent.    State - Supplies the initial signal state of the event object.Return Value:    None.--*/{    //    // Initialize standard dispatcher object header, set initial signal    // state of event object, and set the type of event object.    //    Event->Header.Type = (UCHAR)Type;    Event->Header.Size = sizeof(KEVENT) / sizeof(LONG);    Event->Header.SignalState = State;    InitializeListHead(&Event->Header.WaitListHead);    return;}

For the event structure:

typedef struct _KEVENT {    DISPATCHER_HEADER Header;} KEVENT, *PKEVENT, *PRKEVENT;

typedef struct _DISPATCHER_HEADER {    union {        struct {            UCHAR Type;     // obj type            union {                UCHAR Absolute;                UCHAR NpxIrql;            };            union {                UCHAR Size;  // obj size，unit as sizeof(DWORD)                UCHAR Hand;            };            union {                UCHAR Inserted;                BOOLEAN DebugActive;            };        };        volatile LONG Lock;    };    LONG SignalState;             LIST_ENTRY WaitListHead;  // the objs that wait for this obj  } DISPATCHER_HEADER;

From the above, we can see that for the event, the kernel actually saves a data object and records its basic status and waiting list.

But how does the kernel scheduling thread decide which thread should be suspended and which one can be ready or run to ensure that the thread synchronization and mutex are correct?

As shown in the following ntsetevent code, it calls the kiwaittestsynchronizationobject function internally:

FORCEINLINEVOIDKiWaitTestSynchronizationObject (    IN PVOID Object,    IN KPRIORITY Increment    )/*++Routine Description:    This function tests if a wait can be satisfied when a synchronization    dispatcher object attains a state of signaled. Synchronization objects    include synchronization events and synchronization timers.Arguments:    Object - Supplies a pointer to an event object.    Increment - Supplies the priority increment.Return Value:    None.--*/{    PKEVENT Event = Object;    PLIST_ENTRY ListHead;    PRKTHREAD Thread;    PRKWAIT_BLOCK WaitBlock;    PLIST_ENTRY WaitEntry;    //    // As long as the signal state of the specified event is signaled and    // there are waiters in the event wait list, then try to satisfy a wait.    //    ListHead = &Event->Header.WaitListHead;    ASSERT(IsListEmpty(&Event->Header.WaitListHead) == FALSE);    WaitEntry = ListHead->Flink;    do {        //        // Get the address of the wait block and the thread doing the wait.        //        WaitBlock = CONTAINING_RECORD(WaitEntry, KWAIT_BLOCK, WaitListEntry);        Thread = WaitBlock->Thread;        //        // If the wait type is wait any, then satisfy the wait, unwait the        // thread with the wait key status, and exit loop. Otherwise, unwait        // the thread with a kernel APC status and continue the loop.        //        if (WaitBlock->WaitType == WaitAny) {            Event->Header.SignalState = 0;            KiUnwaitThread(Thread, (NTSTATUS)WaitBlock->WaitKey, Increment);            break;        }        KiUnwaitThread(Thread, STATUS_KERNEL_APC, Increment);        WaitEntry = ListHead->Flink;    } while (WaitEntry != ListHead);    return;}

It sends an activation signal to the list of wait threads for this event. Of course, whether the thread can continue to execute depends on the status of the specific settings. Its core code is the kiunwaitthread function:

VOIDFASTCALLKiUnwaitThread (    IN PRKTHREAD Thread,    IN LONG_PTR WaitStatus,    IN KPRIORITY Increment    )/*++Routine Description:    This function unwaits a thread, sets the thread's wait completion status,    calculates the thread's new priority, and either readies the thread for    execution or adds the thread to a list of threads to be readied later.Arguments:    Thread - Supplies a pointer to a dispatcher object of type thread.    WaitStatus - Supplies the wait completion status.    Increment - Supplies the priority increment that is to be applied to        the thread's priority.Return Value:    None.--*/{    //    // Unlink thread from the appropriate wait queues and set the wait    // completion status.    //    KiUnlinkThread(Thread, WaitStatus);    //    // Set unwait priority adjustment parameters.    //    ASSERT(Increment >= 0);    Thread->AdjustIncrement = (SCHAR)Increment;    Thread->AdjustReason = (UCHAR)AdjustUnwait;    //    // Ready the thread for execution.    //    KiReadyThread(Thread);    return;}

I don't think I need to explain the meaning of kireadythread. Of course, for other synchronization mutex objects, such as mutex, the process of implementing mutex is similar. We will not list them here.

Q5: Can I use the pthread function library on windows?

A: Microsoft official does not seem to release the pthread library, but there is open source code, details please: http://sources.redhat.com/pthreads-win32/

Author: Chen Xi

Date: 13:05:34

Environment: [win7 32-bit OS intel I3 support 64-bit command vs2010; wrk-v1.2; source insight]

Reprinted please indicate the source