Next, switch the Windows NT/2000 environment

Next, switch the Windows NT/2000 environment

Author: Unknown Source: Unknown

A thread is the most basic unit for Windows NT/2000 environment switching. In the <Analysis of Windows NT/2000 environment switching> (nsfocus magazine 12) article, I only discussed the process of switching to S3, this article will try to describe the content that does not involve thread scheduling. Before that, let's take a look at the following code: //----------------------------------------------- // // Enumthreads-information from kpeb and kteb // Only test on Windows 2000 Server Chinese edition // Build 2195 (free )! Programmed by webcrazy // (Tsu00@263.net) on 5-23-2000! // //----------------------------------------------- # Define kteblistoffsetkpeb 0x50 # Define pidoffset 0x9c # Define kpeblistoffset 0xa0 # Define processnameoffset 0x1fc # Define stacktopoffset 0x18 # Define stackbtmoffset 0x1c # Define userteboffset 0x20 # Define stackptroffset 0x28 # Define kteblistoffset 0x1a4 # Define ktebpidoffset 0x1e0 # Define tidoffset 0x1e4 Void displaythreadfromkpeb (void * kpeb) { Char processname [16]; Ulong PID; Ulong tid; Ulong stackbtm, stacktop, stackptr, userteb; Plist_entry kteblisthead, kteblistptr; Kteblisthead = kteblistptr = (plist_entry) (INT) kpeb + kteblistoffsetkpeb ); Do { Void * kteb; Kteb = (void *) (* (ulong *) kteblistptr)-kteblistoffset ); Tid = * (ulong *) (char *) kteb) + tidoffset ); Stackbtm = * (ulong *) (char *) kteb) + stackbtmoffset ); Stacktop = * (ulong *) (char *) kteb) + stacktopoffset ); Stackptr = * (ulong *) (char *) kteb) + stackptroffset ); Userteb = * (ulong *) (char *) kteb) + userteboffset ); Memset (processname, 0, sizeof (processname )); Memcpy (processname, (char *) kpeb) + processnameoffset, 16 ); PID = * (ulong *) (char *) kpeb) + pidoffset ); // Or pid = * (ulong *) (char *) kteb) + ktebpidoffset ); Dbuplint ("% 04x % 08x % 08x % 08x % 08x % 08x % s (% x)/n ", TID, kteb, stackbtm, stacktop, stackptr, userteb, processname, pid ); Kteblistptr = kteblistptr-> flink; } While (kteblistptr-> flink! = Kteblisthead ); } Void enumthreads () { Plist_entry kpeblisthead, kpeblistptr; If (ushort) ntbuildnumber )! = 2195 ){ Dbuplint ("only test on Windows 2000 Server build 2195! /N "); Return; } Dbuplint ("/n TID kteb ADDR stackbtm stacktop stackptr user Teb Process Name (PID )"); Dbuplint ("/n --------------------------------------------------- -----/N "); Kpeblisthead = kpeblistptr = (plist_entry) (char *) psinitialsystemprocess) + kpeblistoffset ); While (kpeblistptr-> flink! = Kpeblisthead ){ Void * kpeb; Kpeb = (void *) (char *) kpeblistptr)-kpeblistoffset ); Displaythreadfromkpeb (kpeb ); Dbuplint ("/N "); Kpeblistptr = kpeblistptr-> flink; } } The output result of the enumthreads function listed in this code in Windows 2000 Server build 2195 is as follows: TID kteb ADDR stackbtm stacktop stackptr user Teb process name (PID) ------------------------------------------------------------- 0004 fe4e19e0 f9019000 f901c000 f901b9c4 00000000 system (8) 000c fe4e0c80 f9021000 f9024000 f9023d34 00000000 system (8) 0010 fe4e0a00 f9025000 f9028000 f9027d34 00000000 system (8) 0014 fe4e0780 f9029000 f902c000 f902bd34 00000000 system (8) 0018 fe4e0500 f902d000 f9030000 f902fd34 00000000 system (8) 001c fe4e0280 f9031000 f9034000 f9033d3400000000 system (8) . . (Omitted) . According to the running results, enumthreads mainly lists all the threads in the system. The output format is the same as that of the SoftICE thread command. Some uninitialized ented kpeb/kteb data items are used in the Code: 1. thread linked list of processes This is an item in the list_entry structure (which occupies two 32-bit pointers (8 bytes) and is located at 50 h after kpeb. The code above is represented by kteblistoffsetkpeb. 2. offset of the thread linked list relative to kteb Located at 1a4h after kteb (defined by kteblistoffset ). For output results such as stackbtm, stacktop, and stackptr, see < > In kteb, see the definition before code. After understanding the underlying code of the enumthreads program segment and the implementation of enumprocesses that I provided last time, I also understood how Windows NT/2000 organizes, manages processes, and threads, this is crucial to understanding thread scheduling. Even so, we need to discuss Thread Scheduling and look at some important data (I will not detail how to obtain the specific location of the data, if you want to know, we recommend that you check again <Windows NT/2000 environment switch> ): 1. Process status // kpeb + 65 h uchar Typical process statuses include running, ready, and idle. It should be noted that there is only one running process in a single processor, and it is not restricted by the value in kpeb. The system obtains it by calling the iogetcurrentprocess kernel routine. I have introduced iogetcurrentprocess in more detail in <Windows NT/2000 Internal data structure>. When the status value in kpeb is 0, the Process status is ready; when the status is 1, the Process status is idle; when the status is 2, the Process status is transition, and so on. As mentioned above, the thread is the basic unit for switching the Windows NT/2000 environment. In fact, the system does not execute the process. The process status and the process priority to be mentioned below are very abstract concepts, this is only a restriction on the thread range during system calling. Microsoft proposed these concepts to hide the thread scheduling behavior inside the system. However, if there is a thread status, it does not mean that you just need to add a value. Once I changed the status of the system process from ready to transition, I could only watch the program code on the screen and could not store the disk. At this time, the system has become lazy and no longer responds to my call. 2. thread status // kteb + 2dh uchar In kteb, a Member State mainly indicates the current thread state, which is located at kteb + 2d (single-byte ). It mainly has the following values (values are taken from SoftICE output results ): 0-initialized (indicating that the thread state is initialized when the state value is 0) 1-ready 2-running 3-standby 4-terminated 5-waiting 6-Transition < > Is described as follows: To quote: -------- The thread States are as follows: Ready When looking for a thread to execute, the dispatcher considers only the pool of threads in Ready State. These threads are simply waiting to execute. Standby A thread in the STANDBY state has been selected to run next on a participant processor. When The correct conditions exist, the dispatcher performs a context switch to this thread. Only one Thread can be in the STANDBY state for each processor on the system. Running Once the dispatcher performs a context switch to a thread, the thread enters the running State and executes. The thread's execution continues until the kernel preempts it to run a higher priority thread, its quantum ends, it terminates, or it voluntarily enters the wait state. Waiting A thread can enter the wait state in several ways: a thread can voluntarily wait on an object to synchronize its execution, the operating system (the I/O system, for example) can wait on the thread's behalf, or an environment subsystem can direct the thread to suspend itself. when the thread's wait ends, depending on the priority, the thread either begins running immediately or is Moved back to the ready state. Transition A thread enters the transition state if it is ready for execution but its kernel stack is Paged out of memory. For example, the thread's kernel stack might be paged out of memory. Once Its Kernel stack is brought back into memory, the thread enters the ready state. Terminated When a thread finishes executing, it enters the terminated state. Once terminated, a thread Object might or might not be deleted. (The object manager sets policy regarding when to delete The object.) If the Executive has a pointer to the thread object, it can reinitialize the thread Object and use it again. Initialized Used internally while a thread is being created. -------- The following mainly analyzes the waiting status: In Windows NT/2000, you can call kewaitforsingleobject and kewaitformultipleobjects to automatically discard the total execution time (quantum ). The thread currently executed by the system is specified by the currentthread member in the processor control block (prcb, which is different from processor control region) of the system. Remember how I got the current thread (take the DWORD Value in FS: 124h )? It actually points to the currentthread member. The definition of prcb is in ntddk. h. The system obtains the kprcb pointer using the following function: _ Kegetcurrentprcb 0008: 80465310 mov eax, [ffdff020] using the kpcr (Processor Control Region) member prcb 0008: 80465315 RET The current thread status of the system is running. The status of other threads is usually several (the maximum is thread_wait_objects + 1, otherwise bsod will appear, but Microsoft also defines maximum_wait_objects, which depends on the parameters you pass to the system) the kwait_block structure indicates that these values and the kwait_reason that will be discussed below indicate the thread waiting reason can also be obtained from ntddk. h. The data in the kwait_block structure of the thread is located in kteb + 6ch. In the previous two cases of context switch, you can either use synchronization objects such as event and semphore, or use the timer kernel object to form a thread waiting column, to indicate the current state of the thread. Because kwait_block, kwait_reason, event, timer, and so on are several rare ented members in Windows NT/2000, you can read the thread wait queue by yourself after knowing the specific location of kwait_block. However, SoftICE has shown you all these internal structures. From the above analysis, the thread status involves a lot of content. I copied a part of the analysis as follows: : U _ kereadstatethread _ Kereadstatethread 0008: 8042f029 mov eax, [esp + 04] 0008: 8042f02d mov Al, [eax + 04] 0008: 8042f030 RET 0004 : BPX _ kereadstatethread if (TID = _ tid) : BL // exit the debugger after this command 00) BPX _ kereadstatethread if (TID = 0x3bc) Break due to BPX _ kereadstatethread if (TID = 0x3bc) (ET = 22.28 seconds) // Analyze the first parameter of _ kereadstatethread (which is also a unique parameter) : What DWORD (@ (esp + 04) // you should understand which kernel object determines the thread status of each thread at each time. The value ff6811e0 is (a) kernel timer object (handle = 0230) for Explorer (398) | | _ Timer Kernel Object | _ HANDLE of this object in the explorer process : Timer DWORD (@ (esp + 04 )) Timer object at ff6811e0 Dispatcher type: 08 Dispatcher size: 000a Signal state: signaled . . (Omitted) . // The timer command in SoftICE only reads the timer Object Data // You can directly read the signalstate member in dispatcher_header (Common dispatcher object header) (see ntddk. h) // The following command # Byte (@ (esp + 04) + 4 )) 00000001 0000000001 "" // 1 indicates signaled. Have you tried changing it from 1 to 0? (Changed from signaled to not signaled) The current state of the thread analyzed above depends on the status of the timer object (Object Pointer: 0xff6811e0) (Jeffrey Richter says signaled indicates when time comes due ). I have already analyzed it from the simplest aspect. The current state of multithreading often depends not only on an object, but also on thread wait list in SoftICE. After talking about so many objects that let the thread wait, let's talk about kwait_reason. In kteb, there is a member dedicated to thread wait reason. It is located at the position of kteb offset 57h, occupying a char space, strictly speaking, this is the reason why the thread is actually in the wait state. The above discussion only explains what kernel object causes this wait reason. DDK documentation defines wait reason in this way Waitreason Specifies the reason for the wait. A driver shoshould set this value to executive, unless it is doing work on behalf of a user and is running in the context of a user thread, in which case it shoshould set this value to userrequest. The user thread is a single-character member of kteb located at the 55 h offset. It is represented by 0 in kernel mode and 1 in user mode. The preceding two values of wait reason are the most common in driver programming (not the most common in system kernel-state code): Executive and userrequest. For other values, see ntddk. h. 3. Process Priority (kpeb + 62 h), thread basic priority (basepriority, kteb + 68 h), thread dynamic priority (Dyn priority, kteb + 33 H) Each of these values occupies one byte. Among them, thread Dyn priority is displayed as current priority in spy ++, while some small tools in Microsoft's windbg and Windows 2000 Server resource Kitts, such as pstat.exe, are directly expressed as priority, however, in SoftICE, it is displayed as Dyn priority. Because priority is not easy to express so many priorities. Since all the content in my article is based on SoftICE analysis, I will keep the name in SoftICE in this article. In fact, Microsoft also provides prioritydecrement in the kteb structure to enable the system to dynamically change the current priority at any time. This is one of the reasons why I prefer Dyn priority. For more information about these priorities, see < >, Which describes the functions of these core values. 4. affinity) As I have not yet conducted conditional multi-processor testing, I am not sure to mention it here. If you have any conditions, I hope you can talk about it. 5. Total thread time (thread quantum) Single-byte, located at kteb + 6B. Specifies the total time (quantum) that the CPU can schedule threads ). In the processor control block, the system has three _ kthread (kteb) structure members: currentthread, nextthread, and idlethread, it represents the threads currently being executed by the system processor, the threads to be called, and the Idle threads of the system. The idle thread is generally just a simple call to kiidleloop until new system interruptions come, to call other threads. In Windows NT/2000, call kiquantumend to determine whether the current thread has used its total time. If the current thread has executed quantum, the nextthread is returned if the nextthread in kprcb is not empty, as the next thread of the system call. The system calls kifindreadythread to find the next ready thread. _ Kiquantumend 0008: 804315b9 push EBP 0008: 804315ba mov EBP, ESP 0008: 804315bc push ECx 0008: 804315bd push EBX 0008: 804315be push ESI 0008: 804315bf push EDI 0008: 804315c0 mov eax, DS: [ffdff020]; kprcb-> eax 0008: 804315c6 mov EDI, eax; kprcb-> EDI 0008: 804315c8 mov eax, FS: [00000124]; current thread's kteb-> eax 0008: 804315ce mov ESI, eax; current thread's kteb-> ESI 0008: 804315d0 call [_ imp _ keraiseirqltodpclevel]; Raise IRQL to dispatch_level, learned DDK friends should be familiar 0008: 804315d6 xor ebx, EBX 0008: 804315d8 mov [EBP-01], Al; save old IRQL 0008: 804315db CMP [ESI + 6B], BL; judge the quantum of the current thread 0008: 804315de JG 804315f2; obtain nextthread when quantum is less than or equal to 0 0008: 804315e0 mov eax, [ESI + 44] 0008: 804315e3 CMP [eax + 69], BL 0008: 804315e6 JZ 80431608 0008: 804315e8 CMP byte PTR [ESI + 33], 10 0008: 804315ec JL 80431608 0008: 804315ee mov byte PTR [ESI + 6B], 7f 0008: 804315f2 mov ESI, [EDI + 08]; kprcb's nextthread-> ESI 0008: 804315f5 cmp esi, EBX; whether kprcb's nextthread is empty 0008: 804315f7 jnz 80431601 0008: 804315f9 mov Cl, [EBP-01] 0008: 804315fc call @ kiunlockdispatcherdatabase 0008: 80431601 mov eax, ESI; returns nextthread 0008: 80431603 pop EDI 0008: 80431604 pop ESI 0008: 80431605 pop EBX 0008: 80431606 leave 0008: 80431607 RET 0008: 80431608 movsx edX, byte PTR [ESI + 33] . . (The code is very long and involves scheduling algorithms. You can only take a look at it yourself) . 6. kpeb of the process to which the thread belongs (kteb + 22ch) It is easier to exchange data between kteb and kpeb. In fact, the above part of internal data can also be found in Linux to implement the corresponding functions, such as the thread quantum in Windows NT/2000 corresponds to the counter member in Linux task_struct. In <Analysis of Windows NT/2000 environment switch>, I pointed out that Windows NT/2000 actually has only two task switches, I also provided part of the code for time interruption last time and the swapcontext code (mainly for switching from the third node to the third node. Every process in Windows has private memory space, private kernel objects (handle table), and so on. These are implemented based on environment switching, it is also the basis of the robustness and reliability of an operating system. There are many articles in http://www.research.microsoft.com/that have been verified. For the implementation of this part, you can also take a look at the following routines (I will not list the code for space limitations ): ⊙ Kifindreadythread ⊙ Kireadythread ⊙ Kiswapthread ⊙ Swapcontext Swapcontext and kiswapthread are the real code for system switching. Do you often see this function in stack trace? It should be easy to understand here. (For details about stack trace, refer to the anatomy of a stack trace section in DDK documentation or the stack and thread commands in SoftICE command reference) In other cases, when the thread automatically gives up execution, it is relatively easy to track kewaitforsingleobject and kesetevent. Let's take the event object as an example. Because we know the event structure, on your lab machine, you can change the signalstate member in dispatcher_header to change the object state. check whether it is clear or signalled (I have provided how to implement it with SoftICE ). Even after you understand the code I provided at the beginning (in fact, simply read the data in a two-way linked list, if it is in Linux, I believe you have no patience to see it ), I also understand the thread wait list defined by kwait_block. After giving you a kernel debugger, I believe that the thread you want to block is the same as the thread. After understanding the thread priority, which thread you want to take up more CPU time can be used. However, I am not sure if you want to restore the robustness and reliability at this time. By the way, the scheduling code in Windows NT/2000 runs on dispatch_level IRQL, which has prevented the interruption of common code running on passive_level. Jeffrey Richter < > It was pointed out that: ⊙ Microsoft doesn't fully document the behavior of the schedior. ⊙ Microsoft doesn't let applications take full advantage of the schedures's features. Microsoft tells you that the scheduler's algorithm is subject to change so that you Can code defensively. From these points, Microsoft has not fixed the scheduling behavior. In fact, Windows NT 4.0 and Windows 2000 have different scheduling algorithms, all the Code mentioned in this article is taken from or tested only in Windows 2000 Server build 2195. My friends who have learned about DDK should know that pssetcreateprocesspolicyroutine and pssetcreatethreadpolicyroutine allow the user to register the fully receivented routine that calls the callback function when the user registers the system to establish and delete the process or thread, we know that they are the two most important functions of Mark russinovich's ntpmon implementation. In fact, in Windows 2000, Microsoft also provides functions related to environment switching that implement similar functions, namely, kesetswapcontextpolicyroutine and kesetthreadselectpolicyroutine. However, these two functions are uninitialized ented, it can only run on a specific version of Windows 2000 (please refer to msdn magazine ). All the code in Linux is public, but it is still difficult to really understand the scheduling code. Besides, it is more complicated than Linux itself (I have not browsed the Linux 2.4.x source code, I can say this by myself) and I can only search for Windows NT/2000 in a lot of assembly code. The analysis is really difficult. So I hope you can tell me (tsu00@263.net) where the error or omission in this article is correct. Thank you! References: 1. Jeffrey Richter < > 2. Peter viscarola and Anthony Mason < > 3. numbench < > 4. Windows 2000 DDK documentation 5. David Solomon and Mark russinovich < >