A journey to the kernel uncovers the mysteries of the system

Kernel journey unveil the mysteries of the system (I)

 

Good materials can bring you more technical ideas. "in-depth analysis of Windows operating systems" is a book that explains the relationship between OS architecture and hardware based on Linux systems, if you like Linux, you can consider reading this book. In the Unix operating system design, the Unix System explained in the Unix V version is somewhat unsatisfactory; currently, FreeBSD does not have much information. Therefore, if you want to analyze a Unix-like system from the perspective of the entire system, this book will be a good reference, it will also help you solve many drawbacks of code writing.

If Intel has an assembly language, "80x86 assembly language and computer architecture" tells you how to compile the assembly program, and under what circumstances each command occupies several bytes, how does a program produce compiled and connected opcode (machine code)? The author will tell you carefully how a mov command occupies a clock cycle, mov and xchg are often used for data passing values. They also tell you the opcode bytes occupied by mov commands.

(1) concepts to be understood

Before analyzing the mechanism of Windows, you need to pay attention to some basic concepts. With these concepts, the use of tools will be conducive to the Analysis of the Mechanism of Windows.

API: Application Program Interface, which contains thousands of API functions.

Plat software Development Kit: the platform Development suite. When developing VC programs, the best habit is to install the latest SDK package, which contains a lot more content than MSDN, Which is outdated, there are also many examples of code that tell you how to write the specifications of Windows programs.

DDK: device driver development kit.

Process concept:

The process is just a container, because the container is used to store the main thread. in Windows, no matter whether the main or WinMain function is created, it is not a process, but a main thread, the main thread is the main body in a process. Understanding the concept of thread is of great help for developing Windows programs. After a process is created, the main Thread is created based on various environmental conditions. The main Thread is divided into two parts: User Mode Thread (User Mode Thread) and Kernel Mode Thread (Kernel Mode Thread ). For the main thread, there are not many bad things we can do (from the thread perspective, discard various environment conditions of the process). When we call the CreateThread function to create a thread, or when we create a thread pool, we can do a lot of bad things, because MS provides some functions for managing threads, at the same time, the kernel data structure of a group of Management threads is also published. The data structure is CONTEXT. The CONTEXT data structure depends on the CPU type; the description of this structure is described in Windows core programming. You can use the GetThreadContext function to obtain the CPU register status of the current thread by initializing a CONTEXT structure. You can use the SetThreadContext function to set the CPU register status of the current thread. However, the trouble is that when we use the GetThreadContext and SetThreadContext functions, the thread must be inactive because a user-state visible active thread is executed asynchronously, the value you obtain when using the above function is different from the actual value. Some time ago, when I was engaged in thread pools, I asked my colleagues about Asynchronization. My colleagues gave me a good answer. After writing test code for a while, I got a deeper understanding of this place.

Thread concept:

It is necessary to separate the following content, which is based on some opinions of the main thread and the particularity of the CONTEXT structure. The kernel also has a CONTEXT structure, which is identical to the structure value of the user State. However, this does not mean that the kernel state references the structure of the user State, the underlying description of the structure mentioned above only indicates that the content stored in the structure is relatively low. If you have carefully read the description of this structure in "Windows core programming", the author may not be right in some places. Of course, everyone understands different technologies, if the explanation here is incorrect, you still need some advice.

The CONTEXT structure is mentioned, and the Get (Set) ThreadContext function can be used to modify the structure, can we pause and resume the running of threads in user mode by performing operations on this structure? The answer is yes, but special processing is required. Because the thread is executed asynchronously, its stack environment is not as stable as the process. In other words, as long as the main thread is running, the stack environment will be stable, but the thread (the thread created using CreateThread) is different from it. Once the thread returns an end, or the thread end function is called, the stack is automatically cleared by the kernel when the thread is created. However, to use the CONTEXT structure to manage threads, you need to be familiar with the thread running mechanism; otherwise, it is difficult to manage threads.

To implement control over multiple threads in the user mode (the control here does not refer to thread synchronization, and thread synchronization is easier to implement. Many documents have detailed explanations ), the best and only way (user mode) is ResumeThread and SuspendThread, because Windows is a multi-task, therefore, it is difficult for you to control the threads in kernel mode during thread scheduling. Therefore, in user mode, you can only use the above two functions to restore and pause threads. As mentioned above, the thread has a distinction between user State and kernel state. When the SuspendThread function is used to suspend a thread, the user State thread is paused. That is, the user State thread is paused, it is still active in the kernel state, but this is enough. At least we can ensure that our programs in the user State can run according to our ideas. What is the kernel state, that's the OS thing.

Threads include:

Registers of the CPU status of a group of codes;

Two stacks: User-mode stack and kernel-mode stack;

TLS (Thread Local Storage), private Storage area;

Unique identifier of a thread, thread ID;

The running environment of the thread;

Virtual Memory:

This concept is equivalent to swap (swap partition) in Unix. To address this problem, we should first understand it in a general statement. The execution of a program in the OS does not use the actual physical address. The execution of the program is executed by combining the OS with the CPU. There is a paging mechanism in the OS, it is used to map the address of a 32-bit program to a physical address. If the program runs in large space, swap (* nix) and PageFile can be used. sys (Win) for Virtual Memory ing, all of which are implemented through the page splitting mechanism.

In Windows, Memory Manager (Memory Manager) is used to implement Memory ing, which also plays a protection role. This protection and ing mechanism prevents a process from breaking into another process or modifying the kernel data.

To get a general understanding, you can consider the virtual Memory mechanism as the Memory address visible during user-state debugging, but it is not the real physical Memory address, and then perform Memory ing through Memory Manager.

/* ++
Virtual Memory Layout on x86 is:
+ ------------------------------------ +
00000000 |
|
|
| User Mode Addresses |
|
| All pages within this range |
| Are potentially accessible while |
| The CPU is in USER mode. |
|
|
+ ------------------------------------ +
7ffff000 | 64 k No Access Area |
+ ------------------------------------ +
80000000 |
| NTLDR loads the kernel, HAL and |
| Boot drivers here. The kernel |
| Then relocates the drivers to the |
| System PTE area. |
|
| Kernel mode access only. |
|
| When possible, the PFN database & |
| Initial non paged pool is built |