Windows and Linux memory management

Windows Memory Management is very rigorous, and memory must be allocated first. Of course, this is true for every operating system. However, Windows is rigorous in the allocation process, which is divided into two phases: retention and submission, the meaning of retention is that a space is reserved in the virtual address space of the process and cannot be used for other purposes. The concept of retention is for the virtual address space, the submission means to map the virtual memory block of the reserved virtual address space to the physical memory. Here, Windows extends the meaning of the physical memory, including the physical memory and disk page files represented by the memory, as well as any backup storage that can be swapped in and out with the real physical memory, the concept of commit is actually a ing, in order to make the virtual memory available, a ing is made to the actual physical storage, that is, changing false to true.

There are several things involved in the two phases of Windows retention and submission. One is when the page table is established, we can imagine a reasonable way, that is, when the memory block is retained, if you do not operate the page table, you only need to insert the virtual memory segment into a data structure that is easy to find, insert, and delete. In the commit phase, you can operate the page table, of course, the memory may not have reached the real physical memory at this time. It is likely that only a slot is allocated to the page file. In this case, the related bits of the page table can be used to describe the slot location and other information, as long as the existence bits of the page table are 0, which is the same as that of Linux. In fact, the submitted memory is mapped to the physical memory only when the meaning of the physical memory is extended. The virtual memory is mapped to the original physical memory only when the memory is accessed, this is absolutely lazy. In fact, we can see that the windows method is not lazy enough. Linux does not have the concept of retained and committed. When an execution thread calls different levels of functions, such as MMAP, malloc, or BRK, in fact, it is equal to retaining the memory area, and it will be directly mapped to the physical memory only when the memory is accessed, there is no connection between the virtual memory and the physical fact. It is true that the false memory is revealed only when the fact cannot be concealed. Linux memory management is absolutely lazy, the access memory can be viewed as memory commit. The reason why Windows uses this method to manage memory is actually to manage all the memory in a more unified way. As long as the memory is submitted, the memory manager must track this memory, whether it is in physical storage or on a disk page file. Linux seems more nonstandard. Linux uses a page table to act as a dual-sided role, which can both search for physical memory and find the location of swap partition memory, in addition, Linux does not have a mechanism to uniformly manage physical storage and swap partition space, by virtue of powerful file system functions and efficient memory management and file management data structures, you can easily achieve efficient memory swap-in and swap-out, removing the coupling between physical storage and swap partitions. On the contrary, in Windows, the uniform and gorgeous appearance is distorted and complicated. For example, if you want to modify the format of the page file, the logic of the memory commit must be involved, in Linux, you only need to change the file_operations. The unified and gorgeous appearance is not just a pleasant look, but also a price, for example, the task of balancing the amount of memory between processes is handed over to the user. In fact, as long as the user can maintain a high degree of flexibility in the memory allocation and management of the process, the memory balance between processes is clearly the responsibility of the operating system, because as long as the memory is submitted, or the physical storage or disk page files occupy a certain amount of space, and these spaces It is shared by all processes. If a process commits too much memory, other processes will be hungry. In this case, the operating system, as a coordinator, cannot actually help, at most, attackers can get rid of the problem. The allocation ratio of various processes in the physical environment depends entirely on the process itself, and the supervision of other processes and the coordination of kernel mechanisms are lost. This seems inferior to Linux, in Linux, the memory management module tries its best to ensure fair allocation between processes. Even if a process allocates a large amount of memory itself, as long as it does not access the memory, these memories won't even occupy swap partitions, let alone physical memory. Of course, if this greedy process accesses these memories, the results will be the same as windows, this process should be easily identified from the program's behavior, but in any case, it is much better than Windows's policy that allows to take up the pitfalls, although the memory is already very cheap, however, for applications with the same growth, memory is still scarce resources. Therefore, the lazy allocation method should be the most economical.

 

 
As a direct result of the above discussion, let's take a look at the stacks in the two systems. In Windows, stack allocation is static, that is, the size of the thread stack is determined in the PE file and generally cannot be dynamically changed during runtime. During stack management, windows uses a slightly complicated but thoughtful method. Windows tries its best to protect its stack from overflow. How can it be protected? In Windows core programming, there is a detailed description, that is, first determine the size of your stack, then, set the first and last pages of the memory block of this size to be retained, and the remaining pages follow the following principles: Assuming the stack grows down, in Windows, the next page being used is set as protection commit. Of course, the page being used must be submitted. The system will be notified whenever the protection commit page is accessed, note that the notification is not an error message, and there is no serious error, because the protection submission page can be accessed and it has already been submitted, but because it has the protection attribute, when the system knows this, it can set the protection submission attribute to the next page, and so on. The stack has a strict sequential access feature, that is to say, the first is that the high address is accessed, and the lower address is not used before the slightly lower address is not used. Of course, unless you use the assembly language completely out of the stack concept, the stack space page of the thread will be submitted one by one in the descending order, and the submitted page will be set to secure the commit until the end of the stack, Windo WS will detect that it will not set the last page as protection commit, but will cause a stack overflow exception. The result of this mechanism in Windows is to effectively protect the data behind the stack from being overwritten by the stack data, but this mechanism does not work every time, for example, if a large array that is sufficient to cause stack overflow is allocated to the stack, the array starts with a stack. If I directly access this first element, in addition, if the memory covered by this element has been committed, it will be finished. If you think the above instance will be discovered by the compiler, consider the following example:
Char s [1];
S-= 100000;
* S = 100;

 

It is very simple and lazy to see how Linux does it. Linux does not allocate a static size for the stack, but uses page-missing interruptions to dynamically increase the stack during runtime, of course, there is no overflow problem if there is no fixed size. As long as the virtual memory is sufficient, the demand for dynamic growth may be met, in Linux, is there any way to protect non-stack data from being damaged by stack data or vice versa? To be honest, no, it is mainly because the implementation of that mechanism is very complicated, and the maintenance of the introduced additional data structure will definitely affect the efficiency. Second, this is a matter of user space, if a programmer fails, he just needs to open it. The kernel does not need to be wiped by him. In fact, if the kernel is used to help him clean his ass, it will not say that the kernel is very clever, therefore, open-source Linux does not have such a complex mechanism and does not have any mechanism to use the kernel. In fact, if the programmer is not qualified, the program he writes will not be able to prevent. Can the machine compete with others? Obviously, no. Even a good operating system is incapable of loving poor programmers!

 

 

Finally, we will discuss the philosophical question of "how to allocate memory and where to allocate it to users or not, the key to this issue should be the usage of the allocated memory and the closeness between the function and the system mechanism. For example, I need a piece of memory to store some structures in my program, such as large database buffering, for example, a string. In this case, the more transparent the allocation, the better. Because the program does not need to communicate with the implementation mechanism, the program can focus more on solving the so-called business problems, however, if a piece of memory is required by a management mechanism, it is necessary to export more information to the user, because this requirement is often focused on the needs of the implementation itself, rather than the interface requirements, for example, the position of the thread stack, because the thread is a mechanism of the operating system, the purpose is to optimize program execution, it actually has nothing to do with the business logic, threads are more used by the management mechanism of program processes than by business processes. In this regard, Linux is much better than Windows. Looking at the parameters called by the clone system, the user must allocate stack space for the thread, Which is silently executed in windows, in fact, Windows tries its best to hide a lot of important information at the underlying layer to users. However, information like the location of the thread stack is still used by many user space management mechanisms, therefore, it is best to give it to the user, and the system does not care too much.