The idea of a modern operating system process address space

What is a heap, what is a stack, what is a data segment, what is a code snippet ... These are the legacy of history. Now programming really doesn't need to care about this! Don't be fooled and haunted by the contents of/proc/xx/{maps,smaps}. Manage your own memory allocations as well. If the program is not written by itself, then look for the person who wrote it.
This article starts with how a running file linking a dynamic library is loaded into the process address space, and discusses my views on the layout of the process's address space. I did not describe in a precise way how the elf or PE file was loaded, but merely expressed a process of thought, in order not to allow the person who saw this article to re-enter the abyss of endless Code analysis, so I just talked about the process and not the details.
1. The structure of the executable file can be run file contains the following content:
Self-descriptive narrative head: Describes the type of the document. Size, run platform, compatible information, ingress address and other meta-data.
Symbol table: The so-called symbol table is the identifier of a function or variable. The entities represented by these identities are finally mapped to an address in the address space.
Code: There is a running binary code of its own. The gist of the program can be run here.
Initialized data: Already initialized data defined in the program. In addition to saving the data symbol itself. You also want to save the data values.
Uninitialized data: Data declared in the program. But it's not initialized yet.

Just save the data symbol and don't save the value.
link library information: Dynamic Library information linked to this run file.
Whether it's a Windows PE file or a Unix elf file, it's nothing more than the above, and the complexity is that they have different formats and too much detail.

2. The executable file is loaded into a so-called executable file that maps the executable files in a file system to a virtual address space. The content contains the executable file itself. The entry address of the program can be found through the meta-data such as self-descriptive narrative head. So the end of this step is to jump to that address to run.

Attention. This step does not load the dynamic library, and the responsibility for loading the dynamic library is the user state. It's usually the interpreter thing.

Let's say you run the following sequence. You know when the dynamic library was loaded by WHO:
Readelf-a/bin/ls
Ldd/bin/ls
Strace-e Trace=open/bin/ls
I've read a lot of articles and I haven't touched on the details. For Linux, it is said that the dynamic library loading is completed in Load_elf_library. Even Maudeca said so. They did not notice the behavior of the C library at all. Is the man in the kernel so advanced? A lot of people think that just to get things done in the kernel, it is very profound. It's not really the case at all!

3. Dynamic library loading because of the 2nd step above, the process address space often has a dynamic library of information, including the library file name Word. Then the library path is indicated based on some environment variables. There is a hope that these libraries will be found under these paths and then loaded into the process address space one after the other.

At the end of the 2nd step, the program has returned to the user state to run, and in general this is a program called an interpreter, which parses the dynamic library information that needs to be loaded, and then loads the dynamic library into it. Some symbolic redirection issues are then resolved.
In fact. There is a lot of work to do before running the main function directly. Loading a dynamic library is just one of the most complex of them.

It is necessary to understand that only with a dynamic library, the work before entering main will be extremely complex, which is a certain price. Revenue is in the load_elf_library of the workload greatly reduced, in other words, you can run the burden of the file is reduced, false assumption for main before the behavior burden, use static link bar.

4. The address space is laid out here. The address space of a process is populated, and the next step is to jump to the main function we have defined. What does the spatial layout look like? If you ask, the average person will answer:
Code Snippet | data Segment | bss| Heap | spare | stack | kernel
Is that right? In a sense yes, but please look at steps 1th and 2nd carefully. I did not deliberately emphasize how the data that can be run files or dynamic library is plugged into these paragraphs, in fact, there is no need to use these paragraphs, these are historical legacy, the real address space layout should be:
Dynamic Library 1| Spare | Dynamic library 2| Dynamic Library 3| can run Program | spare | stack | kernel
Did you see it? No heaps, yes. No heaps.

In the case of existing Linux, in fact, in addition to preserving a brk to represent the heap, the old segments, such as code snippets, data segments, have been completely weakened, or even degenerate into routine or purely to evoke human memory. In my opinion. The heap should also be abandoned. There is only a stack, because it is extremely relevant to the processor. Therefore, reservations are required.

Today's memory management approach is virtual memory management in protected mode. For a process. The 32-bit virtual address has 4G of space, for 64-bit virtual addresses. is larger, is it necessary to divide the memory into sections according to attributes? In fact, in the era of real mode, there are special hardware registers to indicate the starting address of these sections, because the processor is so designed. Therefore, it is necessary to divide the memory into a series of internal contiguous sections, but after 32-bit protection mode, a lot of this division is unnecessary, the segment register in flat mode is still routine but no longer work, whether it is the code snippet or data segment, occupy the entire address space.

What's going on with the heap? Heap is the heap, is also an ancient concept, it is in the address space is a continuous space, generally located in the following stack. Unlike stacks that grow downward, the heap grows upward.

In prehistoric times, everyone was in a shared memory address space, and each process was within a contiguous spatial range. And there is a definite size. As with data segments, where the code segment is contiguous, the heap is a contiguous space, because the stack grows in the opposite direction. It ensures that the memory used by the program will go wrong before it crosses the border.

However, the virtual memory is now exclusive, how to manage the physical memory has been stripped out, so there is no need to use the original way.
Note that there is no problem with internal and external fragments, because in the age of protection mode, a much larger, exclusive address space can be used to efficiently avoid both fragments with a well-designed memory management algorithm. And in prehistoric times. Minimal shared memory is insufficient to accommodate the complex memory management program itself, and it is only possible to design a simple and consistent segmentation convention that divides the space into different extents according to the attributes of the data to avoid memory fragmentation.

5. Say a little bit about the heap essentially, in the exclusive process virtual address space, everything is determined by mmap to be the simple way. Whether it is the data or the code is now mmap the permissions of the section. For Linux, a section is a vm_area_struct. Its permissions reflect whether it maps code or data. So is there a need for a heap? True. It is contiguous in the virtual address space, but continuous is not the most important meaning of it, and many of its other things are embodied in how the user manages the memory. The problem boils down to how to provide a user with a user-specified size of virtual memory, assuming that Linux does not use the heap, but will all the spare not mapped to vm_area_struct space according to the partner system to organize, wouldn't it be better? The partner system itself has the ability to avoid fragmentation.
6. Run-time allocation memory in principle. I tend to use mmap for all memory allocations, even if the kernel is not implementing the virtual memory partner system at the moment.

In fact, the standard practice is when malloc applies less than 128K of memory. Allocated on the heap (now Linux is still using the heap), more than 128K is allocated using MMAP.

Suppose you are very proficient in the use of Linux memory. Would think that my attack on the heap was nonsense. Because I ignored the heap memory and the MMAP application and release mechanism are completely different. It is true in a sense, but unfortunately, I have a profound understanding of the difference. Assuming the man is mallopt, you will find M_mmap_threshold following a sentence:
Mmap ()-allocated memory can be immediately returned to the OS when it was freed, but this is not true for all mem‐
Ory allocated with SBRK (); However, memory allocated by MMAP () and later freed are neither joined nor reused, so
The overhead is greater. default:128*1024.
This is not the negation of mmap, this is not the opposite of my proposal? Yes!
But. The efficiency of BRK and how the user program uses memory has a great relationship. The memory on the heap is freed. If it is not the edge, it will not be released to the operating system. Instead, it continues to remain in the address space, which is managed by the malloc of the C library. It might implement a management algorithm for a similar partner system within malloc, allowing it to manage its own area of memory. But you have to rely on such a C library related to all implementations, such as the advantages for MMAP is not operating system level, but the C library level. For mmap, only you call free, then the underlying will call Munmap, at this time the existence of the address space is no longer usable, the frequent mmap/munmap operation will certainly bring a lot of overhead, and because each mmap address may be far away, Therefore, the advantages of locality are also lost.
I think. Discussing this problem and discussing the various optimizations of TCP in the kernel is a kind of problem. Finally, the result is, do not in the kernel to do such a comparison, completely to the user program or library itself, I believe that the user program can be solved perfectly.

The kernel only needs to give the memory to the user to be able, how to manage how to use all the user program own responsibility. The operating system kernel should strip away such things. For memory segments such as heaps. Its position in the kernel is indeed very embarrassing. It originally represents a memory management method, that is, a continuous memory with the program allocation and release is cut into small segments of different sizes, these segments constitute a heap data structure, indeed. In the prehistoric era of real patterns. Physical memory is really managed, but when the protection pattern clearly distinguishes between kernel space and user space. The heap management of memory has been ported to C library, in fact, they have been in the C library, just that later C library was arranged to the user space. As a result, a heap shadow is left in the kernel-state process address space. But the substance of the heap was long gone. The possibility of such a heap sandwiched between a program and a library exists, and you can know how much by traversing the/proc/xx/maps yourself, and the reason for this embarrassment lies in the inconsistency between the principles of running file layout and the principle of process space layout, The solution to this inconsistency is, of course, subject to the recommendations of the executable file, as it is first loaded.
There is another awkward place to be. Suppose that a process's heap is sandwiched between a running program image and a dynamic library image, such as the following:
Can run program |heap| Dynamic library 1| lots of spare |
There is no doubt that the heap expansion space is limited, followed by a large number of dynamic library 1 can be used for heap expansion of the space is not, but because the dynamic library 1 itself is disqualified, because the heap must be continuous, on Linux it is simply to add BRK to expand.

Assuming that the concept of the heap is canceled, the address space becomes the following:
Can run Program | Spare 1| dynamic LIBRARY 1| spare 2| ...
So spare 1 and spare 2 can be organized into a linked list. In this list, you can use a partner system or any other management algorithm. The continuity of the heap space was only to make memory use more compact, this is the legacy of the times, just like the exact division of Space continuous code segment, data segment, BSS section and now do not need, the continuity of the heap must have a new understanding.

After all, doing things in an age of abundance is not too petty.

Ideas for the modern operating system process address space