"Operating System" heap and memory management overview

-– when it comes to operating system stacks and memory management, that content really is the sea went, from the beginning of the place can continue to expand, but the importance of the content is also ineffable, the blog focused on the summary of the following three: Linux virtual address space layout heap and stack management, heap and stack differences

Some of the expanded knowledge will be involved in the middle, but will not be elaborate.

where to start: the memory layout of the program

To say heap and memory management, then the beginning of the place has to be the program's memory layout, that is, the virtual address space, the following posted a virtual address space map (stolen map):

Analysis: This diagram can let us clearly understand the virtual address space layout (32-bit);

The kernel part does not say, the ability is not enough-_-| |, the important is we know the virtual address space high 1G space is gives the kernel to be possible, certainly this is the Linux kernel, the Windows default assigns 2G to the kernel, certainly also may match the 1G; Interested students can go to understand the interruption, or process switching, the kernel stack and the user stack switch, is how to protect the scene, we have the default kernel of only 1G address space, and this 1G of the kernel space is all process sharing, that is, that sentence, Each process has a 4G virtual address space, but 1G is the kernel space shared by all processes, while the remaining 3G space is the private user address space of all processes, and the state of the program running in kernel and user space is called kernel State and user state. In the kernel State and user state of the running level is different, that is, permissions are different, when will enter the kernel state. The general interrupt will run into the kernel state, the interruption is divided into hardware interrupts and soft interrupts , hardware interrupts, such as when you enter the keyboard is the hardware interrupt, processing-protection of the scene-hardware processing-to get the interrupt vector code-find the Interrupt vector table-execution interrupt to the quantum program-Restore the scene Software interruption, the more common is that we call the system call, the feeling can not be pulled away, about the interruption is the school compiled textbooks to learn; about this piece of content recommended reference * * "Programmer self-cultivation" **12 Chapter; Leave a question, why should have the kernel state and user state.

Continue next, between the kernel space and the stack are environment variables and command-line arguments , which are set before the main function is executed, with regard to environment variables, under Linux, enter env to view environment variables,set To see all the variables,export the ordinary variable into an environment variable, environment variables will be the quilt process inheritance, on the verification of this is also very simple: you set a generic variable on the Linux command line, test= ' AAAA '; View variables: Echo $test; At this point test is a normal variable; you use env | grep test is not found if you use Set | grep test can see the set variable and now wants to let test become an environment variable: Export test; Test becomes an environment variable; now it's time to verify that the child process inherits the environment variable, so simply, the command-line input bash will be a child bash to see if the value of test is still in, exit the child Bash press ctrl+d, and post a demo diagram below. , command-line arguments are too familiar, you can get the command-line arguments through the parameters of the main function, in the argv inside, do not repeat;

3. Next, stack not too familiar, not we use container stack ah; we usually write code, local variables are stored on the stack, and the default stack size of each thread is 1M (can be modified) to maintain the context of function calls, (At this point you should recall the function of the calling convention, the function of the stack frame, etc.), at the same time, it is obvious that the stack growth direction from the high address to the low address;

Leave a question for this piece:

Object Return_fun
{
    //object is a class that returns the objects of a class;
    //Mode 1
    object ret;
    return ret;

    Mode 2: Optimized version return
    object ();

    Problem: Mode 1 and Mode 2 each called several copy constructors.  why.
}

Refer to the "self-cultivation of the programmer" P305

4. memory-Mapped segment: here, the kernel maps the contents of hard disk files directly to memory, and any program can request this mapping via Linux mmap system calls. Memory mapping is a convenient and efficient way of file i/0, so it is used to load dynamic shared libraries. Users can also create an anonymous memory map, which does not have a corresponding file, can be used to store data, in Linux, if the application of a larger memory through the malloc, it will use the mmap of the anonymous address space mapping, this in the following detail;

5. Heap: the same as the stack to store data, from the low address to the high address growth, users can apply for the memory, before the release of the user has been effective, but for C + +, pain and happiness, often because of memory leaks get burned, often envy Java garbage collection mechanism At the same time, different operating systems for the maintenance of the heap is not the same, the next part of the details;

6.BSS segment: uninitialized data segment, but actually does not contain data, only maintain a start address and end address, BSS section of data by default are zero, so in the binary executable file header, in fact, there is no BSS section, so, if someone asked you, The BSS section saves what the space is, you tell him that save the file space, not the memory space, because when the program read the BSS data, the kernel will be transferred to a full 0 pages, does not occur, but it will allocate the corresponding physical memory;
7. Datasection has been initialized, and BSS segment is called data segment, occupy the file space, also accounting for physical memory;
8. Code Snippet: that is, where the program code is stored, but also read-only, such as our char *str = "Hello World"; is stored in a read-only data area, in fact, and code snippets together, so, in the program we can not modify this string;
9. last part: the reserved area is located at the low address of the virtual address space, the physical address is not given, and any reference to it is illegal; For example, our operation on null pointers is illegal;

Finally put the above procrastination, of course, the focus of this blog is the following to say

Heap and memory management

1. First try the maximum amount of memory you can get on Linux and Windows.

Test code:

#include <iostream>

using namespace std;

unsigned maximum = 0;
int main ()
{
    unsigned blocksize[3] = {1024*1024,1024,1};

    int count = 0;
    for (int i = 0; i < 3; ++i)
    {for
        (count = 1;; count++)
        {
            void* block = malloc (maximum + blocksize[i]*c Ount);
            if (block)
            {
                maximum = maximum + Blocksize[i]*count;
                Free (block);
            } else{break
    ;
    }}} printf ("MaxSize is%u m\n", maximum/(1024*1024));
    System ("pause");
    return 0;
}

Windows Execution Results:

Linux Execution results:
There is so much space in the virtual address space that can be used as a heap space, and the 2.6 version of Linux to open up 1.9G of heap space, 2.6 version of the memory map after the increase and stack more and more near, heap can open up 2.9G of space, so such a large space we should how to manage it. Combined with the STL Space Configurator, why should there be space configurator? Because of the frequent application of memory to the operating system is relatively low efficiency, and if the frequent application of small memory, it will cause memory fragmentation problem, so directly to the configurator to throw a large space, let user space to deal with, and Windows and Linux, the underlying memory management is actually similar to the idea, Just more complicated;

Linux Process heap management:

Linux process heap Management is more complex, providing two ways to allocate, that is, two system calls, one is BRK () system call, the other is mmap (); the
function of BRK is to set the end address of the process data segment, that is, it can enlarge or reduce the data segment , we can use that part of the expansion of the data segment as a heap space, which is one of the most common practices. And glibc also has a function called SBRK (), function and brk consistent, in fact, is the brk of a layer of packaging, here is not deeply explored, meaningless;
the role of Mmap () is similar to the Vitualalloc under Windows, The role is to apply to the operating system for a virtual address space, when the virtual address can be mapped to a file when its role is shared memory, when it does not map to the file, then the virtual address space to become anonymous space, can be used as a heap space;

void *mmap
(
    void *start,//      The starting address of the application
    size_t length,    //length
    int port,        
    int flags,
    int FD,
    off_t offset
);

MMAP application of virtual address space is a multiple of the page, so the comparison consumes system performance, so if the application of small memory can not always call the Mmap bar, so the Linux glibc has its own set of allocation algorithm;

Here we focus on mmap how to get anonymous space: Look at the picture

If a vm_area_struct of the mmap map stores a space that is not occupied, the space can be used as a heap;

glibc heap allocation algorithm: For space Applications less than 64 bytes is a method similar to object pooling, and for space applications larger than 512 bytes is the optimal adaptation algorithm (operating system textbook is spoken); For larger than 64 bytes and less than 512 bytes, It will take the best compromise strategy of the above method according to the situation; for applications larger than 128KB, it will use the mmap mechanism to apply space directly to the operating system;

stack and heap comparison application method and efficiency:
The stack is distributed automatically by the system, the heap is requested and recycled by the programmer, and the idle list is also required to traverse the heap allocation, and the efficiency is self-evident; application size limit:
The general default size of the stack is 1M, more than the stack remaining space will be reported overflow error, so the space from the stack is relatively small;
Heap if it is in the new version of the Linxu can get up to 2.9G, under Windows generally 1.5G, up to 1.9G; storage Content lifecycle:
Stacks are generally used to do function call stack frame, store local variables and register values, so when the function is finished, the stack to clear data;
The heap application space holds the data, knowing that the heap is recycled, otherwise it always exists;

Finally, a problem remains: The return value of malloc (0).