Linux memory [Translate]

Resources

Linux Agent Capture Item Description-memory
Memory–part 1:memory Types
Memory–part 2:understanding Process Memory

IPCS in-depth analysis

Virtual memory

In modern operating systems, each process survives within its own memory space. However, the operating system does not directly address the memory address, but instead provides a hardware abstraction layer, which creates virtual memory for each process. The mapping between physical memory and virtual memory is obtained by the CPU using a mapping table, which is maintained by the kernel and each process has a record in the table.
Virtual memory has multiple purposes. First, process isolation is allowed. A process can only communicate with memory in virtual memory. This way it can only access the data associated with the process itself, and cannot see the memory data of other processes. Second, hardware abstraction. The kernel is free to change the mapping between the physical memory address and the virtual memory. It can also choose not to give virtual memory any real physical memory unless it is really needed. And it can be switched to the hard disk when the virtual memory is not used for a long time and the physical memory is tight. In general, the existence of virtual memory makes the kernel more free, and the only limitation is that when the program needs to read the memory, it needs to be converted to where the original data is written. Third, it can treat storage units that are not in RAM as memory. This is the principle of mmap and mapping files. You can think of a file as a memory, so accessing the file is like a memory buffer. This makes writing code more concise. Four, in order to share. Since the kernel knows which processes have been mapped to virtual memory, it avoids repeatedly loading the data into memory, but instead reuses the virtual addresses pointing to physical memory. The result of sharing is that the kernel uses cow (copy on write): When two processes use the same data, but when one changes the data and the other process does not allow the change, the kernel replicates a pre-changed data (a multi-version mechanism similar to the Oracle database).

Fork ()

The most famous case of cow is fork (). In Unix-like systems, fork () is a system call that can replicate an existing process to recreate one. When fork () returns, two processes will be at the same progress (open files, memory, and so on). Because of the cow mechanism, fork () does not actually replicate the memory of a process, and only data that the parent process has changed is replicated in RAM. Since exec () is called immediately by most of the scenes using fork (), the entire virtual memory space is invalidated, and the COW mechanism avoids completely useless memory being copied by the parent process.
Another side effect is that fork () creates a snapshot of the private memory of the process (snapshot) at a very low cost. If you want to do something in the memory of the process, but are afraid to change something, and don't want to debug, even though fork ()

Pages page

Virtual memory is split into pages. The paging size is specified by the CPU and is typically 4KiB. This means that the granularity of management memory in the kernel is 4K. When memory is requested, the kernel returns 1 or more pages, and when the memory is freed is also in page (Oracle default data file format 8K), any well-packaged API, such as malloc, is initiated on the client side.
For each assigned page, the kernel maintains a set of privileges (permission): The page can be read, written, or executed. These privileges are set when allocating memory, or when calling Mprotect (). Page that is not assigned cannot be accessed.

Memory type

Not all of the memory allocated in virtual memory is the same. We can classify it from two directions. The first class is whether the memory is private or shared. The second type is whether memory is file-backed. This creates a classification of 4 memory categories.
Private shared
Anonymous
1. Stack mmap (ANON, SHARED)
2. malloc ()
3. Mmap (ANON, PRIVATE)
4. BRK ()/sbrk ()
File-backed 1. Mmap (FD, PRIVATE) mmap (FD, GKFX)
2. Binary/shared Libraries

Private Memory

Private memory is unique to the process. Most programs use private memory.
Since private memory cannot be accessed by other processes, private memory is subject to cow. The side effect is that even if the memory is private, multiple processes share a piece of physical memory to store the data. For example, binary files and shared libraries.

Shared Memory

Shared memory is designed to implement inter-process communication. When shared memory is modified, other processes can see the change.

Ananymous Memory

Anonymous memory is all in RAM. But the kernel does not assign addresses until the RAM is actually written.

File-backed and Swap

If the memory map is file-backed, the data is loaded from the disk. Most of the time, it only loads when used. But you can also control the kernel to load ahead of time. When physical memory is not enough, the kernel moves some data from RAM to disk. In Linux, swap is a special area on disk. Because of virtual memory, the swap pages are completely transparent to the process.

Linux memory [Translate]