The principle of Shared memory: file mapping

Mmap () is the use of the specified file using virtual memory technology to map into memory, the page table entry in memory, but the mmap () call to establish a page table entry after the address is empty, only when the use of an address space to allocate space for this page, to involve the page replacement, it is necessary to have a certain amount of physical memory to support, If the memory is too small, the pages that have just been placed in memory are swapped to disk, and the performance of the mmap will be compromised.

Mmap's advantages are mainly in the user program random access, operation, file provides a convenient way to operate, followed by the sharing of large quantities of data for different processes to provide efficient means of processing, but also for large files (not one-time memory) of the treatment provided an effective method.

Advantages of File mapping:

The first conclusion: using memory-mapped files to handle large files can improve efficiency.

Why is it?

Let's take a look at this. If the process of not using a memory-mapped file is what, first we have to read the contents of the disk file into memory, then modify it and write it back to disk. The first read disk file is a system call, it first copies the contents of the file from disk to a buffer in the kernel space, and then copies the data to the user space, which is actually two copies of the data. The third step of the write-back also takes two copies of the data.

So we basically have four copies of the data, because large file data volume is large, dozens of GB or even larger, so the cost of copying is very large.

A memory-mapped file is a mechanism provided by the operating system that can reduce this unnecessary copying of data and thus improve efficiency. It is mapped directly to user space by Mmap (), mmap () does not have a copy of the data, the real copy of the data is in the fault processing of the fault, because MMAP () directly map the file to user space, so the interrupt handler function according to this mapping relationship, directly copy the file from the hard disk to the user space , so only a single copy of the data, two copies of the data copy is better than read, so the efficiency of memory mapping is higher than read/write efficiency.

In general, the read write operation can meet the requirements of most file operations, but for some dozens of GB or more of storage that is required for some special application areas, the usual file handling method is obviously not working.

Mmap maps a file or other object into memory. Files are mapped to multiple pages, and if the size of the file is not the sum of the size of all pages, the space that is not used by the last page will be zeroed out. Munmap performs the opposite operation by deleting the object mappings for a specific address area.

When using the mmap mapping file to the process, you can directly manipulate this virtual address for file read and write operations, no longer need to call Read,write and other system calls. However, it is important to note that the memory written directly to the segment does not write content that exceeds the current file size.

One obvious benefit of using shared memory communication is that it is efficient because the process can read and write directly to the memory without requiring any copy of the data. For communications like pipelines and message queues, four copies of the data are required in the kernel and user space, while shared memory copies only two data: one from the input file to the shared memory area, and the other from the shared memory area to the output file. In fact, when you share memory between processes, you do not always have to read and write small amounts of data, and then re-establish the shared memory area when there is new communication. Instead, the shared area is maintained until the communication is complete, so that the data content is kept in shared memory and is not written back to the file. Content in shared memory is often written back to a file when it is de-mapped. Therefore, the use of shared memory communication mode is very efficient.

The principle of Shared memory: file mapping