Principle of memory ing files

I have always been vague about the concept of memory ing files. I don't know what is the difference between it and virtual memory, and the word ing is also confusing. I finally figured it out today... Next, I will explain my understanding of the ing term first, and then distinguish between several confusing concepts. Then, it will be clear what memory ing is.

 

Principle

First of all, the word "ing" means a one-to-one ing with the math, that is, establishing a one-to-one ing relationship. Here

Files on Hard Disk

Location and Process

Logical Address Space

Medium
One-to-one correspondence between areas of the same size, as shown in process 1 in 1. This correspondence is purely a logical concept and does not physically exist because the logical address space of a process does not exist.
. During the memory ing process, there is no actual data copy, the file is not loaded into the memory, but is logically put into the memory, specific to the code, is to establish and initialize the relevant data structure
(Struct address_space), which is implemented by calling MMAP () systematically, so it is highly efficient to establish memory ing.

 

Figure 1. Memory ing principle
 

 

 

 

Since the memory ing is not actually copied, how can the process directly access the files on the hard disk through memory operations? It depends on several related processes after memory ing.

 

MMAP () will
Returns a pointer PTR pointing to an address in the logical address space of the process. In this way, the process no longer needs to call read or write to read and write the file, but only needs to be able to operate through PTR.
File. However, PTR points to a logical address. to operate on the data, you must use MMU to convert the logical address to a physical address, as shown in process 2 in 1. This process has nothing to do with memory ing.

 

Before
As mentioned above, the memory ing is not actually copied. In this case, MMU cannot find the physical address corresponding to PTR in the address ing table, that is, MMU fails, A missing page is interrupted.
The page interrupt response function will find the corresponding page in SWAp. If it cannot be found (that is, the file has never been read into the memory), it will use MMAP () the established ing relationship, from hard
The file is read to the physical memory on the disk, as shown in process 3 in 1. This process has nothing to do with memory ing.

 

If you find that the physical memory is insufficient during data copying, the physical pages that are not currently used are switched to the hard disk through the virtual memory mechanism (SWAP), as shown in process 4 in step 1. This process is also unrelated to memory ing.

 

 

Efficiency

 

Slave
At the code level, reading files from the hard disk into the memory must be copied through the file system, and the data copy operation is implemented by the file system and hardware driver. Theoretically, the efficiency of copying data is one.
. However, using the memory ing method to access files on the hard disk is more efficient than calling the Read and Write systems. Why? The reason is that read () is a system call, where data is performed
Copy: it first copies the file content from the hard disk to a buffer zone in the kernel space. Process 1 in step 2, and then copy the data to the user space. process 2 in step 2, in this process
Now

Two data copies

While MMAP () is also a system call. As mentioned above, no data is copied in MMAP (), and the real data copy is performed when page missing is interrupted. Because MMAP () the interrupt processing function directly copies the file from the hard disk to the user space based on the ing relationship.

One data copy

. Therefore, memory ing is more efficient than read/write.

 

 

Figure 2. Read system calling Principle

The following program operates a file named "mmap_test" on the hard disk by using the read and MMAP methods. The file contains 10000 integers, the program reads them two times in different ways, adds 1, and then writes them back to the hard disk. Through comparison, we can see that the read duration is nearly two to three times that of MMAP.

 

# Include <unistd. h>

# Include <stdio. h>

# Include <stdlib. h>

# Include <string. h>

# Include <sys/types. h>

# Include <sys/STAT. h>

# Include <sys/time. h>

# Include <fcntl. h>

# Include <sys/Mman. h>

 

# Deprecision Max 10000

 

Int main ()

{

Int I = 0;

Int COUNT = 0, FD = 0;

Struct timeval TV1, TV2;

Int * array = (int *) malloc (sizeof (INT) * max );

 

/* Read */

 

Gettimeofday (& TV1, null );

FD = open ("mmap_test", o_rdwr );

If (sizeof (INT) * max! = Read (FD, (void *) array, sizeof (INT) * max ))

{

Printf ("reading data failed.../N ");

Return-1;

}

For (I = 0; I <Max; ++ I)

 

++ Array [I];

If (sizeof (INT) * max! = Write (FD, (void *) array, sizeof (INT) * max ))

{

Printf ("writing data failed.../N ");

Return-1;

}

Free (array );

Close (FD );

Gettimeofday (& TV2, null );

Printf ("time of read/write: % DMS/N", tv2. TV _ usec-tv1. TV _usec );

 

/* MMAP */

 

Gettimeofday (& TV1, null );

FD = open ("mmap_test", o_rdwr );

Array = MMAP (null, sizeof (INT) * max, prot_read | prot_write, map_shared, FD, 0 );

For (I = 0; I <Max; ++ I)

 

++ Array [I];

Munmap (array, sizeof (INT) * max );

Msync (array, sizeof (INT) * max, ms_sync );

Free (array );

Close (FD );

Gettimeofday (& TV2, null );

Printf ("time of MMAP: % DMS/N", tv2. TV _ usec-tv1. TV _usec );

 

Return 0;

}

 

 

 

Output result:

Time of read/write: 154 Ms

Time of MMAP: 68 Ms