Inter-process communication in Linux (5) shared memory (on)

Inter-process communication in Linux (5): Shared Memory (on)

One obvious advantage of using shared memory communication is high efficiency, because the process can directly read and write the memory without any data copying. For communication methods such as photo channels and message queues, You need to copy data four times in the kernel and user space, while the shared memory only copies data twice [1]: one from the input file to the shared memory area, and the other from the shared memory area to the output file. In fact, during Memory sharing between processes, the ing is not always removed after a small amount of data is read and written. When there is new communication, the shared memory area is re-established. Instead, keep the shared area until the communication is complete. In this way, the data content is stored in the shared memory and not written back to the file. The content in the shared memory is often written back to the file when the ing is removed. Therefore, the efficiency of communication with shared memory is very high.

Linux 2.2.x kernel supports multiple memory sharing modes, such as MMAP () system calling, POSIX shared memory, and System V shared memory. For Linux Release versions such as RedHat 8.0 that support MMAP () system calls and System V shared memory, but POSIX shared memory has not yet been implemented, this article will mainly introduce MMAP () principle and Application of System Call and System V shared memory API.

I. How does the kernel ensure that each process can address the memory page in the same shared memory area?

1. Page cache and swapcache: The physical pages of an accessed file reside in the page cache or swap cache. All information on a page is described by the struct page. The struct page contains a ing field pointing to a struct address_space structure. All pages in page cache or swap cache are differentiated based on the address_space structure and an offset.

2. Correspondence between the file and address_space structure: after a specific file is opened, the kernel will create a struct inode structure for it in the memory, where the I _mapping domain points to an address_space structure. In this way, a file corresponds to an address_space structure, and an address_space and an offset can determine a page in a page cache or swap cache. Therefore, to address a specific data, it is easy to locate the corresponding page based on the given file and data offset in the file.

3. When a process calls MMAP (), a buffer of the corresponding size is added to the process space and the access ID is set, however, no mappings are established between process spaces and physical pages. Therefore, when you access the space for the first time, a page missing exception is thrown.

4. For Shared Memory ing, the page missing exception handler first searches the swap cache for the target page (the physical page that conforms to address_space and offset). If it finds the target page, it returns the address directly; if no value is found, the system checks whether the page is in the SWAp area. If yes, it performs a swap operation. If neither of the preceding conditions is met, the handler allocates a new physical page and inserts it into the page cache. The process will eventually update the process page table.

Note: For normal file ing (non-shared ing), the page missing exception handler first searches for corresponding pages in page cache based on address_space and data offset. If not found, the file data has not been read into the memory. The Handler reads the corresponding page from the disk, returns the corresponding address, and updates the process page table.

5. When all processes map to the same shared memory area, the situation is the same. After establishing a ing between a linear address and a physical address, regardless of the return address of the process, the actual access must be a physical page corresponding to the same shared memory area.

Note: A shared memory area can be considered as a file in the SHM of a special file system. The SHM installation point is in the SWAp area.

The above involves some data structures, and it is easier to understand the data structure.

Back to Top

Ii. MMAP () and related system calls

MMAP () system calls allow processes to share memory by ing to the same common file. After a common file is mapped to the process address space, the process can access the file like accessing the common memory without calling read (), write (), and other operations.

Note: In fact, MMAP () system calls are not completely designed for Memory Sharing. It provides different access methods for common files. A process can perform operations on common files like read/write memory. POSIX or System V's shared memory IPC is purely used for sharing purposes. Of course, MMAP ()'s shared memory is also one of its main applications.

1. The MMAP () system call form is as follows:

Void * MMAP (void * ADDR, size_t Len, int Prot, int flags, int FD, off_t offset)

The FD parameter is the description of the file to be mapped to the process space. It is generally returned by open (). At the same time, FD can be specified as-1. In this case, map_anon must be specified in the flags parameter, it indicates that the row is anonymous ing (not involving specific file names, avoiding File Creation and opening. Obviously, it can only be used for Kinship-related inter-process communication ). Len is the number of bytes mapped to the address space of the calling process. It starts from the offset byte at the beginning of the mapped file. The prot parameter specifies the access permission for the shared memory. The following values can be obtained: prot_read (readable), prot_write (writable), prot_exec (executable), and prot_none (inaccessible ). Flags are specified by the following common values: map_shared
, Map_private, map_fixed. Among them, map_shared and map_private are required. map_fixed is not recommended. The offset parameter is generally set to 0, indicating that the ing starts from the file header. The ADDR parameter specifies that the file should be mapped to the starting address of the process space. Generally, a null pointer is specified. At this time, the task of selecting the starting address is left to the kernel for completion. The Return Value of the function is the address mapped from the last file to the process space. The starting address of a process operation can be the valid address of the value. Here, we will not detail MMAP () parameters. You can refer to the MMAP () manual page for further information.

2. The system calls MMAP () to share memory in two ways:

(1) Use the memory ing provided by common files: Suitable for any process; in this case, you need to open or create a file and then call MMAP (). The typical Call Code is as follows:

FD = open (name, flag, mode ); If (FD <0) ...

 

PTR = MMAP (null, Len, prot_read | prot_write, map_shared, FD, 0); MMAP () provides many features and precautions for shared memory communication, we will describe it in the example.

(2) use special files to provide anonymous memory ing: Suitable for processes with kinship. Because of the special kinship between parent and child processes, MMAP () is called first in the parent process (), then call fork (). After fork () is called, the child process inherits the address null after the anonymous ing of the parent process, and also inherits the address returned by MMAP, the Parent and Child processes can communicate through the ing area. Note that this is not a general inheritance relationship. Generally, child processes independently maintain some variables inherited from the parent process. The address returned by MMAP () is maintained by the parent and child processes.

The best way to achieve shared memory for unrelated processes is to use anonymous memory ing. In this case, you do not need to specify a specific file. You only need to set the corresponding flag. See example 2.

3. The system calls munmap ()

Int munmap (void * ADDR, size_t Len)
This call removes a ing relationship from the process address space. ADDR is the address returned when MMAP () is called, And Len is the size of the ing area. When the ing relationship is removed, access to the original ing address will cause a segment error.

4. The system calls msync ()

Int msync (void * ADDR, size_t Len, int flags)
Generally, changes to shared content in the ing space of a process are not directly written back to the disk file. This operation is usually performed only after munmap () is called. You can call msync () to ensure that the file content on the disk is consistent with that in the shared memory area.

Back to Top

Iii. MMAP () Example

The following two examples Use MMAP (): Example 1: two processes map common files to implement shared memory communication; Example 2: The Parent and Child processes implement shared memory through anonymous ing. MMAP () is called by the system. The following is an example of MMAP () ing between common files to implement inter-process communication. We use this example to describe MMAP () features and precautions for implementing shared memory.

Example 1: two processes map common files to implement shared memory communication

Example 1 contains two subprograms: map_normalfile1.c and map_normalfile2.c. Compile two programs. The executable files are map_normalfile1 and map_normalfile2. The two programs use the command line parameter to specify the same file to implement inter-process communication in the Memory sharing mode. Map_normalfile2 tries to open a common file specified by the command line parameter, map the file to the address space of the process, and write the mapped address space. Map_normalfile1 maps the file specified by the command line parameter to the process address space, and then performs read operations on the mapped address space. In this way, two processes use the command line parameter to specify the same file to implement inter-process communication in the Memory sharing mode.

The following are two program codes:

/* ------------- Map_normalfile1.c -----------*/ # Include <sys/Mman. h> # Include <sys/types. h> # Include <fcntl. h> # Include <unistd. h> Typedef struct { Char name [4]; Int age; } People; Main (INT argc, char ** argv) // map a normal file as shared mem: { Int FD, I; People * p_map; Char temp; FD = open (argv [1], o_creat | o_rdwr | o_trunc, 00777 ); Lseek (FD, sizeof (people) * 5-1, seek_set ); Write (FD, "", 1 ); P_map = (People *) MMAP (null, sizeof (people) * 10, prot_read | prot_write, Map_shared, FD, 0 ); Close (FD ); Temp = 'a '; For (I = 0; I <10; I ++) { Temp + = 1; Memcpy (* (p_map + I). Name, & temp, 2 ); (* (P_map + I). Age = 20 + I; } Printf ("initialize over \ n "); Sleep (10 ); Munmap (p_map, sizeof (people) * 10 ); Printf ("umap OK \ n "); } /* ------------- Map_normalfile2.c -----------*/ # Include <sys/Mman. h> # Include <sys/types. h> # Include <fcntl. h> # Include <unistd. h> Typedef struct { Char name [4]; Int age; } People; Main (INT argc, char ** argv) // map a normal file as shared mem: { Int FD, I; People * p_map; FD = open (argv [1], o_creat | o_rdwr, 00777 ); P_map = (People *) MMAP (null, sizeof (people) * 10, prot_read | prot_write, Map_shared, FD, 0 ); For (I = 0; I <10; I ++) { Printf ("Name: % s age % d; \ n", (* (p_map + I). Name, (* (p_map + I). Age ); } Munmap (p_map, sizeof (people) * 10 ); }

 

Map_normalfile1.c first defines a people data structure (the data structure is used here because the data in the shared memory area is usually in a fixed format, which is determined by the communication processes, the structure is generally representative ). Map_normfile1 first open or create a file, and set the file length to 5 people structure sizes. Then, 10 people structures are set starting from the return address of MMAP. Then, the process sleep for 10 seconds, wait for other processes to map the same file, and finally unmap.

Map_normfile2.c is just a simple ing of a file. It reads 10 people structures from the address returned by MMAP () in the format of people data structure, outputs the read values, and unmaps them.

After compiling the two programs into the executable files map_normalfile1 and map_normalfile2, run./map_normalfile2/tmp/test_shm on one terminal. The output result is as follows:

Initialize over Umap OK

 

Run map_normalfile2/tmp/test_shm on another terminal after map_normalfile1 outputs initialize over and umap OK (to save space, the output result is a result after sorting ):

Name: B age 20; Name: C age 21; Name: d age 22; Name: e age 23; Name: F age 24; Name: G age 25; Name: H age 26; Name: I age 27; Name: J age 28; Name: K age 29;

 

After map_normalfile1 outputs umap OK, run map_normalfile2 and output the following results:

Name: B age 20; Name: C age 21; Name: d age 22; Name: e age 23; Name: F age 24; Name: age 0; Name: age 0; Name: age 0; Name: age 0; Name: age 0;

 

Conclusion drawn from the running results of the program

1. The content length of the finally mapped file will not exceed the initial size of the file, that is, the ing cannot change the file size;

2. The size of the valid address space that can be used for process communication is generally limited by the size of the mapped file, but not completely limited by the file size. The opened file is truncated to 5 people structure sizes, and 10 people data structures are initialized in map_normalfile1. when appropriate (map_normalfile1 outputs initialize over and before umap OK) when map_normalfile2 is called, map_normalfile2 will output the values of all 10 people structures, which will be discussed in detail later.

Note: in Linux, the memory protection is based on pages. Even if the mapped file has only one byte size, the kernel will allocate a page size memory for the ing. When the size of the mapped file is smaller than the size of a page, the process can access the page size starting with the return address of MMAP () without errors. However, if you access an address space other than a page, an error occurs, which will be further described later. Therefore, the size of the valid address space used for inter-process communication does not exceed the file size and the sum of the size of a page.

3. Once a file is mapped, the process that calls MMAP () accesses the returned address to a memory area, which is temporarily out of the influence of files on the disk. All operations on the address space returned by MMAP () are only meaningful in the memory. Only after munmap () is called or msync () is the corresponding content in the memory written back to the disk file, the size of the file cannot exceed the size of the file.

Example 2: the parent-child process achieves shared memory through anonymous ing

# Include <sys/Mman. h> # Include <sys/types. h> # Include <fcntl. h> # Include <unistd. h> Typedef struct { Char name [4]; Int age; } People; Main (INT argc, char ** argv) { Int I; People * p_map; Char temp; P_map = (People *) MMAP (null, sizeof (people) * 10, prot_read | prot_write, Map_shared | map_anonymous,-1, 0 ); If (Fork () = 0) { Sleep (2 ); For (I = 0; I <5; I ++) Printf ("child read: The % d people's age is % d \ n", I + 1, (* (p_map + I). Age ); (* P_map). Age = 100; Munmap (p_map, sizeof (people) * 10); // In fact, The ing is automatically removed when the process is terminated. Exit (); } Temp = 'a '; For (I = 0; I <5; I ++) { Temp + = 1; Memcpy (* (p_map + I). Name, & temp, 2 ); (* (P_map + I). Age = 20 + I; } Sleep (5 ); Printf ("parent read: The First People, s age is % d \ n", (* p_map). Age ); Printf ("umap \ n "); Munmap (p_map, sizeof (people) * 10 ); Printf ("umap OK \ n "); }

 

Measure the test taker's knowledge about the output result of the program and the anonymous memory shared by the parent and child processes:

Child read: The 1 People's age is 20 Child read: The 2 people's age is 21 Child read: The 3 people's age is 22 Child read: The 4 people's age is 23 Child read: The 5 people's age is 24 Parent read: The First People, s age is 100 Umap Umap OK

 

Back to Top

4. Access to the address returned by MMAP ()

As mentioned in the previous discussion about the sample running structure, Linux uses a page-based management mechanism. For MMAP () ing common files, the process adds a space in its own address space. The space size is specified by the Len parameter of MMAP (). Note, the process may not be able to effectively access all new spaces. The valid address that a process can access depends on the size of the mapped part of the file. Simply put, the minimum number of pages that can accommodate the size of the mapped part of the file determines the size of the address space that can be effectively accessed by the process starting from the address returned by MMAP. If the size of this space exceeds the threshold, the kernel returns different signals to the process based on the severity. Available instructions:

Note: The mapped part of the file, instead of the entire file, determines the size of the space that the process can access. In addition, if you specify the offset part of the file, note that it is an integer multiple of the page size. The following is an example of access to the process ing address space:

# Include <sys/Mman. h> # Include <sys/types. h> # Include <fcntl. h> # Include <unistd. h> Typedef struct { Char name [4]; Int age; } People; Main (INT argc, char ** argv) { Int FD, I; Int pagesize, offset; People * p_map; Pagesize = sysconf (_ SC _pagesize ); Printf ("pagesize is % d \ n", pagesize ); FD = open (argv [1], o_creat | o_rdwr | o_trunc, 00777 ); Lseek (FD, pagesize * 2-100, seek_set ); Write (FD, "", 1 ); Offset = 0; // here offset = 0 is compiled into version 1; offset = pagesize is compiled into Version 2 P_map = (People *) MMAP (null, pagesize * 3, prot_read | prot_write, map_shared, FD, offset ); Close (FD ); For (I = 1; I <10; I ++) { (* (P_map + pagesize/sizeof (people) * I-2). Age = 100; Printf ("access page % d over \ n", I ); (* (P_map + pagesize/sizeof (people) * i-1). Age = 100; Printf ("access page % d edge over, now begin to access page % d \ n", I, I + 1 ); (* (P_map + pagesize/sizeof (people) * I). Age = 100; Printf ("access page % d over \ n", I + 1 ); } Munmap (p_map, sizeof (people) * 10 ); }

 

As noted in the program, compile the program into two versions. The two versions are mainly reflected in the size of the mapped part of the file. The file size is between one page and two pages (the size is pagesize * 2-99). The mapped part of version 1 is the entire file, the mapped part of version 2 is the file size minus the rest of a page, less than a page size (size: pagesize-99 ). The program tries to access each page boundary. Both versions attempt to map pagesize * 3 bytes in the process space.

The output result of version 1 is as follows:

Pagesize is 4096 Access page 1 over Access page 1 edge over, now begin to access page 2 Access page 2 over Access page 2 over Access page 2 edge over, now begin to access page 3 Bus Error // The ing file overwrites two pages in the process space. At this time, the process tries to access the third page.

 

The output result of version 2 is as follows:

Pagesize is 4096 Access page 1 over Access page 1 edge over, now begin to access page 2 Bus Error // The mapped file overwrites a page in the process space. At this time, the process tries to access the second page.

 

Conclusion: It is convenient to use the system to call MMAP () for inter-process communication, and the interface on the application layer is very simple. The internal implementation mechanism covers Linux Storage Management and file systems. You can refer to the important data structures for better understanding. Later in this topic, we will introduce the implementation of system V shared memory.

 

References

[1] understanding the Linux kernel, 2nd edition, by Daniel P. bovet, marcocesati, which focuses on each topic and has a clear context.

[2] UNIX Network Programming Volume 2: inter-process communication, Author: W. Richard Steven S, Translator: Yang jizhang, Tsinghua University Press. MMAP () is described in detail.

[3] source code analysis for Linux kernel (I): source code analysis related to MMAP () is provided by Mao decao, Hu Ximing, and Zhejiang University Press.

[4] MMAP () Manual

About the author

Zheng yanxing, a doctorate degree from the National Defense University. Contact info:
Mlinux@163.com