Shared memory can be said to be the most useful way to communicate between processes and the fastest form of IPC. Two different processes A, B shared memory means that the same physical memory is mapped to process A, B's respective process address space. Process A can instantly see Process b update of the data in shared memory, and vice versa. Because multiple processes share the same chunk of memory, it is necessary to have some sort of synchronization mechanism, mutual exclusion locks and semaphores.
One obvious benefit of using shared memory traffic is that it is efficient because processes can read and write directly to memory without requiring copies of any data. For communication like pipelines and Message Queuing, four copies of the data are required in the kernel and user space, while shared memory copies only two times of data [1]: One from the input file to the shared memory area and the other from the shared memory area to the output file. In fact, when sharing memory between processes, the mappings are not always read and written, and when there is new communication, the shared memory area is again established. Instead, keep the shared area until the communication is complete so that the data content is kept in shared memory and not written back to the file. Content in shared memory is often written back to the file when the mapping is released. Therefore, the use of shared memory communication mode is very high efficiency.
The Linux 2.2.x kernel supports a variety of shared memory modes, such as mmap () system calls, POSIX shared memory, and System V shared memory. Linux distributions such as Redhat 8.0 support mmap () system calls and System V shared memory, but have not implemented POSIX shared memory, this article will mainly introduce the mmap () system call and System V shared Memory API principle and application.
How the kernel ensures that each process addresses the memory pages of the same shared memory area
1, page cache and swap cache pages in the distinction: a file access to the physical pages are residing in page cache or swap cache, a page of all information by struct page to describe. A field in the struct page is a pointer mapping that points to a struct ADDRESS_SPACE type structure. All pages in page cache or swap cache are differentiated according to the address_space structure and an offset.
2, file and address_space structure of the corresponding: a specific file in the open, the kernel will be in memory for it to establish a struct inode structure, where the i_mapping domain point to a address_space structure. Thus, a file corresponds to a address_space structure, a address_space and an offset can determine a page in the page cache or swap cache. Therefore, when you are addressing a data, it is easy to find the page based on the number of offsets given to the file and data within the file.
3, the process calls Mmap (), only in the process space to add a corresponding size of the buffer, and set the corresponding access identity, but did not establish process space to the physical page mapping. Therefore, the first time you access the space, a page fault exception is raised.
4, for shared memory mapping, the page fault exception handler first in the swap cache to find the target page (in line with the Address_space and offset physical pages), if found, directly return the address, if not found, then determine if it is in the swap zone (swap area), If it is, a swap operation is performed, and if neither of the above is true, the handler assigns a new physical page and inserts it into the page cache. The process will eventually update the Process page table.
Note: For mapping common file conditions (unshared mappings), the page-fault exception handler will first look for pages in page cache based on Address_space and data offsets. If it is not found, the file data is not yet read into memory, the handler reads the corresponding page from disk and returns the address, and the Process page table is also updated.
5, all processes in mapping the same shared memory area, the situation is the same, in the establishment of a linear address and the mapping between the physical address, regardless of the process of their respective return address, the actual access is necessarily the same shared memory area corresponding to the physical page.
Note: A shared memory area can be viewed as a file in the special file system SHM, and the SHM installation point is on the swap area.
Some data structures are involved and it is easier to understand problems around data structures.
|
back to the top of the page |
|
Ii. mmap () and its related system calls
Mmap () system calls enable shared memory to be implemented by mapping the same common file between processes. After the normal file is mapped to the process address space, the process can access the file as normal memory, without having to call read (), write (), and so on.
Note: In fact, mmap () system calls are not designed entirely for shared memory. It itself provides a different way of accessing normal files, and processes can manipulate normal files like memory. The shared memory IPC for POSIX or System V is purely for shared purposes, and of course mmap () is one of the main applications of shared memory.
1, mmap () system call form is as follows:
void* mmap (void * addr, size_t len, int prot, int flags, int fd, off_t offset)
Parameter FD is a file descriptor that is about to be mapped to a process space, typically returned by open (), and FD can be specified as-1, specifying the Map_anon in the flags parameter to indicate an anonymous mapping (without involving a specific filename, avoiding the creation and opening of the file). It is obvious that it can only be used for relational interprocess communication. Len is the number of bytes mapped to the calling process address space, starting with the offset byte at the beginning of the mapped file. The prot parameter specifies the access rights for shared memory. The following values are preferable: prot_read (readable), prot_write (writable), prot_exec (executable), Prot_none (inaccessible). Flags are specified by the following constant values: map_shared, Map_private, map_fixed, where, map_shared, map_private must choose one, and map_fixed is not recommended. The offset parameter is typically set to 0, which indicates that the mapping starts at the file header. The parameter addr specifies that the file should be mapped to the starting address of the process space and is typically assigned a null pointer, at which point the task of selecting the starting address is left to the kernel to complete. The return value of the function is the address that the final file maps to the process space, and the process can directly manipulate the starting address as a valid address for that value. The parameters of mmap () are no longer detailed here, and readers can refer to the mmap () manual page for further information.
2, the system calls Mmap () for shared memory two ways:
(1) Memory mapping provided using normal files: for any process; At this point, you need to open or create a file, then call Mmap (), and the typical calling code is as follows:
Fd=open (name, flag, mode);
if (fd<0)
...
|
Ptr=mmap (NULL, Len, prot_read| Prot_write, map_shared, FD, 0); There are a number of features and areas to note through the mmap () way of communicating shared memory, which we will specify in the example.
(2) Use special files to provide anonymous memory mappings: For relationships between processes, because of the special affinity of parent-child processes, call Mmap () in the parent process, and then call Fork (). After the call to fork (), the child process inherits the address space of the parent process's anonymous mapping and also inherits the address returned by Mmap (), so that the parent-child process can communicate through the mapped area. Note that this is not a general inheritance relationship. In general, a child process maintains a separate number of variables inherited from the parent process. The address returned by Mmap () is maintained jointly by the parent-child process.
The best way to implement shared memory for a relational process should be to use an anonymous memory map. At this point, you do not have to specify a specific file, just set the appropriate flag, see Example 2.
3, System call Munmap ()
int Munmap (void * addr, size_t len)
The call unlocks a mapping relationship in the process address space, the address that is returned when the mmap () is invoked, and Len is the size of the map area. addr When the mapping relationship is lifted, access to the original mapped address will cause a segment error to occur.
4, System call Msync ()
int Msync (void * addr, size_t len, int flags)
In general, changes to shared content in a mapped space are not directly written back to the disk file, and are often performed after the call to Munmap (). You can implement the contents of the file on the disk consistent with the content of the shared memory area by calling Msync ().
|
back to the top of the page |
|
Iii. mmap () example
Here are two examples of using mmap (): Example 1 gives two processes to achieve shared memory traffic by mapping common files; Example 2 gives the parent-child process the shared memory through an anonymous mapping. There are a lot of interesting places in the system call Mmap (), the following is an example of mmap () mapping common files to implement interprocess communication, and we illustrate the characteristics and considerations of mmap () implementation of shared memory by this example.
Example 1: Two processes to achieve shared memory communication by mapping common Files
Example 1 contains two subroutines: map_normalfile1.c and map_normalfile2.c. Compile two programs, the executable files are Map_normalfile1 and Map_normalfile2 respectively. Two programs specify the same file through command-line arguments to implement interprocess communication in shared memory mode. Map_normalfile2 attempts to open a normal file specified by a command-line argument, maps the file to the address space of the process, and writes the mapped address space. Map_normalfile1 maps the file specified by the command-line argument to the process address space, and then performs a read operation on the mapped address space. In this way, two processes specify the same file through the command-line arguments to implement interprocess communication in shared memory mode.
Here are two program codes:
* *-------------map_normalfile1.c-----------* *
#include <sys/mman.h>
#include <sys/types.h>
#include <fcntl.h>
#include <unistd.h>
typedef struct{
Char name[4];
int age;
}people;
Main (int argc, char** argv)//Map a normal file as shared mem:
{
int fd,i;
People *p_map;
char temp;
Fd=open (argv[1],o_creat| o_rdwr| o_trunc,00777);
Lseek (fd,sizeof (people) *5-1,seek_set);
Write (FD, "", 1);
P_map = (people*) mmap (null,sizeof (people) *10,prot_read| prot_write,map_shared,fd,0);
Close (FD);
temp = ' a ';
for (i=0; i<10; i++)
{
temp = 1;
memcpy ((* (P_map+i)). Name, &temp,2);
(* (P_map+i)). Age = 20+i;
}
printf ("Initialize over/n");
Sleep (10);
Munmap (P_map, sizeof (people) *10);
printf ("Umap ok/n");
}
* *-------------map_normalfile2.c-----------* *
#include <sys/mman.h>
#include <sys/types.h>
#include <fcntl.h>
#include <unistd.h>
typedef struct{
Char name[4];
int age;
}people;
Main (int argc, char** argv)//Map a normal file as shared mem:
{
int fd,i;
People *p_map;
Fd=open (argv[1],o_creat| o_rdwr,00777);
P_map = (people*) mmap (null,sizeof (people) *10,prot_read| prot_write,map_shared,fd,0);
for (i = 0;i<10;i++)
{
printf ("Name:%s Age%d;/n", (* (P_map+i)). Name, (* (p_map+i));
}
Munmap (p_map,sizeof (people) *10);
}
|
MAP_NORMALFILE1.C first defines a people data structure, where the data structure is used because of the fact that the shared memory area is often in a fixed format, which is determined by the processes of the communication and is universally representative in the way it is structured. Map_normfile1 first opens or creates a file and sets the length of the file to 5 people structure size. Then, starting with the return address of mmap (), 10 people structures are set up. The process then sleeps for 10 seconds, waits for the other processes to map the same file, and finally unlocks the map.
MAP_NORMFILE2.C simply maps a file and reads 10 people structures from the address returned from Mmap () in the format of the people data structure, outputting the read values and then unblocking the mappings.
The two programs are compiled into executables map_normalfile1 and Map_normalfile2, and run first on a terminal./MAP_NORMALFILE2/TMP/TEST_SHM, the program outputs the following results:
After the Map_normalfile1 output initialize over, the output umap OK, run on another terminal Map_normalfile2/tmp/test_shm, will produce the following output (in order to save space, The result of the output is slightly sorted out):
Name:b age ; Name:c age ; Name:d age ; Name:e age ; Name:f Age 24;
Name:g age ; Name:h age ; Name:i age ; Name:j age ; Name:k age 29;
|
After the map_normalfile1 output umap OK, the run Map_normalfile2 outputs the following results:
Name:b age ; Name:c age ; Name:d age ; Name:e age ; Name:f Age 24;
Name: Age 0; Name: Age 0; Name: Age 0; Name: Age 0; Name: Age 0;
|
conclusions that can be drawn from the operating results of the program
1, the final content of the mapped file will not exceed the original size of the file itself, that is, mapping can not change the size of the file;
2. The amount of valid address space that can be used for process communication is largely limited by the size of the mapped file, but not entirely limited by the file size. The open file is truncated to 5 people structure sizes, and 10 people data structures are initialized in Map_normalfile1, at the appropriate time (Map_normalfile1 output initialize over, output umap OK before calling Map_normalfile2 will find that map_normalfile2 will output the values of all 10 people structures, followed by a detailed discussion.
Note: In Linux, memory protection is based on the page, even if the mapped file has only one byte size, the kernel also allocates a page size memory for the mapping. When the mapped file is less than a page size, the process can access a page size that starts at the mmap () return address without error, but if access to an address space other than a page results in an error, it is further described later. Therefore, the valid address space available for interprocess communication does not exceed the size of the file and the size of a page.
3, once the file is mapped, the call to the mmap () of the process of access to the return address is a memory area of access, temporarily disconnected from the impact of the disk files. All operations on the mmap () return address space are only meaningful in memory, and only after the Munmap () or Msync () is invoked, the contents of the memory are written back to the disk file, and the content is still not larger than the file size.
Example 2: The parent-child process implements shared memory through anonymous mapping
#include <sys/mman.h>
#include <sys/types.h>
#include <fcntl.h>
#include <unistd.h>
typedef struct{
Char name[4];
int age;
}people;
Main (int argc, char** argv)
{
int i;
People *p_map;
char temp;
P_map= (people*) mmap (null,sizeof (people) *10,prot_read| prot_write,map_shared| map_anonymous,-1,0);
if (fork () = = 0)
{
Sleep (2);
for (i = 0;i<5;i++)
printf ("Child read:the%d people ' s The age is%d/n", I+1, (* (P_map+i)).
(*p_map). Age = 100;
Munmap (p_map,sizeof (people) *10); In fact, when the process terminates, the mapping is automatically lifted.
Exit ();
}
temp = ' a ';
for (i = 0;i<5;i++)
{
temp = 1;
memcpy ((* (P_map+i)). Name, &temp,2);
(* (P_map+i)). Age=20+i;
}
Sleep (5);
printf ("Parent read:the-People,s is%d/n", (*p_map). age);
printf ("umap/n");
Munmap (p_map,sizeof (people) *10);
printf ("Umap ok/n");
}
|
Examine the output of the program, and realize that the parent-child process anonymous shared memory:
Child Read:the 1 People ' s 20
Child Read:the 2 people ' s 21
Child Read:the 3 People ' s 22
Child Read:the 4 People ' s 23
Child Read:the 5 people ' s 24
Parent Read:the-A-people,s is 100
Umap
Umap OK
|
|
back to the top of the page |
|
Iv. access to Mmap () return address
It has been mentioned in the previous discussion of the example running structure that Linux uses a page-style management mechanism. For mmap () mapping a normal file, the process adds a space in its own address space, which is specified by the Len parameter of the mmap (), noting that the process is not necessarily capable of effectively accessing all new space. The valid address size that a process can access depends on the size of the portion of the file being mapped. Simply put, the minimum number of pages that can accommodate the size of the mapped part of a file determines the size of the address space that the process starts with the address returned from Mmap () and can be effectively accessed. Beyond this space, the kernel returns a different signal to the process depending on the severity of the excess. You can use the following illustration to illustrate:
Note: The file is mapped rather than the entire file determines the amount of space the process can access, in addition, if you specify the offset portion of the file, be sure to pay attention to the size of the page integer multiple times. The following is an example of an access to the process mapping address space:
#include <sys/mman.h>
#include <sys/types.h>
#include <fcntl.h>
#include <unistd.h>
typedef struct{
Char name[4];
int age;
}people;
Main (int argc, char** argv)
{
int fd,i;
int pagesize,offset;
People *p_map;
pagesize = sysconf (_sc_pagesize);
printf ("PageSize is%d/n", pagesize);
FD = open (argv[1],o_creat| o_rdwr| o_trunc,00777);
Lseek (Fd,pagesize*2-100,seek_set);
Write (FD, "", 1);
offset = 0; Here offset = 0 compiled into version 1;offset = pagesize compiled into version 2
P_map = (people*) mmap (null,pagesize*3,prot_read| Prot_write,map_shared,fd,offset);
Close (FD);
for (i = 1; i<10; i++)
{
(* (p_map+pagesize/sizeof (People) *i-2)). Age = 100;
printf ("Access page%d over/n", i);
(* (p_map+pagesize/sizeof (People) *i-1)). Age = 100;
printf ("Access page%d edge over, now begin to access page%d/n", I, i+1);
(* (p_map+pagesize/sizeof (People) *i)). Age = 100;
printf ("Access page%d over/n", i+1);
}
Munmap (p_map,sizeof (people) *10);
}
|
As noted in the program, the program is compiled into two versions, and two versions are mainly reflected in the size of the file being mapped. The size of the file is between one page and two pages (size: pagesize*2-99), the mapped portion of version 1 is the entire file, and version 2 of the file is mapped to a file size minus the remainder of a page, less than a page size (size: pagesize-99). In a program that attempts to access each page boundary, two versions attempt to map the number of pagesize*3 bytes in the process space.
The output of version 1 is as follows:
PageSize is 4096
Access page 1 over
Access page 1 edge over, now begin to access page 2
Access page 2 over
Access page 2 over
Access page 2 edge over, now begin to access page 3
The bus error //mapped file covers two pages in the process space, at which point the process attempts to access the third page
|
The output of version 2 is as follows:
PageSize is 4096
Access page 1 over
Access page 1 edge over, now begin to access page 2
The bus error //mapped file overwrites a page in the process space, at which point the process attempts to access the second page
|
Conclusion: It is convenient to use system call MMAP () to realize interprocess communication, and the interface is very concise on the application layer. The internal implementation mechanism area involves the contents of Linux storage Management and file system, and can refer to the relevant important data structure to deepen understanding. Later in this topic, the implementation of System V shared memory is described.
Resources
[1] Understanding the Linux Kernel, 2nd Edition, by Daniel P. Bovet, Marco Cesati, focused on the themes, clear venation.
[2] UNIX Network programming Volume II: Interprocess communication, Author: W.richard Stevens, translator: Yang Jizhang, Tsinghua University Press. The mmap () is described in detail.
[3] Linux kernel source code scenario analysis (above), Maudeca, Hu Himing, Zhejiang University Press, gave Mmap () related source analysis.
[4]mmap () Manual
