Common traps and Analysis of Linux shared memory usage

Original article: http://davidlu1001.blogbus.com/logs/85182548.html

Shared memory allows multiple processes to access the same memory space. It is the fastest available IPC format. It is designed to reduce the running efficiency of other communication mechanisms. It is often used in conjunction with other communication mechanisms, such as semaphores, to achieve synchronization and mutual exclusion between processes. Other processes can "Connect" the same shared memory segment to their own address space. All processes can access the addresses in the shared memory. If a process writes data to the shared memory, the changes will be immediately seen by other processes that access the same shared memory. The use of shared memory greatly reduces the memory consumption in the Process of large-scale data processing. However, there are many traps in the use of shared memory. If you do not pay attention to it, the program may crash.
Exceeds the size limit of shared memory?
On a Linux server, the total size of shared memory is limited. The size is defined by the shmmax parameter (in bytes ), you can determine the value of shmmax by executing the following command:
# Cat/proc/sys/kernel/shmmax
If the total size of the shared memory created on the machine exceeds this limit, the following information may occur when the standard error perror is used in the program:
Unable to attach to shared memory
Solution:
1. Set shmmax
The default value of shmmax is 32 MB. Set the shmmax parameter to 2 GB using one of the following methods:
By directly changing the/proc file system, you can change the shmmax default settings without restarting the machine. I am using the following command to put it in the/etc/rc. Local Startup file:
#> Echo "2147483648">/proc/sys/kernel/shmmax
You can also use the sysctl command to change the shmmax value:
# Sysctl-W kernel. shmmax = 2147483648
Finally, by inserting this Kernel Parameter into the/etc/sysctl. conf Startup File, you can make this change permanently valid:
# Echo "kernel. shmmax = 2147483648">/etc/sysctl. conf

2. Set shmmni
Now let's look at the shmmni parameter. This Kernel Parameter is used to set the maximum number of shared memory segments within the system range. The default value of this parameter is 4096. This value is sufficient and does not need to be changed.
You can determine the value of shmmni by executing the following command:
# Cat/proc/sys/kernel/shmmni
4096

3. Set Shmall
Finally, let's look at the Shmall shared memory kernel parameters. This parameter controls the total amount of shared memory that the system can use at one time (in pages ). In short, the value of this parameter should always be at least:
Ceil (shmmax/page_size)
The default Shmall size is 2097152. You can use the following command to query the Shmall:
# Cat/proc/sys/kernel/Shmall
2097152
Shmall's default settings should be enough for us. Note: On the i386 platform, the page size of Red Hat Linux is 4096 bytes. However, you can use bigpages to configure a larger memory page size.
What are the problems with multiple shmat operations?
When a shared memory segment is created for the first time, it cannot be accessed by any process. To make the shared memory area accessible, you must use the shmat function to attach it to your own process space, so that the process is connected to the shared memory. This function is declared in Linux/SHM. h:
# Include <sys/types. h>
# Include <sys/SHM. h>

Void * shmat (INT shmid, const void * shmaddr, int shmflg );
The shmid parameter is the return value of shmget () and an identifier;
The shmflg parameter indicates the access permission. If it is set to 0, no access restriction is set. Several permissions are defined in <bits/SHM. h>:

# Define shm_rdonly 010000/* attach read-only else read-write */
# Define shm_rnd 020000/* Round attach address to shmlba */
# Define shm_remap 040000/* take-over region on attach */

If shm_rdonly is specified, the shared memory only has read permission.

The shmaddr parameter is an additional point of shared memory. Different values have different meanings:

Ø if it is null, the kernel selects an idle memory zone. If it is not empty, the return address depends on whether the caller specifies the shm_rnd value for the shmflg parameter. If it is not specified, the shared memory area is appended to the address specified by shmaddr; otherwise, the additional address is the address (shmlba, a common address) after the shmaddr round-down address ).

Ø generally, the shmaddr parameter is set to null.

After shmat () is successfully called, a pointer pointing to the shared memory area is returned. With this pointer, you can access the shared memory area. If the pointer fails,-1 is returned.

Shows the ing relationship:
Figure 1.1 shared memory ing

Shmaddr indicates the starting address of the block memory in the virtual memory space when the physical memory space is mapped to the virtual memory space of the process, because we generally do not know which addresses are not occupied in the process, it is difficult to specify the physical space memory to be mapped to the virtual memory address of the process. Generally, the kernel will specify the address itself:
Void PTR = shmat (shmid, null, 0 );

In this way, it is okay to mount a shared memory for one call, but a process can mount the same shared memory for multiple times. The physical memory points to the same block, if shmaddr is null, the linear address space returned each time is different. In addition, the reference count pointing to this shared memory will increase. That is, multiple linear spaces of a process point to the same physical address. In this way, if the linear address of the process that has previously attached this shared memory is not shmdt, that is, the applied linear address is not released, the virtual memory space of the process will be continuously consumed, it is very likely that the linear space of the process will be used up and the next shmat or other operations will fail.

Solution:

You can determine whether the shared memory pointer to be applied is null to identify whether the shared memory is mounted for the first time. If it is used for mounting, exit.

Void * PTR = NULL ;...
If (null! = PTR)

Return;

PTR = shmat (shmid, PTR, 0666 );

Appendix:
The shmat function maps the ID number shmid shared memory to the address space of the calling process. The mapped address is determined by the shmaddr and shmflg parameters. The criterion is:
(1) If the shmaddr parameter is null, the system automatically determines the first address of the shared memory link to the process space.
(2) If the shmaddr value is not null and the shmflg parameter does not specify the shm_rnd flag, the system uses the address shmaddr to link the shared memory.
(3) If the shmaddr parameter value is not null and the shmflg parameter specifies the shm_rnd flag, the system will align the address shmaddr and link it to the shared memory. The shm_rnd option indicates an integer alignment. The constant shmlba represents a multiple of the Low-boundary addresses. The formula "shmaddr-(shmaddr % shmlba) it means moving the shmaddr address to an integer multiple of the Low-boundary address.
L shmget creates shared memory. when the key is the same, what will happen?

Shmget () is used to create a shared memory area or access an existing shared memory area. This function is defined in the header file Linux/SHM. h. The prototype is as follows:

# Include <sys/IPC. h>
# Include <sys/SHM. h>

Int shmget (key_t key, size_t size, int shmflg );
The parameter key is the key value obtained from ftok;

The size parameter specifies the memory size in bytes;

The shmflg parameter is an operation flag. Its macros are defined as follows:

Ipc_create: When shmget is called, The system compares this value with the key in other shared memory areas. If the same key exists, it indicates that the shared memory area already exists, the identifier of the shared memory area is returned. Otherwise, a new shared memory area is created and its identifier is returned.

Ipc_excl: This macro must be used with ipc_create. Otherwise, it does not make sense. When the shmflg parameter is set to ipc_create | ipc_excl,-1 is returned if the memory zone already exists. The error code is eexist.

Note: When you create a new shared memory partition, the size value must be greater than 0. If you access an existing shared memory partition, set the size to 0.

Generally, when we create shared memory, we use shmget in a process to create shared memory,

Int shmid = shmget (Key, size, ipc_create | 0666 );

In another process, use shmget and the same key to obtain the created shared memory,

Int shmid = shmget (Key, size, ipc_create | 0666 );

If the key of the created process is the same as that of the mounted process and the corresponding size is different, Will shmget fail?

The size of the created shared memory can be adjusted, but the size of the created shared memory can only be adjusted to a smaller value.
For example, shm_id = shmget (Key, 4194304, ipc_creat );

A 4 m shared memory is created. If the shared memory is not deleted, we will use it again.

Shm_id = shmget (Key, 10485760, ipc_creat );

When you create a 10 m shared memory, the following error message is returned when you use the standard error output:

Shmget error: invalid argument

However, if we use

Shm_id = shmget (Key, 3145728, ipc_creat );

When creating a 3 m shared memory, no error message is output, but the size of the shared memory is changed to 3145728. This also indicates that when using the shared memory, the key is used as the unique identifier of the shared memory. The size of the shared memory cannot be different from that of the shared memory.

What problems will this cause?

When multiple processes can create shared memory, if the key is the same, and the size of the shared memory to be created by a process is smaller than that of another process, A process with large Shared Memory creates shared memory first, and a process with a small Shared Memory creates shared memory. A process with a small shared memory gets the shared memory of a large shared memory process, and modify the size and content of the shared memory, which may cause a large shared memory process to crash.

Solution:

Method 1:

When all the shared memory is created, use exclusive creation, that is, use ipc_excl Tag:

Shmget (Key, size, ipc_create | ipc_excl );

When the shared memory is mounted, use exclusive creation to determine whether the shared memory has been created. If not, handle the error.

Shmid = shmget (Key, size, ipc_create | ipc_excl );

If (-1! = Shmid)

{

Printf ("error ");
}

Shmid = shmget (Key, size, ipc_create );
Method 2:
Although they all want their programs to be able to pre-define a unique key value with other programs, they are not always possible, because your program cannot select a key value for a shared memory. Therefore, set the key to ipc_private. In this way, the operating system will ignore the key, create a new shared memory, specify a key value, and return the ipc id of the shared memory. The ID of the new Shared Memory identifier tells other processes that the shared memory can be created by sending child processes or writing files or pipelines, that is to say, this method does not use keys to create shared memory. The operating system guarantees uniqueness.

Does ftok always generate a unique key value?

An ID value must be specified for the system to establish IPC communication (such as message queue and shared memory. Generally, this ID value is obtained through the ftok function.

The ftok prototype is as follows:

Key_t ftok (char * pathname, int proj_id)

The name of the file you specified when pathname is used. proj_id is the sub-serial number.

In the general UNIX implementation, the index node number of the file is taken out, and the return value of key_t is obtained by adding the sub-number before it. For example, if the index node number of a specified file is 65538, it is converted into a hexadecimal value of 0x010002, and the value of proj_id you specified is 38, it is converted into a hexadecimal value of 0x26, then the final key_t return value is 0x26010002.

To query the node number of a file index, use LS-I.

However, after the reconstruction file is deleted, the index node number is allocated by the operating system according to the usage of the current file system. Therefore, the index node number obtained is different from the original one.

Based on the file (or directory) name specified by pathname and the number specified by the proj_id parameter, the ftok function generates a unique key value for the IPC object. In practice, it is easy to understand that, when the proj_id is the same, as long as the file (or directory) name remains the same, ftok can always return consistent key values. However, this understanding is not completely correct and may leave a hidden trap for application development. Because ftok implementation has this risk, that is, when multiple processes accessing the same shared memory call the ftok function successively, if the file (or directory) specified by pathname) if it is deleted and re-created, the file system will assign the new I node information to this file (or directory) with the same name, so the ftok called by these processes can return normally, however, the obtained key values cannot be the same. The possible consequence is that these processes originally intended to access the same shared memory object. However, due to their different key values, the shared memory to which the process actually points is no longer consistent; if these shared memories are created, no errors are reported during the entire application running process. However, data transmission through a shared memory object cannot be achieved.

Therefore, to ensure that the key_t value remains the same, either ensure that the ftok file is not deleted or that a fixed key_t value is specified without ftok.

If a file that generates the key_t value has been deleted, it is likely that the key_t value of the shared memory used by the file will conflict with the key_t value of another process, as shown in the following situation:

Process 1 uses file 1 to generate key10000 for ftok, and process 2 uses file 2 to generate key 11111 for ftok. At this time, if both process 1 and process 2 need to download the file, update the file content to the shared memory. In this case, process 1 and process 2 must first download the file, delete the previous shared memory, and then use ftok to generate a new key, this key is used to apply for a new shared memory to load new problems, but it may be that file 2 is large, download is slow, and file 1 is small, download is slow, because both file 1 and file 2 are modified, the node number occupied by file 1 may be occupied by file 2, at this time, if the key generated by ftok of the downloaded file 1 is 11111, it will conflict with the shared memory of process 2 that does not have the 11111 key yet, resulting in problems.

Solution:

Method 1:

When you use ftok to obtain the key of a downloaded file in a program that has file downloading operations, you need to avoid conflicts. For example, if the shared memory is obtained exclusively, then, add one to the key and then obtain the shared memory until there is no conflict.

Method 2:

Before downloading the file, perform a music video on the previous file and "Occupy" the node number of the file to prevent other shared memory keys from being obtained.

In addition:

We recommend that you do not use the ftok method to obtain the key when you create a process to notify other processes of the connection, but use the file or inter-process communication method to notify the process.

Shared Memory deletion traps?

When the process stops using the shared memory area, you must use the shmdt function to disconnect from the shared memory area. This function is declared in sys/SHM. H. Its prototype is as follows:

# Include <sys/types. h>
# Include <sys/SHM. h>
Int shmdt (const void * shmaddr );
The shmaddr parameter is the return value of the shmat function.
After the process is out of the shared memory area, the shm_nattch in the data structure shmid_ds will be reduced by 1. However, the shared block memory still exists. Only when the shm_attch value is 0, that is, no process can use the shared memory area. The shared memory area is deleted from the kernel. Generally, when a process is terminated, the shared memory zone attached to it is automatically detached.

We pass

Int shmctl (INT shmid, int cmd, struct shmid_ds * BUF );

To delete the existing shared memory.

The first parameter, shmid, is a token returned by shmget.

The second parameter, CMD, is the action to be executed. It has three values:

Command description
Ipc_stat sets the value associated with the shared memory for data reflection in the shmid_ds structure.
Ipc_set if the process has the corresponding permissions, set the value associated with the shared memory to the value provided in the shmid_ds data structure.
Ipc_rmid: deletes the shared memory segment.

The third parameter, Buf, is a pointer to a structure containing the shared memory mode and permissions. It can be deleted by default as 0.

If the shared memory has been disconnected from all the processes that access it, the system will immediately delete the identifier of the shared memory after calling the ipc_rmid sub-command and delete the shared memory zone, and all relevant data structures;
If another process is still connected to the shared memory, the shared memory will not be immediately deleted from the system but set to ipc_private after the ipc_rmid sub-command is called, and marked as "deleted" (you can see the Dest field using the IPCS command). The shared memory will not disappear from the system until all existing connections are disconnected.

It should be noted that once the shared memory is deleted through shmctl, the shared memory will no longer accept any new connections, even if it still exists in the system! Therefore, it is clear that it is safe to delete a new connection after the shared memory is deleted. Otherwise, if a new connection still occurs after the delete operation, these connections may fail!