Common pitfalls and analysis for Linux shared memory usage

The so-called shared memory is the fastest available IPC form that allows multiple processes to access the same piece of memory space. is designed for inefficient operation of other communication mechanisms. It is often used in conjunction with other communication mechanisms, such as semaphores, to achieve synchronization and mutual exclusion between processes. Other processes can "connect" the same piece of shared memory to their own address space. All processes have access to the addresses in the shared memory. If a process writes data to this shared memory, the changes are instantly seen by other processes that have access to the same shared memory. The use of shared memory greatly reduces the amount of memory consumed during large-scale data processing, but there are many pitfalls in the use of shared memory, which can easily cause a program to crash if not noticed.

Exceed the size limit of shared memory?

On a Linux server, the overall size of shared memory is limited, and this size is defined by the SHMMAX parameter (in bytes), and you can determine the value of SHMMAX by executing the following command:

1. # Cat/proc/sys/kernel/shmmax

If the total size of shared memory created on the machine exceeds this limit, using standard error perror in your program may cause the following information to appear:

1. Unable to attach to shared memory

Workaround:

1. Set SHMMAX

The default value for SHMMAX is 32MB. Generally use one of the following methods to set the SHMMAX parameter to 2GB:

By changing the/proc file system directly, you can change the default settings for SHMMAX without restarting the machine. I use the following command in the/>etc/rc.local startup file:

1. # echo "2147483648" >/proc/sys/kernel/shmmax

You can also use the SYSCTL command to change the value of SHMMAX:

1. # sysctl-w kernel.shmmax=2147483648

Finally, you can make this change permanent by inserting the kernel parameter into the/etc/sysctl.conf startup file:

1. # echo "kernel.shmmax=2147483648" >  > /etc/sysctl.conf

2. Set Shmmni

Let's look at the Shmmni parameter. This kernel parameter is used to set the maximum number of shared memory segments within a system range. The default value for this parameter is 4096. This value is sufficient and usually does not need to be changed.

You can determine the value of the Shmmni by executing the following command:

1. # Cat/proc/sys/kernel/shmmni
2. 4096

3. Set Shmall

Finally, let's look at the Shmall shared memory kernel parameters. This parameter controls the total amount of shared memory (in pages) that the system can use at one time. In short, the value of this parameter should always be at least:

1. Ceil (Shmmax/page_size)

The default size of Shmall is 2097152 and can be queried using the following command:

1. # Cat/proc/sys/kernel/shmall
2. 2097152

The default setting for Shmall should be sufficient for us.

Note: The page size of Red Hat Linux on the i386 platform is 4096 bytes. However, you can use Bigpages, which supports configuring a larger memory page size.

What's wrong with multiple shmat?

When a shared memory segment is first created, it cannot be accessed by any process. In order for the shared memory area to be accessible, it must be appended (attach) to its own process space through the Shmat function, so that the process establishes a connection to the shared memory. The function is declared in linux/shm.h:

1. #include
2. #include
3. void *shmat (int shmid, const void *shmaddr, int shmflg);

The parameter shmid is the return value of Shmget (), which is an identifier;

The parameter SHMFLG is the access permission flag, and if 0, no restricted permission is set. Several permissions are defined in:

1. #define SHM_RDONLY 010000/* Attach read-only else read-write */
2. #define SHM_RND 020000/* Round attach address to Shmlba */
3. #define SHM_REMAP 040000/* Take-over region on Attach */

If Shm_rdonly is specified, then the shared memory area is read only.

The parameter shmaddr is the additional point of shared memory, and different values have different meanings:

If it is empty, the kernel chooses an idle memory area, and if not NULL, the return address depends on whether the caller specifies a shm_rnd value for the SHMFLG parameter, and if not specified, the shared memory area is appended to the address specified by SHMADDR, otherwise the address is shmaddr Round down the address of a low-end boundary address for shared memory (Shmlba, a common site).

? The parameter shmaddr is usually set to NULL.

Shmat () Returns a pointer to the shared memory area after a successful call, which is used to access the shared memory area, or 1 if it fails.

The mapping relationship is as follows:

Figure 1.1 Shared Memory Map

Where shmaddr represents the physical memory space mapped to the process of virtual memory space, the virtual memory space in the block memory of the start address, in use, because we generally do not know which address in the process is not occupied, so it is not good to specify the physical space of memory to map to the virtual memory address of this process, The kernel itself is typically assigned:

1. void ptr = shmat (shmid, null,0);

This mounts a shared memory if a call is not a problem, but a process can mount multiple shmat of the same shared memory, the physical memory points to the same block, and if SHMADDR is null, each return has a different linear address space. And the reference count that points to this shared memory increases. That is, the process multiple linear spaces point to the same physical address. Thus, if the linear address of the process that was previously mounted on the shared memory is not SHMDT, i.e. the requested linear address is not released, the virtual memory space of the process is consumed, and it is likely that the process's linear space will eventually be exhausted, causing the next Shmat or other operation to fail.

Workaround:

It is possible to identify whether the shared memory is being mounted for the first time by determining whether it is empty or not, or if it is used for mounting, if it is to be mounted.

1. void* ptr = NULL;
2. ...
3. if (NULL! = ptr)
4. Return
5. ptr = Shmat (shmid,ptr,0666);

Report:

The function Shmat maps the identification number Shmid shared memory to the address space of the calling process, and the mapped address is determined by the parameters shmaddr and SHMFLG, with the following guidelines:

(1) If the parameter shmaddr value is NULL, the system automatically determines that the shared memory is linked to the first address of the process space.

(2) If the parameter shmaddr value is not null and the parameter SHMFLG does not specify the SHM_RND flag, the system uses the address SHMADDR link to share the memory.

(3) If the parameter shmaddr value is not null and the parameter SHMFLG specifies the SHM_RND flag bit, the system aligns the address shmaddr to the linked shared memory. Where option shm_rnd means rounding, constant Shmlba represents a multiple of a low-boundary address, and the formula "shmaddr– (shmaddr% Shmlba)" means that the address shmaddr is moved to an integer multiple of the low-boundary address.

Shmget Create shared memory, when key is the same, what happens when it goes wrong?

Shmget () is used to create a shared memory area, or to access an existing shared memory area. The function is defined in the header file Linux/shm.h and is prototyped as follows:

1. #include
2. #include
3. int Shmget (key_t key, size_t size, int shmflg);

The parameter key is the key value obtained by Ftok ();

The parameter size specifies the amount of memory in bytes;

The parameter SHMFLG is the action flag bit, and some of its macros are defined as follows:

Ipc_create: When calling Shmget, the system compares this value to the key of the other shared memory area, if there is the same key, indicating that the shared memory area already exists, returns the identifier of the shared memory area, or creates a new shared memory area and returns its identifier.

IPC_EXCL: The macro must be used with ipc_create, otherwise it doesn't make sense. When SHMFLG take Ipc_create | IPC_EXCL, returns 1 if the memory area is found to be present, and the error code is eexist.

Note that when you create a new shared memory area, the value of size must be greater than 0, and if you are accessing an already existing memory share, set the size to 0.

In general, when we create shared memory, we use shmget in a process to create shared memory.

1. Int shmid = shmget (key, size, ipc_create|0666);

In another process, using shmget and the same key to obtain the shared memory that has been created,

1. Int shmid = shmget (key, size, ipc_create|0666);

If the creation process and the hook process key are the same, and the corresponding size is different, will shmget fail?

? The size of the shared memory you have created is adjustable, but the size of the shared memory you have created can only be small, not large

Such as:

1. shm_id = Shmget (key,4194304,ipc_creat);

Create a 4M size shared memory, if this shared memory is not deleted, we can use

1. shm_id = Shmget (key,10485760,ipc_creat);

To create a 10M-size shared memory, using the standard error output will have the following error message:

1. Shmget error:invalid argument

However, if we use:

1. shm_id = Shmget (key,3145728,ipc_creat);

To create a 3M-size shared memory, it does not output an error message, except that the shared memory size is modified to 3145728, which means that the shared memory is used as a unique identifier for shared memory, and the size of shared memory does not differentiate between shared memory.

What does this cause?

When multiple processes can create shared memory, if key is in the same situation, and one process needs to create a shared memory size that is smaller than the shared memory that is being created by another process, the shared memory is created by a process that creates shared memory, and the shared memory is created after a small process of shared memory. The small shared memory process gets the shared memory of the large shared memory process and modifies the size and content of its shared memory (note the following comment supplement), which can cause the large shared memory process to crash.

Workaround:

Method One:

When all of the shared memory is created, use exclusivity to create the IPC_EXCL tag:

1. Shmget (Key, size,ipc_create| IPC_EXCL);

In the case of shared memory hooking, use exclusive creation to determine if the shared memory has been created, if it has not yet been created, and if it has been created, it is hooked up:

1. Shmid = shmget (key, size,ipc_create| IPC_EXCL);
2. If ( -1! = shmid)
3. {
4. Printf ("error");
5. }
6. Shmid = shmget (key, size,ipc_create);

Method Two:

Although you want your program to be able to pre-contract a unique key value with other programs, it is not always possible, because your program cannot choose a key value for a shared memory. Therefore, the key is set to Ipc_private, so that the operating system ignores the key, establishes a new shared memory, specifies a key value, and returns this shared memory IPC identifier ID. The identifier ID of this new shared memory tells other processes to create shared memory by deriving a child process, or writing to a file or pipeline, which means that the method does not use key for creating shared memory, which is guaranteed to be unique by the operating system.

Does Ftok always produce a unique key value?

An ID value must be specified when the system establishes an IPC communication (such as Message Queuing, shared memory). Typically, this ID value is obtained through the Ftok function.

The Ftok prototype is as follows:

1. key_t Ftok (char * pathname, int proj_id)

Pathname When you specify the file name, PROJ_ID is the sub-sequence number.

In a typical UNIX implementation, the index node number of the file is taken out, preceded by a sub-ordinal to get the return value of key_t. If the index node number of the specified file is 65538, the conversion to 16 is 0x010002, and you specify a proj_id value of 38, converted to 16 0x26, then the final key_t return value is 0x26010002.

The method for querying the file index node number is: Ls-i

However, when the rebuild file is deleted, the index node number is assigned by the operating system according to the usage of the file system at that time, so the index node number is different because it is different from the original.

The Ftok function generates a unique key value for the IPC object, based on the file (or directory) name specified by pathname, and the number specified by the PROJ_ID parameter. In practice, it is easy to understand that, in the same situation as PROJ_ID, you can ensure that the Ftok returns a consistently consistent key value as long as the file (or directory) name is not changed. However, this understanding is not entirely correct, and it is possible to bury a very subtle trap in application development. Because the implementation of Ftok has the risk that in the time period when multiple processes accessing the same shared memory call the Ftok function successively, if the file (or directory) pathname specified is deleted and recreated, the file system assigns the new I-node information to the file (or directory) with the same name. As a result, the ftok called by these processes can return normally, but the resulting key values are not guaranteed to be the same. The possible consequence is that these processes originally intended to access an identical shared memory object, but because of their respective key values, the shared memory that the process points to is no longer consistent, and if the shared memory is created, no errors will be reported on the surface during the entire application run. However, the purpose of data transfer through a shared memory object cannot be achieved.

So if you want to make sure that the key_t value is not changed, either make sure that the Ftok file is not deleted, or do not Ftok, specify a fixed key_t value.

If a file that generates a key_t value has been deleted, it is likely that the shared memory key_t value you are using now will conflict with the key_t value of another process, as in the following scenario:

Process 1 Using a file Ftok generated key10000, process 2 using the file and the Ftok generated key 11111, at this time if process 1 and process 2 need to download the file, and the contents of the file update to the shared memory, the process 1 and 2 need to first file, and then delete the previous shared memory, Then use Ftok to generate a new key, and then use this key to apply for new shared memory to load the new problem, but it is possible to file 2 larger, slow download, and file 1 is relatively small, the download is slow, because the file 1 and the file 2 are modified, at this time file 1 occupies a file node number may be occupied before file 2, At this point, if the downloaded file 1 Ftok generates a key of 11111, then there is no 11111 this key process 2 of the shared memory conflict, causing problems.

Workaround:

Method One:

In the download file operation of the program, the download of the file using Ftok to obtain key, the need for conflict avoidance measures, such as exclusive access to shared memory, if unsuccessful, the key is added to the operation, and then to obtain shared memory, until there is no conflict.

Method Two:

Before downloading the file, the previous file for the MV, first "Occupy" The file node number, to prevent other shared memory requests when the key is obtained.

Other than that:

When the creation process notifies other processes to hook up, it is recommended not to use the Ftok method to get the key, but to use the file or interprocess communication to inform.

A trap for shared memory removal?

When the process ends using the shared memory area, the connection to the shared memory area is broken through the function Shmdt. The function is declared in sys/shm.h with the following prototype:

1. #include
2. #include
3. int SHMDT (const void *SHMADDR);

The parameter shmaddr is the return value of the Shmat function.

When the process is out of the shared memory area, the Shm_nattch in the data structure Shmid_ds will be reduced by 1. However, shared segment memory still exists, only after Shm_attch is 0, that is, there is no process to use the shared memory area, the shared memory area is deleted in the kernel. In general, when a process terminates, its attached shared memory area is automatically detached.

We pass:

1. int shmctl (int shmid, int cmd, struct shmid_ds *buf);

To delete the shared memory that already exists.

The first argument, Shmid, is the token returned by Shmget.

The second argument, CMD, is the action to be performed. He can have a value of three:

Command description

• Ipc_stat sets the value of the data in the SHMID_DS structure to reflect the values associated with shared memory.
• Ipc_set if the process has appropriate permissions, set the value associated with the shared memory to the value provided in the SHMID_DS data structure.
• Ipc_rmid Delete the shared memory segment.

The third parameter, buf, is a pointer to the structure that contains the shared memory mode and permissions, which can be deleted by default to 0.

If the shared memory has been disconnected from all the processes that accessed it, the IPC_RMID subcommand is immediately removed, and the shared memory identifier is removed and the shared memory area is deleted, along with any associated data structures;

If there is still another process remaining connected to the shared memory, after calling the IPC_RMID subcommand, the shared memory is not immediately removed from the system, but is set to the Ipc_private state and marked as "deleted" (the Dest field can be seen using the IPCS command) The shared memory will not eventually disappear from the system until all of the connections have been disconnected.

It should be stated that once the shared memory has been deleted through SHMCTL, the shared memory will no longer accept any new connections, even if it still exists in the system! Therefore, it is possible to know that it is not possible to have a new connection after the deletion of the shared memory, it is safe to perform the delete operation, or the connection will fail if a new connection still occurs after the delete operation!

The difference between SHMDT and Shmctl:

SHMDT is to detach the shared memory from the process space so that the shmid in the process is invalidated and cannot be used. But space is reserved.

and Shmctl (sid,ipc_rmid,0) is to delete the shared memory, completely unusable, free space.

Common pitfalls and analysis for Linux shared memory usage