Inter-process communication in Linux (3) Message Queue

Inter-process communication in Linux (3) Message Queue

Message Queue (also called Message Queue) can overcome some shortcomings of the early UNIX communication mechanism. As one of the early UNIX communication mechanisms, the amount of information that a signal can transmit is limited. Although POSIX 1003.1b has extended the real-time signal, it has greatly improved the amount of information transmitted, however, the signal communication method is more like the "instant" communication method. It requires the process that receives the signal to respond to the signal within a certain time range, therefore, the signal can only be meaningful within the life cycle of the receiving process. The information transmitted by the signal is similar to the process-persistent concept. See
Appendix 1; Pipelines, well-known pipelines, and well-known pipelines are typical continuous IPC with processes, and only passing non-formatted byte streams will undoubtedly cause inconvenience to application development, its buffer size is also limited.

A message queue is a linked list of messages. A message can be viewed as a record with a specific format and a specific priority. A process with write permission on a message queue can add new messages to it according to certain rules. A process with read permission on the Message Queue can read messages from the message queue. Message Queues continue with the kernel (see appendix 1 ).

Currently, there are two main types of Message Queues: POSIX message queues and System V message queues. System V message queues are widely used. Considering the portability of applications, newly developed applications should use POSIX message queues whenever possible.

In this series of topics (a deep understanding of Linux inter-process communication (IPC), there are two implementation versions for message queues, signal lights, and shared memory areas: POSIX and System V. The Linux kernel (kernel 2.4.18) supports POSIX traffic signals, POSIX shared memory areas, and POSIX message queues. However, for one of the mainstream Linux releases, redhad8.0 (kernel 2.4.18 ), there is no support for the POSIX inter-process communication API, but it should be time-based.

Therefore, this article mainly introduces the system V message queue and its corresponding APIs.If no statement is made, the System V message queue is used in the following discussion.

I. Basic concepts of Message Queue

1. System V Message Queue continues with the kernel. This message queue will be deleted only when the kernel restarts or is displayed as deleting a message queue. Therefore, the data structure (struct ipc_ids msg_ids) of the Message Queue recorded in the system is located in the kernel. All message queues in the system can find the access entry in the msg_ids structure.
2. A message queue is a linked list of messages. Each message queue has a queue header, which is described using the structure struct msg_queue (see
   Appendix 2 ). The queue header contains a large amount of information about the message queue, including the key value, user ID, group ID, and number of messages in the message queue, the ID of the read/write process of the message queue. You can access this information or set some information.
3. Describes how the kernel establishes a connection with message queue:
   Struct ipc_ids msg_ids is the global data structure of message queues recorded in the kernel, and struct msg_queue is the queue header of each message queue.

It can be seen that the global data structure struct ipc_ids msg_ids can access the first member of each Message Queue header: structkern_ipc_perm; each struct kern_ipc_perm can correspond to a message queue because there is a key_t member key in this structure, and the key uniquely identifies a message queue. The structure of kern_ipc_perm is as follows:

Struct kern_ipc_perm {// global data structure of the Message Queue recorded in the kernel msg_ids can access this structure; Key_t key; // the key value uniquely corresponds to a message queue. Uid_t uid; Gid_t GID; Uid_t cuid; Gid_t cgid; Mode_t mode; Unsigned long seq; }

 

Back to Top

Ii. Operate message queues

There are three types of Message Queue operations:

1. Open or create a message queue
The kernel persistence of Message Queue requires that each Message Queue corresponds to a unique key value within the system range. Therefore, you must obtain a Message Queue description, you only need to provide the key value of the message queue;

Note: The Message Queue description is generated by a unique key value within the system, and the key value can be seen as a path corresponding to the system.

2. read/write operations

Message read/write operations are very simple. for developers, each message is similar to the following data structure:

Struct msgbuf { Long mtype; Char mtext [1]; };

 

Mtype members represent the message type. An important basis for reading a message from a message queue is the message type. mtext is the message content, and the length is not necessarily set to 1. Therefore, for sending a message, you can first preset a msgbuf buffer and write the message type and content, and call the corresponding sending function. For reading a message, first assign such a msgbuf buffer zone, then, read the message into the buffer.

3. Get or set Message Queue attributes:

Message Queue information is basically stored in the message queue header. Therefore, you can allocate a structure similar to the Message Queue header (struct msqid_ds, see
Appendix 2) To return the attributes of the Message Queue. You can also set this data structure.

Message Queue API

1. file name to key value

# Include <sys/types. h> # Include <sys/IPC. h> Key_t ftok (char * pathname, char proj );

 

It returns a key value corresponding to the path pathname. This function does not directly operate on message queues, but calls IPC (msgget ,...) This function is often called before msgget () is used to obtain the Message Queue description. The typical Call code is:

Key = ftok (path_ptr, 'A '); Ipc_id = IPC (msgget, (INT) Key, flags, 0, null, 0 ); ...

 

2. Linux provides a unified user interface for Operating System V inter-process communication (message queue, signal light, and shared memory:
Int IPC(Unsigned intCall, IntFirst, Int
Second, IntThird, Void *PTR, Long
Th);

The first parameter specifies the operation method for the IPC object. There are four operations for the message queue: msgsnd, msgrcv, msgget, and msgctl, the first parameter represents a unique IPC object. The following describes four operations.

• Int IPC(Msgget, intFirst,IntSecond,
  IntThird,Void* PTR,LongTh );
  
  System V calls corresponding to this operation are: int msgget (key_t) First, second ).
• Int IPC(Msgctl, intFirst,IntSecond,
  IntThird,Void* PTR,LongTh)
  
  System V calls corresponding to this operation are: int msgctl (first, second, (struct msqid_ds *) PTR ).
• Int IPC(Msgsnd, intFirst,IntSecond,
  IntThird,Void* PTR,LongTh );
  
  System V calls corresponding to this operation are: int msgsnd (first, (struct msgbuf *) PTR, second, third ).
• Int IPC(Msgrcv, intFirst,IntSecond,
  IntThird,Void* PTR,LongTh );
  
  System V calls corresponding to this operation: int msgrcv (first, (struct msgbuf *) PTR, second, Th, third ),

 

Note: I do not advocate using the system to call IPC (), but prefer to use the System V or POSIX inter-process communication API. The reason is as follows:

• Although the system call provides a unified user interface, it is precisely because of this feature that its parameters cannot provide specific practical meanings (such as naming parameters with first and second ), to some extent, the development is inconvenient.
• As mentioned in the IPC manual, IPC () is unique in Linux. You should pay attention to the portability of programs when writing programs;
• The implementation of the system call is only to encapsulate the System v ipc function, without any efficiency advantages;
• The number of APIs of System V in IPC is small, and the format is concise.

 

3. System V Message Queue API
There are four System V Message Queue APIs, which must contain several header files:

# Include <sys/types. h> # Include <sys/IPC. h> # Include <sys/msg. h>

 

1) int msgget (key_t key, int msgflg)

The parameter key is a key value obtained by ftok, And the msgflg parameter is a flag. This call returns the Message Queue description corresponding to the key.

In either of the following cases, this call creates a new message queue:

• If no message queue corresponds to the key, and msgflg contains the ipc_creat flag;
• The key parameter is ipc_private;

 

The msgflg parameter can be set to the following: ipc_creat, ipc_excl, ipc_nowait, or the result.

Call return:The message queue description is returned. Otherwise,-1 is returned.

Note: Setting the parameter key to the constant ipc_private does not mean that other processes cannot access the message queue. It only means that a new message queue will be created.

2) int msgrcv (INT msqid, struct msgbuf * msgp, int msgsz, long msgtyp, int msgflg );

The system calls to read a message from the Message Queue represented by msgid, and store the message in the msgbuf structure pointed by msgp.

Msqid is the description word of the Message Queue. After a message is returned, it is stored in the address pointed to by msgp. msgsz specifies the length of the mtext member of msgbuf (that is, the length of the message content ), msgtyp is the type of message to be read by the request. The message reading Mark msgflg can be of the following common values or:

• Ipc_nowait if no message meets the condition, the call returns immediately. At this time, errno = enomsg
• If ipc_except and msgtyp> 0 are used together, messages of the first type not msgtyp in the queue are returned.
• Ipc_noerror if the content of the message in the queue that meets the condition is greater than the requested msgsz byte, the message is truncated and the truncation part is lost.

 

The msgrcv manual details the message type when different values are obtained (> 0; <0; = 0). The call will return the message in the message queue.

Msgrcv () has three conditions to remove blocking:

1. The message queue contains messages that meet the conditions;
2. Message Queue represented by msqid is deleted;
3. The process that calls msgrcv () is interrupted by the signal;

 

Call return:The actual number of bytes of the read message is returned. Otherwise,-1 is returned.

3) int msgsnd (INT msqid, struct msgbuf * msgp, int msgsz, int msgflg );

Send a message to the Message Queue represented by msgid. The message to be sent is stored in the msgbuf structure pointed to by msgp. The message size is specified by msgze.

For sending a message, the meaningful msgflg flag is ipc_nowait, indicating whether msgsnd waits when the message queue does not have enough space to accommodate the message to be sent. There are two conditions that cause msgsnd () wait:

• The sum of the current message size and the number of bytes in the current Message Queue exceeds the total capacity of the message queue;
• The number of messages (in bytes) in the current message queue is no less than the total capacity (in bytes) of the Message Queue. At this time, although the number of messages in the message queue is large, however, there is only one byte.

 

There are three conditions for msgsnd () to remove blocking:

1. The preceding two conditions are not met, that is, the message queue has a space to accommodate the message;
2. Message Queue represented by msqid is deleted;
3. The process that calls msgsnd () is interrupted by the signal;

 

Call return:0 is returned. Otherwise,-1 is returned.

4) int msgctl (INT msqid, int cmd, struct msqid_ds * BUF );

The system calls the CMD operation on the message queue identified by msqid. There are three kinds of CMD operations: ipc_stat, ipc_set, and ipc_rmid.

1. Ipc_stat: this command is used to obtain Message Queue information. The returned information is stored in the msqid structure pointed to by the Buf;
2. Ipc_set: this command is used to set the attributes of a message queue. The attributes to be set are stored in the msqid structure pointed to by the Buf. The attributes can be set to msg_perm.uid, msg_perm.gid, msg_perm.mode, and msg_qbytes, it also affects msg_ctime members.
3. Ipc_rmid: deletes the Message Queue identified by msqid;

 

Call return:0 is returned. Otherwise,-1 is returned.

Back to Top

Iii. Message Queue restrictions

The capacity of each Message Queue (the maximum number of bytes) is limited. The value varies depending on the system. The RedHat 8.0 limit is output in subsequent application instances. For more information, see
Appendix 3.

Another limit is the maximum number of messages that each Message Queue can accommodate: In redhad 8.0, this limit is restricted by the capacity of the Message Queue: the number of messages must be smaller than the capacity (in bytes) of the message queue ).

Note: These two restrictions apply to each message queue. The system limits the maximum number of message queues in the system and the maximum number of messages in the system. In general, the actual development process will not exceed this limit.

Back to Top

Iv. MQ Application Instances

The message queue application is relatively simple. The following instances basically cover all the operations on the message queue. At the same time, the program output results help to deepen the understanding of some rules and Message Queue restrictions mentioned above.

# Include <sys/types. h> # Include <sys/msg. h> # Include <unistd. h> Void msg_stat (INT, struct msqid_ds ); Main () { Int gflags, sflags, rflags; Key_t key; Int msgid; Int reval; Struct msgsbuf { Int mtype; Char mtext [1]; } Msg_sbuf; Struct msgmbuf { Int mtype; Char mtext [10]; } Msg_rbuf; Struct msqid_ds msg_ginfo, msg_sinfo; Char * msgpath = "/Unix/msgqueue "; Key = ftok (msgpath, 'A '); Gflags = ipc_creat | ipc_excl; Msgid = msgget (Key, gflags | 00666 ); If (msgid =-1) { Printf ("MSG create error \ n "); Return; } // After creating a message queue, the default attributes of the Output Message Queue Msg_stat (msgid, msg_ginfo ); Sflags = ipc_nowait; Msg_sbuf.mtype = 10; Msg_sbuf.mtext [0] = 'a '; Reval = msgsnd (msgid, & msg_sbuf, sizeof (msg_sbuf.mtext), sflags ); If (reval =-1) { Printf ("Message send error \ n "); } // Output Message Queue attributes after a message is sent Msg_stat (msgid, msg_ginfo ); Rflags = ipc_nowait | msg_noerror; Reval = msgrcv (msgid, & msg_rbuf, 4,10, rflags ); If (reval =-1) Printf ("read MSG error \ n "); Else Printf ("read from MSG queue % d bytes \ n", Reval ); // Output the Message Queue attributes after reading the message from the Message Queue Msg_stat (msgid, msg_ginfo ); Msg_sinfo.msg_perm.uid = 8; // just a try Msg_sinfo.msg_perm.gid = 8 ;// Msg_sinfo.msg_qbytes = 16388; // Verify that the superuser can change the default message queue msg_qbytes. // Note that the value is greater than the default value. Reval = msgctl (msgid, ipc_set, & msg_sinfo ); If (reval =-1) { Printf ("MSG set info error \ n "); Return; } Msg_stat (msgid, msg_ginfo ); // Verify the setting of Message Queue attributes Reval = msgctl (msgid, ipc_rmid, null); // Delete A Message Queue If (reval =-1) { Printf ("unlink MSG queue error \ n "); Return; } } Void msg_stat (INT msgid, struct msqid_ds msg_info) { Int reval; Sleep (1); // only for the convenience of subsequent output time Reval = msgctl (msgid, ipc_stat, & msg_info ); If (reval =-1) { Printf ("Get MSG info error \ n "); Return; } Printf ("\ n "); Printf ("current number of bytes on queue is % d \ n", msg_info.msg_cbytes ); Printf ("number of messages in queue is % d \ n", msg_info.msg_qnum ); Printf ("Max number of bytes on queue is % d \ n", msg_info.msg_qbytes ); // The capacity (number of bytes) of each message queue is limited to msgmnb, and the value varies with the system. When a new message queue is created, the default value of // msg_qbytes is msgmnb. Printf ("PID of last msgsnd is % d \ n", msg_info.msg_lspid ); Printf ("PID of last msgrcv is % d \ n", msg_info.msg_lrpid ); Printf ("Last msgsnd time is % s", ctime (& (msg_info.msg_stime ))); Printf ("Last msgrcv time is % s", ctime (& (msg_info.msg_rtime ))); Printf ("last change time is % s", ctime (& (msg_info.msg_ctime ))); Printf ("MSG uid is % d \ n", msg_info.msg_perm.uid ); Printf ("msg gid is % d \ n", msg_info.msg_perm.gid ); }

 

For program output results, see
Appendix 3.

 

Back to Top

Summary:

Message Queues are more flexible than pipelines and famous pipelines. First, they provide formatted byte streams, which helps reduce the workload of developers. Second, messages have different types, in practice, it can be used as a priority. These two points are incomparable to pipelines and famous pipelines. Similarly, message queues can be reused among several processes, regardless of whether these processes are related or not. This is very similar to the famous pipeline, but message queues continue with the kernel, compared with the famous Pipeline (with the process going on), the pipeline has more vitality and more application space.

Appendix 1:In reference [1], the definition of continuous IPC with process, kernel, and file system is given:

1. As the process continues: IPC persists until the last process that opens the IPC object closes the object. Such as pipelines and named pipelines;
2. Continuous with the kernel: the IPC continues until the kernel reboots or displays the deletion of this object. Such as message queues, signal lights, and shared memory;
3. As the file system continues: IPC continues until the object is displayed and deleted.

 

Appendix 2:
The msg_queue structure is used to describe the Message Queue header, which exists in the system space:

Struct msg_queue { Struct kern_ipc_perm q_perm; Time_t q_stime;/* Last msgsnd time */ Time_t q_rtime;/* Last msgrcv time */ Time_t q_ctime;/* last change time */ Unsigned long q_cbytes;/* current number of bytes on queue */ Unsigned long q_qnum;/* Number of messages in queue */ Unsigned long q_qbytes;/* max number of bytes on queue */ Pid_t q_lspid;/* PID of last msgsnd */ Pid_t q_lrpid;/* last receive PID */ Struct list_head q_messages; Struct list_head q_receivers; Struct list_head q_senders; };

 

The structure msqid_ds is used to set or return message queue information, which exists in the user space;

Struct msqid_ds { Struct ipc_perm msg_perm; Struct MSG * msg_first;/* First message on queue, unused */ Struct MSG * msg_last;/* last message in queue, unused */ _ Kernel_time_t msg_stime;/* Last msgsnd time */ _ Kernel_time_t msg_rtime;/* Last msgrcv time */ _ Kernel_time_t msg_ctime;/* last change time */ Unsigned long msg_lcbytes;/* reuse junk fields for 32 bit */ Unsigned long msg_lqbytes;/* ditto */ Unsigned short msg_cbytes;/* current number of bytes on queue */ Unsigned short msg_qnum;/* Number of messages in queue */ Unsigned short msg_qbytes;/* max number of bytes on queue */ _ Kernel_ipc_pid_t msg_lspid;/* PID of last msgsnd */ _ Kernel_ipc_pid_t msg_lrpid;/* last receive PID */ };

 

// We can see that the above two structures are very similar.

 

Appendix 3:Message Queue instance output result:

Current number of bytes on queue is 0 Number of messages in queue is 0 Max number of bytes on queue is 16384 PID of last msgsnd is 0 PID of last msgrcv is 0 Last msgsnd time is Thu Jan 1 08:00:00 1970 Last msgrcv time is Thu Jan 1 08:00:00 1970 Last change time is Sun Dec 29 18:28:20 2002 MSG uid is 0 Msg gid is 0 // Output when a new message queue is created above Current number of bytes on queue is 1 Number of messages in queue is 1 Max number of bytes on queue is 16384 PID of last msgsnd is 2510 PID of last msgrcv is 0 Last msgsnd time is Sun Dec 29 18:28:21 2002 Last msgrcv time is Thu Jan 1 08:00:00 1970 Last change time is Sun Dec 29 18:28:20 2002 MSG uid is 0 Msg gid is 0 Read from MSG queue 1 bytes // Number of bytes actually read Current number of bytes on queue is 0 Number of messages in queue is 0 Max number of bytes on queue is 16384 // Maximum capacity of each Message Queue (bytes) PID of last msgsnd is 2510 PID of last msgrcv is 2510 Last msgsnd time is Sun Dec 29 18:28:21 2002 Last msgrcv time is Sun Dec 29 18:28:22 2002 Last change time is Sun Dec 29 18:28:20 2002 MSG uid is 0 Msg gid is 0 Current number of bytes on queue is 0 Number of messages in queue is 0 Max number of bytes on queue is 16388 // you can see that the super user can modify the maximum capacity of the Message Queue PID of last msgsnd is 2510 PID of last msgrcv is 2510 // trace the process that operates the Message Queue Last msgsnd time is Sun Dec 29 18:28:21 2002 Last msgrcv time is Sun Dec 29 18:28:22 2002 Last change time is Sun Dec 29 18:28:23 2002 // msgctl () call affects msg_ctime MSG uid is 8 Msg gid is 8

 

References

• Second volume of UNIX Network Programming: inter-process communication, Author: W. Richard Steven S, Translator: Yang jizhang, Tsinghua University Press. Both POSIX and System V message queues are described, which is of great inspiration for program development in Linux.
• Linux kernel source code Scene Analysis (I), Mao decao, Hu Ximing, Zhejiang University Press, provides source code analysis related to system V message queue.
• Bytes.
• Msgget, msgsnd, msgrcv, msgctl Manual

About the author

Zheng yanxing, male, is now pursuing a doctorate degree in network direction from the computer College of the National Defense University. You can contact him by email mlinux@163.com.