Shared Memory for big data exchange between processes

　　 Introduction

Data exchange and sharing between processes is a very important and practical technology. The development and design of large and medium-sized software are mostly completed by the cooperation of many program designers. Generally, one program designer is only responsible for the development of one or more modules, these modules can be dynamic link libraries, applications, or other forms of program components. These independently developed program modules eventually need to run as a whole, that is, to form a system. During system operation, these modules often need to frequently exchange data and share data, it is very easy to implement data exchange between the dynamic link library and its main debugging application, however, it is difficult to exchange data between two applications or with other applications other than the main application. This topic is especially difficult to implement when the exchange data volume is too large and the exchange frequency is too frequent, it also proposes a way to achieve quick exchange of large data volumes through shared memory.

　　Comparison and Selection of communication methods

There are many ways to communicate between processes. Common methods include shared memory, named pipes, anonymous pipelines, and message sending, in addition, you can indirectly implement inter-process data communication tasks through the socket port, configuration file, and registry. These methods have their own advantages and disadvantages. Specifically, You can exclude the use of the configuration file and registry when performing fast data exchange between processes, because the use of pipelines and socket sockets requires support from NICs, you may not consider it. In this way, there are only two communication modes available: Shared Memory and message sending. Because of the large data volume, data cannot be directly carried to the receiver through message parameters during message communication, and can only be transmitted by address. When an application sends data to another application, a wm_copydata system message is sent. Therefore, you can insert a wm_copydata message to the Message Queue to copy data between processes.

When a wm_copydata message is used, the handle of the sending window is specified by the first message parameter, and the second message parameter is the pointer to the copydatastruct data structure related to the same data. The original structure declaration is as follows:

Typedef struct tagcopydatastruct { DWORD dwdata; DWORD cbdata; Pvoid lpdata; } Copydatastruct;

You only need to assign the first address of the data to be sent to lpdata and specify the length of the data block by cbdata. After a message is sent, the receiver program receives the data block in the response function of the wm_copydata message through the second parameter that comes in with the message. However, when wm_copydata is used, only sendmessage () function can be used to send messages, but postmessage () is not used. Although these two functions are very similar, they are used to send messages to the specified window, however, after the sendmessage () function sends a message, it does not return immediately. Instead, it returns only after the message response function of the receiver completes processing and returns the result. During this period, the sender program will be blocked, and the statements after sendmessage () cannot be executed. The postmessage () function returns a message immediately after it is sent, and then the statement can be executed immediately, but the message execution result cannot be obtained. It can be seen that the wm_copydata message sending method is not suitable for frequent and fast data exchange when the amount of data exchanged is large, data loss may occur when data transmission is too frequent.

Compared with the above communication methods between processes, shared memory has obvious advantages. The shared memory is implemented by directly operating the memory ing file, and the memory ing file is the lowest layer mechanism for standalone data sharing, the previous data exchange methods are implemented through memory ing files at the lower layer. Therefore, the use of shared memory can obtain high performance with a small overhead, which is the best solution for fast data exchange with large data volumes.

 Use of shared memory

 

In Windows, no process can read, write, or modify the data of another process (including variables, objects, and memory allocation ), however, the view of the file ing object created in a process can be mapped to multiple other processes. These processes share the same page of the physical memory. Therefore, when a process writes data to the view of the shared file ing object, other processes can immediately obtain data changes. To further improve the data exchange speed, you can also use memory ing files supported by system page files directly in the memory area, obviously, this shared memory method can fully meet the requirements of fast data transmission tasks between processes. The following describes how to allocate and access the application instance of the same shared memory block through the file ing object between two independent processes. In this example, the sender program is responsible for sending data to the receiver program. The file ing object is created and closed by the sender and a unique name is specified for the recipient. The recipient program directly opens the ing object of this file with the unique name and receives the data.

In the sender program, a memory ing file object is created by using the createfilemapping () function. if the object is created successfully, the mapviewoffile () function is used to map the view of the object mapped to the address space, the first address of the ing view is also obtained. It can be seen that shared memory is mainly created through these two functions. The original declaration of these two functions is as follows:

Handle createfilemapping (handle hfile, Lpsecurity_attributes lpfilemappingattributes, DWORD flprotect, DWORD dwmaximumsizehigh, DWORD dwmaximumsizelow, Lptstr lpname ); Lpvoid mapviewoffile (handle hfilemappingobject, DWORD dwdesiredaccess, DWORD dwfileoffsethigh, DWORD dwfileoffsetlow, DWORD dwnumberofbytestomap );

The createfilemapping () function parameter hfile specifies the file handle to be mapped to the process address space, if the handle is invalid, the system creates a file ing object that uses the page file instead of the specified disk file storage. Obviously, in this example, we need to set this parameter to invalid_handle_value for fast data exchange. The flprotect parameter sets the protection attribute that the system takes on the page. Because of the need for read/write operations, therefore, you can set the protection attribute page_readwrite. The double-font parameters dwmaximumsizehigh and dwmaximumsizelow specify the maximum number of bytes in the shared memory zone. The last parameter lpname is used to set a name for the shared memory, the recipient can open it with this name. The hfilemappingobject parameter of the mapviewoffile () function is the handle of the memory file image returned by createfilemapping (). The dwdesiredaccess parameter specifies the access method to its data again and must be the same as createfilemapping () the protection attributes set by the function match. The repeated setting of protection attributes ensures that the application can effectively control data protection attributes. The following describes some key generation for creating shared memory.

Hrecvmap = createfilemapping (invalid_handle_value, null, page_readwrite | sec_commit, 0, 1000000, "datamap "); If (hrecvmap! = NULL) { Lpdata = (lpbyte) mapviewoffile (hrecvmap, file_map_write, 0, 0, 0 ); If (lpdata = NULL) { Closehandle (hrecvmap ); Hrecvmap = NULL; } } // The View of the notification recipient's memory file ing object is enabled. Hwnd hrecv =: findwindow (null, decode_programm ); If (hrecv! = NULL) : Postmessage (hrecv, wm_map_open, 0, 0 );

Data transmission actually writes data from the sender to the shared memory, and then the receiver can immediately remove the data from it. Writing data from the sender program to the shared memory is relatively simple. You only need to use the memcpy () function to copy the data. The key is to promptly notify the recipient that the data has been written to the shared memory, and let it even take it away. The Message notification method is still used here. When data is written to the shared memory, the postmessage () function is used to send messages to the receiver program. The receiver completes Data Reading in the message response function:

// Copy data to shared memory Memcpy (lpdata, recvbuf, sizeof (recvbuf )); // Notify the recipient to receive data Hwnd hdecode =: findwindow (null, decode_programm ); If (hdecode! = NULL) : Postmessage (hdecode, wm_data_ready, (wparam) 0, (lparam) sizeof (recvbuf ));

When the data transmission is over and the program is about to exit, You Need To unmount the mapped memory file ing object view and release the resource. This part of work is mainly completed by functions such as unmapviewoffile () and closehandle:

Hwnd hdecode =: findwindow (null, decode_programm ); If (hdecode! = NULL) : Postmessage (hdecode, wm_map_close, 0, 0 ); If (lpdata! = NULL) { Unmapviewoffile (lpdata ); Lpdata = NULL; } If (hrecvmap! = NULL) { Closehandle (hrecvmap ); Hrecvmap = NULL; }

In the receiving program, after receiving the wm_map_open message sent and sent, the openfilemapping () function opens the file ing object specified by the name "datamap". If the execution is successful, continue to use the mapviewoffile () function to map the view of the ing object of this file to the address space of the receiving application and obtain its first address:

M_hreceivemap = openfilemapping (file_map_read, false, "datamap "); If (m_hreceivemap = NULL) Return; M_lpbreceivebuf = (lpbyte) mapviewoffile (m_hreceivemap, file_map_read, 0, 0 ); If (m_lpbreceivebuf = NULL) { Closehandle (m_hreceivemap ); M_hreceivemap = NULL; }

After the sender program writes data to the shared memory, the receiver receives the message wm_data_ready. In the response function, the receiver copies the data from the shared memory to the local cache for subsequent processing. Similar to the sending program, after receiving program data, you also need to use functions such as unmapviewoffile () and closehandle () to release opened resources such as file views:

// Receive data from the shared memory Memcpy (recvbuf, (char *) (m_lpbreceivebuf), (INT) lparam ); ...... // Release resources before exiting the program Unmapviewoffile (m_lpbreceivebuf ); M_lpbreceivebuf = NULL; Closehandle (m_hreceivemap ); M_hreceivemap = NULL;

　　Summary

It has been tested in practice that the use of shared data to process large amounts of data in fast exchange shows good performance, in terms of data reliability and other aspects is much higher than the way to send wm_copydata messages. This large-capacity, high-speed data sharing processing method has a good use effect in the design of high-speed data transmission communication software. The code described in this article is compiled by Microsoft Visual C ++ 2000 in Windows 6.0.