[2] chrome inter-process communication

[2] chrome inter-process communication 1. the basic mode of chrome Process Communication is Interprocess Communication, which is called IPC (inter-process communication). In a few articles on chrome, there is an article about this. Chrome has three main processes: browser main processes, which we have always named as the boss, and render processes, which we have mentioned earlier, it is a plug-in process. Every plug-in is displayed in chrome in the form of a process. It will be mentioned later when it comes to plug-ins. Both the render process and the plug-in process maintain communication with the boss. The render process and the plug-in process also have connection paths, but multiple render processes or multiple plug-in processes are directly involved, there is no way to communicate with each other. It depends on the coordination of the boss.... Communication between processes requires relying on the features of the operating system. There is not much to play with. In Chrome Named Pipe)But it uses an IPC: channel class to encapsulate the specific implementation details. Channel can have two working modes: client and server. The server and client belong to two processes and maintain a common pipeline name. The server is responsible for creating the pipeline, the client tries to connect to the MPs queue and sends the data to the buffer zone of the MPs queue to read and write data (in chrome, the binary stream, asynchronous Io...) is used ...), complete communication...

Pipeline name negotiation In the socket, We will specify the communication port in advance. If we do not follow this port for access, the wrong door will be pushed out by a random stick. Similarly, if a famous pipeline is expected to walk between two processes, it needs to use an incoming password acceptable to both processes. This is the name of a famous Pipeline. In Chrome (windows...), the names of famous pipelines are in the format of //./pipe/chrome. Id. The ID must be unique, such as process ID, instance address, and random number. Generally, this ID is generated by a process (usually browser process). When another process is created, it is passed as a command line parameter to complete name negotiation... If you do not know and want to know more about the famous pipelines and semaphores in windows, we recommend that you read some professional books, for example, for the Bible-level windows core programming and in-depth analysis of Windows operating systems, you can also view the SDK. The APIS You Need To Know may include: createnamedpipe, createfile, connectnamedpipe, waitformultipleobjects, waitforsingleobject, setevent, and so on...

There are three key roles in the channel: Message: sender, channel: listener, and messageloopforio: Watcher. The channel itself is derived from sender and watcher and has two sides. The Listener is an abstract class, which is implemented by the users of the channel. As the name suggests, sender is the interface for sending messages, and listener is the specific implementation for processing received messages. But what is watcher? If you think Watcher Looks familiar, I will be excited with tears. That's right, before (first part of the first section ...) when talking about message loops, we can see from that table that I/O thread (Remember, in chrome, I/O refers to network I/O ,*_*) will process the registered watcher. Watcher is actually very simple. It can be regarded as a semaphore and a pair with an onobjectsignaled method object. When the message loop detects that the semaphore is enabled, it will call the corresponding onobjectsignaled method...

Figure 5 flowchart of Chrome's IPC Processing

As shown in the following illustration, the entire core IPC process of Chrome is shown in the figure, during which some logic such as error handling is planned. If you want to see the original, you can view the implementation of the channel class by yourself. When a message is sent to a channel of the sending process, the channel puts it in the sending message queue, if the message is still being sent (the sender is blocked ...), check whether the blocking function is removed (use a semaphore waiting for 0 seconds to wait for the function ...), serialize the content in the message queue and write it into the pipeline. The operating system maintains this set of semaphores in the pipeline in asynchronous mode. When a message is written from the sending process buffer to the receiving process buffer, the semaphores at the receiving end are activated. When the message loop of the receiving process follows the watcher check step and the current semaphore is activated, the corresponding onobjectsignaled method of the watcher will be called to notify the channel of the receiving process. A message is sent! The channel will try to receive byte and group messages from the pipeline and call listener to parse the message... It is not difficult to see from the above description that Chrome's Process Communication is the core feature, It is to use the message loop to check the semaphore, rather than directly blocking the pipeline on a semaphore.. In this way, it is closely linked with its multi-threaded model and uses a unified mode to solve the problem. In addition, the thread will not be congested because of the unified check of the message loop, and other work can be better processed. Theoretically, this is by increasing the CPU working time, in exchange for a better experience, there is a lot of capitalist dispatching...

Gentle message loop In fact, many of Chrome's message loops are not so overbearing, and will also be blocked in some semaphores or scenarios. After all, the client is not the server of its home, the CPU cannot be owned by all of them... For example, when I/O threads have no messages and no semaphores are activated, they will be blocked. For details, see the messagepumpforio waitforwork method... However, this type of blocking is centralized and can be modified at any time. Compared with the channel blocking on the semaphore, the shutdown time is shorter...

2. Cross-thread communication between processes and synchronous communication in chrome, any underlying data is thread-unsafe, and the channel is not too old (or Chinese football ?...), It is no exception. In each process, only one thread can be used to operate the channel. This thread is called an I/O thread (it is really a miserable thing if its name is not correct ...). If other threads attempt to overwrite the token, it will cause a big mess... But sometimes (in fact, most of the time...), we need to communicate with other processes from non-io threads. How can this be better? If you have read the thread model I wrote earlier, you can think of it as a simple practice. First, Put Channel operations in the task and put the task in the IO thread queue, let the IO thread handle it. Of course, this kind of thing happens too frequently, and every time it is complicated to do it, there is a proxy class called channelproxy to help you do it all... From the interface point of view, there is no major difference between the channelproxy interface and the channel (otherwise it will not be called the proxy ...), you can use channelproxy to send your messages, just like using a channel. channelproxy will help you complete the rest of the tasks. In addition, channelproxy is better than blue, and has done more things at this level, such: Send synchronous Message... However, the class that can send synchronous messages is not channelproxy, but its subclass, syncchannel. In the channel, all messages are asynchronous (in windows, also called overlapped...), and synchronization logic is not supported. To achieve synchronization, syncchannel does not create another wheel, but adds a wait operation to the channel layer. After the send operation of channelproxy is returned, syncchannel will block itself on a set of semaphores and wait until the packet is returned forever or times out.. In terms of appearance, there is no difference between synchronization and Asynchronization, but you should be careful when using it. using synchronous messages in the UI thread is easy to be merged... 3. the IPC Message format in Chrome has been mentioned for a long time, and another big part has not been mentioned, that is, the message package. If the overhead of data access in multi-thread mode comes from the lock, most of the overhead in multi-process mode comes from message disassembly and transmission between processes. Regardless of the mode, message packaging, serialization, deserialization, and group packages are inevitable as long as the process is different... In Chrome, communication messages between IPC are derived from the IPC: Message class. For messages, serialization and deserialization must be supported. The basic class pickle of messages is to do this. Pickle provides a set of interfaces that can accept input of int, Char, and other types of data. However, in pickle, everything is the same and converted into a binary stream. This binary stream is 32-bit, all-in-oneFor example, if you pass only one bool, it also occupies at least 32 bits. At the same time, the pickle stream has the auto-increment logic (that is, it will first open a buffer, if it is full, will double this buffer ...), so that it can be expanded infinitely. Pickle itself does not maintain any information in the binary stream logic. This task is handed over to the upper-level for processing (which will be mentioned later ...), however, pickle adds a header information for the binary stream, which stores the length of the stream. When the message inherits the pickle, the definition of this header is extended. The Complete Message format is as follows:

Figure 6 chrome IPC Message format

The yellow part is the packet header, with a fixed length of 96 bits, the green part is the package body, and the binary stream is indicated by payload_size. In terms of size, this package is very streamlined, except that the routing bit will be wasted when messages are not routed. Messages are transmitted by binary streams in a famous Pipeline. (A famous pipeline can transmit two types of pipeline streams: binary streams and message streams ...), therefore, payload_size + 96 bits can determine whether a complete package is received... Logically, IPC messages are divided into two categories: routed messages and control messages ). A message is a private and purposeful message. The system securely transmits the message to the destination based on the route information, so that it cannot be peeped. A message is a broadcast message, who want to hear it...

Message serialization Recently I read the source code of Google protocol buffers, Which is used on the server side and used as the standard, code generation tool and framework for internal machine communication protocols. Its main idea is to merge the key/value content into the binary, to help generate a more flexible and reliable binary protocol... In Chrome, This set is not used, but pure binary stream is used as the message serialization method. I think this is due to different application scenarios. On the server side, we are more concerned about protocol stability and scalability, and there are many protocol types involved. However, in chrome, the message format is very uniform, and there is no need for scalability and flexibility. In serialization, although the key/value method is very powerful, but what chrome needs is not flexibility but simplification, so it is better not to use the wheel created by Protocol buffers, but to set up another stove, and it takes a great effort to provide a pure binary message mechanism...

4. defining IPC messages if you have written an MFC program and have a lot of Macros in the MFC program, it is unfortunate that in the IPC message definition in chrome, you need to take a bit more effort, even more bitter; if you have learned how to use the special and special templates for traits, Use templates for function overloading, and use tuple for variable parameters during compilation, if you have any trouble with such a mechanism, we are also sorry. In Chrome, you need to feel it again... However, let's forget the macro and template to see what operations are needed for a message. A standard IPC message definition should be similar to the following:

Class somemessage: Public IPC: Message {public: Enum {id = ...;} somemessage (sometype & data): IPC: Message (msg_routing_control, ID, tostring (data )){...}...};

This is probably the case. You need to derive a subclass from message (or another subclass). This subclass has Unique id valueThis subclass accepts a parameter and you need Serialize this parameter. The two troubles are very clear. If a unique ID value is generated? How can I enable automatic serialization of any parameters more conveniently ?... In Chrome, the answer to these two questions is the macro + template. Chrome has an ID specification for each message, represented by a 16 bits value. The 4-bit high identifies a channel, and the 12-bit low identifies a subid of a message, that is, A maximum of 16 channels can exist between different processes, and 4 K messages can be defined on each channel. Currently, Chrome has used eight channels (IF process a and process B need two-way communication, in chrome, these are two different channels and different messages need to be defined, that is, A two-way process communication relationship that requires two channels ...), they already think that the 16bits ID format is not enough. In the future, it will be extended to 32bits. In this case, chrome defines the Message ID and uses an enumeration class to let it go from high to low, just like this:

Enum somechanic nel_msgtype {somechanic nelstart = 5 <12, somechanic nelprestart = (5 <12)-1, msg1, msg2, msg3,... msgn, somechanic nelend };

This is a message id Declaration for a channel of type 5. Because the first two values are specified, the values of the subsequent enumeration will decrease in sequence, so as long as the uniqueness of the channel type is maintained, the uniqueness of all message IDS can be maintained (of course, the premise is that the message limit cannot be exceeded ...). However, defining an ID is not enough. You also need to define a message subclass using this message id. This step is not only tedious, but most importantly, it violates the DIY principle. To add a message, you need to work in two places. It is intolerable, So Google sacrificed the macro atomic bomb, you need to define the message in the following format:

Ipc_begin_messages (pluginprocess, 3) ipc_message_control2 (outputs, INT/* process_id */, handle/* Renderer handle */) ipc_message_control1 (outputs, bool/* OK to shutdown */) ipc_message_control1 (pluginprocessmsg_pluginmessage, STD: vector <uint8>/* opaque data */) ipc_message_control0 (pluginprocessmsg_browsershutdown) ipc_end_messages (pluginprocess)

This is the macro that defines the pluginprocess message in chrome. I dug it and put it here. If you want to add a message, just add a message similar to ipc_message_control0, this indicates that it is a control message with 0 parameters. Basically, you can understand that ipc_begin_messages is equivalent to completing an enumeration start declaration, and each entry in the middle will add an ID in the enumeration and declare a subclass. This one macro eat, directly forced Beijing roast duck eat superb practices, you can refer to ipc_message_macros.h, or look at the following one macro eat an example...

Multiple macro expansion techniques This is a technique used in chrome to define a macro and expand multiple pieces of code. For the first time, see a similar example: First, define a macro. h with the macro defined in it: # UNDEF super_macro # If defined (first_time) # UNDEF first_time # Define super_macro (Label, type )/ Enum IDs {/ Label ##__ id = 10/ }; # Elif defined (second_time) # UNDEF second_time # Define super_macro (Label, type )/ Class testclass/ {/ Public :/ Enum {id = label #__ ID };/ Testclass (type value): _ value (value ){}/ Type _ value ;/ }; # Endif As you can see, this header file is reentrant. Each time UNDEF is removed from the previous definition, a new definition is determined. Then, you can create a use_macro.h file and use this macro to define the specific content: # Include "macros. H" Super_macro (test, INT) This header file does not need to be placed in the header file protection such as ifundef... define... To be reentrant. In the main function, you can define + include multiple times to expand multiple times: # Define first_time # Include "use_macro.h" # Define second_time # Include "use_macro.h" # Include <iostream> Int _ tmain (INT argc, _ tchar * argv []) { Testclass T (5 ); STD: cout <testclass: ID <STD: Endl; STD: cout <t. _ value <STD: Endl; Return 0; } In this way, you can successfully implement, define at a time, and generate multi-segment code...

In addition, after receiving a message, you also need to process the message. The message receiving function is the onmessagereceived function of the IPC: Channel: Listener subclass. In this function, a batch of macros will be placed. This macro will remind you of the message map mechanism of MFC:

Handler (renderprocesshost, MSG, msg_is_ OK) ipc_message_handler (viewhostmsg_pagecontents, onpagecontents) ipc_message_handler (handler, onupdatedcachestats) handler ()

This is very simple. After expansion, it can basically be regarded as a switch loop, judge the Message ID, and then pass the message to the corresponding function. Compared with the message map of MFC, there are fewer tasks to do... Macro methods can solve the problem of message class declarations and message distribution, but automatic serialization is not supported (the so-called automatic serialization is no matter what type of parameter you are, several parameters can be directly serialized without writing code ...). In C ++, the so-called automatic serialization, automatic type identification, and automatic XXX are usually implemented through templates. These so-called automation is actually achieved through a large amount of manual work beforehand and automatic recurrence of templates. net or Java's automatic serialization is a mountain track, which is the pride of the Porter. Although the last two legs are not moved to the top of the hill, it is a great difference to pay for the cost. For specific implementation skills, please refer to "STL source code analysis", "C ++ New Thinking", or "ipc_message_utils.h" in chrome. It is really not a sentence or two... In short, with macros and templates, you can easily declare a message, which can be passed into a variety of parameters (Here we use exaggerated rhetoric, in fact, as long as it is template automation, there will always be a limit. In the template Implementation of chrome, the number of parameters should not exceed 5, and the types must be basic types, STL containers, etc. In non-BT scenarios, it should be enough ...), you can call send methods such as channel, channelproxy, and syncchannel to send messages to other processes. In addition, you can implement a listener class and use message map to send messages to corresponding processing functions. In this way, the entire IPC system has been built...

Hard-working macros and templates Both macros and templates need to write a large amount of similar code to implement this mechanism. For example, to support 0 ~ For control messages with N parameters, you need to write n + 1 macro. To support serialization of various basic data structures, you need to write ten similar Write Functions and traits... The reason for such painstaking efforts is to make it as simple and convenient as possible for those who use these things, in line with the DIY principle. According to the responsibilities of the designers mentioned earlier, this is a typical act that has led to the suffering of a happy and responsible person. In Chrome, such code can be seen everywhere. I can see that tuple has been used for no less than three times (I have done a set before, And I have vomited blood directly ...), it's so awesome to be so conscientious...