Chromium multi-process mechanism analysis

There are many articles on chromium IPC multi-process analysis, this attempt to explain the chromium multi-process mechanism in an obvious way, the basic mechanism of internal operation of IPC is analyzed .

Chromium How to ensure multi-process performance

For a multi-process application, the core of the solution is the concurrency problem. Because of the underlying IPC mechanism, one of the paths further evolves into concurrent network communication problems, two polygons: threads/Processes and networks .

In chromium, the following key technologies are used:

• Unix Domain Socket (IPC mechanism used under POSIX)
• Libevent (lightweight event-driven network library for listening to ports (file descriptors) in IPC)
• Channelproxy (a thread-safe mechanism for the channel)
• Closures (How threads work)

Get the road! Establish a channel

Writing a single machine under the core of multi-process applications is to establish the way of communication between processes, which can be called channel or pipe. The operating system provides such a basic mechanism, including: Named pipe, shared memory, socket.

Chromium is implemented using UNIX Domain sockets under POSIX. Unix Domain sockets provide a lightweight, stable native socket communication by reusing the standard interface of a network socket. Unix domain naming is derived from the use of the domain parameter defined by the socket as: Pf_unix (Mac OS) or Af_unix, which is used to identify a single-machine communication. The socket is not built with an IP address, but is the FD (file descriptor) in the file system.

The socket API was originally designed for network communication, but later developed an IPC mechanism on the socket's framework, UNIX Domain socket. Although the network socket can also be used for interprocess communication with the same host (via loopback address 127.0.0.1), the UNIX Domain socket is more efficient for IPC: it does not need to go through the network protocol stack, does not need to package unpacking, calculate checksum, maintain serial number and answer, etc. , just copy the application layer data from one process to another. UNIX domain sockets compare to TCP sockets, where the same host is twice times faster than the latter. This is because the IPC mechanism is inherently reliable communication, and network protocols are designed for unreliable communications. UNIX domain sockets also provide both stream-oriented and packet-oriented API interfaces, similar to TCP and UDP, but the message-oriented UNIX domain sockets are also reliable, and messages are neither lost nor sequenced.

The process of using the UNIX Domain socket is very similar to the network socket, and the socket () is called to create a socket file descriptor, and address family is specified as Af_unix,type to select Sock_ The Dgram or Sock_stream,protocol parameter is still specified as 0.

The most obvious difference between the UNIX domain socket and the network socket programming is that the address format is different, the structure of the Sockaddr_un, the network programming socket address is the IP address plus port number, and Unix Domain The address of the socket is the path of a socket type file in the file system.

Chromium uses the Socketpair () API directly under POSIX to create an already connected anonymous pipeline with the following logical structure:

You can see that the IPC mechanism distinguishes between the server and the Client, in fact the anonymous pipeline created through Socketpair () is full-duplex and does not actually differentiate between server/client.

This IPC-established process is done in the main process and requires the use of other mechanisms to inform the child process. Under Android Chrome, complete this operation by passing the FD list. This will be explained later (can find kprimaryipcchannel learning).

When the child process knows the server side of the socket FD, it can connect, send a hello message, authentication can start the communication.

the butler of the communication port

When the port is open, efficiency becomes the key. Generally, this is usually considered in order to handle receiving messages in a timely manner, either polling or implementing callbacks. But it's not that simple.

Although Unix Domain sockets do not go the network stack has not improved performance. But there is a problem with the load that needs to be addressed. It is necessary to use callback mechanisms, and more importantly, to face c10k problems:

c10k problem

The most important feature of the c10k problem is that the design of the program is not good enough, the relationship between the performance and the connection number and machine performance is often non-linear. For example: If the c10k problem is not considered, a classic select-based program can handle 1000 concurrent throughput on the old server, and it will often not handle the throughput of concurrency 2000 on a new server with twice times performance.

Chromium uses the famous third-party network library: Libevent to do the work (plus ACE, Adaptive Communication Environment ). Here is an introduction to its features:

Libevent is a lightweight, open-source, high-performance network library. There are several notable highlights:

A. Event-driven (Event-driven),

B. High-performance lightweight, focus on the network, not as big as Ace

C. Registering event priorities

Basic socket programming is blocking/synchronizing, each operation is returned unless it is completed or an error occurs, so that for each request to be processed using a thread or a separate process, the system resources cannot support a large number of requests (so-called c10k problem), For example, Memory: By default, each thread needs to occupy 2~8m of stack space. POSIX defines a select system call that can be used asynchronously, but because it uses polling to determine whether an FD becomes active, it is inefficient [O (n)], and the number of connections is not enough. Therefore, each system proposes a system call based on asynchronous/callback, such as the IOCP of Linux Epoll,bsd kqueue,windows. Because of the support at the kernel level, it is possible to find active FD with the efficiency of O (1). Basically, Libevent is the encapsulation of these high-efficiency IO, which provides a unified API and simplifies development.

The equivalent of a large butler on the communication port, the ability to enhance the IPC interaction.

The core of Libevent is the event-driven reactor model that is used to solve concurrency problems. In simple terms, it is through an internal loop that starts and responds when the event is triggered, and hangs when there is no event. It also needs to be aware that the handling of event callbacks cannot do too much to avoid congestion. The code in chromium embodies the IPC channel implementation of the following interfaces:

Used with Watchfiledescriptor to asynchronously monitor the I/O readiness//of a file Descriptor.class watcher {public :  //called from Messageloop::run if an FD can is read From/written to  //without blocking  virtual void OnFi lecanreadwithoutblocking (int fd) = 0;  virtual void onfilecanwritewithoutblocking (int fd) = 0; Protected:  virtual ~watcher () {}};

For a comparison of the reactor model and another similar pattern, you can read this article .

Special Reception! line up, line up ...

The IPC channel has been established, but there are a large group of callers. Anyone wants to communicate with IPC at their own convenience, so there is a concurrency problem. Channel itself only want to do a good job in the management of channels, the external. Chromium introduced the channel Proxy for this purpose.

Proxy from the design mode, the function is the service secretary, arrange visitors and channel to meet the time and manner. The same thread can be queued, different threads, sorry, go out and turn right, and then come in. In short, the channel works on only one thread, and the channel Proxy is responsible for arranging the transaction to be processed onto the specified thread.

BOOL Channelproxy::send (message* Message) {  context_->ipc_task_runner ()->posttask (      from_here,      Base::bind (&channelproxy::context::onsendmessage,                 context_, Base::P (scoped_ptr<message> ( message)));  return true;}

threads in chromium are only responsible for building runways, performing transactions, It does not save contextual information for a particular task (to help improve performance), nor does it care what transaction is performed. What is done is entirely up to the business logic of the upper layer. On the main road, the processing time of the task is very valuable and is specific on the UI thread. The process specifies the thread where the channel processing task is located, and the multithreading situation requests the channel to send and process messages in the form of a threaded task. The core of the thread is the implementation of the closure.

make threads more efficient-bare-Ben Threads

Increase the concurrency of your application with two key points:

• Task response is short and fast. Don't have time-consuming transactions.
• Avoid frequent context switches.

The reactor pattern applied in the libevent mentioned above is also intended to achieve the same purpose. The 1th is the application layer of a convention, the second layer is the threading mechanism to protect.

Traditional threads at the time of creation will specify a portal, often is a specific function of the entrance, there will be a loop, the event to be monitored, as well as the corresponding processing, the loop body itself is clear, and can even do some logic judgment work. This is because it holds contextual information about the current task.

The thread on the chromium can avoid this kind of context switch, the thread itself does not save the information of the task, the task is transparent to it, the thread is only responsible for invoking its operation, can be regarded as a bare-Ben state, without any burden. The task running on the thread is implemented in a typical command pattern. It must solve two problems:

• The task itself runs on a thread, and it can save contextual information by itself.
• The parameters of the task (also a context), you can also save it by closing the package.

Closures in C + + have a lot of attempts, chromium in particular, it also reference the design of Tr1::bind . About the threading implementation of chromium here is not much to explain.

Chromium Process Architecture

Logically, chromium treats the process in which the UI resides as the main process, named Browser, the process in which the page is rendered, and other business processes, all of which are child processes. Including renderer, GPU, etc.

The main process itself, in addition to initializing itself, is responsible for creating child processes, as well as informing the establishment of channel information (file descriptors). The conceptual model is as follows:

For chromium, contents already represents a browser capability, which is the embedder of implementing browser business. However, different embedder or different platform processes may have different choices, such as starting a browsing process with different behavior (such as on Android, there is no need to start a new process, the direct initial browsermainrunner. Linux, however, can run the main process in the form of service process. ), the way to initialize the sandbox is different. Chromium is to centralize all the initiated operations of the process for Embedder and master processes to start the new process (Embedder is responsible for initiating the browser process, the master process starts the new process, and starts the different sub-processes in Contentmainrunner according to the parameters. ):

* In a single-process model, the master process does not start the subprocess directly, but instead creates a new thread (Inprocessrendererthread) through Createinprocessrendererthread (), and also passes in the channel descriptor.

Looking further at the main process and the renderer process, the two sides are responsible for the interface, then there are two classes (inherited) Ipc::sender and Ipc::listener functions: Renderprocesshost and Renderthread. Yes, that's renderthread!. Renderprocess also exists, just a logical representation of a process, with only a small fraction of the code. Using Inprocessrendererthread to initialize Renderthread as a single-process mode, you can see that channel_id_ is processed in the Renderthread.

void Inprocessrendererthread::init () {  render_process_.reset (new Renderprocessimpl ());  New Renderthreadimpl (CHANNEL_ID_);}

Line and public lines-distribution of messages

When distributing the message, it is divided into two ways: broadcast and leased line. In chromium, a page is identified to each other on different threads, browser, and renderer at both ends of the routed ID. If you want to whisper, specify a routed ID, which is called routed message, which is a leased line. Another type of message is broadcast, not identity-sensitive, and this type of message is control message.

Renderview (inherited from Renderwidget) is used to register its own code with the IPC channel at Renderwidget::D oinit ():

BOOL result = Renderthread::get ()->send (create_widget_message); if (result) {  renderthread::get () Addroute (routing_id_, this);  ......}

In addition, when the browser and render thread are initialized, a set of filters is added to the IPC channel for other functions to be used.

References

• Unix Domain Socket IPC
• Libevent Source Depth Analysis
• Adaptive Communication Environment
• Potential performance bottleneck in Linux TCP
• A proposal to add a polymorphic Function Object Wrapper to the standard Library

• Analysis of WEBKIT2 multi-process mechanism

Chromium multi-process mechanism analysis