Chromium multi-process mechanism and chromium Mechanism

Chromium multi-process mechanism and chromium Mechanism

There are a lot of articles on Chromium multi-process analysis. This article attempts to explain Chromium multi-process mechanism in a simple way and parse the basic mechanism of IPC internal operation.

How Chromium ensures the performance of multiple processes

The core of a multi-process application is to solve the concurrency problem:ThreadAndIPC.

A multi-process interaction program is very similar to city traffic management. I would like to make an analogy:

Traffic Management Problems	Solution	Multi-process applications
Chelaile	Purchase restriction. Don't buy a car if you have something to do.	There are many transactions.
Narrow Path	Limit rows. Cars on the road can only be restricted before they can be expanded.	Process Communication load is limited.
Temporary accident	Shunting. Divide lanes with different functions to avoid mutual interference.	Blocking caused by task waiting
Traffic condition information is missing	Active monitoring, peak load on the road.	If you try to use IPC when it is unavailable, it will lead to waiting.

Chromium usesNon-blocking, multi-thread, and status listeningSolution! The following key technologies are used:

• Unix Domain Socket (IPC Mechanism Used in POSIX)
• Libevent (a lightweight event-driven network library used to listen to ports (file descriptors) in IPC ))
• ChannelProxy (provides a thread-safe mechanism for the Channel)
• Closure (thread Operation Mode)

Let's start from scratch!
Build the road first! Establish a channel

The core of writing a multi-process application under a single machine is to establish communication between processes, which can be called channel or pipe. The operating system provides such basic mechanisms as named pipe, shared memory, and socket.

Chromium is implemented using Unix Domain Socket in POSIX. Unix Domain Socket Provides Lightweight and stable local socket communication by reusing standard interfaces of network socket. The Unix Domain name is defined as PF_UNIX (Mac OS) or AF_UNIX When socket is used to identify as a single-host communication. The establishment of socket is not aimed at IP addresses, but by FD (file descriptor) in the file system ).

The socket API was originally designed for network communication, but later an IPC Mechanism was developed on the socket framework, namely the UNIX Domain Socket. Although the network socket can also be used for inter-process communication between the same host (through loopback address 127.0.0.1), the UNIX Domain Socket is more efficient for IPC: No network protocol stack is required, you do not need to package, unpack, calculate, and verify the checksum, maintain the serial number, and respond. Instead, you only need to copy application layer data from one process to another. Compared with TCP sockets, the transmission speed of UNIX domain sockets on the same host is twice that of the latter. This is because the IPC Mechanism is essentially reliable communication, and the network protocol is designed for unreliable communication. UNIX Domain Socket also provides two API interfaces, namely stream-oriented and data packet-oriented, similar to TCP and UDP. However, message-oriented UNIX Domain Socket is reliable, and messages are neither lost nor disordered.

The process of using UNIX Domain Socket is very similar to that of network socket. You must first call socket () to create a socket file descriptor. The address family is specified as AF_UNIX, And the type can be SOCK_DGRAM or SOCK_STREAM, the protocol parameter is still set to 0.

The most obvious difference between UNIX Domain Socket and network socket programming is that the address format is different. It is represented by the addr_un struct. The socket address of network programming is the IP address and port number, the UNIX Domain Socket address is the path of a socket-type file in the file system.

Chromium directly uses socketpair () API in POSIX to create connectedAnonymousPipeline. The logical structure is as follows:

We can see that the IPC Mechanism distinguishes the Server and Client. In fact, the anonymous pipeline created through socketpair () is full-duplex and does not actually distinguish the Server/Client.

The process of this IPC is completed in the main process, and other mechanisms need to be used to notify the sub-process. In Android Chrome, this operation is completed by passing the FD list. This will be explained later (you can findKPrimaryIPCChannelLearning ).

After the sub-process knows the socket FD on the Server end, it can connect and send hello message. After Authentication, it can start communication.

Communication Port Manager

When a port is opened, efficiency becomes critical. Generally, in this case, you may consider either polling or callback to process received messages in a timely manner. But it is not that simple.

Although the Unix Domain Socket does not go through the network stack, it cannot improve the performance. However, there are also load problems that need to be solved. It is necessary to use the callback mechanism, and more importantly, it faces the C10K problem:

C10K Problem

The biggest feature of the C10K problem is that the performance and connections of poorly designed programs and the correlation between machine performance are often non-linear. For example, if you have not considered the C10K problem, a classic select-based program can handle 1000 concurrent throughput on the old server, it often cannot handle 2000 of the throughput of concurrency on new servers at 2x performance.

Chromium uses the well-known third-party concurrent network library libevent to do this (In addition, ACE and adaptive communication environment ). The following describes its functions:

Libevent is a lightweight open-source high-performance network library with several notable highlights:

A. event-driven ),

B. High Performance and lightweight, focusing on the network, not as bloated as ACE

C. register the event priority

Basic socket programming is congested/synchronized. Each operation will be returned unless it has been completed or an error occurs. In this way, a thread or a separate process must be used to process each request, system resources cannot support a large number of requests (the so-called c10k problem), such as memory: by default, each thread needs to occupy 2 ~ 8 MB of stack space. Posix defines that asynchronous select system calls can be used, but it uses the round robin method to determine whether a fd is active, which is less efficient [O (n)]. There are multiple connections, it still persists. Therefore, various systems propose asynchronous/callback-based system calls, such as Linux epoll, BSD kqueue, and Windows IOCP. Because of the kernel support, you can use the O (1) Efficiency to find the active fd. Basically, libevent encapsulates these efficient IO and provides unified APIs to simplify development.

In short: "Don't wait! As long as it is ready, I will notify you! "Is a typical Hollywood law. This is equivalent to setting up a big butler on the communication port to improve the capability of IPC interaction.

The core of Libevent is to apply the event-driven Reactor mode commonly used to solve concurrency problems. Simply put, an internal loop is used to start and respond when an event is triggered, and then suspend when there is no event. it also needs to note that the Event Callback processing cannot do too many things to avoid congestion. the code in Chromium reflects the following interface implemented by the IPC Channel:

// Used with WatchFileDescriptor to asynchronously monitor the I/O readiness// of a file descriptor.class Watcher { public:  // Called from MessageLoop::Run when an FD can be read from/written to  // without blocking  virtual void OnFileCanReadWithoutBlocking(int fd) = 0;  virtual void OnFileCanWriteWithoutBlocking(int fd) = 0; protected:  virtual ~Watcher() {}};

For comparison between Reactor mode and another similar mode, read this article.

Special reception! Queuing, queuing...

The IPC channel has been established, but there are a large group of callers. Anyone wants to perform IPC communication at their own convenience, so there is a concurrency problem. The Channel itself only wants to do a good job of Channel management and is dedicated to external applications. Chromium introduces Channel Proxy.

From the perspective of the design pattern, Proxy serves as the Secretary of the Service to arrange the time and method for the visitor to meet the Channel. The same thread can wait in the queue. Sorry for different threads. Turn right when you go out and then come in. In short, the Channel only works on one thread, and the Channel Proxy is responsible for arranging the transactions to be processed to the specified thread.

bool ChannelProxy::Send(Message* message) {  context_->ipc_task_runner()->PostTask(      FROM_HERE,      base::Bind(&ChannelProxy::Context::OnSendMessage,                 context_, base::Passed(scoped_ptr<Message>(message))));  return true;}

Threads in Chromium are only responsible for establishing runways and executing transactions,It does not save the context information of a specific task.(Helps improve performance), and does not care about what transactions are being executed. The upper-layer business logic determines the operation to perform. On the main road, the processing time of a task is very valuable, especially on the UI thread. The thread where the Channel processes the task is located. In the case of multithreading, the following thread tasks request the Channel to send and process messages. The core of a thread is the implementation of closures.

 

Make threads more efficient-streaking threads

There are two main points to improve the concurrency of applications:

• Task response is short and fast. Do not have time-consuming transactions.
• Use as few locks as possible.
• Avoid frequent context switching.

The Reactor mode used in libevent also needs to achieve the same purpose. The first point is an agreement at the application layer, and the second layer is the guarantee of the thread mechanism.

When a traditional thread is created, it will specify an entry, which is usually already an entry to a specific function. There will be a loop in it to monitor the event and handle it accordingly, the loop body itself is clear and may even make some logical judgment. this is because it has mastered the context information of the current task.

Threads on Chromium can avoid such context switching. The thread itself does not store the task information, and the task is transparent to it. The thread is only responsible for calling the operation and can be regarded as a streaking state, there is no burden. tasks running on threads are implemented in a typical Command mode. It must solve two problems:

• When a task runs in a thread, the context information can be saved by itself.
• The parameters of a task (also a context) can be saved by the closure.

The closure has many attempts in C ++. Chromium particularly describes the design of tr1: bind. We will not explain the thread implementation of Chromium here.

 

Chromium Process Architecture

Logically, Chromium regards the process in which the UI is located as the main process and names it Browser. The process in which page rendering is located and other business processes are all subprocesses. Including Renderer and GPU.

In addition to initializing itself, the main process is also responsible for creating sub-processes and notifying the establishment of channel information (file descriptor ). The concept model is as follows:

For Chromium, Contents already represents a browser capability, on which it is an Embedder that implements browser services. However, different Embedder or processes on different platforms may have different options. For example, starting a browser main process may have different behaviors. (For example, you do not need to start a new process on Android, directly initialize BrowserMainRunner. In Linux, the main Process can be run in the form of Service Process .), The method for initializing a SandBox is different. Chromium is a combination of operations to start all processes for Embedder and the main process to start a new process (Embedder is responsible for starting the Browser process, and the main process then starts a new process, and start different sub-processes in ContentMainRunner according to the parameters .) :

* In a single-process model, the main process does not directly start the process. Instead, it creates a new thread (InProcessRendererThread) through CreateInProcessRendererThread (), which will also pass in the channel descriptor.

For more information, see the main process and Renderer process. The interfaces on both sides are two classes that have both (inherited) IPC: Sender and IPC: Listener functions: RenderProcessHost and RenderThread. That's right. RenderThread! RenderProcess also exists, which is only a logical representation of a process, with only a small part of code. In single-process mode, calling InProcessRendererThread to initialize RenderThread is used as an example. We can see that channel_id _ is passed in to RenderThread for processing.

void InProcessRendererThread::Init() {  render_process_.reset(new RenderProcessImpl());  new RenderThreadImpl(channel_id_);}

Leased Line and public line-message distribution

Messages are divided into two types: Broadcast and leased line. In Chromium, a page is located in different threads, and both ends of Browser and Renderer are identified by routed IDs. If you want to whisper, specify a routed id, which is called a Routed Message and a leased line. Another type of Message is broadcast without distinguishing identities. Such messages are Control messages.

RenderView (inherited from RenderWidget) is used to register your own code with IPC Channel in RenderWidget: DoInit ():

bool result = RenderThread::Get()->Send(create_widget_message);if (result) {  RenderThread::Get()->AddRoute(routing_id_, this);  ......}

In addition, a set of Filters will be added to the IPC Channel during Browser and Render Thread initialization for other functions.

 

References

• Unix Domain Socket IPC
• In-depth libevent source code analysis
• Adaptive Communication Environment
• Potential performance bottleneck in Linux TCP
• A Proposal to add a Polymorphic Function Object Wrapper to the Standard Library

• Analysis of WebKit2 multi-process mechanism