Explore GRPC: Protocols

This is a creation in Article, where the information may have evolved or changed.

RC3 version for TIKV The most important function is to support the GRPC, it means that the back of everyone can be very convenient to use their favorite voice docking tikv.

GRPC is based on the HTTP/2 protocol, to understand deeply grpc, understand the HTTP/2 is necessary, here first briefly introduce HTTP/2 related knowledge, and then in the introduction Grpc is based on HTTP/2.

http/1.x

HTTP protocol can be considered at this stage of the Web above the most common protocol, in a long time ago, many applications are based on the http/1.x protocol, http/1.x protocol is a text protocol, readability is very good, but actually not efficient, the author has encountered several problems:

Parser

To parse a complete HTTP request, first we need to be able to read the HTTP header correctly. The HTTP header uses separate fields and separates them from \r\n the body \r\n\r\n . After parsing the header, we can get the body size from inside the header to content-length read the body.

This process is actually not efficient, because we need to read multiple times to parse a complete HTTP request, although there are many optimizations in the code implementation, such as:

• Read a chunk of data into buffer at a time to avoid multiple IO read

• Stream parsing in a direct-matching manner when reading \r\n

But the way it works is still going to cost a high-performance service. In fact, the main problem is that the http/1.x protocol is a text protocol, is for people to see, the machine is not friendly, if you want to machine-friendly, binary protocol is a better choice.

If you are interested in parsing http/1.x, you can study the next http-parser, a very efficient and compact C library, see a lot of frameworks are integrated with this library to deal with http/1.x.

Request/response

http/1.x Another problem is its interactive mode, one connection at a time can only ask a reply, that is, after the client sends the request, must wait until response, in order to continue to send the next request.

This set of mechanisms is very simple, but will cause the network connection utilization is not high. If you need to do a lot of interaction at the same time, the client needs to establish multiple connections with the server, but the establishment of the connection is also a cost, so for performance, usually these connections are long connections are kept alive, although for the server to deal with millions of connections is not much of a challenge, but ultimately inefficient.

Push

With http/1.x did push the classmate, probably know how painful, because http/1.x did not push mechanism. So there are usually two ways of doing this:

• The long polling method, which is to hang a connection directly to the server, waits for a period of time (for example, 1 minutes) and re-poll again if the server has returned or timed out.

• Web-socket, through the upgrade mechanism, turns this HTTP connection into a bare TCP, interacting in two directions.

Compared to Long polling, I still prefer to web-socket a bit, after all, more efficient, just web-socket behind the interaction is not the traditional meaning above the HTTP.

Hello HTTP/2

While the http/1.x agreement may still be the most widely used protocol in the Internet today, as the scale of WEB services continues to expand, http/1.x becomes increasingly stretched, and we urgently need another set of better protocols to build our services, so we have HTTP/2.

HTTP/2 is a binary protocol, which means it is almost 0 readable, but fortunately, we still have a lot of tools, such as Wireshark, that can parse it out.

Before you know HTTP/2, you need to know some common terms:

• Stream: A bidirectional stream in which a connection can have more than one streams.

• Message: The request,response above the logic.

• Frame:: The smallest unit of data transfer. Each Frame belongs to a specific stream or to the entire connection. A message may have more than one frame composed.

Frame Format

Frame is the smallest data transfer unit within HTTP/2, and a frame is defined as follows (copied directly from the official website):

+-----------------------------------------------+|                 Length (24)                   |+---------------+---------------+---------------+|   Type (8)    |   Flags (8)   |+-+-------------+---------------+-------------------------------+|R|                 Stream Identifier (31)                      |+=+=============================================================+|                   Frame Payload (0...)                      ...+---------------------------------------------------------------+

Length: That is frame, the default maximum length is 16KB, if you want to send a larger frame, you need to display the settings max frame size.
Types of type:frame, such as data,headers,priority.
Flag and R: Reserved bit, can be regardless of first.
Stream Identifier: Identifies the stream that belongs to, and if 0, indicates that the frame belongs to the entire connection.
Frame Payload: Different formats depending on the Type.

As you can see, the format definition of Frame is very simple, according to the official agreement, the pro can be very convenient to write one out.

Multiplexing

HTTP/2 supports the multiplexing of connections via stream, increasing the utilization of connections. Stream has many important features:

• A connection can contain multiple streams, and the data sent by multiple streams does not affect each other.

• Stream can be used unilaterally or shared by client and server.

• Stream can be closed by any segment.

• The Stream determines the order in which the frames are sent, and the other end is processed in the order in which they are received.

• Stream is identified by a unique ID.

Here is the Stream ID, if the client created Stream,id is odd, if it is created by the server, the ID is even. ID 0x00 and 0x01 have specific usage scenarios and are not used.

Stream ID cannot be reused, and if a connection has been assigned the above ID, the client creates a new connection. The server sends a Goaway frame to the client to force the client to create a new connection.

In order to increase a connection above the stream concurrency, you can consider the increase SETTINGS_MAX_CONCURRENT_STREAMS in tikv inside, we have encountered this value is smaller, the overall throughput on the problem.

It is also important to note that although a connection can handle more requests, a connection is far from enough. A connection usually has only one thread to handle, so it does not take full advantage of the server multicore. At the same time, each request codec or a cost, so with a connection or there will be bottlenecks.

In Tikv there is a version, we believe that a connection to run more streams this way no problem, let the client only with a connection and tikv interaction, the results found that performance is completely useless, not only the thread CPU to handle the connection is full, the overall performance will not go, Then we switched to multiple connections, and things got better.

Priority

Because a connection allows multiple streams to send a frame above, under some scenarios, we still want the stream to have a priority, which makes it easy to assign different resources to different requests on the end. For example, for a Web site, priority loading of important resources, and for some less important picture what, then use a low priority.

We can also set the Stream Dependencies to form a streams priority tree. Assuming that Stream A is parent,stream B and C are its children, B's weight is 4,c's weight is 12, assuming that A can now be assigned to all resources, then B can allocate only 1/3 of the resources in C.

Flow Control

HTTP/2 also supports flow control, and receiver may not want to process this data if the sender sends the data too fast, the receiver side may be too busy, too stressed, or just want to allocate resources to a particular stream. For example, if the client requests a video screen for the server, but at this point the user pauses to watch, the client may tell the server not to send the data.

Although TCP also has flow control, it is only effective for one connection. HTTP/2 there will be multiple streams on a connection, and sometimes we just want to control some streams, so HTTP/2 provides a flow control mechanism separately. Flow control has the following features:

• Flow control is one-way. Receiver can choose to set window size for stream or the entire connection.

• Flow control is trust-based. Receiver will simply give sender advice on its initial connection and stream's flow control window size.

• Flow control cannot be banned. When the HTTP/2 connection is established, the client and server Exchange SETTINGS frames, which is used to set the flow control window size.

• Flow control is hop-by-hop, not end-to-end, which means we can use a middleman to control the flow.

It is important to note that the default window size of HTTP/2 is 1 KB, the actual value is too small, and in tikv we set it directly to a minimum of.

HPACK

In an HTTP request, we usually carry a lot of meta-information on the header of the request to describe the resource to be transferred and its related properties. In the http/1.x era, we used plain text protocol, and use \r\n to separate, if we want to transfer a lot of metadata, it will lead to a very large header. In addition, most of the time, in a connection above the majority of requests, in fact, the header is not much, for example, our first request may GET /a.txt , followed by GET /b.txt , two requests the only difference is the URL path is not the same, but we still have all the other fields completely sent again.

HTTP/2 in order to result in this problem, the use of HPACK. Although HPACK's RFC documents look scary, the principle is very simple to understand.

HPACK provides a static and dynamic table, and static table defines common HTTP header fields, such as Method,path. When sending a request, the two sides know what field to send As long as the field is indexed in the static table.

For dynamic table, initialized to NULL, if the two sides of the interaction, found that there is a new field, added to the dynamic table above, so that the subsequent request can be like static table, just need to bring the relevant index on it.

Also, to reduce the size of the data transfer, use Huffman to encode. This is no longer a detailed explanation of how HPACK and Huffman are encoded.

Summary

The above is just a list of some of the characteristics of HTTP/2, and some, such as push, and different frame definitions are not mentioned, you are interested, you can refer to the HTTP/2 RFC document.

Hello GRPC

GRPC is Google based on HTTP/2 and Protobuf, to understand the GRPC protocol, just need to know how GRPC is transmitted on the HTTP/2.

Grpc typically have four modes, unary,client streaming,server streaming and bidirectional streaming, which are stream for the underlying HTTP/2 and are still a set of Request + response model.

Request

GRPC's request typically includes request-headers, 0 or more length-prefixed-message, and EOS.

Request-headers Direct use of the HTTP/2 Headers, in Headers and continuation frame inside the distribution. The headers defined are mainly call-definition and custom-metadata. Call-definition inside includes Method (in fact is used HTTP/2 POST), Content-type and so on. And Custom-metadata is the application-level customization of any key-value,key is not recommended to use the grpc- beginning, because this is reserved for GRPC follow-up.

Length-prefixed-message is mainly distributed in the DATA frame, it has a compressed flag to indicate whether the message is compressed, if it is 1, indicating that the message is compressed, and the compression is defined in the Heade R inside the message-encoding inside. Followed by a four-byte message length and the actual message.

EOS (End-of-stream) will take this flag in the final DATA frame END_STREAM . To indicate that the stream is not sending any data, it can be closed.

Response

Response mainly contains response-headers,0 or multiple length-prefixed-message and trailers. If an error is encountered, you can also return trailers-only directly.

Response-headers mainly includes Http-status,content-type, Custom-metadata and so on. Trailers-only also has http-status, content-type and trailers. The trailers includes Status and 0 or more custom-metadata.

Http-status is our usual HTTP 200,301,400, which is no longer explained in general. Status is the status of Grpc, and Status-message is the Message of GRPC. Status-message uses the Percent-encoded encoding method, the specific reference here.

If the last received HEADERS frame inside, with trailers, and have END_STREAM this flag, then it means response EOS.

Protobuf

The service interface of GRPC is defined based on protobuf, and we can associate the service with HTTP/2 very conveniently.

• Path:/Service-Name/{method name}

• Service-name:?( {proto package name} "." ) {service name}

• Message-type:{fully qualified proto message name}

• Content-type: "Application/grpc+proto"

Postscript

Above is just a simple understanding of the GRPC protocol, you can see that the cornerstone of GRPC is HTTP/2, and then use the PROTOBUF protocol above to define the service RPC. Although it may seem simple, it is very difficult to support GRPC if a language is not supported by HTTP/2,PROTOBUF.

The sad reminder is that Rust just doesn't have HTTP/2 support, and there's just one available PROTOBUF implementation. In order to support GRPC, we team to pay a great effort, but also took a lot of detours, from the original use of pure Rust RUST-GRPC project, to later C-GRPC encapsulated Grpc-rs, there are many can be said, behind the slow way. If you are interested in grpc and rust, you are welcome to participate in the development.

Grpc-rs:https://github.com/pingcap/grpc-rs

Author: Tang Liu