HTTPS and HTTP/2

Source: Internet
Author: User

http://geek.csdn.net/news/detail/188003

HTTPS protocol principle analysis of the problem that the HTTPS protocol needs to solve

When HTTPS is born as a security protocol, there are two major security issues that you will have to face:

    • Identity verification

      Ensure the authenticity of both sides of the communication. Straightforward, a want to communicate with B, a how to confirm that B's identity is not forged by C.

      (by C Forged B's identity with a communication, called Man-in-the-middle attack)

    • Communication encryption

      The confidentiality and integrity of the communication depend on the algorithm and the key, and how the communication parties choose the algorithm and the key.

Can solve the above two problems at the same time, can ensure that the real and effective communication between the two parties to take an effective algorithm and the key to communicate, then completed the original intent of the Protocol security.

Before we describe how the HTTPS protocol solves two security problems, we first understand several concepts.

    • Digital certificates

      A digital certificate is a digital file that identifies the identities of both parties in Internet communications and is issued by a CA.

    • Ca

      A CA (certification authority) is the issuing authority for digital certificates. As an authority, it approves the identity of the applicant and issue a digital certificate, so that we only need to verify the digital certificate to determine the real identity of the other party.

    • The relationship between HTTPS protocol, SSL protocol, TLS protocol, handshake Protocol

      HTTPS is an abbreviation for the hypertext Transfer Protocol over Secure Socket layer, which is HTTP over SSL, which can be understood as an SSL-based HTTP protocol. HTTPS protocol security is implemented by the SSL protocol (currently used in this article, based on the TLS 1.2 analysis).

      The SSL protocol is a kind of logging protocol, which has good extensibility and can easily add sub-protocol, and the Handshake protocol is a sub-protocol of SSL protocol.

      The TLS protocol is a later version of the SSL protocol, and the SSL protocol covered in this article defaults to the TLS protocol version 1.2.

The security of HTTPS protocol is implemented by SSL protocol, and the current TLS protocol 1.2 version contains four core sub-protocols: Handshake protocol, key Configuration switch protocol, Application data protocol and alarm protocol.

The core of resolving authentication and communication encryption is the handshake protocol, which is followed by a focus on the handshake protocol.

Handshake Protocol

Handshake protocol is the role of the two sides of the communication to identify, negotiate security connection parameters (encryption algorithm, key, etc.), to ensure that both identities are true and negotiated algorithms and keys to ensure communication security.

The introduction of handshake protocol is limited to client authentication, and one-way authentication is the most common authentication method of Internet company.

First, let's look at the protocol interaction, as shown in 1:


Figure 1 Handshake Protocol

Next, take the handshake protocol process of the Wireshark crawl interface as an example, for each protocol message analysis.

ClientHello message

The purpose of the ClientHello message is to send a set of parameters that the client can use to establish an encrypted channel, which is sent to the server at once.

The message content includes information such as the expected protocol version (TLS 1.2), the available cipher suite (Cipher Suites), the client random number, and the Extended field contents (Extension), as shown in 2.


Figure 2 ClientHello

Serverhello message

The purpose of the Serverhello message is to select the appropriate parameter in the ClientHello parameter collection and send the parameters that the server uses to establish the encrypted channel to the client.

Message content includes information such as the Protocol version taken (TLS 1.2), the cipher suite used (Cipher suite), the server-side random number (randomly), the session ID used to recover the session (session ID), and the extended field, as shown in 3.

自此客户端与服务端的协议版本、密码套件已经协商完毕。

The session ID issued by the server here can be used for subsequent recovery sessions. If the client carries a session ID in ClientHello and the server acknowledges it, both parties generate a new set of keys directly from the master key to continue the communication. Reduce the two network round trip to one network round trip, improve the efficiency of channel establishment.


Figure 3 Serverhello

Certificate message

The purpose of the certificate message is to send the server-side certificate details to the client for server-side authentication.

Message content: The certificate chain issued by the server, shown in 4.

Service side in order to ensure that the issued certificate can be correctly identified by the client, it is necessary to issue this certificate issued by the CA certificate, constitute a certificate chain, to ensure that the client can be based on the certificate chain of information in the system configuration to find the root certificate, and through the root certificate of the public key layer down to verify the legitimacy of the certificate.
, the 58 server issued two certificates: its own certificate and the issuing CA's certificate. The root certificate can be found directly by issuing the CA's certificate information.


Figure 4 Certificate

客户端本地校验服务端证书,若校验通过,则客户端对服务端的身份验证便完成了。

Certificate This phase solves the authentication problem at both ends. With the power of the CA, the certificate is issued through a CA and the work of authentication is given to the CA for processing.
We recognize the identity of the certificate holder as long as it is a CA that we recognize. Because of the CA's intervention, the problem of the man-in-the-middle attack is resolved because the middleman does not have a server certificate available for client authentication.

Serverkeyexchange message (may not be sent)

The purpose of the Serverkeyexchange message is to pass the additional parameters of the key exchange provided by the server to the client. Some algorithms do not require additional parameters, then Serverkeyexchange messages are not sent.

Message content: Additional parameters for key exchange, as shown in 5.


Figure 5 Serverkeyexchange

5, the service side issued the "EC Diffile-hellman" key exchange algorithm required parameters.

Serverhellodone message

The purpose of the Serverhellodone message is to notify the client that the Serverhello phase of the data has been sent, waiting for the client to next message.

Clientkeyexchange message

The purpose of the Clientkeyexchange message is to send the data that the client needs to provide for the key exchange to the server.

When we choose the RSA key exchange algorithm, the content of this message is the pre-master key for generating the master key that is encrypted by the certificate public key.

6, since the chosen key exchange algorithm is "EC Diffie-hellman", the Clientkeyexchange message sends the client parameters required by the "EC Diffie-hellman" algorithm.


Figure 6 Clientkeyexchange

When the Clientkeyexchange is sent, both ends have the full key data and the random number to generate the master key, and each end calculates the master key according to the selected algorithm.

至此,ClientKeyExchange发送后,两端均可生成主密钥,密钥交换问题便解决了。

Some readers may have some doubts about the use of random numbers, I think the addition of random numbers is to improve the randomness of the key.
Since the key generated directly by the client is likely not enough to be random, and the master key is seeded with the two random numbers provided on both sides, the key created is guaranteed to be closer to the real random keys.

Changecipherspec message

After the above six messages, we have solved the identity authentication problem, the cipher suite selection problem, the key exchange problem. Both parties have also generated six encryption keys that are actually used through the master key.

The purpose of the CHANGECIPHERSPEC message is to declare that subsequent messages are encrypted with a key. After this message, we will not see the clear text on the Wireshark.

Finished message

The function of the finished message is to calculate the summary of all messages in the handshake phase and send it to the other side for verification to avoid being tampered with by the middleman during the communication.

HTTPS protocol Summary

Since then, HTTPS how to ensure the security of communications, through the introduction of the handshake protocol, we have some understanding.

However, before we can fully use HTTPS, we also need to consider a well-known problem--https performance.

Compared with the HTTP protocol, the HTTPS protocol is more time-consuming to establish the data channel, and if deployed directly into the app, it will reduce the efficiency of data transmission and affect the user experience indirectly.

Next, the HTTPS Performance Saver--HTTP2 protocol is introduced.

Protocol Introduction to the newest-HTTP2 protocol

With the rapid development of the Internet, the http1.x protocol has developed rapidly, but when the app page contains dozens of requests, the limitations of the http1.x protocol are exposed:

    • Each request and response requires a separate link to make the request (the connection field solves some of the problems) and wastes resources.
    • Each request and response needs to add complete header information, and the application data transfer is less efficient.
    • By default, encryption is not performed, and data is easily monitored and tampered with during transmission.

HTTP2 was born to solve the problem that http1.x exposed.

Speaking of HTTP2 had to mention Spdy.
Due to http1.x exposure, Google has designed a new protocol called Spdy. Spdy introduces a new logic layer in the TCP layer and HTTP layer of the five-tier protocol stack to improve efficiency. The spdy is an intermediate layer that is well compatible with the HTTP layer and does not need to modify the HTTP layer to improve the speed of application data transfer.
Spdy through multiplexing technology, the client and server only need to maintain a link to multiple data interaction, improve communication efficiency.
And HTTP2 Pence was developed based on the idea of Spdy.
By introducing the concept of flow and frame, we inherit the multiplexing of spdy and add some practical features.

What are the characteristics of HTTP2? The HTTP2 features not only address the above-mentioned issues, but also some features that make the HTTP protocol more usable.

    • Multiplexing
    • Compression header information
    • Prioritize requests
    • Support for service-side push messages to clients

In addition, HTTP2 is currently used in the actual use, only for the HTTPS protocol scenario, through the handshake phase ClientHello and Serverhello extension field negotiation, so the current HTTP2 usage scenarios, are the default security encryption.

The following describes the HTTP2 protocol negotiation and multiplexing and compression header information two characteristics, the implementation of the partial use of Okhttp source (based on parent-3.4.2) for analysis and introduction.

Okhttp is currently the most widely used Android open source network library to support HTTP2, to Okhttp as an example of HTTP2 features can also be convenient for readers to learn about okhttp in advance to facilitate subsequent access okhttp.

Agreement negotiation

The negotiation of the HTTP2 agreement is carried out at the handshake stage.

The negotiation is done through the Handshake Protocol extension Extension field, and the new application Layer Protocol negotiation field is negotiated.

During the ClientHello phase of the handshake protocol, the client fills in the list of supported protocols into the application Layer Protocol negotiation field for selection by the server. 7 is shown below:


Figure 7 ALPN1

After the server receives the ClientHello message, select the appropriate protocol as the subsequent application layer protocol in the list of protocols supported by the client. 8 is shown below:


Figure 8 ALPN2

In this way, the HTTP2 protocol is negotiated at both ends.

When HTTP2 does not appear, Spdy also expands the Next_protocol_negotiation field by extending the field to negotiate Spdy with the NPN protocol. However, because the NPN protocol negotiation is too complex, the HTTPS protocol is more intrusive, after the emergence of ALPN negotiation agreement, it is gradually eliminated. Therefore, the agreement of this article is negotiated and introduced for the NPN protocol negotiation.

Multiplexing of protocol features

HTTP2 in order to optimize the waste of TCP performance of http1.x, the concept of multiplexing is proposed.

The meaning of multiplexing

In HTTP2, requests under the same domain name can be transmitted over the same TCP link, so that multiple requests do not have to establish links separately, saving the overhead of establishing a link.

To achieve this, HTTP2 presents the concept of flow and frame, the flow represents the request and the response, while the request and response specific data is wrapped as frames, the data transmitted in the link is separated by the stream ID and frame type processing. Figure 9 is a multiplexed abstraction, with each block representing one frame, and the same color block representing the same stream.


Figure 9 Http2_stream

So how is HTTP2 multiplexed implemented?

Due to the many scenarios of network requests, we chose one of the paths to describe:

    1. Client and server TCP channels in a domain name have been established
    2. A newly created client request is sent and responded to by a connected TCP channel
Multiplexing implementations

By default we have added each parameter to create the Request object R and created the call object C through the request object. And in the standalone thread, call the C.execute () method to perform a synchronous request operation.

After Okhttp invokes the Execute method, it is actually performed by a series of interceptor.

Interceptor is executed according to the order of addition, in which we are concerned with Retryandfollowupinterceptor, ConnectInterceptor0, Callserverinterceptor.

1. In Retryandfollowupinterceptor, Okhttp created a Streamallocation object for us, and Streamallocation contains an address object based on the URL creation.

The URL field of the address class differs from the URL field of the request class, and the URL field of the address class does not include the path and query fields, only the scheme and authority sections. This has played a significant role in the equal operation of connection multiplexing.

2. In Connectinterceptor, the address of the Streamallocation object is matched sequentially to the address of each connection object in the connection pool, and the connection that match the success and satisfy some conditions are reused. Creates a http2xstream based on a matching connection for subsequent read and write operations.

The address match in the connection pool is primarily through address url,url because it only contains scheme and authority so it can be used for domain name matching, which is the basis of okhttp multiplexing based on the domain name level.
In fact, the real stream read and write operation is framedconnection and framedstream,connection and Http2xstream is abstract in the specific operation of the class, in order to facilitate the use of the upper layer.

3. In Callserverinterceptor, Http2xstream creates framedstream for request sending and caches framedstream with corresponding streamid bindings so that when response arrives , which can be followed by the corresponding framedsteam according to the Streamid index.

After Framedstream sends the request, wait is called when the Readresponseheaders method is executed, and the current thread is suspended.
And when the framedconnection read thread receives the STREAMID message, the Framedstream is queried in the cache and the corresponding thread wakes up for response decoding.

The method of realizing the multiplexing of Okhttp is summarized:

    1. Select the available connection by matching the existing connection address in the connection pool with the requested address.
    2. Framedstream created through Http2xstream caches the Framedstream object and streamid mappings to framedconnection after the request is sent.
    3. After receiving the message, Framedconnection parses the frame information, selects the cached Framedstream in the map through the parsed Streamid, and wakes up the Framedstream for response processing.

In my opinion, HTTP2 is a custom protocol that is well-compatible with the HTTP protocol format, which distributes data to each request via stream and subdivides the request data in detail by frame.

Compression header information for the Protocol feature

HTTP2 in order to solve the problem that the head information is too large in http1.x, the solution is to compress the head information. The specific compression method, then introduced the Hpack.

The Hpack compression algorithm is specifically designed for HTTP2 header compression services. To achieve the purpose of compressing the header information, Hpack caches the header field as an index and represents the header field by the index ID. Client and Server Maintenance Index table, communication process as far as possible to use the index to communicate, after receiving the Index Query Index table, in order to resolve the real head information.

Hpack Index Table is divided into Dynamic Index table and static index table, dynamic Index table is the index table which is dynamically maintained during HTTP2 protocol communication, and Static Index table is the index table in hard-coded into protocol.

As the basis for analyzing Hpack compression header information, you need to first describe how the Hpack is represented in the index and the header string.

Index

The index is represented by an integer number, because the hpack needs to take into account the compression and codec issues, so the integer number structure is defined as 10:


Figure Ten Int_strut

    • Category identification

      Classification of Hpack categories by category identification, guidance of subsequent codec operations, common 1,01,01000000 and other eight categories.

    • First byte low integer type

      The first byte excludes the remaining bits of the category ID, which is used to represent the low-level integer. If the value is greater than the capacity that the remaining bits can represent, subsequent bytes are required to represent the high-level integer.

    • End identity

      Indicates whether this byte is an integer parse-terminated byte.

    • High-level integer

      The remaining 7bit of the byte is used to fill the integer high.

The end ID + high-order integer byte can be 0 or more, depending on the size of the data.
For example, if you want to indicate that the class is 1 and the index is 2, then 10000010 is used, and no extra bytes are required to increase the high-order integer.

Header string

The header string requires an explicit length, so the first byte of the data consists of type ID + data length. 11 is shown below:


Figure One String_strut

    • Type identification

      Whether choose Huffman Code, 1 for choice, 0 is not selected, okhttp by default does not choose Huffman code.

    • Data length

      Identifies the length of the data, represented by the integer notation mentioned above.

    • Data content

      Binary data.

Decoding instances

The following comprehensive okhttp source analysis Hpack decode header field process.

Readers interested in the coding section can refer to RFC 7541 or directly analyze Okhttp source code.

When we need to decode the header field, first parse the first byte of the header field (the first byte of the Hpack header field is divided into 8 categories, pick 3 of the category descriptions), and the first byte is used to guide the parsing rules for the current header field:

    • 1xxxxxxx

      The category ID is 1, which represents a header field that receives an index of K and V.

      K, V Value: The index value of the KV pair is obtained by parsing the Hpack integer, and the corresponding head field can be mapped according to the index value, and the compression efficiency is the highest.

    • 01xxxxxx

      The category ID is 01, which represents a header field that receives a K index, a V as the original field, and a dynamic index table that needs to be added.

      K Value: The value of the K-value index is obtained by parsing the Hpack integer, and the corresponding header field is mapped by the index value.

      V Value: The original field of the V value is obtained by parsing the Hpack string.

      You also need to insert the Dynamic Index table after you get the K and V values.

    • 01000000

      01000000 delegates receive a K, v are the original fields, and need to join the Dynamic Index Table header field.

      K, V value: Gets the K, v original field by parsing the Hpack string, and inserts the Dynamic Index table.

Also have not joined the Dynamic Index Table, adjust index table size and other categories, here does not expand, interested can see Okhttp source code implementation.

The core method of Okhttp parsing header information is implemented as follows:

void Readheaders () throws IOException {while (!source.exhausted ()) {int b = Source.readbyte () &0xFF;if (b = =0x80) {10000000Category ID is 1, but index is 0ThrowNew IOException ("Index = = 0"); }Elseif (b &0x80) = =0x80) {1NNNNNNNThe category is 1 and the integer index is parsed by Readindexedheader.Intindex = readInt (b, prefix_7_bits);Get the full header field by index Readindexedheader (Index-1); }Elseif (b = =0x40) {0100000001000000 for KV are the original field, parse the string to get the K value, v value, and insert dynamic table Readliteralheaderwithincrementalindexingnewname (); }Elseif (b &0x40) = =0x40) {01NNNNNN01xxxxxx means the K value is the index, the V value is the original string, the integer index and string are parsed, and inserted into the dynamic tableIntindex = readInt (b, prefix_6_bits); Readliteralheaderwithincrementalindexingindexedname (Index-1); }Elseif (b &0x20) = =0x20) {001NNNNNCategory 001, meaning update dynamic list capacity Maxdynamictablebytecount = ReadInt (b, prefix_5_bits);if (Maxdynamictablebytecount <0 | | Maxdynamictablebytecount > Headertablesizesetting) {throw new IOException ( "Invalid Dynamic Table Size Update "+ Maxdynamictablebytecount);} Adjustdynamictablebytecount (); } else if (b = = 0x10 | | b = = 0) {//000?0000-ignore never indexed bit. //this class represents KV as the original string, parsing the string in turn, and not inserting the parsed KV value into the dynamic table. Readliteralheaderwithoutindexingnewname (); } else {//000? Nnnn-ignore never indexed bit. //similar to the previous category, but the K value is index, the V value is the original string int  index = readInt (b, prefix_4_bits); Readliteralheaderwithoutindexingindexedname (index-1);}}}  

Compression effect

The head field of the K value "accept-encoding" and the V value "gzip, deflate" can be replaced by the index value 15 in HTTP2 to achieve the effect of the head field compression.

The "Accept-charset" header field is represented by 14 for the head K value, while the value values are written to the stream according to the Hpack rule encoding.

With Hpack, an app with less change in head field, each header field will be reduced to 4 bytes, the compression effect is very obvious.

HTTPS and HTTP/2

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.