HTTPS protocol detailed (iv): TLS/SSL handshake process

1. Handshake and key negotiation process

Client Authentication server based on RSA handshake and key exchange for an example TLS/SSL handshake process

Look at a hand-drawn timing diagram.

(1). Client_hello
    Client initiates a request to transmit the request information in clear text, including the version information, the Cipher Suite candidate list, the compression algorithm candidate list, the random number, the extended field information, The relevant information is as follows:
    ?  supported maximum TSL protocol version versions, from low to high SSLv2 SSLv3 TLSv1 TLSv1.1 TLSv1.2, currently basically no longer using versions below TLSV1 ;
    ?  Client-supported cryptographic Suites cipher Suites list, each cipher suite corresponds to the combination of four functions in the previous TLS principle: the authentication algorithm Au (authentication), the key exchange algorithm Keyexchange ( Key negotiation), symmetric encryption algorithm ENC (information encryption) and Information digest Mac (integrity check);
    ?  supported compression algorithm compression methods list for subsequent information compression transmission;
    ?  random number random_c, which is used for subsequent key generation;
    ?  Extended field extensions, support protocol and algorithm parameters and other ancillary information, etc., the common SNI is an extended field, followed by a separate discussion of the role of the field.
(2). Server_hello+server_certificate+sever_hello_done
     server_hello, The server returns the negotiated information results, including the option to use the Protocol versions version, the selected cryptographic Suite cipher suite, the selected compression algorithm compression method, the random number random_s, etc., where the random number is used for subsequent key negotiation;
    ? server_certificates, the server-side configuration of the corresponding certificate chain, for authentication and key exchange;
    ? server_hello_done, notifies the client Server_hello the end of the message sent;

(3). Certificate Verification
The client verifies the legality of the certificate, and if the validation passes for subsequent communication, otherwise prompts and actions are made depending on the error condition, including the following:
?[certificate chain]The credibility of the trusted certificate path, as described earlier in this article;
? The certificate is revoked revocation, there are two kinds of way offline CRL and online OCSP, different client behavior will be different;
? Validity expiry date, whether the certificate is in the valid time range;
? Domain, verify that the certificate domain name matches the current access domain name, matching rules follow-up analysis;
(4). Client_key_exchange+change_cipher_spec+encrypted_handshake_message
(a) Client_key_exchange, after the legitimacy verification is passed, the client calculates the random number pre-master, and encrypts the certificate public key and sends it to the server;
(b) At this point the client has obtained all the information needed to calculate the negotiation key: two plaintext random number random_c and random_s with their own calculation generated pre-master, calculated to obtain the negotiated key;
Enc_key=fuc (Random_c, random_s, Pre-master)
(c) Change_cipher_spec, the client notifies the server that subsequent communications are encrypted using a negotiated communication key and encryption algorithm;
(d) Encrypted_handshake_message, combining the hash value of all previous communication parameters with other related information to generate a piece of data, using the negotiation key session secret and algorithm encryption, and then sent to the server for data and handshake verification;
(5). Change_cipher_spec+encrypted_handshake_message
(a) The server decrypts the encrypted pre-master data with the private key, and calculates the negotiated key based on the two plaintext random numbers random_c and random_s previously exchanged: Enc_key=fuc (Random_c, random_s, pre-master);
(b) Computes the hash value of all previously received information and decrypts the Encrypted_handshake_message sent by the client, verifying the correctness of the data and the key;
(c) Change_cipher_spec, after the verification is passed, the server also sends Change_cipher_spec to inform the client that subsequent communication is encrypted using the negotiated key and algorithm;
(d) Encrypted_handshake_message, the server also combines all current communication parameter information to generate a piece of data and uses the negotiation key session secret and algorithm to encrypt and send to the client;

(6). Handshake End
The client calculates the hash value of all received information, and uses the negotiated key to decrypt the Encrypted_handshake_message, verifies the data and keys sent by the server, and verifies that the handshake is completed by the verification.
(7). Encrypted communication
Begins encrypting communication with the algorithm using the negotiation key.
Attention:
(a) The server can also request authentication client, namely bidirectional authentication, can send client_certificate_request information in procedure 2, the client sends Client_certificate and CERTIFICATE_VERIFY_ in Process 4 first Message information, the authentication method of the certificate is basically the same, Certificate_verify_message is using the client's private key encryption of a section based on the negotiated communication information to obtain data, the server can use the corresponding public key decryption and verification;
(b) Depending on the key exchange algorithms used, such as ECC, the negotiation details are slightly different and generally similar;
(c) Sever key exchange's role is that when the server certificate does not carry enough information, it is sent to the client to calculate pre-master, such as a DH-based certificate, the public key is not included in the certificate and needs to be sent separately;
(d) Change cipher spec can actually be used to notify the end of the current use of encrypted communication mode, not in-depth analysis;
(e) ALTER message is used to indicate the state change or error message in the handshake or communication process, the general alarm information trigger condition is the connection is closed, received illegal information, information decryption failure, the user cancels the operation and so on, after receiving the alarm message, the communication will be disconnected or the receiver decides whether to disconnect.

2. Session Cache Handshake Process

In order to speed up the handshake, reduce the performance and resource consumption of the Protocol (specifically analyzed in the following article), the TLS protocol has two types of session caching mechanisms: The session ID sessions are logged to the conversation log session ticket.
Session ID by the server-side support, the standard fields in the protocol, so basic all servers are supported, server-side save session ID and negotiated communication information, Nginx 1 m memory can save about 4,000 session ID machine-related information, occupy more server resources;
Session ticket requires both server and client support, belongs to an extension field, supporting a range of about 60% (no reliable statistics and sources), the negotiated communication information is encrypted and sent to the client to save, the key only the server knows, the server resource is very small.
In contrast, the main point is to save the negotiation information in different ways, similar to the session and cookie in HTTP.
In the case of both, the (Nginx implementation) takes precedence over the use of session_ticket.
The handshake process is as follows:

Note: Although the handshake process has 1.5 back and forth, the first application data sent by the client to the server does not need to wait for the information returned by the server, so the handshake delay is 1*rtt.
(1). Session ID
(a) If a connection has been established between the client and the server, the server will return the session ID after the handshake succeeds and save the corresponding communication parameters in the server;
(b) If the client needs to establish a connection with the server again, the logged information is sent to the server in the session ID in Client_hello;
(c) The server retrieves the cache record based on the session ID received, and if no retrieval of the arrival cache expires, follow the normal handshake process;
(d) If the corresponding cache record is retrieved, the CHANGE_CIPHER_SPEC and encrypted_handshake_message information are returned, and the two messages Act similarly, encrypted_handshake_message Is the hash value to the current communication parameter and the Master_secret;
(f) If the client is able to verify the data encrypted through the server, the client also sends Change_cipher_spec and encrypted_handshake_message information;
(g) When the server verifies that the data is passed, the handshake is successful and the normal encrypted data communication begins.
(2). Session Record Sessions Ticket
(a) If a connection has been established between the client and the server, the server carries the encrypted Session_ticket information in the New_session_ticket data and the client saves it;
(b) If the client needs to establish a connection with the server again, the extended field Session_ticket in Client_hello carries the encrypted information and sends it to the server together;
(c) The server decrypts the Sesssion_ticket data, if it can decrypt the failure, then follow the normal handshake process;
(d) If the decryption is successful, the Change_cipher_spec and encrypted_handshake_message information is returned, and the two information function is similar to the session ID;
(f) If the client is able to verify the data encrypted through the server, the client also sends Change_cipher_spec and encrypted_handshake_message information;
(g) When the server verifies that the data is passed, the handshake is successful and the normal encrypted data communication begins.

3. Rebuilding the connection

Rebuilding the connection renegotiation the TLS connection that is being used, the new authentication and key negotiation process, characterized by the ability to re-authenticate, update the key or algorithm without disconnecting the current data transfer, so that both server-side storage and cached information can be maintained. Both the client and the server are able to initiate the process of rebuilding the connection, which is currently not supported by Windows & XP and SSL 2.0.
(1). Server Rebuild Connection

Server-side rebuild connections typically occurs when a client accesses a protected data. The basic process is as follows:
(a) A valid TLS connection is established and communicated between the client and the server;
(b) Client access to protected information;
(c) Return of hello_request information on the server side;
(d) The client sends Client_hello information after receiving hello_request information and begins to reestablish the connection.
(2). Client Rebuild Connection

The client rebuilds the connection generally in order to update the communication key.
(a) A valid TLS connection is established and communicated between the client and the server;
(b) The client needs to update the key and proactively issue Client_hello information;
(c) After receiving Client_hello information from the server, it cannot immediately identify the non-application data, so it will be submitted to the next processing, after processing will be returned to notify the information to require rebuilding the connection;
(d) The server does not immediately stop sending data to the client until the connection is determined, but the client is not sent any more information to the server, possibly at the same time or with the cached data;
(e) After the server identifies the rebuild connection request, sends the SERVER_HELLO message to the client;
(f) The client is also unable to immediately determine the information non-application data, the same is submitted to the next processing, after processing will return to notify the information to require rebuilding the connection;
(g) The process by which the client and server begin a new rebuild connection.

4. Key Calculation

The previous section mentions the two random numbers transmitted in plaintext Random_c and random_s and the pre-master exchanged between the server and the client via encryption, and three parameters as the basis of the key negotiation. This section discusses the basic computational procedures for key negotiation and the use of keys during communication.
(1). Calculate Key
Involving the parameters random client and random server, Pre-master, master secret, key material, compute keys, both the server and the client have these basic information, the interchange method is described in the previous section, the calculation process is as follows:

(a) The client uses cryptographic algorithms such as RSA or Diffie-hellman to generate pre-master;
(b) Pre-master with random client and random server two stochastic numbers through Pseudorandomfunction (PRF) calculation to get master secret;
(c) Master secret combined with the random client and randomly server two stochastic numbers to get the Key material by iterative computation;
Here are some important records that can solve some of the love in-depth study of the doubts of friends, copy of the material, share to everyone:
(a) Premaster secret the first two bytes is the version number of TLS, which is a more important version number to check the handshake data, because in the client Hello phase, clients send a list of cryptographic suites and the version number of the currently supported SSL/TLS to the server. and is transmitted in clear text, if the handshake packet is cracked, the attacker is likely to string the packet, select a less secure encryption suite and version to the server, so that the data can be cracked. Therefore, the server needs to decrypt the ciphertext in the Premaster version number with the previous Client Hello phase of the version number, if the version number is lower, then the string changed, immediately stop sending any message. (copy)
(b) A random number is required for both the client and the server, so that the generated key will not be the same every time. Because the certificate is static in the SSL protocol, it is necessary to introduce a stochastic factor to guarantee the randomness of the negotiated key.
For RSA key exchange algorithm, the Pre-master-key itself is a random number, plus the random in the Hello message, three random numbers through a key exporter to finally export a symmetric key.
The presence of the pre master is that the SSL protocol does not trust each host to produce a completely random random number, and if the random number is not random, then the pre-master secret is likely to be guessed, then only the pre-master secret as the key is inappropriate, so you must introduce New random factors, then the client and server plus pre-master secret three random number generated by the key is not easy to guess, a pseudo-random may not be random, but three pseudo-random is very close to random, each increase of one degree of freedom, the randomness is not one of the increase.
(2). Key Usage
Key after 12 rounds of iterative calculation will get to 12 hash values, grouped into 6 elements, the list is as follows:

(a) Mac key, encryption key, and IV are a set of cryptographic elements that are used by both the client and the server, but both sets of elements are acquired simultaneously on both sides;
(b) Clients use client group elements to encrypt data, servers to decrypt with client elements, servers to encrypt with server elements, and client to decrypt with server elements;
(c) Two-way communication in different directions using different keys, crack communication needs to crack at least two times;
(d) encryption key for symmetric encryption of data;
(e) IV is used as an initialization vector for many cryptographic algorithms, specifically to study symmetric encryption algorithms;
(f) Mac key is used for data integrity checking;
(3). Data encryption Communication Process
(a) Partitioning the application layer data into a suitable block;
(b) To prevent replay attacks by numbering the Shard data;
(c) Compressing the data using a negotiated compression algorithm;
(d) Calculating MAC values and compressing the data to form the transmitted data;
(e) Encrypting data using the client encryption key and sending it to server servers;
(f) After the server receives the data, it uses the client encrytion key to decrypt, verify the data, decompress the data, and reassemble.
Note: The calculation of Mac values includes two hash values: Client Mac key and hash (number, package type, length, compressed data).

5. Grasping packet Analysis

About the grab bag no longer detailed analysis, according to the previous analysis, the basic situation can be matched, according to the usual positioning problem process, the individual to make some points that need attention:
     (1). Grab the packet HTTP communication, Can clearly see the head of the communication and the plaintext of the message, but HTTPS is encrypted communication, unable to see the HTTP protocol related headers and data of the plaintext information,
     (2). Packet capture HTTPS communication consists of three processes: TCP establishes a connection, TLS handshake, TLS encrypted communication, the main analysis of HTTPS communication handshake establishment and status information.
     (3). Client_hello
can know the highest protocol version number supported by the client based on version information, if it is a low-version protocol such as SSL 3.0 or TLS 1.0. It is very important to note that some handshake failures may occur because of low version; The
determines whether SNI is supported based on the server_name field in the Extension field, which is supported, otherwise not supported, and is useful for locating handshake failures or certificate return errors;
Session ID is the standard protocol part, if no connection is established, the corresponding value is NULL, and NOT NULL, the corresponding connection was established and cached; The
Session Ticke T is the extension protocol part, there is the field Description protocol support Sesssion ticket, otherwise unsupported, exists and the value is NULL, indicating that the connection was not established and cached before, exists and the value is not empty, indicating that there is a cache connection.
    (4). Server_hello
     The maximum version of the protocol supported by the server can be inferred from the TLS version field, and different versions may cause a handshake to fail;
     the encryption protocol based on Cipher_suite information to determine the priority of server support;
    (5). Ceritficate
     server configuration and return certificate chain, according to the certificate information and in the server configuration file comparison, to determine whether the request and expectations are consistent, if not consistent, Whether to return the default certificate.
    (6). Alert
     Alert Information alert indicates that the cause of the connection failure is the type of alarm, which is important for locating the problem

HTTPS protocol detailed (iv): TLS/SSL handshake process