Fiddler crawl HTTPS setting and its principle

Iddler crawl HTTPS setting and its principle

2018-02-02

Directory

1 HTTPS handshake process
2 Fiddler Crawl HTTPS process
3 Fiddler crawl HTTPS settings
Reference

1 HTTPS handshake process

HTTPS is not a new protocol for the application layer. Only the HTTP communication interface portion is replaced with SSL (Secure Sockets Layer) and TLS (Secure Transport Layer protocol). That is, HTTP is called HTTPS (http Secure), which adds encryption and authentication mechanisms.

HTTPS = HTTP + authentication + encryption + Integrity protection

The handshake process is as follows:

The first step: The client initiates a clear request: sends a set of cryptographic rules that it supports, and a random number (Random_c) to the server

The second step: the server selects a set of encryption rules and hash algorithm , and its own identity information to the certificate (CA: Contains Web site address, encrypted public key, certification authority and other information) and a random number (random_s) to the client

Step three: Client response to Server

Verify the legality of the certificate (whether the authority that issued the certificate is legitimate, whether the website address contained in the certificate is the same as the address being accessed, and so on). If the certificate is trusted, a small lock is displayed in the Explorer bar, or the certificate is not prompted for the letter.

If the certificate is trusted, or if the user accepts an untrusted certificate, the client does the following things:

1. Generate Password: The browser generates a random number of passwords (Pre_master) and is encrypted with the public key (Enc_pre_master) in the CA certificate for transmission to the server.
2. Calculate negotiation key: Enc_key=fuc (Random_c, random_s, Pre-master)
3. Generate handshake information: Use the agreed hash to calculate the handshake message, and use the negotiated key Enc_key and the agreed-upon algorithm to encrypt the message.
4. Send the following information to the server:
   1. Server random number password Enc_pre_master encrypted with public key
   2. The client sends a notification to the server, "later we all have to use the agreed-upon algorithm and the negotiation key to communicate the OH."
   3. The client encrypts the generated handshake information.

The fourth step, the server receives the data sent by the client to do the following four things:

1. Private key decryption: Decrypts the password pre_master from the received enc_pre_master using its own private key.
2. Calculate negotiation key: Enc_key=fuc (Random_c, random_s, Pre-master)
3. Decryption handshake message: Use the negotiation key Enc_key to decrypt the handshake message from the client and verify that the hash is consistent with the client.
4. Generate handshake messages use the negotiated key Enc_key and the agreed-upon algorithm to encrypt a handshake message and send it to the client. There are two data to be sent here:
   1. Server to the client's notification, "Listen to you, we will use the agreed algorithm and negotiation key for communication Oh."
   2. The handshake information generated by the server encryption.

The fifth step, the client gets handshake information decryption, handshake end.

The client decrypts and computes the hash of the handshake message, which is the end of the handshake process if it is consistent with the hash of the server.

Sixth step, normal encrypted communication

After the handshake succeeds, all communication data will be encrypted and decrypted by the previously negotiated key Enc_key and the agreed-upon algorithm.

Here the browser and the Web site to send encrypted handshake message and verify, the purpose is to ensure that both sides have obtained a consistent password and hash algorithm, and can be normal encryption and decryption data, for the subsequent transmission of real data to do a test.

From the handshake process, we can learn:

• The legality of the website can be confirmed by CA
• Through Enc_key encryption and decryption, in the transmission process, in order to ensure that the enc_key is not broken, after the client with the public key encryption, the server side with the private key decryption, the private key only the server side, so even if the message is intercepted, it can not be cracked.
• Hash algorithm ensures the integrity of the message

Hash algorithm ensures

2 Fiddler Crawl HTTPS process

As we all know, Fiddler is a good proxy tool that can crawl protocol requests for debugging.

The Fiddler crawl HTTPS protocol is mainly carried out in the following steps:

In the first step, fiddler intercepts the HTTPS request sent by the client to the server, fiddler masquerading as the client sending a request to the server for handshake.

The second step, the server sends back the corresponding, fiddler obtains the CA certificate to the server, decrypts with the root certificate Public key, verifies the server data signature, obtains the public key of the server CA certificate. Then fiddler forges its own CA certificate and passes the impersonation server certificate to the client browser.

The third step, and the normal process of the client's operation, the client based on the returned data for certificate verification, generate password pre_master, with fiddler forged certificate public key encryption, and generate HTTPS communication with the symmetric key Enc_key.

In the fourth step, the client passes important information to the server and is intercepted by fiddler. Fiddler will intercept the ciphertext with its own forged certificate of the private key to get and calculate the HTTPS communication with the symmetric key Enc_key. Fiddler the symmetric key is passed to the server using the server certificate public key encryption.

The fifth step, the same as the normal process of server-side operation, the server with the private key after the establishment of trust, and then send an encrypted handshake message to the client.

Sixth step, fiddler intercept the ciphertext sent by the server, with the symmetric key to solve, and then use their own forged certificate private key encryption to the client.

The seventh step, the client gets encrypted information, with the public key to solve, verify the hash. The handshake process is formally completed, and the client and server side establish a "trust".

How does fiddler act as a third party between the server and the client in the subsequent normal encrypted communication process?

Server-Client: Fiddler received the ciphertext sent by the server, with a symmetric key to unlock, to obtain the plaintext sent by the server. Encrypted again, sent to the client.
Client-to-server: The client is encrypted with a symmetric key and is intercepted by the fiddler, and the decryption obtains plaintext. Encrypted again, sent to the server side. Since Fiddler has been communicating with a symmetric key enc_key, the information is transparent to the entire HTTPS communication process.

3 Fiddler crawl HTTPS settings

As can be seen from the above, the key to the success of the Fiddler Crawl HTTPS protocol is the root certificate (the root certificate is the certificate issued by the CA Certification Center to itself and is the starting point of the trust chain.) Installing the root certificate means trusting the CA certification Center. ), which is the starting point of a chain of trust, which is also the key to fiddler forged CA certificates to gain client and server-side trust.

Without a trusted fiddler root certificate, you can neither decrypt the public key nor let the client trust the fiddler forged CA certificate to get pre_master.

Next we'll see if the settings let fiddler crawl the HTTPS protocol.

Fiddler Menu->tools->telerik Fiddler Options

Click ' Yes ' in the popup dialog

After clicking ' Yes (Y) ' in the popup dialog, we can see that the root certificate of fiddler is added to "Trusted Root Certification Authorities" by Internet option of IE.

Reference

[1] talking about HTTPS and fiddler crawl HTTPS protocol

[2] How HTTPS works

[3] Modify configuration enables Fiddler to view HTTPS requests

Fiddler crawl HTTPS setting and its principle