Three ways to decrypt HTTPS traffic

Reprinted from: https://imququ.com/post/how-to-decrypt-https.html Jerry Qu

Web Security is a systematic project, and any slight negligence can cause the entire security barrier to fall apart. Take HTTPS, it's "content encryption, data integrity, identity authentication" three security assurances, will also be illegal root certificate, server configuration error, SSL Library vulnerability, private key theft , and so on risk. Many students think that as long as the site address before the visit has a small green lock is absolutely safe, not actually. This paper introduces the three most common methods and principles of HTTPS traffic decryption, and discusses the security risk of HTTPS.

Man-in-the-middle

Man-in-the-middle (Man-in-the-middle, short-MITM), can communicate with the network at both ends of the connection, the exchange of their received data, so that the communication at both ends of the view that they are directly talking to each other, in fact, the entire conversation is controlled by intermediaries. In short, in the real server view, the middleman is the client, and the real client will think the middleman is the service side.

There are various ways to achieve a man-in-the-middle attack, and no discussion is underway. Some common Http/https-grab debugging tools are designed to intercept traffic by creating a local proxy service and then modifying the browser proxy settings, and they work in unison with a man-in-the-middle attack. The tools I've used are: Fiddler, Charles, and whistle. I am in the "http Agent principle and implementation (a)" The HTTP General agent described in the article, playing the role of the HTTP middleman. "

This article mainly discusses the HTTPS middleman, the simple schematic is as follows:

Server <---> Local Proxy <---> Browser         ^                 ^       https (1)          HTTPS ( 2)

The above HTTPS (1) connection, is the middleman impersonating the client, the connection with the server, because the HTTPS server generally does not authenticate the client identity, this step is usually not a problem. For HTTPS (2) connection, the intermediary wants to impersonate the server, must have the corresponding domain name certificate private key, and the attacker to get the private key, only through these means:

1. Go to the Web server to get it;
2. Issuing certificates from CA;
3. Issue your own certificate.

To prevent the first two points, the site needs to do a good job in all aspects of security, from the host security to the site security (to avoid the private key stolen), from the domain name resolution security to the domain name mailbox security (avoid attackers to re-visa). The attacker himself issued a certificate that cannot be verified by the system's built-in root certificate and cannot be used by default for man-in-the-middle attacks.

For Fiddler, the key to decrypting HTTPS traffic is that they import their own certificates into the list of trusted root certificates in the system so that their self-signed certificates can be trusted by the browser. Enter the "https"tab in the Fiddler settings, check the relevant functions, you can successfully decrypt and modify HTTPS traffic. In this case, you can see the certificate chain in the browser:

RSA Private Key

I wrote in the article "Using Wireshark to debug HTTP/2 traffic": The principle of Wireshark is to read and analyze the network card data directly, there are two ways to let it decrypt HTTPS traffic:

1. If you have an HTTPS website encrypted private key, you can use to decrypt the encrypted traffic of this website;
2. Some browsers support the use of symmetric keys used in TLS sessions in external files that can be used by Wireshark encryption.

This article introduces the second scenario, this article briefly introduces the first kind.

Open the SSL protocol settings for Wireshark, refer to the IP, port, protocol, and certificate private keys (the private key must be in PEM format):

Then access the private key corresponding to the website, you can see the traffic has been decrypted:

Is the encrypted data in HTTP/1 transmitted in a way that can decrypt HTTP/2 traffic? The conclusion is: No! The specific reasons are analyzed slowly below.

We know that the TLS handshake phase requires key exchange and server-side authentication for two important operations, the key exchange is to generate a non-secure data channel only the communication between the two parties know the shared key premaster Secret, resulting in the Master Secret and subsequent symmetric encryption Session Key and MAC key. The purpose of client-side authentication is to ensure that a legitimate server is connected to the private key of the Web site.

The most common key exchange method is RSA, and the following diagram clearly describes the process:

Client random and Server random are transmitted in plaintext and can be viewed directly by the intermediary. After the client generates Premaster Secret, it is sent with the service-side certificate public key, and if the server has the corresponding private key, it can be successfully decrypted Premaster Secret. At this point, the client and the server have the same clients, random, and Premaster Secret, each of which can work out the same successor required Key.

As you can see, this method incorporates two steps for key exchange and server-side authentication, and if the server can decrypt Premaster Secret, it means that the server has the correct private key. The middleman has no private key, cannot get premaster Secret, and cannot decrypt subsequent traffic.

For Wireshark, after configuring the private key of a website, it is easy to understand the encrypted traffic that decrypts the site "key exchange using RSA".

Clearly, there is a big problem with RSA key exchange: There is no forward security forward secrecy. This means that the attacker can save the encrypted traffic that is heard, and once the private key is received, all traffic can be decrypted successfully.

In fact, most of the current HTTPS traffic is ECDHE key exchange. ECDHE is a DH (Diffie-hellman) algorithm that uses elliptic curve (ECC). Is the DH key exchange process:

The Server DH Parameter in is signed with the certificate private key, and the client uses the certificate public key to verify the service-side legitimacy. Compared to RSA key exchange, DH is passed Premaster Scret into the Parameter required to pass the DH algorithm, and each side calculates Premaster Secret.

In this case, because the Premaster Secret does not need to be exchanged, the middleman cannot get Premaster Secret and Master Secret even if there is a private key. This means that Wireshark cannot decrypt encrypted traffic using ECDHE for key exchange by configuring RSA Private key. Of course, after using ECDHE, although the middleman will not be able to decrypt the traffic before the private key, but he can implement a MITM attack to decrypt the traffic after, so the private key is still to be kept good.

Compared to RSA can be used for both key exchange, but also for digital signature; The ECC side is quite clear: ECDHE is used for key exchange, and ECDSA is used for digital signatures. The current key exchange + Signature has three main options:

• RSA key exchange, RSA digital signature;
• ECDHE key exchange, RSA digital signature;
• ECDHE key exchange, ECDSA digital signature;

Here are some of the three key exchange sites available in Chrome:

Key exchange

Only tlsv1.2+ is available in HTTP/2 and hundreds of ciphersuite are disabled (see: TLS 1.2 Cipher Suite Black List). In fact, the ciphersuite allowed by HTTP/2 must adopt a key exchange algorithm with forward security, which does not allow the use of RSA key exchange. This is why RSA Private Key cannot decrypt HTTP/2 encrypted traffic.

Sslkeylogfile

Both Firefox and Chrome SSLKEYLOGFILE Save the Premaster Secret or Master Secret generated by each HTTPS connection when the file path exists in the system environment variable. With this file, Wireshark can easily decrypt HTTPS traffic, even with ECDHE, a key exchange with forward security, such as:

Decrypt HTTP2 over TLS

For a detailed description of this scenario, refer to the article "Using Wireshark to debug HTTP/2 traffic".

SSLKEYLOGFILEThe file is the most important encryption information in HTTPS data transmission, if not for debugging purposes, generally no one will actively configure this environment variable, so this scheme basically does not affect the security of HTTPS.

Summarize

Fiddler Such tools implement HTTPS traffic decryption by importing the root certificate into the system, acting as a middleman. To prevent real HTTPS man-in-the-middle attacks, the Web site needs to keep its own certificate private key and domain name authentication information, in order to prevent bad CA illegally issued their own website certificate to third parties, but also to enable Certificate Transparency, HTTP public Key Pinning and other strategies; users should not trust the certificate of unknown origin, let's not arbitrarily import the certificate to the root certificate list, but also to develop the habit of frequently checking the common Web site certificate chain.

RSA key exchange does not have forward security, which means that all encrypted traffic can be undone once the private key is compromised. To do this, the Web site needs to enable the use of ECDHE as the key exchange Ciphersuite, or directly use ECC certificate, users need to discard the ECDHE not support the antique operating system and browser.

For browsers, HTTPS has no secrets, and with browser-generated SSLKEYLOGFILE files, Wireshark can easily decrypt HTTPS traffic. In addition, if the browser is installed malicious extension, even if access to secure HTTPS Web site, the submitted data can be intercepted. This client is subject to security issues caused by the attacker's control and cannot be resolved through HTTPS.

Three ways to decrypt HTTPS traffic