HTTP proxy and SPDY protocol (1)

HTTP proxy is the most classic and common proxy protocol. It is widely used in the company's Intranet environment. Generally, employees need to configure an HTTP proxy for the browser to access the Internet. At first, the HTTP Proxy was also used to go over the "Kung Fu network". However, due to the continuous development of "Kung Fu network", the common HTTP Proxy has long been ineffective. However, there are still many people who use plain text HTTP Proxy protocol and software such as stunnel to encrypt the wall. Sometimes such a proxy is also called an HTTPS proxy.

Later, proxy protocols such as WebVpn via SPDY were introduced, featuring direct support from Chrome. What can the HTTP Proxy protocol Proxy do? Can I only proxy HTTP or HTTPS, or can I implement SOCKS proxy? In short, it is very confusing. In the fqsocks project, python is used to implement various mainstream variants of HTTP proxy and finally understand the true meaning after different titles. This article attempts to sum up two.

Proxy Basics

The principle of all proxies is similar. The network topology is as follows:

[Client] <-TCP connection-> [proxy] <-TCP connection-> [server]

The proxy holds the connection with the client in the left hand and the connection with the server in the right hand, and then copies data between two TCP connections. For different proxy protocols, the difference is what the TCP connection runs on, how the data is packaged, and how the package is split. No TCP connection is established between the client and the server. The connection between the client and the server can be realized only when VPN works at the IP packet layer.

HTTP traffic, proxy connection in plaintext

This is the simplest HTTP proxy. The method is that the TCP Connection established between the client and the proxy is in plain text, that is, it is not encrypted by SSL. The data transmitted in the TCP connection is the http post and GET in plain text. For this kind of proxy method, the packages sent from the client to the server almost do not need to be modified and sent to the proxy, and the return is almost the same as that of the server. The packet sending and receiving process is as follows:

[Client]-http get-> [proxy]-http get-> [server] [client] <-200 OK [proxy] <-200 OK [server]

HTTPS traffic, proxy connection in plaintext

In this way, the TCP Connection established between the client and the proxy is still in plain text, that is, it is not encrypted by SSL. However, the client sends an HTTPS request instead of an HTTP request. Because HTTPS requests are encrypted over SSL, as a proxy, although the TCP Connection established between the client and the client is not encrypted over SSL, the data transmitted in the middle is encrypted over SSL. Therefore, the proxy cannot check the content of the HTTPS request to know what the target server is.

Therefore, the client must use the http connect request to tell the proxy what the server is to CONNECT to before sending an HTTPS request, and then wait until the proxy server's right-hand TCP connection is stable, the client can send and receive HTTPS traffic. The proxy simply copies data in two TCP connections and does not know what the client and server are doing. In fact, by using http connect, the client and server can exchange HTTPS traffic. After http connect, the TCP connection between the client and the proxy is "equivalent" to a TCP connection established with the server. Of course we have already said that this direct connection is just an illusion. The packet sending and receiving process is as follows:

[Client]-http connect a. B. c. d: port-> [proxy]-TCP SYN-> [server]
[Client] <-200 OK-[proxy] <-tcp syn ack-[server] # The 200 OK here has different meanings from the 200 OK below
[CLIENT]-ssl client hello-> [proxy]-ssl client hello-> [server] # SSL handshake Package 1
[Client] <-ssl server hello-[proxy] <-ssl server hello/CERTIFICATE, etc.-[SERVER] # SSL handshake package 2
[CLIENT]-ssl client certificate, etc.-> [proxy]-ssl client certificate, etc.-> [server] # SSL handshake package 3
[Client] <-ssl server finished-[proxy] <-ssl server finished-[SERVER] # SSL handshake package 4
[Client]-SSL-encrypted http get-> [proxy]-SSL-encrypted http get-> [server]
[Client] <-SSL-encrypted 200 OK-[proxy] <-SSL-encrypted 200 OK-[server]
After the CONNECT operation, the proxy does not care about the traffic in the TCP connection of the left and right hands, so this is also possible:
[Client]-http connect a. B. c. d: port-> [proxy]-TCP SYN-> [server]
[Client] <-200 OK-[proxy] <-TCP SYN ACK-[server]
# The 200 OK here has different meanings from the 200 OK below
[Client]-any request-> [proxy]-any request-> [server]
[Client] <-any response-[proxy] <-any response-[server]

Because TCP connections can be used to run anything after CONNECT, of course, they can also be used to run another Proxy protocol, such as SOCKS proxy.

As long as the SOCKS proxy is running on a port that is allowed by http connect such as 443, Internet content outside of HTTP can be accessed through the HTTP-CONNECT + SOCKS proxy within the company's intranet.