An overview of HTTP proxy and SPDY protocols

HTTP proxies are the most classic and most common proxy protocols. Its use is very extensive, commonly seen in the company intranet environment, the General staff need to configure an HTTP proxy browser to access the Internet. At first, HTTP proxies were also used to climb g*w, but because the g*w was evolving, ordinary HTTP proxies had long been invalidated. However, there are still many people using plaintext HTTP proxy protocol combined with software such as Stunnel to turn Q, sometimes this agent is called HTTPS agent. Later, there is the Webvpn via Spdy and other proxy protocols, characterized by the direct support of the Chrome browser. Plus what can the HTTP proxy protocol delegate? Is it possible to proxy HTTP or proxy HTTPS, or is it possible to implement the SOCKS proxy? In short, very confusing. The Fqsocks project uses Python to implement the various mainstream variants of HTTP proxies, and finally understand the true meaning of different titles. This article attempts to sum up one or two.

Agent Basics

All agents, the principle of which is similar. The network topology is the same:

[Client] <-tcp connection-> [agent] <-tcp connection-> [Server]

The surrogate left hand is connected to the client, the right hand holding the connection to the server, and then the data between the two TCP connections to the copy. A variety of different agent protocols, different just TCP connections on what is running on the protocol, how the data is packaged, unpacking. There is no situation where a TCP connection is established between the client and the server. Only the VPN this layer in the IP package to achieve the connection between the client and the server.

HTTP traffic, clear-text proxy connections

This is the simplest kind of HTTP proxy. The TCP connection established between the client and the agent is plaintext, which is not SSL-encrypted. The data transmitted in a TCP connection is the post of the plaintext HTTP and get these. For this kind of proxy work, the client sends a packet to the server almost without modification to the agent, and can get almost the same return as the server. The process of its receiving and contracting is as follows

[Client]-http get-> [proxy]-http get-> [Server]
[client] <-200 OK [proxy] <-200 OK [Server]

HTTPS traffic, plaintext proxy connection

In this mode of operation, the TCP connection established between the client and the agent is still plaintext, which is not SSL-encrypted. However, the request made by the client is not an HTTP request, but an HTTPS request. Because the HTTPS request is SSL-encrypted, as an agent, although the TCP connection that the client has established is not SSL-encrypted, the data transferred in the middle is SSL-encrypted. The agent cannot know what the destination server is by checking the content of the HTTPS request. Therefore, the client must use the HTTP connect request before sending HTTPS request to tell the agent to connect the server is what, then wait for proxy server The TCP connection of the right hand and all hold steady, the client can send and receive HTTPS traffic. The agent is only a simple one in two TCP connections to do data in the copy, do not know what the client and server are doing. In fact, the use of HTTP CONNECT, the client and the server in addition to the exchange of HTTPS traffic, any traffic is possible. After HTTP connect, the TCP connection between the client and the agent is "equivalent" to the TCP connection established between the server. Of course, as we have said before, this direct connection is only an illusion. Its receiving and contracting process is as follows:

[Client]-http CONNECT a.b.c.d:port-> [proxy]-tcp syn-> [Server]
[client] <-200 ok-[proxy] <-tcp SYN ack-[Server] # here's 20.  0 OK has different meanings than the following OK
[client]-SSL client hello-> [proxy]-SSL client hello-> [Server] # SSL handshake Package 1
[client] <-ssl SERVER hello-[agent] <-SSL server hello/certificate-[Server] # SSL handshake Package 2
[client]-SSL client certificate-> [proxy]-ssl client Certificate et-> [Server] # SSL Handshake Package 3
[client] <-ssl SERVER finished-[proxy] <-ssl server finished-[Server] # SSL Handshake Pack 4
[ Client]-ssl encrypted HTTP get-> [proxy]-ssl encrypted HTTP get-> [Server]
[client] <-ssl encrypted ok-[proxy] <-ssl encrypted ok-[Server]

Because of Connect, the TCP connection between the right-hand and the traffic, the agent is completely ignore, so this is also possible:

[Client]-http CONNECT a.b.c.d:port-> [proxy]-tcp syn-> [Server]
[client] <-200 ok-[proxy] <-tcp SYN ack-[Server] # here's 20. 0 OK has different meanings than the following OK
[client]-any request-> [proxy]-any request-> [Server]
[client] <-any response-[proxy] <-any response-[Server]

Because the TCP connection can be used to run arbitrary things after connect, of course, it can be used to run another agent agreement, such as socks agent.

As long as the socks agent is running in 443 such HTTP CONNECT allowed ports, through the http-connect+socks agent can be in the company intranet access to the Internet content outside the HTTP.

HTTP traffic, SSL-encrypted proxy connection

Traditional HTTP proxies, which are not encrypted between the client and the agent. Because GFW can know the destination server you access through the HTTP proxy from the plaintext traffic, you can disconnect the connection from it. In order for the HTTP Agency agreement to continue to bear the brunt of the wall, people invent a new way. Its topological structure is as follows

[Client] <-tcp connection-> [stunnel client mode] <-TCP over SSL connection-> [Stunnel server mode] <-tcp connection-> [agent] <-tcp connection-> [Server]

Although the topological structure is complex, the concept is actually very simple. Because both the client and the proxy do not directly support SSL TCP connections, a conversion is done via Stunnel, which is encrypted by the client after Stunnel client mode, and then decrypted in a server segment mode that is useful stunnel on the agent. This GFW folder in the middle to see the traffic is SSL encrypted, unable to filter the URL keyword. The process of receiving the contract is as follows:

[Client]-http get-> [stunnel client mode]-SSL encrypted HTTP get-> [stunnel server mode]-http get-> [proxy]-http get-> [Server]
[client] <-200 ok-[stunnel client mode] <-SSL encrypted ok-[stunnel server mode] <-200 ok-[proxy] <-200 ok-[Server]