Example of proxy service implementation in python

The principle of proxy service is very simple, taking browser and web server as an example. It's nothing more than browser
Send A request to the B proxy, and then the B proxy sends the request to the C web service, and then the C reponse-> B->.
To write a web proxy service, you must first understand the http protocol. Of course, do not go deeper, unless you need to implement powerful functions: Modify XX information,
Server Load balancer. An http request consists of three parts: the request line, the message header, and the request body;
For more information, see. The following is a normal GET Request Header (the Cookie part does not take screenshots, and the system w7 is used ):



You can see the first line: GET is the request method,/is the path, followed by the Protocol version; the second line is the request header, which is in the form of key-value pairs;
The GET method has no body. Post has a body. In addition, the request method headers are basically the same, and each row ends with \ r \ n.
The basic request method is as follows:

GET Request to GET the resource identified by Request-URI
POST attaches new data to the resource identified by Request-URI
HEAD Request to obtain the Response Message Header of the resource identified by Request-URI
The PUT Request server stores a resource and uses Request-URI as its identifier.
The DELETE Request server deletes the resource identified by Request-URI.
TRACE Request information received by the server for testing or diagnosis
CONNECT reserved for future use
OPTIONS requests query server performance, or query resource-related OPTIONS and requirements
However, after a proxy is used, the request obtained from the proxy service is as follows:


What is the difference between the resource path of the first line and the first image. When a proxy request is set on the browser, the whole url is used as the resource path. Therefore, we need to delete the domain name, and the proxy server sends the modified request to the target
Web server. This is so simple. Of course, the CONNECT method is special and special, so let's talk about other methods first.
Basic Ideas:
1. The agent server runs the listener. When a client browser request arrives, the client handle (or descriptor) is obtained through accept );
2. Use the client descriptor to receive the request sent by the browser and separate the first line to modify the first line and obtain the method,
The part to be removed is indicated by targetHost except http.
3. The method, request, and targetHost can be obtained through step 1, which can be processed differently according to different methods,
Except for CONNECT, GET, POET, PUT, and DELETE, GET, POET, PUT, and DELETE are basically the same, so the first line is processed, for example:
Copy codeThe Code is as follows:
GET http://www.a.com/https/1.1
Replace
GET, HTTP, 1.1

In this case, targetHost is the red part. The default request port 80 is port 80. If the targetHost has a port (such as www.a.com: 8081 ),
The branch office port is required, and the port is 8081 at this time. Connect to the target server target Based on targetHost and port. The implementation code is as follows:
Copy codeThe Code is as follows:
Def getTargetInfo (self, host): # process targetHost to obtain the URL and port and return the value.
Port = 0
Site = None
If ':' in host:
Tmp = host. split (':')
Site = tmp [0]
Port = int (tmp [1])
Else:
Site = host
Port = 80
Return site, port
Def commonMethod (self, request): # process methods other than CONNECT
Tmp=self.tar getHost. split ('/')
Net = tmp [0] + '//' + tmp [2]
Request = request. replace (net, '') # replace unnecessary parts of the first line
TargetAddr = self. getTargetInfo (tmp [2]) # Call the above function
Try:
(Fam, _, addr) = socket. getaddrinfo (targetAddr [0], targetAddr [1]) [0]
Failed t Exception as e:
Print e
Return
Self.tar get = socket. socket (fam)
Self.tar get. connect (addr) # connect to the target web Service

The 4th step is complete, and the requestafter the 3rd step can be sent to the web server by self.tar get. send (request.
5. In this step, the web server's reponse response is forwarded directly to the client through the proxy service. I used a non-blocking select statement. You can try epoll.
The basic steps are as follows. The methods and functions used can be improved. For example, the main function uses multiple threads or multi-process. How to choose ......
But the idea is similar. If you want to test, chrome installs the SwitchySharp plug-in and sets the proxy port to 8083;
Firefox plug-in autoproxy.
The handling of connect is still in progress (it would be better if there is a Boyou to help), so now this agent does not support https protocol.
The proxy service can obtain all the information about the http protocol. To learn more about http, using the proxy server is a good method.
The following code is attached:
Copy codeThe Code is as follows:
#-*-Coding: UTF-8 -*-
Import socket, select
Import sys
Import thread
From multiprocessing import Process
Class Proxy:
Def _ init _ (self, soc ):
Self. client, _ = soc. accept ()
Self.tar get = None
Self. request_url = None
Self. BUFSIZE = 4096
Self. method = None
Self.tar getHost = None
Def getClientRequest (self ):
Request = self. client. recv (self. BUFSIZE)
If not request:
Return None
Cn = request. find ('\ n ')
FirstLine = request [: cn]
Print firstLine [: len (firstLine)-9]
Line = firstLine. split ()
Self. method = line [0]
Self.tar getHost = line [1]
Return request
Def commonMethod (self, request ):
Tmp=self.tar getHost. split ('/')
Net = tmp [0] + '//' + tmp [2]
Request = request. replace (net ,'')
TargetAddr = self. getTargetInfo (tmp [2])
Try:
(Fam, _, addr) = socket. getaddrinfo (targetAddr [0], targetAddr [1]) [0]
Failed t Exception as e:
Print e
Return
Self.tar get = socket. socket (fam)
Self.tar get. connect (addr)
Self.tar get. send (request)
Self. nonblocking ()
Def connectMethod (self, request): # CONNECT processing can be added here
Pass
Def run (self ):
Request = self. getClientRequest ()
If request:
If self. method in ['get', 'post', 'put', "DELETE", 'have ']:
Self. commonMethod (request)
Elif self. method = 'connect ':
Self. connectMethod (request)
Def nonblocking (self ):
Inputsregistry.self.client,self.tar get]
While True:
Readable, writeable, errs = select. select (inputs, [], inputs, 3)
If errs:
Break
For soc in readable:
Data = soc. recv (self. BUFSIZE)
If data:
If soc is self. client:
Self.tar get. send (data)
Elif soc is self.tar get:
Self. client. send (data)
Else:
Break
Self. client. close ()
Self.tar get. close ()
Def getTargetInfo (self, host ):
Port = 0
Site = None
If ':' in host:
Tmp = host. split (':')
Site = tmp [0]
Port = int (tmp [1])
Else:
Site = host
Port = 80
Return site, port
If _ name __= = '_ main __':
Host = '2017. 0.0.1'
Port = 8083
Backlog = 5
Server = socket. socket (socket. AF_INET, socket. SOCK_STREAM)
Server. setsockopt (socket. SOL_SOCKET, socket. SO_REUSEADDR, 1)
Server. bind (host, port ))
Server. listen (5)
While True:
Thread. start_new_thread (Proxy (server). run ,())
# P = Process (target = Proxy (server). run, args = () # multi-Process
# P. start ()