The principle of Agent service is very simple, take the browser and Web server. It's a browser.
Send request to B agent, b agent and then send request to C Web services, then C reponse->b->a.
To write a Web proxy service you must first understand the HTTP protocol, of course, not much more in-depth, unless you want to achieve a powerful function: Modify the XX information,
Load balancing and so on. The HTTP request consists of three parts: the request line, the message header, the request body;
Detailed online have, want to know can look. Below is a normal GET request header (the cookie part does not screen itself, the system W7 used):
You can see the first line: Get is the request method,/is the path, after the protocol version, the second row is the request header, are key value pairs;
The Get method has no body. Post has the body, in addition, the request method head is basically consistent, each line end is \ r \ n.
The basic request method is as follows:
Get request gets the resource identified by Request-uri
Post appends new data to the resource identified by Request-uri
The head request gets the response message header of the resource identified by Request-uri
The put request server stores a resource and uses Request-uri as its identity
Delete requests that the server delete the resources identified by the Request-uri
TRACE requests that the server echo the requested information received, primarily for testing or diagnostics
CONNECT reserved for future use
Options request query server performance, or query resource-related choices and requirements
However, after using the proxy, the request from the agent service is as follows:
Compared to the first picture, what's the difference ... The resource path for the first row is incorrect. When you set up a proxy request on the browser, the entire URL as a resource path, so we want to delete the domain name, and then the proxy server in the modified request sent to the target
Web server. It is so simple, of course, connect method special, want special treatment, so first say other method.
The basic idea:
1, the proxy server runs the monitor, when has the client browser request arrives, through accept () obtains client handle (or is called the descriptor);
2, using the client descriptor to receive the request from the browser, separated out the first act to modify the first line and get method,
To remove the part of the http://, remove the part of the Targethost to express it.
3, through the 2nd step to obtain methods method, request and Targethost, this step can be based on different ways to do different processing,
Because get, poet, put, delete and so on except connect processing basically consistent, therefore handles the first line, for example:
Copy Code code as follows:
Get http://www.a.com/HTTP/1.1
Replaced by
get/http/1.1
At this point targethost is the red part, the default request 80 port, at this time port is 80; If there are ports in targethost (such as www.a.com:8081),
will be the sub-branches port, at this time port is 8081. It then connects to target server based on Targethost and port, implementing the following code:
Copy Code code as follows:
def gettargetinfo (self,host): #处理targetHost获得网址和端口, as the return value.
Port=0
Site=none
If ': ' In Host:
Tmp=host.split (': ')
SITE=TMP[0]
Port=int (Tmp[1])
Else
Site=host
Port=80
Return Site,port
def commonmethod (self,request): #处理除CONNECT以外的方法
Tmp=self.targethost.split ('/')
net=tmp[0]+ '//' +tmp[2]
Request=request.replace (NET, "") #替换掉首行不必要的部分
Targetaddr=self.gettargetinfo (tmp[2]) #调用上面的函数
Try
(FAM,_,_,_,ADDR) =socket.getaddrinfo (targetaddr[0],targetaddr[1]) [0]
Except Exception as E:
Print E
Return
Self.target=socket.socket (FAM)
Self.target.connect (addr) #连接到目标web服务
4, this step is good to do, according to the third step after processing request can be self.target.send (request) sent to the Web server.
5, this step of the Web server reponse response through the proxy service directly to the client line, I used a non-blocking select, you can try Epoll.
The basic step is this, the use of method functions can be improved, such as the main function part of the use of multithreading or multiple processes, how to choose ...
But that's pretty much the way it goes. If you want to test, chrome install Switchysharp plug-in, set up, agent Port 8083;
Firefox plugin autoproxy.
The handling of connect is still in the process (if you have Bo friends to help better), so now this agent does not support HTTPS protocol.
The proxy service can get all the information about the HTTP protocol, and it's a good idea to learn HTTP and use a proxy server.
The following code is attached
Copy Code code as follows:
#-*-Coding:utf-8-*-
Import Socket,select
Import Sys
Import Thread
From multiprocessing import Process
Class Proxy:
def __init__ (SELF,SOC):
Self.client,_=soc.accept ()
Self.target=none
Self.request_url=none
Self. bufsize=4096
Self.method=none
Self.targethost=none
def getclientrequest (self):
Request=self.client.recv (self. BufSize)
If not request:
Return None
Cn=request.find (' \ n ')
FIRSTLINE=REQUEST[:CN]
Print Firstline[:len (Firstline)-9]
Line=firstline.split ()
SELF.METHOD=LINE[0]
SELF.TARGETHOST=LINE[1]
Return request
def commonmethod (self,request):
Tmp=self.targethost.split ('/')
net=tmp[0]+ '//' +tmp[2]
Request=request.replace (NET, "")
Targetaddr=self.gettargetinfo (Tmp[2])
Try
(FAM,_,_,_,ADDR) =socket.getaddrinfo (targetaddr[0],targetaddr[1]) [0]
Except Exception as E:
Print E
Return
Self.target=socket.socket (FAM)
Self.target.connect (addr)
Self.target.send (Request)
Self.nonblocking ()
def connectmethod (self,request): #对于CONNECT处理可以添加在这里
Pass
def run (self):
Request=self.getclientrequest ()
If Request:
If self.method in [' Get ', ' POST ', ' put ', ' DELETE ', ' HAVE ']:
Self.commonmethod (Request)
Elif self.method== ' CONNECT ':
Self.connectmethod (Request)
def nonblocking (self):
Inputs=[self.client,self.target]
While True:
Readable,writeable,errs=select.select (inputs,[],inputs,3)
If errs:
Break
For Soc in readable:
Data=soc.recv (self. BufSize)
If data:
If Soc is self.client:
Self.target.send (data)
Elif Soc is self.target:
Self.client.send (data)
Else
Break
Self.client.close ()
Self.target.close ()
def gettargetinfo (self,host):
Port=0
Site=none
If ': ' In Host:
Tmp=host.split (': ')
SITE=TMP[0]
Port=int (Tmp[1])
Else
Site=host
Port=80
Return Site,port
If __name__== ' __main__ ':
Host = ' 127.0.0.1 '
Port = 8083
Backlog = 5
Server = Socket.socket (socket.af_inet,socket. SOCK_STREAM)
Server.setsockopt (socket. Sol_socket,socket. so_reuseaddr,1)
Server.bind ((Host,port))
Server.listen (5)
While True:
Thread.start_new_thread (Proxy (server). Run, ())
# p=process (target=proxy (server). Run, args= ()) #多进程
# P.start ()
