Write an HTTP proxy with Golang, can catch the packet and scientific internet

This is a creation in Article, where the information may have evolved or changed.

Write an HTTP proxy with Golang, can catch the packet and scientific internet

Origin

Because of the need to restrict the Internet at work, just let our app surf the Internet, meaning that all the domain names or IPs requested by the app are released, and other domain names are banned, so I need to grab the app's HTTP request. Internet search, win under the Fiddler good, but I use a Linux system, fiddler not cross-platform, looked for under the Linux capture Package software, of course tcpdump and Wireshare is strong enough, can achieve my small requirements, But the use of a certain complexity, the threshold is slightly higher. Find other similar software on the Internet quite a lot, Charles, Nproxy and so on, finally found that mitmproxy most in line with my appetite,

But Mitmproxy installation relies too much, Python is so, accidentally error, so, think of using Golang to achieve similar functions, I also like the go language, if you can quietly write your favorite code, is how happy. Think well on the practice, let's do it, of course everything has a simple to complex process, the following features slowly increased.

Characteristics

Features, or functions, the following are implementations or features that will be implemented

• [X] HTTP Proxy
• [X] Crawling of HTTP request Responses
• [] Modify the HTTP request
• [] Repeat Request
• [] Simultaneous monitoring of multiport ports
• [] Support SOCKS5, WebSocket, HTTPS protocol
• [] interface supports both terminal and Web Forms

Installation

git clone https://github.com/sheepbao/gomitmproxy.gitcd

Use

• HTTP Proxy

No parameters, representing HTTP proxy, default port 8080, change port with-port

• HTTP Grab Bag

Plus-m parameter, which indicates crawling of HTTP requests and responses

• HTTP Proxy Science Online

  First you have to have a server outside the wall, such as Ali Hong Kong servers, for the server in the diagram, assuming its IP address is: 22.222.222.222

8888

808022.222.222.222:8888

Then the browser set proxy, IP is localhost, port is 8080, you can achieve scientific Internet access

A brief analysis of source code

对于网络编程，Anything is a socket!实现http代理并不难，简单地说就是用代理服务器代替客户端去请求web服务，然后在把请求的结果返回给客户端。先来个：

    ifReq. Method = ="CONNECT"{b: = []byte("http/1.1 Connection established\r\n"+"proxy-agent:golang_proxy/"+ Version +"\r\n\r\n") _, Err: = conn. Write (b)ifErr! =Nil{logger. Println ("Write Connect err:", err)return}    }Else{req. Header.del ("Proxy-connection") Req. Header.set ("Connection","Keep-alive")ifErr = req. Write (Conn_proxy); Err! =Nil{logger. Println ("Send to server Err", err)return}    }

3. gomitmproxy连接服务器的host，并将客户端的请求发送给服务器，如果没加 -m 参数，那直接将客户端和服务器的io口通过gomitmproxy连接，gomitmproxy很像水管的连接器，把断开的水管连接起来。

//two IO ports connection  func  Transport (conn1, conn2 net. Conn) (err error) {Rchan: = make  (chan  error,< Span class= "Hljs-number" > 1 ) Wchan: = make  ( Chan  error, 1 ) go  mycopy (conn1, CONN2,    Wchan) go  mycopy (conn2, Conn1, Rchan) select  {    case  err = <-wchan: case  err = <-rchan: } return }func  mycopy (src io. Reader, DST io. Writer, ch chan  <-Error) {_, Err: = Io. Copy (DST, src) ch <-err}  

如果加了 -m 参数，表示要截取http请求和响应。

//Print HTTP requests and responsesfuncHttpdump (req *http. Request, resp *http. Response) {deferResp. Body.close ()varRespstatusstrstringRespstatus: = resp. StatusCode Respstatusheader: =int(Math. Floor (float64(Respstatus/ -)))SwitchRespstatusheader { Case2: Respstatusstr = Green ("<--"+ StrConv. Itoa (Respstatus)) Case3: Respstatusstr = Yellow ("<--"+ StrConv. Itoa (Respstatus)) Case4: Respstatusstr = Magenta ("<--"+ StrConv. Itoa (Respstatus)) Case5: Respstatusstr = Red ("<--"+ StrConv. Itoa (Respstatus))} FMT. Println (Green ("Request:") FMT. Printf ('%s%s%s\n ', Blue (req. Method), req. RequestUri, RESPSTATUSSTR) forHeadername, Headercontext: =RangeReq. Header {fmt. Printf ("%s:%s\n", Blue (Headername), Headercontext)} fmt. Println (Green ("Response:")) forHeadername, Headercontext: =RangeResp. Header {fmt. Printf ("%s:%s\n", Blue (Headername), Headercontext)} respbody, err: = Ioutil. ReadAll (resp. Body)ifErr! =Nil{logger. Println ("Read RESP body err:", err)}Else{acceptencode: = resp. header["Content-encoding"]varRespbodybin bytes. Buffer w: = Bufio. Newwriter (&respbodybin) w.write (respbody) W.flush () for_, Compress: =RangeAcceptencode {SwitchCompress { Case "gzip": r, Err: = gzip. Newreader (&respbodybin)ifErr! =Nil{logger. Println ("gzip Reader err:", err)}Else{deferR.close () respbody, _ = Ioutil. ReadAll (R)} Break             Case "Deflate": r: = Flate. Newreader (&respbodybin)deferR.close () respbody, _ = Ioutil. ReadAll (R) Break}} FMT. Printf ("%s\n",string(Respbody)) } FMT. Printf ("%s%s%s\n", Black ("####################"), Cyan ("END"), Black ("####################"))}

4. gomitmproxy将请求服务器的响应发送给客户端。

        resp, err := http.ReadResponse(bufio.NewReader(conn_proxy), req)        ifnil {            logger.Println("read response err:", err)            return        }        true)        ifnil {            logger.Println("respDump err:", err)        }        _, err = conn.Write(respDump)        ifnil {            logger.Println("conn write err:", err)        }

Summarize

Source stacking on GitHub, you are welcome to guide and contribute!

  The more you know, the more you know your don ' t know!