How to turn off HTTP connections in Golang How to close Golang ' s HTTP connection

One of our services is to write with go, in the test to find a few hours after it will be core, and the core of the time no stack information, the simple analysis of the service in the number of HTTP services are growing, and our development machine's FD limit is only 1024, When the number of connections to the process that the service belongs to is increased to the system's FD limit, it is killed by the operating system ...

In this service, we will periodically initiate a POST request to an HTTP server because the request is very infrequent, so we want to do it in a short-connect way. The request code might look like this:

func dialtimeout (Network, addr string) (net. Conn, error) {return net. Dialtimeout (Network, addr, time. Second*post_remote_timeout)}func dorequest (URL string) xx, error {transport: = http. transport{Dial:dialtimeout,} client: = http.         client{Transport: &transport,} content: = requestcontent{}//fill content here POSTSTR, err: = json. Marshal (content) if err! = Nil {return nil, err} resp, err: = client. Post (URL, "Application/json", bytes. Newbuffer (POSTSTR)) if err! = Nil {return nil, err} defer resp. Body.close () body, err: = Ioutil. ReadAll (resp. Body) If err! = Nil {return nil, err}//Receive body, handle it} 

After running this code for a while, it will be found that there is a heap of established state connections under the process (see all FD under a process with the lsof-p PID) because each time the DoRequest function is called, a new TCP connection is created. If the peer does not first close the connection (to the end of the fin packet), we even call the resp. The Body.close () function still does not change these connections in the established state . Why is that? Only go to the source code to find out.

The HTTP Client is implemented in the

      Golang net package client.go, Transport.go, Response.go, and request.go files. When the application layer calls the client. Do () function, the transport layer will first look for the cached connection associated with the request (the cache is a Map,map key is the request method, the request address and the proxy address, value is a connection description structure called Persistconn), If there is already an old connection that can be reused, the HTTP request is sent and accepted on the old connection, otherwise a new TCP connection is created and the data is read and written on the connection. When the client accepts the entire response, if the application layer does not have
call response. The Body.close () function, the persistconn that just transferred the data will not be added to the connection cache, so that if you start the next HTTP request, the TCP connection is re-established and the PERSISTCONN structure is redistributed, which is not called response. A side effect of body.close ().
      If response is not called. Body.close () also has a problem. If the connection is closed after the request is complete (fin is sent to me by the HTTP server on the side), if response is not called here. Body.close (), you can see that the status of the TCP connection associated with this request is always in the close_wait state (remember?). Close_wait is the semi-open half-closed state of the connection, it is received by the other side of fin and we also sent an ACK, but the local side has not yet sent fin to the end, if this segment does not call close to close the connection, then the connection will remain in the
Close_wait state and will not be reclaimed by the system.

called the response. Body.close () is it foolproof? The body is also called in the above code. Why does Close () have a lot of established state connections? Because in each invocation of the function dorequest (), we will create a new transport and client structure, and when the HTTP request completes and receives the response, the connection will remain in the established state if the HTTP server to the end does not close the connection. How to solve it?
There are two ways of doing this:
The first method is to use a global client, which only sends data on this global client each time in the function dorequest (). But what if I want to use a short connection? Use method Two.
The second method is to set its disablekeepalives parameter to False when transport is allocated, as follows:

        // ...        Transport: = http. transport{                Dial:              dialtimeout,                disablekeepalives:true,        }        Client: = http. client{                Transport: &transport,        }        //...

As you can see from transport.go:L908, when the application layer calls RESP. Body.close (), if Disablekeepalives is turned on, the transport automatically shuts down the local connection. Instead of adding it to the connection cache.

How to turn off HTTP connections in Golang How to close Golang ' s HTTP connection