LIGHTTPD Integrated Golang

This is a creation in Article, where the information may have evolved or changed.

LIGHTTPD Integrated Golang

Author:  nullchenemail:   526624974@qq.com

Brief introduction:

Our business with fastcgi do access, the time to study the next fcgi how to run on the Httpserver, mainly fcgi and httpserver communication, here a simple record. Because Qzhttp is non-open source, here to lighttpd-bit object, to learn. This article is divided into two parts, the first part of the simple analysis lighttpd how to communicate with the FASTCGI application, on this basis, the second part of the first part of the conclusion of the verification, a simple golang with fastcgi way to integrate into the LIGHTTPD, Let LIGHTTPD management Golang process, so that multi-process + multi-threaded + multi-coprocessor mode.

The first part

For the sake of clarity, we first perform the following roles by function:

Fastcgi client:
     The function of the fastcgi client is mainly to convert the received http request into a fastcgi request and send it to fastcgiserver. After receiving the response, it converts the response into http format and sends it to the http requester. Lighttpd has the function of fastcgi client.

Fastcgi server:
     The main function of fastcgi server is mainly to manage the process. And execute fastcgi application in each process. This allows the fastcgi application to focus on the business logic. Lighttpd has the function of fastcgi server.

Fastcgi application:
     The main function of this role is to accept the request sent by fastcgi server (lighttpd), decode according to fastcgi protocol (generally provided by the library, for example, we write fcgi is the library used), and do business logic processing (business logic processing part Generally refers to the part of the fcgi code we usually write).

Based on the LIGHTTPD fastcgi module and the Pyhton Flup module, we explored how the fastcgi program written by Python works in lighttpd.

Let's start by looking at a python implementation of the Hello World fastcgi program as follows:

#!/usr/bin/python
from flup.server.fcgi import WSGIServer

def app(environ, start_response):
    start_response('200 OK', [('Content-Type', 'text/plain')])
    return ['Hello World!\n']

if __name__ == '__main__':
    WSGIServer(app).run()

Save the above code as HELLO_WORLD.FCGI and run it in LIGHTTPD. Obviously, the communication with LIGHTTPD is implemented in the Flup.server.fcgi.WSGIServer module, then we follow up the module further, find the key code as follows (note: The following code excerpt from the Flup module, in order to read the convenience of a lot of streamlining):

FCGI_LISTENSOCK_FILENO = 0 --> Standard input
Sock = socket.fromfd(FCGI_LISTENSOCK_FILENO, socket.AF_INET,
                                     socket.SOCK_STREAM)
While self._keepGoing:
     r, w, e = select.select([sock], [], [], timeout)
     If r:
         clientSock, addr = sock.accept()
         # After the previous step, a connection is established, and then the read event is processed on the connection and decoded according to the fastcgi protocol, and then the request is processed by the business logic to obtain a response, and the response is answered.

Where the _socket.fromfd_ document is as follows (from the Python Standard Library manual)

Socket.fromfd (FD, Family, type[, Proto])

Duplicate the file descriptor fd (an I Nteger as returned by a file object ' s Fileno () method) and build a socket object from the result Address family, socket ty PE and protocol number is as for the socket () function above. The file descriptor should refer to a socket, which is not checked-subsequent operations on the object may fail if th E file descriptor is invalid. This function was
rarely needed, but can be used to get or set socket options on a
socket passed to a program as St Andard input or output (such as a
server started by the Unix inet daemon). The socket is assumed to
being in blocking mode.

As can be seen, thefastcgi application is the standard input (fd=0) for the accept operation to establish a connection, and then read and write. However, we know that the standard input does not support the accept operation, so you can guess that lighttpd will _fastcgi Application_ when starting fastcgi application The standard input is associated to a data structure that supports the accept operation. Along this line of thought we come to the realization of a lighttpd. In Lighttpd's mod_fastcgi.c fcgi_spawn_connection Find the following code:

FCGI_LISTENSOCK_FILENO=0
fcgi_fd = socket(socket_type, SOCK_STREAM, 0))
setsockopt(fcgi_fd, SOL_SOCKET, SO_REUSEADDR, &val, sizeof(val))
bind(fcgi_fd, fcgi_addr, servlen)
listen(fcgi_fd, 1024)

switch ((child = fork())) {
    case 0: {    
        dup2(fcgi_fd, FCGI_LISTENSOCK_FILENO)

        close(fcgi_fd)
        ......
        execve(arg.ptr[0], arg.ptr, env.ptr);    ---->_Fastcgi application_ program is executed here。
    }
}

where *dup2 (FCGI_FD, Fcgi_listensock_fileno) *

This line of code is key, pointing the standard input of the child process to a unixsocket fd in the listen state.
Therefore, it is possible to accept the standard input in the fastcgi application.

Conclusion: Thus we can get a conclusion. FastCGI Server (LIGHTTPD) interacts with FASTCGI application data steps as follows:

* fastcgi server bind, listen a Unix socket to get an fd
* fastcgi server fork
* child process dup2 (fd, stdin) --> point the stdin of the child process to a unix socket in the listen state
* Perform fastcgi application in child process
* fcgi app accept stdin waiting for the connection to arrive and process
* fcgi server connect corresponding unixsocket and send data

Part II

According to the conclusion of the first part. This gives the following Golang code, which can be run in fastcgi mode in LIGHTTPD.

package main
import (
    "net"
    "net/http"
    "net/http/fcgi"
    "os"
    "time"
)

type FastCGIApp struct{}

func (s *FastCGIApp) ServeHTTP(resp http.ResponseWriter, req *http.Request) {

    msg := req.FormValue("msg")
    if msg == "sleep" {
        time.Sleep(time.Second * 60)
    }
    resp.Write([]byte(msg))
}

func main() {

    listener, _ := net.FileListener(os.Stdin)
    app := new(FastCGIApp)
    fcgi.Serve(listener, app)

}

Handling of the above code:

0. Save as try_fastcgi.go
1. go build try_fastcgi.go get the executable file try_fastcgi
2. mv try_fastcgi test.fcgi
3. Place test.fcgi in the position corresponding to lighttpd.

The LIGHTTPD configuration is as follows:

server.modules += ( "mod_fastcgi" )
fastcgi.server = (
    ".fcgi" =>
    ( "fcgi-num-procs" =>
                 (
                   "socket" => socket_dir + "/fastcgi-test.socket",
                   "bin-path" => server_root + "/cgi-bin/test.fcgi",
                   "max-procs" => 1,
                   "min-procs" => 1,
                   "broken-scriptfilename" => "enable",
                 )
    ),
)