Write HTTP middleware in go language

This is a creation in Article, where the information may have evolved or changed. This article by Bole Online-codefor translation. without permission, no reprint!
English Source: Justinas. Welcome to join the translation team.

In the process of Web development, middleware generally refers to the components of the application that encapsulate the original information and add additional functionality. I don't know why, middleware is often a less popular concept. But I think it's terrific.

First, a good middleware has a single function, pluggable and self-disciplined. This means that you can put it into the application at the interface level and work well. Middleware doesn't affect your code style, it's not a framework, it's just an extra layer of processing your request process. There is no need to rewrite the code: If you want to use a middleware, add it to the app; If you change your mind, just get rid of it. It's so simple.

Take a look at Go,HTTP Middleware is very popular, and so is the standard library. Perhaps it doesn't seem obvious that the functions in the Net/http package, such as Stripprefix and Timeouthandler It is the middleware we have defined above: encapsulating the process and adding additional actions when processing input or output.

My recent Go pack Nosurf is also a middleware. I have deliberately designed this from the outset. In most cases, you don't have to care about CSRF checks at the application level at all . Nosurf, like other middleware, is very independent and can be used with tools that implement the standard library net/http interface.

You can also use middleware to do these:

• Mitigate BREACH attacks with hidden lengths
• Frequency limit
• Block Malicious automated programs
• Provide debugging information
• Add HSTs, X-frame-options head
• Gracefully recover from the anomaly
• and many others.

Write a simple middleware

In the first example, I wrote a middleware that only allows the user to access the server from a specific domain (with domain information in the HTTP Host header). Such middleware can protect applications from " host spoofing Attacks"

Defining types

For convenience, let's define a type for this middleware, called singlehost.

Type singlehost struct {    handler     http. Handler    Allowedhost String}

Contains only two fields:

• The encapsulated Handler. If it is a valid Host Access, we call this Handler.
• The allowed host value.

Since we have the field name lowercase, it makes the field visible only to our own package. We should also write an initialization function.

Func newsinglehost (Handler http. Handler, allowedhost String) *singlehost {    return &singlehost{handler:handler, Allowedhost:allowedhost}}

Processing requests

Now is the actual logic. In order to implement HTTP. Handler, our type Order implements a method:

Type Handler Interface {        servehttp (Responsewriter, *request)}

This is how we implement it:

Func (S *singlehost) servehttp (w http. Responsewriter, R *http. Request) {    Host: = R.host    If host = = S.allowedhost {        s.handler.servehttp (W, R)    } else {        W. Writeheader (403)    }}

The Servehttp function simply checks the Host header in the request:

• If the Host header matches the allowedhost of the initialization function setting , the servehttp method that encapsulates the handler is called .
• If the Host header does not match, the 403 Status code (forbidden) is returned.

In the latter case, the servehttp method that encapsulates handler is not called at all. So the encapsulated handler doesn't have any output at all, and in fact it doesn't even know that there's a request coming.

Now we have finished our own middleware to put it into the application. This time we do not put the Handler directly into the net/http service, but first packaged the Handler into the middleware.

singlehosted = Newsinglehost (MyHandler, "example.com") http. Listenandserve (": 8080", singlehosted)

A different approach

The middleware we just wrote is simply too simple, just a few lines of code. In order to write such a middleware, a less general approach is introduced. Because Go supports the first type and the closure of functions, and has a concise http. Handlerfunc Wrapper, we can implement it as a simple function instead of writing a separate type. The following is a function-based version of the middleware.

func singlehost (handler http. Handler, allowedhost string) http. Handler {ourfunc: = func (w http. Responsewriter, R *http. Request) {host: = R.host if host = = Allowedhost {handler. Servehttp (W, R)} else {W.writeheader (403)}} return HTTP. Handlerfunc (Ourfunc)} 

Here we declare a simple function called singlehost , which accepts a Handler and allows the host name. Inside the function, we created a function similar to the previous version of servehttp . This intrinsic function is actually a closure, so it can be accessed from outside the singlehost . Finally, we use this function as http through handlerfunc . Handler.

Use Handler or define an http. The Handler type depends entirely on you. For a simple case, a function is sufficient. But with the complexity of middleware functionality, you should consider defining your own data structure and putting logic in separate ways.

In fact, both methods of the standard library are used. Stripprefix is a function that returns HANDLERFUNC. Although Timeouthandler is also a function, it returns a custom type that handles requests.

more complicated situations.

Our singlehost Middleware is very simple: check one of the properties of the request first, then either do nothing, pass the request directly to the encapsulated Handler,or return a response on your own, and never let the encapsulated Handler processing this request. However, in some cases, not only will trigger some actions based on the request, but also do some finishing work after the encapsulation Handler processing, such as modifying the response content.

Easy to add data

If we want to add some data after the contents of the encapsulated handler output, we just need to continue calling Write () after the handler is finished :

Type appendmiddleware struct {    handler http. Handler}func (a *appendmiddleware) servehttp (w http. Responsewriter, R *http. Request) {    a.handler.servehttp (W, R)    w.write ([]byte ("middleware says Hello.")}

The response content should now contain the contents of the encapsulated handler , plus middleware says hello.

Problem is

Doing other response content operation is more troublesome. For example, if we want to write some data before responding to the content. If we call Write ()before the encapsulated handler , then the encapsulated handler is good for losing the HTTP Status Codes and HTTP control of the head. Because the first call to Write () outputs the header directly.

To modify the original output (for example, to replace some of these strings), change the specific HTTP headers, and set different status codes are not possible for the same reason: when the encapsulated handler returns, The above data has already been sent to the client.

In order to deal with this requirement, we need a special responsewriter that can be used as buffer , which can collect, hold output for modification and so on, and then send it to the client. We can pass this buffer Responsewriter to the encapsulated handler, not the real RW, This avoids sending data directly to the client.

Fortunately, Such a tool does exist in the Go standard library. the responserecorder in Net/http/httptest is this: it saves the status code, a dictionary that holds the response header, and accumulates the output in buffer the. Although it was used for testing (the package name implies this), it is a good fit for our needs.

Let's look at an example of using responserecorder

type modifiermiddleware struct {handler http. Handler}func (M *modifiermiddleware) servehttp (w http. Responsewriter, R *http. Request) {rec: = Httptest. Newrecorder ()//passing a responserecorder instead of the original RW m.handler.servehttp (REC, R)//after this Finishes, we have the response recorded//and can modify it before copying it to the original RW//We copy the Ori Ginal headers first for k, V: = Range rec. Header () {W.header () [k] = v}//and set an additional one w.header (). Set ("X-we-modified-this", "Yup")//and the status code, as this call writes out the headers W.writeheader (41    8)//The body hasn ' t been written (to the real RW) yet,//So we can prepend some data.    W.write ([]byte ("middleware says hello again.")) Then write out the original body W.write (rec. Body.bytes ())} 

Here is the output of our packaged handler . Without our middleware package, the original handler will only output Success!.

http/1.1 418 I ' m a teapotx-we-modified-this:yupcontent-type:text/plain; Charset=utf-8content-length:37date:tue, Sep 18:41:39 gmtmiddleware says hello again. success!

This approach provides a great convenience. The encapsulated handler is now completely under our control: even after its return, we can manipulate the output in any way.

share data with other handlers

In different situations, middleware may need to expose specific information to other middleware or applications. For example,Nosurf needs to provide the user with a way to get the CSRF Key and the cause of the error (if there is an error).

A suitable model for this requirement is to use a hidden mapthat will be http. The request pointer points to the data that is needed, and then exposes a package-level (handler level) function to access the data.

I also used this model in Nosurf. Here, I created a global context map. Note that because the Go map is not secure by default, a mutexis required.

Type csrfcontext struct {    token string    reason Error}var (    contextmap = make (map[*http. Request]*csrfcontext)    Cmmutex    = new (sync. Rwmutex))

Use handler to set the data, and then get the data through the exposed function Token () .

Func Token (req *http. Request) string {    Cmmutex.rlock ()    defer cmmutex.runlock ()    ctx, OK: = Contextmap[req]    if!ok {            Return ""    }    return Ctx.token}

You can find the complete implementation in the Nosurf code base context.go.

Although I chose to implement this requirement myself in Nosurf, there is actually a handygorilla/context package that implements a generic mapof the Save request information. In most cases, this package is sufficient to meet your needs and prevent you from stepping on the pit when you implement a shared storage. It even has its own middleware that cleans up request information after the request process is over.

Summarize

The purpose of this article is to draw the attention of go users to the concept of middleware and to demonstrate Some of the basic components of using go writing middleware. Although go is a relatively young development language,go has a very nice standard HTTP interface . This is one of the reasons why writing middleware with Go is a very simple and even enjoyable process.

However, there is still a lack of high-quality HTTP tools for Go . Most of the Go middleware Ideas I mentioned before have not been implemented. Now that you know how to write middleware with Go, why not make one yourself?

PS, you can find all the middleware examples in this article in a GitHub gist.