Using go to build a restful JSON API

Original Address http://thenewstack.io/make-a-restful-json-api-go/
This article not only discusses how to build restful JSON APIs with go, but also discusses how to design restful APIs. If you've ever had an API that doesn't follow a good design, you'll end up writing rotten code to use the Junk API. Hopefully, after reading this article, you'll be able to get a better understanding of how good APIs should be.

What is the JSON API?

Before JSON, XML is a mainstream text format. I am fortunate that both XML and JSON have been used, and there is no doubt that JSON is the obvious winner. This article does not delve into the concept of JSON API, which can be found in a detailed description in jsonapi.org.

Sponsor Note

SPRINGONE2GX is a conference dedicated to app developers, solutions, and data architects. Topics are specific to the program Ape (yuan), the popular open source technology used by architects, such as: Spring IO Projects,groovy & Grails,cloud Foundry,rabbitmq,redis,geode,hadoop and Tomcat and so on.

A basic web Server

A restful service is essentially first a Web service. The following is an example of the simplest Web server that simply returns a request link for any request:

package mainimport (        "fmt"        "html"        "log"        "net/http")func main() {        http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {            fmt.Fprintf(w, "Hello, %q", html.EscapeString(r.URL.Path))    })    log.Fatal(http.ListenAndServe(":8080", nil))}

Compiling This example will run the server and listen on port 8080. Try using http://localhost:8080 to access the server.

Add a route

When most standard libraries start to support routing, I find that most people don't know how they work. I have used several third-party router in my project. The most impressive is the MUX router in the Gorilla WEB Toolkit.

Another popular router is Julien Schmidt's contribution to Httprouter

package mainimport (        "fmt"        "html"        "log"        "net/http"        "github.com/gorilla/mux")func main() {        router := mux.NewRouter().StrictSlash(true)        router.HandleFunc("/", Index)        log.Fatal(http.ListenAndServe(":8080", router))}func Index(w http.ResponseWriter, r *http.Request) {        fmt.Fprintf(w, "Hello, %q", html.EscapeString(r.URL.Path))}

To run the above example, you first need to install the package "Github.com/gorilla/mux". You can use the command to go get traverse the entire source code to install all the dependencies that are not installed.

Translator Note:

You can also use the go get "github.com/gorilla/mux" Direct install package.

The above example creates a simple router, adds a "/" route, and assigns an index handler response to access to the specified endpoint. This is where you will find that links such as Http://localhost:8080/foo, which are also accessible in the first example, do not work in this example, and this example will only respond to link http://localhost:8080.

Create more basic routes

We already have a route in the previous section and it's time to create more routes. Let's say we're going to create a basic Todo app.

package mainimport (    "fmt"    "log"    "net/http"    "github.com/gorilla/mux")func main() {    router := mux.NewRouter().StrictSlash(true)    router.HandleFunc("/", Index)    router.HandleFunc("/todos", TodoIndex)    router.HandleFunc("/todos/{todoId}", TodoShow)    log.Fatal(http.ListenAndServe(":8080", router))}func Index(w http.ResponseWriter, r *http.Request) {    fmt.Fprintln(w, "Welcome!")}func TodoIndex(w http.ResponseWriter, r *http.Request) {    fmt.Fprintln(w, "Todo Index!")}func TodoShow(w http.ResponseWriter, r *http.Request) {    vars := mux.Vars(r)    todoId := vars["todoId"]    fmt.Fprintln(w, "Todo show:", todoId)}

Now we have added two routes on the basis of the previous example, namely:

• ToDo Index Route:http://localhost:8080/todos
• ToDo Show Route:http://localhost:8080/todos/{todoid}

This is the beginning of a restful design. Note that the last route we added is a variable named todoId . This will allow us to pass the variable to the route and then get the appropriate response record.

Basic style

With routing, you can create some basic todo styles for sending and retrieving data. Use classes in some other languages for this purpose, and use structs in go.

package mainimport “time”type Todo struct {    Name        string    Completed   tool    Due         time.time}type Todos []Todo   

Note:

The last line defines the Todos type Todo of slice. You'll see how to use it later.

Return JSON

Based on the basic style above, we can simulate the real response and list the Todoindex based on the static data.

func TodoIndex(w http.ResponseWriter, r *http.Request) {    todos := Todos{    Todo{Name: "Write presentation"},    Todo{Name: "Host meetup"},    }    json.NewEncoder(w).Encode(todos)}

This creates a static slice of Todos and is encoded in response to a user request. If you visit at this point http://localhost:8080/todos , you will get the following response:

[    {        "Name": "Write presentation",        "Completed": false,        "Due": "0001-01-01T00:00:00Z"    },    {    "Name": "Host meetup",        "Completed": false,        "Due": "0001-01-01T00:00:00Z"    }]

A slightly better style.

You may have discovered that, based on the previous style, Todos returned not a standard JSON packet (the JSON format definition does not contain uppercase letters). Although the problem is a little trivial, we can still solve it:

package mainimport "time"type Todo struct {    Name      string    `json:"name"`    Completed bool      `json:"completed"`    Due       time.Time `json:"due"`}type Todos []Todo

The above code example adds the struct tags on top of the original, which allows you to specify the encoding format of the JSON.

File splitting

We need to do a little refactoring on this project. Now a file contains too much content. We will create several files and reorganize the contents of the files as follows:

• Main.go
• Handlers.go
• Routes.go
• Todo.go

Handlers.go

package mainimport (    "encoding/json"    "fmt"    "net/http"    "github.com/gorilla/mux")func Index(w http.ResponseWriter, r *http.Request) {    fmt.Fprintln(w, "Welcome!")}func TodoIndex(w http.ResponseWriter, r *http.Request) {    todos := Todos{        Todo{Name: "Write presentation"},        Todo{Name: "Host meetup"},    }    if err := json.NewEncoder(w).Encode(todos); err != nil {        panic(err)    }}func TodoShow(w http.ResponseWriter, r *http.Request) {    vars := mux.Vars(r)    todoId := vars["todoId"]    fmt.Fprintln(w, "Todo show:", todoId)}

Routes.go

package mainimport (    "net/http"    "github.com/gorilla/mux")type Route struct {    Name        string    Method      string    Pattern     string    HandlerFunc http.HandlerFunc}type Routes []Routefunc NewRouter() *mux.Router {    router := mux.NewRouter().StrictSlash(true)    for _, route := range routes {        router.        Methods(route.Method).            Path(route.Pattern).            Name(route.Name).            Handler(route.HandlerFunc)    }    return router}var routes = Routes{    Route{        "Index",        "GET",        "/",        Index,    },    Route{        "TodoIndex",        "GET",        "/todos",        TodoIndex,    },    Route{        "TodoShow",        "GET",        "/todos/{todoId}",        TodoShow,    },}

Todo.go

package mainimport "time"type Todo struct {    Name      string    `json:"name"`    Completed bool      `json:"completed"`    Due       time.Time `json:"due"`}type Todos []Todo

Main.go

package mainimport (        "log"        "net/http")func main() {    router := NewRouter()    log.Fatal(http.ListenAndServe(":8080", router))}

Better routing

Part of the refactoring above is the creation of a more detailed route file, with a struct in the new file that contains more detailed information about the route. In particular, we can specify the requested action through this struct, such as GET, POST, delete, and so on.

Log Web Log

In the previous split file, I also have a longer-term consideration. As you'll see later, I'll be able to easily decorate my HTTP handlers with other functions after splitting. In this section we will use this feature to allow our web to log Web Access requests like other modern web sites. In go, there is currently no web logging package, and there is no standard library to provide the appropriate functionality. So we have to implement one ourselves.

On the basis of the previous split file, we create a logger.go new file called and add the following code to the file:

package mainimport (    "log"    "net/http"    "time")func Logger(inner http.Handler, name string) http.Handler {    return http.HandlerFunc(func(w http.ResponseWriter, r       *http.Request) {        start := time.Now()        inner.ServeHTTP(w, r)        log.Printf(            "%s\t%s\t%s\t%s",            r.Method,            r.RequestURI,            name,            time.Since(start),        )    })}

This way, if you visit http://localhost:8080/todos , you will see the following log output in the console.

2014/11/19 12:41:39 GET /todos  TodoIndex       148.324us

Routes file starts crazy ... Continue refactoring

Based on the above split, you will find that the Routes.go file will become bigger and larger as you continue to evolve at this pace. So we continue to split this file. Split it into the following two files:

• Router.go
• Routes.go

Routes.go regression

package mainimport "net/http"type Route struct {    Name        string    Method      string    Pattern     string    HandlerFunc http.HandlerFunc}type Routes []Routevar routes = Routes{    Route{        "Index",        "GET",        "/",        Index,    },    Route{        "TodoIndex",        "GET",        "/todos",        TodoIndex,    },    Route{        "TodoShow",        "GET",        "/todos/{todoId}",        TodoShow,    },}

Router.go

package mainimport (    "net/http"    "github.com/gorilla/mux")func NewRouter() *mux.Router {    router := mux.NewRouter().StrictSlash(true)    for _, route := range routes {        var handler http.Handler        handler = route.HandlerFunc        handler = Logger(handler, route.Name)        router.            Methods(route.Method).            Path(route.Pattern).            Name(route.Name).            Handler(handler)    }    return router}

Do more things.

Now that we have a good template, it is time to reconsider our handlers and let handler do more things. First we add two lines of code to the Todoindex.

func TodoIndex(w http.ResponseWriter, r *http.Request) {    todos := Todos{        Todo{Name: "Write presentation"},        Todo{Name: "Host meetup"},    }    w.Header().Set("Content-Type", "application/json; charset=UTF-8")    w.WriteHeader(http.StatusOK)    if err := json.NewEncoder(w).Encode(todos); err != nil {        panic(err)    }}

The new two lines of code let Todoindex handler do two more things. First, return the JSON that the client expects and tell the content type. Then explicitly set a status code.

The description content type that Net/http server in go does not display in the header will try to guess the content type for us, but it is not always accurate. So when we know the content type, we should always set the type ourselves.

Wait, where's the database?

If we continue to construct restful APIs, we need to consider a place for storing and retrieving data. But this is beyond the scope of this article, so there is a simple implementation of a coarse data store (coarse to no thread-safe mechanism).

Create a file named repo.go , with the following code:

package mainimport "fmt"var currentId intvar todos Todos// Give us some seed datafunc init() {    RepoCreateTodo(Todo{Name: "Write presentation"})    RepoCreateTodo(Todo{Name: "Host meetup"})}func RepoFindTodo(id int) Todo {    for _, t := range todos {        if t.Id == id {            return t        }    }    // return empty Todo if not found    return Todo{}}func RepoCreateTodo(t Todo) Todo {    currentId += 1    t.Id = currentId    todos = append(todos, t)    return t}func RepoDestroyTodo(id int) error {    for i, t := range todos {        if t.Id == id {            todos = append(todos[:i], todos[i+1:]...)            return nil        }    }    return fmt.Errorf("Could not find Todo with id of %d to delete", id)}

Add an ID to todo

Now we have a rough database. We can create an ID for Todo that identifies and insights Todo item. The data structure is updated as follows:

package mainimport "time"type Todo struct {    Id        int       `json:"id"`    Name      string    `json:"name"`    Completed bool      `json:"completed"`    Due       time.Time `json:"due"`}type Todos []Todo

Update Todoindex Handler

After the data is stored in the database, it is not necessary to generate the data in handler and retrieve the database directly from the ID to get the corresponding content. Modify the handler as follows:

func TodoIndex(w http.ResponseWriter, r *http.Request) {    w.Header().Set("Content-Type", "application/json; charset=UTF-8")    w.WriteHeader(http.StatusOK)    if err := json.NewEncoder(w).Encode(todos); err != nil {        panic(err)    }}

Posting JSON

All of the preceding APIs are the corresponding get requests and can only output JSON. This section will add an API to upload and store json. routes.goAdd the following route to the file:

Route{    "TodoCreate",    "POST",    "/todos",    TodoCreate,},

The Create Endpoint

This creates a new router, and now creates a endpoint for the new route. handlers.goAdd handler in file TodoCreate . The code is as follows:

func TodoCreate(w http.ResponseWriter, r *http.Request) {    var todo Todo    body, err := ioutil.ReadAll(io.LimitReader(r.Body, 1048576))    if err != nil {        panic(err)    }    if err := r.Body.Close(); err != nil {        panic(err)    }    if err := json.Unmarshal(body, &todo); err != nil {        w.Header().Set("Content-Type", "application/json;   charset=UTF-8")        w.WriteHeader(422) // unprocessable entity        if err := json.NewEncoder(w).Encode(err); err != nil {            panic(err)        }    }    t := RepoCreateTodo(todo)    w.Header().Set("Content-Type", "application/json;   charset=UTF-8")    w.WriteHeader(http.StatusCreated)    if err := json.NewEncoder(w).Encode(t); err != nil {        panic(err)    }}

In the above code, we first get the body of the user request. Note that we use it when getting the body io.LimitReader , which is a good way to prevent your server from being attacked by malicious means. Imagine if someone sent you a 500GB json.

After the body is read, its contents are decoded into the todo struct. If the decoding fails, we do not just return a status code like ' 422 ', but also return a JSON containing the error message. This enables the client not only to know that an error has occurred, but also to understand where the error occurred.

Finally, if everything goes well, we'll return the status Code 201 to the client, and we'll also return the created entity content to the client, which the client might use later in the operation.

Post JSON

After all the work is done, we can upload the JSON string and test it. Sample and return results are as follows:

curl -H "Content-Type: application/json" -d ‘{"name":"New Todo"}‘ http://localhost:8080/todosNow, if you go to http://localhost/todos we should see the following response:[    {        "id": 1,        "name": "Write presentation",        "completed": false,        "due": "0001-01-01T00:00:00Z"    },    {        "id": 2,        "name": "Host meetup",        "completed": false,        "due": "0001-01-01T00:00:00Z"    },    {        "id": 3,        "name": "New Todo",        "completed": false,        "due": "0001-01-01T00:00:00Z"    }]

The things we haven't done

Now that we have a good start, we have a lot of things to do in the back. Here's what we haven't done yet:

• Versioning-if we need to modify the API, and this will result in significant changes? Maybe we can start with a prefix that adds/v1 for all routes.
• Identity authentication-Unless this is a free/open API, we may need to add some authentication mechanisms. Recommended Learning JSON Web tokens
• Etags-If your build needs to be extended, you may need to implement Etags

What's left?

All the projects are small at the start, but soon the development starts to get out of hand. If I want to take this to the next level and be ready to put it into production, there are additional things to do:

• A lot of refactoring
• Encapsulate these files into packages, such as JSON helpers,decorators,handlers, and so on.
• Test... Yes, this can't be ignored. We haven't done any tests yet, but for a product, this is a must.

How to get the source code

If you want to get the source code for this example, the repo address is here: Https://github.com/corylanou/tns-restful-json-api

Using go to build a restful JSON API