Go language RESTful JSON API creation

This is a creation in Article, where the information may have evolved or changed.

Go language development restful JSON API

RESTful APIs are widely used in Web project development, and this article explains how the go language steps through the RESTful JSON API and also touches on the topic of restful design.

Maybe we used a variety of APIs before, and when we came across a poorly designed API, it was almost like a crash. I hope that after this article, we can have a preliminary understanding of well-designed restful APIs.

What is the JSON API?

JSON, many Web sites use XML for data exchange. If you use XML and then touch JSON, there's no question that you'll find the world so beautiful. This does not go into the introduction of JSON API, interested in reference Jsonapi.

A basic Web server

Basically, restful services are first and foremost Web services. So we can first look at how the basic Web server in the go language is implemented. The following example implements a simple Web server, and for any request, the server responds to the requested URL back.

package mainimport (    "fmt"    "html"    "log"    "net/http")func main() {    http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {        fmt.Fprintf(w, "Hello, %q", html.EscapeString(r.URL.Path))    })    log.Fatal(http.ListenAndServe(":8080", nil))}

The basic Web server above uses the Go Standard library's two basic functions Handlefunc and Listenandserve.

func HandleFunc(pattern string, handler func(ResponseWriter, *Request)) {    DefaultServeMux.HandleFunc(pattern, handler)}func ListenAndServe(addr string, handler Handler) error {    server := &Server{Addr: addr, Handler: handler}    return server.ListenAndServe()}

Running the above basic Web service, you can access http://localhost:8080 directly from the browser.

> go run basic_server.go

Add route

Although the standard library contains router, I find that many people feel confused about how it works. I have used a variety of different third-party router libraries in my projects. The most noteworthy is the MUX router of the Gorilla WEB Toolkit.

Another popular router is from Julien Schmidt's bag called Httprouter.

package mainimport (    "fmt"    "html"    "log"    "net/http"    "github.com/gorilla/mux")func main() {    router := mux.NewRouter().StrictSlash(true)    router.HandleFunc("/", Index)    log.Fatal(http.ListenAndServe(":8080", router))}func Index(w http.ResponseWriter, r *http.Request) {    fmt.Fprintf(w, "Hello, %q", html.EscapeString(r.URL.Path))}

To run the above code, first use go get to get the source code for the MUX router:

> go get github.com/gorilla/mux

The above code creates a basic router that assigns the index processor to the request "/" and executes the index processor when the client requests http://localhost:8080/.

If you are careful enough, you will find that the previous basic Web services access HTTP://LOCALHOST:8080/ABC can respond normally: ' Hello, '/abc ', but after you add a route, you can only access http://localhost:8080. The reason is simple, because we only added the "/" resolution, the other routes are invalid routes, so all are 404.

To create some basic routes

Now that we've joined the route, we can add more routes.

Let's say we're going to create a basic Todo app, so our code goes like this:

package mainimport (    "fmt"    "log"    "net/http"    "github.com/gorilla/mux")func main() {    router := mux.NewRouter().StrictSlash(true)    router.HandleFunc("/", Index)    router.HandleFunc("/todos", TodoIndex)    router.HandleFunc("/todos/{todoId}", TodoShow)    log.Fatal(http.ListenAndServe(":8080", router))}func Index(w http.ResponseWriter, r *http.Request) {    fmt.Fprintln(w, "Welcome!")}func TodoIndex(w http.ResponseWriter, r *http.Request) {    fmt.Fprintln(w, "Todo Index!")}func TodoShow(w http.ResponseWriter, r *http.Request) {    vars := mux.Vars(r)    todoId := vars["todoId"]    fmt.Fprintln(w, "Todo Show:", todoId)}

Here we have added two additional routes: Todos and todos/{todoid}.

This is the beginning of the RESTful API design.

Note that the last route we add a variable to the route is called Todoid.

This allows us to pass the ID to the route and can use the specific record to respond to the request.

Basic model

The route is now ready, and it is time to create the model, which can be used to send and retrieve data. In the go language, model can be implemented using structs, whereas model in other languages is generally implemented using classes.

package mainimport (    "time")type Todo struct {    Name      string    Completed bool    Due       time.Time}type Todos []Todo

Above we define a todo struct, which is used to represent a backlog. In addition, we define a type Todos, which represents a to-do list, an array, or a shard.

You will see later that this will become very useful.

Returns some JSON

We have a basic model, so we can simulate some real responses. We can simulate some static data lists for Todoindex.

package mainimport (    "encoding/json"    "fmt"    "log"    "net/http"    "github.com/gorilla/mux")// ...func TodoIndex(w http.ResponseWriter, r *http.Request) {    todos := Todos{        Todo{Name: "Write presentation"},        Todo{Name: "Host meetup"},    }    json.NewEncoder(w).Encode(todos)}// ...

Now we have created a static Todos shard to respond to the client request. Note that if you request Http://localhost:8080/todos, you will get the following response:

[    {        "Name": "Write presentation",        "Completed": false,        "Due": "0001-01-01T00:00:00Z"    },    {        "Name": "Host meetup",        "Completed": false,        "Due": "0001-01-01T00:00:00Z"    }]

A better model

For seasoned veterans, you may have found a problem. Each key in the response JSON is a first-letter answer, although it seems trivial, but it is not customary to respond to the JSON key capitalization. So here's how you can solve this problem:

type Todo struct {    Name      string    `json:"name"`    Completed bool      `json:"completed"`    Due       time.Time `json:"due"`}

In fact, it is very simple to add a tag attribute to the struct, which gives you complete control over how the struct is organized (marshalled) into JSON.

Split code

So far, all of our code is in one file. Seems messy, it's time to split the code. We can split the code into the following multiple files as a function.

We are going to create the following file and then move the code to the specific code file:

• Main.go: Program entry file.
• Handlers.go: route-dependent processor.
• Routes.go: Route.
• Todo.go:todo the relevant code.

  Package Mainimport ("Encoding/json" "FMT" "Net/http" "Github.com/gorilla/mux") Func Index (w http. Responsewriter, R *http. Request) {fmt. Fprintln (W, "welcome!")} Func Todoindex (w http. Responsewriter, R *http.  Request) {todos: = todos{todo{name: "Write Presentation"}, Todo{name: "Host Meetup"},} if err: = Json. Newencoder (W). Encode (Todos); Err! = Nil {panic (err)}}func todoshow (w http. Responsewriter, R *http. Request) {vars: = Mux. Vars (r) Todoid: = vars["Todoid"] fmt. Fprintln (W, "Todo Show:", Todoid)}  

package mainimport (    "net/http"    "github.com/gorilla/mux")type Route struct {    Name        string    Method      string    Pattern     string    HandlerFunc http.HandlerFunc}type Routes []Routefunc NewRouter() *mux.Router {    router := mux.NewRouter().StrictSlash(true)    for _, route := range routes {        router.            Methods(route.Method).            Path(route.Pattern).            Name(route.Name).            Handler(route.HandlerFunc)    }    return router}var routes = Routes{    Route{        "Index",        "GET",        "/",        Index,    },    Route{        "TodoIndex",        "GET",        "/todos",        TodoIndex,    },    Route{        "TodoShow",        "GET",        "/todos/{todoId}",        TodoShow,    },}

package mainimport "time"type Todo struct {    Name      string    `json:"name"`    Completed bool      `json:"completed"`    Due       time.Time `json:"due"`}type Todos []Todo

package mainimport (    "log"    "net/http")func main() {    router := NewRouter()    log.Fatal(http.ListenAndServe(":8080", router))}

Better routing.

In our refactoring process, we created a more versatile routes file. This new file takes advantage of a structure that contains multiple information about the route. Note that here we can specify the type of request, such as GET, POST, delete, and so on.

Output Web Log

In the split routing file, I also included an ulterior motive. As you'll see later, it's easy to use another function to decorate the HTTP processor after splitting.

First we need to have the ability to log Web requests, just like many popular Web servers do. In the go language, there is no Web Log package or feature in the standard library, so we need to create it ourselves.

package loggerimport (    "log"    "net/http"    "time")func Logger(inner http.Handler, name string) http.Handler {    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {        start := time.Now()        inner.ServeHTTP(w, r)        log.Printf(            "%s\t%s\t%s\t%s",            r.Method,            r.RequestURI,            name,            time.Since(start),        )    })}

Above we define a logger function that can be used to wrap the handler.

This is a very standard idiom in the go language. In fact, it is also the idiomatic way of functional programming. Very effective, we just need to pass handler into the function, then it will wrap the incoming handler, add Web logs and time-consuming statistics.

Apply the Logger decorator

To apply the logger modifier, we can create router, we simply package all our current routes into it, and the Newrouter function is modified as follows:

func NewRouter() *mux.Router {    router := mux.NewRouter().StrictSlash(true)    for _, route := range routes {        var handler http.Handler        handler = route.HandlerFunc        handler = Logger(handler, route.Name)        router.            Methods(route.Method).            Path(route.Pattern).            Name(route.Name).            Handler(handler)    }    return router}

Now run our program again and we can see that the log is probably as follows:

2014/11/19 12:41:39 GET /todos  TodoIndex       148.324us

This routing file is crazy ... Let's refactor it.

The routing routes file has now become slightly larger, and we'll break it down into multiple files:

• Routes.go
• Router.go

package mainimport "net/http"type Route struct {    Name        string    Method      string    Pattern     string    HandlerFunc http.HandlerFunc}type Routes []Routevar routes = Routes{    Route{        "Index",        "GET",        "/",        Index,    },    Route{        "TodoIndex",        "GET",        "/todos",        TodoIndex,    },    Route{        "TodoShow",        "GET",        "/todos/{todoId}",        TodoShow,    },}

package mainimport (    "net/http"    "github.com/gorilla/mux")func NewRouter() *mux.Router {    router := mux.NewRouter().StrictSlash(true)    for _, route := range routes {        var handler http.Handler        handler = route.HandlerFunc        handler = Logger(handler, route.Name)        router.            Methods(route.Method).            Path(route.Pattern).            Name(route.Name).            Handler(handler)    }    return router}

and take some responsibility.

So far, we've got some pretty good boilerplate code (boilerplate), and it's time to revisit our processor. We need a little more responsibility. First modify the Todoindex and add the following two lines of code:

func TodoIndex(w http.ResponseWriter, r *http.Request) {    todos := Todos{        Todo{Name: "Write presentation"},        Todo{Name: "Host meetup"},    }    w.Header().Set("Content-Type", "application/json; charset=UTF-8")    w.WriteHeader(http.StatusOK)    if err := json.NewEncoder(w).Encode(todos); err != nil {        panic(err)    }}

Two things have happened here. First, we set the response type and tell the client that it expects to accept JSON. Second, we explicitly set the response status code.

The net/http server in the go language tries to guess the output content type for us (though not always accurate), but since we know exactly what the response type is, we should always set it ourselves.

Wait a moment, where is our database?

Obviously, if we were to create a restful API, we needed some place to store and retrieve data. However, this is not within the scope of this article, so we will simply create a very primitive simulation database (non-thread-safe).

We create a repo.go file with the following content:

package mainimport "fmt"var currentId intvar todos Todos// Give us some seed datafunc init() {    RepoCreateTodo(Todo{Name: "Write presentation"})    RepoCreateTodo(Todo{Name: "Host meetup"})}func RepoFindTodo(id int) Todo {    for _, t := range todos {        if t.Id == id {            return t        }    }    // return empty Todo if not found    return Todo{}}func RepoCreateTodo(t Todo) Todo {    currentId += 1    t.Id = currentId    todos = append(todos, t)    return t}func RepoDestroyTodo(id int) error {    for i, t := range todos {        if t.Id == id {            todos = append(todos[:i], todos[i+1:]...)            return nil        }    }    return fmt.Errorf("Could not find Todo with id of %d to delete", id)}

Add ID to Todo

We created the simulation database, we used and gave the ID, so we need to update our TODO struct accordingly.

package mainimport "time"type Todo struct {    Id        int       `json:"id"`    Name      string    `json:"name"`    Completed bool      `json:"completed"`    Due       time.Time `json:"due"`}type Todos []Todo

Update our Todoindex

To use the database, we need to retrieve the data in Todoindex. Modify the code as follows:

func TodoIndex(w http.ResponseWriter, r *http.Request) {    w.Header().Set("Content-Type", "application/json; charset=UTF-8")    w.WriteHeader(http.StatusOK)    if err := json.NewEncoder(w).Encode(todos); err != nil {        panic(err)    }}

POST JSON

So far, we just output json, and it's time to go into storing some JSON.

Add the following route to the Routes.go file:

Route{    "TodoCreate",    "POST",    "/todos",    TodoCreate,},

Create route

func TodoCreate(w http.ResponseWriter, r *http.Request) {    var todo Todo    body, err := ioutil.ReadAll(io.LimitReader(r.Body, 1048576))    if err != nil {        panic(err)    }    if err := r.Body.Close(); err != nil {        panic(err)    }    if err := json.Unmarshal(body, &todo); err != nil {        w.Header().Set("Content-Type", "application/json; charset=UTF-8")        w.WriteHeader(422) // unprocessable entity        if err := json.NewEncoder(w).Encode(err); err != nil {            panic(err)        }    }    t := RepoCreateTodo(todo)    w.Header().Set("Content-Type", "application/json; charset=UTF-8")    w.WriteHeader(http.StatusCreated)    if err := json.NewEncoder(w).Encode(t); err != nil {        panic(err)    }}

First we open the requested body. Note that we use IO. Limitreader. This is a good way to protect your servers from malicious attacks. What if someone wants to send 500GB JSON to your server?

After we read the body, we deconstruct the todo struct. If it fails, we make the correct response, using the appropriate response code 422, but we still use the JSON response back. This allows the client to understand that the error has occurred and has a way of knowing exactly what happened.

Finally, if all is passed, we respond to the 201 status code, which indicates that the requested entity was created successfully. We also respond back to the JSON that represents the entity we created, which contains an ID that the client might need to use next.

Post some JSON

Now that we have the pseudo repo and the create route, we need to post some data. We use curl to achieve this by using the following command:

curl -H "Content-Type: application/json" -d '{"name": "New Todo"}' http://localhost:8080/todos

If you visit again through Http://localhost:8080/todos, you will probably get the following response:

[    {        "id": 1,        "name": "Write presentation",        "completed": false,        "due": "0001-01-01T00:00:00Z"    },    {        "id": 2,        "name": "Host meetup",        "completed": false,        "due": "0001-01-01T00:00:00Z"    },    {        "id": 3,        "name": "New Todo",        "completed": false,        "due": "0001-01-01T00:00:00Z"    }]

Things we haven't done yet.

Although we have a very good start, but there are many things do not do:

• Version control: What if we need to modify the API and the results change completely? Maybe we need to add/v1/prefix at the beginning of our route?
• Authorization: Unless these are public/free APIs, we may also need authorization. It is recommended to learn something about JSON web tokens.

ETag-If you are building something that needs to be extended, you may need to implement the ETag.

What else?

For all projects, the start is small, but it quickly becomes out of control. But if we want to take it to another level and make him productive, there are some extra things to do:

• Massive refactoring (refactoring).
• Create several packages for these files, such as some JSON helpers, modifiers, processors, and so on.
• Test, so that you can't forget this. We haven't done any tests here. For production systems, testing is a must.

Source

https://github.com/corylanou/...

Summarize

For me, the most important thing to remember is that we want to build a responsible API. Sending the appropriate status codes, headers, etc., are key to the widespread use of APIs. I hope this article will allow you to start your own API as soon as possible.

Reference links

• Go language RESTful JSON API implementation
• JSON API
• Gorilla Web Toolkit
• Httprouter
• JSON Web Tokens
• ETag