A practical tutorial on Go Building microservices (iii.)

This is a creation in Article, where the information may have evolved or changed.

Original Address
Reprint please specify the original and translation address

In the third section, we're going to let our accountservice do something useful.

• Declaring an account structure
• A storage that embeds a key-value pair to store the account structure
• The serialization structure is JSON and is used for our Accounts/{accountid} HTTP Service

Source

All the code in this blog can be obtained from the branch P3.

git checkout P3

Declaring an account structure

In our project, create a model folder in the Accountservice folder

mkdir model

Under the Model folder, create a file with the name Account.go and write the following code:

package modeltype Account struct {    Id string `json:"id"`    Name string `json:"name"`}

This declares the account, which contains the ID and name. The case of the first letter indicates the scope (uppercase =public, lowercase = in-package call).
In the declaration, we also used the built-in Json.marshal function to support parameter serialization.

Embedding a key-value pair for storage

Here, we will use the BOLTDB to store the key-value pairs. This package is simple, quick and easy to integrate. We can use go get to get this package

go get github.com/boltdb/boltdb/b

After that, in the/goblog/accountservice folder, create a folder dbclient, boltclient.go the file in Dbclient. To make mocking easier, we first declare an interface that is used to define the method that the implementation needs to follow.

package dbclientimport (        "github.com/callistaenterprise/goblog/accountservice/model")type IBoltClient interface {        OpenBoltDb()        QueryAccount(accountId string) (model.Account, error)        Seed()}

In the same file, we implement this interface. Define a bolt that is encapsulated first. The structure of a pointer to a DB

// Real implementationtype BoltClient struct {        boltDB *bolt.DB}

Here is the implementation of OPENBOLTDB (), and we will then add the remaining two functions.

func (bc *BoltClient) OpenBoltDb() {        var err error        bc.boltDB, err = bolt.Open("accounts.db", 0600, nil)        if err != nil {                log.Fatal(err)        }}

This part of the code may seem a little strange, but we are actually binding a method function to a struct. Our structure implicitly implements one of the three methods.
We need a "bolt client" instance in some places. Let's declare where it will be used, create/goblog/accountservice/service/handlers.go, and create an instance of our struct:

package service    import (          "github.com/callistaenterprise/goblog/accountservice/dbclient"  )    var DBClient dbclient.IBoltClient

Update Main.go to open the database when he starts:

func main() {        fmt.Printf("Starting %v\n", appName)        initializeBoltClient()                 // NEW        service.StartWebServer("6767")}// Creates instance and calls the OpenBoltDb and Seed funcsfunc initializeBoltClient() {        service.DBClient = &dbclient.BoltClient{}        service.DBClient.OpenBoltDb()        service.DBClient.Seed()}

Our microservices now create a database at startup. However, we also need to refine the code before running:

Seed some accounts at startup

Open Boltclient to add the following code:

Start seeding Accountsfunc (BC *boltclient) Seed () {initializebucket () seedaccounts ()}//creates an "Acc Ountbucket "in our boltdb. It would overwrite any existing bucket of the same name.func (BC *boltclient) initializebucket () {bc.boltDB.Update ( Func (TX *bolt. TX) Error {_, err: = Tx. Createbucket ([]byte ("Accountbucket")) if err! = Nil {return FMT. Errorf ("Create Bucket failed:%s", err)} return nil})}//Seed (n) Make-Believe Acco UNT objects into the acountbucket bucket.func (BC *boltclient) seedaccounts () {total: = + for I: = 0; I & Lt Total i++ {//Generate a key 10000 or larger key: = StrConv. Itoa (10000 + i)//Create An instance for our account struct ACC: = model. account{Id:key, Name: "Person_" + StrConv.              Itoa (i),}  Serialize the struct to JSON jsonbytes, _: = json. Marshal (ACC)//Write the data to the Accountbucket Bc.boltDB.Update (func (TX *bolt. TX) Error {b: = tx. Bucket ([]byte ("Accountbucket")) Err: = B.put ([]byte (Key), jsonbytes) return ERR})} FMT. Printf ("Seeded%v fake accounts...\n", total)}

Want to know how the update function of the bolt API works. You can refer to Boltdb's documentation

Now let's add the query function:

Func (BC *boltclient) queryaccount (AccountId string) (model. Account, error) {//Allocate a empty account instance we'll let json.        Unmarhal populate for us in a bit. Account: = model. account{}//Read an object from the bucket using Boltdb.view err: = Bc.boltDB.View (func (TX *bolt. TX) Error {//Read the bucket from the DB b: = Tx.                Bucket ([]byte ("Accountbucket"))//Read The value identified by our accountId supplied as []byte Accountbytes: = B.get ([]byte (AccountId)) if accountbytes = = Nil {return FMT. Errorf ("No account found for" + accountId)}//Unmarshal the returned bytes into the Accou NT struct we created at//the top of the function JSON.                Unmarshal (Accountbytes, &account)//Return nil to indicate nothing went wrong, e.g no error return nil})//If There were an error, return the error if err! = Nil {return model.        account{}, err}//Return the account struct and nil as error. Return account, nil}

Annotations make it easier for you to understand. This function will search for BOLTDB with a provided accountid, then return an account structure or error

Now you can try running:

> go run *.goStarting accountserviceSeeded 100 fake accounts...2017/01/31 16:30:59 Starting HTTP service at 6767

Provide account service via HTTP

Let's modify the/accounts/{accountid} route in/service/routes.go so that he returns a seeded account struct. Open Routes.go replace func with the Getaccount function (w http. Responsewriter, R *http. Request). We will then create the Getaccount function:

Route{        "GetAccount",             // Name        "GET",                    // HTTP method        "/accounts/{accountId}",  // Route pattern        GetAccount,},

After that, update service/handlers.go. Add the Getaccount function:

var DBClient dbclient.IBoltClientfunc GetAccount(w http.ResponseWriter, r *http.Request) {    // Read the 'accountId' path parameter from the mux map    var accountId = mux.Vars(r)["accountId"]        // Read the account struct BoltDB    account, err := DBClient.QueryAccount(accountId)        // If err, return a 404    if err != nil {        w.WriteHeader(http.StatusNotFound)        return    }        // If found, marshal into JSON, write headers and content    data, _ := json.Marshal(account)    w.Header().Set("Content-Type", "application/json")    w.Header().Set("Content-Length", strconv.Itoa(len(data)))    w.WriteHeader(http.StatusOK)    w.Write(data)}

This getaccount function conforms to the handler function format defined in gorilla. So when Gorilla finds a request/accounts/{accountid}, it is routed to the Getaccount function. Let's run for a try:

> go run *.goStarting accountserviceSeeded 100 fake accounts...2017/01/31 16:30:59 Starting HTTP service at 6767

Use Curl to request this API. Remember, we joined 100 accounts.id starting from 10000.

> curl http://localhost:6767/accounts/10000{"id":"10000","name":"Person_0"}

Yes, our microservices now answer JSON data over HTTP.

Performance

Let's look at memory and CPU usage separately:

Memory utilization after startup

2.1MB, good. Added 300kb after adding embedded boltdb and some other routing code, compared to the initial consumption. Let's test 1K req/s with Gatling. Now we're really returning the real account data and serializing it to JSON.

Memory usage under stress testing

31.2MB. Adding an embedded database does not consume too much resources, compared to the simple Return data service in chapter two

Performance and CPU Usage

1k req/s with a single core of about 10%. Boltdb and JSON serialization do not add too much consumption. By the way, take a look at the above Java program, with three times times more CPU resources

The average response time is still less than 1ms.
Let's try a bigger stress test, 4k req/s. (You may need to increase the OS level to handle the maximum number of requests)

Memory Usage 4k REQ/S

About 12MB grows about 4 times times. Memory growth is likely due to a go run or gorilla increase in the number of internal Goroutine to process requests concurrently.

4k REQ/S Performance

CPU usage is approximately 30%. When we run on a 16GB Ram/core i7 notebook, io or file access will become a bottleneck before the CPU.

The average delay rose to 1ms,95% 's request under 3MS. We saw an increase in latency, but I think the accountservice performance is good.

Summarize

We will discuss unit test in the next article. We will use Goconvey mocking Boltdb client at the same time.

Thank you , please.