Build a testable Go Web app

This is a creation in Article, where the information may have evolved or changed.

Almost every programmer agrees that testing is important, but testing has many ways to get people writing tests back. They may run slowly, may use duplicate code, and may test too much at one time to cause the failure of the test to be difficult to locate. In this article, we will discuss how to design sourcegraph unit tests to make them easy to write, easy to maintain, run fast and can be used by others. We hope that some of the patterns mentioned here will help others who write the Go web app, while welcoming suggestions for our testing methods. Before you start testing, take a look at our framework overview.

Xin Xin Xin Translated 4 days ago 0 Person Top top translation of good Oh!

Framework Like other web apps, our website has three floors: Web front-end to serve HTML; The HTTP API is used to return JSON; Data store, run a SQL query against the database, and return to the go fabric or slice. When a user requests a Sourcegraph page, the front end receives an HTTP page request and initiates a series of HTTP requests to the API server. The API server then starts querying the data store, the data store returns the data to the API server, then encodes it in JSON format, returns it to the Web front end server, and the front end uses the Go Html/template package to display and format the data as HTML. The frame diagram is as follows: (For more details, see Recap of our Google I/O talk about building a large-scale code search engine in Go.)

Xin Xin Xin Translated 4 days ago 0 Person Top top translation of good Oh!

Test V0 When we first started building sourcegraph, we wrote the test in the easiest way to run. Each test will enter the database to initiate an HTTP GET request to the test API endpoint. The test parses the HTTP return content and compares it to the expected data. A typical v0 test is as follows: Func testlistrepositories (t *testing. T)  {  tests := []struct { url string; insert []interface{};  want []*repo }{    {"/repos", []*repo{{name:  "foo"}}, []* repo{{name:  "foo"}}},    {"/repos?lang=go", []*repo{{lang:  "Python"}},  nil},    {"/repos?lang=go", []*repo{{lang:  "Go"}}, []*Repo{{Lang:   "Go"}}},  }  db. Connect ()   s := http. Newservemux ()   s.handle ("/",  router)   for _, test := range  Tests {    func ()  {      req, _ :=  http. Newrequest ("GET",  test.url, nil)       tx, _ := db. Db. Dbmap.begin ()       defer tx. Rollback ()       tX.insert (Test.data ...)       rw := httptest. Newrecorder ()       rw. Body = new (bytes. Buffer)       s.servehttp (rw, req)       var  got []*repo      json. Newdecoder (rw. Body). Decode (&got)       if !reflect. Deepequal (got, want)  {        t.errorf ("%s: got %v ,  want %v ",  test.url, got, test.want)       }     } ()   }} It's easy to write a test at first, but it becomes painful as the app evolves. Over time, we have added new features. More features result in more tests, longer uptime, and longer our dev cycle. More features also need to be changed and added new URL paths (now about 75), most of which are quite complex. The interior of the sourcegraph is also becoming more complex, so we want to test it independently of the other layers.

Xin Xin Xin Translated 4 days ago 0 Person Top top translation of good Oh!

We encountered some problems in the test: 1. Testing is slow because they are interacting with the actual database-inserting test cases, initiating queries, and rolling back each test transaction. Each test runs about 100 milliseconds, as we add more tests to accumulate. 2. Testing is difficult to refactor. The test used a string to write dead http path and query parameters, which means that if we want to change a URL path or query parameter set, we have to manually update the URL in the test. This pain is exacerbated by the complexity and number of URLs that we have to route. 3. There is a large number of scattered and fragile sample codes. Installing each test requires that the database is healthy and has the correct data. Such code is reused in multiple cases, but the difference is sufficient to introduce a bug in the installation code. We found ourselves spending a lot of time debugging our tests rather than the actual app code. 4. Test failures are difficult to diagnose. As the app becomes more complex, because each test accesses three application tiers, the root cause of the test failure is difficult to diagnose . Our tests are more like integration tests than unit tests. Finally, we present the need to develop a publicly released API client. We want the API to be easy to imitate so that our API users can write well-tested code.

Xin Xin translated 4 days ago 0 human top top   translation is good Oh!

Advanced test target : As our app evolves, we realize the need to meet these high-demand tests: Target clear: We need to test each layer of the app individually. full : All three layers of our app will be tested. Quick : Testing needs to run very fast, which means no more database interaction. DRY : Although every layer of our app is different, they share a number of common data structures. The test needs to take advantage of this to eliminate duplicate sample code. easy to imitate : API external users should also be able to use our in-house test mode. Projects built on our API should be able to easily write good tests.   After all, our web front end is not unique-it's just another API user. How do we rebuild the test Well-written, maintainable tests and good, maintainable application code are inseparable. Refactoring the application code allows us to greatly improve our test code, which is the step for us to improve our testing.

Xin Xin translated 4 days ago 0 human top top   translation is good Oh!

1. Build a go HTTP API client The first step in simplifying testing is to write a high-quality client with Go for our API. Before, our website was the Angularjs app, but since we primarily serve static content, we decided to move the front-end HTML generation to the server. In doing so, our new front end can use the Go API client and API server to communicate. Our client go-sourcegraph is open source, and the Go-github library has a huge impact on it. The client code (especially the endpoint code that gets the warehouse data (repository) is as follows: Func newclient () *client {c: = &client{baseurl:defaultbaseurl} c.repositories = &reposervice{c} return C}type Reposervice struct{C *client}func (c *reposervice) Get (name string) (*repo, error) {resp, err: = http. Get (FMT. Sprintf ("%s/api/repos/%s", C.baseurl, name)) If Err! = Nil {return nil, err} defer resp. Body.close () var repo repo return &repo, JSON. Newdecoder (resp. Body). Decode (&repo)} Previously, our V0 API tests had written a large number of URL paths and built-in HTTP requests in a AD-HOC way, and now they can use this API client to build and initiate requests.

Xin Xin Xin Translated 3 days ago 0 Person Top top translation of good Oh!

2. Interface for unified HTTP API client and Data Warehouse Next, we unify the HTTP API and the Data warehouse interface. Previously our API http. Handlers directly initiates the SQL query. Now we have API http. Handlers only needs to parse http.request and then call our Data Warehouse, and the Data warehouse and HTTP API client implement the same interface. Refer to the HTTP API client (*reposervice) above. Get method, we also have now (*repostore). Get: Func Newdatastore (DBH modl. Sqlexecutor) *datastore {s: = &datastore{dbh:dbh} s.repositories = &repostore{s} return s}type RepoStore Stru ct{*datastore}func (S *repostore) Get (name string) (*repo, error) {var Repo *repo return Repo, S.db.select (&r EPO, "select * from Repo WHERE name=$1", name)} Unifying these interfaces places the behavior description of our web App in one place, making it easier to understand and infer. And we can reuse the same data types and parameter structures in the API client and Data Warehouse.

Xin Xin Xin Translated 3 days ago 0 Person Top top translation of good Oh!

3. Centralized URL path definition Before, we had to redefine the URL path in multiple tiers of the app. In the API client, our code is like this RESP, err: = http. Get (FMT. Sprintf ("%s/api/repos/%s", C.baseurl, name)) This is an easy way to throw an error because we have more than 75 path definitions, and many are complex. A centralized URL path definition means that the path is refactored from the API server in a new package. The path definition is declared in the path package. Const Repogetroute = "Repo" func newapirouter () *mux. Router {m: = Mux. Newrouter ()//define the Routes M.path ("/api/repos/{name:.*}"). Name (Repogetroute) return m}while the HTTP. Handlers were actually mounted in the API server Package:func init () {m: = Newapirouter ()//Mount handlers m.ge T (Repogetroute). Handlerfunc (handlerepoget) http. Handle ("/api/", M)} and HTTP. Handlers is actually mounted in the API server package: Func init () {m: = Newapirouter ()//Mount handlers M.get (Repogetroute). Handlerfunc (handlerepoget) http. Handle ("/api/", M)} Now we can use the path package to generate URLs in the API client instead of writing them to death. (*reposervice). The Get method is now as follows: var apirouter = Newapirouter () func (S *reposervice) Get (name string) (*repo, error) {URL, _: = Apirouter.get (Repogetro Ute). URL ("name", name) resp, err: = http. Get (S.baseurl + URL. String ()) if err! = Nil {return nil, err} defer resp. Body.close () var repo []repo return repo, json. Newdecoder (resp. Body). Decode (&repo)}

Xin Xin Xin Translated 3 days ago 0 Person Top top translation of good Oh!

4. Creating a clone of a non-uniform interface Our V0 tests also tested paths, HTTP processing, SQL generation, and DB queries. Failures are difficult to diagnose and tests are slow. Now we have independent testing of each layer and we mimic the function of the adjoining layer. Because each layer of the application implements the same interface, we can use the same clone interface in all three layers. The implementation of imitation is a simple simulation function structure that can be specified in each test: Type Mockreposervice struct {get_ func (name string) (*repo, error)}var _ Repointerface = Mockreposervice{}func (S Mock Reposervice) Get (name string) (*repo, error) {if s.get_ = = Nil {return nil, nil} return s.get_ (name)}fu NC newmockclient () *client {return &client{&mockreposervice{}}} The following are the uses in the test. We modeled the reposervice of the data Warehouse and tested API HTTP using the HTTP API client. Handler. (This code uses all of the above methods.) ) Func testrepoget (t *testing. T)  {   setup ()    defer teardown ()    var  Fetchedrepo bool   mockdatastore.repo. (*mockreposervice). Get_ = func (name string)   (*repo, error)  {        if name !=  "foo"  {            T.errorf ("Want get %q, got %q",  "foo",  repo. URI)        }       fetchedrepo =  true       return &repo{name}, nil   }    repo, err := mockapiclient.repositories.get ("foo")    if  Err != nil { t.fatal (Err)  }   if !fetchedrepo { t. Errorf ("!fetchedrepo")  }}

xin Xin Xin translated 3 days ago 0 Human top top   good translation!

Review of advanced Test objectives Using the above pattern, we achieved the test target. Our code is: Target Clear : one level of testing. Comprehensive : Three application tiers are tested. Quick : The test runs quickly. DRY: We have combined the common interfaces of three application tiers and reused them in application code and testing. easy to imitate : A clone implementation is available in three application tiers, and external API users who want to test libraries built on Sourcegraph can also be used. The story of how to rebuild and improve Sourcegraph's test is finished. These patterns and examples work well in our environment, and we hope that these patterns and examples will also help other people in the go community, and it is clear that they are not correct in every scenario, and we are sure there is room for improvement. We are constantly trying to improve the way we do things, so we are happy to hear your suggestions and feedback--tell us about your experience with go writing tests!

Xin Xin Xin Translated 3 days ago 0 Person Top top translation of good Oh!

All translations in this article are for learning and communication purposes only, please be sure to indicate the translator, source, and link to this article.
Our translation work in accordance with the CC agreement, if our work has violated your rights and interests, please contact us promptly