Sub-test of advanced test methods after Golang 1.7, sub-benchmark test (subtest Sub-benchmarks)

This is a creation in Article, where the information may have evolved or changed.

Introduced

After go1.7, the testing package T and B introduced a run method for creating subtests and Sub-benchmarks. Subtests and Sub-benchmarks allow developers to better handle failures in tests, better control which test cases to run, control parallel test operations, and test code more succinctly and more maintainable.

Table-driven Tests Foundation

First, let's talk about how to write test code that's common in go.

A series of related test checks can be implemented by traversing the slices of the test case, with the following code:

func TestTime(t *testing.T) {    testCases := []struct {        gmt  string        loc  string        want string    }{        {"12:31", "Europe/Zuri", "13:31"},     // incorrect location name        {"12:31", "America/New_York", "7:31"}, // should be 07:31        {"08:08", "Australia/Sydney", "18:08"},    }    for _, tc := range testCases {        loc, err := time.LoadLocation(tc.loc)        if err != nil {            t.Fatalf("could not load location %q", tc.loc)        }        gmt, _ := time.Parse("15:04", tc.gmt)        if got := gmt.In(loc).Format("15:04"); got != tc.want {            t.Errorf("In(%s, %s) = %s; want %s", tc.gmt, tc.loc, got, tc.want)        }    }}

The test function must start with test, and the first name must be capitalized.
The above test method is called Table-driven test, can reduce the duplication of code.

Table-driven Benchmarks

Before go1.7 is not able to use Table-driven method of benchmarks, if you want to test different parameters need to write different benchmark function, before go1.7 common benchmarks test code is as follows:

func benchmarkAppendFloat(b *testing.B, f float64, fmt byte, prec, bitSize int) {    dst := make([]byte, 30)    b.ResetTimer() // Overkill here, but for illustrative purposes.    for i := 0; i < b.N; i++ {        AppendFloat(dst[:0], f, fmt, prec, bitSize)    }}func BenchmarkAppendFloatDecimal(b *testing.B) { benchmarkAppendFloat(b, 33909, 'g', -1, 64) }func BenchmarkAppendFloat(b *testing.B)        { benchmarkAppendFloat(b, 339.7784, 'g', -1, 64) }func BenchmarkAppendFloatExp(b *testing.B)     { benchmarkAppendFloat(b, -5.09e75, 'g', -1, 64) }func BenchmarkAppendFloatNegExp(b *testing.B)  { benchmarkAppendFloat(b, -5.11e-95, 'g', -1, 64) }func BenchmarkAppendFloatBig(b *testing.B)     { benchmarkAppendFloat(b, 123456789123456789123456789, 'g', -1, 64) }

After go1.7, use the Table-drive method code as follows:

func BenchmarkAppendFloat(b *testing.B) {    benchmarks := []struct{        name    string        float   float64        fmt     byte        prec    int        bitSize int    }{        {"Decimal", 33909, 'g', -1, 64},        {"Float", 339.7784, 'g', -1, 64},        {"Exp", -5.09e75, 'g', -1, 64},        {"NegExp", -5.11e-95, 'g', -1, 64},        {"Big", 123456789123456789123456789, 'g', -1, 64},        ...    }    dst := make([]byte, 30)    for _, bm := range benchmarks {        b.Run(bm.name, func(b *testing.B) {            for i := 0; i < b.N; i++ {                AppendFloat(dst[:0], bm.float, bm.fmt, bm.prec, bm.bitSize)            }        })    }}

Each B. Run creates a separate benchmark.
You can see that the new encoding method is more readable and maintainable on the line.

If you want the sub-test to execute concurrently, use the B.runparallel

Table-driven tests using subtests

After Go1.7, the Run method is used to create the subtests, and the code in the previous Table-driven tests base is re-written as:

func TestTime(t *testing.T) {    testCases := []struct {        gmt  string        loc  string        want string    }{        {"12:31", "Europe/Zuri", "13:31"},        {"12:31", "America/New_York", "7:31"},        {"08:08", "Australia/Sydney", "18:08"},    }    for _, tc := range testCases {        t.Run(fmt.Sprintf("%s in %s", tc.gmt, tc.loc), func(t *testing.T) {            loc, err := time.LoadLocation(tc.loc)            if err != nil {                t.Fatal("could not load location")            }            gmt, _ := time.Parse("15:04", tc.gmt)            if got := gmt.In(loc).Format("15:04"); got != tc.want {                t.Errorf("got %s; want %s", got, tc.want)            }        })    }}

The test code for the table-driven tests base before go1.7 runs the result:

--- FAIL: TestTime (0.00s)    time_test.go:62: could not load location "Europe/Zuri"

Although both use cases are wrong, the latter use cases are not able to run after the first use case is fatalf.

The test code running with run results in:

--- FAIL: TestTime (0.00s)    --- FAIL: TestTime/12:31_in_Europe/Zuri (0.00s)        time_test.go:84: could not load location    --- FAIL: TestTime/12:31_in_America/New_York (0.00s)        time_test.go:88: got 07:31; want 7:31

Fatal causes Subtest to be skipped, but does not affect the tests of other subtest and parent test.

For each sub-test, the Go Test command prints a single line of test summaries. They are separated and independently counted. This allows us to perform a more granular test, fine-grained to each input and output.

Filtering execution Test Cases

Subtests and Sub-benchmarks can use-run or-bench flag
To filter the test cases to run. -run Or-bench flag followed by a '/' split regular expression used to develop a specific test case.

• Perform a sub-test that matches "in Europe" under Testtime

  $ go test -run=TestTime/"in Europe"--- FAIL: TestTime (0.00s)  --- FAIL: TestTime/12:31_in_Europe/Zuri (0.00s)      time_test.go:85: could not load location

• Perform a sub-test that matches "12:[0-9]" under Testtime

  $ go test -run=Time/12:[0-9] -v=== RUN   TestTime=== RUN   TestTime/12:31_in_Europe/Zuri=== RUN   TestTime/12:31_in_America/New_York--- FAIL: TestTime (0.00s)  --- FAIL: TestTime/12:31_in_Europe/Zuri (0.00s)      time_test.go:85: could not load location  --- FAIL: TestTime/12:31_in_America/New_York (0.00s)      time_test.go:89: got 07:31; want 7:31

$ go test -run=Time//New_York--- FAIL: TestTime (0.00s)    --- FAIL: TestTime/12:31_in_America/New_York (0.00s)        time_test.go:88: got 07:31; want 7:31

Func (*t) Parallel

func (t *T) Parallel()

Use T. Parallel () to enable the test and other sub-tests to execute concurrently.

 tc := tc这个地方很关键，不然多个子测试可能使用的tc是同一个。

func TestGroupedParallel(t *testing.T) {    for _, tc := range testCases {        tc := tc // capture range variable        t.Run(tc.Name, func(t *testing.T) {            t.Parallel()            if got := foo(tc.in); got != tc.out {                t.Errorf("got %v; want %v", got, tc.out)            }            ...        })    }}

Func (*b) runparallel

func (b *B) RunParallel(body func(*PB))

Runparallel runs a benchmark in parallel. It creates multiple goroutines and distributes B.N iterations among them. The number of goroutines defaults to Gomaxprocs. To increase parallelism for non-cpu-bound benchmarks, call Setparallelism before Runparallel. Runparallel is usually used with the Go TEST-CPU flag.

The body function would be is run in each goroutine. It should set up any goroutine-local state and then iterate until PB. Next returns FALSE. It should not use the Starttimer, Stoptimer, or Resettimer functions, because they has global effect. It should also not call Run.

Runparallel concurrent execution benchmark. Runparallel creates multiple goroutine and then distributes B.N iterative tests to these goroutine. The number of Goroutine is gomaxprocs by default. If you want to increase the number of benchmark for non-cpu-bound, call Setparallelism before executing runparallel.

Do not use Starttimer, Stoptimer, or Resettimer functions these functions, because these functions are all global effect.

package mainimport (    "bytes"    "testing"    "text/template")func main() {    // Parallel benchmark for text/template.Template.Execute on a single object.    testing.Benchmark(func(b *testing.B) {        templ := template.Must(template.New("test").Parse("Hello, {{.}}!"))        // RunParallel will create GOMAXPROCS goroutines        // and distribute work among them.        b.RunParallel(func(pb *testing.PB) {            // Each goroutine has its own bytes.Buffer.            var buf bytes.Buffer            for pb.Next() {                // The loop body is executed b.N times total across all goroutines.                buf.Reset()                templ.Execute(&buf, "World")            }        })    })}

My Test example

Benchmark Test Code

Func Benchmarkproductinfo (b *testing.  B) {//B.resettimer () testcases: = []string{"Pn3", "P7", "p666"} for _, ProductId: = Range testcases {// B.setparallelism B.run (ProductId, func (b *testing).        B) {for i: = 0; i < B.N; i++ {Mgodb.ecngetproductinfoofproductid (productId)} })}}func Benchmarkproductinfoparalle (b *testing. B) {//B.resettimer () testcases: = []string{"Pn3", "P7", "p666"} for _, Tproductid: = Range Testcases {/ /b.setparallelism ProductId: = Tproductid B.runparallel (func (b *testing). PB) {for B.next () {Mgodb.ecngetproductinfoofproductid (productId)}})}}f UNC Benchmarkproductlock (b *testing.  B) {//B.resettimer () testcases: = []string{"Pn3", "P7", "p666"} for _, ProductId: = Range testcases {// B.setparallelism B.run (ProductId, func (b *testing).            B) {for i: = 0; i < B.N; i++ {    Mgodb.checkproductlockstatus (ProductId)})}}func Benchmarkproductlockparallel (b *testing. B) {//B.resettimer () testcases: = []string{"Pn3", "P7", "p666"} for _, Tproductid: = Range Testcases {/ /b.setparallelism ProductId: = Tproductid B.runparallel (func (b *testing). PB) {for B.next () {mgodb.checkproductlockstatus (ProductId)}}}}

• execute the following test command

    go test-bench=. " 

  Result

    benchmarkproductinfo/pn3-4 10000 107704 ns/opbenchmarkproductinfo/p7-4 10000 108921 ns/opbenchmarkproductinfo/p666-4 10000 107163 Ns/opbenchmarkproduc TINFOPARALLE-4 10000 113386 ns/opbenchmarkproductlock/pn3-4 10000 10041                8 ns/opbenchmarkproductlock/p7-4 20000 97373 ns/opbenchmarkproductlock/p666-4 20000 96905 ns/opbenchmarkproductlockparallel-4 10000 108399 ns/op  

• Execute the following test command

  go test -bench=ProductInfo

  The filter test function name contains the ProductInfo test case, the result:

  BenchmarkProductInfo/pn3-4                 10000            111065 ns/opBenchmarkProductInfo/p7-4                  10000            118515 ns/opBenchmarkProductInfo/p666-4                10000            111723 ns/opBenchmarkProductInfoParalle-4              10000            118641 ns/op

• Execute the following test command

  go test -bench=oductInfo

  The filter test function name contains the Oductinfo test case, the result:

  BenchmarkProductInfo/pn3-4                 10000            107338 ns/opBenchmarkProductInfo/p7-4                  10000            109848 ns/opBenchmarkProductInfo/p666-4                10000            109344 ns/opBenchmarkProductInfoParalle-4              10000            114351 ns/op

• Execute the following test command

  go test -bench=ProductInfo/p7

  The filter test function name contains ProductInfo and the child test name contains P7 test cases, and we can note that parallel tests are also performed. Results:

  BenchmarkProductInfo/p7-4                  10000            109045 ns/opBenchmarkProductInfoParalle-4              10000            117569 ns/op

Test code

Func testcheckproductlockt (t *testing. T) {testcases: = []string{"A1", "A2", "A3"} for _, ProductID: = Range testcases {t.Log (ProductID) T. Run (ProductID, func (t *testing). T) {_, ret: = Mgodb.ecngetproductinfoofproductid (ProductID) if ret! = Success {T.FA Talf ("Faield")})}}func Testcheckproductlocktparalle (t *testing.        T) {testcases: = []string{"A1", "A2", "A3"} for _, Tproductid: = Range testcases {productID: = Tproductid T.Log (ProductID) T.run (ProductID, func (t *testing. T) {T.parallel () _, ret: = Mgodb.ecngetproductinfoofproductid (ProductID) if ret! = Succe SS {t.fatalf ("Faield")})}}func Testuseridmatchrole (t *testing.  T) {reqdata: = []struct {ProductID string UserID string Roletype string} {"PN2", "48176d26e860975e96518b80a3520407", "HR"}, {"PN2", "48176d26e860975e96518b80a3520407 "," CEO "}, {" PN2 "," 48176d26e860975e96518b80a3520407 "," CTO "},} for _, Data: = Rang E reqdata {//T.Log (data) T.run (FMT. Sprint ("%s%s", data. ProductID, data. Roletype), func (t *testing. T) {if ret: = Checkusermatchproductrole (data. ProductID, data. UserID, data. Roletype); Ret! = Success {t.error ("not Match")})}}func Testuseridmatchroleparall (t *testing .  T) {reqdata: = []struct {ProductID string UserID string Roletype string} {"PN2", "48176d26e860975e96518b80a3520407", "HR"}, {"PN2", "48176d26e860975e96518b80a3520407", "CEO"}, {"PN2", "481        76d26e860975e96518b80a3520407 "," CTO "},} for _, Tdata: = Range Reqdata {//data: = Tdata//Important T.Log (data) T.run (FMT. Sprint ("%s%s", data. ProductID, data. Roletype), func (t *testing. T) {T.parallel () if ret: = Checkusermatchproductrole (Data. ProductID, data. UserID, data. Roletype); Ret! = Success {t.error ("not Match")})}}

• Execute the following test command

  go test -bench="."

  Results

  ---fail:testcheckproductlockt (0.00s) ecn_test.go:626:a1---fail:testcheckproductlockt/a1 (0.00s) ecn_test . Go:630:faield ECN_TEST.GO:626:A2---fail:testcheckproductlockt/a2 (0.00s) Ecn_test.go:630:faield ECN _TEST.GO:626:A3---fail:testcheckproductlockt/a3 (0.00s) Ecn_test.go:630:faield---fail:testcheckproductlocktpa Ralle (0.00s) ecn_test.go:642:a1 ecn_test.go:642:a2 ecn_test.go:642:a3---FAIL:TESTCHECKPRODUCTLOCKTP ARALLE/A1 (0.00s) Ecn_test.go:647:faield---fail:testcheckproductlocktparalle/a2 (0.00s) ECN_TEST.GO:647:FA      Ield---fail:testcheckproductlocktparalle/a3 (0.00s) Ecn_test.go:647:faield---fail:testuseridmatchrole (0.00s) ecn_test.go:668: {pn2 48176d26e860975e96518b80a3520407 HR}---fail:testuseridmatchrole/%s_%spn2hr (0.00s) ECN _test.go:671:not match ecn_test.go:668: {pn2 48176d26e860975e96518b80a3520407 CEO}---fail:testuseridmatchrole/%s _%spn2ceo (0.00s) ECn_test.go:671:not match ecn_test.go:668: {pn2 48176d26e860975e96518b80a3520407 CTO}---fail:testuseridmatchrole/ %s_%spn2cto (0.00s) ecn_test.go:671:not match---fail:testuseridmatchroleparall (0.00s) ecn_test.go:692: {PN2 48176d26e860975e96518b80a3520407 HR} ecn_test.go:692: {pn2 48176d26e860975e96518b80a3520407 CEO} ecn_test.go:692  : {pn2 48176d26e860975e96518b80a3520407 CTO}---fail:testuseridmatchroleparall/%s_%spn2hr (0.00s) ecn_test.go:696: Not match---fail:testuseridmatchroleparall/%s_%spn2cto (0.00s) ecn_test.go:696:not Match---FAIL:TESTUSERIDM Atchroleparall/%s_%spn2ceo (0.00s) Ecn_test.go:696:not match

  In the test code we added the T.log print, by printing a comparison between the concurrent version and the non-concurrent version of the output, you can see the non-concurrent version of the test is actually executed in sequence, and the concurrent version of the test is executed concurrently.

  Reference URL

  Using subtests and Sub-benchmarks
  Interpretation of the Golang of 2016: The speed of ascension, gradually beyond