Golang JSON performance analysis

As an important data format, Json has good readability and self-description, and is widely used in various data transmission scenarios. The Go language natively supports this data format serialization and deserialization, the internal use of reflection mechanism implementation, performance is a bit poor, in highly dependent JSON parsing applications, often become a performance bottleneck, fortunately, there are many third-party libraries to help us solve this problem, but so many libraries, For people like me who have the choice of difficult disease, how to choose, the following will be for everyone to analyze

Ffjson

go get -u github.com/pquerna/ffjson

The primary reason for poor native library performance is the use of a lot of reflection mechanism, in order to solve this problem, Ffjson generated code by precompilation, type of judgment in the pre-compilation phase has been determined to avoid the reflection at runtime

But it also requires one more step before compiling, requiring the Ffjson code to be executed by the code, ffjson <file.go> which file.go is a go file containing the JSON struct definition. Note here that the Ffjson is a code generation tool provided by this library, directly executing the above go get will install the tool in the $GOPATH/bin directory, $GOPATH/bin add to the $PATH environment variables inside, you can access the global

In addition, if there are structures that do not want Ffjson to generate code, you can add annotations

// ffjson: skiptype Foo struct {   Bar string}// ffjson: nodecodertype Foo struct {   Bar string}

Easyjson

go get -u github.com/mailru/easyjson/...

Easyjson's ideas and Ffjson are consistent, both to add a precompiled process, pre-generated serialization of the corresponding structure of the deserialization code, in addition, Easyjson also abandoned some native library support some of the unnecessary features, such as: Key type declaration, key Case insensitive and so on to achieve higher performance

Generate code Execution easyjson -all <file.go> , if you do not specify -all parameters, only //easyjson:json generate code for the structure with

//easyjson:jsontype A struct {    Bar string}

Jsoniter

go get -u github.com/json-iterator/go

This is a very magical library, Didi developed, unlike Easyjson and Ffjson are used pre-compiled, and 100% compatible native library, but the performance is super good, do not know how to achieve, if anyone knows, can you tell me?

Using the above, you just put all the

import "encoding/json"

Replaced by

import "github.com/json-iterator/go"var json = jsoniter.ConfigCompatibleWithStandardLibrary

It's going to be all right, no need to move.

Codec-json

go get -u github.com/ugorji/go/codec

In fact, this library contains a lot of content, JSON is just one of the functions, older, more cumbersome to use, performance is not very good

Jsonparser

go get -u github.com/buger/jsonparser

Strictly speaking, this library does not belong to the JSON serialization library, but provides some JSON parsing interface, when used to set the value of the structure itself, in fact, each call will need to re-parse the JSON object, performance is not very good

As the name implies, this library is just a parsing library, and there is no serialized interface

Performance testing

Some performance tests have been made on these JSON libraries, and the test code is: Https://github.com/hatlonely/hellogolang/blob/master/internal/json/json_benchmark_ Test.go, here are the results of the tests on my Macbook (actual results are related to the version of the library and the machine environment, it is recommended to test it again):

BenchmarkMarshalStdJson-4                    1000000          1097 ns/opBenchmarkMarshalJsonIterator-4               2000000           781 ns/opBenchmarkMarshalFfjson-4                     2000000           941 ns/opBenchmarkMarshalEasyjson-4                   3000000           513 ns/opBenchmarkMarshalCodecJson-4                  1000000          1074 ns/opBenchmarkMarshalCodecJsonWithBufio-4         1000000          2161 ns/opBenchmarkUnMarshalStdJson-4                   500000          2512 ns/opBenchmarkUnMarshalJsonIterator-4             2000000           591 ns/opBenchmarkUnMarshalFfjson-4                   1000000          1127 ns/opBenchmarkUnMarshalEasyjson-4                 2000000           608 ns/opBenchmarkUnMarshalCodecJson-4                  20000        122694 ns/opBenchmarkUnMarshalCodecJsonWithBufio-4        500000          3417 ns/opBenchmarkUnMarshalJsonparser-4               2000000           877 ns/op

From the above results can be seen:

1. Easyjson both serialization and deserialization are optimal, serialization is 1 time times higher and deserialization is 3 times times higher
2. Jsoniter performance is also very good, close to Easyjson, the key is no pre-compilation process, 100% compatible native library
3. Ffjson's serialization boost is not obvious, and deserialization is 1 time times higher
4. Codecjson is not much worse or worse than the original library.
5. Jsonparser is not suitable for this scenario, performance improvements are not obvious, and there is no deserialization

So comprehensive consideration, we recommend that you use Jsoniter, if the pursuit of extreme performance, consider Easyjson

Reference links

Ffjson:https://github.com/pquerna/ffjson
Easyjson:https://github.com/mailru/easyjson
Jsoniter:https://github.com/json-iterator/go
Jsonparser:https://github.com/buger/jsonparser
Codecjson:http://ugorji.net/blog/go-codec-primer

Reprint please indicate the source
This article Links: http://hatlonely.github.io/2018/01/28/golang-json-performance analysis/

Golang JSON performance analysis