Thoughts on the best practice of Go language in the Reading production environment

This is a creation in Article, where the information may have evolved or changed.

Recently read an article on best practices for Go product development, go-in-procution. The author summarizes their use of go development process of many practical experience, many of us actually use, in view of this, here simply write the book, follow-up I also strive to be able to translate this article. Later I used SoundCloud to refer to the original author.

Development environment

In SoundCloud, each person uses a separate gopath, and in Gopath directly follows the code path defined by go to clone the code.

$ mkdir-p $GOPATH/src/github.com/soundcloud$ cd $GOPATH/src/github.com/soundcloud$ git clone git@github.com: Soundcloud/roshi

For go, the usual project management should be the following directory structure:

Proj/src/modulea/a.go moudleb/b.go App/main.go Pkg/bi n/

Then we set up the Proj path in the Gopath, so we can compile and run it. This was nothing, but if we were to submit the code to GitHub and allow another developer to use it, we wouldn't be able to submit the whole Proj thing, and if it did, it would hurt. The developers outside may cite this:

Import "Github.com/yourname/proj/src/modulea"

But in our own code, we can directly:

Import "Github.com/yourname/proj/modulea"

If the outside developers need to follow the removal of the SRC reference, can only set the Gopath to the Proj directory, if the import more, it will cause people to crash.

I have been this thing for a long time, and finally looked at the vitess code, found the above way, feel very good.

Project directory Structure

If the number of files in a project is not many, just put it in the main package, do not need to split into multiple packages, such as:

Github.com/soundcloud/simple/readme.md Makefile main.go main_test.go support.go support_test.go

If there is a public class library, it is split into separate package processing.

Sometimes, a project may include multiple binary applications. For example, a job might require a server, a worker, or a janitor, in which case, create multiple subdirectories as different main packages, placing separate binary applications. Use a different subdirectory to implement common functions.

Github.com/soundcloud/complex/readme.mdmakefilecomplex-server/main.go main_test.go handlers.go handlers_test.    Gocomplex-worker/main.go main_test.go process.go process_test.goshared/foo.go foo_test.go bar.go Bar_test.go

I have a slightly different approach, mainly referring to vitess, I like to set up a total cmd directory, and then set up a separate subdirectory inside, so there is no need to guess the directory is a library or application.

Code style

Code style This is nothing to say, directly using GOFMT solution, usually we also agreed to gofmt time without any other parameters.

It's a good idea to configure your editor to automatically gofmt when you save your code.

Google recently released the code specification for go, and SoundCloud has made some improvements:

• Avoid naming function return values unless you can clearly indicate the meaning.

• Try to use make and new sparingly, unless you really need to, or know beforehand what size you need to allocate.

• Use struct{} as the tag value instead of bool or interface{}. For example set we use map[string]struct{} to achieve, not map[string]bool.

If a function has more than one parameter, and the single line length is long and needs to be split, it is best not to use Java:

Don ' t do this.func process (DST io. Writer, ReadTimeout, writetimeout time. Duration, allowinvalid bool, Max int, src <-chan util. JOB) {//...}

Instead, use:

Func process (DST io. Writer, ReadTimeout, writetimeout time. Duration, allowinvalid bool, Max int, src <-chan util. Job,) {//...}

Similarly, when constructing an object, it is best to pass in the relevant parameters when initializing, rather than setting them later:

F: = foo. New (foo. config{Site: "zombo.com", Out:os. Stdout, Dest:conference. keypair{Key: "Gophercon", value:2014,},})//Don ' t do this.f: = &foo{}//or, even worse:new (Foo ) F.site = "zombo.com" f.out = os. Stdoutf.Dest.Key = "Gophercon" F.dest.value = 2014

If some variables are followed by other actions to get, I think they can be set up later.

Configuration

SoundCloud uses the GO flag package to pass configuration parameters instead of configuration files or environment variables.

The flag configuration is defined in the main function, not at the global scope.

Func main () {var (payload = flag). String ("payload", "ABC", "Payload data") Delay = flag. Duration ("Delay", 1*time. Second, "write delay") flag. Parse ()//...}

I have reservations about using flag as the delivery of configuration parameters. If an application requires very many configuration parameters, the use of flag comparison makes the egg ache. In this case, the use of the configuration file is better, I personally prefer to use JSON as a configuration, for reasons here.

Log

SoundCloud uses the log log of Go, and they also show that their log does not require too many other features, such as the log rating. For log, I've written a reference to Python's log, here. This log supports the log level and supports custom Loghandler.

SoundCloud also mentioned a concept of telemetry, I really do not have a good way to translate, according to my understanding may be the program's information collection, including response time, QPS, memory run errors and so on.

Usually telemetry there are two ways to push and pull.

Push mode is the initiative to send information to a specific external system, while the pull mode is to write it to a place, allowing the external system to obtain the data.

Both of these methods have different positioning, if you need to see the data in a timely manner, the push mode is a good choice, but this mode may take up too much resources, especially in the case of large amounts of data, it consumes CPU and bandwidth.

SoundCloud seems to be using a pull model.

On this point I am very much agree, we have a service, we need to send their information to a statistical platform for the follow-up information, at the beginning, we use push mode, each produced a record, we directly through the HTTP push to the back of the statistical platform, finally, with the increase in pressure, the entire statistical platform was we hung up, Denial of service. Finally, we used to write the data to local, and then through another program pull and then send the way to resolve.

Test

SoundCloud uses Go's testing package for testing, and then uses flag to do the integration test as follows:

+build Integrationvar FOOADDR = flag. String (...) Func Testtoo (t *testing. T) {f, err: = foo. Connect (*FOOADDR)//...}

Because go test also supports flag delivery like go build, it will be synthesized by default to a main package, which is then processed in the flag parse.

This way I do not use, I am in the test case directly written dead a global configuration, mainly for the convenience of the root directory to go to test. Processing. However, I feel very flexible in the way I use flag, and if it is possible to consider it later.

Go's testing package does not provide a strong function, such as not providing assert_equal such things, but we can through the reflect. Deepequal to solve.

Dependency Management

This piece is really what I want to solve too. Now our code is very violent with go get to solve the dependency problem, this is actually very risky, if a dependency package changes the interface, then we can go to the wrong time.

SoundCloud uses a vendor way to manage dependencies. In fact, it is very simple, it is to copy all the dependencies to their own projects, as their own code to use. This, however, requires regular maintenance of the dependency pack updates.

If you are introducing an executable package, create a _vendor folder under your own project directory (so that the relevant tool for go, such as go test, ignores the folder's contents). Use _vendor as a separate gopath, for example, copy Github.com/user/dep to _VENDOR/SRC/GITHUB.COM/USER/DEP. Then add the _vendor to your Gopath, as follows:

Go? = Gogopath: = $ (CURDIR)/_vendor:$ (Gopath) All:buildbuild: $ (GO) Build

If you are introducing a library, put it in the vendor directory and vendor as the package prefix, such as copy Github.com/user/dep to VENDOR/USER/DEP, and change all relevant import statements.

Because we do not need to frequently do the go get update processing for these introduced projects, most of the time this is worth it.

I started with a similar approach, but I do not call vendor, but called 3rd, and later for convenience or decided to change to direct go get, although it is known that the risk is relatively large. The possibility of subsequent use of GODEP may be a good solution.

Build and deploy

SoundCloud uses go build to build the system directly during the development process, and then uses a makefile to handle the formal build.

Because SoundCloud mainly deploys many stateless services, similar to Heroku provides a very simple way:

$ git Push bazooka master$ Bazooka Scale-r <new>-N 4 ... $ # validate$ Bazooka Scale-r <old>-N 0 ...

In this regard, we directly use a simple makefile to build the system, as follows:

All:build build:go Install ${src}clean:go clean-i ${src}test:go test ${SRC}

The release of the application takes the most primitive SCP to the target machine in the way it is restarted, but is now testing using salt to publish the app. And for the application to start, stop these, we use Supervisor to manage.

Summarize

In general, this article explains in detail the many experiences in the product development process with go, hope to be helpful to everybody.