Some tips on the Go Project

This is a creation in Article, where the information may have evolved or changed.
SpaceX

Recently the small partners have just completed the advertising system, the second direct service to the business of the project. Stepping on some pits, more gain a lot of knowledge. summed up with you to share, nothing tall still technology, are around the small skills, deepen the understanding of the go language, suitable for beginners, not to spray the birds.

Package Management

Many people think that the package management of Go is not friendly and deep feeling. Especially on GitHub, I fork to my own directory, and if the original author has a library that references the path below, it's a hassle.

Another is version management, everyone's gopath below the same library may have different versions, the official provides a GODEP to control the version, I see a lot of open source projects are also in use. But what if you want to manage dependencies other than go?

We use relative paths to centralize the referenced libraries into submodule, such as:

Package Management

We put all language third-party libraries into Tinder, including Thrfit IDL files shared among multiple groups. Go third-party libraries are placed under the Golang/lib directory, the shared internal library is placed under Golang/src/common, and each time the install compiles the program, the Gopath is assigned to the relative directory under the current project.

Update 20160629: Now relies on using Govender, third-party IDL uses submodule

Timeout control and request tracking

Due to the high latency requirements of the business, we set a 50ms time-out. You will certainly think of using Channel and timer to control, but we also want to track the request in a series of operations inside, otherwise the Debug log a large number of, unable to locate.

At this time think of the Golang.org/x/net/context Library, the official documentation is very detailed, for cross-API calls is very convenient, vitess in a large number of use of this library.

A Context carries a deadline, a cancelation signal, and other values across

API boundaries.

The Context ' s methods is called by multiple goroutines simultaneously.

Type Context Interface {

Deadline returns the time when work done on behalf of this context

should be canceled. Deadline returns Ok==false when No. Deadline is

Set. Successive calls to Deadline return the same results.

Deadline () (Deadline time. Time, OK bool)

Done returns a channel that's closed when work done on behalf of this

Context should be canceled. Done could return nil if this context can

Never be canceled. Successive calls to do return the same value.

See Http://blog.golang.org/pipelines for more examples by

A done channel for cancelation.

Done () <-chan struct{}

Value returns the value associated with this context for key, or nil

If no value is associated with key. Successive calls to Value with

The same key returns the same result.

Use context values request-scoped data transits

Processes and API boundaries, passing optional parameters to

Functions.

Packages that define a Context key should provide Type-safe accessors

UserKey is the key for user.  User values in contexts. It is

unexported; Clients use user. Newcontext and user. Fromcontext

Instead of using this key directly.

Value (Key interface{}) interface{}

}

A request to come, every time the flow to carry the context. Context, and first detects if the timeout is exceeded, and if it is timed out or canceled, it returns directly. In addition context. The Context carries each request ID, which is the field passed by the business, and if it is empty, a UUID is generated internally to identify it.

Code logic for final execution

The timeout parameter is passed by the business, generating a context based on timeout. Context, the final function is either by CTX. The done timeout is returned, or the business result is retrieved from the RR channel. A real business request will open an anonymous goroutine, incoming context. The Context carries the logid, the internal log will print the Logid first.

Cancel Signal

At the entrance of each time-consuming request (REDIS/MYSQL), the timeout is detected first.

Goroutine and Panic

This piece is not fine, unlike actor has a father-son relationship, function derived from the Goroutine if Panice will hang up the entire program, such as the following code:

Error example

The beginning of the program, such as the original thought will capture the do_something produced panic, or too young ah. To place the recover in the Go Func entry.

Cache Dirty Data

We will cache the user information in Redis, expires 6 hours, if no more fallback to the database, there is also a program built-in LRU cache.

After the program upgrade, found that the test logic is not correct, the UID is always 0,fix this problem, the cache then appears dirty data. Then there are two ways to choose the 2nd one.

1. Bulk purge of invalid caches using Redis-port

2. Update the program again, internal revision error data

PHP Thrift Timeout problem

The problem is a headache, and people on the Internet have encountered a timeout (thrift) pit. There are three timeouts at the bottom. Connect, send and recv, initially set the 100ms, the daily large number of super-times error on the line, then we will recv timeout to 1000ms line on the quiet.

The other two connect, send is still 100ms, we prefer the lower-level driver timeout time is slightly longer, by the business layer to control the timeout (context library).

Object Pool

The object pool is something different from the connection pool, two concepts. The connection pool refers to the long connection of Redis/mysql, resident memory. While object pooling is an internal instance, using object pooling can reduce program GC pressure. There are two types of sync currently in use. Pools of pool and channel simulations. The official has a pair of sync. A detailed description of the Pool, the object will be collected between two GC release, and the channel will be resident.

Object Pool

The code is simple and easy to understand, and when the channel has data on GET, it returns without a direct New. As for the channel buffer size, it depends on the business pressure.

Internal Service Registration

In the Global Map Registration service, this is also the Go program standard, the most famous is the official database library registered MySQL driver code

Service Registration

Implemented in driver. Driver interface Services, directly registered in can be used, directly according to name to find Driver.

Serveronrun

There is a large number of initialization requirements for the service internals, which can be thrown directly into the init () function for global variables, but not in Init () for dependent external services (Mysql/redis/servervice) when the connection handle does not exist at program startup.

One approach is to write the Init_xx () method in each module and then invoke it after main () starts initializing the external configuration, but maintaining it in main is cumbersome.

So to define serveronrun globally, each initialization that cannot be done by Init () is registered here, and finally executed by main traversing Serveronrun.

Thrift Field Change Issues

Business upgrade changes, often add fields, and in order to be compatible with existing code, must be set to optional. In addition, sometimes encountered in order to change the field type from int to string problem, more trouble, in the early or to design a good

JSON serialization

There are a lot of JSON serialization requirements inside the program, the official is slightly slower, using the more popular ffjons