Freewheel practical experience based on Go--GC is a big pit (no business scenario), the Web framework is not unified, and C + + performance is difficult to compare

Excerpt from: Http://www.infoq.com/cn/news/2017/06/freewheel-experience-on-go

The go language is one of the main forces of the Freewheel company, and in its view, in the context of a service-oriented architecture, go is well suited to do some of the languages that are relatively independent and feature more specific microservices. Based on a combination of existing programming languages, computational frameworks (such as Hadoop, Java, Ruby, C + +), freewheel the Go language as the default programming language for lightweight services or APIs, combining it with Python for smaller-grained work. constitutes the entire technical language stack of freewheel.

Freewheel the "pit" that goes on go

Although from 2012 Go 1.0 released to the team to use go to write projects, which experienced roughly three years of time, but because in the GC and many other problems need go itself to do a part of the iteration, Freewheel also need to control the impact of technology on the customer within a controllable range, As a B-to-b enterprise, it has adopted a more gradual approach to the application of the go language to its own production platform.

In this process, Freewheel also experienced two more important "pits".

• Issues with GC

As most people know, the go language garbage collector has some flaws, especially the intermittent pauses that are prone to unpredictable whole processes. Like some large background service programs, such as game servers, app containers, due to large memory, the number of memory objects, GC to complete a recovery cycle, may take a few seconds or longer, during this time, the entire service process is blocked, paused, in the external view is service interruption, no response. Freewheel also encountered a similar problem when using Go 1.4: The ad prediction team uses go to implement the scheduler, which normally runs without problems, However, once the 1.4 version of the GC is triggered, the system's downgrade is very strong, causing the task to accumulate very serious, triggering the alarm, while its processing will degrade a lot, it will also affect the normal operation of other upstream and downstream systems.

Therefore, freewheel at that time mainly took three kinds of countermeasures: first, does not use go in very critical, the service process stability requirements of the system, second, the introduction of Kafka such as the ability to persist Message Queuing, to cache and re-release such a way to solve the problem, so that the system can withstand the impact, and digest it in the back; third, try to reuse the objects that have been created to prevent go from frequently creating recycled objects.

After the go 1.5 to 1.7 release, the GC's system performance has been continuously improved and continuously increased (from seconds to milliseconds). For systems that are not critical to the living environment in the freewheel, the GC after Go 1.7 can be reached within acceptable ranges and levels.

• Changes to built-in data structures

Many people know that the dictionary types provided by the go language are not concurrency-safe, and that some of the built-in data structures such as map behavior change due to the fast development of the go language. Because map is a reference type, even if a function is called, the parameter copy still points to the mapping m, so multiple goroutine concurrently write the same mapping m. For example, if the map is read and written simultaneously by multiple threads, the fatal error:concurrent map read and map write errors occur.

Prior to the Go 1.6 version, the map could support concurrent read and write, but the program developed by Freewheel after upgrading to 1.6 also found that map created a problem of read and write competition.

For this problem, there are two common solutions, one is the above mentioned locking (including Universal lock and read-write lock), the second is the use of channel serialization processing. Freewheel's approach also relies on two main aspects, one is to make the lock granularity design finer, make the concurrency dependence less, and in the different data structure, the selectivity can be higher. such as array and slice, the former is better choice.

Freewheel chief architect Liu Hao believes that when concurrency does not assume that go can handle all the work perfectly, engineers need to combine and learn from traditional (mature) programming languages such as Java or C for concurrency, before they start to think clearly about the size of concurrency, the granularity of locks, etc. And there is a very clear design and understanding of how the system will run.

At the same time, Liu Hao also frankly, with the rapid development of go, freewheel internal also has a variety of different voices, such as operations team will feel the rapid development phase of the language stability is not enough, its characteristics and data structure will be due to version upgrade and other reasons to make great changes, and some changes can not be completely forward-compatible. In this process freewheel summary is: No matter how to achieve, all the system concurrency to do a good support, in the application layer to do protection and control, so that the system can be run normally.

Why Freewheel not all use the system to rewrite go

From a programming paradigm perspective, the go language is a transformational faction, not an improved one. For the object-oriented ideology represented by languages such as C + +, Java and C #, in general, many companies around the world have a more conservative attitude towards the go language, most of which hold limited absorption (as can be seen from the heat distribution of Go).

Even if freewheel in practice, there are many more obvious advantages to go than other classes of programming languages, such as writing parallelism, an order of magnitude higher than the interpretation language of Python, such as many of the lightweight APIs developed by Python earlier, Because the global interpretation of the relationship between lock Gil and facing the extra overhead of interprocess communication, the lightweight API is migrated to go, as well as the advantages of go in concurrency, suitable for multi-user simultaneous uploading, simultaneous invocation of the API scene. But its internal team did not use go to rewrite the whole system. The main reasons are two points:

First, Freewheel has a lot of existing algorithm implementations, want to cut into the go on the face of huge costs and cost, second, compared to C or c++,go in the high-performance aspects have not fully proven themselves. On the Web server side, it does not currently have a particularly good pop-up framework like ROR, Django, or PHP. For freewheel, the entire AD server is not allowed to have a noticeable downgrade situation (especially when GC), so for this very critical system, it is not fully use go to write. So Liu Hao also believes that go in the extended use of the scene level may also need to do some major changes.

In addition, Freewheel is based on the Go Web program and is currently using the Gorilla framework. But from the use and evaluation of several mainstream frameworks, such as Martini, Revel, and Gocraft/web (which can be referenced on GitHub), the go community does not yet have a dominant web framework. If go wants to extend its tentacles longer, this may be the direction of its future.

However, in general, go in high concurrency, development efficiency and other characteristics of the advantages, determines the go in the freewheel will be more and more deep adoption. Liu Hao said: "In addition to some of the existing businesses rely on Hadoop, spark and other infrastructure, for the new business and functionality, the go language will be our first choice." ”

In addition, Freewheel hopes that the programming language it uses can be shared across continents, across time zones, and approved by all engineers, so go might be the best choice. In the process, freewheel also evaluated many other languages, such as Scala, rust, and so on, but ultimately wins the benefits of go in terms of learning costs, unifying practices, and community size.

Freewheel practical experience based on Go--GC is a big pit (no business scenario), the Web framework is not unified, and C + + performance is difficult to compare