Gopher China Congress

This is a creation in Article, where the information may have evolved or changed.

Two days ago to Shanghai to participate in the Gopher China Convention, see a lot of uh great gods and friends, feel good. And see a lot of enterprises, especially large enterprises have begun to scale the use of Golang, that the design and performance of Golang itself, has been recognized by everyone. Of course, there are a lot of topics, not confined to the Golang.

Go Language Core

The most heavyweight topic is the rain scar "Go 1.4 Runtime". The main points of the Go memory allocator, garbage collector and Goroutine Scheduler three blocks of content. I've read his Go notes before, which is pretty good. This paper expounds the implementation design of the Go language itself in detail, and provides some references for more reasonable utilization of golang. Plus he himself is also very low-key, really is the feeling of hermit tall man. His speech this time is also quite good. Smooth thinking and peaceful language, and sometimes humorous, make people sound interesting. The only regret is that he prepared the PPT can actually talk about 3 hours, but the journey is only 45min, although we still strive to listen to some more, but still do not have the intention.

Another super topic is Robert Griesemer's speech. He is one of the three authors of Go, and one of the developers of Google V8 and Java Hotspot VM, a huge halo! His speech was about gofmt. The go language itself has a lot of peripheral tools, such as GOFMT, Godoc. The formatting tool GOFMT helps you unify the code style in a straightforward way. Go itself with parser package, it can be very good to parse the go source to the syntax tree. The only thing to spit groove is the q&a link, the question of children's shoes with anyone can not understand the "English" to ask, haha ha!

Go and high-speed concurrency

Many people look at the Go language, mainly because of the simplified concurrency model: it is easy to create lightweight user-goroutine, use channel to communicate between Goroutine, the runtime itself is dispatched fast enough. Of course, in the topic of the speech will not be less, such as da da chat in the game practice,Goroutine(codis author) and Zhou Yang in the Distributed system implementation,Qleelulu( The application of Fawave in the ad auction system, and the application of the crescent-Moon -to-peer cache network.

In summary, the high-speed concurrency needs to be considered in detail:

• The distance of the CPU in the data

Dada's game project to ensure high speed, a lot of data are directly memory data. Large data in memory can significantly affect the efficiency of the GC. So to do, reduce the number of data objects in memory . Then the memory data must be written to the physical storage. Dada makes a direct mapping of data to MySQL in memory, called an in- memory database , and supports transactional processing. I understand that the actual operation in the in-memory database is not copied directly to the MySQL operation. Its own model is not complex, so many operations are operations of a single object, or some field of an object. Dada also mentions that the dispatch code of the memory database is actually generated in code. In other words, a simple logical description can represent a data operation. Then transactions are obviously necessary in business, while manipulating different objects is equivalent to manipulating multiple data tables.

In addition, this memory database uses a log mechanism similar to the traditional database. Memory modifications can be done instantly, and then synced to MySQL with a custom transaction log description. MySQL actually writes the currently executed operation in Binlog, and the subsequent execution is actually performed into the stored data.

• Proper centralized processing

The concentration of the antisense, is scattered. Scattered understanding, in fact, there are many possibilities. For example, repeated IO operations, RPC operations, and so on. From the perspective of Go itself, the problem of dispersion is goroutine leakage . If you put all kinds of logic into a separate goroutine, when the logic blocks, the goroutine is created and cannot be ended. This has put a lot of pressure on the scheduler so that other logic-created goroutine can not be executed for a long time, slowing down the entire business. The simple way to do this is to pass the channel, passing the logic that needs to be handled to some goroutine execution. In other words, create some goroutine to deal with some kind of logic, passing logic parameters through the channel.

The remaining problem is the decentralization of external resource scheduling. The most common way to use Redis is with pipeline. RPC can also use long-link duplex mode calls. Simply let the various commands be executed together, not separately.

• Control Locks and Timer

Locks are necessary for many exclusive operations, and there is no way to circumvent them. But the granularity of the lock is the key to control. If the size of the lock is large, the data is locked or the logic is much, then the other operation can only wait. For example, a global map modification of a 10k object requires locking to prevent concurrency conflicts. But as long as the object locked in 1W is not operational. So it's easiest to split the 10k object's map into 10 1K maps. This way each small map will only affect 1k data, not the original 10k, even if it is locked. However, the granularity of lock in practice needs to be judged by business, but it is a unified pursuit to reduce granularity.

The timer is actually a combination of a timed goroutine and a channel with an expiry signal. The more the timer, the more obviously it is goroutine. The scheduler's consumption increased significantly. If you still need the timing logic operation, you can go look at the time Wheel algorithm, reduce the creation timer. In addition, simply do not use, in their own logic to determine whether the time is beyond the limit of the problem.

• Summarize

High-concurrency services, some of the details of go itself, are helpful to improve efficiency, but not by the go language itself enough, but also need to cooperate with various means to truly meet the actual business requirements.

Go and Virtualization

The most popular Go language project is Docker. Many great gods are studying and using Docker. As a lightweight virtualization technology, there are many advantages compared to Xen, which is favored by everyone. I'm not familiar with Docker applications and don't have much practical experience with virtualization, so listen to the big gods and grow your knowledge.

after is a well-known friend of the Docker community. His speech is mainly about devops, from development, testing to deployment, operation and maintenance of the entire solution. In order to unify at the development, testing and deployment level, the system environment needs to be well unified. The direct approach is virtualization, such as a completely similar virtual image. The advent of Docker has greatly reduced the cost of virtualization. And the management of Docker images is very much like the git-like version management model, which is very friendly to developers. In addition, Docker mirroring is efficient for system resources and can be used as a production deployment. As a result, future devops trends are likely to be containerops.

I don't know much about Docker, listening to a lot of future trends, and the rocket between Docker and the river. The best way to use it now is to use Docker virtualization application, not virtual data storage . It's like Nginx proxy n instances, which manipulate the feeling of unifying data. In addition, the presentation also mentions Microsoft Windows support for Docker. Ten days ago Docker also released the Windows Client Preview. It looks like Docker has a lot of hope for virtualization across the platform.

Go and various fields

Go language localization is the system level language, many aspects will have the deep application.

A lecture is about the field of telecommunications, network function Virtualization (NFV, networks functions virtualization). The connection between telecommunication network devices is a network of network units, such as base stations-node devices-gateway devices and various physical devices. Communication between physical devices requires the interaction of hardware protocols. The more types of network devices, the more complex the interaction protocol is. Go's use, is on the ordinary server device, realizes the function of the network unit, namely the software simulates the hardware. At the same time, the virtual software can balance the sudden change of the network faster than the hardware device. However, the telecommunications network to the unicom rate, unicom quality has very strict requirements, the current program is still testing stage, do not know whether success or not.

There is also a lecture in the financial field that adds a lot of knowledge. I used to think that the financial system, especially the banking system, was a very old system. But in fact they are also evolving with new technologies. Because of the particularity of the financial field, the security and regulatory requirements for data are very high. And the complexity of the financial business is far greater than the general commercial business. The financial IT system itself is fully coupled, and the architecture is very stable. Of course, they are also trying to use virtualization technology to make adjustments within the architecture. While the overall structure is unlikely to change, new technologies are being used in a variety of emerging businesses.

There is also a very enlightening speech, written in the Go language of the network operating system, similar to the Chrome OS. Why we should consider the problem of high concurrency, because services and operations are centralized, such as micro-bo data processing is Sina computer room is doing. If the center is removed, the complexity of the operation will obviously be much lower. At the same time, the data is scattered and distributed across client nodes. This is actually very much like Bitcoin, there is no center, the data are present in the user's client. Updates to the data are propagated through the client's peer. The analogy is that Sina's data is actually distributed on the computers of individual microblogging users. When you use Weibo, you're actually connected to the microblogging network, and you can sync the data you need from other clients (usually just the data on your timeline). This is my understanding, do not know whether the great God really mean. At present, there are still many problems, such as data dispersion and synchronization, network node synchronization and so on. But such a model is really groundbreaking.

A little disappointment

Seven cows are the pioneer of the Go language business in China. Xu Xiwei is also one of the earliest users and evangelists of the Go language. Just this time to talk about an HTTP test tool, it is really disappointing. The highlight is the invention of a DSL to simplify the description of the HTTP test. Seven cows can also have a lot to talk about, such as cloud storage redundancy mechanism, multimedia file processing. Therefore, he is really disappointed to have such a subject.

Many friends

In the Go language circle for a long time, but also a lot of acquaintances. Used to be online chat discussion, this time can see himself, very happy. Dada and Crescent is a local friend of Xiamen, occasionally also meet chat, or very familiar. But the feeling of two people online is different from the onstage speech, Dada is too shy.

Lunny is the author of Xorm, who seems to be a very attentive uncle. Betago is very high, very serious look. Mao Jian also a little soft feeling. Of course, Asta Xie's impression and photos of the same, very enthusiastic and enthusiastic, but also hard for the children of the question of the shoe hand microphone. Congress also had been his hard-working, busy before. is a very good person.

I am just an ordinary programmer, there is no big on the work or name, do not seek to become the great God in the eyes of everyone. To understand and learn what I want, to do my job well, to live a simple life, I will be satisfied.

The Go language is very hot in China. A large number of Go language developers in the Gopher China to meet, exchange technology, make friends. I also see that there are a lot of great gods in-depth study of the use of the Go language. I'm just a beginner's child and need a lot of exercise. Hope to make unremitting efforts, study hard, day after day!!!