Xu Xiwei: My ten years with Go language

This is a creation in Article, where the information may have evolved or changed. Author | Xu Xiwei Editor | Little Wisdom September 20, 2007, the discussion about the design of a new language officially began, the new language is the later Go. Today, the Go language has been released to version 1.9, and it has been a decade-old journey. In the decade, the go language won the TIOBE Annual Language Awards (2009/2016), and many start-ups developed early with Go, including today's cloud giants Docker, which also spawned projects like Kubernetes. In China on the other side of the ocean, Google Trends shows that Go's unusually hot is more shocking to the father of the go language. And all this, with a technical person named Xu Xiwei inseparable.

The Go language has just passed its 10 anniversary. And to say that my fate with Go, also began ten years ago (2007). This decade, is the Go growth of ten years, but also I grew up 10 years.

February 2007, I set up the Jinshan laboratory, the main focus of distributed storage direction. This is a crucial turning point in my career. I've been working on the WPS Office suite for 7 years before. This has two important challenges for me: one is role-switching, from the architect of software development to the product manager doing business, and the other in the field of technology, from desktop software development to distributed back-end server development.

The first problem I faced was technology selection. Almost without any entanglements, we chose Java as the main development language. As a team in the field of early-entry back-end development, this is a pragmatic choice. But TrustNo1 in the "Storm in a Teacup" published in the programmer's magazine made me know the Erlang language and was deeply impressed by Erlang's programming ideas. I realized that this kind of programming thought would greatly unleash the productivity of server-side programming. To drive the quest for best practices in the service side, and to further drive more people to understand and study Erlang, I launched the ECUG community. The ECUG has been 10 years now. We will have a ECUG Con convention at the end of the decade, together with technical danale from all walks of life, to share the experience of this year's service-side development.

Although we chose Java, we then made a small-scale attempt to develop with Erlang. In 2008 we launched an amateur program that uses Erlang to write a storage system (a minimalist version). This attempt allowed me to make further judgments about Erlang: Although Erlang's programming ideas are good, the Erlang language itself is not suitable for large projects.

By the March 2009, when I joined the Grand Innovation Institute to restart the distributed storage project, I faced the second technology selection. This time I picked the hardest route: developing in C + +. It's a hard road. The biggest reason I made this choice was that I wanted to be able to create a new wheel that would have the advantage of Erlang programming and overcome the disadvantages of the Erlang language. If this can be done, it will be a great innovation, and all the server developers will benefit from it. So, under the blood of the heart, a project called Cerl was born. During the first version we stored, we actually spent much more time in the Cerl library than we did in the store itself, both in terms of code volume and effort.

cerl project has undergone 2 large versions. Cerl 1.0 fully follows Erlang's programming philosophy, and the main programming paradigm is as follows:

• can start any number of processes (where the process is abstract, the implementation is fiber/coprocessor), the maximum number of processes is limited by the memory size.

• Each process has process mailboxes, which send messages to each other through the process mailbox.

• messages are synchronized messages and asynchronous messages, and synchronization messages block waiting for each other to return messages.

• The Web server is a single-process model (a single-threaded model that you understand) that processes only one message at a time.

• there is no lock.

• ...

Based on the programming model of Cerl 1.0, we have found some of these problems in practice:

• "deadlock" occurs when network server A and B send synchronization messages to each other. Because both parties may be dealing with messages that cannot be responded to in a timely fashion, this "deadlock" eventually manifests itself as a timeout (but in fact the timeout is more frightening and is the killer of server performance).

• The solution to the deadlock is to change the synchronization message to asynchronous, but this has a big impact on programming complexity and the procedural semantics become obscure. Essentially, this is a return to the traditional asynchronous programming model, abandoning the greatest advantage of the Erlang programming model.

• ...

After discovering the "deadlock" problem, we immediately reflected on our entire programming model, decided to reconstruct the entire programming model, the CERL2.0 key points are as follows:

• can start any number of processes (where the process is abstract, the implementation is fiber/coprocessor), the maximum number of processes is limited by the memory size.

• There is no process mailbox.

• only synchronous messages between processes (to send asynchronous messages, to start a new process and send synchronization messages to achieve the same effect).

• The network server is a multi-process model, and each request is handled by a separate process.

• There is a lock.

The key difference between the two versions of the programming model is whether to reject the lock or reject the asynchronous message. Cerl 1.0 (that is, the Erlang programming model) rejects the lock, and Cerl 2.0 rejects the asynchronous message. Based on Cerl 2.0 we implemented the second and third editions of distributed storage. It turns out that this version of the server programming model has a much smaller mental burden when the concept of asynchronous messaging is completely eliminated.

Then, as you know, the Go language was released later. Our team looked at the server programming model and the Cerl 2.0 was exactly the same as the Go language (including all the details of the decision). A team dinner talk about this matter, Li Daobing said: We hurriedly put Cerl open source Bar, or so Go popular it will not be a chance. I really and Innovation Institute Dean Bryan Chan Talk about Cerl open-source things, the New Year said: No problem, you send an e-mail application bar, leave a voucher. However, I did not send this e-mail at last. After trying to write a week's code with Go, I had a conclusion: I wasn't going to let Cerl out. Because someone has already done its goal, and far more than expected.

So, in June 2011, when we left the Grand Innovation Institute to start seven Qiniu, I faced the third technology selection. This time, I was very determined to choose the Go language. To this end I also sent a special email to the team, a paragraph in the message said:

We face a lot of choices in the entrepreneurial process, and there will be many choices that will prove to be wrong, but today I can be sure that choosing Go will be the right choice for us.

• go really is a revolutionary language, its popularity will be of great significance to industrial development.

• go is still a small language, and we're not just going to recruit go programmers, but more importantly convince them that the go language is the direction of professional skills with great prospects.

• Seven the user of the cow is a programmer, we need to build the image of the expert in the user's mind.

Seven Qiniu start the first business is cloud storage, we chose to fully use Go to implement our storage system. This is the world's first cloud storage to write with go, and the first cloud service to write with go. By the year 2014, when we decided to move into big data, we were once again faced with the problem of technology selection. Frankly speaking, we have been struggling for some time. In terms of ecology, the choice of Java, or the language of a certain JVM platform, such as Scala, has significant advantages. Especially for our big data business in charge of Chen Chao, the avid fan of Scala (gossip: Chen's network ID is CRAZYJVM), the choice of the JVM platform seems to be a more unquestionable choice. So what do we struggle with? We believe that go will occupy the entire infrastructure sector in the future, and big data is undoubtedly one of the most important areas. So it is a problem to do the selection now, or to do the selection for the future.

Speaking of which, I would mention an interesting detail. When Chen Chao joined the seven cows, I said to him: "Regardless of the future you will use any language, but into seven cows must write go." So he was forced to write the Go code for one months. One months later, I asked him how he felt, and he said, "You can understand why you recommend go, write the go code and don't want to go back to writing Scala code." "Not long after, when he started the Pandora Big Data platform project, he came across what I said about the selection problem. After a very thoughtful thought, he decided to use Go as a Pandora. It was an unexpected decision for me. Because I do not have the inclination in this matter, and Chen Chao I think he will probably still prefer Scala. Just recently (National day before) a chat, Chen recalled this matter, summed up why he chose Go: very low learning costs, very low mental burden. If you use Scala, you need to train for new recruits and worry about writing bad Scala code. But with Go newcomers do not train directly, several times code Review after the basic will be able to know how to write a good quality Go code.

10 years went by, go is no longer a minority language. More and more people are using go, like go. Do not remember when, but one day through Google Trends search Golang found the world Go the most fire region is in China, that moment really very happy.

What will happen in the next 10 years? I know some people are looking forward to the iteration of the Go language feature. But if you have this idea you may be disappointed because Go won't change much in the next 10 years. The ability to predict and control long-term demand changes is one of the greatest charms of Go. This can be compared to go in C (C language different version of the specification difference is very small), but because go to solve more problems, it is actually more difficult to do this. In the next 10 years go will continue to build on the ecosystem of server-side development while actively exploring other potential application markets.

Extended reading

In QCon 2015 Shanghai Station, Xu Xiwei once did "Go language development status" theme share. For the Go language, he mentions three predictions:

• The first forecast, I mentioned in 2012 Sina Weibo, the Go language will certainly surpass C and Java in 10 years, become the first language of the leaderboard. Today, we may be able to believe slightly, but in 2012, not many people will believe that even today, most people will feel too early, after all, it has not ranked in the top five.

• The second forecast, the Go language will be in two years to dominate the cloud computing field.

• A third prediction, Go will rule the next 10 years. (from Deferpanic founder Lan Eyberg).

For more information, please click on the following title to read:

"Go will rule the next 10 years?" Analysis of the development status of Go language

Extension Practice

The text mentions the Scala Mad witch Chen teacher and he launched the Pandora Big Data platform project, in this QCon Shanghai station, we have the honor to invite the teacher Chen to share the technology selection, the component design, the system tuning and the structure evolution process, welcome to click "reads the original" Explore more technology sharing.

In addition, top technology experts such as Google, Uber, Paypal, LinkedIn, and Airbnb are also coming to share cutting-edge hands-on experience. Conference registration last week, next high-performance more convenient forward-looking technology please lock QCon Shanghai 2017. If you have questions, please contact the Ticket Manager Hanna, Tel: 15110019061,:qcon-0410.

Author Introduction

Xu Xiwei , CEO of seven Qiniu. ECUG Community Sponsor, domestic go language Practice circle recognized go language experts, and has "go language programming." More than 10 years of experience in the Internet industry, has worked in Jinshan, Shanda Technical research, is WPS2005 's chief architect. 2011 founded seven Qiniu.

Today's recommended text

Click on the image below to read

Xu Xiwei: The meaning of programming is to make the world a little more interesting