Say this "Why am I moving from python to go

This is a creation in Article, where the information may have evolved or changed. author CMGS 2015.05.17 15:47* Wrote 7891 words, was paid attention by 143 people, gained 97 likes

Say this "Why am I turning from Python to go"

words 3748 read 24227 comments like Bayi

Well read this article why do I move from Python to go, it seems that the author is also KSO light office/Enterprise Fast team. As one of the engineers in the fast-from-scratch period (always by the handsome brother said they changed my left bug), and happened to be python/go double repair (fog actually I was Rust Party), in fact, I was rejected, Duang Duang Duang, then write a little bit.

Come on, first of all, the author says Python is a dynamic language.

Python is a dynamic language, not a strongly typed system. For a variable, we sometimes do not know what type it is, and then there may be a run-time error such as INT + string.

In Python, you can allow a function with the same name to appear, and the latter will overwrite the previous function, and one of our system's most serious errors is because of this.

In fact, if it is a static check, Pylint and pyflakes can do it, although not with the static compiler language of go, but also enough. If you remember correctly, it was asked to do a static check before submitting the whole group. I think this problem more should be the quality of personnel to avoid, after all, Onion also said that the code to write their own to look back to see if it can be reconstructed, can do better. Not that lazy, but the Python dynamic features are too flexible, Python: Blame me?

In addition, the function as the first object, in Python is feature,go to write a mock, it is almost abusive.

This has always been a place where many people spit out python, but think about what Python was developed to solve? We just want to use him to high-performance server development above, in fact, it is a bit difficult.

If I remember correctly, whether it is the light office or the fast disk, is heavy IO does not heavy CPU, the maximum time is the data block encryption that block, I was in the Java write. In addition the high-performance server Choose Go is also abuse not to be, various cautious avoid GC. In most extreme cases, the performance of PyPy is sufficient, and I don't think it is sufficient.

Python's Gil leads to the inability to really multi-threading, and everyone might say I'm not going to end up with multiple processes. However, if some computations involve multi-process interaction, the communication overhead between processes has to be considered.

In fact, Python has a macro that can bypass the GIL, but the architecture is well designed to avoid, to the asynchronous piece I would say.

Stateless distributed processing is convenient for using multiple processes, such as handling HTTP requests, and we are mounting more than 200 Django servers behind Nginx to handle HTTP, but so many processes naturally lead to high overall machine load.

But even if we use multiple Django processes to handle HTTP requests, Python still can't handle some of the extra requests. So we use Openresty to implement high-frequency HTTP requests using LUA. This in turn leads to the use of two development languages, and some logic has to write two different codes.

If that's true, you're still using Gateway written five years ago? The traffic distribution layer based on the Django route? Four years ago, when I left, I had a small-scale use of the Flask+gevent Demo test, whether performance or load than the synchronous model of Django has the advantage. If it is the same as Django, I can only say that the more regrettable, after all, Jinshan new staff contest is the number I and a few small partners to write the real-time synchronization of online document editing system, with this set of technology.

So it's an engineering problem, not a language problem. Python gives you so many tools, hard to choose a traditional, old fashion, Python: Blame me?

The Django Network is a synchronous block, that is, if we need to access a service outside, Django cannot handle any other logic while waiting for the result to return (except, of course, multithreading). If it takes a long time to access an external service, it means that our entire service is completely unavailable for almost a long time.

In order to solve this problem, we can only continue to open the Django process, but also to ensure that all services can quickly handle the response, but think this is really a very unreliable thing.

Synchronization model is not a bad thing, because the overhead is low enough, many business scenarios with the synchronization model but will achieve better results, such as watercress. The biggest problem with the synchronization model is that it's long enough to wait for the IO-intensive business, and instead of changing the language, it's a reminder that the architecture needs to be changed.

Although Tornado is asynchronous, Python's MySQL library does not support asynchrony, which means that if we access the database in tornado, we may still face the fact that the entire service is not available due to database problems.

Tornado is having this problem, but gevent has been solved. I have replied to a question in node. js that for node, there is only one asynchronous model that can be selected, and Python is too many choices. In addition, Pypy+tornado+redis can abuse a variety of long-connected scenes, such as I have written to my factory a push service.

In fact, the biggest problem of the asynchronous model is the fragmentation of the code logic, because it is triggered by the event, so we are all through callback related processing, so the code inside often appear to do one thing, pass a callback, and then callback inside and pass callback situation, The result is that the entire code logic is very confusing.

This is really not, if there is no ES6 JavaScript, may really have Callback hell, but this is Python Ah! Python has long been the implementation of the lvalue binding alas, yield that posture than some of the daily blowing language do not know where to go, of course, I am talking about the full version of Python3 yield. Even incomplete Python 2 yield is fully sufficient for asynchronous expression evaluation, Tornado gen.coroutine.

Synchronous form-write asynchronous, in Python powerful companies in the early popularity, this is an engineering problem, not a language problem. Of course I blame Python for this kind of thing, Python: Blame me?

Python does not have native coprocessor support, although it is possible to support the process by gevent,greenlet this patch, but after all, the Python source code is changed. In addition, the yield of Python can also be a simple co-simulation, but after all, can not cross the stack, the limitations are very large, do not know whether the 3.x version has been improved.

Whether it is gevent or Greenlet are not modified Python source code, in fact, the goods has become the standard of Py2 coroutine, plus the open source greenify, basically all the libraries can be smooth asynchronous, including MySQL and other C-level Lib 。 Since the use of this technology, watercress Python dev all kinds of cool not to.

When I first used the Python development project, I did not successfully install the package required for the project, the installation of the successful MySQL library has been a long time. Later, it was a colleague who packaged the entire Python directory for me so I could run the project normally. In other words, now that there is Docker, it is one thing that makes people happy.

While deploying the Python service, we need to install a bunch of packages on the server, which is very troublesome, although it is possible to solve the deployment problem by puppet,salt these automation tools, but compared to the static compilation language only throw a binary file, it is much more convenient.

Just as I was developing a Docker-based platform, Docker wasn't really used to deploy it. First of all, Python has virtualenv this tool, in fact, compared to packet management and packet isolation, python than go high do not know where to go. When Python is talking to Git, the Go Dev also has to think about how I can make the import package stable on a single version (of course there are a lot of third-party scenarios now). Virtualenv + PIP is fully automated for Python deployment, so I think that's the problem with tool chain selection. After all, is a more than 10 years old monster, Python what situation has not seen Ah, a variety of packaging tools to choose, forcibly said Python deployment inconvenient, Python: Blame me?

Python is very flexible and simple, the ability to write C dozens of lines of code can be done, Python line of code can not be solved. But it is too simple, but many students can not be in-depth thinking of the code, the entire architecture for careful consideration. Come a demand, PA PA, the keyboard knocked out the speed of implementation, the result is more and more chaotic code, resulting in the entire project code out of control.

Once I know that there is a post asking whether Python will reduce programmer programming ability, I can only say that this is really very relevant. You do not think deep-seated things strange language does not make sense, that good, Go inside Goroutine is how to realize, a with socket goroutine minimum can do how much memory, think about? Any language has its own advantages and disadvantages, all need the executor to judge, blindly feel simple will not think deeply that there is a problem. In addition, the code confusion I think or engineering control is not enough, watercress has more than 10W line of Python implementation, although not perfect, generally do not mess with this goal.

Also, C write dozens of lines to fix the Python line to solve this is definitely a major feature, productivity ah, staffing Ah, recruit the cost Ah, from the project, Python in this piece is completely add sub-item, not every project requires extreme concurrency, the ultimate efficiency, Doing projects is a lot of time to choose.

Although both Java and PHP are the best programming languages (everyone is so competitive), I prefer a simpler language. While Openresty, though powerful, Lua is still a dynamic language, and will encounter some of the previous dynamic language problems. Finally, before Jinshan Xu Xiwei used the go, the former fast-plate architect Onion also used go, so we naturally chose go.

Openresty with Lua If you look at it in terms of dynamic language, it's really not, at best, a simple C. Xu Xiwei Go when most are CPP, onion at present I do not know what he used to write a business, but he certainly has no language inclination. Either Leo or UFA, one with Python in Java, he chooses what language to use from the actual project.

Error, well, if a student who has a neat language may be really sick of the grammar of Go, especially the last return value of the contract is error.

This is actually go style, whether it's go FMT or error style,go is actually trying to smooth out the style problem between different engineers. No longer a waste of time for one indent and brace position. This is not a bad way, but I personally think that the return value of rust is not friendly.

Gc,java GC Development for 20 years, go just so little time, GC is definitely not perfect. So we still can't write the code as we want, or the GC may be the whole service under the large request volume. So sometimes, the use of object pool, memory pool must be used, although the code is ugly point, but somehow performance up.

1.4 Start Go is 100% accurate GC, in addition to the lag ah, completely and how you use the object, can inline never refer to most of the scene is completely enough, so the impact of GC is minimal. Really want to use the pool ... Can only say why not choose Java.

Natural parallel support, because Goroutine and channel, with go write distributed applications, write concurrency program exception easy. Without the callback of the egg, the code logic is fragmented and the code logic is sequential.

This is a cost, goroutine memory consumption calculation (of course 1.3 or 1.4 began to be greatly improved, the memory minimum limit has been lost), the channel cross-threading performance loss (jumper lock), as well as the Goroutine control force of almost 0 and so on. In short, this is not the killer characteristics, we all have, is convenient a bit, but also have their own shortcomings. For example, we use Go bar, often compared to the egg spawn out of the goroutine how graceful shutdown, but sometimes complicate things.

Performance, go performance may not be able to catch up with c,c++ and openresty, but it's also pretty tough. In our project, a go process is now deployed on a single machine, fully capable of doing what was done in the previous 200 Python processes, with lower CPU and mem usage.

I am not rigorous measurement probably gevent+py2 can achieve the same logic go implementation of the 30%~40%,pypy+tornado can achieve 80%~90%, mixed with a number of calculations and connection processing. The main thing is to see the business scenario, pure CPU bound of course is go good, pure IO bound you just use C also useless ah.

Operations deployment, directly compiled into binary, thrown onto the server, more than Python needs to install a bunch of environment that is simply too much. Of course, if there are CGO, we also need to throw the corresponding dynamic library to the past.

We are now based on the host version of the glibc there are 2 sets of compilation environment, it seems to be a simple deployment, compiled to pit you dead. In addition, although the disk is cheap, these lines of code is a few m, cluster synchronization deployment time in some cases will really go out of the basket.

Development efficiency, although go is a static language, but I personally feel that the development efficiency is really very high, intuition above the same as Python. For me personally, the best example is that I quickly developed a lot of open source components, such as Ledisdb,go-mysql, with go, and the first versions were done in a very short time. For our project, we also completed the first version of refactoring with go in one months, and released it.

Go development efficiency is compared to C, compared to Python, probably the latter only need 3 days ...

In short, Go is not bad, Python is not no, do engineering, no outside is to consider costs, time costs, labor costs, maintenance costs and so on. Go and Python have a mutual, it's a choice. Of course, you have to say Python is not, Python: Blame me?