Manual memory management in the Go language

This is a creation in Article, where the information may have evolved or changed.

Introduced Note: If you have a different view of this article, please correct me-I am not an expert in this field. We have collected performance statistics on the use of defer and panic from a large number of go users. Unlike other Apm/error record companies, our focus is not to tell you that there is a problem, but rather how to actually solve the problem. That's why we want to use go. This makes us see what is good, bad and ugly for everyone. When you hide a big problem in your project, if you're just looking back and forth, you usually can't find it until you test it. Many performance issues may have been around for several weeks.

A song Translated 2 weeks ago 0 Person Top top translation of good Oh!

Usually most of these problems can be easily passed pprof and other methods to analyze the target program to solve the problem. Once you suspect that a place is wrong, you should look at this first (pprof). However, sometimes, when the source of the problem cannot be found for a few days, the ape begins to complain. is a problem with the database! Problems with the Framework! is a cat (computer-aided program) problem! (cat? ...). is a problem with the garbage collector! Every time this point, a guy in the team jumps out and he uses a better language (-_-! , so if you are not a software engineer, you will be confused by him. I'm not talking about language, cats (cat again?). ) or frame. I just want to guide you through a more efficient approach.

translated 2 weeks ago 0 human top top   translation is good Oh!

I do want to talk about the garbage collector. It is a performance problem that is often mentioned in most languages, but if the problem can be solved easily, it will not attract people's attention. The argument of the Against the garbage collector, for quite some time, has been a quick catch up with hard-coded compilations. Of course, if you know everything you need to know about target compilers, target architectures, target operating systems, and so on, it makes sense to choose one or two items to optimize, but basically, you're better than some of the best compiler authors. You may have the ability to do so. However, this is a fairly high requirement. In fact, there are many garbage collector, such as  azul's memory collector, that people find quite good.

gones945 Translated 1 weeks ago 1 human top top   translation is good Oh!

background /h2> "C programmers think that memory management is too important to be handled by the computer itself. Lisp programmers think that memory management is too important to be handled by users. "-- bjarne Stroustrup To start using-most modern programming languages use automated garbage collection mechanisms. The has a garbage collector either in Java or C #. Also, there are some explanatory languages. The John McCarthy (John McCarthy) invented a "garbage collector" on his way back in 1958. This was the second year of the successful launch of Sputnik. And then he invented something else. So, this concept has been quite a long time, but some developers seem to be uncomfortable with the concept. Why is it? To answer this question, we should look at the memory management of various methods.

Happybks Translated 2 weeks ago 0 Human Top top   Good translation!

memory Managed methods Here are some methods of memory management memory, you can: * Allocate all memory at compile time * Manage memory manually * Manage memory automatically For our definition, We believe that garbage is the allocated memory that has been used. Also, for this blog, the only difference is that the garbage can be manually cleaned by the programmer (or not cleaned up), or automatically cleaned up by the programming language. There are a number of issues in the process of managing memory. Some of these problems are obvious, some are not easy to detect, some are not easy to solve, and others are not even aware of their existence. This usually just wants to be sure, because--you usually have a garbage collector!! :)

gones945 Translated 1 weeks ago 0 Human top top   Good translation!

management In-memory problem: * Reference memory that is no longer in use * Pointer to unallocated memory * Allocate memory but never release * Use free memory pointer * Not enough memory allocated * Allocate Memory size error Error * Allocating or freeing memory too fast or too slow * Load memory before storing data in memory * Safe removal of sensitive data from memory * Internal fragmentation * External fragmentation Actually--security industry statistics, The error with memory management involves the billions of dollar. This is obviously a problem.

gones945 Translated 1 weeks ago 0 Human Top top   Good translation!

basic Memory management operations Memory management has its uniqueness in computer science, but here are two of the most common and important operations: Assignment: This is the operation that allocates a piece of memory to the specified request. Also ensure that the allocator does not allocate this memory to other places unless you say no. release/reuse: This is when you say--this memory has been used, and you can handle it freely in the way you feel fit. The memory is not "deleted". The memory is not "released" into the air, or anywhere else. What is done is that the memory is marked so that it can be reused the next time the request is made.

gones945 Translated 1 weeks ago 0 Human Top top   Good translation!

about inside Storage Management error hypothesis reserved vs. assigning One thing that many people are puzzled about is the difference between allocating memory and reserving memory. Go language   Set aside a large chunk of memory (reserves a large chunk of memories right off the bat)  . There are a number of reasons for this-one of which is continuous vs. discontinuous memory mapping. You want to reduce the fragmentation as much as possible. For us, we define a reservation as a call to the malloc function, but when we decide to use this memory, the memory allocation really happens. As long as there is no reserved memory space, we can call the malloc function at will. However, when we use memory and consume it, we get an out-of-memory (OOM) error message.

gones945 Translated 6 days ago 0 Human Top top   Good translation!

operating system System interrupts Developers think that if they can allocate and free memory, their programs will not be interrupted and slowed by GC calls. If the above assumptions are true, the problem with this logic is that they forget that the operating system itself can block the allocation of memory, the scheduler switches to other tasks, and all device drivers have interrupt processing. Your code does not exist in isolation, and there are too many factors that cause it to pause. memory release is not necessarily free Using a good GC can actually reduce memory allocation time. Freeing up memory at the same time does not mean releasing the memory immediately, and any version of malloc you use will put it in a release list-  This will help prevent fragmentation.   Of course, the release of the stored time is not free, it can be random, you need to paging, resulting in bad IO. Imagine doing a lot of small memory allocations, and you will cause a lot of fragmentation in order to avoid crashing the stack size that has to be redistributed.

Wancheng Translated 5 days ago 0 Human Top top   Good translation!

fragment I've already mentioned fragments several times-let me tell you a simple example. For example, we have a place where we can dynamically allocate memory. There are 4 slots that we can access. However, every time we request memory, we can only get 2 slots. The first time we allocated, we grabbed the first two slots. Then we assert that we no longer need the first slot, so we "deleted" it. All right, that's great. That's what it looks like now. Now a large database job comes, and we also need a slot. Too bad, our allocation mechanism only allows us to get 2 slots at a time-so what happens? We got the next two slots. Now we have allocated three-fourths of the slot , even if we only used 2 of them. So go on, until all that remains is that we can't clip these already allocated slots in the middle using a lot of free gaps. This is the fragment that makes us mad.

Happybks Translated 1 weeks ago 0 people top top   Good translation!

configuration policy Generally speaking, the classic allocation policy can be in two forms: {An array of single lists or lists}: I will not introduce array implementations, nor do we talk about the more exotic strategies that exist. We say a linked list that is similar to the one above. There are a couple of policy options for finding a suitable address (slot) in a single idle single-link list: First-time adaptation ( primary fit ): We scan the entire list to find the first location that fits our allocation request. Loop first adaptation ( next fit ): Similar to first-time adaptation, but we track the top one visit, and then start looking for the next place to meet the request, so you don't have to start scanning from the head of the list every time. Optimal strategy ( best Fit ): Look for the smallest block in memory that meets the requirements. Worst-case policy ( worst fit ): Look for the largest block in memory that meets the requirements.

Happybks Translated 1 weeks ago 0 people top top   Good translation!

go garbage collection to implement this type of garbage collection immediately, go is a very young language. There is no doubt that there will be a lot of improvements in the future, but it will be an extraordinary task and someone needs to work on it. I'm sure not everyone has such a talent. Two different methods have many advantages in garbage collection: {Trace, reference count} 1.4  the classic STW garbage collector. 1.5   The introduces a concurrency collector. The is both token and clear, which is an implementation of the trace collector. They are non-mobile, which means that the garbage collector will not move the reference like a compressible collector. You can turn off the garbage collector by setting environment variables: gogc=off ./myproggie However, this is meaningless unless you want to run the collector manually-and the environment itself allocates memory- So you produce trash that someone will deal with it. Do you drag trash to a place every day? Of course not, you have more important things to do. As for why go has a garbage collector-I don't think that's enough to drive people to do it. One of the main selling points of this language is concurrency. Traditional memory management is hard to come by, let alone to implement garbage collection in a concurrent environment. See horse ' s mouth.

Happybks Translated 1 weeks ago 0 Person Top top translation of good Oh!

Anti-pattern Finally cut to the chase ... Many developers choose to use high-level languages to manage memory transactions. Many developers often run into memory problems because all memory management is hidden from them. This does not mean that most of the memory is hidden. But this does not mean that they are poor developers, but that these memory transactions are not in their thinking. This should not be the homework they have to do to develop their own code money. I suspect that most of the problems we see with users come from simple but often overlooked patterns, as follows: String connection There is a well-known anti-pattern in Java that we can use to link strings in a ring structure instead of using string generators. Well, go also has its own anti-pattern. The thing to know is that when you concat a string, go assigns a new address to the new value. So, things have become so simple: ? 1234 &NBSP;&NBSP; BLAH&NBSP;:=&NBSP; "stuff"   +&NBSP; " More stuff "   +&NBSP; " Even more  stuff " &NBSP;&NBSP; for     i:=0; i<10*10*10; i++ {       BLAH&NBSP;+=&NBSP;  +&NBSP; " ad  Infinitum " &NBSP;&NBSP; } Many people have more to do with the labor. You can refer to the Stdlib @strings for this way. Join. You can also use bytes. Buffer.writestring.

Happybks Translated 1 weeks ago 1 Person top top translation of good Oh!

manual Manage memory Use cases Do you still want to manage it manually? All right. Some real-world use cases might include the following: * Real large heap-assuming large in-memory database operations; * Manage objects of the same size of the same type; * Real-time system-this is not a good use case because it's not a go-to feature, You'd better choose a completely different language. write it yourself or use it to do Honestly, with all due respect, unless you're an expert (I'm sure not), there's not a lot of good use cases here. If you are, I may suggest that you continue to work with go. Most people do not write underlying device instructions or real-time software. I'm not talking about real-time node.js/websockets and other similar software, we're talking about specific operating systems that exist on your phone instead of Linux's second operating system. &NBSP If you have to do this: * Your disk is heavily fragmented; * Your disk is not thread safe; * You think the garbage collector is confusing, so you can close it and we've already mentioned that there are other problems.

Wancheng Translated 5 days ago 0 Human Top top   Good translation!

or do you manage it yourself? Well, you can use hacky to manage it might not be as effective as in C, just use CGO to link to it. or You can use STDLIB's stuff: sync. Pool or You can also use a buffered channel. In addition, as more and more references are available, see how couchbase  and CloudFlare implement them. Monitoring Remember earlier in this article, I said, most people don't realize a problem until it becomes a big problem or something that you should weigh? We just happen to do it, and for go it's active: We can even remind you before things get unbearable. In addition, since we only support go, our tools are custom-made for go, in addition to the dishwashing tank. (translator added: "Dishwashing" allusions from the World War II idiom "Everything but the kitchen sink", when the enemy artillery fire violently (in addition to the dishwasher, all kinds of shells Qi), now refers to too many things)

Happybks Translated 2 weeks ago 0 people top top   Good translation!

Packaging There are many ways to manually manage memory in the go language, and it doesn't seem to solve the problem we need to solve. I think we should focus on the solution to the problem of garbage collection, as Blade paper put it in the real sense. But you really don't have to resort to these tools. If you find yourself complaining about the garbage collector, then you should immediately take out the pprof and blame the damn garbage collector. If so, are you sure you want to consider improving it?

Happybks Translated 1 weeks ago 0 Person Top top translation of good Oh!

All translations in this article are for learning and communication purposes only, please be sure to indicate the translator, source, and link to this article.
Our translation work in accordance with the CC agreement, if our work has violated your rights and interests, please contact us promptly