Stack space management for Go language

This is a creation in Article, where the information may have evolved or changed.

Translate the original link reprint/reprint please indicate the source

English original link published in 2014/09/15

At CloudFlare, we use the go language to build services and applications. In this blog post, we will analyze the technical features of the go language in depth. One of the most important features of the go language is goroutine. They are less expensive and work together to schedule threads to run. They are used for a wide range of purposes, such as implementing time-out control (timeouts), generators (generators), and competing with each other (racing) across multiple background applications. In order for Goroutine to be able to adapt to more tasks, we must make sure that each goroutine occupies very little memory. At the same time, people should be able to easily create goroutine.

To achieve these goals, the go language management stack looks like many other languages, but its implementation is really very different.

Thread Stacks Introduction

Before we start talking about the go language stack, let's look at how C is managing the stack.

When you start a thread in C, the standard library allocates a chunk of memory to be used as the thread's stack space. It first allocates a piece of memory, tells the kernel its address, and then lets the kernel control the running of the thread. If the allocated memory space is not large enough, the problem becomes complicated.

Let's take a look at the following function:

int a(int m, int n) {    if (m == 0) {        return n + 1;    } else if (m > 0 && n == 0) {        return a(m - 1, 1);    } else {        return a(m - 1, a(m, n - 1));    }}

This is a recursive function. a(4,5)the call will drain all of the stack memory. To avoid this problem, we can adjust the size of the memory space allocated by the standard library to the stack. But increasing this parameter will cause all threads to take up so much stack space, even if these functions do not require recursive invocation. In this case, although your program does not use the allocated stack, it will run out of all memory.

Another solution is to allocate stacks of different sizes to each thread. This allows you to configure the stack size for each thread, making it more cumbersome to create threads. It is often very difficult to decide how much memory a thread will use.

Solutions for Go languages

The Run environment (runtime) of the go language attempts to dynamically allocate stack space when goroutine is needed, rather than allocating a fixed size of memory space to each goroutine. This avoids the need for programmers to determine the size of the stack. The Go development team is trying to switch from one solution to another. The next step is to discuss the old solution and its drawbacks, then introduce the new scenario and the reason for choosing it.

chunked stack (segmented stacks)

A chunked stack is the way the first go language organization stacks. When creating a goroutine, it allocates a 8KB memory space to use for the goroutine stack.

What we are most interested in is when the 8KB stack space is exhausted. To handle this situation, each go function begins with a small piece of detection code. This code checks to see if we have exhausted the allocated stack space. If it is, it will call the morestack function. The morestack function allocates a new piece of memory as the stack space, and fills in the information at the bottom of the stack space (including the previous stack address). After allocating this new stack space, it retries the function that caused the stack to be out of space just now. This process is called stack split. When the stack is split, the stack structure is shown.

At the bottom of the newly allocated stack, a function pointer called Lessstack is also inserted. This function has not been called yet. This is set to be prepared from the return of the function that caused insufficient stack space just now. When we return from that function, it jumps to the lessstack . The lessstack function looks at the information in the data structure stored at the bottom of the stack, and then adjusts the stack pointer (stack pointer). This completes the jump from the new stack block to the old stack block. Next, the newly allocated block stack space can be freed.

The problem of the block-type stack

The chunked stack allows us to expand and shrink the size of the stack as needed. Programmers don't have to spend their energy estimating how much stack goroutine will use. It is not very expensive to create a new goroutine. When the programmer doesn't know how big the stack will be, it can handle the situation very well.

This has always been the way to the previous go language management stack. But there is a problem with this approach. Reducing the stack space is a relatively expensive operation. If there is a stack split in a loop, its overhead becomes negligible. A function expands and then splits the stack. When it returns, it releases the previously allocated block of memory. If it all happens in a loop, the cost is quite large.

This is the so-called thermal splitting problem (hot split problem). It is the main reason that the Go language developer chooses the new stack management method. The new method is called stack copying.

Stack copy method (stack copying)

The stack copy method starts with a block-like stack. When the Goroutine runs and finishes the stack space, the stack overflow check is triggered as in the previous method. However, unlike the previous method of allocating a new block of memory and linking to the old stack memory block, the new method allocates a memory block that is twice times larger and copies the old memory block contents into the new memory block. Doing this means that we don't need to do anything when the stack shrinks back to its previous size. There is no cost to the stack's reduction. Also, when the stack expands again, the environment does not need to do anything more. It can reuse previously allocated space.

How is the stack copied?

The copy of the stack sounds easy, but the actual operation is not that simple. The address of the variable stored on the stack may already be used. This means that the program uses some pointers to the stack. When you move the stack, all pointers to the contents of the stack become invalid. Fortunately, pointers to the contents of the stack themselves must also be stored on the stack. This is necessary to ensure memory security. Otherwise, a program may have access to an already invalid stack space.

Because of the need for garbage collection, we have to know which parts of the stack are being used as pointers. When we move the stack, we can update the pointers in the stack to point them to the new address. All relevant pointers will be updated. We use garbage-collected information to replicate the stack, but not any function that uses the stack has this information. Because a large part of the operating environment is written in C, many of the functions in the running environment are called without pointers, so they cannot be copied. When this happens, we can only go back to the chunked stack and pay the corresponding overhead. (Note: This part of the information is a bit outdated, but still worth reading!) ）

This is why the current running environment developer is rewriting most of the code in the run environment with the go language. Parts that cannot be rewritten with the go language, such as the scheduler's core code and the garbage collector, run on a special stack. The size of this special stack is set by the developer of the running environment.

In addition to making the stack copy possible, these changes allow us to implement parallel garbage collection in the future as well.

Say it again. Virtual memory

Another way to deal with stack space is to allocate a large chunk of virtual memory. Because the physical memory is actually allocated only when the memory address is accessed, it seems that we can simply allocate a large chunk of virtual memory and let the operating system do the rest of the work. But there are several problems with this approach.

First, the 32-bit system has only 4GB of virtual memory, and usually only 3GB of it can be used by the application. It is not uncommon to create millions of goroutine, when you are likely to run out of all virtual memory (even if we assume that the stack only uses 8KB of space).

Second, even if we can allocate a large amount of virtual memory in a 64-bit system, it relies on excessive use (overcommitting) of memory. Excessive use means that we allocate more virtual memory than the actual physical memory space and rely on the operating system to ensure that the physical memory that is needed is allocated. However, excessive use of virtual memory is a risky problem. Because a process really uses more memory space than the actual physical memory, it needs to start freeing up available physical space for new requirements. It usually saves the contents of a piece of memory to disk. This causes the delay to be unpredictable. For this reason, we usually do not use memory too much in the system.

Conclusion

To make Goroutine lightweight, fast, and suitable for most tasks, developers have done a lot of work. The management of stacks is only a small part of it. If you want to learn more about stack duplication, this design document provides more detail.

If you want to learn more about rewriting the go language runtime, you can read the following article in this mailing list.