Translation Go Scheduler: M, P and G

This is a creation in Article, where the information may have evolved or changed.

This is another article about the Go Scheduler (Scheduler). Original: GO scheduler:ms, PS & GS by Uber engineer Povilas.

There are many articles on the Go Scheduler online, such as the Golang Scheduler source code Analysis, more looking at some, can deepen the memory, can also be compared to see if there is inaccurate in the article, a more comprehensive understanding of the GO scheduler.

I decided to delve into the internal mechanics of go, because no one had written about Go scheduler for a long time, and I thought it was a very interesting point of knowledge, so let's get started.

Basic knowledge

The Go runtime manages scheduling, garbage collection, and goroutine operating environments. This article focuses only on the scheduler.

The runtime is responsible for running goroutine and mapping them to the operating system thread. Goroutine is lighter than threads, and it costs little to start. Each goroutine is represented by a G -structure,
The fields of this struct are used to track the stack (stack) and state of this goroutine, so you can think of G = goroutine .

The runtime manages G and maps them to Logical Processor(called P). P can be thought of as an abstract resource or a context that needs to be fetched so that the operating system thread (called M) can run G.

Can be runtime.GOMAXPROCS (numLogicalProcessors) obtained by controlling how much P is available. If you need to adjust this parameter (most of the time you don't need to adjust), set it only once because it requires a STW GC pause.

Essentially, the operating system runs the thread, and the thread runs your code. The trick of Go is that the compiler will insert system calls in some places of the go runtime (such as sending values via channel, calling runtime packages, etc.), so go can notify the scheduler to perform specific operations.

The understanding from analysis of the Go Runtime scheduler

Interaction between M, P, and G

The interaction between M,P , and G is somewhat complex. Take a look at this picture from the Go Runtime Scheduler slideshow from Gao Chao:

As you can see, there are two types of queue in the Go runtime: One is a global queue (rarely used in the SCHEDT structure), and the other is a queue in which each P maintains its own G .

In order to run Goroutine, M needs to hold the context P. M will pop a goroutine from the queue of P and execute.

When you create a new Goroutine ( go func() method), it is put into the queue of P . Of course there is work-stealing a scheduling algorithm, when M executes some G , if its queue is empty, it randomly selects another P, and takes half the G from its queue To your own queue to execute. (Stealing!)

When your goroutine executes a blocked system call (Syscall), the blocked system call is interrupted (intercepted), and if there is some G executing at the moment, the thread is removed from P (Detach). , and then create a new operating system thread (if no idle thread is available) to serve this P.

When the system call continues, the goroutine is put into a local run queue, which is park added to the idle thread by the thread itself (hibernate).

If a goroutine executes a network call, the runtime performs a similar action. The call is interrupted, but because go uses the integrated network Poller, it has its own thread, so it is also given to it.

The go runtime will run another Goroutine if the following goroutine are blocked:

-Blocking Syscall (for example opening a file),
-Network input,
-Channel operations,
-Primitives in the sync package.

Scheduler Trace Debugging

Go can track the runtime scheduler, which is GODEBUG implemented by environment variables:

1	$ godebug=scheddetail=1, schedtrace= ./program

Here is an example of the output:

123456789101112

SCHED0Ms:gomaxprocs=8 idleprocs=7 threads=2 spinningthreads=0 idlethreads=0 runqueue=0 gcwaiting=0 nmidlelocked=0 stopwait=0 sysmonwait=0P0:status=1 schedtick=0 syscalltick=0 m=0 runqsize=0 gfreecnt=0P1:status=0 schedtick=0 syscalltick=0 m=-1 runqsize=0 gfreecnt=0P2:status=0 schedtick=0 syscalltick=0 m=-1 runqsize=0 gfreecnt=0P3:status=0 schedtick=0 syscalltick=0 m=-1 runqsize=0 gfreecnt=0P4:status=0 schedtick=0 syscalltick=0 m=-1 runqsize=0 gfreecnt=0P5:status=0 schedtick=0 syscalltick=0 m=-1 runqsize=0 gfreecnt=0P6:status=0 schedtick=0 syscalltick=0 m=-1 runqsize=0 gfreecnt=0P7:status=0 schedtick=0 syscalltick=0 m=-1 runqsize=0 gfreecnt=0M1:p=-1 curg=-1 mallocing=0 throwing=0 preemptoff= locks=1 dying=0 helpgc=0 spinning=false blocked=false lockedg=-1M0:p=0 curg=1 mallocing=0 throwing=0 preemptoff= locks=1 dying=0 helpgc=0 spinning=false blocked=false lockedg=1G1:status=8()m=0 lockedm=0

Note that the output uses the concepts of G,M , and p and their state, such as the size of the queue of p . If you don't want to care about these details, you can use:

1	godebug= schedtrace= /program.

William Kennedy wrote a good essay explaining the details.

Of course, there is a go-to-own tool go tool trace , which has a UI that allows you to view your programs and run-time conditions. You can read this article: Pusher.

Reference documents

1. Slides by Matthew Dale
2. Columbia University paper:analysis of the Go Runtime scheduler
3. Scalable Go Scheduler Design Doc
4. Hacker News Chat which explains a lot
5. Go Runtime Scheduler slides by Gao Chao
6. Morsmachine article