Golang Scheduler Policy

This is a creation in Article, where the information may have evolved or changed. We all know that the go language is native-supported language-level concurrency, and the smallest logical unit of concurrency is goroutine. Goroutine is a user-state thread provided by the go language, which is, of course, a thread of user-state running on a kernel-level thread. When we create a lot of goroutine, and they are all running on the same kernel thread, we need a scheduler to maintain these goroutine, ensure that all goroutine use CPU, and use CPU resources as fairly as possible. The principle and implementation of this scheduler is worth studying in depth. The main support of the entire scheduler has 4 important structures, namely M, G, P, Sched, the first three are defined in Runtime.h, Sched defined in PROC.C. The SCHED structure is the scheduler, which maintains queues with storage m and G, as well as some state information of the scheduler. M is a kernel-level thread, an M is a thread, Goroutine is running above M, and M is a large structure that maintains a lot of information such as the small Object memory cache (Mcache), the currently executing goroutine, the random number generator, and so on. P Full name is processor, the processor, its main purpose is to perform goroutine, so it also maintains a goroutine queue, which stores all the goroutine that need it to execute, the role of P can be a bit confusing, At first it was easy to clash with M, and the following was the key to talking about their relationship. G is the goroutine implementation of the core structure, g maintenance of the goroutine required stack, program counter and its location m and other information. Understanding the relationship between M, P, G is important to understand the entire scheduler, to illustrate its three relationships: the Hamster (Gopher) is carrying a pile of bricks to be processed in a trolley. M can be regarded as a hamster, p is a car, G is a car in the bricks. # # # # # # # # #启动过程在关心绝大多数程序的内部原理的时候, we're all trying to figure out its start-up process, figuring out how this process is critical to subsequent in-depth analysis. The assembly code in the Asm_amd64.s file _rt0_amd64 is the entire boot process, the core process is as follows: Callruntime args (SB) callruntime osinit (SB) Callruntime Hashinit ( SB) Callruntime Schedinit (SB)//Create a new goroutine to start Programpushq$runtime main F (SB)//entrypushq$0//arg siZecallruntime Newproc (SB) popqaxpopqax//start this mcallruntime mstart (SB) starts the process after the scheduler initializes the runtime Schedinit, Call runtime Newproc to create the first goroutine, the Goroutine will execute the function is runtime main, this first goroutine is called the main goroutine. We wrote the simplest go program "Hello,world" is to completely run in this goroutine, of course, any go program entrance is from this goroutine start. The last call to runtime Mstart is the real execution of the main goroutine created in the previous step. The Scheduler initialization runtime Schedinit function during startup mainly creates a batch of carts (p) based on the Gomaxprocs value set by the user, no matter how large the Gomaxprocs is set, and can only create up to 256 cars (p). These cars (p) are idle after the initial creation, i.e. they are not yet in use, so they are stored in the list of Pidle fields maintained in the scheduler structure (Sched) for future needs. Looking at the runtime main function, you can see that the first thing to do when the main goroutine is started is to create a new kernel thread (gopher m), but this thread is a special thread that is specifically responsible for doing specific things throughout the run-system monitoring (Sysmon). The next step is to go to the Go program's main function to start the GO program execution. The GO program is now up and running. A really work go program, must create a lot of goroutine, so after the go program started running, will add goroutine to the scheduler, the scheduler will be responsible for maintaining the normal execution of these goroutine. The Go keyword corresponds to the interface of the scheduler is runtime Newproc. The runtime Newproc is very simple, it is responsible for making a brick (g), and then put the Brick (g) into the current hamster (M) in the car (P). Each new goroutine need to have a stack of their own, the G-Structure of the Sched field to maintain the stack address and program counters and other information, this is the most basic scheduling information, that is to sayThis goroutine need to save this information when the CPU is discarded, and it needs to be loaded into the corresponding CPU register the next time the CPU is re-acquired. Assuming that a large number of Goroutne have been created at this time, it is up to the scheduler to maintain these goroutine. # # # # #创建内核线程 (M) There are no language-level keywords in the GO program that let you create a kernel thread, you can only create goroutine, and kernel threads can only be created by runtime according to the actual situation. When does runtime create a thread? To the hamster transport brick diagram, brick (G) too much, the hamster (M) and too little, really busy, just have free car (p) not used, then borrow some land from elsewhere (m) come over until the car (p) to run out. Here is a gopher (m) not enough to borrow from elsewhere (m) process, this process is to create a kernel thread (m). The interface function for creating m is: void Newm (void (*FN) (void), P *p) The core behavior of the NEWM function is to invoke the clone system call to create a kernel thread, where each kernel thread starts executing at the runtime Mstart function. The parameter p is a free car (p). Each of the created kernel threads is executed from the runtime Mstart function, and they will be assigned to their own car to move bricks. # # # # # # # # # #调度核心newm接口只是给新创建的M分配了一个空闲的P, which is the equivalent of telling borrowed Gopher (M)--"in the following days, you will use the number 1th car to move bricks, remember the number 1th car; "The Gopher (M) goes to get the car (P) the process is acquirep. Runtime Mstart The code in the P,runtime mstart function on the current m assembly before entering schedule:} else if (m! = &runtime M0) {acquirep (M->NEXTP); m >NEXTP = nil;} Schedule (); If the content of the branch is for the current m assembly on the P,NEXTP is the NEWM allocated to the idle car (P), but only then to really get hands. No p,m is unable to carry out goroutine, just like the ground mouse does not have the car can not be transported bricks the same reason. Corresponding to the action of the Acquirep is Releasep, the M assembly of P to load off, the work is done, the hamster needs to rest, the car also to the parking lot, and then go to sleep. When the Gopher (M) gets its own car (P), it enters the factory and begins to work, which is called the schedule above. The code for simplifying schedule is as follows: Static VoidschedulE (void) {G *gp;gp = Runqget (m->p); if (gp = = nil) GP = findrunnable (); if (m->p->runqhead! = M->p->runqtail &am P;&runtime atomicload (&runtime sched.nmspinning) = = 0 &&runtime atomicload (&runtime· Sched.npidle) > 0)//todo:fast Atomicwakep (); Execute (GP);} The simplification of the schedule function involves a 4-step logic: Runqget, a gopher (M) trying to remove a brick (G) from his own car (P), of course, the result may fail, that is, the hamster's car is empty, no bricks. Findrunnable, if the hamster own car does not have bricks, that can not idle work is it, so the hamster will try to run to the factory warehouse to take a brick to deal with, the factory warehouse may not have bricks ah, this situation, the land rat did not lazy stop work, but quietly run out, Randomly stare at a small partner (gopher) and then try to steal half of the bricks from his car into his car. If many attempts to steal bricks have failed, it shows that there is no brick can be moved, this time the rat will return to the car park, and then sleep rest. If the hamster sleeps, the following process is of course stopped, and the hamster sleeps in the thread sleep. Wakep, to this process, the poor hamster found himself in the car there are many bricks ah, they can not handle it, and then look back at the parking lot there is idle car, immediately ran to the dorm, your sister, incredibly still have a small partner in sleep, directly to the bottom of a foot, "Your sister, incredibly still sleeping, Lao Tzu is almost exhausted, Get up and work and share the job. The little buddy woke up and took his car and went to work. Sometimes, poor hamster ran to the dorm but found no sleep in the small partner, so will be very disappointed, finally had to tell the factory owner-"Parking lot and idle car ah, I can not move, hurriedly from other factories to borrow a gopher to help it." Finally, the factory owner got a new hamster to work on. Execute, the hamster took the bricks into the fire and lit up happily. Note: "Gopher steal brick" called Work stealing, a scheduling algorithm. Here, it seems that the entire factory is functioning normally, impeccable appearance. No, there is a doubt unresolved ah, suppose the rat's car has a lot of bricks, it put a brick into the stove, when to take it out, put in the second brick? Do you want to keep the first piece of brick in practice before you take it out?You coming? It is estimated that the brick behind is really waiting for the flowers to be thanked. Here is to really solve the Goroutine scheduling, context switching problem. # # # # # # # # # # #调度点 When we look at the channel's implementation code, we can see that the runtime Park function is triggered when the channel reads and writes. After Goroutine calls Park, the Goroutine is set to the bit waiting state, discarding the CPU. The Goroutine of Park is in waiting state, and this goroutine is not in the car (P), and if it is not called runtime ready, it will never be executed again. In addition to channel operation, the timer, network poll, etc. may be park goroutine. In addition to park can discard the CPU, calling the runtime gosched function can also let the current goroutine abandon the CPU, but unlike park, Gosched is to set Goroutine to Runnable State, Then put into the scheduler global waiting queue (that is, the factory warehouse mentioned above, this will understand why the factory warehouse will have bricks (G) it). In addition, it is the turn of the system call, some system calls will also trigger the rescheduling. The go language is completely its own package system call, so in the encapsulation system call, can do a lot of hands and feet, that is, enter the system call when the execution of Entersyscall, exit and execute the Exitsyscall function. Only system calls that encapsulate the entersyscall can trigger a reschedule, which will change the state of the Trolley (P) to Syscall. Do you remember the Sysmon thread that was mentioned at the beginning? This system monitoring thread will scan all the cars (p), found a car (p) in the state of the Syscall, it is known that the car (p) encountered Goroutine is making a system call, so the system monitoring thread will create a new Gopher (M) To get this car in the Syscall to Rob, began to work, so that all the bricks in the car (G) can bypass the previous system call waiting. Was robbed of the car, such as the hamster system call back, found that his car did not, can not continue to work, so can only execute the system call Goroutine put back to the factory warehouse, his sleep went. From the Goroutine dispatch point can be seen, the scheduler is still quite rough, scheduling granularity is a bit too big, fairness also did not think so good. In short, this scheduler is still relatively simple. # # # # # # # #现场处理 goroutine on the CPU swap, constantly context switch, must be guaranteed to save the scene and restore the scene, save the scene is Goroutine abandon the CPU when, the value of the relevant register is saved in memory, and the recovery site is the time when Goroutine regain the CPU, it needs to put all the previous register information back into the corresponding register from memory. Goroutine in the active abandonment of the CPU (park/gosched), will involve the call runtime McAll function, this function is also a compilation implementation, mainly Goroutine stack address and program counter saved to the G structure of the Sched field, The McAll was completed on-site preservation. The function to restore the scene is Runtime Gogocall, which is called primarily in execute, which requires reloading the appropriate registers before executing the goroutine. 676 reads