Goroutine related knowledge in go language

Go language from birth to popularity has been three years, most of the pioneers are the background of web development, there are some popular books, system development background of the people in the study of these books, there is always vague feeling, online also has a number of widely circulated articles, can be more or less always some inconsistent with the facts of the technical description. It is hoped that this article will introduce the system knowledge behind Goroutine for Web developers who are less systematic in programming backgrounds.

1. Operating system and Run-time library

For ordinary computer users, it is enough to understand that the application is running on the operating system, but for developers, we need to understand how the programs we write are running on top of the operating system, how the operating system provides services to the application, so that we can identify which services are provided by the operating system, And which services are provided by the runtime of the language we use.

In addition to internal modules such as memory management, file management, process management, peripheral management, and so on, the operating system provides many external interfaces for applications to use, and these interfaces are called "system calls." Starting with the DOS ERA, system call is provided through the form of soft interrupts, that is, the well-known int 21, the program to call the function number into the AH register, put the parameters into the other specified registers, and then call Int 21, after the interrupt returned, program from the specified register (usually AL) Gets the return value. This has not changed until the Pentium 2, P6, for example, Windows through the int 2E to provide system calls, Linux is int 80, but later registers than before, and may be more than one level of Jump table query. Later, Intel and AMD provided more efficient sysenter/sysexit and syscall/sysret instructions to replace the previous interrupt mode, skipping the time-consuming privilege level check and the register stack stack operation, directly completing the Ring 3 code segment to the ring 0 conversion of the code snippet.

What functionality does the system call provide? Use the name of the operating system plus the corresponding interrupt number to Google to get a complete list (Windows, Linux), this list is the communication between the operating system and the application of the Protocol, if the need to exceed the functionality of this protocol, we can only be implemented in their own code, for example, for memory management , the operating system only provides management of process-level memory segments, such as Windows Virtualmemory Series, or Linux BRK, the operating system does not care about how the application allocates memory for new objects or how to do garbage collection, which needs to be implemented by the application itself. If the functionality beyond this protocol cannot be implemented on its own, we will say that the operating system does not support this feature, for example, Linux does not support multithreading until 2.6, and in any case, we cannot make multiple dispatch units that can be run simultaneously and that conform to POSIX 1003.1c semantic standards.

However, we do not need to write programs to invoke interrupts or syscall instructions, because the operating system provides a layer of encapsulation, on Windows, it is NTDLL.DLL, which is often said native API, we do not need to directly call int 2E or syscall, To be exact, we cannot directly invoke int 2E or syscall, because Windows does not expose its calling specification, and direct use of int 2E or syscall does not guarantee future compatibility. There is no such problem on Linux, the list of system calls is public, and Linus is very important to compatibility, will not make any changes, glibc even specifically provided syscall (2) to facilitate the user directly with the number of calls, however, To address the hassle of different versions of compatibility between GLIBC and the kernel, and to improve the efficiency of some calls (such as _nr Gettimeofday), Linux has a layer of encapsulation of some system calls, VDSO (early called linux-gate.so).

However, we write programs rarely directly call ntdll or VDSO, rather, it is through a more layer of encapsulation, which handles parameter preparation and return value format conversions, and error handling and code conversion, which is the runtime of the language we use, and for C, Linux is glibc, Windows is kernel32 (or call MSVCRT), and for other languages, such as Java, the JRE, the runtime of these "other languages" usually ultimately calls glibc or KERNEL32.

The term "runtime" actually includes not only the library files used to link to the compiled target execution program, but also the operating environment of the scripting language or bytecode interpretation language, such as Python,c# 's Clr,java JRE.

Encapsulation of system calls is only a small part of the runtime's functionality, the runtime typically also provides features such as string processing, mathematical calculations, commonly used data structure containers, and so on that do not require operating system support, while the runtime provides more advanced encapsulation for features supported by the operating system, such as iOS with caching and formatting, Thread pool.

So, when we say "a certain language has added a certain function", there are usually several possibilities:

Support new semantics or syntax so that we can describe and solve problems. such as Java generics, Annotation, lambda expressions.

Provides a new tool or class library that reduces the amount of code we develop. Like Python 2.7 's argparse.

A better and more comprehensive encapsulation of system calls enables us to do things that we did not or could not do in the language environment before. such as Java NIO

But any language, including its runtime and operating environment, it is impossible to create features that are not supported by the operating system, as is the case with the go language, no matter how flashy the feature description may seem, it must be done by other languages, except that goes provides more convenient and clearer semantics and support and improves development efficiency.

2. Concurrency and parallelism (concurrency and Parallelism)

Concurrency refers to the logical structure of a program. Non-concurrent program is a bamboo stick to the end, only a logical control flow, that is, sequential execution (sequential) program, at any time, the program will only be in this logical control of the flow of a location. And if a program has multiple independent logical control flows, that is, it can handle (deal) many things at the same time, we say that the program is concurrent. The "at the same time" here is not necessarily true at some point in the clock (that is, the running state rather than the logical structure), but rather: if these logical control streams are drawn as sequential flowcharts, they can overlap on the timeline.

Parallelism refers to the running state of a program. If a program is processed at a time by multiple CPU lines, then we say that the program is running in parallel. (Strictly speaking, we can't say that a program is "parallel", because "parallel" is not the description of the program itself, but describes the running state of the program, but this small text is not so literal, the following when "parallel", refers to the "Run in parallel form") Obviously, parallelism must be required hardware support.

And it's not hard to understand:

Concurrency is a necessary condition for parallelism, if a program itself is not concurrent, that is, there is only one logical control flow, then we can not let it be processed in parallel.

Concurrency is not a sufficient condition for parallelism, a concurrent program that is not parallel if it is processed by only one CPU line (through time-sharing).

Concurrency is only more in line with the nature of the real problem of expression, the original purpose of concurrency is to simplify the code logic, rather than make the program run faster;

These paragraphs are slightly abstract, and we can instantiate these concepts in one of the simplest examples: to write the simplest HelloWorld in C, which is not concurrent, if we build multiple threads and each thread prints a HelloWorld, then the program is concurrent, If the program is running on an older single core CPU, then the concurrent program is not parallel, and if we run it with multi-core, multiple CPUs and multitasking operating systems, then the concurrent program is parallel.

There is also a slightly more complicated example of the fact that concurrency is not necessarily parallel, and that concurrency is not for efficiency, which is the sieve.go of the number of primes in the Go language example. We started a code fragment for each factor from a small size, if the number of currently validated can be removed by the current factor, the number is not prime, if not, the number is sent to the code fragment of the next factor until the last factor is not exhausted, and then the number is prime, and we then start a code fragment for Verify the larger number. This is in line with the logic of our prime calculation, and each factor's code processing fragment is the same, so the program is very concise, but it can not be parallel, because each fragment depends on the previous fragment processing results and output.