Dive into Go Playground Insider

This is a creation in Article, where the information may have evolved or changed.

Brief introduction

In September 2010, we introduced Go Playground, a Web server that consists entirely of Go code and returns the results of a program run.
If you're a go programmer, you've probably already used go playground by reading the Go tutorial or the sample program in the Go document.
You can also use it by clicking the "Run" button on the talks.golang.org slide or a program on a blog (such as a recent blog about strings).
In this article we will learn how go playground is implemented and integrated with other services. The implementation involves different operating systems and running times, and here we assume that all the systems used to write go are basically the same.

Overview

The playground service has three parts:
* A backend that runs on top of Google services. It receives the RPC request, compiles the user program with the GC tool, executes, and returns the output (or compilation error) of the program as an RPC response.
* A front end running on the Gae. It receives an HTTP request from the client and generates the appropriate RPC request to the backend. It also does some caching.
* A JavaScript client implements the user interface and generates an HTTP request to the front end.

Back end

The backend program itself is very simple, so here we don't discuss its implementation. The interesting part is how we securely execute arbitrary user code in a secure environment, while also providing core functionality such as time, network, and file systems.
To isolate user programs from Google's infrastructure, the backend runs them in native Client (or "NaCl"), Native Client (NaCl)-a Google-developed technology that allows the x86 program to execute securely in a Web browser. The backend uses a special version of the GC tool that can generate a NaCl executable file.

(This special tool will be merged into Go 1.3.) To learn more, read the design documentation. If you want to experience NaCl in advance, you can check out a branch that contains all the changes. ）

Local clients limit the amount of CPU and RAM used by programs, and also prevent programs from accessing the network and file system. However, this can lead to a problem with many of the key advantages of the Go program, such as concurrency and network access. In addition, access to the file system is critical for many programs. We need time capabilities to demonstrate high-performance concurrency. Obviously we need networks and file systems to show the benefits of accessing network and file systems.
Although these features are now supported, the first version of playground released in 2010 was not supported. The current time function is supported in November 2009, but is 10. Sleep is not available, and most systems and network-related packages are not supported
A year later, we implemented a pseudo-time on the playground, which allowed the program to have a proper sleep behavior. Newer playground updates introduce pseudo-network and pseudo-file systems, which make the playground tool chain the same as the normal Go tool chain. These newly introduced features are described in detail below.

Pseudo time

CPU time and memory are limited in the program available in playground. In addition, the actual time of application is also limited. This is because every program running in playground consumes background resources and occupies the infrastructure between the client and the backend. Limiting the uptime of each program makes our maintenance more accessible and protects us from denial-of-service attacks.
But these limits become very inappropriate when the program uses the time function function. In the Go Concurrency Patterns speech, an example is presented to illustrate this terrible problem. This is a time function function such as times. Sleep and time. After the example program, when running in the early playground, the hibernation of these programs fails and behaves strangely (sometimes even error)

By using a clever trick, we can make the GO program think it is in hibernation, while actually this hibernation does not take any time. Before we introduce this trick, we need to understand that the scheduler is the principle of managing goroutine sleep.
When a goroutine calls time. Sleep (or other similar functions), the scheduler adds a timer to the suspended timer heap and lets Goroutine hibernate. During this time, a special Goroutine calculator manages the heap. When this special Goroutine calculator starts to work, it first tells the scheduler that it wakes itself up when the next suspended timer in the heap is ready to be timed, and then it begins to hibernate itself. When this special timer is awakened, the first is to detect if there is a timer timeout, if so, wake up the corresponding goroutine, and then go back to hibernation.
Having understood this principle, the trick is simply to change the condition of the timer that wakes the goroutine. The scheduler does not wake up after a period of time, and only waits for a deadlock that all goroutines are blocked to occur.

An internal clock is maintained in the Playground runtime version. When the modified scheduler detects a deadlock, it checks to see if there are some pending timers. If so, it adjusts the time of the internal clock to the time of the earliest timer, and then wakes up the goroutine timer. This keeps repeating, and the program thinks that time is over, and actually hibernation is almost no time consuming.
Details of these scheduler changes are described in PROC.C and TIME.GOC.
Pseudo time solves the problem of exhaustion of background resources, but what about the output of the program? How strange it is to see a dormant program that does almost no time to get the job done correctly!

The following program outputs the current time per second and then exits after three seconds. Try to run it.

func main() {    stop := time.After(3 * time.Second)    tick := time.NewTicker(1 * time.Second)    defer tick.Stop()    for {        select {        case <-tick.C:            fmt.Println(time.Now())        case <-stop:            return        }    }}

How is this done? This is actually the result of working with the backend, front end, and client.
We capture the time of the output to standard output and standard error, and make this time available to the client. Then the client can output at the correct time interval, so that the output is the same as the local program output.

Playground's Run environment package provides a special write function that introduces a small "replay header" before each write to the data. The playback header contains a logical character, the current time, to write the data length. The playback header for a write operation is structured as follows:

0 0 P B <8-byte time> <4-byte data length> <data>

The original output of this program is similar to this:

\x00\x00PB\x11\x74\xef\xed\xe6\xb3\x2a\x00\x00\x00\x00\x1e2009-11-10 23:00:01 +0000 UTC\x00\x00PB\x11\x74\xef\xee\x22\x4d\xf4\x00\x00\x00\x00\x1e2009-11-10 23:00:02 +0000 UTC\x00\x00PB\x11\x74\xef\xee\x5d\xe8\xbe\x00\x00\x00\x00\x1e2009-11-10 23:00:03 +0000 UTC

The front end parses these outputs into a series of events and returns a JSON object to the client's list of events:

{    "Errors": "",    "Events": [        {            "Delay": 1000000000,            "Message": "2009-11-10 23:00:01 +0000 UTC\n"        },        {            "Delay": 1000000000,            "Message": "2009-11-10 23:00:02 +0000 UTC\n"        },        {            "Delay": 1000000000,            "Message": "2009-11-10 23:00:03 +0000 UTC\n"        }    ]}

The JavaScript client (which runs in the user's Web browser) then replays the event using the provided delay interval. It appears to the user that the program is running in real time.

Pseudo file System

Programs built on the Go Local client (NaCl) Toolchain are not able to access the local machine's file system. In order to solve this problem, a file access function (Open, Read, write, and so on) in the Syscall package is operated on a memory file system. This memory file system is implemented by the Syscall package itself. Since the Syscall package is an interface between the go code and the operating system memory, the user program treats the pseudo-file system as if it were a real file system.
The following example program writes data to a file so that the content is copied to standard output. Try to run it (you can edit it too)

func main() {    const filename = "/tmp/file.txt"    err := ioutil.WriteFile(filename, []byte("Hello, file system\n"), 0644)    if err != nil {        log.Fatal(err)    }    b, err := ioutil.ReadFile(filename)    if err != nil {        log.Fatal(err)    }    fmt.Printf("%s", b)}

When a process starts, this pseudo file system joins the device in the/dev directory and A/tmp empty directory. Then the program can operate on the same file system as usual, but after the process exits, all changes to the file system will be lost.

At initialization time, you can upload Zip files (see UNZIP_NACL.GO) so far only in the standard library testing, we will use the decompression tool to provide test data files. But we're going to playground the program to run data from sample documents, blog posts, and Golang tutorials.
For details, see Fs_nacl.go and Fd_nacl.go files (due to the _nacl suffix, these files are added to the Syscall package only if the GOOS is set to NaCl).
This pseudo file system is represented by the Fsys struct. One of the global instances (called FS) is created at initialization time. Various file-related functions operate on FS instead of making real system calls. For example, here is a syscall. Open function:

func Open(path string, openmode int, perm uint32) (fd int, err error) {    fs.mu.Lock()    defer fs.mu.Unlock()    f, err := fs.open(path, openmode, perm&0777|S_IFREG)    if err != nil {        return -1, err    }    return newFD(f), nil}

The file descriptor is recorded by a global fragment called files. Each filename descriptor corresponds to a file, and each of the files provides an implementation of the Fileimpl interface. Here are the implementations of several interfaces:
* Fsysfile represents regular documents and equipment (such as/dev/random),
* standard input and output and standard errors are instances of naclfile, which can use system calls to manipulate real files (this is the only way for programs in the playground to access the external environment,
* Network sockets have their own implementations, which are discussed in the following sections.

Pseudo Network access

Like the file system, the playground network stack is simulated by the Syscall package inside the process, which allows the playground project to use the loopback address (127.0.0.1). However, you cannot request additional hosts.

Run the following executable instance code. This program first listens to the TCP port, then waits for the connection to arrive, then copies the connection data to the standard output, and finally the program exits. In another goroutine, he connects to the listening port, writes the data to the connection, and finally shuts down.

func main() {    l, err := net.Listen("tcp", "127.0.0.1:4000")    if err != nil {        log.Fatal(err)    }    defer l.Close()    go dial()    c, err := l.Accept()    if err != nil {        log.Fatal(err)    }    defer c.Close()    io.Copy(os.Stdout, c)}func dial() {    c, err := net.Dial("tcp", "127.0.0.1:4000")    if err != nil {        log.Fatal(err)    }    defer c.Close()    c.Write([]byte("Hello, network\n"))}

The interface of the network is more complex than the file, so the implementation of the pseudo-network interface is much larger and more complex than the pseudo-file system. Pseudo-networks must simulate read and write timeouts, as well as handling different address types and protocols, and so on.
Specific implementation See NET_NACL.GO. It is recommended to start reading from Netfile because this is the implementation of the network socket for the Fileimpl interface.

Front

The front end of the playground is another simple program (less than 100 lines). Its primary function is to accept HTTP requests from the client and then issue the corresponding RPC requests to the background, while some cache work is done.
The front end provides an HTTP handler, see Http://golang.org/compile. This handler accepts a POST request with the body tag that contains the Go program code to run and an optional version tag (most clients should be ' 2 ').
When the current side receives an HTTP compile request, it first looks at the cache and checks to see if the same compilation request has been done before. If the presence of the same is found, the cached response is returned directly. Caching can prevent the background from being overloaded by popular programs like the Go home page. If the request has not been cached before, the front end sends the appropriate RPC request to the background, then caches the response from the background, analyzes the corresponding event replay (see Pseudo time), and finally returns the JSON-formatted object to the client via an HTTP response (as described above).

Client

Various sites that use playground share some of the same JavaScript code to build user access interfaces (code windows and output Windows, run buttons, and so on), which are used to playground front-end interaction through these interfaces.
Specifically implemented in the Playground.js file of the Go.tool repository, it can be imported through the Go.tools/godoc/static package. Some of the code is more concise and somewhat cumbersome, because it is merged by several different client-side code.
The playground function uses some HTML elements and then forms an interactive playground widget. You can use these functions if you want to add playground to your site.
The transport interface (informally defined, JavaScript script) is designed to be based on the Web site's front-end interaction. HTTPTransport is a transport implementation that can send HTTP-based protocols as described earlier. Sockettransport is another implementation that sends WebSocket (see ' Playing Offline ' below).
In order to comply with [homology policy] (Http://en.wikipedia.org/wiki/Same-origin_policy), various Web server (for example Godoc) through playground in http://golang.org/ Compile service to complete the proxy request. This agent is done through a common go.tools/playground package.

Run offline

Either the Go tour or the present tool can be run offline. This offline feature is great for people who have limited access to the network.
To run offline, these tools run a special version of the Playground backend locally. This particular backend uses the regular go
Tools, which do not have the modifications mentioned above, and use Websocker to communicate with the client.
WebSocket backend implementations are described in the Go.tools/playground/socket package. The code details were discussed in the inside present speech.

Other clients

Playground service is not just for the official use of Go projects (go by example is another example). We are happy that you can use the service at your site. Our only requirement is that you contact us in advance to use a unique user agent in your request (so that we can confirm your identity) and that the services you provide are beneficial to the go community.

Conclusion

Whether it's a godoc, a tour, or a blog,playground has become an integral part of the Go Documentation series. With the recent introduction of pseudo-file systems and pseudo-network stacks, we will be thrilled to refine our learning materials to cover these new content.
But, finally, playground is just the tip of the iceberg, and with the local client (Native client) going to support Go1.3, we look forward to the community making better features.

This article is an article from the Go Advent calendar on December 12, and Go Adventcalendar is a collection of blog posts.

Author Andrew Gerrand

Related articles

• Learn Go from your browser
• Introducing the Go Playground

Original: Inside the Go Playground
Reprinted from: Open source China Community--mitisky, Garfielt, Cmy00cmy, Java Grassroots