Small glimpse Go AST Library and preliminary use

This is a creation in Article, where the information may have evolved or changed.

This article requires you to have experience writing Golang code, which takes about 20 minutes to read.

We've been studying Go's dependency injection (dependencyinjection) recently, which makes it easier to write code that is easy to test later (so you can be lazy). Now learn the AST parsing code, now take out to share with you:)

Tokenizer and Lexical Anaylizer

If you know what Tokenizer and lexical anaylizer are, please skip to the next chapter, if you are not familiar with this, please see the simplest go code below.

package mainfunc main() {     println("Hello, World!")}

What does this go code do? Very simple, the package is main, defines a main function, the main function called the println function, the parameter is "hello,world!". Well, you know, but when you run Gorun, how does go know? Go first to break your code into an understandable component (token), this process is called tokenize. For example, the first line is split into the package and main. At this stage, go is like a little baby who understands me, wants, eats, and so on, but does not make the right sentences. Because "Eat I want" is not make sense, so the word according to a certain grammar string up the process is lexicalanaylize or parse, simple! Unlike the human brain, code that is understood by the program is usually stored in the form of an abstract syntaxtree (AST) for easy verification and lookup.

Go's AST

Let's take a look at the AST library of go to the understanding of the code is not a small baby bar (can run the source code in this), is actually token+parse we saw just the previous chapter code, and by the way the AST print out, the results here

package mainimport (    "go/ast"    "go/parser"    "go/token")func main() {    // 这就是上一章的代码.    src := `package mainfunc main() {    println("Hello, World!")}`    // Create the AST by parsing src.    fset := token.NewFileSet() // positions are relative to fset    f, err := parser.ParseFile(fset, "", src, 0)    if err != nil {        panic(err)    }    // Print the AST.    ast.Print(fset, f)}

In order not to scare you, I'll just print the first 6 lines:

     0  *ast.File {     1  .  Package: 2:1     2  .  Name: *ast.Ident {     3  .  .  NamePos: 2:9     4  .  .  Name: "main"     5  .  }       // 省略之后的50＋行

As can be seen, go resolves the first (2:1) of the second line of the package, the keyword in the text. "Main" is also parsed out, the 9th character in the second line, but the Go parser also gives it a term: AST. Ident, an identifier, or an ID that you often say, as shown in:

Ident +------------+                   |                         Package +-----+    |              v    v           package main

Let's take a look at what the main function is all about.

     6  .  Decls: []ast.Decl (len = 1) {     7  .  .  0: *ast.FuncDecl {     8  .  .  .  Name: *ast.Ident {     9  .  .  .  .  NamePos: 3:6    10  .  .  .  .  Name: "main"    11  .  .  .  .  Obj: *ast.Object {    12  .  .  .  .  .  Kind: func    13  .  .  .  .  .  Name: "main"    14  .  .  .  .  .  Decl: *(obj @ 7)

Here the Func main is parsed into an AST. Funcdecl (Functiondeclaration), and the function's parameter (Params) and function body (body) are naturally also in this funcdecl. The params corresponds to the *ast. FieldList, as the name implies, is the item list, and the function body consisting of curly braces "{}" corresponds to the AST. Blockstmt (blockstatement). If you are not sure, you can refer to the following figure:

                 FuncDecl.Params +----------+                                            |                   FuncDecl.Name +--------+ |                                          v v          +----------------------> func main() {          |                    +->FuncDecl ++    FuncDecl.Body +-+       println("Hello, World!")          |                    +->          +----------------------> }

For the function body of the main function, we can see that the println function is called, which corresponds to EXPRSTMT (expressstatement) in the AST, and the expression of the calling function corresponds to CALLEXPR (callexpression). The parameters of the call must not be missed, because the argument is only a string, so go will classify it as an AST. Basiclis (a literal of basictype). As shown in the following:

+-----+ ExprStmt +---------------+|                                ||    CallExpr   BasicLit         ||        +          +            ||        v          v            |+---> println("Hello, World!")<--+

What else?

    50  .  Scope: *ast.Scope {    51  .  .  Objects: map[string]*ast.Object (len = 1) {    52  .  .  .  "main": *(obj @ 11)    53  .  .  }    54  .  }    55  .  Unresolved: []*ast.Ident (len = 1) {    56  .  .  0: *(obj @ 29)    57  .  }    58  }

We can see that the AST also resolves the scope of the function and the object that corresponds to the scope.

Summary

Go abstracts all identifiable tokens into node, organized by interface, and the relationships between them are as indicative: