Go 1.8RC3 Source Code learning: Scanner

This is a created article in which the information may have evolved or changed.

Objective

The scanner package contains data structures and methods related to the Golang lexical analyzer, and the source code is located in <go-src>/src/go/scanner

Example_test.go

Example_test.go contains an example method that uses the scanner package to perform a lexical scan of the Euler formula

func ExampleScanner_Scan() {    // src is the input that we want to tokenize.    src := []byte("cos(x) + 1i*sin(x) // Euler")    // Initialize the scanner.    var s scanner.Scanner    fset := token.NewFileSet()                      // positions are relative to fset    file := fset.AddFile("", fset.Base(), len(src)) // register input "file"    s.Init(file, src, nil /* no error handler */, scanner.ScanComments)    // Repeated calls to Scan yield the token sequence found in the input.    for {        pos, tok, lit := s.Scan()        if tok == token.EOF {            break        }        fmt.Printf("%s\t%s\t%q\n", fset.Position(pos), tok, lit)    }}

Summarize