Go language Introduction "five": Source Learning-bufio

This is a creation in Article, where the information may have evolved or changed.

Introduced

Package Bufio is also part of Io, but in different packages, so the section is separate.
Which consists bufio.go of scan.go two parts.

Bufio

Bufio's role is to provide a buffer for an existing reader or writer, we know that the operating system IO is a resource bottleneck, should be as few as possible to invoke IO operations, so the large amount of data read or write together is a better choice. How to use:

    w := bufio.NewWriter(os.Stdout)    fmt.Fprint(w, "Hello, ")    fmt.Fprint(w, "world!")    w.Flush() // Don't forget to flush!    // Output: Hello, world!

The source code for reader and writer made a simple package, Bufio. Reader as an example:

// Reader implements buffering for an io.Reader object.type Reader struct {    buf          []byte    rd           io.Reader // reader provided by the client    r, w         int       // buf read and write positions    err          error    lastByte     int    lastRuneSize int}

In addition to including the original reader, there is also a []byte structure, which takes the Read method as an example:

Read reads data into p.//It returns the number of bytes read into p.//The bytes is taken from at most one read on th e underlying reader,//hence n May is less than Len (p).//At EOF, the count would be zero and err would be IO.    Eof.func (b *reader) Read (P []byte) (n int, err error) {n = len (p) if n = = 0 {return 0, B.readerr ()}            if B.R = = B.W {if b.err! = Nil {return 0, B.readerr ()} If Len (P) >= len (b.buf) {            Large read, empty buffer.            Read directly into p to avoid copy.  N, b.err = B.rd.read (p) if n < 0 {panic (errnegativeread)} if n > 0 {b.lastbyte = Int (p[n-1]) b.lastrunesize =-1} return N, B.readerr ( )} B.fill ()//buffer is empty if B.R = = B.W {return 0, B.readerr ()}}// Copy as much as we can n = Copy (P, B.BUF[B.R:B.W]) B.R+ = n b.lastbyte = Int (b.buf[b.r-1]) b.lastrunesize = 1 return n, nil} 

explanation : Each read invokes only one operation of the internal reader,
If the internal buf is less than the provided p, then read directly into p without passing buf.
If the BUF is larger, do a fill operation: 1. Clean up legacy data in buf to BUF head, 2. Read internal reader to buf and move backwards w,w+=n
Finally, a copy operation was made to copy the contents of the BUF to P.

Then take Writer.write as an example:

 //Write writes the contents of P into the buffer.//It returns the number of bytes written.//If nn < len (p    ), it also returns an error explaining//what the Write is Short.func (b *writer) write (P []byte) (nn int, err error) {  For Len (P) > b.available () && B.err = nil {var n int if b.buffered () = = 0 {//Large            Write, empty buffer.            Write directly from p to avoid copy.        N, b.err = B.wr.write (p)} else {n = copy (B.BUF[B.N:], p) B.N + = n B.flush ()    } nn + = n p = p[n:]} if b.err! = nil {return nn, b.err} N: = Copy (B.BUF[B.N:], p) B.N + = n nn + n return nn, nil}  

explanation :
n refers to how many bytes have been written in the BUF
B.available () refers to the number of bytes left in buf that are writable, equal to Len (BUF)-N
B.buffered () is n
To see the process:
First, if n=0, then write p directly to the internal writer
If there is something in the buf, copy the contents of P to buf first, and then flush (that is, Buf writes to writer).
As long as there is not enough space in the BUF (less than Len (p)), the writer will be written continuously.
Finally a little tail, can only temporarily stay in the BUF, waiting for the next flush operation.

Look at the Bufio in the scene you're using. Writer's use:
Small buf, a lot of writing data: This is similar to not adding this buf, only a little bit of tail.
Big buf, small data write: So it is possible not to write, just put the data in the BUF first.

Scanner

The role of scanner is to iterate over a reader using the following methods:

scanner := bufio.NewScanner(reader)    for scanner.Scan() {        fmt.Println(scanner.Text()) // Println will add back the final '\n'    }

The default is read by one line, scan,scanner each time. Text () returns the data for the next row until Eof,scan () returns false.
Let's look at the source code, mainly the scanner structure:

type Scanner struct {    r            io.Reader // The reader provided by the client.    split        SplitFunc // The function to split the tokens.    maxTokenSize int       // Maximum size of a token; modified by tests.    token        []byte    // Last token returned by split.    buf          []byte    // Buffer used as argument to split.    start        int       // First non-processed byte in buf.    end          int       // End of data in buf.    err          error     // Sticky error.    empties      int       // Count of successive empty tokens.    scanCalled   bool      // Scan has been called; buffer is in use.    done         bool      // Scan has finished.}

Each return "one line" (in fact not necessarily a line, so called) is called token, moving to the next token is called a advance, through the split function to do tokenize. Others are some of the more obvious auxiliary fields.

Here is the main split function, the default bufio.NewScanner() code is as follows:

func NewScanner(r io.Reader) *Scanner {    return &Scanner{        r:            r,        split:        ScanLines,        maxTokenSize: MaxScanTokenSize,    }}

Take the branch function as split and see maxscantokensize = 64 * 1024, which means that one line cannot be too long. Otherwise, the error will be thrown unless scanner is used. The buffer () method provides its own buffers and maximum capacity.

In addition to the default scanlines, the system also provides, ScanRunes ScanWords ScanBytes Three split functions, the user can also customize the Split function.