Golang progressive processing of files

This is a creation in Article, where the information may have evolved or changed.

Golang offers the package Bufio. Bufio. Newreader () Creates a default-sized readbuf, which, of course, can also be bufio. Newreadersize.

Func Newreader (rd IO. Reader) *reader Newreader returns a new reader whose buffer has the default size (4096). Func newreadersize (rd IO. Reader, size int) *reader newreadersize returns a new Reader whose buffer has at least the specified size. If the argument io. Reader is already a reader with large enough size, it returns the underlying reader.

Bufio

func  (B *reader)  readbyte ()   (c byte, err error)      Readbyte reads and returns a single byte. if no byte is  available,    returns an error.func  (B *reader)  readbytes ( Delim byte)   (line []byte, err error)     readbytes reads  until the first occurrence of delim in the input,     returning a slice containing the data up to and  Including the delimiter.    if readbytes encounters an error  before finding a delimiter, it returns    the data  read before the error and the error itself  (Often io. EOF).     readbytes returns err != nil if and only if the returned data  does not    end in delim. for simple uses, a  scanner may be more convenient.func  (B *reader)  ReadString (delim byte)   (Line string, err error)     readstring reads until the  first occurrence of delim in the input,    returning  a string containing the data up to and including the     delimiter. If ReadString encounters an error before  Finding a delimiter,    it returns the data read before  the error and the error itself  (Often    io. EOF).  readstring returns err != nil if and only if the returned data     does not end in delim. for simple uses, a scanner  may be more    convenient.

readbyte this interface, and C language fgetc very close, each read a byte. Both Readbytes and ReadString can implement progressive reads as long as the Delim is set to ' \ n '.

package mainimport  "FMT" import  "OS" import  "io" import  "flag" import  "Bufio" var  num_flag = flag. Bool ("n", False, "Num each line") Func usage () {    fmt. Printf ("%s %s\n", OS. Args[0], "filename")}func cat (R *bufio. Reader) {    i := 1    for {         //buf,err := r.readbytes (' \ n ')          buf,err := r.readstring (' \ n ')         if err  == io. eof{            break         }        if *num_flag{             fmt. fprintf (OS. Stdout, "%5d %s",                          i,buf)              i++        }else{             fmt. fprintf (OS. Stdout, "%s", buf)         }    }     return }func main () {    flag. Parse ()     if (flag. Narg ()  == 0) {        cat (Bufio. Newreader (OS. Stdin))     }    for i:=0;i<flag. Narg (); I++{        f,err := os. OpenFile (flag. ARG (i), OS. o_rdonly,0660)         if err != nil{             fmt. fprintf (OS. Stderr, "%s err read&Nbsp;from %s : %s\n ",             os . Args[0],flag. ARG (0), err)             continue         }        cat (Bufio. Newreader (f))         f.close ()     }

Read row by line with Scaner

Func Cat (Scanner *bufio. Scanner) error{for Scanner. Scan () {fmt. PRINTLN (scanner. Text ())//fmt. fprintf (OS. Stdout, "%s\n", scanner. Text ())} return scanner. ERR ()}

Notice why the Scan,text () function is executed to return to the next line? Because the default Split function is scanlines. If you have special needs to split, Func (S *scanner) split (split Splitfunc)

This function can make splitfunc. You can customize your own split function.

It should be noted that scan will remove the split symbol \ n, if the fprintf output, do not add \ n printing, there will be no line wrapping phenomenon, as shown below

Fmt. fprintf (OS. Stdout, "%s", scanner. Text ())

Manu@manu-hacks:~/code/go/self$ go run mycat_v2.go test.txt this is test  file created by goif not existed ,please create this fileif  existed, Please write appendhello world,hello gothis is test  file created by goif not existed ,please create this fileif  existed, please write appendhello world,hello gomanu@manu-hacks:~/code/go/self$  cat test.txt this is test file created by goif not existed  ,please create this fileif existed, Please write appendhello  world,hello gothis is test file created by goif not existed  , Please create this fileif existed, please write appendhello world, Hello go

The code for the invocation section is as follows:

F,err: = os. OpenFile (flag. ARG (i), OS. o_rdonly,0660) ... error: = Cat (Bufio. Newscanner (f)) if err! = nil{FMT. fprintf (OS. Stderr, "%s err read from%s:%s\n", OS. Args[0],flag. ARG (i), error)}