Golang with scanner

This is a creation in Article, where the information may have evolved or changed.

        the articles in this blog ,we will introduce the performance for that who consider performance in your design of software.wo introduce implementtion of some common method.the method "make" that usually malloc memory from the memory from go platform. a class called type Scanner struct.

    typeScannerstruct{R IO. Reader//The reader provided by the client.Split Splitfunc//The function to split the tokens.MaxTokenSizeint       //Maximum size of a token; modified by tests.token []byte    //Last token returned by split.BUF []byte    //Buffer used as argument to split.Startint       //First non-processed byte in buf.Endint       //End of data in BUF.Err error//Sticky error.Emptiesint       //Count of successive empty tokens.ScancalledBOOL      //Scan has been called;DoneBOOL      //Scan has finished.}

We can show scanner. Bytes (), just used Buiild in scanner class that's Origin data,not new malloc memory to restore data.

    func (s *Scanner) Bytes() []byte {        return s.token //refernce from class inner member    }    // Text returns the most recent token generated by a call to Scan    // as a newly allocated string holding its bytes.    string {        returnstring(s.token) //useing memory copy from s.token to new string that is new memeory allocation.    }

    usingText () method,if  theData ofSize is too big,this wouldGetBad performance. Next I'll introduce Scan method,implemention ofScan Code.all methodinchGolang, that bring usaConvenient,but background ofImplements so complex. theManagement ofMemory ofOperationsystemTaked over byGolangPlatform, theUser don ' t care ofHow Memory mallocorFree FOMsystem. atThis is not  theSame toC language.

//reference from Bufio/scan.gofunc(S *scanner) Scan ()BOOL{//this method already malloc memory for BUF        ifS.done {return false} s.scancalled =true                                                                 //Loop until we have a token.          for{//See if we can get a token with the what we already has.             //If we ' ve run out of data but has an error, give the Split function            //A chance to recover any remaining, possibly empty token.             ifS.end > S.start | | S.err! =Nil{Advance, token, err: = S.split (S.buf[s.start:s.end], s.err! =Nil)ifErr! =Nil{ifErr = = Errfinaltoken {S.token = token S.done =true                                                                               return true} s.seterr (Err)return false}if!s.advance (advance) {return false} S.token = TokenifToken! =Nil{ifS.err = =Nil|| Advance >0{s.empties =0}Else{//Returning tokens not advancing input at EOF. s.empties++ifS.empties > -{Panic("Bufio. scan:100 empty tokens without progressing ")                                 } }return true}                                                                                   }//We cannot generate a token with the what is holding.             //If we ' ve already hit EOF or an I/O error, we're done.             ifS.err! =Nil{//Shut it down. S.start =0S.end =0                                                                               return false}//must read more data.             //First, shift data to beginning of buffer if there ' s lots of empty space            //or space is needed.             ifS.start >0&& (S.end = =Len(S.BUF) | | S.start >Len(S.BUF)/2) {Copy(S.buf, S.buf[s.start:s.end])//copy []byteS.end-= S.start S.start =0}//is the buffer full?                                     If So, resize.             ifS.end = =Len(S.BUF) {//Guarantee no overflow in the multiplication below.                 ConstMaxInt =int(^UINT(0) >>1)if Len(s.buf) >= S.maxtokensize | |Len(S.BUF) > MaxInt/2{S.seterr (Errtoolong)return false} newSize: =Len(S.BUF) *2                                                               ifNewSize = =0{newSize = Startbufsize }ifNewSize > S.maxtokensize {newSize = S.maxtokensize                                                                                       } Newbuf: = Make([]byte, newSize)//resize buf                Copy(Newbuf, s.buf[s.start:s.end]) S.buf = Newbuf                                                                        S.end-= S.start S.start =0                                                                             Continue}//Finally We can read some input. Make sure we don ' t get stuck with            //A misbehaving Reader. Officially we don ' t need to does this, but let's            //Be extra careful:scanner are for safe, simple jobs.              forLoop: =0; ; {N, err: = S.r.read (s.buf[s.end:Len(S.BUF)]) S.end + = nifErr! =Nil{S.seterr (ERR) Break}ifn >0{s.empties =0                                                                              Break} loop++ifLoop > Maxconsecutiveemptyreads {s.seterr (io. errnoprogress) Break                                                                                 }                                                                                 }                                                                              }                                                                               }