Java Stream Tokenizer, streamtokenizer

Java Stream Tokenizer, streamtokenizer

Note:JAVA is generally used to solve problems using the handler class.InputBut it may time out when the time is strictly required. I encountered this problem when I solved POJ1823, and then switched to the StreamTokenizer class.Input. It seems that the latter handlesInputHigh efficiency.
The following is a summary:

1. Class java. io. StreamTokenizer canObtainInputStream and analyze it as a Token ).
The nextToken method of StreamTokenizer reads the next tag

2. By default, StreamTokenizer considers the following content as a Token: letters, numbers, symbols other than c and c ++ comments.
For example, if the symbol "/" is not a Token, the content after the annotation is not, and "/" is a Token. Single quotation marks, double quotation marks, and the total content can only be regarded as one Token.

3. To improve efficiency,UseBufferedReader, as follows, creates a StreamTokenizer object

StreamTokenizer st = new StreamTokenizer (new BufferedReader (newInputStreamReader (System. in)));


4. In orderObtainYou can call the nextToken () method of StreamTokenizer.
After the nextToken () method is called, if it is marked as a String, String s = st. sval can be used. If it is an integer, int n = (int) st. nval is used.

St. nextToken ();

Int I = (int) st. nval; // The default format parsed by st. navl is double.

St. nextToken ();

Int j = (int) st. nval;

St. nextToken ();

String s = st. sval;

 

Appendix:

Reproduced in: http:// I .cnblogs.com/EditPosts.aspx? Opt = 1

Key points:

• Class java. io. StreamTokenizer can get the input stream and analyze it as a Token (TAG ). The nextToken method of StreamTokenizer reads the next tag
• By default, StreamTokenizer considers the following as tokens: letters, numbers, symbols other than C and C ++ comments. For example, if the symbol "/" is not a Token, the content after the annotation is not, and "\" is a Token. Single quotation marks, double quotation marks, and content can only be regarded as one Token.
• To count the number of characters in a file, you cannot simply count the number of tokens, because the number of characters is not equal to the Token. According to the Token regulations, the content in the quotation marks is a Token even if it is 10 pages. If you want the content in quotation marks and quotation marks to be counted as tokens, you should use the ordinaryCha () method of StreamTokenizer to treat single quotation marks and double quotation marks as common characters.

// Specify the file input path in the program: String fileName = "d:/ceshi.txt ";

Package szu.edu;

Import java. io. BufferedReader;
Import java. io. FileReader;
Import java. io. IOException;
Import java. io. StreamTokenizer;

/**
* Use StreamTokenizer to count the characters in the file
* The StreamTokenizer class obtains the input stream and analyzes it as a "tag". A tag can be read at a time.
* The analysis process is controlled by a table and many flags that can be set to various States.
* The stream's Mark Generator can recognize identifiers, numbers, referenced strings, and various annotation styles.
*
* By default, StreamTokenizer considers the following content as tokens: letters, numbers, symbols other than C and C ++ comments.
* For example, if the symbol "/" is not a Token, the commented content is not, and "\" is a Token. Single quotation marks, double quotation marks, and content can only be regarded as one Token.
* The program used to count the number of characters in an article. It is not easy to count the number of tokens, because the number of characters is not equal to the Token. According to Token rules,
* The content in the quotation marks is a Token even if it is 10 pages. If you want the quotation marks and content in the quotation marks to be counted as tokens, you should call the following code:
* St. ordinaryChar ('\'); // set single quotation marks to common characters
* St. ordinaryChar ('\ "'); // double quotation marks are set to common characters.
*/
Public class StatisFileChars {

/**
* Count characters
* @ Param fileName: File Name
* @ Return Characters
*/
Public static long statis (String fileName ){

FileReader fileReader = null;
Try {
FileReader = new FileReader (fileName );
// Create a tag Generator for analyzing a given upstream stream
StreamTokenizer st = new StreamTokenizer (new BufferedReader (
FileReader ));

// The ordinaryChar method specifies that the character parameter is a "common" character in the Mark Generator.
// The single quotation marks, double quotation marks, and comments below are common characters.
St. ordinaryChar ('\'');
St. ordinaryChar ('\"');
St. ordinaryChar ('/');

String s;
Int numberSum = 0;
Int wordSum = 0;
Int symbolSum = 0;
Int total = 0;
// The nextToken method reads the next Token.
// TT_EOF indicates the constant that has been read to the end of the stream.
While (st. nextToken ()! = StreamTokenizer. TT_EOF ){
// After the nextToken method is called, The ttype field will contain the type of the newly read tag
Switch (st. ttype ){
// TT_EOL indicates the constant that has been read to the end of the row.
Case StreamTokenizer. TT_EOL:
Break;
// TT_NUMBER indicates that a constant marked by a number has been read.
Case StreamTokenizer. TT_NUMBER:
// If the current tag is a number, the nval field will contain the value of this number
S = String. valueOf (st. nval ));
System. out. println (s );
NumberSum + = s. length ();
Break;
// TT_WORD indicates that a constant with a text mark has been read.
Case StreamTokenizer. TT_WORD:
// If the current tag is a text tag, the sval field contains a character string that gives the text tag
S = st. sval;
WordSum + = s. length ();
Break;
Default:
// If none of the above three types is used, it is an English Punctuation Mark.
S = String. valueOf (char) st. ttype );
SymbolSum + = s. length ();
}
}
System. out. println ("sum of number =" + numberSum );
System. out. println ("sum of word =" + wordSum );
System. out. println ("sum of symbol =" + symbolSum );
Total = symbolSum + numberSum + wordSum;
System. out. println ("Total =" + total );
Return total;
} Catch (Exception e ){
E. printStackTrace ();
Return-1;
} Finally {
If (fileReader! = Null ){
Try {
FileReader. close ();
} Catch (IOException e1 ){
}
}
}
}

Public static void main (String [] args ){
String fileName = "d:/ceshi.txt ";
StatisFileChars. statis (fileName );
}
}

 

Note: In addition to the above usage, there is also a common usage. For basic input operations! (For example, in acm programming, the input efficiency is relatively high .)

BufferedReaderProvides quite fast read operations for almost all problems. But this class may be used to read single characters and lines only. To read tokens and numbers you shoshould useStringTokenizerOrStreamTokenizer.

 

import java.io.*;public class Main{    public static void main(String[] args) throws IOException    {        StreamTokenizer in = new StreamTokenizer(new BufferedReader(new InputStreamReader(System.in)));        //PrintWriter out = new PrintWriter(new OutputStreamWriter(System.out));        int a, b;        while(in.nextToken() != StreamTokenizer.TT_EOF)        {            a = (int)in.nval;            in.nextToken();            b = (int)in.nval;            //out.println(a + b);            System.out.println("a + b = "+(a+b));        }        out.flush();    }}