A simple lexical parser for Java implementations

A simple lexical parser

Lexical parsing (Lexical analysis) is the first stage of compilation. The main task of the lexical analyzer is to read the input characters of the source program, make them into morphemes, generate and output a sequence of lexical units, each lexical unit corresponding to a morpheme. This lexical unit sequence is output to the parser for parsing.

Introduction to the principle of reference http://www.cnblogs.com/yanlingyin/archive/2012/04/17/2451717.html

There's not much to say here.

The following direct code, the relevant comments in the code has been given, the processing of the string is entirely in accordance with their own ideas written.

It reads the Test.txt text document under the E disk:

The results of the operation are as follows:

public class Word {
/**
*1~20 number as the keyword, with the subscript, i+1 is its machine code, 21~40 number is the operator, with the subscript, I+21 is its machine code; 41~60 number is the delimiter,
* The subscript indicates that the I+41 is its machine code, the user-defined identifier, the machine code is 51, the machine code of the constant is 52;
* Unrecognized identifier with a machine code of 0
**/
Key words
Private String keyword[]={"int", "Long", "char", "short", "float", "double",
' If ', ' Else ', ' for ', ' while ', ' return ', ' break ', ' Continue ', ' switch ', ' case ',
"Default", "void", "struct", "static", "Do"};
Operator
Private String operator[]={"+", "-", "*", "/", "%", "=", ">", "<", "!",
"= =", "! =", ">=", "<=", "+ +", "--", "&", "&&", "| |", "[", "]"};
Delimiter
Private String delimiter[]={",", ";", "(", ")", "{", "}", "\", "\" ",": "," # "};
is a number
public boolean isdigit (String str) {
Char ch;
if (str== "") {
return false;
}
for (int i=0;i<str.length (); i++) {
Ch=str.charat (i);
if (ch< ' 0 ' | | Ch> ' 9 ') {
return false;
}
}
return true;
}
Whether it is a letter or a string
public boolean isletter (String str) {
Char ch;
if (str== "") {
return false;
}
for (int i=0;i<str.length (); i++) {
Ch=str.charat (i);
if (ch< ' A ' | | Ch> ' Z ' | | (ch> ' Z ' &&ch< ' a ')) {
return false;
}
}
return true;

}
To determine whether an operator
public boolean isoperator (String str) {
Char ch;
Char CH2;
Non-operator
if (str== "" | | Str.length () ==0| | Str.length () >2) {
return false;
}
if (Str.length () ==1) {
A unary operator
for (int i=0;i<operator.length;i++) {
if (Str.equals (Operator[i])) {
return true;
}
}
return false;
}else if (str.length () ==2) {
Two operators
Ch=str.charat (0);
Ch2=str.charat (1);
if (ch2== ' = ' && (ch== '! ' | | ch== ' > ' | | ch== ' < ' | | ch== ' = ')) {
return true;
}else if (ch== ' + ' &&ch2== ' + ') {
return true;
}else if (ch== '-' &&ch2== '-') {
return true;
}else if (ch== ' | ') &&ch2== ' | ') {
return true;
}else if (ch== ' & ' &&ch2== ' & ') {
return true;
}else{
return false;
}
}
return true;

}
Determines whether the identifier (where the identifier contains the keyword, the following to get the machine code to remove the keyword)
public boolean isidentifier (String str) {
if (str== "") {
return false;
}else if (str.length () ==1) {
if (Isletter (str) | | str== "_") {
return true;
}else{
return false;
}
}else{
Char Ch=str.charat (0);
String str2=str.substring (1, Str.length ()-1);
String STR3;
if ((ch> ' a ' &&ch< ' z ') | | (ch> ' A ' &&ch< ' Z ') | | ch== ' _ ') {
for (int i=0;i<str2.length (); i++) {
Str3=str2.substring (i, i+1);
if (! ( IsDigit (STR3) | | Isletter (STR3))) {
return F

Simple lexical parser for Java implementation