Reading dragon Book compiling principle Lexical Analysis (2) ...

Source: Internet
Author: User

The second form of lexical analysis is of course the automatic generator of the lexical analyzer. Like Lex, Jlex and so on. So how do you get the generator to know what tokens we want to generate?

This involves the question of a unified declarative specification, in other words, you will be able to identify the declarative specification to the generator in the form of a generator. So we just need to complete the declarative specification to complete the lexical analysis part, to the final lexical parser automatic generator will be able to generate corresponding automata for us ...

So how do you give the generator a unified declarative specification? One of the first mathematical tools to understand is the regular expression, which is the basic concept of regular expressions ...

Then we can try to use regular expressions to represent identifiers (IDs) in the C language: [a-za-z_][a-za-z0-9_]* ...

With these specifications, we can look at the generated automata (in fact, a piece of code) exactly what it looks like ... Theoretically speaking, automata is actually a mathematical concept ...

M= (ε,s,q0,f,δ)

  • E: Alphabet (accepted letters)
  • S: State set (how many states exist altogether)
  • Q0: The initial state
  • F: End-of-state set (how many acceptable states are there altogether)
  • Δ: Transfer function (each state accepts what character jumps what state)

This is an example of a simple automaton ...

A string that can be accepted must reach the accepted state at the end of the string (two concentric circles, only state 2 is accepted in the figure)

The figure described in fact is DFA, state machine inside there is a kind called NFA

Reading dragon Book compiling principle Lexical Analysis (2) ...

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.