Behind the browser (I) -- lexical parsing of the HTML language

Source: Internet
Author: User

Thanks to the encouragement from Lao Zhuang (@ Zhuang table Wei), the mouse uncle (@ left ear mouse), and The yanda (@ ), I had the courage to start this series.

Also, I would like to thank @ Yubin xiaofeiyu's question. This is the only serious reply I have received. I will try to answer your question in the layout section ......

From now on, we will assume the browser.

Basic knowledge

For us, HTML is actually a string.

Well, considering that we can't wait until the download is complete and start parsing, we actually have to deal with the "ghost stream ".

To parse a streaming into a correct Dom structure, we need to do two steps:

    • Lexical Analysis: parses the token stream into a word that we can understand. The student name is token.
    • Syntax analysis: end-to-end tag pairing, attribute assignment, parent-child relationship connection, and DOM tree
Lexical: State Machine

The HTML structure is not too complex. We need 90% of the tokens only for the tag start, attribute, tag end, comment, and CDATA nodes.

In fact, the trouble is that we need to handle a lot of fault tolerance because HTML is closely related to SGML. <? And <% or something must also be supported.

Now let's see what these tokens look like:

    • <ABC
    • A = "XXX"
    • </Xxx>
    • />
    • <XXX>
    • Hello world!
    • <! -- XXX -->
    • <! [CDATA [Hello world!]>

According to this analysis, we now start to read characters from the pipeline stream. Well, if it is <, we will all know that this is not a text node!

Then read another character, for example, X. Then we will know that it is not a comment or CDATA. Then we will continue to read it until we encounter> or a space, A complete token is obtained.

In fact, every time we read one character, we have to make a decision, and these decisions are related to the "current status. This is a typical state machine scenario.

In the following section, you can find the state transition diagram of the state machine.

NextCodeThe best way to implement a state machine in C/C ++ and JS is similar: each function is regarded as a state, the parameter is a accepted character, and the return value is the next state function.

(Here I want to emphasize again that a state machine is really something that cannot be encapsulated and never tries to encapsulate it .)

The data status on the figure is like this:

 
VaR DATA = function (c) {If (C = "&") {return characterreferenceindata;} If (C = "<") {return tagopen ;} else if (C = "\ 0") {error (); emittoken (c); return data;} else if (C = EOF) {emittoken (EOF ); return data;} else {emittoken (c); return data ;}};

The Lexical analyzer accepts characters in a simple way, as shown below:

Function htmllexicalparser () {// State functions ...... Function data (){//...... } Function tagopen (){//...... }//...... VaR state = data; this. receiveinput = function (char) {state = State (char );}}

Next we will intuitively feel it (you can open the console to view the output ):

<HTML maaa = A> <br/> <pead> <br/> <title> cool </title> <br/> </pead> <br/> <body> </body> <br/> </ptml> </P> <p>Parse

The slightly clean code can be seen in this gist.

These codes only want to demonstrate the parsing principles of HTML and omit most of the HTML States. If you want to fully implement HTML lexical, the W3C specification has clearly defined the entire state machine for you.

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.