[WebKit Core] JavaScriptCore Depth Analysis--Basic (a) byte code generation and construction of syntax tree

See Horkeychen wrote the article "[WebKit] javascriptcore Analysis--Basic article (c) from script code to JIT compiled code implementation", written very well, deeply inspired. I would like to add some details such as how bytecode is generated, and so on, written by Horkey.

JSC's handling of JavaScript, in fact, is similar to WebKit's handling of CSS in many places, and it has so few parts:

(1) Lexical analysis--out of the word (Token);

(2) Grammar analysis, abstract syntax tree (ast:abstract Syntax trees);

(3) Traversal of the abstract syntax tree-generated bytecode (bytecode);

(4) Execute byte code with interpreter (Llint:low level interpreter);

(5) If the performance is not good enough to use baseline JIT compiled bytecode generation machine code, and then execute this machine code;

(6) If performance is not good enough, use DFG JIT recompile bytecode to generate better machine code, and then execute this machine code;

(7) Finally, if it is not good, the Llvm:low-virtual machine to compile the middle of the DFG code, to generate higher-optimized machines and execution. Next, I'll use a series of articles to describe this process.

Among them, steps 1, 2 is similar, 3, 4, 5 steps of the idea, CSS JIT is also a similar approach, please refer to [1]. Want to write JSC article, with rookie and Yugong Yishan Way, open the JSC tip of the iceberg.

This article mainly describes the details of lexical and syntactic parsing.

First, javascriptcore analysis of the lexical analyzer workflow

This explains the lexical and grammatical workflow:

The working process of the tokenizer is to constantly look for a word (Token) from a string, such as to find a continuous "true" string, creating a tokentrue. The process of working with the lexical device is as follows:

Javascriptcore/interpreter/interpreter.cpp:
Template <typename chartype>

[CPP] view plaincopy

2. Template <parsermode mode> tokentype Literalparser<chartype>::lexer::lex (literalparsertoken< chartype>& token)

4. {

6. while (M_ptr < m_end && Isjsonwhitespace (*m_ptr))

8. ++m_ptr;

12. if (m_ptr >= m_end) {

14. Token.type = Tokend;

16. Token.start = Token.end = m_ptr;

18. return tokend;

20. }

22. Token.type = Tokerror;

24. Token.start = m_ptr;

26. Switch (*m_ptr) {

28. Case ' [':

30. Token.type = Toklbracket;

32. Token.end = ++m_ptr;

34. return toklbracket;

36. Case '] ':

38. Token.type = Tokrbracket;

40. Token.end = ++m_ptr;

42. return tokrbracket;

44. Case ' (':

46. Token.type = Toklparen;

48. Token.end = ++m_ptr;

50. return toklparen;

52. Case ') ':

54. Token.type = Tokrparen;

56. Token.end = ++m_ptr;

58. return tokrparen;

60. Case ', ':

62. Token.type = Tokcomma;

64. Token.end = ++m_ptr;

66. return tokcomma;

68. Case ': ':

70. Token.type = Tokcolon;

72. Token.end = ++m_ptr;

74. return Tokcolon;

76. Case ' "':

78. Return Lexstring<mode, ' "> (token);

80. Case ' t ':

82. if (m_end-m_ptr >= 4 && m_ptr[1] = = ' R ' && m_ptr[2] = = ' u ' && m_ptr[3] = = ' E ') {

84. M_ptr + = 4;

86. Token.type = Toktrue;

88. Token.end = m_ptr;

90. return toktrue;

92. }

94. Break

96. Case '-':

98. Case ' 0 ':

[CPP] view plaincopy

2. ...

4. Case ' 9 ':

6. return Lexnumber (token);

8. }

10. if (M_ptr < m_end) {

12. if (*m_ptr = = '. ') {

14. Token.type = Tokdot;

16. Token.end = ++m_ptr;

18. return Tokdot;

20. }

22. if (*m_ptr = = ' = ') {

24. Token.type = tokassign;

26. Token.end = ++m_ptr;

28. return tokassign;

30. }

32. if (*m_ptr = = '; ') {

34. Token.type = Toksemi;

36. Token.end = ++m_ptr;

38. return tokassign;

40. }

42. if (Isasciialpha (*m_ptr) | | *m_ptr = = ' _ ' | | *m_ptr = = ' $ ')

44. return Lexidentifier (token);

46. if (*m_ptr = = ' \ ') {

48. return lexstring<mode, ' \ ' > (token);

50. }

52. }

54. M_lexerrormessage = String::format ("Unrecognized token '%c '", *m_ptr). Impl ();

56. return tokerror;

58. }

Through this process, a complete JSC World token is generated. Then, parse the syntax to generate an abstract syntax tree.

Javascriptcore/parser/parser.cpp:

[CPP] view plaincopy

1. <span style= "font-family:arial, Helvetica, Sans-serif;" >PassRefPtr<ParsedNode> parser<lexertype>::p arse (jsglobalobject* lexicalglobalobject, debugger* debugger, execstate* debuggerexecstate, jsobject** exception) </span>

[CPP] view plaincopy

2. {

4. ASSERT (Lexicalglobalobject);

6. ASSERT (Exception &&!*exception);

8. int errline;

10. Ustring errmsg;

14. if (parsednode::scopeisfunction)

16. M_lexer->setisreparsing ();

20. m_sourceelements = 0;

24. Errline =-1;

26. ErrMsg = Ustring ();

30. Ustring parseerror = Parseinner ();

32. 。。。

34. ｝

Ustring parser<lexertype>::p Arseinner ()

[CPP] view plaincopy

1. {

2. Ustring parseerror = ustring ();

4. unsigned oldfunctioncachesize = M_functioncache? M_functioncache->bytesize (): 0;

[CPP] view plaincopy

1. Abstract Syntax Tree Builder:

2. Astbuilder context (const_cast<jsglobaldata*> (M_globaldata), const_cast<sourcecode*> (M_source));

3. if (m_lexer->isreparsing ())

4. m_statementdepth--;

5. Scoperef scope = CurrentScope ();

[CPP] view plaincopy

1. Start parsing a node of the build syntax tree:

2. sourceelements* sourceelements = parsesourceelements<checkforstrictmode> (context);

3. if (!sourceelements | |!consume (EOFTOK))

｝
For example, according to the token type, JSC that the input token is a constant declaration, the syntax node is generated using the following template function, and then placed inside the Astbuilder:

[CPP] view plaincopy

2. Javascriptcore/bytecompiler/nodecodegen.cpp:

4. Template <typename lexertype>

6. Template <class treebuilder> treeconstdecllist parser<lexertype>::p arseconstdeclarationlist ( treebuilder& context)

8. {

10. Failiftrue (Strictmode ());

12. Treeconstdecllist constdecls = 0;

14. Treeconstdecllist tail = 0;

16. do {

18. Next ();

20. Matchorfail (IDENT);

22. Const identifier* name = M_token.m_data.ident;

24. Next ();

26. BOOL Hasinitializer = match (EQUAL);

28. Declarevariable (name);

30. Context.addvar (name, Declarationstacks::isconstant | (Hasinitializer?) declarationstacks::hasinitializer:0));

32. Treeexpression initializer = 0;

34. if (Hasinitializer) {

36. Next (treebuilder::D ontbuildstrings); Consume ' = '

38. initializer = parseassignmentexpression (context);

40. }

42. Tail = context.appendconstdecl (M_lexer->lastlinenumber (), tail, name, initializer);

44. if (!CONSTDECLS)

46. Constdecls = tail;

48. } while (Match (COMMA));

50. return constdecls;

52. }

Next, the bytecodegenerator::generate is called to generate bytecode, which is divided into section analysis. Let's take a look at the following syntax tree nodes from JavaScript to generate bytecode:

Javascriptcore/bytecompiler/nodecodegen.cpp:
registerid* Booleannode::emitbytecode (bytecodegenerator& generator, registerid* DST)

[CPP] view plaincopy

1. {

2. if (DST = = Generator.ignoredresult ())

3. return 0;

4. Return Generator.emitload (DST, m_value);

5. }

Here are the articles I am going to write:

First, JavaScriptCore's lexical analyzer workflow analysis;

Second, javascriptcore analysis of the parser work flow;

Three, JavaScriptCore of the bytecode generation process analysis;

Iv. Llint interpreter work flow analysis;

Five, the Baseline JIT compiler's work flow analysis;

Six, DFG JIT compiler workflow analysis;

Seven, LLVM virtual machine workflow analysis;

Viii. future prospects of javascriptcore;

Rough writing, poor expression, hope to write better.

The

first time to get blog update reminders, as well as more technical information sharing, welcome to the personal public platform: Programmer Interaction Alliance (coder_online), sweep the QR code below or search number Coder_online can pay attention to, we can communicate online.

Reference:

1 https://www.webkit.org/blog/3271/webkit-css-selector-jit-compiler/

2 http://blog.csdn.net/horkychen/article/details/8928578

Reprinted from: http://my.oschina.net/coderonline/blog/392971

[WebKit Core] JavaScriptCore Depth Analysis--Basic (a) byte code generation and construction of syntax tree