Detailed introduction to similarities between Python CST and AST

Source: Internet
Author: User

Have you ever encountered intermediate results obtained by syntax analysis in computer related operations, such as CST? Do you want to know the actual application of CST? The following is a detailed introduction to Python CST's practical applications and related code. forget to get what you want.

Similar to the AST, Python CST is the intermediate result obtained by syntax analysis. Their difference is that CST directly corresponds to the matching process of syntax analysis, which is directly generated and contains a large amount of redundant information. The AST skips the redundant information in the middle and directly corresponds to the actual semantics, that is, the analysis result. Use examples to clarify the following:

Suppose there is such an expression,

Python CST is like this:-> indicates from parent node to child node)

 
 
  1. file_input -> stmt -> simple_stmt -> small_stmt -> 
    expr_stmt -> testlist -> test ->or_test ->and_test 
    ->not_test -> comparison -> expr -> xor_expr ->
     and_expr -> shift_expr -> arith_expr -> term -> 
  2. factor -> power -> atom -> (NAME, “a”) 

The AST is:

 
 
  1. (stmt_ty, expr_kind) -> (expr_ty, name_kind) ->(“a”) 

We can see that CST expresses the entire process of analyzing a, from file_input to the final NAME. Every step of derivation has become a node of the tree, and most of the information can be said to be useless. The AST structure is much simpler and more straightforward. It directly indicates that expression a is an expression statement and a is assumed to be a separate statement. The content is a identifier and the value is "". Python syntax analysis generates Python CST instead of AST. Later, Python will call PyAst_FromNode to convert CST to AST.

The CST Node is called a node, and its structure is defined in Node. h:

 
 
  1. typedef struct _node {  
  2. short n_type;  
  3. char *n_str;  
  4. int n_lineno;  
  5. int n_col_offset;  
  6. int n_nchildren;  
  7. struct _node *n_child;  
  8. } node;  
  9. Field  
  10. Description  
  11. n_type   

Node type. The Terminator is defined in token. h, not in graminit. h.

N_str

Content of the string corresponding to the node

N_lineno

Corresponding row number

N_col_offset

Column number

N_nchildren

Number of subnodes

N_child

Subnode array for dynamic memory allocation

Python provides the following functions/macros to operate CST, which are also defined in node. h:

 
 
  1. PyAPI_FUNC(node *) PyNode_New(int type);  
  2. PyAPI_FUNC(int) PyNode_AddChild(node *n, int type,  
  3. char *str, int lineno, int col_offset);  
  4. PyAPI_FUNC(void) PyNode_Free(node *n);  
  5. /* Node access functions */  
  6. #define NCH(n) ((n)->n_nchildren)  
  7. #define CHILD(n, i) (&(n)->n_child[i])  
  8. #define RCHILD(n, i) (CHILD(n, NCH(n) + i))  
  9. #define TYPE(n) ((n)->n_type)  
  10. #define STR(n) ((n)->n_str)  
  11. /* Assert that the type of a node is what we expect */  
  12. #define REQ(n, type) assert(TYPE(n) == (type))  
  13. PyAPI_FUNC(void) PyNode_ListTree(node *);   

The above is similar to Python CST and AST, which are the intermediate results obtained by syntax analysis. Their difference is that CST directly corresponds to the matching process of syntax analysis, which is directly generated and contains a large amount of redundant information. The AST skips the redundant information in the middle and directly corresponds to the actual semantics, that is, the analysis result. If you use examples to describe the relevant content, you will be rewarded.

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.