Configurable syntax analyzer development discipline (6) -- construct a real usable state machine (bottom)

In the previous article, a state machine is created for most of the grammar. This time, we mainly talk about right recursion. Right recursion is not as troublesome as left recursion, because most right recursion statements do not make the syntax tree difficult to operate, however, there are still a few cases where we still want to retain the recursive syntax tree shape, such as C ++ connections, so here we will talk about this problem.

How is right recursion formed? Here we don't want this problem. Let's look at a common grammar. As we have already said in the previous article, if a grammar has a non-Terminator that references another syntax, shift and reduce should be done to insert the state machine from this state machine to that state machine:

 

Here, we need to explain that the Green Arrow is shift, the purple arrow is reduce, and they are all ε edges. Furthermore, if a just ends with B, the last input of A is not a Terminator. However, because it is not a right recursion, it seems that there is no problem now:

We are close to the right recursive shape. A fundamental feature of right recursion is of course recursion (nonsense ). To create a right recursion, we can think about what if A and B are not two rule but the same rule? Of course, it seems that a can access himself:

In fact, this already forms a loop of ε edges. The left recursion is the shift loop, and the right recursion is the reduce loop. They are all the same. Then you may wonder why left recursion is so easy to process since left recursion and right recursion are the opposite. Right recursion does not seem to have any way? In fact, if you only want to check whether a string is an element of a grammar without creating a syntax tree, you can compress the ε reduce edge of this loop into one. Why? As mentioned earlier, we can determine whether a reduce task is caused by left recursion, or whether a shift task is caused by right recursion. As long as shift enters the stack without pressing the state, the right recursive reduce loop, no matter how many times, is actually a pop state, so the problem is gone. Similarly, if you do not process the syntax tree, you can use the same method for left recursion.

However, when you create a syntax tree, you add semantic actions to each edge. At this time, shift and reduce are not simply offset by each other, so you cannot compress the ε reduce side of a loop into one. What should you do?

The method is actually very simple, as long as we walk in the state machine and find there is no way to go, see if there is a right recursive reduce can give us a "try. Why? We still remember that when we compress the entire state to no ε edge, every input needs to match the stack. It is gratifying that no side can generate the same matching structure as the reduce side of the right recursion (but I don't want to prove it here), so this is safe.

At this point, we have made all the situations that do not contain the lookahead state machine clear. If a syntax needs to construct lookahead, it is equivalent to adding a token requirement for the future in the edge matching rule, without changing the structure of syntax analysis in essence. However, we know that there are two kinds of context-independent grammar that are not included here, and the C language occupies all of them. Here are two simple examples:

Variable Declaration: For a structure that has already passed typedef, we can write suchCode: A * B ;. In this case, if A is of type, then the rule of variabledeclarationstatement is required. If expression A is an expression, this requires the rule of expressionstatement. However, for syntax analysis, A is a simple token (except for the typedef type, all types of C language start with a keyword, so if you want to make a simple C-language parser, remove typedef. Ah, hahahahaha), you cannot make predictions during syntax analysis.

There are two methods in this case. The first is to prepare a richer semantic actions so that the symbol table can be constructed during parse. So here, we can earn different branches based on whether a is a type. The other is that we keep an ambiguousstatement syntax tree node and write all the unhandled ambiguities encountered by a subtree In the syntax tree. We may recall, why don't we simply return multiple analysis results with one parser? If you don't do this, there will be 10 such variable declarations in a function, and you will have 1024 results. If we contract the ambiguity to a sub-tree, it is still a result, but there are 10 more sub-trees, the effect is completely different.

Forced type conversion: When writing a C language, it is impossible to disable forced type conversion, but when parser sees code similar to this: (A *****) B, because the type structure and expression structure are different, but you cannot do lookahead when you see "(" -- because this lookahead is infinitely long, the expressions or types in the brackets can be infinitely long. But even if you want to limit it to a limited length, even if you give 100 tokens, there will be thousands of lookahead models, so we should not use lookahead here.

What should we do? We only need to regard this state machine as NDA (because it is now NDA), from deterministic push-down Automaton to non-deterministic push-down Automaton, we can only turn our parser into non-deterministic. Wait until the next article -- the last article in this seriesArticle-- To explain in detail.