Use BOOST.SPIRIT.X3 's rule and action for complex grammar-guided processes

Preface

The basic use method of boost.spirit.x3 is briefly described in the previous article. In four simple examples, we showed how to use the X3 organization to construct a grammar-generating formula that matches the source string and generates a synthetic attribute. In these simple examples, the complex syntax units, which are non-terminators, are created by combining the basic syntax units in the X3 library. But the syntax units in these examples do not have enough, they can only match the Phrase_parse function to tell us whether the source matches, and a simple assignment operation to return a comprehensive property. If I want to complete some user-defined action when the match is successful, how do I complete this requirement? In addition, when you implement a more complex DSL using only the combination of basic grammar units, the resulting formula becomes very complex. These problems are the problems that the humble text will solve.

Semantic Action

Semantic action, while translating idiom meaning function, is a unary Parser provided by X3. It can wrap a grammar unit and a generic functor. This generic functor is called when the unit of the Action wrapper syntax succeeds. Let's start with an example.

1#include <boost/config/warning_disable.hpp>2#include <boost/spirit/home/x3.hpp>3 4#include <string>5#include <iostream>6 7 namespacex3 =boost::spirit::x3;8 9AutoConstprint = [] (auto& ctx) {std::cout << _attr (CTX) <<Std::endl;};Ten  One intMainvoid) A { -STD::stringSource ="_123123_foo"; -Auto ITR =Source.cbegin (); theAuto End =source.cend (); -  -STD::stringattr; -Auto R =Phrase_parse (ITR, End +, (*x3::char_) [Print] - , x3::ascii::space); +  A     return 0; at}

X3::p Arser class template overloads the operator index. When a parser object is an argument with a generic functor object, calling this operator Index,parser returns an object. The type of this object is the type that x3::action is instantiated based on the type of parser and F. The generic functor is called when the x3::action inside the parser match succeeds in discovering the package.

Generic functor (fabricated) that satisfies a condition. Write the condition as a normal functor as follows.

struct action_func{    <typename context>    voidoperatorconst    {         << _attr (CTX) << Std::endl;    }};

A generic functor must have a operator call with a template parameter. X3 is not concerned with the return value of this operator call, so it is optional. The formal parameter type is a type that relies on template parameters and can have a const qualification, but loses write access to the CTX object. The generic lambda in the C++14 standard is exactly what it needs to be, so it's handy to use lambda. Boost is used in Boost::spirit::qi::p Heonix Library to implement the semantic Action, but the limitations are large, the use of user-defined functor also requires some additional code to adapt, X3 design is much more reasonable.

We discuss this template parameter in detail in context. The type of the context is determined by the comprehensive properties of the parser. It is essentially a tuple. The _attr () function, which we use in operator call, can be likened to the std::get<i> () function to get the data passed in the context. _attr () is a comprehensive property of the wrapped sub-parser. There are three of these functions, _val () Gets the general properties of Rule, _where () Gets the current iterator position of the source matching string, and _pass () gets the matching result. _val () and _attr () details, the humble text will unfold when the rule is introduced. Here's a simple example to show how _pass () is used.

//#include ...AutoConstfoo = [] (auto&CTX) {     if(_attr (CTX) > -|| _attr (CTX) <0) _pass (CTX)=false;};intMainvoid) {std::stringSource =" the"; Auto ITR=Source.cbegin (); Auto End=Source.cend (); intresult; Auto R=Phrase_parse (ITR, End, X3::int_[foo], x3::ascii::space, result); return 0;}

When attribute, here is X3::int_ 's comprehensive attribute, greater than 100 or less than 0 when the match is set to fail.

Rule

X3::rule can manage the basic syntactic units that are grouped together and become a more complex parser. Often in practice, our own definition of DSL syntax can be very complex, using rule to manage the basic parsers, is to use a bottom-up approach to build our grammar production, so that the level is more clear. Below, the example is still in C + + identifier.

1#include <boost/config/warning_disable.hpp>2#include <boost/spirit/home/x3.hpp>3 4#include <string>5#include <iostream>6 7 namespacex3 =boost::spirit::x3;8 9 namespaceClientTen { One     usingX3::lexeme; A     usingX3::raw; -     usingX3::char_; -     usingX3::ascii::alnum; the     usingX3::ascii::alpha; -  -x3::rule<classidentifier, std::string>Constidentifier ="identifier"; -AutoConstIdentifier_def = raw[lexeme[(Alpha |'_') >> * (Alnum |'_')]]; +  -Template <typename Iterator, TypeName Context, TypeName attribute> +InlineBOOLParse_rule ( Ax3::rule<classidentifier, std::string>rule, atiterator& First, IteratorConst&Last , -ContextConst& CTX, attribute&attr) -     { -         usingboost::spirit::x3::unused; -         StaticAuto Def = (identifier =identifier_def); -         returnDef.parse (First, last, CTX, unused, attr); in     } - } to  +  - intMainvoid) the { *STD::stringSource ="_identifier"; $Auto ITR =Source.cbegin ();Panax NotoginsengAuto End =source.cend (); -  theSTD::stringstr; +Auto R =Phrase_parse (ITR, End, Client::identifier, X3::ascii::space, str); A  the     return 0; +}

X3::rule in general, to define two template parameters, the first template parameter is an ID, just as an ID parameter, can be just a forward declaration; The second parameter is the general attribute of this rule. The 17th line of code initializes the rule with a string, giving the rule a name, which is optional, and the rule-saved string can play a role in debugging. Colleague, we can see that this rule uses the Const modifier, because our syntax is constant. Defining a Rule object is to declare a syntax, and then we define it, and the 18th line of code is the way to define it. So how are identifier and identifier_def bound together? Below we x3 the source code to explore, a little bit.

X3::p Hrase_parser as the entire parsed entry, this function goes into x3::p hrase_parse_main this function.

// Line 94 in <boost/spirit/home/x3/cord/parse.hpp>  <typename >bool  phrase_parse_main (...) {      /**/      bool r = as_parser (p). Parse (first, last, Skipper_ctx, unused, attr);       /* */}

Phrase_parse_main will invoke the parse method of parser starting with the root node of the static syntax generation constructed from expression Tetmplate. In fact, rule is derived from parser using the CRTP pattern, and rule is a parser,pharse_parse_main that goes into the parse method of rule.

//Line @ <boost/spirit/home/x3/nonterminal/rule.hpp>Template <typename ID, TypeName Attribute,BOOLForce_attribute_>structRule:parser<rule<id, attribute>>{      /*lines of code*/Template<...>rule_defintion<...>operator=(RHS ...); Template<typename Iterator, TypeName Context, TypeName attribute_>BOOLParse (iterator& first, IteratorConst&Last , ContextConst& Context, Unused_type, attribute_& attr)Const      {          returnParse_rule (* This, first, last, context, attr); }      /*we don ' t care now!*/}

The parse member template method of rule invokes an important function, the Parse_rule function. This function could have used a generic template function to handle all common situations, but X3 did not do so. X3 is designed here to make it possible for the user to implement a parse_rule overloaded function. Note that Parse_rule is a non-qualified name (Unqualified name), so the overloaded method of Parse_rule can define the user's namespace under the mechanism of ADL lookup, and rule is actually defined under the user's namespace. From here you can see the idea of X3 good design.

Here's a question, why not include the Rule object in the rule-managed syntax unit? By reading the source code of rule, it can be found that its members have only one string member variable. If you are familiar with the design principles of expression template, you can know that identifier_def is a complex type of information. If we include this identifier_def in rule, we have only two options. Add a template parameter <typename Rhs> Use the CRTP method to inherit again; or use an abstract type to give the assignment to the run time. The first method would make it very inconvenient for the user to define rule, and the second method clearly violates the original purpose of the X3 design. But X3 here came a Jinchantuoqiao! And look at line 22nd of the sample code. X3 uses rule_defintion to really manage complex expression Template, which exists as a static object in the scope of the overloaded parse_rule under the user namespace. ADL finds mates with overload resolution, parses the code of rule into the user-defined Parse_rule function, and continues recursively down by Rule_defintion's Parse Method! X3 The design method here, very amazing!!

For the most part, the way we overload the Parse_rule is only a few lines of code, and it's duplicated. X3 In order to reduce the workload of overloading functions, a macro boost_spirit_define is defined to accomplish this task. However, there is a convention that the Rule_definition object name can only be the object name of rule and a suffix of "_def". Boost_spirit_define is also a variable-parameter macro that allows us to define all the work we need with just one macro.

Problems in MSVC

I just learned from the group's partners in the morning that vs2015 Update 1 has partially supported expression Sfinae features, x3 code can be used without modification. At this point I am downloading the installation Update 1, and later verify that the test results will be given in a later article.

There are some flaws in the previous article. X3 in the vs2015 can not be used in the implementation of the modified Sfinae a bit of a problem, and do not correctly traits out whether the ID has On_error and on_success function, the correct method of modification please refer to this blog post. I wrote Sfinae why did not take effect, niche also need to study, but also please know the great God of the generous enlighten.

Another mention Boost_spirit_define macro, in the previous article, we have to modify this macro. The official example is to allow the use of the following methods.

1 boost_spirit_define (identifier = identifier_def)2//  or3 4       ' _ ' ' _ ')])

Neither of these methods can be compiled in vs2015, but using the methods in this example temporarily satisfies the requirements. I will also verify in the Update1, Decltype bug is not fixed.

Ending

X3 provides the semantic action to handle complex parsing behavior for the user, as well as a rule to manage complex syntactic units for the user. But x3 not give the complex problem to the user, but use the subtle design to avoid the problems caused by static type. Next, the niche talks about spirit performance issues, how to use the spirit of the attention of the high-tech, will be detailed x3 of the black technology, custom parser to extend the functionality of X3.

Use BOOST.SPIRIT.X3 's rule and action for complex grammar-guided processes