"Open source framework that thing 25": the implementation of the framework template engine of the transformation of the record

a bit of understanding design ideas, tiny template engine optimization record! Join the framework Design interest group: http://bbs.tinygroup.org/group-113-1.html

The implementation of the tiny template engine is the way it was compiled, and recently there have been some problems, so I think it is necessary to adjust the compilation method to explain the way to start the implementation of this activity.

Problems with compiling methods

At that time, the use of compiling method, mainly considering that the compilation mode at run time does not have to go through the syntax tree, so the use of the compilation method. However, in practical applications, the following problems have arisen:

Problems with file path conflicts

Since the use of the compilation method, this time exists in a choice, that is: Java source code to the ground or not to the landing of the choice. If the Java file does not fall, then there is no source code to debug if you want to debug your code if you have a problem (though this scenario is not much). If the Java code falls to the ground, there is a problem with the resource file having a conflict in the disk file.

The same problem exists for the class file, and if it does not, then each time the application restarts, the files will have to be recompiled to produce the class file, and if landed, there will be conflicting issues.

Of course, the tiny template engine solves this conflict by adding a configuration item, but due to the addition of a configuration item, the workload of the maintainer is increased objectively, and the maintenance personnel do not understand the Tao A problem occurs when you forget to set up a conflict between a Java file and a class file generated by a template file in multiple apps when multiple tiny applications are deployed in a single server.

PermSize Memory consumption issues

When compiling, because each template file is to generate a class, each macro will also generate a class, in the macro call process, but also to generate some classes. (These classes can not be generated, but because the tiny template engine supports some very useful features, macro calls are compiled at all times, and some inline classes are generated). In this way, a large number of Java classes will be generated, from a very large project, it will lead to PermSize defeated very large. Especially when the system is still debugging, when the template file changes, it is necessary to recompile the generation of new classes, in order to avoid the need to restart the application server to be effective, so the use of their own writing ClassLoader way to achieve the immediate refresh problem, but because of Java garbage collection mechanism, It is decided that the garbage is not collected in time, but because each class has a classloader to support in order to replace it in time, this will further enlarge the memory footprint.

The problem of loading speed is very long

Because macros are available in the tiny template engine, and these macros can exist independently, all macros must be loaded into memory when the app is launched to find them. Therefore, the first boot time, due to compile all the macro files and load, resulting in very slow startup. At a later time, it is also necessary to detect the template file and the generated class is consistent, whether it has been modified, when a project size is relatively large, this time is also relatively long. Especially during the development period, the start-up time is increased by 10 seconds, which makes the developer feel unbearable.

Problem with access speed

There are some problems with the way of compiling.

In order to improve the application startup time, only the macro files are pre-compiled and loaded at startup, and the template files and layout files do not have this treatment, which leads to the time when the access, the first time you visit, you need to compile the template file as a Java file, and then compile the Java file into a class file, If this visit also uses the layout file, and import other template files, then the tragedy, the first visitor may have to wait a few seconds. At the same time, in order to avoid the multiple compilation situation, but also to increase the synchronization lock, which will further affect the efficiency of access.

Specifically haven't tested ClassLoader too much on the performance of how much impact, but Mao Yan estimate is a certain impact, after all, to increase the number of layers to find. Do more work, the work of slow is also natural, people are so, computer is the same truth.

Benefits of using an explanatory approach

Because of the use of interpretation, so there is no need to generate Java source files and class files, so there is no file path conflict, there is no permsize and many classloader memory-intensive problems.

Because of the use of interpretation, the first load, only qualitative scanning part of the relationship, so the scanning speed is very fast, only in the direct execution of the time, the need for more detailed processing, and because there is no need to compile, do not need to do synchronous processing, so loading speed is much higher than the compilation method, In particular, compared to the first load time of the compilation method.

The speed of access to the problem, my original feeling, I feel that the compiler will be faster, after all, it does not have to go through the cloud syntax tree, but the actual execution down, the sense of interpretation of the way roughly a bit of ascension, I analyzed the reason, It can be considered as follows: 1. Due to the Java optimization strategy, resulting in the use of high-frequency access to the depth of performance optimization, the use of interpretation, because of the use of those functions, so can quickly meet the requirements of the Java Virtual machine, the depth of the optimization earlier; 2. More optimized solutions can be used as compared to the method of compilation Scheme, so the time to traverse the syntax tree is made up by avoiding doing something, so feel the performance rather a little bit higher. In short, this compilation to explain, the effect is obvious, all aspects of the overall let me satisfied, especially in front of the implementation of the efficiency of worry about about 50% of the upgrade is to let me be overjoyed. Another surprise is that by changing the way it is compiled to explain execution, the size of the code shrinks by nearly half, from the original 8000+ line to the 4000+ line. At the same time, because you do not have to rely on JDT,ANTLR also rely on the runtime package, but also reduce the size of the war packet 3M.

OK, say so much, then talk about this transformation process.

Because the team to travel to the island, at that time to the task to a left-behind students to complete, but before and after two weeks, did not submit the results of my satisfaction, due to see no subsequent completion of the time node, there is no way, I have to do the old man himself to complete, OK to start, I believe carefully read the following section of the Will have a deep understanding of the development of the ANTLR interpretation engine, and even take my code divert, directly available.

Explain the engine makeover recordExplain engine general control class

Explain the engine general control class is the core of the interpretation engine, because this thing is to tiny template engine custom-written, so if there are students to take to transform, please divert can. Because the class is not big, I directly paste the source up, so that the parents understand and I explained below.

public class Templateinterpreter {terminalnodeprocessor[] terminalnodeprocessors = new terminalnodeprocessor[200]; Map<class<parserrulecontext&gt, contextprocessor> contextprocessormap = new HashMap<Class<    Parserrulecontext>, contextprocessor> ();      Otherterminalnodeprocessor othernodeprocessor = new Otherterminalnodeprocessor ();  public void Addterminalnodeprocessor (Terminalnodeprocessor processor) {Terminalnodeprocessors[processor.gettype ()]    = processor; } public void Addcontextprocessor (Contextprocessor contextprocessor) {contextprocessormap.put (contextprocessor    . GetType (), contextprocessor); } Public Tinytemplateparser.templatecontext Parsertemplatetree (string sourceName, String templatestring) {Char        [] Source = Templatestring.tochararray ();        Antlrinputstream is = new Antlrinputstream (source, source.length);        Set source file name, it'll be is displayed in error report.    Is.name = SourceName;    Tinytemplateparser parser = new Tinytemplateparser (New Commontokenstream new Tinytemplatelexer (IS));    return Parser.template (); } public void interpret (Templateenginedefault engine, Templatefromcontext templatefromcontext, String templatestring, String SourceName, Templatecontext PageContext, Templatecontext context, writer writer) throws Exception {Interpre        T (engine, Templatefromcontext, Parsertemplatetree (SourceName, templatestring), PageContext, context, writer);    Writer.flush (); } public void interpret (Templateenginedefault engine, Templatefromcontext Templatefromcontext, Tinytemplateparser.tem        Platecontext Templateparsetree, Templatecontext PageContext, Templatecontext context, writer writer) throws Exception { for (int i = 0; i < Templateparsetree.getchildcount (); i++) {Interprettree (engine, Templatefromcontext        , Templateparsetree.getchild (i), PageContext, context, writer); }} public Object Interprettree (TemPlateenginedefault engine, Templatefromcontext templatefromcontext, Parsetree tree, Templatecontext PageContext,        Templatecontext context, writer writer) throws Exception {Object returnvalue = null;            if (tree instanceof terminalnode) {Terminalnode Terminalnode = (terminalnode) tree;            Terminalnodeprocessor processor = Terminalnodeprocessors[terminalnode.getsymbol (). GetType ()];            if (processor! = null) {returnvalue = Processor.process (Terminalnode, context, writer);            } else {returnvalue = othernodeprocessor.process (Terminalnode, context, writer); }} else if (tree instanceof Parserrulecontext) {Contextprocessor processor = Contextprocessormap.get (t            Ree.getclass ()); if (processor! = null) {returnvalue = Processor.process (This, Templatefromcontext, (Parserrulecontext) Tre            E, PageContext, context, engine, writer); } if (processor= = NULL | | Processor! = null && Processor.processchildren ()) {for (int i = 0; i < Tree.getchildcount (); i+ +) {Object value = Interprettree (engine, Templatefromcontext, Tree.getchild (i), PageContext, context,                    writer);                    if (value = null) {returnvalue = value;                }}}} else {for (int i = 0; i < Tree.getchildcount (); i++) {                Object value = Interprettree (engine, Templatefromcontext, Tree.getchild (i), PageContext, context, writer);                if (returnvalue = = NULL && value! = null) {returnvalue = value;    }}} return returnvalue;            public static void Write (writer writer, object object) throws IOException {if (Object! = null) {            Writer.write (Object.ToString ());        Writer.flush (); }    }}

This class, so the number of rows is 80 rows, minus 15 rows of import and package, that is, 65 lines, from the function of the class, mainly to complete the following matters:

1. managed the Terminalnodeprocessor and Parserrulecontext
2. Parsertemplatetree: Parse text content get syntax tree
3. Interpret: Interpreting the execution Syntax tree
4. Interpret: Iterates through all nodes and interprets the execution
5. Interprettree: If it is terminalnode then find the appropriate Terminalnode executor to execute, if not found, then by the otherterminalnodeprocessor to handle-actually is to return the string , if it is a parserrulecontext node, then it is executed by the corresponding executor, executed to see if the child node is to be executed, if necessary, continue to execute the child node, or return. If neither of these is true, then traverse all the child nodes to explain the execution.

So the logic is still relatively clear, the most complex core algorithm is only 30 lines, no matter what level of students, look at the code is not any difficulty.

One thing that needs to be confessed is: why Contextprocessor's processing class is saved with a map, and terminalnodeprocessor is an array? The main purpose here is to take into account that the Terminalnode has a type that uses data in a way that is faster.

It says there are two interfaces, one is processing terminalnodeprocessor, the other is dealing with contextprocessor, and the following two interfaces.

Terminalnodeprocessor

Public interface Terminalnodeprocessor<t extends parsetree> {    int getType ();    Object process (T Parsetree, Templatecontext context, writer writer) throws Exception;}

GetType: Used to return the processor to be processed by the type used to explain the engine check is not your dish

1. process: A real place to handle logical implementations

Contextprocessor

Public interface Contextprocessor<t extends parserrulecontext> {    class<t> getType ();     Boolean Processchildren ();     Object process (Templateinterpreter interpreter, Templatefromcontext Templatefromcontext, T parsetree, TemplateContext PageContext, Templatecontext context, Templateenginedefault engine, writer writer) throws Exception; }

1. GetType: Used to return the processor to be processed by the type used to explain the engine check is not your dish
2. Processchildren: Used to tell the engine whether your sons are doing it themselves or let the explanation engine continue. Returns true to allow the engine to continue processing
3. process: A real place to handle logical implementations

At this point, the framework of the entire parsing engine is set up, and all that is left to do is to write the processors.

Terminalnodeprocessor Implementation Class example actually these implementation class really is too simple, I am embarrassed to post out, in order to let everybody see clearly, put a few to say the meaning is goodDoublenodeprocessor

public class Doublenodeprocessor implements terminalnodeprocessor<terminalnode> {public    int getType () {        return tinytemplateparser.floating_point;    }     public Boolean Processchildren () {        return false;    }     Public Object process (Terminalnode Terminalnode, Templatecontext context, writer writer) {        String text= Terminalnode.gettext ();        return double.parsedouble (text);}    }

This article means: if it is a double type of data, the string will be converted to a double value returned.Stringdoublenodeprocessor

public class Stringdoublenodeprocessor implements terminalnodeprocessor<terminalnode> {public    int getType ( ) {        return tinytemplateparser.string_double;    }    public Boolean Processchildren () {        return false;    }    Public Object process (Terminalnode Terminalnode, Templatecontext context, writer writer) {        String text= Terminalnode.gettext ();        Text=text.replaceall ("\\\\\" "", "\" ");        Text=text.replaceall ("[\\\\][\\\\]", "\\\\");        Return text.substring (1, Text.length ()-1);}    }

This article means that if it is a double quotation mark string, then the inside of some of the escape characters are disposed of, and then the outside of the double quotation marks are removed and returned.

The other and this is similar, in short, very simple, want to see the students can see the source code, here is not affixed.

Example of implementation of the Contextprocessor class the processing in this case, said that the actual is nothing complicated, the main reason is that the original in the writing template engine, the runtime of some things to do a good abstraction, so here is just a simple call. Here are 2 examples of a slightly more complex demonstration:Forprocessor

public class Forprocessor implements contextprocessor<tinytemplateparser.for_directivecontext> {public CLASS&L T    Tinytemplateparser.for_directivecontext> GetType () {return TinyTemplateParser.For_directiveContext.class;    } public boolean Processchildren () {return false; } public Object process (Templateinterpreter interpreter, Templatefromcontext Templatefromcontext, Tinytemplateparser.for_directivecontext Parsetree, Templatecontext PageContext, Templatecontext context, Templateenginedefault engine, writer writer) throws Exception {String name = Parsetree.for_expression ().        IDENTIFIER (). GetText (); Object values = Interpreter.interprettree (engine, Templatefromcontext, parsetree.for_expression (). expression (),        PageContext, context, writer);        Foriterator foriterator = new Foriterator (values);        Context.put ("$" +name + "for", Foriterator);        Boolean hasitem = false; while (Foriterator.hasnext ()) {TemplateconText Forcontext=new templatecontextdefault ();            Forcontext.setparent (context);            Hasitem = true;            Object value = Foriterator.next ();            Forcontext.put (name, value); try {interpreter.interprettree (engine, Templatefromcontext, Parsetree.block (), PageContext, Forcontext, WRI            ter);            } catch (Forbreakexception is) {break;            } catch (Forcontinueexception CE) {continue; }} if (!hasitem) {Tinytemplateparser.else_directivecontext Elsedirectivecontext = Parsetree.els            E_directive (); if (elsedirectivecontext! = null) {Interpreter.interprettree (engine, Templatefromcontext, Elsedirectivecon            Text.block (), Pagecontext,context, writer);    }} return null; }}

Here's an explanation of its execution logic:

1. first get the name of the loop variable
2. next get the object to loop
3. then build a loop iterator and put a loop variable in the context
4. Then the loop is actually executed, and if there is a break or continue instruction in the loop, then the execution
5. if the last loop is not executed, check if the Else directive exists, and if it exists, execute the

Isn't it very simple?

Mapprocessor

public class Mapprocessor implements contextprocessor<tinytemplateparser.expr_hash_mapcontext> {public Class& Lt    Tinytemplateparser.expr_hash_mapcontext> GetType () {return TinyTemplateParser.Expr_hash_mapContext.class;    } public boolean Processchildren () {return false; } public Object process (Templateinterpreter interpreter, Templatefromcontext Templatefromcontext, Tinytemplateparser.expr_hash_mapcontext Parsetree, Templatecontext PageContext, Templatecontext context, Templateenginedefault engine, writer writer) throws Exception {list<tinytemplateparser.expressioncontext> ex        pressions = Parsetree.hash_map_entry_list (). expression ();        list<tinytemplateparser.expressioncontext> expressioncontexts = expressions;        map<string, object> map = new hashmap<string, object> (); if (expressions! = null) {for (int i = 0; i < expressions.size (); i + = 2) {String key = int Erpreter.inteRprettree (engine, Templatefromcontext, Expressions.get (i), Pagecontext,context, writer). ToString (); Object value = Interpreter.interprettree (engine, Templatefromcontext, expressions.get (i + 1), PageContext, context,                writer);            Map.put (key, value);    }} return map; }}

This is a map-building processor, and its execution logic is:

1. Create a new map object and loop the put data into the map.
2. finally return to the map object

I have taken the most complex of the two, the other is more simple, so no longer posted, concerned students can go to see the source code.

Summarize

1. Actually write a new language in Java What, nothing difficult, difficult is the fear of your mind, after all, now some open-source framework such as ANTLR support, do lexical analysis, syntax tree construction is very easy one thing, as long as the planning and definition of grammar rules, the subsequent implementation is not much more complex.
2. Good design will benefit you a lot, tiny template engine from the compilation to explain execution, there is no hurt – change, just in a new way to implement the original interface only
3. The depth of the analysis of the problem determines the complexity of your code writing, the last time you discussed with a person said: Why you write not simple, because you think not enough, the analysis is not thin enough
4. at this time the reconstruction is completed, is being tested, will be launched recently.

Welcome to the Open source technology community: http://bbs.tinygroup.org . The Code and framework information in this example will be shared in the community. "Self-write framework" member QQ Group: 228977971, Hands-on, understand the secrets of the Open source framework! Or click Join QQ Group:Http://jq.qq.com/?_wv=1027&k=d0myfX

Copyright NOTICE: This article for Bo Master original article, without Bo Master permission not reproduced.

"Open source framework that thing 25": the implementation of the framework template engine of the transformation of the record