Using Project Lombok to customize the AST transformation

Transfer from http://www.ibm.com/developerworks/cn/java/j-lombok/

Using Project Lombok to customize the AST transformation

When and how to generate extensions for custom code Lombok

Alex Ruiz introduced Project Lombok in this article and explored some of its unique programming features, including comment-driven code generation, as well as concise, compact, and readable code. He then prompts attention to Lombok's more valuable uses: to extend it with a custom AST (abstract Syntax tree) transformation. The extended Lombok allows you to build your own project or domain-specific boilerplate code, but it does require a lot of work. Finally, Alex provides some tips for simplifying the key steps of the process, as well as a free-to-use JavaBeans custom extension.

Even for conservative Java™ developers, lengthy syntax is a weakness of Java language programming. While it is sometimes possible to avoid lengthy by using new languages such as Groovy, many times Java programming is most appropriate and sometimes even required. Then you might want to try Project Lombok, which is an open source, code generation library for the Java platform.

Lombok can easily reduce the code size of boilerplate files in Java applications, so you don't need to encode a lot of Java syntax. But it's not just syntax that makes Lombok so sweet, it's a unique code-generation approach that can open all Java development possibilities.

In this article, I'll introduce Project Lombok and illustrate its advantages, though imperfect, enrich the Java Developer's Toolkit. I'll give you an overview of the Lombok, including how it works and where it works best, and simply lists its pros and cons. Next, I'll introduce you to one of the most useful, but also very complex, Lombok use cases: extend it to a custom code base. This may be your own code or an existing Java template, which is not part of the Lombok library. Either way, the next section of the article will focus on the tips and tricks of extending Lombok, including determining whether it is worthwhile to spend time on the Lombok API, or whether you can better write sample files for your particular application.

The included sample code (see download) expands Lombok to generate the JavaBeans boilerplate code. This license is free to use in Apache 2.0 environments.

What makes Lombok different

Perhaps the main reason for choosing Lombok instead of other code generation tools is that Lombok not only generates Java source or bit code: it transforms the abstract syntax tree (AST) by modifying its structure during the compilation phase. The AST represents a tree that has parsed the source code, which is created by the compiler, similar to the DOM tree model of the XML file. The source code can be trimmed to avoid bloat by modifying (or converting) Ast,lombok, which is different from plain text code generation. The code generated by Lombok is visible to classes of the same compilation unit, unlike direct character-encoding operations with libraries, such as CGLib or ASM.

Lombok supports several mechanisms for triggering code generation, including very popular Java annotations. With Java annotations, developers can modify annotated classes, which are forbidden by the normal Java triggering process.

For examples of Lombok use, refer to the classes in Listing 1:

Listing 1. A simple Java class

public class Person {  private String firstName;  Private String lastName;  private int age;}

adding, and implementing to your code equals hashCode toString is not difficult, but tedious and error-prone. You can use modern Java ides like Eclipse to generate the main sample code, but that's only part of the solution. This saves time and effort, but at the expense of readability and legibility, because sample code usually adds noise words to the application source.

However, Lombok has a smart way to solve boilerplate code problems. In Listing 1, for example, you can Person.java @Data easily generate the required methods by adding comments to the class. Figure 1 shows the code generation for Lombok in Eclipse. In outline view, you can see that the generated method is shown in the compilation class, while the source file is still outside the sample file.

Figure 1. Lombok of the event

Specific details of the Lombok

Lombok supports the popular Java compiler javac as well as Eclipse Compiler for Java (ECJ). Although the two compilers produce similar output, their implementations are completely different. As a result, Lombok comes with two sets of annotation handlers (hooks to code in Lombok and included Code generation logic): one for each compiler. Fortunately, this is transparent, so as a user, we only have to face a set of Java annotations.

Lombok also provides tight integration with eclipse: Saving Java files automatically triggers Lombok code generation (no noticeable delay) and updates Eclipse's outline view to show the members produced, as shown in Figure 1.

For developers who want to know what's going on inside, the Lombok delombok tool will guide you through Maven or ANT command-line access. Delombok Gets the code that is converted by Lombok and generates a generic Java source file based on it. Code that has been processed by Delombok will contain the conversion previously done by Lombok, in the form of plain text. For example, if delombok you apply the code in Figure 1, you will be able to see,,, equals hashCode and toString have been implemented.

Set Lombok

In order to add support for Lombok to the project, you only need to add to the Classpath lombok.jar . Based on the final Lombok conversion, determine if it needs to be included in the lombok.jar class directory at run time. (Lombok the upcoming release, 0.10, will not need to be included at run time lombok.jar .) For more information on Lombok installation, please visit the project homepage (see Resources).

There is no perfect thing: the disadvantage of Lombok

Before you choose Lombok and prepare to apply it in your project, you should know that it has some limitations. The two main areas are:

• the strength of Lombok may be a weakness . The main objection to Lombok's view is that it shows "too much magic". First, by removing Java code verbosity, Lombok changes the way many Java programmers love the language: WYSIWYG. With the Lombok,.java file, it is no longer possible to show what the. class file contains.
  
  Second, as we know, a specific Lombok transformation will fundamentally change the Java syntax. @SneakyThrowsThe conversion is an obvious example. It allows you not to declare the checked exceptions in the method definition, but to throw them away as if they were unchecked exceptions: Listing 2. @SneakyThrows-pretty Sneaky

    Normally, we would need to declare the This method throws Exception  @SneakyThrows public  void DoSomething () { C7/>throw new Exception ();  }

• Lombok the annotation naming convention does not communicate intent . In Lombok, annotations are no longer just metadata: they essentially drive code generation like commands . I believe @GenerateGetter the annotations will be able to @Getter communicate intentions better than the current annotations.

In addition to these Lombok related issues, there are some issues related to Eclipse integration. In most cases, this is due to the fact that Eclipse does not understand the Lombok code generation:

• Eclipse is raised every now and then when it generates code with Lombok NullPointerException . The cause of the problem is not clear now. Shutting down and reopening Eclipse usually resolves this issue.
• Lombok will add trouble to the refactoring in Eclipse. For example, using Eclipse to rename a field that contains Lombok to generate getters and setters methods, you need to press alt-shift-r two times to use the Rename Field dialog instead of performing the In-place fields rename. In the Preview step, you need to deselect GetXXX and setxxxfrom the type you are refactoring.
• Because there is no Java source for Lombok generated code, debugging can become a bit confusing. For example, if you attempt to process getName code that Lombok generates a getter, the Eclipse debugging tool jumps to name the comment for the field @Getter . In addition, when Lombok appears, the Eclipse debugging tool will work as usual.

In general, these issues can be bypassed, and most of the problems in the future may be resolved by Lombok with the Eclipse Dev team. However, it is best to be aware of the technology to be applied. This can add new tools to the toolbox at any time.

Extended Lombok

Lombok generates most of the common Java sample code, including getters, setters, and equals hashCode , for example, just a few. This is useful, but sometimes you also need to generate your own sample code. For example, Lombok does not support some public coding patterns, such as JavaBeans. In some cases, you may also want to generate code that is assigned to a project or domain.

The best use case for scaling is to prototype and experiment with new code patterns in the early stages of the project. These code patterns are becoming more and more mature, so it's easy for Lombok to make changes or enhancements: Just modify the comment handler (the part of the snippet that is attached to the Lombok to generate the code) and compile. All base code will be automatically updated (unless the public conventions in the generated code change, resulting in compilation errors) Once these code patterns are determined, you can select the delombok code. As a result, you can use a regular Java source.

What is an AST conversion?

The AST transformation is the name of the code that is used to modify the abstract syntax tree structure during compilation. Modifying the AST is a better way to generate code by adding additional nodes before converting them to bytecode. (This is relative to the common technique of making text code or directly generating bytecode operations.) Using the AST directly also means that developers need access to the compiler API, which introduces a feature of Lombok that other code generation tools do not have: Custom compilation errors and warnings.

To extend Lombok, you need to identify or create comments that will trigger Lombok code generation. Next, you will need to write a comment handler for each comment that you determine. A comment Handler is a class- aka code generation that implements a pair of Lombok interfaces and transformation logic.

The following sections contain recommendations, from project Setup to testing, which can be useful when creating your own AST transformations. It also includes some code examples that demonstrate the functional Lombok extensions that are used to support JavaBeans. Further articles will be introduced in depth.

Lombok Generating JavaBeans Code

As I mentioned earlier, Lombok currently supports public code mode, but it is not fully covered, including JavaBeans. To demonstrate the Lombok extension, I have written a very simple project for generating JavaBeans full-length (plumbing) code. In addition to showing how to use custom annotation handlers to extend Lombok for Javac and ECJ, the project also packages some useful tools (such as fields and method builders for each compiler) that make the process clearer and simpler.

I took a snapshot of Eclipse 3.6 (Helios) and Lombok Library for version 0.10-beta2 git . The code contains the build JavaBean "binding" setters. The additional zip file (see the Download section) contains the following:

• Ant Build File
• Comments @GenerateBoundSetter and@GenerateJavaBean
• Generates a comment handler for the binding setter (for Javac and ECJ)
• Some JavaBeans plumbing (for example, PropertyChangeSupport the generation of fields)

The attached code has full functionality and is licensed under Apache 2.0. You can get the updated version of the code from GITHUB (see Resources). Here's a quick tour of code features to find inspiration.

If you write code in Listing 3 with my trigger handler, Lombok will generate the code similar to that in Listing 4:

Listing 3. lombok! Generate javabean!

@GenerateJavaBeanpublic class Person {  @GenerateBoundSetter private String firstName;}

Listing 4. Example of generating JavaBean support code

public class Person {public  static final String prop_first_name = "FirstName";    Private String firstName;   Private Propertychangesupport Propertysupport = new Propertychangesupport (this);  public void Addpropertychangelistener (PropertyChangeListener listener) {    Propertysupport.addpropertychangelistener (listener);  }  public void Removepropertychangelistener (PropertyChangeListener listener) {    Propertysupport.removepropertychangelistener (listener);  }    public void Setfirstname (string value) {    string oldValue = FirstName;    FirstName = value;    Propertysupport.firepropertychange (Prop_first_name, OldValue, firstName);}  }

See the Readme.txt file included in the sample code to learn how to build the Eclipse project from the build file of the sample code.

Extended Standard

You will need some criteria when determining what code to generate from the extended Lombok. On the one hand, the code should be simple. As you'll see in the next section, the APIs used to create the Lombok extension are complex, and even generating simple elements requires a lot of code. Extending Lombok is not easy, and maintenance work will be more difficult and costly (this is true for many projects). Extension Lombok should be considered only if the capacity is truly increased.

Getting Started: Javac or ECJ?

In my opinion, any Lombok extension needs to support both Javac and ECJ at the same time, at least for now. Javac is the default compiler used by build tools such as Ant and Maven. However, at the time of writing this article, when used with Lombok, Eclipse provides the most fluid coding experience possible. Supporting two compilers at the same time is critical to improving developer productivity.

Javac and ECJ use a similar AST structure. Unfortunately, their deployment is completely different, which allows you to write two annotation handlers for each note, one for Javac and another for ECJ. The good news is that the Lombok team has started working on the unified AST API, which will eventually implement a single annotation handler for each comment (see Resources) when using two compilers.

Understanding the source code of Lombok

The next step is to understand what's going to happen, and it's best to look at the source code.

Lombok uses non-public APIs to implement its intelligent code generation techniques in Javac and ECJ. Because the code will be inserted into the Lombok, you should have a similar API, even without the same API.

The main problem with non-public APIs is the lack of documentation and reliability. Fortunately, according to the Lombok team, they have not yet encountered a compatibility issue with the new version of Eclipse, which we have a chance to see when Java 7 is released. Currently, the lack of documentation is the biggest problem that has to be addressed. In addition, even with good documentation, learning two different compiler APIs is a tough and time-consuming task. What we need is a "quick and useful guide" for Javac and ECJ-some of which are beyond the scope of this article.

The good news is that the Lombok team has done a lot of work on documenting the AST nodes using Javac and ECJ. It is highly recommended that you read their code. They provide the most common use cases: for example, variable declarations, method implementations, and so on. Reading the source code of Lombok is the quickest way to learn the API of Javac and ECJ. Listing 5 shows an example of the source code owned by Lombok:

Listing 5. Generating local variables with Javac

  /* Final int PRIME = 31; */{    if (!fields.isempty () | | callsuper) {      statements.append (maker. Vardef (maker. Modifiers (flags.final),          primename, maker. Typeident (Javac.getctcint (Typetags.class, "INT")),           maker. Literal));}  }

As you can see, the Lombok team has recorded what blocks produce what. The next time you need to generate a declaration for a local variable, you can go back to this source and use this as a reference.

Do not limit yourself to reading Lombok. java files. The Lombok developer has provided pointers for setting up the build project and for testing the comment handlers. More details on these topics are described in the following sections.

Dependency Management

If you try to automate dependency management in your project, it is difficult to return to manual mode. There are several build tools in the Java architecture to provide dependency management, including Ivy and Maven (see Resources). However, when you create a Lombok extension, the selection narrows to one, and it is Ivy.

One of the reasons to choose Ivy is that all the necessary dependencies, such as Javac, are in the central repository of MAVEN-which excludes maven. Another reason is that Ivy supports management dependencies that are not available in the Maven library. You can easily specify a link to download dependencies. This configuration requires a custom ivysettings.xml configuration file, which is relatively simple.

Ivy is located above Ant and provides dependency management for builds. The Lombok team adopted the optimized version of Ivy that they developed, ivyplusplus (see Resources). This Ivy extension provides some useful Ant targets (targets), such as creating Eclipse and IntelliJ project files from a series of dependencies.

Increase dependency

The following files are required to set up the Lombok extension project:

• build.xml File: Ant Build File:
  • Download Ivyplusplus (from a specific location) the first time the build is called.
  • Specifies the location of the Ivy configuration file.
  • Compile, test, and package your code.
• buildscripts/ivy.xml File: Specifies the dependency of the project.
• buildscripts/ivysettings.xml File: Specifies the library (Maven or just URLs) from which to obtain dependencies.
• Buildscripts/ivy-repo folder: Each containing an XML file, specifying each dependency in the Ivy.xml. These XML files describe a dependency artifact (for example, where to provide the download, the home page, and so on)

You don't have to do repetitive work. To save time and effort, take a look at the build file from Lombok, or from the additional resources and other required content in this article.

Note naming

As mentioned earlier, Lombok's annotations are not only meta-data, but also perform well in communication tasks. They should indicate that they are responsible for triggering some types of code generation. Therefore, I strongly recommend that you put all Lombok-related comments in front of "Generate". In the source code of this article, I have @GenerateBoundSetter named the comment that triggered the JavaBeans related source @GenerateJavaBean . This naming convention gives at least one clue to developers who are unfamiliar with the codebase, that there is a process of generating code in the build environment.

Document Record AST Conversion

The documentation is important when you extend Lombok. The document comment handler will benefit the maintainer of the AST transformation, and the documentation comment will benefit its users .

Document Comment handlers

Code that uses the Javac or ECJ API is not cumbersome to read or understand. Even if it generates the simplest Java code, it is complex and time-consuming. Documenting the note handlers can ease the maintenance work for you and your team. With regard to documentation issues, I found the following to be useful:

• The
• class-level Javadoc comment explains what code was generated in a certain height comment handler. I think the best way to explain what code is generated is to include the sample code in the comments, as shown in Listing 6: Listing 6. Class-level Javadoc of an annotation handler

  /** * instructs Lombok to generate the necessary code To make an annotated Java * Class A JavaBean. * <p> * For example, given this class: * * <pre> * @GenerateJavaBean * public class Person {* *} * </PR e> * Our Lombok annotation handler (for both Javac and Eclipse) would generate * the AST nodes that correspond to this Code: * * <pre> * public class Person {* * Private propertychangesupport Propertysupport * = new Propert Ychangesupport (this); * * public void Addpropertychangelistener (PropertyChangeListener l) {* Propertysupport.addpropertychangelistener (L) ; *} * * public void Removepropertychangelistener (PropertyChangeListener l) {* Propertysupport.removepropertychang Elistener (l); *} *} * </pre> * </p> * * @author Alex Ruiz */

• Typically, the non-Javadoc comments in the codebase explain what the code block generates, as shown in Listing 7: Listing 7. Documenting what a block of code generates

      public void Setfirstname (string value) {    //   final String oldValue = firstName;   firstName = value;   Propertysupport.firepropertychange (Prop_first_name, OldValue,     //       firstName);    }    jcvariabledecl fielddecl = (jcvariabledecl) fieldnode.get ();    Long mods = Tojavacmodifier (accessLevel) | (FieldDecl.mods.flags & STATIC);    Treemaker Treemaker = Fieldnode.gettreemaker ();    list<jcannotation> nonnulls = findannotations (Fieldnode, non_null_pattern);    Return Newmethod (). Withmodifiers (mods)                      . Withname (Settername)                      . Withreturntype (Treemaker.type (Voidtype ( ))                      . Withparameters (Parameters (Nonnulls, Fieldnode)).                      Withbody (Body (propertynamefieldname, Fieldnode))                      . Buildwith (Fieldnode);

Document comments

Adding a class-level Javadoc comment similar to the one we used in the comment handler (in Listing 6) helps annotate the user's knowledge and understanding of what happens when they use these annotations.

Compiler consistency

This tip is useful if you decide to support both Javac and ECJ. When you have two sets of annotation handlers, any bug fixes, changes, or additions should be applied simultaneously to both sets (or branches). The more similar the branch, the faster and safer the change will be. This similarity must occur at both the package level and the file level.

Package-level consistency : the more the better, each branch (Javac and ECJ) should have the same number of classes, using the same name, as shown in 2:

Figure 2. Similarity of Javac and ECJ branch packages

file-level consistency : because these two branches may have more or less similar number of classes, with similar names, comments in each pair of files with the same name must be as similar as possible: fields, method counts, method names, and so on, should all be the same. Listing 8 shows the methods for Javac and ECJ generatePropertySupportField . Note that even for different AST APIs, the implementations of these methods are very similar.

Listing 8. Compare Javac and ECJ annotation handlers

Javac private void Generatepropertychangesupportfield (Javacnode typenode) {if (Fieldalreadyexists (property_support    _field_name, Typenode)) return;    Jcexpression exprforthis = Chaindots (Typenode.gettreemaker (), Typenode, "this"); Jcvariabledecl fielddecl = Newfield (). OfType (Propertychangesupport.class). Withnam E (property_support_field_name). Withmodifiers (PRIVATE | FINAL). Withargs (Exprforthis). Buildwith (t    Ypenode);  Injectfield (Typenode, fielddecl); }//ECJ private void Generatepropertychangesupportfield (Eclipsenode typenode) {if (Fieldalreadyexists (Property_suppor    T_field_name, Typenode)) return;    Expression exprforthis = Referenceforthis (Typenode.get ()); Fielddeclaration fielddecl = Newfield (). OfType (Propertychangesupport.class). Wit Hname (property_support_field_NAME). Withmodifiers (PRIVATE | FINAL). Withargs (exprforthis). Buildwi    Th (Typenode);  Injectfield (Typenode, fielddecl); }

Test AST Conversion

Testing a custom AST conversion is easier than you think, thanks to the testing infrastructure provided by Lombok. To illustrate how easy it is to test the AST transformation, let's take a look at the JUnit test cases in Listing 9:

Listing 9. Unit tests for all ECJ annotation handlers

Import static Lombok. Directoryrunner.compiler.ecj;import Java.io.file;import lombok.*;import Lombok. Directoryrunner.compiler;import Lombok. Directoryrunner.testparams;import org.junit.runner.runwith;/** * @author Alex Ruiz * * @RunWith (Directoryrunner.class) public class TESTWITHECJ implements Testparams {  @Override public Compiler Getcompiler () {    return ECJ;  }  @Override public boolean printerrors () {    return true;  }  @Override Public File Getbeforedirectory () {    return new file ("Test/transform/resource/before");  }  @Override Public File Getafterdirectory () {    return new file ("TEST/TRANSFORM/RESOURCE/AFTER-ECJ");  }  @Override Public File Getmessagesdirectory () {    return new file ("TEST/TRANSFORM/RESOURCE/MESSAGES-ECJ");}  }

The test work is somewhat more or less similar to the following scenario:

1. The test compiles getBeforeDirectory all the Java files in the specified folder, using the getCompiler compiler specified by the Lombok.
2. When programming is complete, test exploits delombok Create a textual representation of the compiled class.
3. The test reads the getAfterDirectory files in the specified folder. These files contain the content of the expected compiled class. The test compares the contents of these files with the files obtained in [2nd]. The file you are comparing must have the same name.
4. The test getMessagesDirectory reads the file from the folder specified in. These files contain the expected compiler messages (warnings and errors). The test compares the contents of these files to the actual values shown during compilation, and if you compile a Java file, you do not need a message file, and there is no expected message. Match by name. For example, if CompleteJavaBean.java a compiler message is expected at compile time, the file containing such a message should be named CompleteJavaBean.java.messages .
5. If the expected value matches the actual value, the test passes; otherwise, it fails.

As you can see, this is a very different but very effective way to test the comment handler:

• Each compiler is one junit test instead of one JUnit test per note handler.
• Unlike a test method for each use case, we have a text file containing the expected build code and an optional text file containing the expected compiler message.
• The test does not care how to use the Javac with the ECJ API. The test verifies that the generated code is correct.

Verify the generated code

The tests I describe are useful in verifying that the comment handler generates the code that you expect. However, you also need to test the generated code to really accomplish the task you expect. To verify the correctness of the generated code attributes, you need to write a Java class that takes your AST transformation, and then write tests to examine the characteristics of the generated code. Test as if the code is what you wrote.

The simplest way to compile and return those tests is to use Ant, which means compiling with javac. Because you have tested and learned that the code generated with ECJ is correct, you do not have to run these tests inside Eclipse, which causes the settings to be severely complicated.

I have included tests for Javac and ECJ comment handlers in the sample code for this article (see download).

Conclusion

Project Lombok is an effective tool for simplifying lengthy Java code. It does this by using Java annotations and compiling APIs in an unusual and intelligent way. Like other tools, it's not perfect. The benefits (code brevity) are paid for: Java code loses its WYSIWYG style, and developers lose some of their favorite IDE features. Before adding Lombok to the toolbox, be sure to consider its pros and cons and determine if the gain is greater than the loss.

If you decide to take Lombok, you might want to extend it to generate your own boilerplate code. At present, although the extension of Lombok is not simple, but it is feasible. This article provides some guidance on extending Lombok and describes how to do it. It takes time and experience to Lombok extensions, or manually create boilerplate code, which is more cost-effective for you to decide.

Using Project Lombok to customize the AST transformation