Using dynamic mounts in Java to implement hot fixes for bugs

Most JVMs have Java's hotswap feature, which most developers think is just a debugging tool. With this feature, it is possible to change the implementation of the Java method without restarting the Java process. A typical example is using the IDE to encode. However, HotSwap can implement this function in a production environment. In this way, you can extend an online application without stopping the program, or fix a small error on a running project. In this article, I will demonstrate dynamic binding, apply runtime code changes to bind, introduce some tool APIs, and a Byte Buddy library, which provides some API code changes that are more convenient.

Suppose there is a running application that performs special processing of the server by validating the x-priority header in the HTTP request. This validation is implemented using the following tool classes:

 class headerutility { Static boolean isprioritycall(httpservletrequest request) {returnRequest.getheader ("X-pirority") !=NULL; }}

Other translation versions (1)

Have you found something wrong? Such errors are common, especially in test code where constant values are decomposed into static field reuse. In less than ideal cases, this error will only be found when the product is installed, where the head is generated from another application and is not misspelled.

It is not difficult to fix such errors. In an era of continuous delivery, redeploying a new version requires just a click of a button. In other cases, however, the change may not be easy, and the redeployment process can be complicated, where downtime is not allowed and it may be better to run with errors. But HotSwap offers us another option: minor changes without restarting the app.

Attach API: Using dynamic attachments to penetrate another JVM

In order to modify a running Java program, we first need a way to communicate with a JVM that is in the running state. Because Java's virtual machine implementation is a managed system, it has the standard API for doing so. The API referred to in the question is called the Attachment API, which is part of the official Java tool. Using the APIs exposed by the running JVM allows the second Java process to communicate with it.

In fact, we have already used the API: it has been applied by debugging and simulation tools such as VisualVM or Java Mission Control . The APIs that apply these attachments are not packaged with the standard Java APIs that are used everyday, but are packaged in a special file called Tools.jar, which contains only a JDK-packaged release version of a virtual machine. Worse still, the location of this JAR file is not set, it is different in Windows, Linux, especially on the Macintosh VM, not only the location of files, even the file names are also different, some distributions are called Classes.jar. Finally, IBM even decided to modify the names of some of the classes contained in the JAR, moving all the Com.sun classes into the COM.IBM namespace, adding a fuss. In Java 9, the messy state was finally cleaned up and Tools.jar was replaced by the Jigsaw module jdk.attach .

Once the JAR (or module) of the API has been positioned, we should make it available to the attachment process. On OpenJDK, the class that is used to connect to another JVM, called Virtualmachine, provides an entry point to any VM running on the same physical machine as the JDK or a common htpspot JVM. After attaching to another virtual machine through the process ID, we are able to run a JAR file in one of the threads specified by the target VM:

// the  Following strings must be provided by us  string processid =  processid (); String jarfilename = jarfilename (); Virtualmachine virtualmachine = virtualmachine.attach (processId); try   {    virtualmachine.loadagent (jarFileName,   "world!" );}  finally   {     virtualmachine.detach ();} 

After receiving a jar file, the target virtual opportunity looks at the jar's program manifest description file (manifest) and locates the class under the Premain-class property. This is very similar to how the VM executes a main method. With a Java proxy, the VM and the specified process ID can find a method named Agentmain, which can be executed by a remote process in the specified thread:

 Public class helloworldagent {Public   static void agentmain(String Arg) {System.out.println ("Hello,"+ arg); }}

Using this API, as long as we know the process ID of a JVM, we can run the code on it, print out a Hello, world! News. It is even possible to communicate with a JVM that is not familiar with a portion of the JDK release, as long as the attached VM is a JDK installer to access the Tools.jar.

Instrumentation API: Modify the target VM's program

Everything is going well for the time being. But in addition to successfully communicating with the target VM, we are not able to modify the code and bugs on the target VM. Subsequent modifications, the Java proxy can define a second parameter to receive an instance of instrumentation. The interfaces that will be implemented later provide access to several underlying methods, one of which is able to modify the code that has already been loaded.

To fix the "x-pirority" typo, let's first assume that Headerutility introduced a fix class, called Typo.fix, in the JAR file of the agent behind the bugfixagent we developed below. In addition, we need to give the agent the ability to replace existing classes by adding can-redefine-classes:true to the manifest file. With these things now, we can use the instrumentation API to redefine the class, which accepts a pair of loaded classes and a byte array to perform the class redefinition:

 Public class bugfixagent {Public   static void agentmain(String arg, Instrumentation inst) throws Exception{//Only if header utility are on the class path; otherwise, //A class can be found within any class loader by iterating//Over the return value of Instrumentation::getallloadedclassesclass<?> headerutility = Class.forName ("Headerutility");//Copy the contents of Typo.fix into a byte arrayBytearrayoutputstream output =NewBytearrayoutputstream ();Try(InputStream input = BugFixAgent.class.getResourceAsStream ("/typo.fix")) {byte[] buffer =Newbyte[1024x768];intLength while(length = input.read (buffer))! =-1) {output.write (buffer,0, length); }        }//Apply the redefinitionInstrumentation.redefineclasses (NewClassdefinition (Headerutility, Output.tobytearray ())); }}

After running the above code, the Headerutility class is redefined to correspond to its patched version. Any subsequent calls to Isprivileged will now read the correct header information. As a small additional note, the JVM may perform a full garbage collection when the class redefinition is applied, and the affected code will be re-optimized. In summary, this can lead to a short-term decrease in application performance. In most cases, however, this is a better approach than a full restart process.

When you apply code changes, make sure that the new class defines exactly the same fields, methods, and modifiers as the classes it replaces. Attempting to modify the class redefinition behavior of any such attribute will result in unsupportedoperationexception. Now the HotSpot team is trying to get rid of this limitation. In addition, the OpenJDK-based dynamic code Evolution virtual machine supports previewing this feature.

Use Byte Buddy to track memory leaks

A simple BUG fix agent like the above example is easier to implement when you are familiar with the instrumentation API. As long as you go deeper, you can run the agent without having to create additional class files manually, but by rewriting the existing class to apply more common code modifications.

BYTE code operation

The compiled Java code presents a series of bytecode instructions. From this point of view, a Java method is nothing more than a byte array, each of which represents a command issued to the runtime, or a parameter of the most recent instruction. The mapping of each byte to its meaning is defined in the Java virtual machine specification, such as Byte 0xb1, which indicates that the VM is returned from a method with a void return type. Therefore, the enhancement of bytecode is to extend the byte numbers of a method, including the additional business logic instructions we want to apply.

Of course, byte-by-bit operations can be particularly cumbersome and error-prone. To avoid manual processing, many libraries provide a higher level of API, and using them does not require us to deal directly with Java bytecode. One of these libraries is called Byte Buddy (I'm the author of the library, of course). One of its functions is the ability to define template methods that can be executed before and after the original code of the method.

Analyze an application that has leaked

The example here assumes that an application has been found to have leaked resources after several weeks of running in the production environment. Bugs like this are difficult to track because it is difficult to reproduce such a similar problem state in an observable test environment. Therefore, in addition to restarting the crashed application, we can also modify the code of the application in the current thread to trace the leak. To find out where the application leak handle is located, we track the lifecycle of any closableobject. With a Java proxy, we can modify any construct, or close any one of these objects. With this information, we have the hope of confirming which object will leak in which case.

Working with templates

To implement this behavior, we first define the template, which contains the constructor that we want to add to any Closeable object, or the code that closes the method call. In order to understand where the leaking object was created or closed, we would like to print the tracking stack at each such time when this occurs. The template method code for this logic is as follows:

 class constructiontemplate {@Advice.onmethodexit static void exit(@Advice. This Object self) {NewRuntimeException ("Created:"+ self). Printstacktrace (); }} class closetemplate {@Advice.onmethodenter static void Enter(@Advice. This Object self) {NewRuntimeException ("Closed:"+ self). Printstacktrace (); }}

The Onmethodexit and onmethodenter annotations are added to the template method to inform Byte Buddy when we want it to be called. Any of their arguments are annotated to specify the values they represent in the redefined method. When the template is applied, Byte Buddy then maps any access to a parameter to load the value of the annotation, as if the method instance is typically represented by this.

To apply this modification to any class that implements the Closable interface, Byte Buddy provides a specific domain language to create a Java proxy for matching types and methods, and to apply the above template when it can be applied:

 Public class tracingagent {Public   static void agentmain(String arg, Instrumentation inst) {NewAgentbuilder.default ()//By default, JVM classes is not instrumented                .ignore (None () )                 . Disableclassformatchanges ()                  .with (AgentBuilder.RedefinitionStrategy.RETRANSFORMATION)                  .type (issubtypeof (Closable.class))                  .transform (builder,  type, loader)  -> builder                         .visit (Advice                                   .to (Constructiontemplate.class)                                   .on (Isconstructor ()))                          .visit (Advice                                  .to (CloseTemplate.class)                                  .on (Named ("Close"). and (Takesarguments (0))). Installon (inst); }}

By default, Byte Buddy does not modify classes that are defined in the Java.* namespace, and they can be modified by setting an invalid ignore match. In addition, Byte Buddy needs to be told not to modify the class file format we discussed earlier when we apply the re-conversion. Only in this way can Byte Buddy automatically detect any existing or future types that implement the Closable interface, and add the above template code to its constructor and close method.

Additional agents

The Byte Buddy also provides a convenient way to attach an agent at run time. To cope with the differences between the Tools.jar location and the virtual machine type, Byte Buddy adds an abstraction layer to the attachment, where the correct settings can be automatically detected. After packaging the above agent into a JAR file, you can attach to a process that has a resource leak by calling the following code:

File jarfile = GetAgent (); String processId = GetProcessID (); Bytebuddyagent.install (Jarfile, processId);

Of course, the above agent will soon create too much output for manual processing. To provide more convenience, it might be more useful to collect the objects that were created before releasing them. By creating a Java proxy, it can also be as simple as writing a Java application to implement this behavior.

Summarize

In this article, we have an in-depth look at the attach API, which makes it possible to inject a Java agent sequentially into any running JVM process. The proxy is represented as a JAR file that contains a class containing a Agentmain method that a remote process can execute in a specified thread. The method can receive an instance of an instrumentation interface as a parameter and can be redefined by reading the already loaded class. The redefinition of the code can be achieved either by replacing the entire class file with a patched version, or by modifying the bytecode of an existing class, which can be done in a relatively simple way such as a byte Buddy.

Using dynamic mounts in Java to implement hot fixes for bugs