The dynamic nature of Java programming--Reprint

Original address: http://www.ibm.com/developerworks/cn/java/j-dyn0429/

This article is the first in this new series of articles that will discuss a series of topics I call the dynamic nature of Java programming . These topics range from the basic structure of the Java binary class file format to the runtime metadata access using reflection, until the new classes are modified and constructed at run time. The common thread throughout the article is the idea that programming on the Java platform is more dynamic than using a language that directly compiles the cost machine code. If you understand these dynamic aspects, you can use Java programming to do things that cannot be done in any other mainstream programming language.

In this article, I'll discuss some basic concepts that are fundamental to the dynamic nature of these Java platforms. The core of these concepts is the binary format used to represent Java classes, including what happens when these classes are loaded into the JVM. This article is not only the foundation of the remaining articles in this series, but also shows some very practical issues that developers encounter when using the Java platform.

Class with binary representation

Developers who use the Java language often do not have to care about the details of what they are doing when compiling their source code with the compiler. But in this series of articles, I'll discuss many behind-the-scenes details from source to execution, so I'll start with a compiler-generated binary class.

The binary class format is actually defined by the JVM specification. Typically these classes are represented by compilers generated from the Java language source code, and they are typically stored in .class files with an extension. However, these features are irrelevant. A number of other programming languages have been developed that can use the Java binary class format, and for some purposes, new class representations have been built and are immediately loaded into the running JVM. In the case of the JVM, the important part is not the source code and how to store the source code, but the format itself.

So what does this class format actually look like? Listing 1 provides the source code for a (very) short class, along with a partial hexadecimal display of the class file that is output by the compiler:

Listing 1. Hello.java source code and (partial) binary class files

public class hello{public    static void Main (string[] args) {        System.out.println ("Hello, world!");}    } 0000:cafe babe 0000 002e 001a 0a00 0600 0c09 ..... .....  0010:000d 000e 0800 0f0a 0010 0011 0700 1207 ..... .....  0020:0013 0100 063c 696e 6974 3e01 0003 2829 .....  <init> () 0030:5601 0004 436f 6465 0100 046d 6169 6e01  V ... code...main.0040:0016 285b 4c6a 6176 612f 6c61 6e67 2f53  . ([ljava/lang/s0050:7472 696e 673b 2956 0c00 0700 0807 0014  tring;) V ... 0060:0C00 1601 000d 4865 6c6c 6f2c 2057 ...  Hello, w0070:6f72 6c64 2107 0017 0c00 1800 1901 0005 Orld  !........... 0080:4865 6c6c 6f01 0010 6a61 7661 2f6c 616e  hello...java/lan0090:672f 4f62 6a65 6374 0100 106a 6176 612f  g/Ob ject...java/00a0:6c61 6e67 2f53 7973 7465 6d01 0003 6f75  Lang/system...ou ...

Inside the binary class file

The binary class representation shown in Listing 1 is first the "Cafe Babe" signature, which identifies the Java binary class format (and, incidentally, a permanent-but largely non-realized-gift to the hard-working barista, They build the Java platform in the spirit of the developer. This signature is just a simple way to verify that a block of data is actually declared as an instance of the Java class format. Any Java binary class (even a class that does not appear in the file system) needs to start with these four bytes.

The rest of the data is less appealing. The signature is followed by a pair of class format version numbers (in this case, the minor version 0 and major version 46― generated by 1.4.1 Javac in hexadecimal notation is 0x2e) followed by the total number of items in the constant pool. The total number of items (in this case, 26, or 0x001a) is followed by the actual constant pool data. This puts all the constants used by the class definition. It includes class and method names, signatures, and strings (which you can identify in the text interpretation to the right of the hexadecimal dump), as well as a variety of binary values.

The length of items in a constant pool is variable, and the first byte of each item identifies the type of item and how it is decoded. Here I do not delve into the details of all of this, and if interested, there are many resources available, starting with the actual JVM specification. The key is that the constant pool contains all references to other classes and methods used by the class, and also contains the actual definition of the class and its methods. Constant pools tend to account for half or more of the binary class size, but may be less on average.

There are also several items behind the constant pool that reference the class itself, its superclass, and the constant pool entry for the interface. These items are followed by information about fields and methods, which are themselves represented by complex structures. The executable code of the method appears in the form of code attributes that are included in the method definition. This code, commonly called bytecode , is represented in the form of a JVM directive, which is one of the topics discussed in the next SectionTo.

In the Java class format, attributes are used for several defined purposes, including the bytecode already mentioned, the constant value of the field, exception handling, and debugging information. However, attributes are not only possible for these purposes. From the outset, the JVM specification has required the JVM to ignore attributes of an unknown type. The flexibility of this requirement makes it possible to extend the use of attributes in the future for other purposes, such as providing the meta information required to use the framework of the user class, which is widely used in the Java-derived C # language. Unfortunately, no hooks have been provided for the use of this flexibility at the user level.

Back to top of page

byte code and stack

The bytecode that makes up the executable part of a class file is actually the machine code that ―jvm― for a particular type of computer. It is called a virtual machine because it is designed to be implemented using software rather than hardware. Each JVM used to run the Java platform application is built around the implementation of the machine.

This virtual machine is actually quite simple. It uses a stack architecture, which means that they are loaded into the internal stack before the instruction operand is used. The instruction set contains all the general arithmetic and logical operations, as well as conditional transfer and unconditional transfer, mount/store, call/return, stack operations, and several special types of instructions. Some directives contain immediate manipulation values, which are encoded directly into the instruction. Other directives directly refer to the values in the constant pool.

Although virtual machines are simple, implementations are not. The early (first generation) JVM is basically an interpreter of the virtual machine bytecode. These virtual machines are actually relatively simple, but there are serious performance issues-the time to interpret the code will always be longer than the time it takes to execute the native code. To mitigate these performance issues, the second generation JVM added an immediate (Just-in-time,jit) conversion. Before the Java bytecode is executed for the first time, JIT technology compiles it to the cost machine code, providing better performance for repeated executions. The performance of contemporary JVMs is even much better, as adaptive technology is used to monitor program execution and selectively optimize frequently used code.

Back to top of page

Loading class

Languages such as C and C + +, which compile the cost machine code, usually need to link this step after compiling the source code. This link process merges the code from individual compiled source files with the shared library code to form an executable program. The Java language is different. In the Java language, a compiler-generated class usually remains intact until it is loaded into the JVM. Even building a JAR file from a class file does not change this ―jar just a container for class files.

A linked class is not a separate step, it is part of the job that the JVM executes when it loads these classes into memory. This step adds some overhead to the initial loading of the class, but also provides a high degree of flexibility for Java applications. For example, when you write an application to use an interface, you can specify its actual implementation to run time. This post-compilation approach to assembling applications is widely used in the Java platform, a common example of servlet.

The rules for loading classes are described in detail in the JVM specification. The basic principle is to load the class only when it is needed (or at least it appears to be loaded ―JVM has some flexibility in actual loading, but must maintain a fixed class initialization order). Each loaded class may have other classes that it depends on, so the loading process is recursive. The class in Listing 2 shows how this recursive loading works. The Demo class contains a simple main method, creates an Greeter instance of it, and invokes a greet method. The Greeter constructor creates an Message instance that is then used in the greet method call.

Listing 2. Class loading the source code of the demo

public class demo{public    static void Main (string[] args) {        System.out.println ("**beginning execution**");        Greeter Greeter = new Greeter ();        System.out.println ("**created greeter**");        Greeter.greet ();    }} public class greeter{    private static message S_message = new Message ("Hello, world!");        public void greet () {        s_message.print (System.out);    }} public class message{    private String m_text;        Public Message (String text) {        m_text = text;    }        public void print (Java.io.PrintStream PS) {        ps.println (m_text);    }}

javasetting parameters on the command line -verbose:class prints the tracking record for the class loading process. Listing 3 shows some of the output from running the Listing 2 program with this parameter:

Listing 3. Partial output of the-verbose:class

[Opened/usr/java/j2sdk1.4.1/jre/lib/rt.jar] [Opened/usr/java/j2sdk1.4.1/jre/lib/sunrsasign.jar] [Opened/usr/java/j2sdk1.4.1/jre/lib/jsse.jar] [Opened/usr/java/j2sdk1.4.1/jre/lib/jce.jar] [Opened/usr/java/j2sdk1.4.1/jre/lib/charsets.jar] [Loaded Java.lang.Object From/usr/java/j2sdk1.4.1/jre/lib/rt.jar] [Loaded java.io.Serializable From/usr/java/j2sdk1.4.1/jre/lib/rt.jar] [Loaded java.lang.Comparable From/usr/java/j2sdk1.4.1/jre/lib/rt.jar] [Loaded java.lang.CharSequence From/usr/java/j2sdk1.4.1/jre/lib/rt.jar] [Loaded java.lang.String From/usr/java/j2sdk1.4.1/jre/lib/rt.jar] ... [Loaded Java.security.Principal From/usr/java/j2sdk1.4.1/jre/lib/rt.jar] [Loaded java.security.cert.Certificate From/usr/java/j2sdk1.4.1/jre/lib/rt.jar] [Loaded demo]**beginning execution**[loaded greeter][loaded message]**created Greeter**hello, World! [Loaded Java.util.hashmap$keyset From/usr/java/j2sdk1.4.1/jre/lib/rt.jar] [Loaded Java.util.hashmap$keyiterator From/usr/java/j2sdk1.4.1/jre/lib/rt.jar]

This lists only the most important parts of the output-the complete tracking record consists of 294 lines, and I deleted most of them, forming the list. The initial set of classes loaded (279 in this case) is Demo triggered when you try to load the class. These classes are the core classes that each Java program, however small, uses. Demo mainThis initial mount order is not affected even if all code for the method is deleted. However, different versions of the class library involve different classes and names.

In the above list, the part that Demo follows the class is more interesting. The order here shows Greeter that the class is loaded only when you are ready to create an instance of the class. However, the class Greeter uses a Message static instance of the class, so you Greeter must first load the class before you can create an instance of the class Message .

When a class is loaded and initialized, many operations are done inside the JVM, including decoding the binary class format, checking compatibility with other classes, verifying the order of bytecode operations, and finally constructing the java.lang.Class instance to represent the new class. This Class object is the basis for all instances of the JVM creating a new class. It is also the identity of the loaded class itself-for the same binary class that is loaded into the JVM, there can be multiple replicas, each with its own Class instance. Even though these replicas share the same class name, they are separate classes for the JVM.

Non-regular (class) path

Classes loaded into the JVM are controlled by the class loader . A bootstrapper class loader is built into the JVM, which is responsible for loading the basic Java class Library classes. This special class loader has some special features. First, it loads only the classes found on the boot classpath. Because these are trusted system classes, the bootstrapper loader skips a lot of validation of regular (untrusted) classes.

The bootstrapper is not the only class loader. For starters, the JVM defines an extension class loader for classes loaded in the standard Java extension API, and a system class loader for classes that load on generic classpath, including application classes. Applications can also define their own class loader for special purposes, such as reloading of run-time classes. This adds a class loader that derives from a java.lang.ClassLoader class (possibly indirectly derived) that provides core support for building an inner class representation (instance) from a byte array java.lang.Class . Each well-constructed class is "owned" in a sense by the class loader that loads it. Class loader usually preserves the mappings of the classes they load, so that when a class is requested again, it can be found by name.

Each class loader also retains a reference to the parent class loader, which defines the class loader tree, the root of which is the bootstrapper loader. When an instance of a particular class (identified by name) is required, no matter which class loader initially processes the request, the parent class loader is generally checked before attempting to mount the class directly. If there is a multilayer class loader, this step is performed recursively, so this means that the class is usually visible not only in the class loader that is loading the class, but also for all descendant class loader. This also means that if there is more than one class loader on a chain that can load a class, then the class loader at the top of the tree is the class loader that actually loads the class.

In many environments, Java programs use multiple application class loader. The Java EE framework is an example. Each Java EE application that the framework loads needs to have a separate class loader to prevent classes in one application from interfering with other applications. The framework code itself will also use one or more other class-loader, which is also used to prevent interference from the application or from the application. The entire class loader collection forms a hierarchical structure of trees that can be loaded into different types of classes at each level.

Loader tree

As a practical example of the class loader hierarchy, figure 1 shows the class loader hierarchy defined by the Tomcat servlet engine. Here the Common class loader is mounted from a JAR file of a specific directory installed by Tomcat and is intended to be used to share code between the server and all WEB applications. The Catalina loader is used to load Tomcat's own classes, and the shared loader is used to load classes shared between Web applications. Finally, each WEB application has its own loader for its private class.

Figure 1. Tomcat class Loader

In this environment, it can be confusing to track the appropriate loader for requesting new classes. To do this, the setContextClassLoader methods and methods are getContextClassLoader added to the class in the Java 2 platform java.lang.Thread . These methods allow the framework to set the class loader so that the class loader can be used for the application when running code in each application.

Ability to load independent class collections This flexibility is an important feature of the Java platform. Although this feature is useful, it can be confusing in some cases. One confusing aspect is the old problem of dealing with JVM classpath. For example, in the Tomcat class loader hierarchy shown in Figure 1, classes loaded by the Common class loader must not directly access the classes loaded by the WEB application (by name). The only way to associate these classes is by using the interfaces that are visible through the two class sets. In this example, it is included in the Java servlet implementation javax.servlet.Servlet .

The problem occurs for whatever reason the code is moved between the class loader. For example, when J2SE 1.4 moves the JAXP API for XML processing to a standard distribution, problems arise in many environments because the applications in those environments previously relied on loading their own chosen XML API implementations. With J2SE 1.3, the problem can be resolved as long as the appropriate JAR file is included in the user class path. In J2SE 1.4, the standard versions of these APIs are now in the extended classpath, so they will typically overwrite any implementations that appear in the user class path.

Using multiple class loader can also cause other types of confusion. Figure 2 shows an example of class identity Crisis , which is a crisis that occurs when the two standalone class loader loads an interface and its associated implementation. Crisis An instance of a class from one loader cannot be considered an interface that implements a different loader, even if the name and the binary implementation of the interface and class are the same. This confusion can be lifted in Figure 2 by moving the interface class I to the space in the System class loader. The class A still has two separate instances, but they all implement the same interface I .

Figure 2. Class Identity Crisis

Back to top of page

Conclusion

The Java class definition, along with the JVM specification, defines a very powerful framework for run-time assembly code. By using a class loader, Java applications can use multiple versions of a class, otherwise these classes can cause conflicts. The flexibility of the class loader even allows you to dynamically reload the modified code while the application continues to execute.

Here, the flexibility of the Java platform is to some extent at the cost of starting the application at a higher cost. Before the JVM can start executing even the simplest application code, it must load hundreds of separate classes. This startup cost typically makes the Java platform more suitable for long-running server-type applications, relative to frequently used applets. Server applications also benefit most from the flexibility of code assembly at runtime, so it's no surprise that the Java platform is becoming more and more favored for this development.

In part 2nd of this article series, I'll cover another aspect of using the dynamic foundation of the Java platform: the Reflection API (Reflection API). Reflection enables the execution code to access internal class information. This can be a great tool for building flexible code, and you can hook up your code at run time without using any source code links between classes. But as with most tools, you have to know when and how to use it for maximum benefit. Read the dynamics of Java programming part 2nd to understand the tricks and pros and cons of effective reflection

The dynamic nature of Java programming--Reprint