Java 9 is finally going to include the jigsaw project.

When Jigsaw was finally released in Java 9, the project had a history of more than eight years.

Reproduced in: http://www.itxuexiwang.com/a/liunxjishu/2016/0228/180.html?1456925937

In the first few years, it had to compete with two other similar Java specification requests (Java specification request) called the JSR 277 Java module System and the JSR 294 Enhanced modularity support (improved modularity supports). It also led to conflicts with the OSGi community, where it was feared that the jigsaw project would be an unnecessary and incomplete functional duplication, forcing Java developers to choose between two incompatible modules.

In the early days, the project was not staffed enough to be interrupted even when Sun was incorporated into Oracle in 2010. Until 2011, the strong need for a modular system in Java was reiterated, and the work was fully restored.

The next three years was an exploratory phase, ending in 2014 in July, when several Java enhancement proposals (Java Enhancement proposal) were established, including the JEP 200 modular JDK (Modular jdk), JEP 201 Modular source codes (Modular source code) and Jep 220 Modular runtime Mirroring (Modular run-time image), as well as the final JSR 376 Java Platform Module system (Java Platform Module systems). The last item above defines a real Java module system, which will be implemented as a new Jep in the JDK.

In July 2015, the JDK was divided into which modules had been roughly determined (see JEP), the JDK's source code was also refactored to accommodate this change (see Jep 201), and the run-time mirroring (run-time image) was also ready for modularity (see Jep 220). All of this can be seen in the current preview version of JDK 9.

The code developed for JSR 376 will soon be deployed to the JDK repository, but it is regrettable that the modular system itself is not yet experienced. (Currently, the preview version of Java 9 already includes the modular functionality.) --the translator's note)

Driving force

In the history of the Jigsaw project, there have been some changes in its driving force. Initially, it just wanted to modularize the JDK. But when people realize that the tool can be used in the Code of libraries and applications, it can be very profitable, and its scope is expanded. #p # pagination Title #e#

The growing and indivisible Java runtime

The size of the Java runtime continues to grow. But before Java 8, we had no way to install a subset of the JRE. All Java installation packages contain distribution versions of the various libraries, such as XML, SQL, and Swing APIs, which are included, whether we need them or not.

This is not a serious problem for computing devices of medium size (such as desktop PCs and laptops), but this is serious for small devices, such as routers, TV boxes, and cars, and other small places to use Java. With the current containerized trend, there are also related requirements in the server domain, because reducing the size of the mirror means reducing costs.

Java 8 introduces the functionality of the compact profile, which defines a subset of three Java SE. Alleviate this problem to some extent, but they can only work in a strictly limited scenario, the profile is too rigid to cover all the current and future requirements for using the JRE functionality.

Jar/classpath Hell

Jar Hell and Classpath hell is a humorous statement that refers to the problem caused by the flaws in the Java class loading mechanism. Especially in large applications, they can produce painful problems in a variety of ways. Some of the problems are caused by other problems, while others are independent.

Unable to express dependencies

A jar file cannot express what other jars it relies on in a way that the JVM can understand. As a result, users are required to manually identify and satisfy these dependencies, which requires the user to read the document, locate the correct project, download the jar file, and add it to the project.

Also, some dependencies are optional, and a jar relies on another jar only if the user is using a feature-specific feature. This will make the process more complicated. #p # pagination Title #e#

The Java runtime cannot detect this dependency until it actually uses a dependency. If this happens, an exception will occur NoClassDefFoundError , causing the running app to crash.

Build tools like Maven can help solve this problem.

Transitive dependency

It may be sufficient for an application to run with just a few libraries, but these libraries will require some additional libraries. The problem will become more complex, and it will grow exponentially as the energy consumed and the likelihood of error.

Similarly, build tools can provide some help on this issue.

Covered

Sometimes, different jar packages in Classpath may contain fully qualified classes, such as two different versions of the same library that we use. Because the class is loaded from the first jar package in Classpath, this version of the variant will "mask" all other versions, making them unusable.

If these different variants are semantically differentiated, it can lead to various levels of problems, from hard-to-find abnormal behavior to very serious errors. To make matters worse, the manifestation of the problem is uncertain. This depends on the order of the jar files in the classpath. In different environments, there may be differences, such as between the developer's IDE and the production machine where the code eventually runs.

Version conflict

This problem arises if there are two required libraries in the project that depend on different versions of the third library. #p # pagination Title #e#

If two versions of this library are added to the classpath, the final behavior is unpredictable. First, because of the masking problem described earlier, only one of the two versions of the class will be able to load in. Even worse, if a class is in a jar package, but the other classes it accesses are not in the package, this class can also be loaded. The result is that the code calls to this library will be mixed in two versions.

In the best case scenario, attempting to access code that does not exist in the loaded class will result in a noticeable NoClassDefFoundError error. But in the worst case, the difference between versions is only semantically, and the actual behavior is slightly different, which introduces bugs that are hard to spot.

It is difficult to identify the unpredictable behavior caused by this situation, and it cannot be solved directly.

Complex class-Loading mechanism

By default, all classes are loaded by the same one, and ClassLoader in some scenarios it may be necessary to introduce additional loaders, such as allowing the user to load new classes and extend the application.

This can quickly lead to complex class-loading mechanisms that produce unpredictable and difficult-to-understand behavior.

Between packages, there is only a weak encapsulation mechanism

If the class is in the same package, the Java visibility modifier provides a great way to implement encapsulation between these classes. However, to cross the boundary between packages, you can use only one visibility: public.

Because the class loader puts all the loaded packages together, the public class is visible to all other classes, so if we want to create a feature that is available to a jar and is unavailable outside of the jar, there is no way to implement it. #p # pagination Title #e#

Manual security

A direct result of weak encapsulation between packages is that security-related features will be exposed to all the code in the same environment. This means that malicious code has the potential to bypass security restrictions and access critical functionality.

Starting with Java 1.1, there is a hack way to prevent this: whenever a security-related code path is entered, it is called and the SecurityManager decision is not to allow access. More precisely, it should be called on every such path. In the past, there were some gaps in the omission of calls to them in some places, which bothered Java.

Startup performance

Finally, the Java runtime takes a long time to load and JIT-compile all the required classes. One reason is that the class loading mechanism performs a linear scan of all jars under Classpath. Similarly, when you identify the usage of an annotation, you need to explore all the classes under Classpath.

Goal

The goal of the Jigsaw project is to solve the problem described above by introducing a language-level mechanism for modular large-scale systems. This mechanism will be used in the JDK itself, and developers can use it in their own projects.

It is important to note that not all goals are of equal importance to the JDK and to our developers. Some are more relevant to the JDK, and most of them do not have a significant impact on everyday programming (this contrasts with recent language modifications, such as lambda expressions and default methods). However, they will still change the development and deployment of large projects.

Platform for scalability

After the JDK is modularized, the user can pick out the features they need and create their own JRE, which contains only the modules they need. This helps to maintain Java as a key player in small equipment and container areas. #p # pagination Title #e#

In this proposed specification, the Java SE platform and its implementation are allowed to be decomposed into a set of components that developers can assemble to form a custom configuration that contains only the functionality that the application actually needs. --JSR 376

Reliable Configuration

With this specification, a module is able to declare dependencies on other modules. The runtime environment can analyze these dependencies during the compile period (compile-time), build period (Build-time), and Startup (Launch-time), and fail soon if there is a lack of dependency or dependency conflicts.

Strong package

One of the main goals of the Jigsaw project is to have the module export only specific packages, and the other packages are private to the module.

A private class in a module is like a private domain in a class. In other words, the boundaries of a module not only determine the visibility of classes and interfaces, but also define its accessibility. --mark Reinhold wrote the article "Project Jigsaw: Transforming a grand blueprint into a focusable point"

Improve security and maintainability

In modules, strong encapsulation of the internal API can greatly improve security, because the core generation??? It is hidden from the rest of the code that is not necessary to use them. Maintenance will also become easier because we can make it easier to make the module's public API smaller.

The internal API, which is freely implemented with the Java SE platform, is not only a security risk, but also a maintenance burden. The proposed specification provides strong encapsulation, so that components that implement the Java SE platform can block access to their internal APIs. --JSR 376#p# pagination Title #e#

Improve performance

Because the boundaries of the code used can be defined more clearly, existing optimization techniques can be used more effectively.

Many pre-Ahead-of-time optimization and full program (Whole-program) optimization techniques are more efficient because it is possible to know that a class only references classes in several specific components, and it does not refer to any class that is loaded at run time. --JSR 376

Core Concepts

Because Modularity is the goal, the Jigsaw project introduces the concept of module, which is described as follows:

A named, self-describing program component that contains code and data. Modules must be able to contain Java classes and interfaces, organized as packages, and can contain native code in the form of dynamically loaded libraries (Dynamically-loadable library). The data for the module must be able to contain static resource files and user-editable configuration files. --java Platform Module System: requirements (draft 2)

In order to be able to understand the module based on a certain context, we can think of well-known libraries, such as libraries in Google Guava or Apache Commons (such as collections or IO), as modules. Each library can be divided into multiple modules, depending on the granularity the author wants to divide.

This is true for applications as well. It can be used as a monomer (monolithic) module, or it can be split. The size and cohesion of the project will be an important factor in determining how to divide it into modules.

By planning, when you organize your code, the module becomes a general tool in the Developer Toolbox.

#p # pagination Title #e#

Developers are now able to take into account some of the standard program components, such as language-level classes and interfaces. The module will be another program component, like classes and interfaces, that will work at all stages of program development. --mark Reinhold's article "Project Jigsaw: Transforming a grand blueprint into a focusable point"

Modules can be further combined into various configurations of the development phase, namely, the compile, build, install, and run phases. For Java users like us, this can be done (often referred to as the developer module in this case), and it can also be used to dissect the Java runtime itself (which is often referred to as the Platform module at this point).

In fact, this is the JDK's current modular planning.

(Click to enlarge image)

Characteristics

So, how does the module work? A look at the needs of the jigsaw project and the JSR 376 will help us understand it.

Dependency Management

To solve the problem of "Jar/classpath hell", a key feature of the Jigsaw project is dependency management. Let's take a look at these related components.

Declaration and Resolution

The module will declare which other modules it needs to compile and run. The module system uses this information to identify all required modules in a transitive manner, ensuring that the initial module compiles and runs. #p # pagination Title #e#

Instead of relying on specific modules, we can rely on a set of interfaces. The module system will attempt to identify the module, which implements the interface on which it depends, and the system binds it to the corresponding interface.

Version of

The module will be versioned. They are able to mark their own versions (to a large extent can be in any format, as long as they can fully represent the order), and the version can be used to limit dependencies. These two pieces of information can be overwritten at any stage. The module system enforces the requirement of configuration to meet all constraints at all stages.

The jigsaw project does not necessarily support multiple versions of a module in one configuration. But, wait a minute, then how to solve the problem of jar hell? Good question!

Version selection-for the same module, pick the most appropriate version in a different set of versions-and not as a specification to complete the task. So, as I wrote above, the module system recognizes the required modules for compilation, and at runtime it may use a different module, based on the assumption that only one version of the module exists in the environment. If there are multiple versions, then the upstream step (such as the developer or the build tool he uses) must make a choice, and the system will only verify that it satisfies all constraints.

Packaging

The module system enforces strong encapsulation at all stages. This is implemented around an export mechanism, in which case only the package exported by the module can be accessed. The encapsulation SecurityManager is independent of the security checks performed.

The specific syntax for this proposal is not yet defined, but Jep 200 provides some key semantic XML instances. As a sample, the following code declares the java.sql module.

#p # pagination Title #e#<module>  <!-- 模块的名字 -->  <name>java.sql</name>  <!-- 每个模块都会依赖java.base -->  <depend>java.base</depend>  <!-- 这个模块依赖于java.logging和java.xml       模块，并重新导出这些模块所导出的API包 -->  <depend re-exports="true">java.logging</depend>  <depend re-exports="true">java.xml</depend>  <!-- 这个模块导出java.sql、javax.sql以及       javax.transaction.xa包给其他任意的模块 -->  <export><name>java.sql</name></export>  <export><name>javax.sql</name></export>  <export><name>javax.transaction.xa</name></export></module>

From this code snippet we can see that java.sql depends on java.base , java.logging as well java.xml . When we introduce the different export mechanisms later, we will be able to understand the other declarations above.

Export

The module declares a specific package for export, and only the types contained in those packages can be exported. This means that other modules can only see and use these types. More strictly, other modules must explicitly declare dependencies on the modules that contain these types before they can be exported to the corresponding module.

It is interesting that different modules can contain packages of the same name, and that these modules can even be exported.

In the example above, the java.sql packages are exported java.sql , javax.sql as well javax.transaction.xa .

Re-export

We can also re-export the API (or part of it) in a module that it depends on. This will provide support for refactoring, and we can split or merge modules without destroying dependencies because the initial dependencies can continue to exist. The reconstructed modules can export the same package as before, even though they may not contain all the code. In extreme cases, there is a so-called #p# page title #e# aggregator Module (aggregator module) that can contain no code at all, just as an abstraction of a set of modules. In fact, this is what the compact profile provided in Java 8 does.

From the example above, we can see that the java.sql API that it relies on is re-exported, that is, java.logging and java.xml .

Restrict Export

To help developers (especially those who are modular JDK) to have a smaller interface for their exported API, there is an optional restricted export (qualified export) mechanism that allows a module to declare some packages to be exported only for a specific set of modules. So when using the "standard" mechanism, the module that exported the feature did not know (nor cared) who would access the packages, but by restricting the export mechanism, a module could be allowed to limit the dependencies that could be generated.

Configuration, phase, and Fidelity (Fidelity)

As mentioned earlier, one of the goals of JEP 200 is that modules can be combined into various configurations at various stages of development. This can be true for platform modules, so that you can create a mirror similar to a full JRE or JDK, which is used by the compact profile introduced by Java 8 and any custom configuration that contains a collection of specific modules (and their cascade dependencies). Similarly, developers can use this mechanism to combine different variants of their applications.

At compile time (compile-time), the code to be compiled can only see the packages that are exported in the configured module collection. During the build period (Build-time), with a new tool (which might be called jlink), we were able to create binary runtime mirrors that contained only specific modules and their dependencies. During the installation period (Launch-time), the image can look like it contains only a subset of the modules it has.

We are also able to replace the modules that implement the authorization criteria (endorsed standard) and the standalone technology (standalone technology), which can be replaced with newer versions at any stage. This overrides the deprecated authorization standard overloading mechanism (endorsed standards override mechanism) and the extension mechanism (see below.) ) #p # page Title #e#

All aspects of a module system, such as dependency management, encapsulation, and so on, run in exactly the same way at all stages, unless for a specific reason, at some stage.

All information related to the module, such as version, dependency, and package export, is described in the code file, which is independent of the IDE and the build tool.

The technology of Full program optimization of performance

In a modular system, the strong encapsulation technology makes it easy to automatically calculate where a particular piece of code is used. This makes program analysis and optimization techniques more feasible:

Quickly find JDK and application classes, early detection of bytecode, positive cascade (aggressive inlining) content like lambda expressions and other compiler optimizations; build JVM-specific memory images that are more efficient than class files when loaded , pre-compiling the method body into native code, and removing unused fields, methods, and classes. --jigsaw Project: Objectives & Requirements (draft 3)

There are a number of technologies called Full program Optimization (Whole-program optimization) that implement at least two of these technologies in Java 9. There is also a tool that can be used to analyze a given set of modules and use the above optimization techniques to create a higher performance binary image.

Annotations

Currently, to automatically discover classes with annotations (such as the configuration class for spring annotation annotations), you need to scan all classes under a specific package. This is usually done when the program starts, which slows down the startup process to a considerable extent.

The module will provide an API that allows the caller to recognize all classes with the given annotations. An expected way is to create an index for such a class, which is created when the module is compiled. #p # pagination Title #e#

Integrates with existing concepts and tools

Diagnostic tools, such as stack trace information, will be updated, which will contain information about the module. They are also integrated into the reflection API so that they can be used in the same way as the action class, and contain version information that can be reflected or overloaded at run time.

The module is designed to??? We "reduce the hassle (with a minimum of fuss)" as much as possible when using the build tool. After the compiled module can be used in classpath, but also as a module to use, so that the library developers do not need for the CLASSPATH application and module-based applications to create a number of components respectively.

The interoperability with other modular systems is also planned, the most notable of which is OSGi.

Although the module can hide the package from other modules, we can still perform white-box testing on the classes and interfaces contained in the module.

Packages for specific OS

The module system is designed with the Guarantee manager file format in mind, such as RPM, Debian, and Solaris IPS. Not only can developers use existing tools to create a set of module collections as packages for specific OS, but they can also invoke other modules that are installed in the same mechanism.

Developers are also able to package a set of modules that comprise an application as packages for a specific OS, and "end users can install and invoke the packages they make in accordance with the common practice of the target system." Based on the above introduction, we can know that only the non-existent modules in the target system must be packaged in.

Dynamic configuration

Running apps can create, run, and publish stand-alone module configurations. In these configurations, developers and platform modules can be included. This can be useful for container class schemas, such as Ides, application servers, or other Java EE platforms. #p # pagination Title #e#

Non-compatibility

As is customary in Java, these changes are implemented with a strong focus on backwards compatibility, and all standard and non-deprecated APIs and mechanisms can continue to be used. However, the project may depend on other structures that lack documentation, so that when migrating to Java 9, it will require some extra work.

The internal API is not available

With strong encapsulation, each module is able to explicitly declare which types are part of its API. The JDK will use this feature to encapsulate all the internal APIs, so they will become unusable.

This may be the largest part of the coverage in the incompatibilities that Java 9 brings. But this is also the most obvious, because it causes compilation errors.

So, what is an internal API? There is no doubt that all the content is in sun.* the package. If it is in a com.sun.* package, or if annotations are used, @jdk.Exported it is still available in the Oracle JDK and is not available if there are no annotations.

An example of a particular problem is a sun.misc.Unsafe class. It is used in many projects to achieve critical tasks or high performance-demanding code that may not be available in the future, triggering a lot of discussions. However, it has been suggested in a related communication that it is still available through an obsolete command line tag. Considering that it is not possible to put all of its functions into the public API, this may be a necessary tradeoff.

Another example is com.sun.javafx.* all the content in the package. These classes are critical to building JavaFX controls, and they have a certain number of bugs to modify. Most of the functionality in these classes will be used as a target for publishing.

Merge jdk and jre#p# pagination title #e#

With the extensible Java Runtime, it allows us the flexibility to create run-time mirroring, and the JDK and JRE lose their unique features, which are just two forms in the module mix.

This means that the two artifacts will have the same structure, including the same directory structure, and any code that relies on it, such as a subdirectory in the original JDK directory that is known as the JRE, will not function properly.

The internal jar is not available

An lib/rt.jar lib/tools.jar internal jar like and such will be unavailable. Their content will be stored in a specific implementation of the file, the format of these files is not clearly stated, it is possible to change.

Any code that assumes the existence of these files will not run correctly. This may cause some switching problems for the IDE or other tools that rely heavily on these files.

New URL pattern for run-time mirrored content

At run time, some APIs return URLs for Class and resource files (such as Classloader.getsystemresource). Before Java 9, they were jar URLs in the following format:

jar:file:<path-to-jar>!<path-to-file-in-jar>

The jigsaw project will use the module as a container for the code files, and a single jar will be unavailable. This requires a new format, so these APIs will return the JRT URL:

jrt:/<module-name>/<path-to-file-in-module>

If you use the instance returned by these APIs to access the code for the file (such as Url.getcontent), it will run the same way as it does now. However, if you rely on the #p# page header #e# structure of the jar URL (such as manually building them or parsing them), you will get an error.

Remove authorization standard overloading mechanism

Some Java APIs are called "Standalone technologies" (Standalone technology), and they are created outside of Java Community Process (such as JAXB). For them, it is possible to upgrade their dependencies or use alternative implementations. The licensing standard overloading mechanism allows the installation of these standard alternative versions in the JDK.

This mechanism is deprecated in Java 8 and will be removed in Java 9 and replaced by the upgradeable module described above.

Remove extension mechanism

With the extension mechanism, custom APIs can be used by all applications running in the JDK without having to name them in Classpath.

This mechanism is deprecated in Java 8 and will be removed from Java 9. Some features that are inherently useful will be preserved.

What are you going to do next?

We have a brief overview of the history of the jigsaw project, see what is driving its development and discuss its objectives, and how to achieve these goals through some features. What else can we do besides waiting for Java 9?

Get ready

We should do some preparatory work for our project to check whether they depend on what is being removed in Java 9.

At the very least, manual searches are no longer required to check for internal API dependencies. Starting with Java 8, the JDK contains the Java Dependency Analysis tool (Java Dependency analytical Tools), named #p# page title #e#jdeps (some internal packages, official documents for Windows and UNIX). It can list all the packages that a project relies on. If you use the-parameter at run time jdkinternals , it will list almost all of the internal APIs used by the project.

"Almost everything" is said because it does not recognize all the packages that are not available in Java 9. This will at least affect the package that JavaFX belongs to, and you can view the JDK-8077349. (by using this search, I could not find any other flaws except for the missing features.) ）

There are at least three jdeps plugins for maven: Apache, Philippe Marschall, and myself. For now, the last one is the only jdeps plug-in that jdkinternals causes build failures when the report relies on an internal API. (now, Apache plug-ins will also prompt for build failures in the presence of an internal API dependency, see Infoq's news.) --the translator's note)

Discuss

The latest news from the jigsaw project is from the Jigsaw-dev mailing list. I will also continue to discuss this topic in my blog.

If you are concerned that a particular API is not available in Java 9, then you can view the mailing list for the related OPENJDK project, as they will be responsible for developing the published version.

Using

Early trial builds of Java 9 are already available. However, the JSR 376 is still in the development phase, and there are many changes to the modular system that are not yet available in these builds. (in the current trial version, Jigsaw is included, but the latest news is that Java 9 has been released for six months.) In fact, in addition to the strong package, other functions are already ready.

Send the collected messages to the Jigsaw-dev mailing list for feedback to the project. Finally, quote Jep 220 (near) the end of the paragraph: #p # page Title #e#

It is not possible to determine the full impact of these changes in an abstract way, so it is important to rely on extensive internal testing, especially for external testing. [......] If there are changes that can put an unbearable burden on developers, deployer, or end users, we'll look at ways to mitigate their impact.

In addition, there is a global Java user group ADOPTOPENJDK, which can be a good way to connect early adopters.

About the author

Nicolai Parlog is a software developer who is passionate about Java. He continues to read, think and write about Java-related content, and he relies on coding for a living, and also for this pleasure. He is a long-term contributor to multiple open source projects and uses blogs to record software development-related content on Codefx. You can focus on Nicolai on Twitter.

Java 9 is finally going to include the jigsaw project.