Re-view jar package conflict problem and solution

Jar package conflict is a commonplace problem, almost every Java program ape inevitably encountered, and can also think of the usual reason is generally the same jar package due to MAVEN delivery dependencies and other reasons have been introduced a number of different versions, can be used to rely on exclusion, Rely on management and other general ways to try to solve the problem, but these methods can really completely solve the conflict problem? The answer is in the negative. The author of the article as "Re-look", because the previous understanding of the problem of jar package conflict only stay in front of those, until the work encountered a series of jar package conflict problems, it is not so easy to find that the problem has a re-understanding, Next, this article will focus on the problem nature of jar package conflict and related solutions to these two points.

Jar package Conflict Problem I. The nature of conflict

What is the nature of the jar package conflict? Google has not found a satisfactory complete definition for half a day. In fact, we can summarize the result from the jar package conflict, and give the following definition here (if there is something wrong here, welcome to make bricks-):

Java applications, for some reason, do not load the correct classes and cause their behavior to be inconsistent with expectations.

In particular, there are two cases: 1) The application relies on the same jar package has multiple different versions, and select the wrong version of the JVM can not load the required classes or load the wrong version of the class, for the convenience of the narrative, I call the First Class jar package conflict problem 2) The same class (the fully qualified name of the class) appears in several different dependent jar packages, that is, there are multiple versions of the class, and because the order of the jar loads causes the JVM to load the wrong version of the class, called the Second class jar problem . The result of these two situations is actually the same, the application will not load the correct class, and its behavior naturally inconsistent with expectations, the following two types are detailed analysis.

1.1 Several different versions of the same jar package appear

With the jar package iterative upgrade, we rely on the open-source or intra-company JAR Package tool will have a number of different versions, and version upgrade naturally avoids the class method signature changes, even the class name replacement, and our current application is often dependent on a specific version of a class M , Because Maven's delivery dependency resulted in multiple versions of the same jar package, when Maven's quorum mechanism chose the wrong version, the class m was removed in that version, or the method signature was changed, causing the application to find the required class m Or you cannot find a specific method in class M , the first type of jar conflict problem occurs. The following three prerequisites for the occurrence of this type of conflict can be summed up:

• Multiple versions of the same jar package appear in the dependency tree due to Maven's delivery dependency
• There are interface differences between multiple versions of the jar package, such as class name replacement, method signature replacement, and applications that rely on classes or methods that have changed
• MAVEN's quorum mechanism chooses the wrong version

1.2 The same class appears in several different jar packages

What is the problem when the same class appears in the two or more different jar packages that the application relies on? We know that the same class loader will only be loaded once for the same class (many different ClassLoader say, this is also a way to solve the jar package conflict, which is discussed later), then when a class appears in more than one jar package, fake A, B , C such as the length of the path the jar depends on, the order in which it is declared, or the file loading order of the file system, the class loader will not load the class in the remaining jar package after it is first loaded from Jar a . So the question is: if the application needs a class version in Jar package B at this point, and the class differs in Jar package a and b (different methods, different members, etc.), the JVM loads the jar package a In the class version, inconsistent with expectations, naturally there will be a variety of strange problems.

From the above description, you can find the following three prerequisites for conflicting problems with different jar packages:

• The same class M appears in multiple dependent jar packages, for convenience of narration, assuming two: a and B
• There are differences in the class M in Jar a and B , whether the method signatures are different or the member variables are different, as long as the behavior and expected inconsistencies of the classes that actually load can be done. If the classes in Jar a and B are exactly the same, then the class loader, regardless of which jar package is loaded first, gets the same version of Class M , without any impact, and there is no weird problem with jar-package collisions.
• The loaded class M is not the desired version, that is, the wrong jar package was loaded

Ii. causes of conflict 2.1 MAVEN arbitration mechanism

At present, MAVEN is a big line, when it comes to the first kind of jar package conflict problem, we have to mention the dependency mechanism of MAVEN. Transitive dependency is a new feature introduced by Maven2.0, which allows us to focus on the directly dependent jar packages, and for the indirectly dependent jar packages, Maven introduces them implicitly by parsing the dependency-package pom files obtained from the remote Repository, which is a great convenience for our development, but at the same time, it brings common problems --version conflict, where there are multiple versions of the same jar package, there is a quorum mechanism for MAVEN to decide which version to choose, but maven's choice is not always what we expect , which is one of the most common causes of jar packet collisions. First look at MAVEN's arbitration mechanism:

• Take precedence over the version declaration specified in the dependency management <dependencyManagement> element, at which point the following two principles are not valid
• If no version is declared, it is arbitrated according to the "Short path First" principle (Maven2.0), which is to select the shortest path in the dependency tree.
• If the path length is the same, the arbitration is based on the principle of "first declaration precedence", that is, the first declared version in Pom is selected

From the MAVEN arbitration mechanism, it can be found that, in addition to the first arbitration rule (which is also a common means of resolving jar package conflicts), the following two principles, for the same jar package different versions of the choice, Maven choice is a little "wishful thinking", Maybe this is the two conclusions that the MAVEN research team came up with when they summed up a lot of project-dependent management experience, or found that there was no single way to meet all of the scenarios, which might be appropriate for most scenarios, but it doesn't necessarily suit me-the current application , Because each application has its own specificity, which version to rely on, MAVEN has no way to help you get it done, if you don't use <dependencyManagement> to manage it properly, is doomed to escape the first kind of jar package conflict.

Load order of 2.1 jar packages

For the second type of jar-package conflict problem, there is a class conflict between several different jar packages, which is more tricky than the first type of problem. Why do you say that? In this case, two different jar packages, assuming a, B, their names are different, maybe even completely unrelated, and if it's not a conflict, you probably won't find that they have a common class! For the two jar packages A and B, Maven is powerless because Maven will only arbitrate for you for different versions of the same jar, which are different jar packages that go beyond Maven's dependency management. At this point, when both A and B appear under the classpath of the application, there is a potential risk of conflict, that is, the order of loading of A and B determines the class version that the JVM eventually chooses, and if the choice is wrong, there will be a bizarre second type of conflict.

So what are the factors that determine the load order of the jar packages? Specific as follows:

• The load path where the jar package is located, or in other words, the class loader that loads the jar package is at the level of the JVM ClassLoader tree structure. Because the JVM class loads the parent-delegation mechanism, the higher-level ClassLoader loads the class under its load path first, as the name implies, the boot ClassLoader (bootstrap ClassLoader, also called the startup ClassLoader) is the first to load its path under the jar package, The second is the extension classloader (extension ClassLoader), which is again the system ClassLoader (Systems ClassLoader, which is the application loader Appclassloader), where the jar packages are loaded with different loading paths, Determines the difference in the order in which it is loaded. For example, when we configure the resin environment for a Web application in eclipse, it is necessary to carefully consider whether the dependent jar package is added to Bootstrap Entries or User Entries in.
• File-System file load order. This factor is easily overlooked and is often the culprit for all sorts of bizarre conflicts caused by inconsistent circumstances. Because Tomcat, resin and other containers ClassLoader get the file list under the load path is not sorted, which depends on the order of the underlying file system return, then when the file system between different environments is inconsistent, there will be some environment is not a problem, some environment conflict. For example, for Linux operating systems, the order of return is determined by the order of the Inode, if the test environment of the Linux system and the online environment is inconsistent, it is very likely to have a typical case: test environment is not a problem, but the first line on the issue of conflict, The best way to circumvent this problem is to try to ensure that the test environment is consistent with the line.

Iii. the appearance of conflict

What issues can be caused by jar package conflicts? Usually occurs at compile or run time, mainly divided into two types of problems: one is more intuitive and the most common error is to throw a variety of runtime exceptions, there is a kind of more obscure problem, it will not error, its manifestation is the application behavior inconsistent with expectations, sub-article listed as follows:

• java.lang.ClassNotFoundException, which is not found in the Java class. Typically, this type of exception is due to the fact that, without declaring the version in dependency management, Maven chooses the wrong version at the time of the quorum, and this version lacks a class that we need to cause the error. For example, when the Httpclient-4.4.jar upgrade to Httpclient-4.36.jar, the class org.apache.http.conn.ssl.NoopHostnameVerifier is removed, if we need to be 4.4 version at this time, and use the NoOp Hostnameverifier this class, and when Maven Quorum chooses 4.6, it causes the classnotfoundexception exception.
• Java.lang.NoSuchMethodError, which means that no specific method is found, the first and second types of conflicts can cause the problem-the loaded class is incorrect. If the first type of conflict is caused by an inconsistency between the wrong version of the jar and the class interface in the required version of the JAR package, For example, when the Antlr-2.7.2.jar upgrade to Antlr-2.7.6.jar, the interface Antlr.collections.AST.getLine () changes, and when Maven Quorum chooses the wrong version and loads the wrong version of the class AST, Causes the exception, and if the second type of conflict is caused by the inconsistency of the same class interface with different jars, the typical case : Apache's Commons-lang package, 2. X upgrade to 3.x, the package name is changed directly from Commons-lang to Commons-lang3, some interfaces are also changed, due to the different package name and transitive dependency, there are often two kinds of jar package at the same time under Classpath, Org.apache.commons.lang.StringU Tils.isblank is one of the different interfaces, and a Nosuchmethoderror exception can occur due to the loading order of the jar package, which causes the wrong version of the StringUtils class to be loaded.
• java.lang.NoClassDefFoundError,java.lang.LinkageError and so on, the reason and the above similarities, do not make specific case analysis.
• There are no error exceptions, but the behavior of the application is inconsistent with expectations . This kind of problem is also due to the runtime loading of the wrong version of the class caused, but unlike the previous, the conflicting class interface is consistent, but the implementation logic is different, when we load the class version is not the implementation logic we need, there will be behavior and expected inconsistency problem. Such problems usually occur in our own internal implementation of the multiple jar package, due to the package path and the class name is not canonical, and so on, resulting in two different jar packages have a consistent but logical implementation of the same class with the same name, causing this problem.

Solution one, troubleshooting and resolution

1. If you have an exception stack information, you can locate the class name that caused the conflict based on the error message, and then in Eclipse CTRL+SHIFT+T or in idea you CTRL+N will find that the class exists in multiple dependent jar packages
2. If step 1 cannot locate the conflicting class from which jar, you can add the JVM parameter at application startup -verbose:class or the -XX:+TraceClassLoading log will print the load information for each class, such as from which jar package
3. After locating the jar package for the conflict class, mvn dependency:tree -Dverbose -Dincludes=<groupId>:<artifactId> view the version of the jar package that was introduced
4. After identifying the source of the jar package, if it is a first class jar package conflict, you can <excludes> exclude the unwanted jar package version or the dependency management <dependencyManagement> In the case of the second type Jar package conflict, if it can be ruled out, then use <excludes> to drain the unnecessary jar package, if not platoon, you need to consider the upgrade of the jar package or a different jar package. Of course, in addition to these methods, you can also solve the problem from the class loader perspective, you can refer to the blog post-if the jar package conflict is unavoidable, how to implement the JAR packet isolation, its thinking is worthy of reference.

Second, effectively avoid

From the previous section of the solution can be found that when the second type of jar package conflict, and the conflict jar package can not be ruled out, the problem becomes quite tricky, at this time to deal with the conflict problem will require a large cost, so the best way is to avoid the conflict before the effective ! Like the database deadlock problem, deadlock avoidance and deadlock prevention is very important, if you wait until the deadlock, the conventional approach can only be rolled back and restarted part of the transaction, which is stretched. So how can we effectively circumvent jar package collisions?

2.1 Good habits: dependency Management

For the first type of jar package conflict, it is common practice to use <excludes> to exclude unwanted versions, but the problem with this approach is that each time a jar package with transitive dependencies is introduced, it is cumbersome to exclude each. MAVEN provides a mechanism for centralizing the management of dependent information, that is, relying on the management element <dependencyManagement>, unified version management of the dependent jar package, once and for all. It is common practice to declare as many versions of the dependent jar package as possible in the parent module's Pom file, and simply reference the widget in the child Pom.

As an example, when development determines that the httpclient version used is 4.5.1, it can be configured in the parent pom as follows:

...<Properties><httpclient.version>4.5.1</Httpclient.version></Properties><dependencymanagement> < dependencies> <dependency> << Span class= "Hljs-name" >groupid>org.apache.httpcomponents</ groupid> <artifactid>httpclient </artifactid> <version>${ Httpclient.version}</version> </ dependency> </dependencies> Span class= "Hljs-tag" ></DEPENDENCYMANAGEMENT> ...        

The following dependencies are then configured for each sub-pom that needs to rely on the jar package:

...    <dependencies>      <dependency>        <groupId>org.apache.httpcomponents</groupId> <artifactId>httpclient</artifactId> </dependency> </dependencies>...

2.2 Conflict Detection Plugin

For the second type of jar package conflict, the core is that the same name class appears in a number of different jar packages, if manually to troubleshoot the problem, you need to open each jar, and then compare each other to see the same name of the class, how much waste of energy ah?! Fortunately, this time-consuming and laborious physical activity can be handed to the program to do. Maven-enforcer-plugin, this powerful maven plugin, with the extra-enforcer-rules tool, can automatically scan the jar package to detect and print the conflict, ashamed of I did not even hear the work before the existence of such a plug-in, and perhaps did not encounter such a conflict in the work, such as the rise posture. The principle is also relatively simple, by scanning the jar in the class, record each class corresponding JAR package list, if there are multiple conflicts, so do not have to delve into, we just need to focus on how to use it.

In the application module Pom that eventually needs to be packaged and run , the Maven-enforcer-plugin dependency is introduced, and the problem is discovered and resolved in the build phase. For example, for multi-module projects with the parent POM, the plug-in dependencies need to be declared in the application module's POM. There may be questions about children's shoes here, so why not declare the plugin dependency in the parent pom? Wouldn't it be possible to reuse the application submodule that relied on it? The emphasis here is on "Packaged application Module POM" because conflict detection is focused on the final integrated application and whether there is a conflict in the application runtime, and each of the different application modules depends on the set of jar packages that are different, resulting in < The ignoreclasses> list is also different, so you can only introduce the plug-in for the application module POM separately.

Let's look at the example usage as follows:

...<Plugin><Groupid>org.apache.maven.plugins</Groupid><Artifactid>maven-enforcer-plugin</Artifactid><version>1.4.1</Version><Executions><Execution><Id>enforce</Id><Configuration><Rules><Dependencyconvergence/></Rules></Configuration><Goals><Goal>enforce</Goal></Goals></Execution><Execution><Id>enforce-ban-duplicate-classes</Id><Goals><Goal>enforce</Goal></Goals><Configuration><Rules><Banduplicateclasses><Ignoreclasses><ignoreclass>javax.*</Ignoreclass><ignoreclass>org.junit.*</Ignoreclass><ignoreclass>net.sf.cglib.*</Ignoreclass><ignoreclass>org.apache.commons.logging.*</Ignoreclass><Ignoreclass>org.springframework.remoting.rmi.rmiinvocationhandler</Ignoreclass></Ignoreclasses><Findallduplicates>true</Findallduplicates></Banduplicateclasses></Rules><Fail>true</Fail></Configuration></Execution></executions> < dependencies> <dependency> <groupid>org.codehaus.mojo</groupid> <artifactid> Extra-enforcer-rules</artifactid>  <version>1.0-beta-6</version > </dependency> </dependencies></PLUGIN>     

Maven-enforcer-plugin is constrained by a number of predefined standard rules and user-defined rules to constrain MAVEN's environmental factors, such as MAVEN version, JDK version, etc., which have many useful features, which can be found in the official website. and extra Enforcer rules is the Mojohaus Project under the development of the maven-enforcer-plugin to provide additional rules of the plug-in, which includes the previous repetition of the detection function, the specific usage can be found in the official website, It is not described in detail here.

Typical case first class jar package conflict

This kind of jar package conflict is the most common and relatively good solution, already in the third, the appearance of the conflict in this section of the list of some cases, here do not repeat the list.

The second kind of jar package conflict Spring2.5.6 and spring3.x

Spring2.5.6 and spring3.x, split from a single module into multiple modules, the jar package name (Artifactid) also changes from spring to Spring-submodulename, as
Spring-context, SPRING-AOP and so on, and there are a few interface changes (in the process of jar package upgrade, this is also unavoidable). Because of the different jar packages, through Maven's transitive dependency mechanism, both versions of Spring are often present in classpath, causing potential conflict issues.

Sherlockyb
Links: https://www.jianshu.com/p/100439269148
Source: Pinterest
Copyright belongs to the author. Commercial reprint please contact the author for authorization, non-commercial reprint please specify the source.

Re-view jar package conflict problem and solution