Go Android Dex Sub-package scheme

Reprinted from: https://m.oschina.net/blog/308583

When the functionality of an app becomes more complex, and the amount of code becomes more and more, one day you may suddenly encounter the following phenomena :

1. Generated APK before 2.3 machine cannot install, prompt install_failed_dexopt

2. The number of methods is too large, error at compile time, prompt:

Conversion to Dalvik format failed:unable to execute Dex:method ID not in [0, 0xFFFF]: 65536

This problem occurs because :

1. Android2.3 and previous versions of memory used to perform dexopt (for optimizing Dex files) are allocated only 5M

2. A DEX file supports a maximum of 65,536 methods.

In response to the above problems, there have been many solutions, the most used is the plug-in, will be some of the independent function of a separate apk, when opened with Dexclassloader dynamic loading, and then use the reflection mechanism to invoke the plug-in class and methods. This is a solution to the problem: But there are two problems with this solution:

1. Plug-in is only suitable for some relatively independent modules;

2. The class and method of the plug-in must be called through the reflection mechanism, so it must be paired with a set of plug-in frameworks;

Due to the existence of these problems, through continuous research, we have the Dex subcontracting solution. Simply put, the idea is to package the compiled class file into two Dex, bypassing the limit on the number of Dex methods and checking the installation, and then dynamically loading the second Dex file at run time. Facebook has encountered similar problems, for reference:

https://www.facebook.com/notes/facebook-engineering/under-the-hood-dalvik-patch-for-facebook-for-android/10151345597798920

There is a passage in this article:

However, there was no-the-could break our app up this way--too many of our classes is accessed directly by the Android Framework. Instead, we needed to inject our secondary Dex files directly into the System class loader.

It's easier said than done: In addition to the first Dex file (that is, the only Dex file that the normal APK package contains), other Dex files are placed in the installation package in a resource way. and is injected into the classloader of the system in the application OnCreate callback. Therefore, for those classes that have been referenced before injection (and the jar they are in), they must be placed in the first Dex file.

The following is a simple demo that describes the Dex subcontracting scenario, which is implemented in two steps:

The entire demo directory is structured like this, and I'm going to put Secondactivity,mycontainer and Dropdownview in the second Dex package, and the rest in the first Dex package.

First, compile time packet

The entire compilation process is as follows:

In addition to the box two target, the other is the standard process of compiling. And these two target is our subcontracting operation. First look at Spliteclasses target.

Since we are just a demo here, there are very few files placed in the second package, which is the three files mentioned above. After the package is ready, start generating the Dex file, first packaging the first Dex file:

The first Dex is generated by packaging the ${classes} (the folder under which you want to package to the first Dex file). The second Dex is then generated and packaged into a resource file:

As you can see, the files in ${secclasses} are packaged to generate Dex and added to the AP file (the packaged resource file). In this way, the subcontracting is complete, then we will analyze how to dynamically inject the second Dex package into the system's ClassLoader.

Second, the DEX sub-package injection ClassLoader

When it comes to injection, it's about Android's ClassLoader system.

As can be seen, on the leaf node, we can use Dexclassloader and Pathclassloader, by looking at the development documentation, we found that they have the following usage scenarios:

1. With regard to Pathclassloader, the document reads: Android uses this class for IT system class loader and for its Application class loader (s ),

So, Android app is to use it to load;

2. Dexclass can load Apk,jar, and Dex files, but Pathclassloader can only load apk files that have been installed in the system (that is, the/data/app directory).

Know the use of the two scenarios, the following to analyze the specific loading principle, can be seen, two leaf nodes of the class are inherited Basedexclassloader, and the specific class loading logic in this class:

Basedexclassloader:

[Java] view plaincopy

2. @Override

4. Protected class<?> Findclass (String name) throws ClassNotFoundException {

6. list<throwable> suppressedexceptions = new arraylist<throwable> ();

8. Class C = pathlist.findclass (name, suppressedexceptions);

10. if (c = = null) {

12. ClassNotFoundException Cnfe = new ClassNotFoundException ("didn ' t find class \" "+ name +" \ "on path:" + pathList);

14. for (Throwable t:suppressedexceptions) {

16. cnfe.addsuppressed (t);

18. }

20. Throw Cnfe;

22. }

24. return C;

26. }

The above function shows that when we need to load a class, actually from the pathlist to need, consult the source code, found that PathList is an instance of the Dexpathlist class. OK, then analyze the Findclass function in the Dexpathlist class,

Dexpathlist:

[Java] view plaincopy

2. Public Class Findclass (String name, list<throwable> suppressed) {

4. for (Element element:dexelements) {

6. Dexfile dex = Element.dexfile;

8. if (dex! = null) {

10. Class clazz = dex.loadclassbinaryname (name, Definingcontext, suppressed);

12. if (clazz! = null) {

14. return clazz;

16. }

18. }

20. }

22. if (dexelementssuppressedexceptions! = null) {

24. Suppressed.addall (Arrays.aslist (dexelementssuppressedexceptions));

26. }

28. return null;

30. }

The approximate logic of the above function is to traverse an array of Dex files (each Dex file is actually a Dexfile object) (element array, element is an inner class), then load the required class file sequentially until it is found.

As we see here, the injected solution comes to the surface, and if we put the second Dex file into the element array, we should be able to find it directly when we load the class in the second Dex package.

With this hypothesis, to perfect the demo.

In our custom baseapplication OnCreate, we perform an injection operation:

[Java] view plaincopy

2. public string inject (string LibPath) {

4. Boolean hasbasedexclassloader = true;

6. try {

8. Class.forName ("Dalvik.system.BaseDexClassLoader");

10. } catch (ClassNotFoundException e) {

12. Hasbasedexclassloader = false;

14. }

16. if (Hasbasedexclassloader) {

18. Pathclassloader Pathclassloader = (pathclassloader) sapplication.getclassloader ();

20. Dexclassloader Dexclassloader = new Dexclassloader (LibPath, Sapplication.getdir ("Dex", 0). GetAbsolutePath (), LibPath, Sapplication.getclassloader ());

22. try {

24. Object dexelements = Combinearray (getdexelements (Getpathlist (Pathclassloader)), Getdexelements (GetPathList ( (Dexclassloader)));

26. Object pathList = getpathlist (Pathclassloader);

28. SetField (PathList, Pathlist.getclass (), "dexelements", dexelements);

30. return "SUCCESS";

32. } catch (Throwable e) {

34. E.printstacktrace ();

36. Return android.util.Log.getStackTraceString (e);

38. }

40. }

42. return "SUCCESS";

44. }

This is the key function of injection, analyze This function:

The parameter libpath is the file information for the second DEX package (which contains the full path, which we originally packaged into the assets directory) and then uses Dexclassloader to load it (why it must be loaded using Dexclassloader, Review the above usage scenario) and then get the element array in the dexpathlist in Pathclassloader by reflection (the first Dex package loaded, loaded by the system), and an element array in Dexpathlist in Dexclassloader (just loaded with the second Dex package), after merging two element arrays and assigning them to the element array of Pathclassloader, The injection is complete.

Now try to start the app and start Secondactivity (in the second Dex package) in Testurlactivity (in the first Dex package) to start successfully. Such a scheme is feasible.

But there are still a few points to note with the Dex subcontracting scheme:

1. Since the second Dex package is injected dynamically in the application OnCreate, if the Dex package is too large, it will slow down the app startup, so it is important to note during Dex subcontracting that the second Dex package should not be too large.

2. Due to the limitations of the 1th above, if our app is becoming bloated and bulky, it will often be used with the Dex subcontracting and plug-in solution to load some non-core standalone functions into plug-ins and core functions to be re-loaded.

Go Android Dex Sub-package scheme