APK slimming, how to achieve a compression effect of up to 53%

APK slimming, how to achieve a compression effect of up to 53%

Non-GE @ Ali Poly security

1. How do I think about this thing ?

APK is the file format of the Android system installation package, about this topic is actually a cliché topic, whether it is inside the company, or outside the network, predecessors have summed up a lot of methods and laws. However, with the rapid development of mobile technology in recent two years, some new ways of thinking and optimization have gradually emerged and matured. In the course of practice, I step over some pits, harvest some experience, here to do a thinking and summary, so the essay to everyone, I hope that we are engaged in related work when there are helpful and reference, but also a point, I hope we jointly explore this open topic.

about why the apk to lose weight, this does not say, only from three aspects nagging, for users (or customers), the larger the apk, in the download installation process, they consume more traffic, installation wait time will be longer, for the product itself, means that the download conversion rate will be lower (because of the competition, The user has more opportunities to choose the best, most functional, best performance and smallest package); For research and development, it is an opportunity to optimize technology improvement.

To lose weight, let's find out the causes and problems of fat. According to the target-path-resource thinking mode, to find the reasons and problems have the following path, one is to shoot the head, according to their own experience and judgment, or even subjective imagination; second, to search engines to find keywords, browse various Technical forum Listening technology How to say, see all kinds of technical article extraction refining The third is to use a measurable tool or method to find the problem.

The first two do not repeat, I am here to talk about the third method. Using a measurable tool or method to analyze, the so-called 工欲善其事, its prerequisite. The device can be forged on its own, or it can be used off-the-shelf. Here is an online APK analysis tool, because it is an external tool, so please use the process, do not upload unpublished products, for data security, the author here to take a github on the open-source Android project as a slimming example.

2. Looking for problems

Nimbledroid, a PhD entrepreneurship team at Columbia University, developed a system that analyzes the performance indicators of Android apps in both static and dynamic ways, where static analysis can analyze the big file leaderboards in APK installation package, The size of various well-known SDK and the proportion of the overall code, the size of various types of files and ranking, the number of well-known SDK methods and the proportion of all the methods in Dex, Nonsense said, below the HD uncensored big picture look at the value bar.

If you want to use analytics to analyze your product, please log in and upload your product's APK package, all of which are currently free to use, and if you want to analyze products that have already been published on Google Play, you can click on "Play Apps" to view them, and you can use the search function to view the results based on the app name and package name. Again, please do not upload any unpublished products.

Login

Upload apk file

Summary of the analysis results, you can see some overview of the information, apk file size, total number of methods

File size Analysis Details page, large file list, listed here is more than 100k files in the apk file ranking, here the file size refers to the size of the APK file

The size of the various well-known SDKs and the percentage of the code as a whole are now recognizable as Android Support,jackson JSON parser, Google Play Services, Paypal, Glide, OkHttp, Facebook S DK, Fabric, Gson, etc., application represents the part of the code that you write in your app

Size and rank of various types of files

Various well-known SDKs represent the ratio of the number of methods in all Dex

List of methods of various well-known SDKs

Reading this apk inside is not a refreshing feeling! I compare this analysis tool to our home-bought smart scale, which can weigh weight, fat content, bone weight, bone density, muscle content, and so on, so we have found some problems, and then we can relate these problems to our previous experience and the reason why we have a shot head.

So, we can then analyze the data to sort out our optimization goals.

1. In the big file leaderboard, there are 11 PNG files larger than 100k, remember, this is compressed AH;

2. In the large file leaderboard, the size of the RESOURCES.ARSC is close to 2M, which is also an optimization point;

3. Large file rankings, Classes.dex close to 3m,classes.dex is the carrier of the code, the optimization needs to subdivide, and then to look at the leaderboard of the subdivision SDK;

4. Components in the ring chart, Android support, Jackson JSON parser and Google Play services are the top three of the three-party library;

5. File type leaderboards, PNG, Dex and ARSC are the top three;

3, combing the optimization objectives

So our goal is no tooth decay, no, is the following goal:

1. PNG image optimization;

2. Optimization of RESOURCES.ARSC files;

3. Code optimization

3.1 Picture optimization attempts

First of all is the first goal, picture optimization, slow, we look at these figures why so big first, exactly, why these figures in the APK (in fact, zip file) so large, well, on tool analysis.

This time with a few simple tool combinations, the system comes with the cmd is good.

The results of the command execution are as follows

Well, all PNG files are actually stored in the store in the apk, about the zip in the store and deflate, see.

Colloquially, when a file is stored stored in a zip, it means that the file is not compressed, and if it is defl:n, it is compressed by deflated normal way to zip storage.

This seems a bit unreasonable, PNG into the zip intact, of course, the final output of the APK will be larger. So, how to solve it? I first tasted the use of Android Gradle plugin way, found that aaptoptions and packagingoptions have failed to solve the problem. Find an open source project on GitHub Andresguard, try integration into the project, and look at the results as follows:

Before optimization:

10536027 bytes

After optimization:

Normal zip Compression: 8786265 bytes (compressed by nearly 17%)

With 7zip compression: 8567150 bytes (nearly 19% compressed)

And look at what this tool did, in contrast, before and after opening up the resource confusion

Before optimization

After optimization

1. Resource (PNG, XML, JPG, etc.) name confusion, resource path name confusion and name length compression;

2. The PNG file that was originally stored in stored format to zip is changed to deflated (normal compressed storage) mode;

3. Accidental discovery of RESOURCES.ARSC, meta-inf/*. SF and meta-inf/*. MF becomes smaller, and the size of the file after decompression is also smaller.

Anti-compilation artifact JADX apk to find the truth

The relative path of the original apk in the resource (PNG, XML, and properties files) is stored in the meta-inf/*. SF and meta-inf/*. MF and calculate the SHA1 value for each resource file and store it in these two files, as for why this and the two SHA1 have what difference and function please refer to the network on this aspect of the knowledge of the article, is beyond the topic of this article so don't repeat it here.

For RESOURCES.ARSC files

It is easy to see that it is a resource file index table, so see here you should understand why these three files will become smaller.

3.2 An unexpected discovery

Look down at RESOURCES.ARSC and find an interesting thing:

This will become an optimization point, remove the useless translation resources, introduce some third-party SDK, often these SDKs with a lot of translation resources inside, such as Android Support Library, we take a look at the effect.

Suppose we only keep English, of course, it's just an experiment.

7zip compression: 8220738 bytes (compressed by nearly 22%, add 3 points)

Of course, the real project is not possible, but the mosquito meat is also meat!

In fact, I want to say that this provides an optimization idea, is to use gradle configuration to kill useless resources, the same can be used on so local library, resolution (Gradle configuration has deprecated) on.

The Gradle configuration example is as follows:

Remember to pack in the middle of android{} oh. So, somebody's got to ask, is there a swelling in the ABI without x86? It is said that Intel provides a solution called Houdini, is a running on the x86 device middleware, you can transcode arm to x86 instructions, but inefficient, some operational type, such as calculation MD5 and SHA1, or even worse than Java, I have done a test comparison, is another topic, here do not repeat, interested readers can be moved.

So far, we are moving towards the first goal, inadvertently discovering the relationship between the first and second goals, so we are using the resource obfuscation tool to achieve the second goal.

With 7zip compression, we have 2-point compression for the entire package, which is a more than expected result.

3.3 Image Optimization Methods

With regard to the first goal, our path is not over yet, and the path of the brain to figure out is to compress PNG, non-alpha to JPG, what else? So go to a variety of technical forum stroll around, ask a variety of technology Daniel, combing the path as follows:

1. Manual lint Check, manually delete the code does not refer to the resources, the actual effect varies.

Open "Analyze" in Android Studio and select "Inspect Code ...", select the entire project, then click "OK"

Configuration such as

2. Open Shrinkresources in Gradle script

The script reference is as follows

Shrinkresources with minifyenabled use better, see shrinkresources usage and attention

7zip compression: 8115283 bytes (compressed by nearly 23%, add one more point)

3. Use the image compression tool, compress the size of the PNG map, convert the non-alpha diagram to JPG form, about this colleague and the network of the Daniel have been sorted out very detailed, I do a brief summary here, for details, please see the Appendix Reference.

• Using Tinypng, I just want to say that we do products in the company, this scheme is used with caution, upload any unpublished product content to the external network, it is possible to cause data leakage, so use this scenario with caution. Here's an alternative.
• Wasted
• Pngquant
• Imagealpha
• Imageoptim
• The above tools are too scattered, there is no integrated tool, the answer is "yes", @ imagemin children's shoes development
• Msimageresourceshelper of the development of children's shoes in brother-in-law
• PNG turns into JPG format with no specific effects.

4. The ultimate big kill, PNG to WEBP, about WEBP, for more details please refer to Google official documents and Android developer online Reference

First on:

7zip compression: 4926912 bytes (compressed by nearly 53%, add 30 points)

Right, it is 30 points, the current size of the APK is not half the size of the original apk, and I do, a line of code wood has changed, only with some tools!

Speaking, I have to eat diet pills, Wood has a hunger strike, weight is half light!!!

However, the project is not currently available because there are two pits

• On some of Samsung's models, some of the alpha background part of the graph will have a very obvious black line, here is not, the problem is currently through the white list of the way not to make WEBP figure to deal with;
• In the Xiaomi 2 4.xx mobile phone, failed to correctly identify the XML file described in the WEBP picture, resulting in the interface to load the XML layout file, file loading WEBP failure, Error said resource file not found, causing the app to crash. Trace discovery is the Millet Machine Agent class resource for Miuiresource, but this miuiresource failed to correctly identify WEBP, so the load resource file failed, preliminary decision, currently no solution, so can only reluctantly abandon this optimization program.

As for the first goal, the optimization of the picture resource is written here.

3.4 Code Optimization

The second goal has been achieved, leaving the third goal, the code optimization, combing the following optimization path:

1. Turn on Proguard code optimization

Will proguardfiles Getdefaultproguardfile (' proguard-android.txt '), ' proguard-project.txt '

Change to Proguardfiles getdefaultproguardfile (' proguard-android-optimize.txt '), ' proguard-project.txt '

Please see appendix for the points of attention when you turn on code optimization.

2. Removal of unused libraries

If the minimum version supported by the APK is API14, and the code does not use an API higher than API14, you can consider taking out the entire Android support library.

3. Use a smaller library replacement scheme

If you only use Google stats, don't integrate the entire Google Play services, just the parts you need.

4. Regular cleanup of obsolete code

Periodically remove useless logic and outdated business function modules, as well as obsolete A/b test code.

5. The business module uses the plug-in framework, the code dynamically pulls from the cloud

Plug-in, this is another topic, here do not repeat.

APK slimming down the final result

10536027 bytes compressed to 4926912 bytes, compressed by nearly 53%

Summary

1. Turn on resource obfuscation and resource compression in scripts

2. Replace zip with 7zip

3. Open code obfuscation optimization and useless resource deletion in Gradle script

4. Use the compression tool to compress the image size with a smaller diagram

5. Removal of useless resources, languages, local so library, two-party tripartite library and resolution

6. Use a smaller library

7. Try to completely kick off your project with Android support Library

8. Regularly clean up your code

9. Try to write the interface with H5, the picture cloud gets

10. Try the plug-in business module

11. Find all files stored in the store in the Zip folder (not limited to the raw directory), try to compress, and load these resources in an alternative scenario

12. Try the WEBP image loading scheme to find a breakthrough

Finally, continue to learn and try new optimization solutions.

This article is dedicated to "only thin body and products can not live up to" the technology!!!

Non-GE @ Ali Poly Security, more technical articles, please click on the Ali Poly Security Blog

APK slimming, how to achieve a compression effect of up to 53%