A bloody case triggered by ProGuard in Android 4.0

Case reproduction:
Modify the Setting in the Android 4.0 source code. After adding a function, compile it in eng mode. Everything works, and then submit the code to the server. The next day, there was a bad news. The newly added functions of Setting could not be used, and an error was reported when one click was made.

Case Analysis:
I have compiled and tested the code locally before uploading the code. Why ??!! Take care of it, and run adb logcat to capture logs for analysis. If you don't know what to do, the error message displayed in the log is ClassNotFound, that is, the corresponding class is not found, is it possible that I missed it during the upload )? Check the submission record immediately. Check submitted files one by one. All files are submitted! Why does ClassNotFound appear? What should we do at this time? After analysis, we can know that the local test is usually compiled in the eng mode, and this error is reported when the server is compiled in the usr mode. Are the results compiled by eng and usr inconsistent ??!!

Further case analysis:
Select the usr compilation mode locally and compile the code for testing. The error is the same as that of the server, that is, ClassNotFound.

1. Since the compiled code in eng mode can run properly, it proves that the Code logic and functions are OK.

2. The ClassNotFound problem occurs when usr is compiled, which proves that the compilation results are different due to the different compilation modes of usr and eng.

There is indeed a class missing in the class (which will be replaced by lostClass. java later ).

Case Investigation:
Hypothesis 1:
Isn't lostClass. java compiled in the compilation process?

Verification 1:
Check whether lostClass. java is compiled during compilation.

1. First check method.
Open the Android. mk file in the Settings folder in the source code and we can find the following code:

LOCAL_SRC_FILES: = $ (call all-java-files-under, src)

Add the following content after the Code:

$ (Warning SevenTest LOCAL_SRC_FILES: $ (LOCAL_SRC_FILES ))

After saving the file, compile the file on the terminal. We can see that the src file information is output on the terminal. Copy to the file and find lostClass. java. You can find it, so you can overturn the 4 hypothesis.

2. method 2.
View the APK compilation and packaging process. 1:

 

Based on the above process, we can find the mk file corresponding to each step. Here we find AndroidSouceCode/build/core/definitons. mk to comment out the following code:

[Plain]
Define compile-java
$ (Hide) rm-f $ @
$ (Hide) rm-rf $ (PRIVATE_CLASS_INTERMEDIATES_DIR)
$ (Hide) mkdir-p $ (dir $ @)
$ (Hide) mkdir-p $ (PRIVATE_CLASS_INTERMEDIATES_DIR)
$ (Call unzip-jar-files, $ (PRIVATE_STATIC_JAVA_LIBRARIES), $ (PRIVATE_CLASS_INTERMEDIATES_DIR ))
$ (Call dump-words-to-file, $ (PRIVATE_JAVA_SOURCES), $ (PRIVATE_CLASS_INTERMEDIATES_DIR)/java-source-list)
$ (Hide) if [-d "$ (PRIVATE_SOURCE_INTERMEDIATES_DIR)"]; then \
Find $ (PRIVATE_SOURCE_INTERMEDIATES_DIR)-name '*. Java' >$ (PRIVATE_CLASS_INTERMEDIATES_DIR)/java-source-list ;\
Fi
$ (Hide) tr ''' \ n' <$ (PRIVATE_CLASS_INTERMEDIATES_DIR)/java-source-list \
| Sort-u >$ (PRIVATE_CLASS_INTERMEDIATES_DIR)/java-source-list-uniq
$ (Hide) $ (1)-encoding UTF-8 \
$ (Strip $ (PRIVATE_JAVAC_DEBUG_FLAGS ))\
$ (If $ (findstring true, $ (LOCAL_WARNINGS_ENABLE), $ (xlint_unchecked ),)\
$(2 )\
$ (Addprefix-classpath, $ (strip \
$ (Call normalize-path-list, $ (PRIVATE_ALL_JAVA_LIBRARIES ))))\
$ (If $ (findstring true, $ (LOCAL_WARNINGS_ENABLE), $ (xlint_unchecked ),)\
-Extdirs ""-d $ (PRIVATE_CLASS_INTERMEDIATES_DIR )\
$ (PRIVATE_JAVACFLAGS )\
\@$ (PRIVATE_CLASS_INTERMEDIATES_DIR)/java-source-list-uniq \
| (Rm-rf $ (PRIVATE_CLASS_INTERMEDIATES_DIR); exit 41)
# Seven Annotation
# $ (Hide) rm-f $ (PRIVATE_CLASS_INTERMEDIATES_DIR)/java-source-list
# $ (Hide) rm-f $ (PRIVATE_CLASS_INTERMEDIATES_DIR)/java-source-list-uniq
$ (Hide) jar $ (if $ (strip $ (PRIVATE_JAR_MANIFEST),-cfm,-cf )\
$ @ $ (PRIVATE_JAR_MANIFEST)-C $ (PRIVATE_CLASS_INTERMEDIATES_DIR ).
Endef
 
Define transform-java-to-classes.jar
@ Echo "target Java: $ (PRIVATE_MODULE) ($ (PRIVATE_CLASS_INTERMEDIATES_DIR ))"
$ (Call compile-java, $ (TARGET_JAVAC), $ (PRIVATE_BOOTCLASSPATH ))
# Seven Annotation
# $ (Hide) rm-rf $ (PRIVATE_CLASS_INTERMEDIATES_DIR)
Endef

Define compile-java
$ (Hide) rm-f $ @
$ (Hide) rm-rf $ (PRIVATE_CLASS_INTERMEDIATES_DIR)
$ (Hide) mkdir-p $ (dir $ @)
$ (Hide) mkdir-p $ (PRIVATE_CLASS_INTERMEDIATES_DIR)
$ (Call unzip-jar-files, $ (PRIVATE_STATIC_JAVA_LIBRARIES), $ (PRIVATE_CLASS_INTERMEDIATES_DIR ))
$ (Call dump-words-to-file, $ (PRIVATE_JAVA_SOURCES), $ (PRIVATE_CLASS_INTERMEDIATES_DIR)/java-source-list)
$ (Hide) if [-d "$ (PRIVATE_SOURCE_INTERMEDIATES_DIR)"]; then \
Find $ (PRIVATE_SOURCE_INTERMEDIATES_DIR)-name '*. Java' >$ (PRIVATE_CLASS_INTERMEDIATES_DIR)/java-source-list ;\
Fi
$ (Hide) tr ''' \ n' <$ (PRIVATE_CLASS_INTERMEDIATES_DIR)/java-source-list \
| Sort-u >$ (PRIVATE_CLASS_INTERMEDIATES_DIR)/java-source-list-uniq
$ (Hide) $ (1)-encoding UTF-8 \
$ (Strip $ (PRIVATE_JAVAC_DEBUG_FLAGS ))\
$ (If $ (findstring true, $ (LOCAL_WARNINGS_ENABLE), $ (xlint_unchecked ),)\
$(2 )\
$ (Addprefix-classpath, $ (strip \
$ (Call normalize-path-list, $ (PRIVATE_ALL_JAVA_LIBRARIES ))))\
$ (If $ (findstring true, $ (LOCAL_WARNINGS_ENABLE), $ (xlint_unchecked ),)\
-Extdirs ""-d $ (PRIVATE_CLASS_INTERMEDIATES_DIR )\
$ (PRIVATE_JAVACFLAGS )\
\@$ (PRIVATE_CLASS_INTERMEDIATES_DIR)/java-source-list-uniq \
| (Rm-rf $ (PRIVATE_CLASS_INTERMEDIATES_DIR); exit 41)
# Seven Annotation
# $ (Hide) rm-f $ (PRIVATE_CLASS_INTERMEDIATES_DIR)/java-source-list
# $ (Hide) rm-f $ (PRIVATE_CLASS_INTERMEDIATES_DIR)/java-source-list-uniq
$ (Hide) jar $ (if $ (strip $ (PRIVATE_JAR_MANIFEST),-cfm,-cf )\
$ @ $ (PRIVATE_JAR_MANIFEST)-C $ (PRIVATE_CLASS_INTERMEDIATES_DIR ).
Endef

Define transform-java-to-classes.jar
@ Echo "target Java: $ (PRIVATE_MODULE) ($ (PRIVATE_CLASS_INTERMEDIATES_DIR ))"
$ (Call compile-java, $ (TARGET_JAVAC), $ (PRIVATE_BOOTCLASSPATH ))
# Seven Annotation
# $ (Hide) rm-rf $ (PRIVATE_CLASS_INTERMEDIATES_DIR)
Endef
After executing the compilation command on the terminal, we can find the following two files in the path: AndroidSourceCode/out/target/common/obj/APPS/Settings_intermediates/classes:

Java-source-list and java-source-list-uniq can be found by searching for lostClass. java.

You can see from the above: lostClass. java is compiled. for verification, we can use the jd-gui tool to view AndroidSourceCode/out/target/common/obj/APPS/Settings_intermediates/classes. jar. Check that the jar file contains the lostClass. java.

Hypothesis 2:
Settings.apk encountered an exception during packaging.

Verification 2:
After comparing the output information compiled in eng mode and usr mode, we find that the dex files of the two copies are different. As follows:

Usr mode:

Copying: out/target/common/obj/APPS/Settings_intermediates/proguard. classes. dex

Eng mode:

Copying: out/target/common/obj/APPS/Settings_intermediates/noproguard. classes. dex

Through comparison, we can know that proguard. classes. dex is smaller than noproguard. classes. dex. The proguard here is obfuscation. Also, we can say that the usrpattern implements the code, And the settings.apk compiled by usrpattern is different from the size compiled by eng mode. Will ProGuard the code in usr mode cause problems? I do not know whether Hypothesis 2 is correct or not.

Suppose 3:
An exception occurs when the compilation system performs ProGuard on the code in usr mode.

Verification 3:
First, let's take a look at the ProGuard tool;

ProGuard introduction:

ProGuard is a free java class File compression, optimization, obfuscator. It detects and deletes unused classes, fields, methods, and attributes. It deletes useless instructions and uses bytecode for maximum optimization. It uses meaningless names to rename classes, fields, and methods.

Role of ProGuard:
1. Create compact code, fast loading, and smaller memory usage.

2. Increase the difficulty of decompiling libraries used by programs and programs.

3. Delete the code from the source file that is not called.

After learning about the role of ProGuard, let's look back at lostClass. java. This class is mainly called in Settings_heads.xml, but it is not called in other ways. Is this optimized by ProGuard for unused code?

Open the Android. mk file in the source code Settings and you can see the following code:

LOCAL_PROGUARD_FLAG_FILES: = proguard. flags

Continue to view proguard. flags in the current path. The Code is as follows:

[Plain]
# Keep all Fragments in this package, which are used by reflection.
-Keep class com. android. settings. * Fragment
-Keep class com. android. settings. * Picker
-Keep class com. android. settings. * Settings
-Keep class com. android. settings. wifi. * Settings
-Keep class com. android. settings. deviceinfo .*
-Keep class com. android. settings. bluetooth .*
-Keep class com. android. settings. applications .*
-Keep class com. android. settings. inputmethod .*
-Keep class com. android. settings. MasterClear
-Keep class com. android. settings. MasterClearConfirm
-Keep class com. android. settings. accounts .*
-Keep class com. android. settings. fuelgauge .*

# Keep all Fragments in this package, which are used by reflection.
-Keep class com. android. settings. * Fragment
-Keep class com. android. settings. * Picker
-Keep class com. android. settings. * Settings
-Keep class com. android. settings. wifi. * Settings
-Keep class com. android. settings. deviceinfo .*
-Keep class com. android. settings. bluetooth .*
-Keep class com. android. settings. applications .*
-Keep class com. android. settings. inputmethod .*
-Keep class com. android. settings. MasterClear
-Keep class com. android. settings. MasterClearConfirm
-Keep class com. android. settings. accounts .*
-Keep class com. android. settings. fuelgauge .*
Check the ProGuard syntax to find out which classes will not be fully processed by ProGuard. Add the following code:

-Keep class com. android. settings. xxxx .*

Xxxx indicates the package name of lostClass. java.

Go to the terminal again and switch to usr mode to compile the code. The test is successful, but there are several small functions that still cannot be used normally. The above at least proves that assumption 3 is correct, that is, the problem arising during the ProGuard process.

These small functions do not have a common feature. When you click a Button, an error is reported, indicating that the corresponding method cannot be found. These buttons Add the onClick Method to the xml file, for example:

[Html]
<Button
Android: id = "@ + id/JumpButton"
Android: onClick = "JumpFuction"
Android: layout_width = "0dip"
Android: layout_height = "wrap_content"
Android: layout_weight = "1"/>

<Button
Android: id = "@ + id/JumpButton"
Android: onClick = "JumpFuction"
Android: layout_width = "0dip"
Android: layout_height = "wrap_content"
Android: layout_weight = "1"/> you can view Example -- A complete Android Application on the official ProGuard website. You also need to add the following code in proguard. flags:

-Keepclassmembers class * extends android. content. Context {
Public void * (android. view. View );
Public void * (android. view. MenuItem );
} The explanation provided on the official website is: We're also keeping possible onClick handlers in custom Context extensions, since they might be referenced from XML layout files.

That is to say, some onClick events are defined in xml. Therefore, add the above content to the proguard. flags file to ensure that these onClick trigger methods are not optimized by ProGuard. Add the above Code, compile, test, and pass !!

Case Review:
The entire case seems confusing, but the entire APK compilation process is hidden behind it. Something seemingly impossible at the beginning (the compilation results of usr and eng modes are inconsistent). After a step-by-step Trace, I found the "shocking secret" behind it ". ProGuard is a free open-source project used to compress, optimize, and confuse code. The Android source code integrates proguard, Which is enabled in usr and usrdebug modes by default, this can be found in AndroidSouceCode4.0/build/core/package. mk:

Ifndef LOCAL_PROGUARD_ENABLED
Ifneq ($ (filter user userdebug, $ (TARGET_BUILD_VARIANT )),)
# Turn on Proguard by default for user & userdebug build
LOCAL_PROGUARD_ENABLED: = full
Endif
Endif

(PS: Qualcomm's source code is similar to google's native code, while MTK commented out the LOCAL_PROGUARD_ENABLED: = full, that is, the results compiled by usr, usrdebug, and eng modes are not obfuscated .)

Of course, in addition to ProGuard, there is also a DexGuard that can be used to optimize and protect Android applications, but this product is charged (the cheapest is 350 €... The conversion speed is nearly 3000 faster than RMB... O (random □random) o ).

At last, the truth was revealed in the bright autumn of an adult.