About Android hotfix Nuwa

About Android hotfix Nuwa

Then I said that Nuwa has a pitfall. Someone may ask what kind of pitfall Nuwa has. This article will summarize the pitfall you have encountered on Nuwa. If you have encountered other pitfalls, please leave a message, I will add them to the article. Of course, some of them are not Nuwa's pitfalls. It is classified as ClassLoader's method for hot fixing and exposing problems.

When there are some pitfalls excludeclassthere is no reference ing.txt, resulting in failure to exclude some classes that do not need to be processed

Without obfuscation, Nuwa has no problems in this aspect, but once obfuscation occurs, some classes are injected instead of bytecode, the reason is that Nuwa processes the mixed jar, and the mixed jar package name and class name have changed, the configured excludeClass cannot be used to actively not inject bytecode, unless you add the obfuscated class name, But before obfuscation, we do not know the name of the obfuscated class. Some people say that I can confuse it first. After obfuscation, check the mapping File and find the corresponding obfuscated class name, it can be added to excludeClass. Don't you think it hurts, and it is very likely that errors may occur. Is there any better way? Of course.

A mapping.txt file will be created in the outputsdirectory. Can We parse this file and restore the obfuscated class to the original class name? The general content of this file is like the following.

android.support.graphics.drawable.AnimatedVectorDrawableCompat$1 -> android.support.a.a.c:    android.support.graphics.drawable.AnimatedVectorDrawableCompat this$0 -> a    629:629:void 
  
   (android.support.graphics.drawable.AnimatedVectorDrawableCompat) -> 
   
        632:633:void invalidateDrawable(android.graphics.drawable.Drawable) -> invalidateDrawable    637:638:void scheduleDrawable(android.graphics.drawable.Drawable,java.lang.Runnable,long) -> scheduleDrawable    642:643:void unscheduleDrawable(android.graphics.drawable.Drawable,java.lang.Runnable) -> unscheduleDrawableandroid.support.graphics.drawable.AnimatedVectorDrawableCompat$AnimatedVectorDrawableCompatState -> android.support.a.a.d:    int mChangingConfigurations -> a    android.support.graphics.drawable.VectorDrawableCompat mVectorDrawable -> b    java.util.ArrayList mAnimators -> c    android.support.v4.util.ArrayMap mTargetNameMap -> d    473:503:void 
    
     (android.content.Context,android.support.graphics.drawable.AnimatedVectorDrawableCompat$AnimatedVectorDrawableCompatState,android.graphics.drawable.Drawable$Callback,android.content.res.Resources) -> 
     
          507:507:android.graphics.drawable.Drawable newDrawable() -> newDrawable    512:512:android.graphics.drawable.Drawable newDrawable(android.content.res.Resources) -> newDrawable    517:517:int getChangingConfigurations() -> getChangingConfigurationsandroid.support.graphics.drawable.AnimatedVectorDrawableCompat$AnimatedVectorDrawableDelegateState -> android.support.a.a.e:    android.graphics.drawable.Drawable$ConstantState mDelegateState -> a    424:426:void 
      
       (android.graphics.drawable.Drawable$ConstantState) -> 
       
         430:434:android.graphics.drawable.Drawable newDrawable() -> newDrawable 439:443:android.graphics.drawable.Drawable newDrawable(android.content.res.Resources) -> newDrawable 448:452:android.graphics.drawable.Drawable newDrawable(android.content.res.Resources,android.content.res.Resources$Theme) -> newDrawable 457:457:boolean canApplyTheme() -> canApplyTheme 462:462:int getChangingConfigurations() -> getChangingConfigurations
       
      
     
    
   
  

The first line is the mixed class name corresponding to the original class name.->Split, followed by the mixed variable name corresponding to the original variable name, or use->Split, but indented with four spaces at the beginning. The last step is obfuscation of methods. The first step is the number of rows of methods.:Split. Separate Two numbers and then use one.:, Followed by the confusing method name corresponding to the original method name, which is also used->Split. Methods and variables are of type. Are you sure you want to parse it? Yes, regular expressions. Don't rush to write code. before writing code, let's see if there is any wheel ready for use. Search proguard on github, no results... Another keyword is retrace. Why is retrace? Because proguard comes with a script called retrace, which can be restored from the obfuscated exception information to the exception information of the original class. The result is displayed. Select java in the code. The last one on the first page is. Here I fork this warehouse to my own warehouse, see address https://github.com/lizhangqu/retrace

Of course, it cannot be completely and directly used. In fact, we use three classes, one of which is ClassMapping. java, one is MethodMapping. java, and Retrace. java, as for how to transform, it's up to you. The source code is in front of you, and you won't transform it yet? After the transformation, the full class name after obfuscation is passed in, and the original full category is returned, so that the comparison with excludeClass can be correctly processed.

The class that has not been modified is patched. Why?

We modified a complicated class and prepared to apply a patch. We found that the patch class is not one and contains a lot of other classes. Why? When fixing the package, use mapping of the formal package to fix the bug, modify the original class, and change the class, but some classes do not change because of the confusion relationship (obfuscation will remove some useless methods, and those useless methods may be added when fixing the package ), this causes some classes to not be modified, but will also appear in the repair package. Of course, the probability of such a situation is quite large, but the number of classes is not necessarily there. There is one more and one more .... How to solve it... There is no solution, so there will be more... The maximum size of the patch package is increased. This situation can only be avoided as much as possible. For example, do not modify the original indentation when repairing the package. Accidentally re-format the code, maybe the original code is not formatted, you have changed the entire class in this format, including the internal class of this class, so that the number of patch classes will increase. Therefore, the code changes should be minimized during the patch process.

Trap 2. Application directly referenced classes cannot be patched

Why does this happen?

. As a result, the Application class is marked with the mark for verification. Then the class directly referenced by the Application cannot be patched, and the exception will be reported after a dozen patches.Class ref in pre-verified class resolved to unexpected implementation

How can this problem be solved?

Directly referenced classes cannot be patched, but indirectly referenced classes can be typed. It is okay to change directly referenced classes to indirect references. How can this problem be solved? Create an intermediate class, such as PatchUtil, which has an init method. The input parameter is Application, and all the logic in the Application is transferred to PatchUtil. Then, the Application references the PatchUtil class for calling, in the end, a class directly referenced by a large volume is converted into an indirect reference, and PatchUtil is converted into a directly referenced class. Therefore, a large volume of classes that cannot be patched become a class that cannot be patched, it is worth it.

Trap 3: bytecode Injection

What is the cause of injection failure (confusion and private constructor )?

If the code is not obfuscated, bytecode injection is okay, so this is still caused by obfuscation. After obfuscation, many classes are gone <init>, or <init> is changed to <clinit>. Why? I guess it is caused by removing useless methods. Another special case is the private constructor. For example, in a singleton case, there is only one private constructor. the bytecode of the private constructor does not include <init> or even more powerful, there is no <clinit>. In this case, injection is not allowed, and the Nuwa logic is to judge whether the name is equal to <init> and at the end of the constructor. However, in actual tests, the bytecode is not <init> or <clinit> in most classes after obfuscation.

How to solve the injection failure problem?

Can I insert a member variable? The actual test result is no... I do not know the specific reason. If there is no constructor, you can insert a constructor to it, but it cannot be displayed. This may also be a problem, for example, there was a private constructor. If you plug in another public constructor, it must be a problem, so you just need to insert a static initialization code for it, reference Hack directly in this Code. class. Just like this

static{    System.out.println(com.package.Hack.class);}

As for how to insert this code... I did not insert data with asm, So I replaced the code used to insert the byte code in Nuwa with the javassist code. For how to insert the code, see the following article.

No error is reported if the injected bytecode cannot be found.

The original byte code injection of Nuwa is to inject such code into the constructor.

System.out.println(Hack.class);

What is the problem with this code? Use your mind to think about it. In case some classes are loading Hack. class was used before, and we accidentally injected this piece of code into it, then the program will run crash immediately, so we can't help but let this piece of code be executed, the answer is yes. You can use an if statement to never enter this if statement. The following is a method. Of course, you can use other similar code.

if(Boolean.FALSE.booleanValue()){    System.out.println(Hack.class)}

In this way, the code will never be executed. Even if a class that should not be used is extracted, the program will not crash. At most, a log is output on the console, indicating that the referenced class cannot be found. The actual test result is that even if the log is reported, the patch can still be applied.

If gradle 1.5 or above is not supported, how can this problem be solved?
Hook Solution
The hook method is the same as 1.2.3, except that the gradle of hook 1.5 is much easier to process than 1.2.3. For details, refer to the implementation of the AndHotFix hook task named transfromClassesWidthDexForRelease or transfromClassesWidthDexForDebug, execute our injection operation before this task. Use the transform api
In addition to hook, you can also use the new gradle1.5 api to solve the problem, that is, the transform interface. For specific implementation, refer to the previous article Android hotfix and use Gradle Plugin1.5 to transform the Nuwa plug-in, this method has a disadvantage. The class we handle is the class before it is obfuscated, after processing the patch, you need to perform a code-level obfuscation operation based on the configuration file and the mapping file in the Code. Of course, this operation is fully automated. You can use the code for obfuscation, the disadvantage is that the internal class of a class is patched. So it is not particularly appropriate, but the hook method is more flexible. Pit 5. How can I prevent illegal patch package tampering if the patch package does not undergo signature verification?
The patch package must be verified on the app side. Therefore, the prerequisite for verification is to sign the patch. How should we sign the patch? Refer to Ctrip's packaging script ghost. The following is my modified script.

public static signedApk(Logger logger, def variant, File apkFile) {    if (!apkFile.exists())        return;    def signingConfigs = variant.getSigningConfig()    if (signingConfigs == null) {        logger.error "no need to sign"        return;    }    def args = [JavaEnvUtils.getJdkExecutable('jarsigner'),                '-verbose',                '-sigalg', 'MD5withRSA',                '-digestalg', 'SHA1',                '-keystore', signingConfigs.storeFile,                '-keypass', signingConfigs.keyPassword,                '-storepass', signingConfigs.storePassword,                apkFile.absolutePath,                signingConfigs.keyAlias]    def proc = args.execute()}public static zipalign(Project project, File apkFile) {    if (apkFile.exists()) {        def sdkDir        Properties properties = new Properties()        File localProps = project.rootProject.file("local.properties")        if (localProps.exists()) {            properties.load(localProps.newDataInputStream())            sdkDir = properties.getProperty("sdk.dir")        } else {            sdkDir = System.getenv("ANDROID_HOME")        }        if (sdkDir) {            def cmdExt = Os.isFamily(Os.FAMILY_WINDOWS) ? '.exe' : ''            File dest = new File("${apkFile.absolutePath}.zipalign");            def argv = []            argv << '-f'    //overwrite existing outfile.zip            // argv << '-z'    //recompress using Zopfli            argv << '-v'    //verbose output            argv << '4'     //alignment in bytes, e.g. '4' provides 32-bit alignment            argv << apkFile.absolutePath            argv << dest.absolutePath  //output            project.exec {                commandLine "${sdkDir}/build-tools/${project.android.buildToolsVersion}/zipalign${cmdExt}"                args argv            }            if (apkFile.exists()) {                apkFile.delete()            }            dest.renameTo(apkFile)        } else {            throw new InvalidUserDataException('$ANDROID_HOME is not defined')        }    }}

Then, the client needs to verify the signature of the current app Based on the patch.

Pit 6. High maintenance cost of ASM bytecode Injection

This is not a pitfall of Nuwa, but Nuwa uses ASM to inject bytecode. the readability of ASM is too poor, and it is difficult for people who do not understand bytecode, therefore, you must improve code readability and reduce maintenance costs.

How to reduce maintenance costs

Replace asm with javassist. Compared with asm, javassist may have a poor performance, but it is definitely user-friendly to developers in terms of readability, because it is written in java code. Next we will demonstrate the Code mentioned before injection.

if(Boolean.FALSE.booleanValue()){    System.out.println(Hack.class)}

ClassPool classPool = ClassPool. getDefault (); // The Hack class is dynamically generated here and inserted into classpatch. Because the byte code generated by javassist depends on this class, the CtClass hackClass = classPool is dynamically generated here. makeClass ("com. lizhangqu. hack. hack ") byte [] hackBytes = hackClass. toBytecode () hackClass. defrost () classPool. insertClassPath (new ByteArrayClassPath ("com. weidian. hack. hack ", hackBytes ))

The original function prototype for Nuwa to inject bytecode is as follows:

private static byte[] referHackWhenInit(InputStream inputStream) {}

The input parameter is InputStream, and the returned value is the byte array of bytecode. We will write this injection function without changing the function prototype.

private    static byte[] referHackByJavassistWhenInit(ClassPool classPool, InputStream inputStream) {        CtClass clazz = classPool.makeClass(inputStream)        CtConstructor ctConstructor = clazz.makeClassInitializer()        ctConstructor.insertAfter("if(Boolean.FALSE.booleanValue()){System.out.println(com.weidian.hack.Hack.class);}")        def bytes = clazz.toBytecode()        clazz.defrost()        return bytes    }

The input parameter has a ClassPool parameter, which is the previous ClassPool and contains the Hack class. The key here is the makeClassInitializer function. This function is used to generate a piece of static initialization code. If it does not exist, a new one will be created. If it exists, a result will be returned, then we insert a bytecode at the end, that is

if(Boolean.FALSE.booleanValue()){System.out.println(com.lizhangqu.hack.Hack.class);}

After the insertion is complete, convert it to a byte array. Remember to call the defrost Method to restore the object. Otherwise, an exception occurs. The final production code is like this.

static{    if(Boolean.FALSE.booleanValue(){        System.out.println(com.lizhangqu.hack.Hack.class);    }}

7. Compatibility of Android versions

What is the compatibility between Android 6.0 and?
In actual situations, I tested three system versions, namely 4.4, 5.0, and 6.0. What are the actual test results? There is no problem in patching the three system versions, the only system that requires special processing may be 6.0. Why is 6.0, because 6.0 has an additional runtime permission application.

Android 6.0 dynamic permission application pitfalls

Why is this a pitfall? During the test, I put the patch in the sdcard root directory for testing. In this case, it corresponds to the 6.0 system, in addition to declaring the read/write sdcard in the manifest file, you also need to dynamically apply for permissions. for users, the read/write sdcard is a dangerous permission and requires the user's active authorization. Therefore, 6.0 of the systems, if your patch is in sdcard, you may need to add code similar to this for permission application.

Int permission = ContextCompat. checkSelfPermission (getApplicationContext (), Manifest. permission. WRITE_EXTERNAL_STORAGE); if (permission! = PackageManager. PERMISSION_GRANTED) {Log. e ("TAG", "unauthorized"); ActivityCompat. requestPermissions (this, new String [] {Manifest. permission. WRITE_EXTERNAL_STORAGE}, 100 );}

After that, the patch can be created normally as long as the user authorizes it.

The above is a brief description of some pitfalls I encountered recently and a simple solution. If you encounter other pitfalls, please leave a message.