Android Incremental Upgrade simple implementation (attached source)

As the mobile hardware continues to improve, the resolution of the mobile phone installation package is also getting bigger. When the Nokia,moto era, a mobile phone app if there is 1MB that is big, 2MB is very much. Although the network, storage has been greatly improved, but the traffic is not cheap to the app changed a title to download a few megabytes of the program installation package. The implementation of the Android incremental download is described today. Have patience to see the principle first, the following practice!

The principle of incremental upgrade

Today we are going to implement an incremental upgrade of similar applications. In fact, the principle of incremental upgrade is very simple, that is, the application of the old version of the APK and the new version of the APK differential, to get the updated part of the patch, such as the old version of the APK has 5 m, the version of the 8 m, the updated part may be only about 3 m (it is necessary to note that the resulting difference Because of the need to include some context-sensitive things, the advantage of using differential upgrade is obvious, then you do not need to download the full 8M file, only need to download the update section, and the update part may be only 3, 4M, can greatly reduce the loss of traffic.
After the user has downloaded the differential packets, they need to be combined on the phone side. You can refer to the old version of the software (mostly under the/data/), copied to the SD card or the cache, and they are combined with the previous differential patch to get a new version of the APK app, if not unexpected, This generated apk is consistent with the apk that you did before to make the difference.

Operations for incremental upgrades

After understanding the basic principles, we will gradually solve each of the difficulties. The first is the generation of differential packet patches. If you've ever done an Android phone OTA upgrade, you should be aware that in the patch folder in Update.zip There are files that need to have the same name as the system files, but with the xxx.p suffix, they are the resulting differential patch files. We can use the OTA system upgrade differential generation tool to generate a differential patch file for our single app apk. The production of the OTA system differential packet, using the command:

./ota_from_target_files-n-i < old package > < new Package > < differential package name >
The code in the lookup ota_from_target_files shows that the differential packet is generated in the function writeincrementalotapackage, and the common is created inside the function. Difference This class, we continue to follow, in common.py class Difference (object): You can see:

Diff_program = Diff_program_by_ext.get (EXT, "Bsdiff")

So far we've seen the tools available in Android that we used to make differential delta upgrade packages, "Bsdiff", a very bull X Open source binary differential tool. Related to the portal related code address or in the Android code directory \external\bsdiff Bsdiff is a binary differential tool, the corresponding Bspatch is the corresponding patch synthesis tool needs to be noted is the incremental upgrade of the patch package, is required on the server side, that is, the PC-side completion, the approximate process, such as making patches when the call Bsdiff function, based on two different versions of the binary files, generate patch files.

Command: Bsdiff oldfile newfile patchfile
Example: Bsdiff xx_v1.0.apk xx_v2.0.apk Xx.patch

The resulting patch pack xx.patch placed on the upgrade server for users to download the upgrade, the corresponding multi-version needs to be different versions, for a larger version span, it is recommended that the whole package upgrade. After downloading the Xx.patch patch package, the user needs to use the patch's APK, the old version of the original system installed APK and the Bspatch tool of the patch synthesis. The old version of the system apk can be obtained through the copy system Data/app directory apk file, and the patch synthesis of the Bspatch can be modified by the Bspatch source code, encapsulated into a so library for mobile phone side call.

The Bspatch command format is:
Bspatch oldfile NewFile Patchfile

Is the same as the parameter at differential. The new apk can be used for installation. The above is just a simple operation principle, the incremental upgrade also involves many other aspects, for example, upgrade patch check and other issues, you can refer to the Android source code in the relevant operation Bootable\recovery\applypatch, this article is only an analysis, this is not a table. Insufficient incremental upgrade is not a perfect upgrade method, at least the following two points are insufficient: 1. Incremental upgrades are generated by the difference between two app versions, and you can't guarantee that users will be up to date each time, so you have to differentiate between each version you publish and the latest version. So that all versions of the user can have a differential upgrade, which is cumbersome compared to the original whole package upgrade, but can be generated in batches with automated scripts. 2. The success of the incremental upgrade is based on the premise that the user's mobile phone must have an APK that allows you to copy it and use the same version as your server for the differential, so that it exists, for example, that the system's built-in APK cannot be acquired, cannot be incrementally upgraded, and that the content has been modified ( such as the cracked version of the APK), this is not an incremental upgrade, in order to prevent the composite patch error, it is best to pre-patch the old version of the apk before the Sha1sum check, to ensure the consistency of the base package. The small experiment says useless, the practice is the kingly way. Here is a simple practice to test the correctness of the theory before.

├──bsdiff-4.3//bsdiff Source code path, the official website obtains
│├──bsdiff.1
│├──bsdiff.c
│├──bspatch.1
│├──bspatch.c
│└──makefile
├──bsdiff-4.3.tar.gz
├──bsdiff4.3-win32//windows pc-side test Tool
│├──binary Diff.txt
│├──bsdiff.exe
│├──bspatch.exe
│└──license
├──bspatch//test tools on the mobile side
├──OLDAPK1.6.2.APK//older version of APK
└──NEWAPK1.8.0.APK//new version of APK

APK to do the test, in the shell into the Test\bsdiff4.3-win32 folder, and run the command:

1	bsdiff.exe oldAPK1.6.2.apk newAPK1.8.0.apk apk.patch

The original apk (2.94M), the new version of (3.24M), get the patch file is 1.77M, users need to download just 1.77M, the traffic saved a lot. Let's merge them on the computer side.


bspatch.exe oldAPK1.6.2.apk new.apk apk.patch


After execution, get a composite version of the app named new.apk. This is the same as our newapk1.8.0.apk.
Now write a little Android demo and test this tool. Add native support directly when creating an Android project, add the following code to the CPP file

#include "com_droidupdate_jni_patchutil.h" #include "bzlib_private.h" #include "bzlib.h" #include <stdlib.h># Include <stdio.h> #include <string.h> #include <err.h> #include <unistd.h> #include <fcntl.h > #include <android/log.h>static off_t offtin (U_char *buf) {off_t y;y = buf[7] & 0x7f;y = y * 256;y + = buf[6]; y = y * 256;y + = Buf[5];y = y * 256;y + = Buf[4];y = y * 256;y + = Buf[3];y = y * 256;y + = Buf[2];y = y * 256;y + = Buf[1];y = y * 256;y + = Buf[0];if (buf[7] & 0x80) y =-y;return y;} int applypatch (int argc, const char* argv[]) {FILE * F, *CPF, *DPF, *EPF; Bzfile * cpfbz2, *dpfbz2, *epfbz2;int Cbz2err, Dbz2err, Ebz2err;int fd;ssize_t oldsize, newsize;ssize_t Bzctrllen, Bzdatal  En;u_char header[32], Buf[8];u_char *oldstr, *newstr;off_t oldpos, newpos;off_t ctrl[3];off_t lenread;off_t i;if (argc! = 4) Errx (1, "Usage:%s oldfile newfile patchfile\n", argv[0]);/* Open patch file */if ((f = fopen (Argv[3], "r")) = = NULL) er R (1, "fopen (%s)", argv[3]);/*File format:0 8 "BSDIFF40" 8 8 x 8 y 8 sizeof (newfile) x bzip2 (control block) 32+x Y bzip2 (diff block) 32+x+y??? Bzip2 (extra block) with control block a set of triples (x, y, z) meaning "add x bytes from Oldfile to x bytes from the diff Block Copy y bytes from the extra block; Seek forwards in Oldfile by Z bytes ". *//* Read Header */if (fread (header, 1, F) < +) {if (feof (f)) Errx (1, "corrupt patch\n"); Err (1, "Fread (%s)", argv[3 ]);} /* Check for Appropriate magic */if (memcmp (header, "BSDIFF40", 8)! = 0) Errx (1, "corrupt patch\n");/* Read lengths from He  Ader */bzctrllen = Offtin (header + 8), Bzdatalen = Offtin (header + +), newsize = Offtin (header +), if ((Bzctrllen < 0) || (Bzdatalen < 0) | | (NewSize < 0)) Errx (1, "corrupt patch\n");/* Close patch file and re-open it via libbzip2 at the right places */if (fclose (f)) Err (1, "FCL OSE (%s) ", argv[3]), if (CPF = fopen (argv[3)," r ") = = NULL) Err (1," fopen (%s) ", argv[3]); if (Fseeko (CPF, +, Seek_set) Err (1, "Fseeko (%s,%lld)", Argv[3], (long Long), if (cpfbz2 = Bz2_bzreadopen (&cbz2err, CPF, 0, 0, NULL, 0)) = = N ULL) Errx (1, "bz2_bzreadopen, Bz2err =%d", cbz2err), if (DPF = fopen (Argv[3], "r")) = = NULL) Err (1, "fopen (%s)", argv[3]); F (Fseeko (DPF, + Bzctrllen, Seek_set)) Err (1, "Fseeko (%s,%lld)", Argv[3], (Long long) (+ + Bzctrllen)); if ((dpfbz2 = B Z2_bzreadopen (&dbz2err, DPF, 0, 0, NULL, 0)) = = null) Errx (1, "bz2_bzreadopen, Bz2err =%d", dbz2err); if (EPF = fopen ( ARGV[3], "r")) = = NULL) Err (1, "fopen (%s)", argv[3]), if (Fseeko (EPF, + Bzctrllen + Bzdatalen, Seek_set)) Err (1, "Fseeko" (% S,%lld) ", Argv[3], (Long Long) (+ Bzctrllen + bzdatalen)); if (epfbz2 = Bz2_bzreadopen (&ebz2err, EPF, 0, 0, NULL, 0 ) = = NULL) Errx (1, "bz2_bzreadopen, Bz2err =%d", ebz2err); if ((FD = open (argv[1], o_rdonly, 0)) < 0) | | ((oldsize = Lseek (FD, 0, seek_end)) = =-1) | | ((Oldstr = (u_char*) malloc (oldsize + 1)) = = NULL) | | (Lseek (FD, 0, seek_set)! = 0) | | (Read (FD, OLDSTR, oldsize)! =Oldsize) | | (Close (FD) = =-1)) Err (1, "%s", argv[1]), if ((Newstr = (u_char*) malloc (newsize + 1)) = = NULL) Err (1, NULL); oldpos = 0;newpos = 0;while (newpo  s < newsize) {/* Read control Data */for (i = 0; I <= 2; i++) {lenread = Bz2_bzread (&cbz2err, cpfbz2, buf, 8); if ((Lenread < 8) | | ((Cbz2err! = BZ_OK) && (cbz2err! = bz_stream_end))) Errx (1, "corrupt patch\n"); Ctrl[i] = Offtin (BUF);};/ * Sanity-check */if (newpos + ctrl[0] > NewSize) errx (1, "corrupt patch\n");/* Read diff string */lenread = Bz2_bzread (& Amp;dbz2err, dpfbz2, Newstr + Newpos, ctrl[0]); if (Lenread < ctrl[0]) | | ((Dbz2err! = BZ_OK) && (dbz2err! = bz_stream_end))) Errx (1, "corrupt patch\n");/* ADD old data to diff string */for (i = 0; i < ctrl[0]; i++) if ((Oldpos + i >= 0) & & (Oldpos + i < oldsize)) Newstr[newpos + i] + = Oldstr[oldpos + i];/* Adjust pointers */newpos + = Ctrl[0];oldpos + = ctrl[0];/* Sanity-check */if (Newpos + ctrl[1] > NewSize) errx (1, "corrupt patch\n");/* ReaD Extra String */lenread = Bz2_bzread (&ebz2err, epfbz2, Newstr + Newpos, ctrl[1]); if ((Lenread < ctrl[1]) | | ((Ebz2err! = BZ_OK) && (ebz2err! = bz_stream_end))) Errx (1, "corrupt patch\n");/* Adjust pointers */newpos + = Ctrl[1];oldpos + = ctrl[2];};/ * Clean up the bzip2 reads */bz2_bzreadclose (&cbz2err, cpfbz2); Bz2_bzreadclose (&dbz2err, dpfbz2); Bz2_bzreadclose (&ebz2err, epfbz2); if (fclose (CPF) | | fclose (DPF) | | fclose (EPF)) Err (1, "fclose (%s)", argv[3]);/* Write the new file */if ((fd = open (argv[2], o_creat | O_trunc | O_wronly, 0666)) < 0) | | (Write (FD, NEWSTR, newsize)! = newsize) | | (Close (FD) = =-1)) Err (1, "%s", argv[2]); free (newstr); free (oldstr); return 0;} Jint jnicall java_com_droidupdate_jni_patchutil_applypatchtooldapk (jnienv *penv,jclass clazz, jstring OldPath, Jstring NewPath, jstring patchpath) {const char* Poldpath = Penv->getstringutfchars (OldPath, JNI_FALSE); Const char* PN Ewpath = Penv->getstringutfchars (NewPath, jni_false); Const char* PPATCHpath = Penv->getstringutfchars (Patchpath, jni_false); const char* Argv[4];argv[0] = "Bspatch"; argv[1] = POldPath; ARGV[2] = pnewpath;argv[3] = ppatchpath;int ret = -1;ret = Applypatch (4, argv);p env->releasestringutfchars (OldPath, PO Ldpath);p env->releasestringutfchars (NewPath, Pnewpath);p env->releasestringutfchars (PatchPath, PPatchPath); return ret;}

When you need to publish the upgrade package, make a new pack of Windows Bsdiff.exe. patch file, and then we'll download it when the program detects a new version. A. patch file, and then call this JNI function to generate a latest version of the APK file from the. patch file and the current version comparison, and then install it via Application/vnd.android.package-archive!

Here is the tool and Android test source I put it on one of my other blogs. Direct access download is required.

It 100,000 why? Android Incremental Upgrade detailed

Android Incremental Upgrade simple implementation (attached source)