Simple implementation of Android incremental upgrade (with source code)

Simple implementation of Android incremental upgrade (with source code)

As mobile phone hardware continues to improve, the installation package for improving resolution is also growing. In the NOKIA and MOTO era, if a mobile APP had 1 MB, It would be big, and 2 MB was no longer enough. Although the network and storage have been greatly improved, the traffic is not cheap enough to go to the APP to change the title to download a program installation package of several megabytes. Today we will introduce the implementation of Android incremental download. Be patient and take a look at the principles first!

Principle of incremental upgrade

Today, we will implement incremental upgrades for similar applications. In fact, the principle of incremental upgrade is very simple, that is, first, the old Apk version of the application is different from the new Apk version to obtain the updated part of the patch, for example, the old APK version has 5 MB, the new version has 8 M, and the updated part may only be around 3 M (it should be noted that the size of the differential package is not a simple subtraction, because it actually needs to include some context-related things), the advantage of using a differential upgrade is obvious, so you don't need to download the entire 8 M file, you just need to download the update part, the updated part may only be 3 or 4 M, which can greatly reduce the traffic loss.
After the user downloads the differential package, they need to be combined on the mobile phone end. You can refer to the method of copying the old software (mostly under/data/) on the mobile phone to the SD card or cache and combining them with the previous differential patch, get a new version of the apk application. If nothing happens, the generated apk is the same as the one you used to make the difference.

Incremental upgrade operations

After understanding the basic principles, we will gradually solve various difficulties. The first is the generation of the patch of the differential package. If the Android mobile phone otaupgrade is completed, the patch folder in update.zip contains files with the same name as the system file but suffixed with xxx. p. They are the generated differential patch files. We can use the differential generation tool for OTA system upgrade to generate the differential patch file for a single application apk. The following command is used to create the OTA system differential package:

./Ota_from_target_files-n-I <旧包> <新包> <差分包名>
According to the ota_from_target_files code, the difference package is generated in the WriteIncrementalOTAPackage function, and the common is created in this function. difference class, we continue to follow up, in the common. class Difference (object): In py, you can see:

Diff_program = DIFF_PROGRAM_BY_EXT.get (ext, "bsdiff ")

Now we can see that the android tool "bsdiff" is used to create a differential incremental update package. This is a very good binary differential tool. introduction to the Code address related to the portal or \ external \ bsdiff is a binary difference tool under the android code directory, the corresponding bspatch is the patch synthesis tool. Note that the incremental update Patch must be completed on the server side, that is, on the PC side. The general process is as follows, call the bsdiff function when creating a patch, and generate a patch file based on two binary files of different versions.

Command: bsdiff oldfile newfile patchfile
Example: bsdiff xx_v1.0.apk xx_v2.0.apk xx. patch

Place the generated patch package xx. patch on the upgrade server for users to download and upgrade. Different versions must be different for different versions. if the version span is large, we recommend that you upgrade the entire package. After downloading the xx. patch package, you need to use the apk corresponding to the patch, that is, the old apk installed in the system and the bspatch tool integrated with the patch. The earlier version of the apk can be obtained through the apk file under the copy System data/app directory, and the bspatch patch synthesis can be encapsulated into a so library by slightly modifying the bspatch source code, for mobile phone calls.

The bspatch command format is:
Bspatch oldfile newfile patchfile

The difference is the same as that of time-sharing. A new apk can be merged for installation. The above is just a simple operation principle. The incremental upgrade also involves many other aspects, such as upgrading patch verification. For details, refer to the bootable \ recovery \ applypatch operations in the android source code, this article is only an analysis, not a table here. Insufficient incremental upgrade is not a perfect upgrade method. There are at least two shortcomings: 1. incremental updates generate patches based on the differences between the two application versions. You cannot guarantee that the updates are updated to the latest version every time, therefore, you must make a difference between each version you release and the latest version so that users of all versions can perform a differential upgrade. This operation is more complicated than the original full-package upgrade, however, it can be generated in batches using automated scripts. 2. the premise for successful incremental upgrade is that the user's mobile phone must have an apk that allows you to copy and is consistent with the version used by your server for the differential version. This will exist, for example, the built-in apk of the system cannot be obtained and incremental upgrades cannot be performed. For some of the apk versions that are consistent with your differential version but have modified the content (such as the cracked apk version), incremental upgrades cannot be performed, to prevent merging patch errors, it is best to verify the sha1sum of the old apk version before merging the patch to ensure the consistency of the basic package. The experiment is useless, and practice is king. The following is a simple practice to check the correctness of the previous theory.

│ ── Bsdiff.exe
│ ── Bspatch.exe
│ ── LICENSE
── Bspatch // mobile testing tool
── OldAPK1.6.2.apk // The old apk version
── NewAPK1.8.0.apk // The New apk version

Run the following command in the shell to enter the test \ bsdiff4.3-win32 Folder:

1	bsdiff.exe oldAPK1.6.2.apk newAPK1.8.0.apk apk.patch

In the original apk (2.94 M) and new version (3.24 M), the resulting patch file is 1.77 M. Users only need to download 1.77 M, saving a lot of traffic. Next we will merge them on the computer.


bspatch.exe oldAPK1.6.2.apk new.apk apk.patch


After the execution, the app named new.apk is merged. This is the same as newapk1.8.0.apk.
Now write an android DEMO and test this tool. Add native support directly when creating an android project, and add the following code to the CPP file:

#include "com_droidupdate_jni_PatchUtil.h"#include "bzlib_private.h"#include "bzlib.h"#include 
 
  #include 
  
   #include 
   
    #include 
    
     #include 
     
      #include 
      
       #include static off_t offtin(u_char *buf) {off_t y;y = buf[7] & 0x7F;y = y * 256;y += buf[6];y = y * 256;y += buf[5];y = y * 256;y += buf[4];y = y * 256;y += buf[3];y = y * 256;y += buf[2];y = y * 256;y += buf[1];y = y * 256;y += buf[0];if (buf[7] & 0x80)y = -y;return y;}int applypatch(int argc, const char* argv[]) {FILE * f, *cpf, *dpf, *epf;BZFILE * cpfbz2, *dpfbz2, *epfbz2;int cbz2err, dbz2err, ebz2err;int fd;ssize_t oldsize, newsize;ssize_t bzctrllen, bzdatalen;u_char header[32], buf[8];u_char *oldStr, *newStr;off_t oldpos, newpos;off_t ctrl[3];off_t lenread;off_t i;if (argc != 4)errx(1, "usage: %s oldfile newfile patchfile\n", argv[0]);/* Open patch file */if ((f = fopen(argv[3], "r")) == NULL)err(1, "fopen(%s)", argv[3]);/* File format: 0 8 "BSDIFF40" 8 8 X 16 8 Y 24 8 sizeof(newfile) 32 X bzip2(control block) 32+X Y bzip2(diff block) 32+X+Y ??? bzip2(extra block) with control block a set of triples (x,y,z) meaning "add x bytes from oldfile to x bytes from the diff block; copy y bytes from the extra block; seek forwards in oldfile by z bytes". *//* Read header */if (fread(header, 1, 32, f) < 32) {if (feof(f))errx(1, "Corrupt patch\n");err(1, "fread(%s)", argv[3]);}/* Check for appropriate magic */if (memcmp(header, "BSDIFF40", 8) != 0)errx(1, "Corrupt patch\n");/* Read lengths from header */bzctrllen = offtin(header + 8);bzdatalen = offtin(header + 16);newsize = offtin(header + 24);if ((bzctrllen < 0) || (bzdatalen < 0) || (newsize < 0))errx(1, "Corrupt patch\n");/* Close patch file and re-open it via libbzip2 at the right places */if (fclose(f))err(1, "fclose(%s)", argv[3]);if ((cpf = fopen(argv[3], "r")) == NULL)err(1, "fopen(%s)", argv[3]);if (fseeko(cpf, 32, SEEK_SET))err(1, "fseeko(%s, %lld)", argv[3], (long long) 32);if ((cpfbz2 = BZ2_bzReadOpen(&cbz2err, cpf, 0, 0, NULL, 0)) == NULL)errx(1, "BZ2_bzReadOpen, bz2err = %d", cbz2err);if ((dpf = fopen(argv[3], "r")) == NULL)err(1, "fopen(%s)", argv[3]);if (fseeko(dpf, 32 + bzctrllen, SEEK_SET))err(1, "fseeko(%s, %lld)", argv[3], (long long) (32 + bzctrllen));if ((dpfbz2 = BZ2_bzReadOpen(&dbz2err, dpf, 0, 0, NULL, 0)) == NULL)errx(1, "BZ2_bzReadOpen, bz2err = %d", dbz2err);if ((epf = fopen(argv[3], "r")) == NULL)err(1, "fopen(%s)", argv[3]);if (fseeko(epf, 32 + bzctrllen + bzdatalen, SEEK_SET))err(1, "fseeko(%s, %lld)", argv[3],(long long) (32 + bzctrllen + bzdatalen));if ((epfbz2 = BZ2_bzReadOpen(&ebz2err, epf, 0, 0, NULL, 0)) == NULL)errx(1, "BZ2_bzReadOpen, bz2err = %d", ebz2err);if (((fd = open(argv[1], O_RDONLY, 0)) < 0)|| ((oldsize = lseek(fd, 0, SEEK_END)) == -1) || ((oldStr =(u_char*) malloc(oldsize + 1)) == NULL)|| (lseek(fd, 0, SEEK_SET) != 0)|| (read(fd, oldStr, oldsize) != oldsize) || (close(fd) == -1))err(1, "%s", argv[1]);if ((newStr = (u_char*) malloc(newsize + 1)) == NULL)err(1, NULL);oldpos = 0;newpos = 0;while (newpos < newsize) {/* Read control data */for (i = 0; i <= 2; i++) {lenread = BZ2_bzRead(&cbz2err, cpfbz2, buf, 8);if ((lenread < 8)|| ((cbz2err != BZ_OK) && (cbz2err != BZ_STREAM_END)))errx(1, "Corrupt patch\n");ctrl[i] = offtin(buf);};/* Sanity-check */if (newpos + ctrl[0] > newsize)errx(1, "Corrupt patch\n");/* Read diff string */lenread = BZ2_bzRead(&dbz2err, dpfbz2, newStr + newpos, ctrl[0]);if ((lenread < ctrl[0])|| ((dbz2err != BZ_OK) && (dbz2err != BZ_STREAM_END)))errx(1, "Corrupt patch\n");/* Add old data to diff string */for (i = 0; i < ctrl[0]; i++)if ((oldpos + i >= 0) && (oldpos + i < oldsize))newStr[newpos + i] += oldStr[oldpos + i];/* Adjust pointers */newpos += ctrl[0];oldpos += ctrl[0];/* Sanity-check */if (newpos + ctrl[1] > newsize)errx(1, "Corrupt patch\n");/* Read extra string */lenread = BZ2_bzRead(&ebz2err, epfbz2, newStr + newpos, ctrl[1]);if ((lenread < ctrl[1])|| ((ebz2err != BZ_OK) && (ebz2err != BZ_STREAM_END)))errx(1, "Corrupt patch\n");/* Adjust pointers */newpos += ctrl[1];oldpos += ctrl[2];};/* Clean up the bzip2 reads */BZ2_bzReadClose(&cbz2err, cpfbz2);BZ2_bzReadClose(&dbz2err, dpfbz2);BZ2_bzReadClose(&ebz2err, epfbz2);if (fclose(cpf) || fclose(dpf) || fclose(epf))err(1, "fclose(%s)", argv[3]);/* Write the new file */if (((fd = open(argv[2], O_CREAT | O_TRUNC | O_WRONLY, 0666)) < 0)|| (write(fd, newStr, newsize) != newsize) || (close(fd) == -1))err(1, "%s", argv[2]);free(newStr);free(oldStr);return 0;}jint JNICALL Java_com_droidupdate_jni_PatchUtil_applyPatchToOldApk(JNIEnv *pEnv,jclass clazz, jstring oldPath, jstring newPath, jstring patchPath) {const char* pOldPath = pEnv->GetStringUTFChars(oldPath, JNI_FALSE);const char* pNewPath = pEnv->GetStringUTFChars(newPath, JNI_FALSE);const char* pPatchPath = pEnv->GetStringUTFChars(patchPath, JNI_FALSE);const char* argv[4];argv[0] = "bspatch";argv[1] = pOldPath;argv[2] = pNewPath;argv[3] = pPatchPath;int ret = -1;ret = applypatch(4, argv);pEnv->ReleaseStringUTFChars(oldPath, pOldPath);pEnv->ReleaseStringUTFChars(newPath, pNewPath);pEnv->ReleaseStringUTFChars(patchPath, pPatchPath);return ret;}
      
     
    
   
  
 

When upgrading the package, make a new package with bsdiff.exe of windows. patch file, and then we will download this file when the program detects the new version. patch file, and then call this JNI function. compare the patch file with the current version to generate an apk file of the latest version, and then use application/vnd. android. package-archive to install it!

The following is the tool and Android test source code. I put it on another blog. You need to directly access the download. Android incremental upgrade details