Analysis on incremental upgrade of android applications

I haven't updated my blog for a long time. It's a fallen phenomenon. I tried to write a blog several times, but I was always delayed by all kinds of things. I will write more articles to record it later.
Background
As the size of android applications increases and application versions change, user upgrades become a problem. google is also aware of the loss of user traffic caused by constantly updating applications, the Smart App update mentioned on Google I/O, that is, the incremental update or differential update of an application, is supported in the new version of Google Play, chatting with a group of friends one day is a topic in this area. I just studied it with curiosity.
Principle of incremental upgrade
Today, we will implement incremental upgrades for similar applications. In fact, the principle of incremental upgrade is very simple, that is, first, the old Apk version of the application is different from the new Apk version to obtain the updated part of the patch, for example, the old APK version has 5 MB, the new version has 8 M, and the updated part may only be around 3 M (it should be noted that the size of the differential package is not a simple subtraction, because it actually needs to include some context-related things), the advantage of using a differential upgrade is obvious, so you don't need to download the entire 8 M file, you just need to download the update part, the updated part may only be 3 or 4 M, which can greatly reduce the traffic loss.

After the user downloads the differential package, they need to be combined on the mobile phone end. You can refer to the method of copying the old software (mostly under/data/) on the mobile phone to the SD card or cache and combining them with the previous differential patch, get a new version of the apk application. If nothing happens, the generated apk is the same as the one you used to make the difference.
Incremental upgrade operations
After understanding the basic principles, we will gradually solve various difficulties. The first is the generation of the patch of the differential package. If the Android mobile phone otaupgrade is completed, the patch folder in update.zip contains files with the same name as the system file but suffixed with xxx. p. They are the generated differential patch files. We can use the differential generation tool for OTA system upgrade to generate the differential patch file for a single application apk.
The following command is used to create the OTA system differential package:
[Java]
./Build/tools/releasetools/ota_from_target_files-n-I <old package> <new package> <differential package name>

./Build/tools/releasetools/ota_from_target_files-n-I <old package> <new package> <differential package name>

Small lab
Practice is king. The following is a simple practice to check the correctness of the previous theory. Download the lab package (http://download.csdn.net/detail/by317966834/5260384), unzip the file as follows
[Html]
├ ── Bsdiff-4.3 // bsdiff source code path, the official website to get
│ ── Bsdiff.1
│ ── Bsdiff. c
│ ── Bspatch.1
│ ── Bspatch. c
│ ── Makefile
── Bsdiff-4.3.tar.gz
── Bsdiff4.3-win32 // test tool for windows PC
│ ── Binary diff.txt
│ ── Bsdiff.exe
│ ── Bspatch.exe
│ ── LICENSE
── Bspatch // mobile testing tool
── Ireader1.6.2.0(v352.16.apk // The old apk version
── IReader1.8.0.1(v40).apk // The New apk version

Run the following command in the shell to enter the test \ bsdiff4.3-win32 Folder:
[Html]
Bsdiff.exe ../ireader1.6.2.0(v352.16.apk ../ireader1.8.0.1(v402.16.apk ../ireader. patch
In the original apk (2.94 M) and new version (3.24 M), the resulting patch file is 1.77 M. Users only need to download 1.77 M, saving a lot of traffic.
Next we will merge them on the computer.
[Html] view plaincopybspatch.exe ../ireader1.6.2.0(v3520..apk ../new.apk ../ireader. patch
After the execution, the app named new.apk is merged. I verify it under ubuntu and we can see that they are the same.

Category and ireader. patch, push through adb to the directory you have permission to operate, preferably under/sdcard/, and then set the bspatch execution permission, repeat the above command, you can combine the new version of apk, install and view the verification version later.
Additional reading
The difference upgrade using bsdiff is not the optimal method yet. google optimized the difference algorithm in its Chromium project. The optimized version is called huagua Courgette, it is said that performance optimization is not an order of magnitude. An official example is as follows:
Full update 10,385,920
Bsdiff update 704,512
Courgette update 78,848
You can study it.
I am a little busy recently. I will try to encapsulate the incremental upgrade later and compile the Merged Code into a lib library for java to call. If you are interested in shoes, perform the following operations on your own ~~~

 

Supplement:
Many people do not know how to encapsulate it as lib. In fact, this is the same as the general android C library. If I don't understand jni-related knowledge, I can't find the original test project, below is a simple example for your reference.

[Java]
# Include <stdio. h>
# Include "com_hm25_newstart_bspatch.h"
 
# Include <bzlib. h>
# Include <stdlib. h>
# Include <stdio. h>
# Include <string. h>
# Include <err. h>
# Include <unistd. h>
# Include <fcntl. h>
# Include <android/log. h>
 
 
Static off_t offtin (u_char * buf)
{
Off_t y;
 
Y = buf [7] & 0x7F;
Y = y * 256; y + = buf [6];
Y = y * 256; y + = buf [5];
Y = y * 256; y + = buf [4];
Y = y * 256; y + = buf [3];
Y = y * 256; y + = buf [2];
Y = y * 256; y + = buf [1];
Y = y * 256; y + = buf [0];
 
If (buf [7] & 0x80) y =-y;
 
Return y;
}
 
Int applypatch (int argc, char * argv [])
{
FILE * f, * cpf, * dpf, * epf;
BZFILE * cpfbz2, * dpfbz2, * epfbz2;
Int cbz2err, dbz2err, ebz2err;
Int fd;
Ssize_t oldsize, newsize;
Ssize_t bzctrllen, bzdatalen;
U_char header [32], buf [8];
U_char * old, * new;
Off_t oldpos, newpos;
Off_t ctrl [3];
Off_t lenread;
Off_t I;
 
If (argc! = 4) errx (1, "usage: % s oldfile newfile patchfile \ n", argv [0]);
 
/* Open patch file */
If (f = fopen (argv [3], "r") = NULL)
Err (1, "fopen (% s)", argv [3]);
 
/*
File format:
0 8 "BSDIFF40"
8 8 X
16 8 Y
24 8 sizeof (newfile)
32 X bzip2 (control block)
32 + x y bzip2 (diff block)
32 + X + Y ??? Bzip2 (extra block)
With control block a set of triples (x, y, z) meaning "add x bytes
From oldfile to x bytes from the diff block; copy y bytes from
Extra block; seek forwards in oldfile by z bytes ".
*/
 
/* Read header */
If (fread (header, 1, 32, f) <32 ){
If (feof (f ))
Errx (1, "Upload upt patch \ n ");
Err (1, "fread (% s)", argv [3]);
}
 
/* Check for appropriate magic */
If (memcmp (header, "BSDIFF40", 8 )! = 0)
Errx (1, "Upload upt patch \ n ");
 
/* Read lengths from header */
Bzctrllen = offtin (header + 8 );
Bzdatalen = offtin (header + 16 );
Newsize = offtin (header + 24 );
If (bzctrllen <0) | (bzdatalen <0) | (newsize <0 ))
Errx (1, "Upload upt patch \ n ");
 
/* Close patch file and re-open it via libbzip2 at the right places */
If (fclose (f ))
Err (1, "fclose (% s)", argv [3]);
If (cpf = fopen (argv [3], "r") = NULL)
Err (1, "fopen (% s)", argv [3]);
If (fseeko (cpf, 32, SEEK_SET ))
Err (1, "fseeko (% s, % lld)", argv [3],
(Long) 32 );
If (cpfbz2 = BZ2_bzReadOpen (& cbz2err, cpf, 0, 0, NULL, 0) = NULL)
Errx (1, "BZ2_bzReadOpen, bz2err = % d", cbz2err );
If (dpf = fopen (argv [3], "r") = NULL)
Err (1, "fopen (% s)", argv [3]);
If (fseeko (dpf, 32 + bzctrllen, SEEK_SET ))
Err (1, "fseeko (% s, % lld)", argv [3],
(Long) (32 + bzctrllen ));
If (dpfbz2 = BZ2_bzReadOpen (& dbz2err, dpf, 0, 0, NULL, 0) = NULL)
Errx (1, "BZ2_bzReadOpen, bz2err = % d", dbz2err );
If (epf = fopen (argv [3], "r") = NULL)
Err (1, "fopen (% s)", argv [3]);
If (fseeko (epf, 32 + bzctrllen + bzdatalen, SEEK_SET ))
Err (1, "fseeko (% s, % lld)", argv [3],
(Long) (32 + bzctrllen + bzdatalen ));
If (epfbz2 = BZ2_bzReadOpen (& ebz2err, epf, 0, 0, NULL, 0) = NULL)
Errx (1, "BZ2_bzReadOpen, bz2err = % d", ebz2err );
 
If (fd = open (argv [1], O_RDONLY, 0) <0) |
(Oldsize = lseek (fd, 0, SEEK_END) =-1) |
(Old = malloc (oldsize + 1) = NULL) |
(Lseek (fd, 0, SEEK_SET )! = 0) |
(Read (fd, old, oldsize )! = Oldsize) |
(Close (fd) =-1) err (1, "% s", argv [1]);
If (new = malloc (newsize + 1) = NULL) err (1, NULL );
 
Oldpos = 0; newpos = 0;
While (newpos <newsize ){
/* Read control data */
For (I = 0; I <= 2; I ++ ){
Lenread = BZ2_bzRead (& cbz2err, cpfbz2, buf, 8 );
If (lenread <8) | (cbz2err! = BZ_ OK )&&
(Cbz2err! = BZ_STREAM_END )))
Errx (1, "Upload upt patch \ n ");
Ctrl [I] = offtin (buf );
};
 
/* Sanity-check */
If (newpos + ctrl [0]> newsize)
Errx (1, "Upload upt patch \ n ");
 
/* Read diff string */
Lenread = BZ2_bzRead (& dbz2err, dpfbz2, new + newpos, ctrl [0]);
If (lenread <ctrl [0]) |
(Dbz2err! = BZ_ OK) & (dbz2err! = BZ_STREAM_END )))
Errx (1, "Upload upt patch \ n ");
 
/* Add old data to diff string */
For (I = 0; I <ctrl [0]; I ++)
If (oldpos + I> = 0) & (oldpos + I <oldsize ))
New [newpos + I] + = old [oldpos + I];
 
/* Adjust pointers */
Newpos + = ctrl [0];
Oldpos + = ctrl [0];
 
/* Sanity-check */
If (newpos + ctrl [1]> newsize)
Errx (1, "Upload upt patch \ n ");
 
/* Read extra string */
Lenread = BZ2_bzRead (& ebz2err, epfbz2, new + newpos, ctrl [1]);
If (lenread <ctrl [1]) |
(Ebz2err! = BZ_ OK) & (ebz2err! = BZ_STREAM_END )))
Errx (1, "Upload upt patch \ n ");
 
/* Adjust pointers */
Newpos + = ctrl [1];
Oldpos + = ctrl [2];
};
 
/* Clean up the bzip2 reads */
BZ2_bzReadClose (& cbz2err, cpfbz2 );
BZ2_bzReadClose (& dbz2err, dpfbz2 );
BZ2_bzReadClose (& ebz2err, epfbz2 );
If (fclose (cpf) | fclose (dpf) | fclose (epf ))
Err (1, "fclose (% s)", argv [3]);
 
/* Write the new file */
If (fd = open (argv [2], O_CREAT | O_TRUNC | O_WRONLY, 0666) <0) |
(Write (fd, new, newsize )! = Newsize) | (close (fd) =-1 ))
Err (1, "% s", argv [2]);
 
Free (new );
Free (old );
 
Return 0;
}
 
JNIEXPORT jint JNICALL java_com_hm25_newstart_bspatch_applypatch (JNIEnv * env,
Jobject obj, jstring old, jstring new, jstring patch ){
Int argc = 4;
Char * argv [argc];
Argv [0] = "bspatch ";
Argv [1] = (* env)-> GetStringUTFChars (env, old, 0 );
Argv [2] = (* env)-> GetStringUTFChars (env, new, 0 );
Argv [3] = (* env)-> GetStringUTFChars (env, patch, 0 );

Int ret = applypatch (argc, argv );

(* Env)-> ReleaseStringUTFChars (env, old, argv [1]);
(* Env)-> ReleaseStringUTFChars (env, new, argv [2]);
(* Env)-> ReleaseStringUTFChars (env, patch, argv [3]);
Return ret;
}
Android. mk:
[Java]
LOCAL_PATH: = $ (call my-dir)
Include $ (CLEAR_VARS)
 
# This is the target being built.
LOCAL_MODULE: = libbspatch
 
 
# All of the source files that we will compile.
LOCAL_SRC_FILES: = \
Com_hm25_newstart_bspatch.c
 
 
# No static libraries.
LOCAL_STATIC_LIBRARIES: = \
Libbz
 
 
# Also need the JNI headers.
LOCAL_C_INCLUDES + = \
$ (JNI_H_INCLUDE) external/bzip2
 
# No special compiler flags.
LOCAL_CFLAGS + =
 
Include $ (BUILD_SHARED_LIBRARY)