Android OTA upgrade 2: script ota_from_target_files

Original article: http://blog.csdn.net/zjujoe/article/details/6209393

Android OTA upgrade 2: script ota_from_target_files

Author: Song Lixin

Email: zjujoe@yahoo.com

Preface

We have introduced the compilation process of Ota package. The core part is a Python script: ota_from_target_files. Now let's analyze this script.

First look at help

Without any parameters, let's take a look at its help:

1. $./Ota_from_target_files
3. Given a target-files zipfile, produces an OTA package that instils
5. That build. An incremental OTA is produced if-I is given, otherwise
7. A full OTA is produced.
11. Usage: ota_from_target_files [flags] input_target_files output_ota_package
13. -B (-- board_config) <File>
15. Deprecated.
17. -K (-- package_key) <key>
19. Key to use to sign the package (default is
21. "Build/target/product/security/testkey ").
23. -I (-- incremental_from) <File>
25. Generate an incremental OTA using the given target-files zip
27. The starting build.
29. -W (-- wipe_user_data)
31. Generate an OTA package that will wipe the User Data Partition
33. When installed.
35. -N (-- no_prereq)
37. Omit the timestamp prereq check normally authorized ded at the top
39. The build scripts (used for developer OTA packages which
41. Legitimately need to go back and forth ).
43. -E (-- extra_script) <File>
45. Insert the contents of file at the end of the update script.
47. -M (-- script_mode) <mode>
49. Specify 'Amend' or 'edify 'scripts, or 'auto' to pick
51. Automatically (this is the default ).
53. -P (-- path) <dir>
55. Prepend <dir>/bin to the list of places to search for Binaries
57. Run by this script, and perform CT to find jars in <dir>/framework.
59. -S (-- device_specific) <File>
61. Path to the python module containing device-specific
63. Releasetools code.
65. -X (-- extra) <key = value>
67. Add a key/value pair to the 'extras' dict, which device-specific
69. Extension code may look.
71. -V (-- verbose)
73. Show command lines being executed.
75. -H (-- Help)
77. Display this usage message and exit.

Simple translation:

-B is out of date and is no longer used.

-K key used for signature

-I is used to define the comparison package when an incremental OTA package is generated

-W: whether to clear the userdata Partition

-N: whether or not the instance is being upgradedNoCheck the timestamp. By default, it can only be upgraded based on the old version.

-E: define additional running scripts.

-M is defined in the script format. Currently, there are two types: Amend & edify, where Amend is an old format. Corresponding, different interpreters will be used during the upgrade. By default, ota_from_target_files generates two scripts at the same time. This provides maximum flexibility.

-P defines the path of some executable files used by the script

-S defines the path of the extra running script

-X defines the key/value pairs that may be used by additional running scripts

-V old friend, redundant mode, allows the script to print the executed command

-Old friend h, that's enough.

We call the following command to generate our upgrade package:

 

./Build/tools/releasetools/ota_from_target_files/

-M Auto/

-P out/host/linux-x86/

-K build/target/product/security/testkey-N/

Out/target/product/{product-name}/obj/packaging/target_files_intermediates/Your product-name=-target_files-eng.w.uid=.zip {output_zip}

View content again

Ota_from_target_files is a Python script, so it will be smoother if you understand python.

There are 1000 lines of files. During the analysis, we only paste code snippets. For the complete file, see:

Build/tools/releasetools/ota_from_target_files (from Android 2.2)

 

Entry: Main

According to the python Convention, the Code executed separately starts from _ main:

944 if __name__ == '__main__':

945   try:

946     main(sys.argv[1:])

947   except common.ExternalError, e:

948     print

949     print "   ERROR: %s" % (e,)

950     print

951     sys.exit(1)

It calls the main function:

 

1. 844 def main (argv ):
2. 845
3. 846 def option_handler (O, ):
4. 847 if o in ("-B", "-- board_config "):
5. 848 pass # deprecated
6. 849 Elif o in ("-k", "-- package_key "):
7. 850 options. package_key =
8. 851 Elif o in ("-I", "-- incremental_from "):
9. 852 options. incremental_source =
10. 853 Elif o in ("-W", "-- wipe_user_data "):
11. 854 options. wipe_user_data = true
12. 855 Elif o in ("-n", "-- no_prereq "):
13. 856 options. omit_prereq = true
14. 857 Elif o in ("-e", "-- extra_script "):
15. 858 options. extra_script =
16. 859 Elif o in ("-M", "-- script_mode "):
17. 860 options. script_mode =
18. 861 Elif o in ("-- worker_threads "):
19. 862 options. worker_threads = int ()
20. 863 else:
21. 864 return false
22. 865 return true
23. 866
24. 867 ARGs = Common. parseoptions (argv, _ Doc __,
25. 868 extra_opts = "B: K: I: D: wne: M :",
26. 869 extra_long_opts = ["board_config = ",
27. 870 "package_key = ",
28. 871 "incremental_from = ",
29. 872 "wipe_user_data ",
30. 873 "no_prereq ",
31. 874 "extra_script = ",
32. 875 "script_mode = ",
33. 876 "worker_threads ="],
34. 877 extra_option_handler = option_handler)
35. 878
36. 879 if Len (ARGs )! = 2:
37. 880 common. usage (_ Doc __)
38. 881 SYS. Exit (1)

 

Save the option you set to the options variable. It is a python class. We can understand it as a C struct.

 883   if OPTIONS.script_mode not in ("amend", "edify", "auto"):

884     raise ValueError('unknown script mode "%s"' % (OPTIONS.script_mode,))

Script_mode can only be one of Amend/edify/auto, and auto is currently supported by both options.

It can be understood to be forward compatible (early Android uses Amend)

 886   if OPTIONS.extra_script is not None:

887     OPTIONS.extra_script = open(OPTIONS.extra_script).read()

Read the content of the additional script. (If any)

 889   print "unzipping target target-files..."

890   OPTIONS.input_tmp = common.UnzipTemp(args[0])

Unpack the input package.

 
 

  
  
892   if OPTIONS.device_specific is None:  
  
  
893     # look for the device-specific tools extension location in the input  
  
  
894     try:  
  
  
895       f = open(os.path.join(OPTIONS.input_tmp, "META", "tool-extensions.txt"))  
  
  
896       ds = f.read().strip()  
  
  
897       f.close()  
  
  
898       if ds:  
  
  
899         ds = os.path.normpath(ds)  
  
  
900         print "using device-specific extensions in", ds  
  
  
901         OPTIONS.device_specific = ds  
  
  
902     except IOError, e:  
  
  
903       if e.errno == errno.ENOENT:  
  
  
904         # nothing specified in the file  
  
  
905         pass  
  
  
906       else:  
  
  
907         raise  
 
 
  

It is useless to process device-specific extensions.

 909   common.LoadMaxSizes()

910   if not OPTIONS.max_image_size:

911     print

912     print "  WARNING:  Failed to load max image sizes; will not enforce"

913     print "  image size limits."

914     print

It is useless to read the parameter that sets the image size.

 916   OPTIONS.target_tmp = OPTIONS.input_tmp

917   input_zip = zipfile.ZipFile(args[0], "r")

918   if OPTIONS.package_key:

919     temp_zip_file = tempfile.NamedTemporaryFile()

920     output_zip = zipfile.ZipFile(temp_zip_file, "w",

921                                  compression=zipfile.ZIP_DEFLATED)

922   else:

923     output_zip = zipfile.ZipFile(args[1], "w",

924                  compression=zipfile.ZIP_DEFLATED)

Set the output file. If you want to sign (our case), you need a temporary output file.

 926   if OPTIONS.incremental_source is None:

927     WriteFullOTAPackage(input_zip, output_zip)

928   else:

929     print "unzipping source target-files..."

930     OPTIONS.source_tmp = common.UnzipTemp(OPTIONS.incremental_source)

931     source_zip = zipfile.ZipFile(OPTIONS.incremental_source, "r")

932     WriteIncrementalOTAPackage(input_zip, source_zip, output_zip)

Based on the parameters, call the incremental and non-incremental creation zip creation functions. We use the non-incremental mode.

 934   output_zip.close()

935   if OPTIONS.package_key:

936     SignOutput(temp_zip_file.name, args[1])

937     temp_zip_file.close()

939   common.Cleanup()

941   print "done."

The signature (if needed) is processed.

 

The main function is writefullotapackage.

Main function: writefullotapackage

345 def writefullotapackage (input_zip, output_zip ):

346   if OPTIONS.script_mode == "auto":

347     script = both_generator.BothGenerator(2)

348   elif OPTIONS.script_mode == "amend":

349     script = amend_generator.AmendGenerator()

350   else:

351     # TODO: how to determine this?  We don't know what version it will

352     # be installed on top of.  For now, we expect the API just won't

353     # change very often.

354     script = edify_generator.EdifyGenerator(2)

First, we obtain the script generator. for their implementation, see the script: edify_generator.py.

 356   metadata = {"post-build": GetBuildProp("ro.build.fingerprint", input_zip),

357               "pre-device": GetBuildProp("ro.product.device", input_zip),

358               "post-timestamp": GetBuildProp("ro.build.date.utc", input_zip),

359               }

Obtain some environment variables from the android environment variables. Google knows its meaning.

 361   device_specific = common.DeviceSpecificParams(

362       input_zip=input_zip,

363       input_version=GetRecoveryAPIVersion(input_zip),

364       output_zip=output_zip,

365       script=script,

366       input_tmp=OPTIONS.input_tmp,

367       metadata=metadata)

Device-related parameters are not described in detail.

 369   if not OPTIONS.omit_prereq:

370     ts = GetBuildProp("ro.build.date.utc", input_zip)

371     script.AssertOlderBuild(ts)

If necessary, add an Assert statement to the script and require that the update zip package be used only to upgrade the old system.

 373   AppendAssertions(script, input_zip)

If necessary, add an Assert statement to the script, requiring that the update zip package be used only on the same device, that is, the Ro. Product. device packages of the target device are the same as those in update.zip.

 

374   device_specific.FullOTA_Assertions()

Callback is used to call device-related code. The call time is about to start upgrading. Similar:

Fullota_installend incrementalota_assertions incrementalota_verifyend. Do not go into details.

 376   script.ShowProgress(0.5, 0)

In the upgrade script, add a statement that shows the progress. Parameter 1 indicates the ratio of the current operation time to the overall time (to the next similar statement or to the end. Parameter 2 is used to control the display speed. For example, 50 indicates that the operation is estimated to be completed within 50 seconds, and the progress bar is required to display the progress of this part in 50 seconds. 0 indicates that the update is not automatically updated, and manual control (using setprogress)

 378   if OPTIONS.wipe_user_data:

379     script.FormatPartition("userdata")

If necessary, add a statement to the script and erase the userdata partition.

 381   script.FormatPartition("system")

Add a statement in the script to erase the system partition.

 382   script.Mount("MTD", "system", "/system")

Add the statement in the script and install the system partition to the/system directory.

383   script.UnpackPackageDir("recovery", "/system")

384   script.UnpackPackageDir("system", "/system")

Add a statement in the script to copy the recovery and content in the system to the/system directory. The recovery directory contains a patch and the script for applying the patch.

 386   symlinks = CopySystemFiles(input_zip, output_zip)

387   script.MakeSymlinks(symlinks)

Line 3 copies the file from the input zip package/system to the output zip package/system. Because this process does not support linking files, it returns these files. Continue processing on Row 3. This row creates these link files. All link files point to toolbox

 389   boot_img = File("boot.img", common.BuildBootableImage(

390       os.path.join(OPTIONS.input_tmp, "BOOT")))

391   recovery_img = File("recovery.img", common.BuildBootableImage(

392       os.path.join(OPTIONS.input_tmp, "RECOVERY")))

393   MakeRecoveryPatch(output_zip, recovery_img, boot_img)

This is complicated. makerecoverypatch does two things:

1. Generate a patch in the output zip package: recovery/recovery-from-boot.p (patch for boot. IMG and recovery. IMG), which is located at: System/recovery-from-boot.p

2. Generate a script in the output zip package: recovery/etc/install-recovery.sh, which is located at system/etc/install-recovery.sh.

The script content is:

#!/system/bin/sh

if ! applypatch -c MTD:recovery:2048:6a167ffb86a4a16cb993473ce0726a3067163fc1; then

  log -t recovery "Installing new recovery image"

  applypatch MTD:boot:2324480:9a72a20a9c2f958ba586a840ed773cf8f5244183 MTD:recovery f6c2a70c5f2b02b6a49c9f5c5507a45a42e2d389 2564096 9a72a20a9c2f958ba586a840ed773cf8f5244183:/system/recovery-from-boot.p

else

  log -t recovery "Recovery image already installed"

fi

 395   Item.GetMetadata(input_zip)

Obtain the permission information of each file in the system directory from Meta/filesystem_config.txt.

 396   Item.Get("system").SetPermissions(script)

Add a statement to the script and set the file permissions and owner in the system directory.

 398   common.CheckSize(boot_img.data, "boot.img")

Check whether the file size of boot. IMG exceeds the standard.

 399   common.ZipWriteStr(output_zip, "boot.img", boot_img.data)

Put boot. IMG in the output zip package.

 400   script.ShowProgress(0.2, 0)

402   script.ShowProgress(0.2, 10)

Update the progress bar.

 403   script.WriteRawImage("boot", "boot.img")

Add a statement in the script to write boot. IMG to the boot partition.

 405   script.ShowProgress(0.1, 0)

Update the progress bar.

 406   device_specific.FullOTA_InstallEnd()

Callback, same as before.

 408   if OPTIONS.extra_script is not None:

409     script.AppendExtra(OPTIONS.extra_script)

If there are additional scripts, add them.

 411   script.UnmountAll()

Add the statement in the script and all umount partitions.

 412   script.AddToZip(input_zip, output_zip)

1) output the previously generated script to: META-INF/COM/Google/Android/Updater-script (for edify)

 
 
 

  
  
assert(getprop("ro.product.device") == "thedevicename" ||  
  
  
  
  
  
       getprop("ro.build.product") == "theproductname");  
  
  
  
  
  
show_progress(0.500000, 0);  
  
  
  
  
  
format("MTD", "system");  
  
  
  
  
  
mount("MTD", "system", "/system");  
  
  
  
  
  
package_extract_dir("recovery", "/system");  
  
  
  
  
  
package_extract_dir("system", "/system");  
  
  
  
  
  
symlink("dumpstate", "/system/bin/dumpcrash");  
  
  
  
  
  
symlink("toolbox", "/system/bin/cat", "/system/bin/chmod",  
  
  
  
  
  
        "/system/bin/chown", "/system/bin/cmp", "/system/bin/date",  
  
  
  
  
  
        "/system/bin/dd", "/system/bin/df", "/system/bin/dmesg",  
  
  
  
  
  
        "/system/bin/fb2bmp", "/system/bin/getevent", "/system/bin/getprop",  
  
  
  
  
  
        "/system/bin/hd", "/system/bin/id", "/system/bin/ifconfig",  
  
  
  
  
  
        "/system/bin/iftop", "/system/bin/insmod", "/system/bin/ioctl",  
  
  
  
  
  
        "/system/bin/kill", "/system/bin/ln", "/system/bin/log",  
  
  
  
  
  
        "/system/bin/ls", "/system/bin/lsmod", "/system/bin/mkdir",  
  
  
  
  
  
        "/system/bin/mount", "/system/bin/mv", "/system/bin/netstat",  
  
  
  
  
  
        "/system/bin/newfs_msdos", "/system/bin/notify", "/system/bin/printenv",  
  
  
  
  
  
        "/system/bin/ps", "/system/bin/reboot", "/system/bin/renice",  
  
  
  
  
  
        "/system/bin/rm", "/system/bin/rmdir", "/system/bin/rmmod",  
  
  
  
  
  
        "/system/bin/route", "/system/bin/schedtop", "/system/bin/sendevent",  
  
  
  
  
  
        "/system/bin/setconsole", "/system/bin/setprop", "/system/bin/sleep",  
  
  
  
  
  
        "/system/bin/smd", "/system/bin/start", "/system/bin/stop",  
  
  
  
  
  
        "/system/bin/sync", "/system/bin/top", "/system/bin/umount",  
  
  
  
  
  
        "/system/bin/vmstat", "/system/bin/watchprops",  
  
  
  
  
  
        "/system/bin/wipe");  
  
  
  
  
  
set_perm_recursive(0, 0, 0755, 0644, "/system");  
  
  
  
  
  
set_perm_recursive(0, 2000, 0755, 0755, "/system/bin");  
  
  
  
  
  
set_perm(0, 3003, 02755, "/system/bin/netcfg");  
  
  
  
  
  
set_perm(0, 3004, 02755, "/system/bin/ping");  
  
  
  
  
  
set_perm_recursive(1002, 1002, 0755, 0440, "/system/etc/bluez");  
  
  
  
  
  
set_perm(0, 0, 0755, "/system/etc/bluez");  
  
  
  
  
  
set_perm(1002, 1002, 0440, "/system/etc/dbus.conf");  
  
  
  
  
  
set_perm(1014, 2000, 0550, "/system/etc/dhcpcd/dhcpcd-run-hooks");  
  
  
  
  
  
set_perm(0, 2000, 0550, "/system/etc/init.goldfish.sh");  
  
  
  
  
  
set_perm(0, 0, 0544, "/system/etc/install-recovery.sh");  
  
  
  
  
  
set_perm_recursive(0, 0, 0755, 0555, "/system/etc/ppp");  
  
  
  
  
  
set_perm_recursive(0, 2000, 0755, 0755, "/system/xbin");  
  
  
  
  
  
show_progress(0.200000, 0);  
  
  
  
  
  
show_progress(0.200000, 10);  
  
  
  
  
  
assert(package_extract_file("boot.img", "/tmp/boot.img"),  
  
  
  
  
  
       write_raw_image("/tmp/boot.img", "boot"),  
  
  
  
  
  
       delete("/tmp/boot.img"));  
  
  
  
  
  
show_progress(0.100000, 0);  
  
  
  
  
  
unmount("/system");  
 
 
  

2) copy the update program OTA/bin/Updater from the input zip package to the: META-INF/COM/Google/Android/update-binary in the output zip package

 413   WriteMetadata(metadata, output_zip)

Write the obtained metadata to the file in the output package: META-INF/COM/Android/metadata

In this case, we have to get an update.zip package. You can start upgrading.

Thoughts

1) Although the recovery partition update mechanism is provided, the statement that triggers the update is not displayed. Therefore, the rediscovery partition is not updated by default. Probably for security reasons. However, sometimes you need to update the recovery partition (for example, the hardware configuration and partition table of the device are changed). How can this problem be solved?