"Go" Android compilation system details (iii)--the compilation process

Original URL: http://www.cloudchou.com/android/post-276.html

The original author of this article: Cloud Chou. Welcome reprint, please indicate the source and this article link 1. Overview

The third step in compiling Android is to compile with the MKA command, and of course we can use MAKE–J4, but we recommend using the MKA command. Because MKA will automatically calculate the number of-j options, let's not dwell on how much this number is (which is actually the number of cores for all CPUs). At compile time we can take the goal that we need to compile, if you want to build recovery, then use Mka recoveryimage, if you want to generate OTA package, you need to use Mka otapackage, followed by all the available targets. Also note that there are some goals that just play the role of retouching, that is, the need to work with other targets, a total of 4 pseudo-targets for retouching:

• 1) showcommands shows the commands used during compilation
• 2) Incrementaljavac for incremental compilation of Java code
• 3) Checkbuild for testing the modules that need to be inspected
• 4) All if you compile the target using the all modifier, all modules will be compiled

The most frustrating thing about Android compiling systems is the variables, hundreds of variables we can't remember what they mean, and we don't know what those variables are. For this purpose I made a reference site for compiling variables android.cloudchou.com, you can find the variable in the site, it can tell you the meaning of the variable, but also give you a sample value of the variable, but also explained in detail the compiler system each makefile role, so you look at the code of the compilation system not a Fog

The core files for compiling are build/core/main.mk and Build/core/makefile, Main.mk's main role is to check whether the compilation environment meets the requirements, determine the product configuration, determine the modules that the product needs to use, and define a number of targets for developers to use, such as DROID,SDK, but the rules that generate these goals are defined primarily in makefile, and the kernel's compilation rules are placed in the build /core/task/kernel.mk

We'll start with an overall introduction to the Main.mk execution process, and then analyze the key code used to compile the default target on Linux. Makefile mainly defines the generation rules for each goal, so it is no longer detailed about its execution process, and if interested in the generation rules for each goal, you can view the http://android.cloudchou.com/build/core/Makefile.php

2. MAIN.MK execution Process 2.1 verifying the compilation environment and establishing the product configuration

• 1) Set shell variable to bash, cannot use other shell
• 2) Close the Make suffix rule, rcs/sccs the rule, and set a rule: when a rule fails, delete all targets
• 3) Verify make version, Cygwin can use any version of make, but Linux or Mac can only use version 3.81 or version 3.82
• 4) Set Pwd,top,topdir,build_system variables, define the default target variable, but temporarily do not define the default target generation rule
• 5) contains build/core/help.mk, the makefile defines two goals for help and out, helps for display assistance, out for verifying that the compilation system is correct
• 6) contains build/core/config.mk,config.mk made a lot of configuration, including product configuration, include the makefile, will establish the output directory series of variables, but also set up the product series variables, the subsequent introduction of products configuration, this will be more detailed introduction
• 7) contains build/core/ CLEANBUILD.MK, the makefile will contain all the cleanspec.mk of the project, written cleanspec.mk the project will define a special cleanup step before each compilation, CLEANBUILD.MK will perform these cleanup steps
• 8) Check the compilation environment, first detect the last compilation results, if the last verified version and the version of the test is consistent, then no longer detect, and then detect and write the results of the compilation

2.2 includes other makefile and compile target detection

• 1) If the target contains Incrementaljavac, then compile the target with incremental Javac for incremental compilation
• 2) Set the value of the emma_instrument variable, Emma is the library used to test code coverage
• 3) contains build/core/definistions.mk, the makefile defines many auxiliary functions
• 4) contains build/core/qcom_utils.mk, which defines some auxiliary functions and macros for the high-pass board.
• 5) contains build/core/dex_preopt.mk, which defines some macros that optimize the DEX code makefile
• 6) Detect whether there is User,userdebug,eng in the compilation target, if any, tell the user to place in buildspec.mk or use lunch settings, detect the target_build_variant variable, see if it is valid
• 7) includes Build/core/pdk_config.mk, PDK is mainly able to improve the existing equipment upgrade capability, to help equipment manufacturers to adapt to the new version of Android faster

2.3 Setting up the configuration according to Target_build_variant

• 1) If the compilation target has SDK,WIN_SDK or Sdk_addon, then set Is_sdk_build to True
• 2) If Have_selinux is defined, add properties to build prop at compile time ro.build.selinux=1
• 3) If the target_build_variant is user or userdebug, then tags_to_install + = Debug If the user does not define disable_dexpreopt to True, and is User mode, Then set with_dexpreopt: = True, this option will open the APK pre-optimization, the APK into Odex code files and apk resource file
• 4) Determine the enable_target_debugging variable, which is true by default, and it is false when Build_variant is the user. If the value of this variable is true, the compilation property of the set ROM is ro.debuggable to 1, otherwise the set ro.debuggable is 0
• 5) If Target_build_variant is Eng, then Tags_to_install is Debug,eng, and set the compilation property of Rom Ro.setupwizard.mode to optional, because ENG mode does not want to install the wizard
• 6) If Target_build_variant is tests, then tags_to_install: = Debug ENG Tests
• 7) Set SDK related variables
• 8) Add some additional compilation properties
• 9) Define SHOULD-INSTALL-TO-SYSTEM macro function
• 10) If no target is defined in addition to the target, the default target will be used to compile

2.4 Makefile that contains all modules to be compiled

If the compilation target is clean clobber installclean Dataclean, then the set Dont_bother is true, and if Dont_bother is false, all modules to be compiled are included in the

1) If the host operating system and architecture are DARWIN-PPC (MAC), then the prompt does not support compiling the SDK and setting Sdk_only to True

2) If the host operating system is windows, set Sdk_only to True

3) Depending on whether Sdk_only is true, compile the host OS type, build_tiny_android value, set the Sudbidrs variable

4) store all product_* related variables to the Stash_product_vars variable, and later verify that it is modified

5) based on whether the value of One_shot_makefile is empty and contains different makefile

6) Perform Post_clean steps and ensure that the product-related variables are not changed

7) Check if a file is added to All_prebuilt

8) include other makefile that must be included after all ANDROID.MK are included

9) Convert the known_custom_modules into an installation path to get the variable custom_modules

10) Define a dependency between modules, $ (all_modules.$ (m). REQUIRED)) variables indicate dependencies between modules

11) Calculate the values of the following variables: Product_modules,debug_modules,eng_modules,tests_modules,modules_to_install,overridden_packages, Target_gnu_modules,all_default_installed_modules

12) contains Build/core/makefile

13) Define the variable Modules_to_check

2.5 Defining multiple targets

This section defines a number of goals, Prebuilt,all_copied_headers,files,checkbuild,ramdisk,factory_ramdisk,factory_bundle,systemtarball, Boottarball,userdataimage,userdatatarball,cacheimage,bootimage,droidcore,dist_files,apps_only,all_modules,docs , sdk,lintall,samplecode,findbugs,clean,modules,showcommands,nothing.

Subsequent articles will list all available targets

3 execution process when compiling the default target

Before describing the execution process of compiling the default target, let's introduce the variables of the All_ series, otherwise it is difficult to understand the source of these variables when looking at the code, these variables are built after all the modules are included, and each module has a corresponding makefile for compiling. These makefile will contain a compilation type corresponding to the makefile, such as PACKAGE.MK, and these makefile will eventually contain base_rules.mk, in BASE_RULES.MK will add values for all series variables. All of these variables and their origins can be viewed in detail in android.cloudchou.com:

• 1) All_docs full path of all documents, All_docs assignment in droiddoc.mk, All_docs + + $ (full_target)
• 2) all_modules The simple name set of all the modules of the system, the compilation system also defines two other variables for each module, all_modules.$ (Local_module). BUILT The Generation Path all_modules.$ (local_module) for all modules. Installed the respective installation paths for all modules, see Http://android.cloudchou.com/build/core/definitions.php#ALL_MODULES for details.
• 3) All_default_installed_modules all the default modules to be installed, set in Build/core/main.mk and Build/core/makfile
• 4) All_module_tags use all tag sets defined by local_module_tags, each tag corresponds to a all_module_tags. Variable, see http://android.cloudchou.com/for details Build/core/definitions.php#all_module_tags
• 5) All_module_name_tags is similar to All_module_tags, but its value is the name of all modules of a tag, see http://android.cloudchou.com/build/core/ Definitions.php#all_module_name_tags
• 6) All_host_installed_files installed on the PC of the program collection
• 7) All_prebuilt will be copied the set of full paths of precompiled files installed
• 8) All_generated_sources A collection of source code files generated by some tools, such as Aidl generates Java source files
• 9) All_c_cpp_etc_objects all asm,c,c++, and the full path of the C code files generated by Lex and YACC
• All_original_dynamic_binaries is not optimized, and there is no compressed dynamic link library
• All_sdk_files will be placed in the SDK file
• All_findbugs_files all the XML files used by the FINDBUGS program
• License file for the All_gpl_module_license_files GPL module
• android_resource_generated_classes) The type of the compiled class of Java code generated by the ANDROID resource file

3.1 Key Codes

The code that defines the default target is located in Main.mk:

123	. Phony:droiddefault_goal: = droid$ (default_goal):

The goals that the droid targets depend on are:

1234567

Ifneq ($ (Target_build_apps),) ... droid:apps_only #如果编译app, then the droid relies on apps_only target Else......droid:droidcore dist_files # The default depends on the Droidcore target and the dist_files target endif

Dist_files target relies primarily on tools for packaging, all of which add dependencies with Dist-for-goals macros:

1234567891011

$ (call Dist-for-goals, Dist_files, $ (emma_meta_zip)) system/core/mkbootimg/android.mk$ (call Dist-for-goals, Dist_ Files, $ (local_built_module)) system/core/cpio/android.mk:13:$ (Call dist-for-goals,dist_files,$ (LOCAL_BUILT_ MODULE)) system/core/adb/android.mk:88:$ (call Dist-for-goals,dist_files sdk,$ (local_built_module)) system/core/ fastboot/android.mk:68:$ (Call Dist-for-goals,dist_files sdk,$ (local_built_module)) external/guava/android.mk:26:$ (Call Dist-for-goals, Dist_files, $ (local_built_module): Guava.jar) external/yaffs2/android.mk:28:$ (call Dist-for-goals, Dist_files, $ (local_built_module)) external/mp4parser/android.mk:26:$ (call Dist-for-goals, Dist_ Files, $ (local_built_module): Mp4parser.jar) external/jsr305/android.mk:25:$ (call Dist-for-goals, Dist_files, $ ( Local_built_module): Jsr305.jar) frameworks/support/renderscript/v8/android.mk:29:#$ (call Dist-for-goals, Dist_ Files, $ (local_built_module): Volley.jar) frameworks/support/volley/android.mk:29:#$ (call Dist-for-goals, Dist_ Files, $ (local_bUilt_module): Volley.jar)

Let's look at the goals that Droidcore targets depend on:

1234567891011121314151617181920212223

droidcore:files systemimage \ #system. img$ (installed_bootimage_target) \ # boot.img$ (installed_recoveryimage_target) \ #recovery. img$ (installed_userdataimage_target) \ #data. img$ (INSTALLED_ Cacheimage_target) \ #cache. img$ (installed_files_file) # INSTALLED-FILES.TXTIFNEQ ($ (Target_build_apps),) .... else$ ( Call Dist-for-goals, Droidcore, $ (internal_update_package_target) #cm_find5-img-eng.cloud.zip $ (internal_ota_packag E_target) \ # Cm_find5-ota-eng.cloud.zip $ (symbols_zip) \ # Cm_find5-symbols-eng.cloud.zip $ (INSTALLED_FILES_FILE) \ # Installed-files.txt $ (installed_build_prop_target) \# System/build.prop $ (built_target_files_package) \# cm_find5- Target_files-eng.cloud.zip $ (installed_android_info_txt_target) \# Android-info.txt $ (INSTALLED_RAMDISK_TARGET) \# R Amdisk.img $ (installed_factory_ramdisk_target) \# factory_ramdisk.gz $ (installed_factory_bundle_target) \# cm_find5- Factory_bundle-eng.cloud.zip) endif

System.img, Boot.img, Recovery.img, Data.img,cache.img,installed_files.txt's generation rules are defined in makefile, in/HTTP You can see the detailed build rules in android.cloudchou.com/build/core/makefile.php and look at the goals that the files target depends on:

123	Files:prebuilt $ (modules_to_install) $ (installed_android_info_txt_target)

Prebuilt Target relies on $ (all_prebuilt), Android-info.txt's generation rules are defined in Target/board/board.mk, while $ (modules_to_install) The target is a collection of all the modules to be installed, the calculations are complex, and now, as an example of compiling the default target under Linux, the code involved is organized as follows:

1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950 5152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989910010110210310410510610 7108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147 1481491501511521531541551561571581591601611621631641651661671681691701711721731741751761771781791801811821831841851861871 8818919019119219319419519619719819920020120220320420520620720820921021121221321421521621721821922022122222322422522622722 8229

... Tags_to_install: =ifneq (, $ (user_variant)) ..... Ifeq ($ (user_variant), userdebug) Tags_to_install + = Debug Else ... endifendififeq ($ (target_build_variant), Eng) tags_to_i Nstall: = Debug Eng...endififeq ($ (target_build_variant), tests) Tags_to_install: = Debug Eng testsendififdef is_sdk_ Buildtags_to_install: = Debug Engelse #!sdkendif......#------------------------------------------------------------# Define a function that, given a list of module tags, returns# non-empty if the module should be installed In/system.&nbs p;# For more goals, anything not tagged with the ' tests ' tag should# be installed In/system.define should-install-to-syst Em$ (if $ (filter tests,$ (1)), true) Endefifdef is_sdk_build# for the SDK goal, anything with the ' samples ' tag should be# in Stalled In/data even if that module also have "eng"/"Debug"/"User". Define should-install-to-system$ (if $ (filter samples Te sts,$ (1)), true) Endefendif ... #接下来根据配置计算要查找的subdirs目录ifneq ($ (Dont_bother), True) ... ifeq ($ (sdk_only), true) include $ (toPdir) Sdk/build/sdk_only_whitelist.mkinclude $ (topdir) development/build/sdk_only_whitelist.mk# Exclude TOOLS/ACP When cross-compiling Windows under Linuxifeq ($ (findstring linux,$ (UNAME)),) Subdirs + = build/tools/acpendif else# ! Sdk_onlyifeq ($ (build_tiny_android), true) Subdirs: = Bionic System/core system/extras/ext4_utils system/extras/su Build/libs build/target BUILD/TOOLS/ACP external/gcc-demangle external/mksh external/openssl EXTERNAL/YAFFS2 External /zlibelse#! Build_tiny_androidsubdirs: = $ (TOP) Full_build: = trueendif#! build_tiny_androidendif# before we go and include all of the module makefiles, stash away# the product_* values so that LA ter we can verify they is not Modified.stash_product_vars:=trueifeq ($ (stash_product_vars), True) $ (call Stash-product-v ARS, __stashed) endif....ifneq ($ (one_shot_makefile),) include $ (one_shot_makefile) Custom_modules: = $ (Sort $ (call get-tagged-modules,$ (All_module_tags)) Full_build: =...else # one_shot_makefile## Include all of the makefilesIn the system# # Can ' t use First-makefiles-under here because#--mindepth=2 makes the prunes not work.subdir_makefile S: = $ (Shell build/tools/findleaves.py--prune=out--prune=.repo--prune=.git $ (subdirs) android.mk)  include $ ( subdir_makefiles)  ENDIF # one_shot_makefile......ifeq ($ (stash_product_vars), True) $ (call Assert-product-vars, __ stashed) endif...#-------------------------------------------------------------------# Fix up Custom_modules to refer To installed files rather than# just bare module names. Leave unknown modules alone in case# they ' re actually full paths to a particular file.known_custom_modules: = $ (Filter $ (a Ll_modules), $ (custom_modules)) Unknown_custom_modules: = $ (Filter-out $ (all_modules), $ (custom_modules)) CUSTOM_ MODULES: = $ (call module-installed-files,$ (Known_custom_modules)) $ (unknown_custom_modules#----------------------- --------------------------------------------# figure out We module sets.## of the modules defined by the component Makefi les,#Determine what do we actually want to Build. ifdef Full_build # The base list of modules to build for this product is s Pecified # By the appropriate product definition file, which is included # by Product_config.make. Product_modules: = $ (products.$ (internal_product). product_packages) # Filter out the overridden PACKAGES before doing expansion product_modules: = $ (filter-out $ (foreach P, $ (product_modules), $ (packages.$ (p). OVERRIDES)), $ (product_modules)) $ (call expand-required-modules,product_modules,$ (Product_modules)) Product_files: = $ (Call Module-installed-files, $ (product_modules)) ifeq (0,1) $ (Info product_files for $ (target_device) ($ (Internal_ PRODUCT)):) $ (foreach p,$ (product_files), $ (info: $ (p))) $ (Error done) Endifelse # i ' re not doing a full build, and is probably only including # a subset of the module makefiles. Don ' t try to build any modules # requested by the product, because we probably won ' t has the rules # to build them. Product_fiLES: =endif# When modules is tagged with debug eng or tests, they is installed# for those variants regardless of what th E Product Spec Says.debug_modules: = $ (sort $ (call Get-tagged-modules,debug) $ (Call Module-installed-files , $ (products.$ (internal_product). Product_packages_debug)) Eng_modules: = $ (sort $ (call Get-tagged-modules,eng) $ (Call module-installed -files, $ (products.$ (internal_product). Product_packages_eng)) Tests_modules: = $ (sort $ (call get-tagged-modules,tests) $ (Call Module-install Ed-files, $ (products.$ (internal_product). product_packages_tests)) # Todo:remove the 3 places in the tree so use all_default_installed_modules# and get rid of I T from the list.# todo:the shell is chosen by magic. Do we still need this?modules_to_install: = $ (sort $ (all_default_installed_modules) $ (product_files) $ (foreach tag,$ (Tags_to_install), $ ($ (tag) _modules)) $ (Call Get-tagged-modules, shell_$ (Target_shell)) $ (custom_modules)) # Some packages may override others using-local_overrides_packages.# Filter out (don't install) an Y overridden Packages.overridden_packages: = $ (call get-package-overrides,$ (Modules_to_install)) ifdef Overridden_ packages# Old_modules_to_install: = $ (modules_to_install) Modules_to_install: = $ (filter-out $ (foreach p,$ (overrid Den_packages), $ (P)%/$ (P). apk), $ (Modules_to_install)) endif#$ (Error filtered out# $ (filter-out $ (modul Es_to_install), $ (old_modules_to_install)))  # Don ' t include any of the GNU targets in the SDK. It's OK (and necessary) # to build the host tools, but nothing that's going to being installed# on the target (including Stati c libraries). Ifdef is_sdk_build Target_gnu_modules: = $ (Filter $ (target_out_intermedi ATES)/% $ (target_out)/% $ (target_out_data)/%, $ ( Sort $ (call Get-tagged-modules,gnu)) $ (info removing from SDK:) $ (foreach d,$ (target_gnu_modules), $ (info: $ (d))) Modules_to_install: = $ (filter-out $ (target_gnu_module S), $ (Modules_to_install))   # Ensure every module listed in product_packages* gets something installed # Todo:shoul D Do you builds and not just the SDK? $ (foreach m, $ (products.$ (internal_product). Product_packages), $ (if $ (strip $ (all_modules.$ (m)). Installed)),, $ (Warning $ (all_modules.$ (m). MAKEFILE): Module ' $ (M) ' in Product_packages have nothing to install!))) $ (foreach m, $ (products.$ (internal_product). Product_packages_debug), $ (if $ (strip $ (all_modules.$ (m)). Installed)),, $ (Warning $ (all_modules.$ (m). MAKEFILE): Module ' $ (M) ' in Product_packages_debug have nothing to install!))) $ (foreach m, $ (products.$ (internal_product). Product_packages_eng), $ (if $ (strip $ (all_modules.$ (m)). Installed)),, $ (Warning $ (all_modules.$ (m). MAKEFILE): Module ' $ (M) ' in Product_packages_eng have nothing to install!))) $ (foreach m, $ (products.$(internal_product). product_packages_tests), $ (if $ (strip $ (all_modules.$ (m)). Installed)),, $ (Warning $ (all_modules.$ (m). MAKEFILE): Module ' $ (M) ' in product_packages_tests have nothing to install!))) endif # Install All of the host Modulesmodules_to_install + = $ (sort $ (modules_to_install) $ (all_host_installed_ FILES))  # build/core/makefile contains extra stuff that we don ' t want to pollute this# top-level Makefile with. It expects that all_default_installed_modules# contains everything that's built during the current make but it also furth er# extends All_default_installed_modules. All_default_installed_modules: = $ (Modules_to_install) include $ (build_system)/makefilemodules_to_install: = $ (Sort $ (All_default_installed_modules)) All_default_installed_modules: = endif # dont_bother# These is additional goals that we build, with order to make sure That there# was as little code as possible in the tree that doesn ' t build.modules_to_check: = $ (foreach m,$ (all_modules), $ (All_modules.$ (M). CHECKED)  # If you would like to build all goals, and not skip any intermediate# steps, you can pass the ' all ' Modifi Er goal on the COMMANDLINE.IFNEQ ($ (Filter all,$ (makecmdgoals)),) Modules_to_check + = $ (foreach m,$ (all_modules), $ (All_ modules.$ (M). BUILT)) endif # for easier debuggingmodules_to_check: = $ (sort $ (Modules_to_check)) #$ (Error Modules_to_check $ ( Modules_to_check))

"Go" Android compilation system details (iii)--compilation process detailed