In-depth understanding of Bitcode features in iOS development

Objective

iOS developers know that the latest Xcode7, the new project opens the Bitcode setting by default. And most of the developers are trapped by this sudden bitcode function, causing the project to fail to compile. And those projects that failed to compile because of bitcode have one thing in common, that is, a library or framework that links third-party binaries, and these frameworks or libraries do not contain bitcode (what is called something), This causes the project compilation to be unsuccessful. So whenever this happens, most people are directly setting up Xcode to turn off the Bitcode feature, all without generating bitcode. Don't go into the hidden principle behind this switch. Please give me a good shot.

LLVM is currently the compiler tool chain used by Apple, Bitcode is the LLVM compiler of the intermediate code of a coding, LLVM can be understood as the front end of the programming language, such as C/c++/oc/swift, LLVM backend can be understood as the assembly instructions on each chip platform or executable machine instruction data, then, Bitcode is located in both the direct middle code. LLVM's compiling principle is that the front-end is responsible for translating the project program source code into the Bitcode intermediate code, and then converting it to the corresponding assembly instructions and translating the machine code according to the different target machine-chip platforms. This design allows the LLVM to become a compiler architecture, It is easy to invent new languages (frontend) on top of the LLVM architecture, and to support new CPU (back-end) command output under the LLVM architecture, although Bitcode is just an intermediate code that cannot be run on any platform, but it can be transformed into any supported CPU architecture, This includes a CPU architecture that is not yet invented, which means that the Bitcode feature is now turned on to submit an app to the App Store, and later if Apple has a new phone and the CPU is completely redesigned, The Apple backend server can be compiled from the bitcode of this app into an executable program on the new CPU, which can be downloaded and run by new phone users.

Historical Review

Before the iphone came out, Apple's main compiler technology was to use a slightly improved GCC toolchain to compile code from the Objective-c language to produce native executable programs on the specified machine processor. The executable program generated by the compiler is called "Fat Binaries"-- Similar to the Windows under the PE format of the EXE and the Linux under the elf format of binary, the difference is that a "Fat Binary" can contain many versions of the same program, So the same executable can run on different processors. This is the technology that makes it easy for Apple's hardware to migrate from PowerPC to PowerPC64 processors, and later migrated to Intel and Intel64 processors. The downside of this scenario is that multiple executable code is stored in the same file, which is useless except for the one that is currently being executed by the machine, and white space. This is known in the market as "Universal Binary", where Apple migrated from PowerPC to Intel processors (a binary file contains both a PowerPC version and an Intel version). Slowly later, it also supports the inclusion of Intel 32bit and Intel 64bit. In a fat binary, while the operating system is running according to the processor type dynamic selection of the correct binary plate run, but the application to support different platforms of the processor, the application itself to occupy more space. There are also some slimming tools, such as lipo, that can be used to remove fat Binaries in the current machine are not supported or redundant executable code to achieve the purpose of slimming, Lipo will not change the program execution logic, just the size of the file is thin.

Compiler status

With the development of mobile Internet, the size of the mobile device is becoming more and more important, mainly because the mobile device will not have a large hard disk drive on the computer. And Apple has long migrated from the original ARM processor to its own design a4,a5,a5x,a6,a7,a8 , a8x,a9,a9x and subsequent A10 processors, their instruction set has changed and the original arm design is different, all these changes are the iOS operating system and XCODE/LLVM compiler tool to the upper level of the programmer to a degree of transparency, The compiled program contains many code-execution versions. When faced with this problem, it is becoming increasingly necessary for Apple to move into the LLVM compiler architecture and use Bitcode. From the very beginning, compile OpenGL into a specific GPU instruction to the clang compiler (LLCM c/ OC Compile front end) supports objective-c improvements and acts as the default compiler for Xcode.

LLVM provides a virtual instruction set mechanism that translates the execution code (machine code) of the specified supported processor architecture. This makes it possible to develop a tool chain that is completely based on the LLVM architecture for the compilation of iOS applications. LLVM's Virtual universal instruction set can be represented in a number of formats:

• A compilation format called an IR text representation (like assembly language);
• Converted to the format of the binary data representation (like the target code), this binary format is what we call the Bitcode.

Unlike the traditional set of executable instructions, Bitcode maintains the type and signature of function functions, for example, in a traditional set of executable instructions, a series of Boolean values (<=8) can be compressed into a single byte, but in Bitcode they are individually represented. In addition, the logical operation ( For example, register 0 operations) are also represented by their corresponding logical representations ( $R=0 ); When these bitcode are to be converted to the instruction set of a particular machine platform, he can replace them with assembly instructions optimized for a specific machine platform: xor eax, eax . ( This assembly instruction is also register <eax> Clear 0 operation).

However Bitcode he is not completely independent of the processor platform and the calling convention. The size of the register is a fairly important feature in the instruction set, and it is well known that the 64bit registers can store more data than the 32bit registers, The Bitcode of the Bitcode and 32bit platforms that generate the 64bit platform are significantly different, and the calling convention can be defined based on function definitions or function calls, which determine whether the function's parameter passing is a register value or a stack. Some programming languages also have preprocessing directives like sizeof (long), which are translated before Bitcode is generated. Typically, for support fastcc (fast calling convention) The 64bit platform that is called generates its consistent Bitcode code.

Apple's requirements

Here, let's think about why Apple requires watchOS and tvOS apps to upload bitcode by default. Since uploading the bitcode to his own central server, he can optimize the binary for the target installation of the app, reduce the download size of the installation package, and, of course, iOS developers can upload multiple versions rather than package them in a single package, but this will take up more storage space. The most important thing is to allow Apple to sign the application in the background server without exporting any key to the terminal developer.

The bitcode of uploading to the server is more beneficial to Apple: The new CPU is designed for new instruction set in the future, you can continue to compile the executable files on the new CPU from this bitcode for users to download and install.
But the Bitcode to developers is: before useless bitcode, when the application ran out, the developer can be based on the resulting crash log and the upload to the Apple server binary file debug symbol table information can restore the program run to the end of the call stack information, Troubleshoot the problem. But after using the Bitcode, the user installed binary system is not generated by the developer side, but the Apple server after optimization generated, its corresponding debug symbol information is lost, it can not be said before the restoration of the scene to find reasons.

Currently, watchOS and tvOS applications must be uploaded with a bitcode version of the package. iOS app release requirements for Bitcode are optional and can be turned off by the user in Xcode's project settings. Equivalent to adding a tag at compile time: Embed-bitcode-marker (Debug build) Embed-bitcode (Package/True agency). This parameter in the Clang compiler is-fembed-bitcode, The parameters of the Swift compiler are-embed-bitcode.

Practice is the truth

We should actually get two test code to practice and test it better. Do two Tests, prepare two C language source code for the first time to continue testing; the second time one of these changes to the assembly language source code, and then a C code and a assembler code to repeat the previous test steps to check the difference between the two.

• 1. All two of the following are OBJECTIVE-C codes:

TEST.M:

#import <Foundation/Foundation.h>void greeting(void){    NSLog(@"hello world!");}

DEMO.M:

#import <Foundation/Foundation.h>void demo(void){    NSLog(@"demo func");}

The target file with Bitcode is compiled into ARM64 format with clang TEST.O DEMO.O:

wuqiong:~ apple$ xcrun -sdk iphoneos clang -arch arm64 -fembed-bitcode -c test.m demo.m

Then package the two target files into a static library file:

wuqiong:~ apple$ xcrun -sdk iphoneos ar  -r libTest.a test.o demo.oar: creating archive libTest.a

Use the shell command Otool to see if the target file contains Bitcode segments:

-l test.o |grep bitcode  sectname __bitcode  sectname __bitcode

If you see the output of 2 lines, it means that the sectname __bitcode two target files in this static library contain Bitcode.

• 2. Replace one of the DEMO.M with assembly language and then participate in the compilation:

  Use the following command to convert DEMO.M C code to ARM64 assembly language format Demo.s:

wuqiong:~ apple$ xcrun-sdk iphoneos clang-arch arm64-s demo.mwuqiong:~ apple$ cat Demo.s. Section __TEXT,__text,r Egular,pure_instructions. ios_version_min9,2. Globl _demo. Align2_demo:; @demo. Cfi_startproc; Bb#0: STP x29, x30, [SP,#-16]! mov x29, spLtmp0:. CFI_DEF_CFA w29,16LTMP1:. Cfi_offset w30,-8LTMP2:. Cfi_offset w29,-16 adrp x0, [email protected] add x0, x0, [email protected] bl _nslog ldp x29, x30, [sp],  #16 ret. cfi_endproc. Section __text,__cstring,cstring_literalsl_.str:; @.str. Asciz  "demo func". Section __data,__cfstring. Align 4; @_ Unnamed_cfstring_l__unnamed_cfstring_:. Quad ___cfconstantstringclassreference. Long  1992; 0x7c8. Space 4. Quad L_.str. Quad 9; Span class= "Hljs-number" >0x9. Section __data,__objc_imageinfo,regular,no_dead_stripl_objc_image_info:. Long 0. Long 0.subsections_via_symbol     

Then delete demo.m this c source code, leaving only test.m and demo.s :

wuqiong:~ apple$ rm demo.m

Now, let's take test.m this c source code and dmeo.s this assembler source code together with -fembed-bitcode parameters to generate the target code and package it as a static library:

wuqiong:~ apple$ xcrun -sdk iphoneos clang -arch arm64 -fembed-bitcode -c test.m demo.swuqiong:~ apple$ xcrun -sdk iphoneos ar -r libTest.a test.o demo.o

We then run the Otool tool to check whether the 2 target files contained in this new static library have Bitcode segments:

-l libTest.a | grep bitcode  sectname __bitcode

Unexpectedly, this time, only one line sectname __bitcode of output, this shows that the two target files, there is a no bitcode segment, even if we compile the parameters -fembed-bitcode are not used. As to which one does not take the Bitcode segment, We must know that the target file compiled from ARM64 assembly language does not take.

Then it comes to a conclusion that the generation of Bitcode is compiled from the upper-level languages above the assembly language, and, as before, he is an intermediate code between the upper language and the assembler (machine language).

At present, in our daily iOS application development, we generally do not need to use the assembly level to optimize the code. So we mainly focus on third party (open source) C code, especially audio and video encoding decoding these computationally intensive project code, the key calculation of the code for a specific platform has a corresponding platform for the compilation version implementation, Of course there is the implementation of C, but the default compilation is generally used in the assembly version, which will lead us to compile this open source code even if you take the -fembed-bitcode parameters and just let some of the project C code target file with Bitcode segment, and that decimal assembly code of the target file without the Bitcode segment, so that the compilation of this library to the upper-level developers to use, it will appear in the package upload or real-time debugging because Xcode default open Bitcode function and link failure, Causes no real-machine debugging or can not upload apps to AppStore.

The purpose of this article

Recently in my Camp David comrades to do mobile phone audio and video live app, debugging when mobile phone capture audio and video, video with H264 encoding, audio using AAC code, through the RTMP protocol to the fight Fish live channel release media stream, project needs FFMPEG and libx264 two open source projects, When compiled for the iOS framework library to provide students with the problem, they encountered bitcode problems, although can be taken directly to close bitcode to avoid errors, but comrade-in-arms curiosity must be met, Scientia, must let it know.

libx264VideoLAN Foundation manages a video codec of the Open source project, its extensive use of the various platforms of multimedia assembly instructions are optimized, when compiled into a library without bitcode, completely according to the official Autotools compilation method is not any problem; We had to turn off assembly optimizations when compiling libraries with Bitcode, and we ./configure could add parameters to disable the assembly during the execution phase. --disable-asm However, the implementation mechanism of this option in the configure script is problematic. Cause it still calls the assembly function, but the assembly of the Code is not compiled in, The link stage that causes the project to be built and packaged for true organization will burst out of the wrong sign, so that the best of both worlds cannot be achieved. For a mild degree of obsessive-compulsive disorder, the previous FFMPEG and libx264 The compilation script for the project has been improved and patched. It is now possible to compile a ffmpeg and libx264 frame with all bitcode in one click.

FFmpegNeed to rely on libx264 .

Auto-compile script project location on GitHub:
Https://github.com/Diveinedu-CN/FFmpeg-iOS-build-script.git

In-depth understanding of Bitcode features in iOS development