Openembedded (1): Introduction to openembedded

Document directory

• 2.1 scripts and Data
• 2.2 bitbake and OE metadata
• 3.1 bitbake directory
• 3.2 metadata directory
• 3.3 build directory
• 3.4 sources directory

1 Overview

Openembedded (OE) is an open-source embedded Linux system build environment. Both openmoko and poky use the customized openembedded environment. Openmoko is an open software and hardware design mobile phone platform. Poky is designed to be a customized openembedded environment. The poky environment contains a UI scheme called Sato, which is sometimes called poky. In fact, we can develop other GTK +/macthbox-based UI solutions in the poky environment.

To build an embedded Linux system, you need to build the boot module, kernel, and file system. This is a very complex process, especially the construction of file systems. A typical file system may need to download, compile, and install hundreds of software packages. If you have done LFS (Linux from scratch), you can understand the complexity of manually creating a file system. Because the embedded system is cross-compiled (compiled in the PC environment and run in the embedded environment), the building process is more complex.

Openembedded is designed to simplify the embedded system construction process. Openembedded simplifies the embedded system construction process despite its disadvantages such as large size and slow speed. For example, after installing the required tools, the entire process of building poky is to run the following two commands after entering the poky directory:

. poky-init-build-envbitbake poky-image-sato

Then wait until the build is complete. The build process may take 12-36 hours. The complexity of the target system, the network speed, and the software and hardware environment will affect the build time.

After a successful build, we can develop applications in this environment. In this case, we only need to compile the changed part. After the application development is complete, we can manually install it on the target system for testing. You can also re-build the file system as needed. In this case, oe only compiles the changed software package and replays the file system image once. It takes about 10-15 minutes to redo a file system image.

2. openembedded2.1 scripts and data that programmers see

In the programmer's opinion, openembedded is an automatic building system consisting of some scripts (shell and Python scripts) and data.

The script implements the build process, including fetch, unpack, patch, configure (if autotool is used), compile, and install) package, staging (discussed later), package_write_ipk, and file system construction.

Data mainly provides information in two aspects:

• Build information for specific software packages. How to obtain source code and patches? How can we build it, using makefile or autotool? What files need to be output to the target compiling environment? What files do I need to install? A description file is required for each software package. In fact, different versions of each software package have a description file.
• Dependencies between software packages. What host platform tools and Target Platform tools need to be built before building Software Package? Which software packages does Package A depend on during compilation? Which software packages does Package A depend on at runtime? Which software packages should a target contain? These dependencies link hundreds of software packages to form a complete system.

Scripts and data are already playing together. With the improvement of the system, the independence of the two is getting stronger and stronger, just as the pants filled with mud in besieged city can stand alone without relying on the legs of the master, data and scripts also hope to develop independently from the other party. A separate script is called bitbake. We hope that the independent data is OE metadata.

2.2 bitbake and OE metadata

Bitbake can be considered as a cooking tool to implement specific steps. Regard metadata as a recipe. If there are only cooking tools and no recipes, we don't know what to do. If you only have recipes and no cooking tools, you cannot complete the specific operations.

Bitbake, as a cooking tool, is a stable part of the build system. A variety of cooking tools can be improved, and recipes can be selected as needed.

OE metadata, as a recipe, is the part that represents changes in the build system. Changes come from two aspects:

• Although the recipe system is constantly improved, it includes software packages that most embedded systems may use. But there will still be new software packages developed. The existing software package will also have a new version.
• Each system has its own unique interests and tastes. Openmoko and poky are customized OE environments. They are customized with OE metadata. According to the needs of the mobile phone platform, openmoko adds and customizes some software packages and dependencies between the software packages. The main style of poky is the simplified GTK +/macthbox system. The openmoko environment has more than recipe files, and the poky environment has more than recipe files.

We can customize our metadata as needed. Openmoko and poky are both good start points. A good choice is to use poky as the starting point, and then retrieve your needs from openmoko. Of course, we always need to develop unique applications according to the design objectives.

3. openembedded in the file system

The most important directories in the OE environment are the bitbake directory for the tool, the metadata directory, and the build directory for the build.

3.1 bitbake directory

This directory contains our cooking tool bitbake. We use it, but usually do not need to access it

3.2 metadata directory

In poky, the metadata directory is meta. The metadata directory in openmoko is openembedded. In the metadata directory, there are three Real Metadata directories. They are: classes, Conf, and packages.

3.2.1 packages directory

All recipe files (suffixed with. bb) are placed in the package directory. Each relatively independent software package or build task has its own subdirectory under the package directory. You can have multiple recipe files in a sub-directory. They may be different versions of the same software package. It may also describe different build objectives based on the same software package.

Some recipes are simple, while others are complicated. A simple recipe only describes how to build a software package. The most complex recipe is the recipe for building a file system, such as poky-image-Sato in the poky environment and openmoko-devel-image in openmoko. This recipe file is not long or even short, but it involves hundreds or even thousands of other recipe files through dependencies. The images sub-directory in the packages directory is the recipe for building a file system.

3.2.2 classes directory

This directory contains the recipe class file (suffixed with. bbclass ). Class file contains some bitbake task definitions, such as how to configure and install. The recipe file inherits the definition of these tasks. For example, if we add a software package using autotool, we only need to inherit autotools. bbclass from the recipe file:

inherit autotools

Bitbake knows how to use the autotool tool to configure, compile, and install it. All recipe files automatically inherit base. bbclass. Base. bbclass provides default implementation for most bitbake tasks.

A recipe file can inherit multiple class files. In future articles, we will introduce bitbake tasks and discuss in detail the inheritance of bitbake. At present, we only need to know that the inheritance class file is a method of reusing the construction process.

3.2.3 conf directory

The conf directory contains the configuration file of the compilation system (with. conf as the suffix ). Bitbake executes bitbake. conf at startup. Bitbake. conf loads the local. conf file provided by the user. Then, the configuration files of the machine sub-directories and distro sub-directories are loaded Based on the hardware platform (MACHINE) and the release target (distro) defined in the local. conf file.

The machine subdirectory contains configuration files related to the hardware platform. The distro sub-directory contains the configuration files related to the release target. The configuration file is responsible for setting the environment variables used inside bitbake. These variables affect the entire build process.

3.3 build directory

Build is a big kitchen for Embedded cooking systems. The entire build process is completed in the TMP subdirectory of the build directory. The conf sub-directory in the build directory contains the user's configuration file local. conf.

The tmp directory contains seven subdirectories: cache, cross, rootfs, staging, work, deploy, and stamps. The cache is the cache directory used internally by bitbake. Cross is the interaction compiler generated during the build process. An interactive compiler is a compiler that runs on the host platform and is used to compile the code of the target platform. Rootfs is a temporary root file system created before the file system image is created. We usually do not need to access these three directories at work. The other four directories are accessed: staging, work, deploy, and stamps.

3.3.1 staging directory

Software Package B may depend on the header file and library file provided by Software Package A during construction, or it may need to use the tool generated by Software Package C. The staging directory is where these output files are stored.

We must use the "depends" variable in the recipe file to declare the dependency during the build. Bitbake will build Software Package A and Software Package C before building Software Package B, and put the header files, library files, and tools required by Software Package B in the staging directory. In this way, you can obtain the required header files, library files, and tools from the staging directory when building Software Package B.

3.3.2 work directory

The unpacking, patching, configuration, compilation, and installation of all software packages are carried out in the Work directory. Therefore, the work directory contains the complete source code of the entire embedded system. The working directory is divided into several subdirectories based on the differences between the hardware platform, issuer, and target platform. All software packages are placed in the corresponding subdirectory. Each software package has an independent directory. Because the software package is always built based on a recipe file, the directory where the software package is located is the working directory of the corresponding recipe file. We will come back when discussing bitbake and recipe files, and take a closer look at the working directory of recipe files.

3.3.3 deploy directory

This is the directory for storing output results. The images directory stores the file system images and kernel images generated after the build is successful. The IPK directory stores the installation packages of each software package. After modifying and building the software package, we can manually install the corresponding installation package under the IPK directory on the target platform. After confirming that there are no problems, create a file system image.

3.3.4 stamps directory

Similar to the work directory, the stamps directory is divided into several subdirectories Based on the hardware platform, issuer, and target platform. After each bitbake task is completed, the software package will touch the timestamp of the corresponding task in the corresponding subdirectory. Sometimes we manually delete the timestamp of a software package and force bitbake to re-build the software package.

3.4 sources directory

The sources directory of the OE environment is a storage room for storing source code packages downloaded from the Internet. The fetch task downloads the code and puts it in the sources directory.

I usually create a public sources directory outside the OE directory tree. Then, create a soft link in the OE directory to point to the sources directory. In this way, if I compile multiple versions in multiple OE environments at the same time, the sources directories of these OE environments can point to the same public sources directory. I asked openmoko and poky to use the same public sources directory. In this way, a large number of software packages required to build the system are stored in the public sources directory. When re-compiling a target version, you can save the download time. The entire construction process can be completed without an Internet connection. In the future, even if the link of some software packages fails, we can still build a complete system.

4 Conclusion

While automating the embedded system construction process, openembedded records the previous construction experience in the form of scripts. We can read the OE script to learn how to build an embedded system more intuitively.

No matter in windows, Linux, or RTOS, or whether using C/C ++, Java, or scripts, programmers are always separating the changes from the changes. Openembedded has separated scripts and metadata, and separated changed metadata from unchanged metadata. If openembedded is more effective, the associated configurations are concentrated, and the custom system may be simpler.

This article is a brief introduction to openembedded. Next we will learn more about bitbake's World Customs, the rules and practices of recipe files, and what happened in the tmp/work directory. The following article assumes that the reader has compiled and run poky or openmoko. The example in this article is mainly based on poky.

Regardless of how openmoko and openedhand (a poky developer has been acquired by Intel) develop, we are facing a complete set of Embedded Linux Solutions. As long as we want to, we can see every detail of the system, each line of code, including each line of script for building the system itself.