Customized embedded Linux release

Customized embedded Linux release

The popularity of cheap Iot board means that it will not only control applications, but also control the entire software platform. So how to build a custom release for a cross-compiled application for a specific purpose? As Michael J. Hammel explained here, it is not as difficult as you think.

 

Why is it customized?

In the past, many embedded projects used off-the-shelf releases, and for various reasons, they were stripped away to only the basic necessary things. First, remove unnecessary packages to reduce the occupied storage space. At startup, embedded systems generally do not need a large amount of storage space and available storage space. During the operation of the embedded system, a large number of operating system files may be copied from the non-volatile memory to the memory. Second, removing unused packages can reduce the possible attack surface. If you don't need them, there is no need to mount these vulnerable packages. Finally, removing unused packages can reduce the overhead of release management. If there is dependency between packages, it means that any packet request is updated from the upstream, they must be synchronized. Then there may be verification nightmares.

However, removing a package from an existing release is not as easy as you said. Removing a package may break various dependencies with other packages, and may change dependencies in the upstream release management. In addition, some packages cannot be easily removed because of native integration during boot or runtime. All these are the management of platforms outside the project, and may lead to unexpected development latency.

A popular choice is to use build tools provided by upstream release vendors to build a custom release. Both Gentoo and Debian provide this bottom-up Build method. The most popular of these build tools is Debian's debootstrap utility. It extracts pre-built Core Components and allows users to select the packages they are interested in to build their own platforms. However, debootstrap was initially only available on the x86 platform, although now with ARM (or other platforms) options. Debootstrap and Gentoo catalysts still need to remove dependency management from local projects.

Some people think that it is much easier for others to manage platform software (like Android) than to manage it themselves. However, those releases are all versatile. When you use them on a lightweight, resource-limited Iot device, you may think twice about any resources that are taken away from your hands.

 

The cornerstone of system guidance

A customized Linux release requires many software components. The first one is tool chain toolchain. A tool chain is a set of tools used to compile software. Including (but not limited to) a compiler, linker, binary operating tool, and standard C library. A tool chain is specially built for a specific target hardware device. If a tool chain built on an x86 system is intended for Raspberry Pi, this tool chain is called a cross-compilation tool chain. When working on small embedded devices with limited memory and storage, it is best to use a cross-compilation toolchain. It should be noted that even applications that use scripting languages such as JavaScript that need to run on a specific platform for a specific purpose need to be compiled using the cross-compilation tool chain.

Figure 1. Compile dependencies and boot sequence

The cross-compilation toolchain is used to build software components for the target hardware. The first component is the bootstrap loader. When the computer motherboard is powered on, the processor (which may vary depending on the design) tries to jump to a specific memory location to start running the software. The memory location is the place where the boot loader is saved. The hardware may have a built-in Boot Loader which may directly copy data from its storage location or to the memory before it runs. There may also be multiple boot loaders. For example, the boot loader in the first phase may be in the hardware's NAND or NOR flash. Its only function is to set hardware to facilitate the execution of the second-stage boot loader-for example, a boot loader that can be loaded and run in the SD card.

The boot loader can obtain sufficient information from the hardware, load Linux to the memory, jump to the correct position, and effectively hand over control to Linux. Linux is an operating system. This means that, in this design, it not only monitors the hardware and the upstream software (that is, the application) to provide services, but it does not actually do anything. Linux Kernel is usually a variety of firmware blocks. The pre-compiled software objects usually contain the private IP (knowledge assets) of the devices used by the hardware platform ). When building a custom release, it may require required firmware blocks that are not provided by the Linux kernel source code tree before compiling the kernel.

The application is stored in the root file system. This root file system is built through compilation and integrates various software libraries, tools, scripts, and configuration files. In general, they all provide various services, such as network configuration and USB device mounting, which are required by the project applications to be run.

In general, a complete system build requires the following components:

1. A cross-compilation tool chain
2. One or more boot Loaders
3. Linux Kernel and related firmware Blocks
4. A root file system that contains libraries, tools, and utilities
5. Customized applications

 

Start building with appropriate tools

Components of the tool chain can be built manually, but this is a very complicated process. Fortunately, existing tools can easily complete this process. The best tool for building a cross-compilation toolchain may be Crosstool-NG, which uses the same kconfig menu system as the Linux kernel to build every detail and aspect of the toolchain. The key to using this tool is to find the correct configuration items for the target platform. Configuration items usually include the following:

1. Target architecture, such as ARM or x86.
2. Byte sequence: In general, Intel adopts this sequence. In general, Intel uses this sequence. In general, ARM or other platforms use this sequence ).
3. The CPU type known to the compiler. For example, GCC can use-mcpuOr--with-cpu.
4. Supported floating point type, if any, for example, GCC can be used-mfpuOr--with-fpu.
5. Binary utility binutils, C library, and specific version information of the C compiler.

Figure 2. Crosstool-NG configuration menu

The first four can generally be obtained from the processor manufacturer's documentation. For newer processors, they may not be easy to find, but, like Raspberry Pi or BeagleBoards (and their descendants and branches ), you can find related information in places like embedded Linux Wiki.

Binary utilities, C libraries, and C compiler versions are separated from other tool chains provided by any third party. First, each of them has multiple providers. Linaro provides the most advanced version for the latest Processor type, and is committed to consolidating this support into upstream projects such as the gnu c library. Although you can use various provider tools, you may still want to use the GNU tool chain or the same Linaro version.

Another important choice in Crosstool-NG is the Linux kernel version. This option will get the header file headers for various tool chain components, but it does not need to be the same as the Linux kernel to be booted on the target hardware. It is important to select a Linux kernel that is not updated than the target hardware. If possible, select a kernel with a longer LTS kernel than the Kernel used by the target hardware.

For most developers who are not familiar with building custom releases, the creation of tool chains is the most complex process. Fortunately, the binary tool chain of most hardware platforms can be obtained. If there is a problem with building a custom tool chain, you can search for pre-built tool chains in places such as embedded Linux Wiki online.

 

Boot options

After the toolchain is built, the next task is to guide the loader. Bootstrap loader is used to set hardware so that more and more complex software can use it. The pilot loader in the first phase is generally provided by the target platform manufacturer and is typically burned to hardware-based storage similar to EEPROM or NOR flash. The first-stage boot loader will enable the device to boot from here (for example, an SD memory card. Raspberry Pi's boot loader is like this, so there is no need to create a custom boot loader.

Despite this, many projects have added a second-stage boot loader to facilitate the execution of a diverse task. One of these tasks is to provide a boot animation without using the Linux kernel or a user space tool like plymouth. A more common second-stage Boot Loader task is to provide network-based boot or make the disk connected to PCI available. In that case, a third-stage boot loader, such as GRUB, may be necessary for the system to run.

Most importantly, the boot loader loads the Linux kernel and starts running it. If the first-stage Boot Loader does not provide a mechanism to pass kernel parameters at startup, it must be provided in the second-stage boot loader.

Many open-source boot loaders are available. The U-Boot project is usually used on ARM platforms like Raspberry Pi. CoreBoot is generally used on x86 platforms like Chromebook. The boot loader is specific to the target hardware. The selection of Boot Loader depends on the project requirements and target hardware (you can search for the list of open source Boot Loader programs online on the network ).

 

Now it's time for the launch of Linux

The boot loader loads the Linux kernel to the memory and then runs it. Linux is like an extended Boot Loader: it configures hardware and prepares to load advanced software. The core of the kernel sets and provides the memory shared between applications and hardware, and provides the task manager to allow multiple applications to run simultaneously; initialize hardware components that have not been configured by the boot loader or are not completed, and enable the man-machine interaction interface. The kernel may not be configured to do this on its own. However, it can contain an embedded, lightweight file system, which is well known as initramfs or initrd, they can be created independently of the kernel and used to assist in hardware configuration.

Another thing for Kernel operations is to download binary blocks (usually referred to as firmware) to hardware devices. A firmware is an object file precompiled in a specific format. It is used to initialize a specific hardware in a boot loader or a place that the kernel cannot access. Many of these firmware objects can be obtained from the Linux kernel source warehouse, but many other firmware objects can only be obtained from specific hardware vendors. For example, devices that often provide their own firmware include digital TV tuner or Wi-Fi network card.

The firmware can be loaded from initramfs or after the kernel starts the init process from the root file system. However, when you create a customized Linux release, the process of creating a kernel is often the process of obtaining various firmware.

 

Lightweight core platform

The last thing the Linux kernel does is to try to run a dedicated program called the init process. The private program name may be init or linuxrc or the name passed to the kernel by the loader. The init process is stored in a file system that can be accessed by the kernel. In the case of initramfs, the file system is stored in the memory (it may be placed by the kernel itself or by the boot loader ). However, initramfs is generally not complete enough to run more complex applications. Therefore, another file system is needed, which is a well-known root file system.

Figure 3. Build the root configuration menu

The initramfs file system can be built using the Linux kernel. However, a more common practice is to create a file using a project called BusyBox. BusyBox combines many GNU utilities (such as grep or awk) into a single binary file to reduce the size of the file system. BusyBox is usually used to start the creation process of the root file system.

However, BusyBox is specially lightweight. It does not intend to provide all the tools required by the target platform, or even the tools provided are simplified in functions. BusyBox has a "sister" project called Buildroot, which can be used to obtain a complete root file system, providing various libraries, utilities, and scripting languages. Like Crosstool-NG and Linux kernels, BusyBox and Buildroot allow you to use the kconfig menu system to customize configurations. More importantly, the Buildroot system automatically processes dependencies. Therefore, the selected utility ensures that the software required by the program is also built and installed to the root file system.

Buildroot can generate a root file system package in multiple formats. However, it is important to note that the file system is archived. A single utility and library are not packaged in Debian or RPM format. Use Buildroot to generate a root file system image, but its content is not a separate package. Even so, Buildroot provides support for opkg and rpm package manager. This means that, although the root file system does not support package management, customized applications installed on the root file system can manage packages.

 

Cross-compilation and scripting

One feature of Buildroot is the ability to generate a temporary tree. This directory contains libraries and utilities that can be used to cross-compile other applications. Using the temporary tree and cross-compilation toolchain makes it possible to compile applications other than Buildroot on the host system rather than on the target platform. With rpm or opkg package management software, these applications can be installed on the root file system of the target platform using the package management software at runtime.

Most custom systems are built around the idea of using scripting to build applications. To run scripts on the target platform, there are multiple options available on Buildroot, including Python, PHP, Lua, and Node. js-based JavaScript. It also supports applications that require OpenSSL encryption.

 

What to do next

The compilation process of Linux kernel and boot loader is the same as that of most applications. Their build systems are designed to build a dedicated software location. Crosstool-NG and Buildroot are meta-built metabuild. Meta-build encapsulates a series of software sets with their own build systems into a build system. Reliable metadata construction includes Yocto and OpenEmbedded. The benefit of Buildroot is that it can easily encapsulate higher-level meta-build to automate the Build Process of customized Linux releases. After doing so, the option of Buildroot pointing to the project-specific cache repository will be enabled. Using a cache repository can accelerate the development process and provide snapshot creation without worrying about changes in the upstream repository.

An example of implementing an advanced build system is PiBox. PiBox is a meta build that encapsulates various tools discussed in this article. It aims to add a general GNU Make target architecture around all tools to generate a core platform that can build or distribute other software. The PiBox media center and kiosk project are the application layer software installed on the core platform to generate a building platform. The Iron Man project expands this application for the purpose of home automation, which integrates voice management and IOT device management.

However, PiBox cannot do anything without these core software tools. In addition, if you do not have a deep understanding of the build process of a custom release, you will not be able to correctly run PiBox. In addition, without the long-term dedication of the PiBox development team to this project, there will be no PiBox project, and it has completed a large number of tasks in the Custom release build.

Via: http://www.linuxjournal.com/content/custom-embedded-linux-distributions

Author: Michael J. Hammel Translator: qhwdw Proofreader: wxy

This article was originally compiled by LCTT and launched with the honor of Linux in China