Build a Distcc-based android distributed compilation environment

Many articles have been published on the Internet about the establishment of Ditscc distributed compiling environment, but they are basically outdated. So I read a lot of articles, took a lot of detours, and finally sorted out a correct environment setup step and implemented zeroconf. This article does not involve the implementation of Server Load balancer.

The environment described in this article is built based on Ubuntu 12.04 64bit, And the android version is 4.1 (the same is estimated for other versions ). Android build compilation environment will not go into details, please refer to the official document https://source.android.com/source/initializing.html as much as possible

1. Introduction to Distcc

Distcc is a program that distributes compilation tasks of C, C ++ and other programs to multiple hosts on the network. The working principle of Distcc is: to build an execualble by compiling C/C ++ with GCC, there are four stages: preprocessing :. c. i, compiled by cc :. i. s, compiled by cc :. s. o, completed by the as link :. from o to executable files, collect2 completes the third stage, which is the efficiency bottleneck. Distcc is a compiler drive. In the gcc-c stage, it distributes the preprocessing output to the specified server array and collects the results. The parallel compilation of GNU Make's "-j" can use distcc to accelerate compilation. Distcc itself does not actually participate in any compilation process, but is only the front end of a compiler. Add distributed features to the compiler and participate in some management and simple Load Balancing functions. Distcc completes preprocessing locally (using gcc-E to expand the header file and macro), sends the result to the worker host in the cluster, and the worker host completes compilation (using gcc-c) return the compilation result concurrently, and finally link it locally. The distcc distributed Compilation Program can be divided into two parts: distcc: distributing the compilation task on the client to the compilation host distccd: the compilation host starts the distccd daemon to receive the compilation task and return the compilation result.

2. Install Distcc

If you do not want to take a detour, do not

sudo apt-get install distcc

Do not download the installation package at https://code.google.com/p/distcc/downloads/list.
After I install these two methods, a compilation error is generated on the Compilation host. As a result, almost all tasks are locally compiled. This makes no sense.

The recommended method is:

svn checkout http://distcc.googlecode.com/svn/trunk/ distcc-read-only

Code checkout and compile it by yourself. After getting the code, you should carefully read the INSTALL file, which is very detailed. It is better than any other articles on the Internet that describe how to build the distcc environment. Of course, android involves cross-compilation, which may be different.
To smoothly compile the installation package, INSTALL the following dependencies as described in the INSTALL document:

sudo apt-get install gcc make python python-dev binutils-dev alien libavahi-client-dev

To compile the deb installation package, install alien.
Ubuntu 12.04 supports zeroconf, which can be viewed in ps. avahi-daemon is running. However, to compile the installation package that supports zeroconf, libavahi-client-dev must be installed. I will not repeat what zeroconf is. If everything is ready, execute the following command:

./autogen.sh./configure --with-avahisudo make deb

The with-avahi parameter indicates that the build package supports zeroconf. If everything goes well, you will get the build installation package under the packaging directory of the source code.
Distcc_3.2rc1-1_amd64 distcc-server_3.2rc1-1_amd64
Install them:

dpkg -i distcc_3.2rc1-1_amd64 distcc-server_3.2rc1-1_amd64

Note:
In order to implement zeroconf, I first did an experiment on the Virtual Machine and the local machine. After continuing to configure it, I can find the host. The same environment is built on another computer, but zeroconf cannot find that computer. I can use avahi-discover to view all the hosts under Distributed Compiler. This shows that the zeroconf Implementation Program avahi can find this host, and distcc can also find it theoretically by using avahi to find the host. Continue tracing/var/log/syslog
File, see the line similar to the following

(daemon_proc) Browsing for 'x86_64-unknown-linux-gnu_4.4.3_distcc._tcp'.

That is to say, when distcc searches for a host through avahi, both the host and the gcc version are required to be consistent, but the gcc minor version does not affect the compilation result. Decisively modify the code. In the zeroconf. c file:

......# if (dcc_get_gcc_version(version, sizeof(version)) &&# dcc_get_gcc_machine(machine, sizeof(machine))) {## dcc_make_dnssd_subtype(stype, sizeof(stype), version, machine);# } else {rs_log_warning("Warning, failed to get CC version and machine type.\n");strncpy(stype, DCC_DNS_SERVICE_TYPE, sizeof(stype));stype[sizeof(stype)-1] = 0;# }......

In this way, when distcc finds the host, the parameter is "_ distcc. _ tcp. At least on my side, the modification is OK.

3. Configure distcc

The configuration files involved are mainly in the "/etc/distcc/" folder.
The content in client. allow is

# Indicates that the IP address of 192. 168. * can be sent to the compiled task 192.168.0.0/16.

Copy the compilation tool of the android source package prebuilts to the specified directory:/usr/local/distcc/prebuilts/
Modify commands. allow. sh:

numwords=1allowed_compilers="/usr/bin/cc/usr/bin/c++/usr/bin/c89/usr/bin/c99/usr/bin/gcc/usr/bin/g++/usr/bin/*gcc-*/usr/bin/*g++-*/usr/local/distcc/prebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.6/bin/arm-linux-androideabi*/usr/local/distcc/prebuilts/gcc/linux-x86/host/i686-linux-glibc2.7-4.6/bin/i686-linux*"......

Make sure that the content of the newly added two rows exists. Otherwise, the server will encounter a compilation error.

Under the "/etc/distcc/" folder
The hosts file content is:

localhost+zeroconf

It seems that the IP address that zeroconf can find can only match one CIDR block, so you can add another IP address in the file.

Under the "/etc/init. d/" folder
The content of the distcc file is:

......OPTIONS="--zeroconf --log-file=/var/log/distccd.log --daemon --stats --job-lifetime=1200"USER=distccPROG="distccd"PIDFILE=/var/run/$PROG.pidEXEC="/usr/bin/distccd"......

The "-- zeroconf -- log-file =/var/log/distccd. log" parameter is used to support zeroconf and log.
Restart distccd again.

sudo /etc/init.d/distcc reload

After the preceding steps are completed, the server configuration is basically completed (which must be configured on the client). To enable distcc compilation on the client, continue with the following steps:

4. android distcc compilation Configuration

The following describes the necessary conditions for android to successfully start the distcc compilation task. Many changes are not elegant.

Make sure that

export PATH=/usr/bin/distcc:$PATH

It is not recommended to write data to environment variables.

Under the "build/core/combo" folder
TARGET_linux-arm.mk files:

......ifeq ($(strip $(TARGET_TOOLS_PREFIX)),)TARGET_TOOLCHAIN_ROOT := /usr/local/distcc/prebuilts/gcc/$(HOST_PREBUILT_TAG)/arm/arm-linux-androideabi-4.6TARGET_TOOLS_PREFIX := $(TARGET_TOOLCHAIN_ROOT)/bin/arm-linux-androideabi-endif......

Select. mk file:

......include $(BUILD_COMBOS)/$(combo_target)$(combo_os_arch).mk$(combo_target)CC := distcc $($(combo_target)CC)$(combo_target)CXX := distcc $($(combo_target)CXX)......

Finally Start Compilation

. ./build/envsetup.shlunchmake -jN

You can.

5. Monitor distcc compilation tasks
Distcc comes with distccmon-text, which can be used to enable regionalization monitoring.

# Refresh distccmon-text 3 once every 3 seconds

You can also use distccmon-gnome to start a graphical monitoring program.

sudo apt-get install distccmon-gnomedistccmon-gnome &

You can.