How to accelerate compilation of C ++ code

C ++The Code has always been in the face of the world with high performance and high profile, but the compilation speed is only low-key. For example, if I use the source code of my current job, even if I use Incredibuild to mobilize nearly a hundred machines, it would take four hours to build a complete build. It's horrible !!! Although normal development usually does not require a complete build locally, compiling several related projects is enough for you to wait for a long time. This is called monkey around und, which is quite an image ). Think about the scenario that has been working on a single core at GHz for several years-put a book in front of you and click the build button to read the book for a while ~~~ The past is terrible.

As you can imagine, if you do not pay attention to it, the Compilation speed is very likely to become a bottleneck in the development process. So why is C ++ compilation so slow?

One of the most important reasons is the compilation model of C ++'s basic "header file-source file:

1. each source file, as a compilation unit, may contain hundreds or even thousands of header files. In each compilation unit, these header files are read from the hard disk and parsed.

2. Each compilation unit will generate an obj file, so these obj files will be linked together, and this process is difficult to parallel.
Here, the problem lies in repeated load and resolution of numerous header files and intensive disk operations.

The following describes how to speed up compilation from various perspectives, mainly aiming at the key issue raised above.

I. Code

1. Use a pre-declaration in the header file instead of directly containing the header file.

Do not think that you only add a header file. This effect may be infinitely magnified due to the "included" feature of the header file. Therefore, we need to streamline the header file as much as possible. In many cases, it is difficult to declare a class in a namespace in front of the namespace, and the direct include operation is much more convenient. You must resist this temptation. The class members and function parameters should also be referenced as much as possible and pointers, create conditions for the pre-declaration.

2. Use Pimpl Mode

All Pimpl is called Private Implementation. The interfaces and implementations of traditional C ++ classes are confused, while Pimpl completely isolates the interfaces and implementations of classes. In this way, as long as the public interfaces of the class remain unchanged, only the cpp needs to be compiled for class implementation modifications. At the same time, header files provided by the class to the outside world will be much simpler.

3. highly modular

Modularization means low coupling, and mutual dependency is minimized. There are actually two meanings here. First, the changes in one header file between files should not cause re-compilation of other files; second, the modifications to one project between the project and the project, try not to cause too much compilation for other projects. This requires that the header file or project content must be single, and nothing should be inserted in it, thus causing unnecessary dependencies. This can also be called cohesion.

Take the header file as an example. Do not put two irrelevant classes or macro definitions that are unrelated to each other in a header file. The content should be as single as possible, so that the files containing them do not contain unwanted content. I remember that we once did this. We found out the header files in the most "hot" code and divided them into several independent small files. The effect was quite impressive.

In fact, the refactoring we did last year separates many DLL files into two parts: the UI and Core, which have the same effect-improving development efficiency.

4. Delete redundant header files

Some codes have been developed and maintained over the last ten years, and there are countless people involved. It is very likely that they contain useless header files or duplicate include files. Removing these redundant include files is quite necessary. Of course, this is mainly for cpp, because for a header file, it is difficult to define whether a specific include is redundant. It depends on whether it is used in the final compilation unit, this may be used in a compilation unit, but not in another compilation unit.

I have previously written a Perl script to automatically remove these redundant header files. In a project, more than 5000 include files are removed.

5. Pay special attention to inline and template

This is two more "advanced" mechanisms in C ++, But they force us to include the implementation in the header file, which adds the header file content, this slows down compilation. Before using it, consider it.

2. Comprehensive Skills

1. Pre-compiled header file PCH)

Place common but infrequently modified header files in pre-compiled header files. In this way, at least in a single project, you do not need to load and parse the same header file once and again in each compilation unit.

2. Unity Build

The Unity Build method is very simple. It includes all cpp into one cpp (all. cpp) and only compiles all. cpp. In this way, we only have one compilation unit, which means that we do not need to reload and parse the same header file, because only one obj file is generated, you do not need to perform intensive disk operations during the connection. It is estimated that there will be a 10 x improvement. Let's take a look at this video to see how it works and how it works.

3. ccache

Compiler cache, By caching the last compiled results, enables rebuild to greatly speed up keeping the results the same. We know that if it is a build, the system will compare the Time of the source code with the target code to determine whether to re-compile a file, this method is not completely reliable (for example, the Code of the previous version is obtained from svn), while the principle of ccache judgment is the file content, which is more reliable than other methods.

Unfortunately, Visual Studio does not support this function yet-you can add a new command, such as cache build, between build and rebuild, rebuild is useless.

4. Do not have too many Additional Include Directories

The compiler locates your include header file and searches based on the include directories you provide. As you can imagine, if you provide 100 contained directories and a header file is under 100th directories, it is very painful to locate it. Organize your include directories and keep them as concise as possible.

Iii. Compile Resources

To increase the speed, either reduce the number of tasks or increase the number of workers, the previous two aspects are all about reducing the number of tasks. In fact, the number of workers who increase the compilation speed is still very important.

1. Parallel Compilation

If you buy a 4-core or 8-core cpu, each build means that 8 files are compiled in parallel. The speed is superb. If your boss doesn't agree, let him read this article: Hardware is Cheap, Programmers are Expensive

2. Better Disks

We know that disk operations are a major cause of slow Compilation speed. In addition to minimizing disk operations, we can also speed up disk operations. For example, when the above eight cores work together, the disk is likely to become the biggest bottleneck. Buy a 15000 rpm disk, or an SSD, or RAID 0 disk. In short, the sooner the better.

3. Distributed Compilation

The performance of a machine is always limited. The idle cpu resources in the network and the build server dedicated for compilation can help you compile to fundamentally solve our compilation speed problem, if you want to build a project for more than one hour within two minutes, you will know that you must not have it-Incredibuild.

4. parallel operations can also be done.

This is an extreme situation. If you use Incredibuild and are not satisfied with the final compilation speed, what should you do? In fact, as long as you jump out of the framework of thinking, the Compilation speed can still be a qualitative leap-the premise is that you have enough machines:

Suppose you have solution A and solution B, and B depends on A, so you must Build B after. It takes one hour for A and B to Build each, so it takes two hours in total. But must B be built after? By jumping out of this framework, you will have the following solutions:

Build A and B at the same time.

The build of release A is successful. Although the build of Release B fails, it only fails on the final link.

◦ Re-link the project in B.

In this way, by parallel compilation of build and B of A, and finally link the project in B, the entire Compilation speed should be controlled within 1 hour and 15 minutes.

I hope this article will help you.