Continuous integration across teams: Several Basic Contradictions

 

Continuous integration within a single team is already a mature practice. Cross-team integration has encountered many problems, including the long test run time,
There are some corresponding solutions to these problems, such as reasonable branch policies and hierarchical integration.

I would like to discuss several basic contradictions and the ideal solutions.

 

1. Conflicts between parallel development and integration

This is an essential problem. If all functions are completed step by a single developer, integration is not a major problem.
Because the internal integration of the team already has a lot of mature practices, the previous assumption can be changed to "if all the functions are completed step by the same team, integration is not a big problem ".
This provides us with an idea: If we allocate all the parts to be integrated to a team for completion, the difficulty of integration will be greatly reduced.

In traditional large-scale development, development teams are usually divided by module, such as the UI, network communication, and database access,
The protocol, and so on, are different teams. in the end, in order to complete the visible functions of a user, multiple teams need to be coordinated for parallel development, and then joint debugging, that is, integration. that is to say, features need to be integrated.
If we divide teams by features, cross-team integration will be converted into internal integration, and there are already a lot of mature practices for internal integration.

For a comprehensive comparison between the feature team and the module team, see <choose
Feature teams over component teams for agility
>

The first principle to reduce integration costs is to avoid integration.
. Reduce cross-team integration requirements, if not avoided.

Dividing teams by feature will bring about a major problem, that is, people from different teams may modify the same file and class at the same time,
The same function, because different features will involve the same functional module, such as database, network, and UI. This is the second question we want to discuss.

 

2. Conflicts Between text-based Merging and semantic-based logic

Modern version control systems can automatically merge texts intelligently. However, the merged results are not what we expect,
It can only be verified through tests. In addition, version control tools are powerless due to conflicts at the text level. Therefore, manual intervention is required.The first principle is to avoid merging.
.
There are several possible ways to achieve this goal:

1. Plug-in-based architecture: This isPrinciple of opening/closing
   . Add any new features,
   You do not need to modify the original file or class, but add files or classes. An example of a large scale is the eclipse ecosystem.
   If you think of all the plug-ins of eclipse as the required features of a complete system, some developers of these features have never even met each other,
   Code-level integration with other plug-ins is not required during development, but they can work together harmoniously.

2. Code organization based on small files, small classes, and small functions: This isSingle Responsibility Principle
   Application,
   "Small" is only the external expression of the Code. The true meaning is a single responsibility. Taking C language as an example, consider two extreme situations. one extreme is to write the entire system in the main function,
   Another extreme is that each independent function is written as an independent function, and each file contains only one function. in the previous case, any changes made by any team must be merged with changes made by other teams,
   The combined behavior is verified through a comprehensive test. in the latter case, the merge requirement is greatly reduced. Only when you change the same function, and the test can take the impact scope as the boundary.
   Of course, we do not need to be as extreme as the last case, but at least specify the direction forward.

This kind of solution also brings about a problem, that is, we cannot design such a good plug-in system from the very beginning. It can only be an evolutionary architecture.
During this period, the interface changes between plug-ins and frameworks will pose a challenge to integration, that is, how old customers and new services can get along peacefully. This is the third question we will discuss.

 

3. Conflicts between dependency stability and dependent evolution and upgrade

This is a common problem, especially for hierarchical architecture or development projects in the "Platform + product" mode.

Ideally, the so-called ideal situation refers to the collective ownership of code/the complete reconstruction functions of the IDE/comprehensive automated test cases/, etc,
Modifications to Customer Code caused by changes in public APIs can be completed and submitted by a team at a time. However, this is impossible in a large number of legacy projects. In this case,
"Third-Party code line" (see <Software
Configuration Management Patterns
>) Or "Change Control and modification group" (responsible for implementing changes and submitting) is an optional solution

Here we would like to discuss another solution that is relatively more common on a large scale but will introduce management costs and be somewhat risky.Backward compatible,
Or Versioning
. Example:

1. We know that the COM technology is designed to solve the problem of dll version hell. The new version of COM component can get along with old customers in peace,
   The reason is that the COM component upgrade does not directly modify the old interface, but adds a new interface, retains the old interface, and provides the capability query interface, so that new and old customers can obtain their own needs.
   This is an application in the "Extended object" mode. This is API-level backward compatible/versioning.

2. The client of subversion. subversion can negotiate the version with the server,
   Select a protocol that everyone understands. This is protocol-level backward compatibility/versioning.

3. XML. XML is designed for extension and can be used to implement backward compatible messages/protocols.

However, in actual projects, we do not want the old and old API/old protocols to exist for a long time. Instead, we want to delete the old API/old protocol,
All old customers use new APIs and protocols. Therefore, we need to apply the above scheme in stages and provide management or technical means to ensure that all new and old customers are completed at the end of the transition period.
The risk here is that once chaotic and powerless management allows the old interface to exist for a period of time, it will always exist.

 

The above three problems are hard to be completely solved in large-scale legacy systems. Therefore, these solutions or ideas are as follows,
We only hope to provide some considerations when developing a new system.