Continuous Integration Practice Maturity Model

1 Overview
Continuous integration tests its maturity from five dimensions: "Configuration Management", "Build", "test", "deployment and release", and "Team habits, each dimension has five levels: "entry", "Beginner", "medium", "advanced", and "crazy ". At present, the average level of the industry is concentrated in two dimensions: "entry" and "Beginner.
During the evaluation, different levels cannot be upgraded. That is to say, even if some entries in the "Beginner" have already been done, if there are entries in the "entry", they can only be rated as "entry.
This model aims to help the team better understand the current situation and the direction of improvement. This model is a simple measure of the main dimensions of continuous integration. There may be assumptions behind it that are not suitable for the team and each entry is not explained in detail, for targeted comprehensive evaluation of team continuous integration, contact the PMO department.
Due to technical and project differences, the effort and benefits of different teams to reach the same level may vary greatly, so it is not suitable to use this model for comparison between teams.
2. configuration management dimension
Entry:
Use version management tools
Use private branch
Function Test Case Local Management
Production Code Version Management
Submit code every day and write comment.
Beginner:
Branch policies that can be integrated by applications after the minimum unit of work is completed
All build, test, and deployment scripts are managed by version
All test code and data are centrally managed
Application configuration in the test environment is managed by version
The comment of code check-in is clear and contains the necessary metadata, such as the bug number.
Rd uses a fully standard development environment and does not have scripts, data, or test environments for its own use.
Moderate:
Same test code and production code
Application configuration in the production environment is managed by version
The scripts running on the continuous integration platform are managed by versions, so you do not need to log on to the platform to modify the scripts.
Each build has version tracing, and you do not need to re-build historical versions from the source code.
QA uses a standard development and testing environment
Database Version Management
Feature test data is managed by version
Advanced:
No project assets are saved locally only by team members.
Rd and QA use the same development and testing environment.
Crazy:
The development, testing, and production environments are managed by versions and can be cloned with one click.
All test data is managed by version

Learn more about the configuration management status of the team through the following questions:
What branch management policies are used?
How often and how is the comment quality of code checked in by team members?
What steps does a rd need to establish a complete working environment on a brand new machine?
What are the steps for a QA to build a complete working environment on a brand new machine?
What is the difference between the working environment of RD and QA?
What are the differences between RD working environments?
What are the differences between QA work environments?
How is the test data managed?
Is RD developed in sandbox? If not, what are the dependencies?
How is the local and platform build scripts managed?
How is the application configuration managed in the test environment and production environment managed?
How are test tools and test cases managed?
What are the dependencies of the testing environment? Is it easy for everyone to run in their own work environment?
Are all team members using the same script for local build?
What private code, scripts, and pre-deployed environments do team members have?
3. Build dimensions
Entry:
Execute the build steps with the help script
Phase build, such.
Shell script as an automated build development script
Beginner:
After the code is checked in, the building process containing automated tests is run.
The building result is effectively notified to the team.
Everyone knows how to run local build
There are no outstanding environment problems in the basic construction process
Automated build tasks are completed using automated build scripts.
Dedicated personnel maintain the building environment.
Moderate
Run the automated smoke test on the local and Platform
Run the full set of function tests and performance tests on the platform.
Make full use of multiple machines to run multiple builds.
The Build Process integrates detailed reports.
Allows you to flexibly configure specific tasks for each build phase.
Synchronize and upgrade environment dependencies of various machines through scripts
Advanced
Building steps are fully optimized
The building process can analyze the report, record the trend of key indicators, and make improvements.
Make full use of multiple machines for a single build task.
Check out the project in the new development environment and use a script to build a complete development and testing environment and successfully generate deployment packages.
Crazy
If the running time exceeds the limit, the operation fails.

learn more about the team building status through the following questions:
how to manage compilation dependencies?
how to manage module version dependencies?
how long is the Compilation Time?
are the building environments and methods of each person consistent?
What are the steps for automated building?
what are the main causes of automated building failure?
is build grid used? How to use it?
how to manage machines in the grid? How to synchronize and update the environment?
Local build content? When to run? How long does it take?
which CI platform is used? How to configure it?
how to adjust the content contained in the build?
4. Test dimension
some test-related items are available in other dimensions. For more information, see.
Getting Started:
some automated function tests
concentrated tests at the end of the project
defect Tracking System Management defects
only a small number of unit tests are available, it has not yet played a significant role.
novice:
the minimum unit of work includes manual testing.
the team has benefited from accumulating a certain number of unit tests.
run the automatic regression test at build time.
manual participation in the testing process.
automated function testing plays a certain role in ensuring a certain number of tasks.
the complete automated function test works frequently, not less than once a day.
moderate:
the minimum unit of work includes automated testing.
automated tests at different levels play the role of quality assurance.
static code check and related measures
you can recommend a beta version during construction
automated testing
advanced:
general unit testing, make good results.
the automated test coverage rate is high, and the test work is effectively dispersed in the development stage.
most manual tests are Exploratory tests.
crazy:
unit test coverage rate of 100%.
automated testing provides reliable quality assurance that is automatically deployed after successful building.

learn more about the team building status through the following questions:
what levels of tests are available?
what is the test submission process? How long does it take? How many people are involved?
what is the proportion of the centralized test phase to the entire project cycle?
what is the cooperation process between QA and RD?
what are the differences between automated testing, quantity, and quality?
when is automated testing generally written, maintained, managed, and run?
when and how to perform regression testing?
what are the status quo of automated acceptance testing, performance testing, and security testing?
How do I use static code checks?
what is the unit test coverage rate?
when and what manual tests?
How do I choose which build to test?
5 deployment and release dimensions
entry-level:
manual deployment with support for auxiliary scripts
manual launch process based on the documentation
beginner:
complete deployment script support
standard deployment to the test environment
semi-automatic launch process through the platform
moderate:
select a specified build output for automatic deployment
you can recommend a build as a release candidate version.
advanced:
Publish a key to the production environment, one-click rollback.
automated verification after deployment to the production line.
crazy:
restore production environment with one click

learn more about the team's deployment and release status through the following questions:
what is the launch process?
do I need to pass online steps or parameters through work documents, emails, or an online system?
how to monitor the deployment quality?
How do I perform rollback?
how to deploy data to the development environment? What are the steps? How many people are required?
how to deploy the service to the test environment? What are the steps? How many people are required?
are there scripts for deployment?
what are the differences between the deployment environments of team members?
How do I select an appropriate building for deployment?
6. Team habits
Getting Started:
at least one person knows the build status at any time
regular code submission habits
dedicated personnel maintain the continuous integration platform and script
beginner:
professional care platform construction status
the minimum work unit is merged to the target branch immediately after completion.
all users are aware of the current build status.
the building fails and is repaired in time or roll back, no code is submitted that is irrelevant to the repair build during the failure.
perform local automated verification before checking in
Team members perform the same local verification before checking in the Code
failed build overnight
moderate:
failed build repair takes less than half an hour.
the creator is responsible for repairing the failed build. Everyone is concerned about the build status.
Team members write comprehensive unit tests.
each person submits at least one valid submission to the target branch every day for the minimum unit of work.
the team understands the continuous integration platform and script content, which can be maintained by everyone.
advanced:
the delivery team is responsible for the stable construction of the continuous integration platform.
the delivery team is responsible for continuous integration of script development.
crazy:
A valid submission to the target branch about one hour for the minimum unit of work, and few build failures are found.

Learn more about the team's deployment and release status through the following questions:
How does RD define the meaning of work completion?
What are the code standards for Team check-in?
Are you sure you want to run the local build before checking in the code?
How can we ensure that the failed platform is built and handled by someone? Is it handled by the person who checked in the code?
What is the frequency of building failures?
Who maintains the automated build script in the team?
Who maintains the CI platform in the team? What permissions does the CI platform have?
How can the team improve the unit testing level for each person?

(Author: http://hi.baidu.com/baiduqa/blog/item/73173aed4e2ab8f6cf1b3e6a.html: wenjian)