20th back to test Risk Management

Test risks are inevitable and always exist. Therefore, it is very important to manage test risks. We must try our best to reduce the risks in the test, ensure the quality to the maximum extent, and meet the customer's needs. The main risks involved in testing are:

1. Inaccurate understanding of quality requirements or product characteristics leads to errors in testing scope analysis. The results are not tested in some places or the verification criteria are incorrect;
2. Test cases are not executed by. For example, some test cases are intentionally or unintentionally omitted;
3. Temporary/sudden changes in requirements lead to design modifications and code rewriting, and the test time is not enough;
4. Quality standards are not very clear, such as applicability testing, benevolent and wise;
5. Test Case design is not in place, ignoring some boundary conditions, deep logic, and user scenarios;
6. The test environment is generally not exactly the same as the actual running environment, resulting in errors in the test results;
7. Some defects are not frequently discovered. If the code quality is poor and there are many Software defects, the possibility of missing defects is high;
8. Regression testing generally does not run all test cases. It is selective and brings risks.

The first three risks can be avoided, and 4) to 7) The four risks cannot be avoided, which can be minimized. The last regression test risk can be avoided, but it usually exists out of time or cost considerations.

There are some effective test risk control methods for the risks of the above software testing, such:

• If the test environment is incorrect, you can list all entries to be checked in advance. After the test environment is set, other personnel will check the items listed one by one;
• Some test risks may have very serious consequences. Can they be converted to other low risks that do not cause serious consequences. For example, on the eve of product release, a serious defect is found in a new function that is not very important. If this defect is corrected, it may lead to a defect in the original function. At this time, the risk of dealing with this defect is very high. The countermeasure is to remove the new feature (diasble) and transfer this risk;
• When some risks are inevitable, we try to reduce the risk. For example, the risk of "undiscovered defects in the Program" always exists. We need to improve the test case coverage rate (for example, 99.9%) to reduce this risk;

In order to avoid, transfer or reduce risks, risk management plans and risk control strategies should be prepared in advance, and some emergency and effective solutions should be formulated for risk handling, such:

• Leave room for estimation of resources, time, and cost, instead of 100%;
• Before the project starts, include some elements that may change or are difficult to control in the risk management plan;
• Train reserve personnel for each key technical personnel, prepare for the movement of personnel, and take some measures to ensure that once the personnel leave the company, the project will not be seriously affected, and the project can continue;
• Develop document standards and establish a mechanism to ensure that documents are generated in a timely manner;
• Conducts mutual review of all tasks and identifies problems in a timely manner, including exchange of different testers on different test modules;
• Routine tracking of all processes to promptly detect signs of risks and avoid risks.

To avoid risks, you must completely change the management mode of the test project. To avoid risks of the test, you must establish a management awareness that prevents risks from happening before they occur. Compared with traditional software testing, the whole-process test management method can not only effectively reduce product quality risks, in addition, software product defects can be avoided in advance, defect feedback cycle can be shortened, and the testing cycle of the entire project can be shortened.

Reference: Very effective policies during test execution

Supplementary materials: risk management methods

Risk management can be divided into five steps: risk identification, risk analysis, risk planning, risk control and risk tracking.

1. Risk Identification
Risk identification is a systematic method to determine the factors that threaten the project plan. The identification methods include risk checklists, brainstorming meetings, flowchart analysis, and project personnel interviews. The first two methods are commonly used. The risk checklist is based on the risks that have been encountered in similar projects in the past. For example, a certain technology is used during development, individuals or project teams that have such technical development experience can point out the problems they have encountered when using this technology; brainstorming sessions can focus on the scope, progress, cost, and quality issues that may arise in the project, and discuss and list possible risks of the project. Specific problems should be analyzed for different projects to identify the risk events that may actually occur on the project.

2. Risk Analysis
Risk analysis can be divided into qualitative risk analysis and quantitative risk analysis. Qualitative risk analysis is a process of evaluating the impact and likelihood of identified risks, so as to sort the risks based on the potential impact of risks on the project objectives. It is important to clarify specific risks and guide risk response. Quantitative risk analysis is to quantitatively analyze the probability of each risk and its impact on the project objectives. It also analyzes the overall risk degree of the project.
The impact of different risks on the project varies, mainly in the following three aspects:

• The nature of the risk, that is, the problems that may arise when the risk occurs;
• The scope of the risk, that is, the severity of the risk and its overall distribution;
• The time of the risk, that is, when the risk can be felt and how long the risk remains.

Accordingly, the weighted coefficient of risk estimation is determined to obtain the project risk estimation. Then, by quantifying, selecting, and sorting risks, you can know which risks must be addressed, which are acceptable, and which are negligible. For risk management, we should focus on those risks with a large influence, a wide range of impacts, a high probability, and possible stages.

3. Risk plan
The responsibility, resources, time, activities, response measures, results, and owner should be taken into account when preparing a risk action plan. Establishing a warning threshold is one of the main activities in the risk planning process. The threshold is closely integrated with the quantitative goal in the project, and the warning level of the target is defined.
This phase involves different types of plans, such as reference plans, benchmark plans, and contingency plans.

• The reference plan is a reference point for comparison with the current suggestion.
• The baseline plan is the basis of the proposed plan and the initial position of the proposed project implementation.
• The contingency plan is a suggested supplementary plan established on the basis of the benchmark plan, including the triggering point for initiating emergency response measures.

At this stage, there are specific tasks such as consolidation and interpretation, selection and refinement, and support and persuasion.

4. Risk Control Methods
The methods used mainly include risk avoidance, risk weakening, risk bearing and risk transfer.

• Risk Avoidance: the risk or risk trigger conditions are eliminated by changing the software project plan to protect the target from impact. This is an advance risk response strategy. For example, adopt more mature technologies, clarify ambiguous requirements, increase resources and time, reduce the scope of work of the project, and avoid unfamiliar subcontractors.
• Risk weakening: reduce the probability or result of a risk event to an acceptable level. Of course, the probability reduction is more effective. For example, select a simpler development process, perform more system tests, develop software prototype systems, and add backup designs.
• Risk Exposure: accept risks and consider how to deal with the risks after they occur without changing the Project Plan (or having no appropriate policies to cope with the risks. For example, contingency plans, risk response procedures, and even emergency reserves and monitoring are developed to respond to emergencies at random. In practice, if a software project is in progress and some people want to leave the project team, they can develop emergency plans to ensure that reserve personnel are available and determine the procedures for project team members to leave, and Procedures for the handover.
• Risk Transfer: instead of eliminating risks, the results of software project risks, together with the power to respond, are transferred to a third party (a third party should be aware of the risks and be able to withstand them ). This is also a pre-approval strategy, such as signing different types of contracts or signing compensatory contracts.

5. Risk tracking
After the risk is controlled, We must track the risk in a timely manner and track the risk well:

• Monitors the status of risks, such as whether the risks have occurred, exist, or disappear;
• Check whether risk countermeasures are effective and whether the tracking mechanism is running;
• Constantly identify new risks and develop countermeasures.

You can use the following methods to effectively track risks:

• Risk audit: The Project Administrator should help the project team check whether the monitoring mechanism is implemented. The project manager should carry out regular risk reviews, especially in key areas of the project, and track major risk factors.
• Re-assess risks; Develop new response plans for unanticipated risks.
• Deviation Analysis: the project manager should regularly compare with the benchmark plan to analyze the deviations in cost and time. For example, failure to complete on schedule or budget exceeding are potential problems.
• Analysis of Technical Indicators: The analysis of technical indicators mainly compares the differences between the original and actual technical indicators, for example, the test fails to meet the performance requirements.

To predict the future, please read the next decomposition: comprehensive application of the 21st Test Case Design Method

Copyright, software testing

-For the series of discussions, see: Software Testing romance-intermediate and advanced series (sequence)