Some problems existing in CMM evaluation

Part of this article refers to an article in the February 2000 IEEE Journal. The author analyzes several problems in SCE from the purchaser and software enterprise, and finally asks: "Is CMM evaluation credible?" "The CMM evaluation of Chinese software enterprises is full of impetuous, show and utilitarian atmosphere at the beginning." The whole process of CMM evaluation, we see is greatness government industry competent department, has always loved to join in the news media, have to earn white not profit intermediary agencies, certificate with the Director of the Evaluation division and is willing to pay to buy a yell software business. CMM evaluation of this strong utilitarian, so that "gaming the assessment" has become a software enterprise up and down the consensus and "not to announce the secret." I hope this article on the domestic software industry to correctly treat the CMM should play a role, do not like some software companies to find a "certificate with", "hand-paid, hand-handed" director of the evaluation Division. After all, to improve the core competitiveness of enterprises is the most important.

1 Qualification of Evaluation personnel

SEI's qualification requirements for the Director's assessor are stricter, but the assessment is carried out by an assessment team.

Question 1: It is difficult to assess the qualifications of team members.

Generally, it is difficult for the evaluation team to achieve the technical and management background of the software engineering required by SEI.

Question 2: The evaluators are mostly managers, and the technical quality is generally weak.

This issue also includes the Director Assessor, who has been working out of the software development practice for many years, and the Director Assessor has even deliberately avoided technical questions while asking questions, but repeatedly asked management questions. Because the technical quality of the evaluators is generally weak, the key practical interpretation ability of the CMM involving the engineering part is poor. Even in interviews with employees, when the employee mentions technical issues, the evaluators diverge from the topic and move on to management issues.

2 Time pressure of assessment

The general assessment of the time is about a week, to carry out a considerable amount of work, time pressure is very high. The original assessment method required a single interview, and later was pressured by time, and the new version added "group Interview" (Group interview).

Question 3: Group interviews are supposed to save time, and in fact they tend to obscure the problem and not find the real situation.

The interview of Lenovo Testing department is the way of group interview.

For the assessment team, due to time constraints, they are often unwilling to use their insights and professional judgment, but satisfied with the checklist on a tick, which is commonly known as "Checklist obsessive-compulsive disorder."

Question 4: "Checklist obsessive-compulsive disorder" also gives software companies a chance to exploit.

Software companies can work hard to make documents more "easier for the assessment team to work".

3 CMM Questionnaire

The same set of questionnaires is used in CMM evaluation. Even Americans criticize that "it is unthinkable to use the same set of tests for doctors, architects, or lawyers, but for the evaluation of software companies, there is a sure set of unchanging questionnaires." Think about it, these software companies may be in the development of our defense system! ”

Question 5: The same set of maturity questionnaires.

The CMM evaluation allows companies to train their employees before they are evaluated, and the role of the training is often distorted. Many software companies even "abet" the staff in the maturity Questionnaire on how to answer the "more mature" point.