What does a 10-fold difference in programmer programming productivity mean?

In software engineering research, the most proven conclusion is that for two different programmers with the same experience, there may be a 10-fold gap in efficiency and quality. The researchers also found that this gap also applies to the team level, that is, the same team in the same industry.

Changes in personal efficiency in software development

First, we discovered a huge gap between different people in programming productivity, which was completed by sackman, Erikson, and grant in 1960. They studied professional programmers with an average of 7 years of working experience and found that the ratio of time for the best and worst programmers to write new Code was; the number of debugging times was; the program size is, and the execution efficiency is. They also found that the programmer's experience has nothing to do with code quality or efficiency.

After studying the results of sackman, Erickson, and grant in detail, we can find that the methods they use have many defects, for example, developers who use low-level programming languages and high-level programming languages are combined for research. However, even taking these defects into account, their data still shows that the gap between the best programmers and the worst programmers can be more than 10 times.

After that study, many other studies on professional programmers have proved that the programmer level is also divided into three levels (for details, see references in this book ).

In addition, many rumors also support this idea. When I was still working at Boeing In The Middle Of 1980s, a project team of about 80 programmers was facing the risk of failing to complete a key task on time. This project is very important for Boeing, so they replaced half of the 80 people in the project with another person, and the man-in-Law completed all programming work, the software was delivered on time. I didn't work in this project group or know this genius, but this story was told by someone I trusted, so I believe it is true.

This gap is not limited to the software industry. A study by norm Augustine pointed out that there is a situation in all walks of life, including writing, rugby, invention, and police work, that is, the output of the top 20% talents in the industry accounts for 50% of the total output of the industry, whether these outputs are scores, patents, detected cases or software. You can think about it. It still makes sense. We all know that some students are superior to other students, as do athletes, artists, and even parents. Since this difference exists in all groups of people, how can software development be exceptional?

Negative effects of huge gaps

Augustine's research found that some people do not have any substantial contribution (for example, a forward who cannot score, an inventor without a patent, or a detective who cannot solve the case ), the actual gap between people may be larger than the data mentioned above.

This seems to be the case in the software industry. About 10% of the lab participants were unable to complete the lab tasks in several published research projects on software development efficiency. These research reports often say, "so we excluded the results of these participants from the dataset ." But in real life, if someone "cannot complete the task", you cannot simply "exclude their results from the data set. You may have to wait for them to complete, or assign another person to complete their work. There is an interesting (and terrible) hint, that is, in the software industry, almost 10% of people contribute negative to project output.

As before, this is also consistent with our feelings in real life. I believe that many people will be able to find people who have worked with each other that match this description.

What makes a true "10 times programmer"

Many people do not like to be labeled as "10 times" because they think people will say, "We have a super programmer in our team who is cool, everyone does not want to communicate with him. Without him, the overall efficiency will be higher."

Generally, any practical definition of 10 times the programmer must consider the impact of such a programmer on other members of the team. I also know that there are cool super programmers. However, in more cases, the super programmers are just average programmers at the ordinary level, and even cannot reach the ordinary level. They just use a cool appearance to make their performance look less bad. The real super programmers I 've worked with, in addition to their technical skills, usually have a good team spirit (although sometimes there are some exceptions ).

Measure the personal productivity of programmers

As many studies have pointed out that the efficiency of different programmers may be 10 times different, many people have come up with the idea of measuring their personal efficiency in their own organizations. In any case, this idea involves measuring the productivity of "active" programmers and the productivity mentioned in general research is very different.

Software Engineering Research typically uses the time required to complete a task, how many lines of code can be written per hour or per month, or other criteria to measure productivity. But if you try to use these standards to measure productivity in a business environment, you will encounter many problems.

Productivity = number of lines of code generated every month

Software design is a non-deterministic process. For the same problem, different designers/developers will make completely different solutions. If we use the number of lines of code produced every month (or similar standards) to measure productivity, by default, programmers who use 10 times of code to solve the same problem have 10 times of productivity. Obviously, this is not always the case. For example, a programmer may have a wonderful design idea. As a result, only 10% of the code is used to solve the problem that ordinary programmers need 100% of the Code to solve.

Some people have asserted that the code written by Great programmers is always shorter. In fact, there may be some relationship between the programming level and the simplicity of the code, but I do not want to draw such a broad conclusion. I just want to say that great programmers always try to write the code clearly, and the result is usually a shorter code. However, sometimes the clearest, simplest, and most obvious design requires a little more code than the more "clever" design. In this case, I think Great programmers will also use a little more code to avoid overly clever design. In any case, there is a problem with the idea of "number of lines of code produced every month" to measure productivity.

There is a story in the dilbert cartoon: The boss said that every time a bug is found, he will be rewarded with 10 yuan. Everyone shouted that he had made the money this time, and someone else wanted to "Write a minivan" in this way. The story illustrates this problem, that is, if you use the code output to measure productivity, some people will use this to write a lot of code that may not be necessary at all. The problem here is not the "amount of code" standard, but the old management philosophy, that is, "people only do things that will be investigated ". But you must be careful not to investigate the error.

Productivity = function?

Some of the problems caused by "code output every month" can be measured by the functional points. The feature is a set of "Synthetic" standards for measuring program sizes. The input, output, query, and file quantity are all taken into account as parameters for determining the program size. Inefficient design/programming styles do not produce more functional points, so the functional points standards do not involve the amount of code. However, it has a more practical problem, that is, you need professionals to calculate Function Points (many companies do not have such talents), and the functional points and personal outputs correspond very roughly, therefore, it cannot be used to determine the personal productivity of programmers.

Complexity

Managers often say: "I always give the most difficult and complex programming tasks to the best programmers. Therefore, no matter how it is measured, his productivity seems to be always lower than others, but it may take two times to let others do the same thing ." This is normal, but it also affects how we define and measure productivity.

Is there a way to measure personal productivity?

The difficulties mentioned above make many people think that it is difficult to measure personal productivity, and no one can do it. However, I think it is possible to correctly measure personal productivity, but pay attention to the following points.

• Do not expect to be able to understand the actual overall situation of personal productivity with a single measurement criterion
  You can refer to the statistics collected in sports competitions. We cannot even use a single standard to determine the level of the battery in a baseball match. We must consider hitting rate, full base hitting, base running score, base uploading rate, and other factors. The data alone is not enough. We have to prove the significance of the data. If the batchers' advantages and disadvantages cannot be judged by simple standards, is it possible for programmers to have such complicated personal productivity? What we should use is not a separate standard, but a set of standard combinations. This set of combined standard tasks gives us a deeper understanding of personal productivity. For example, this set of criteria may include the percentage of tasks completed on time and the score of managers 1 ~ 10. colleagues' scores: 1 ~ 10. The number of lines of code generated each month, the average number of defects in each line of code, the ratio of improper fixing, and so on.
• Do not think that as long as there is a certain standard (either individually or in combination), you can carefully identify the productivity of different individuals.
  Remember: these personal productivity criteria only help you identify problems, but do not answer these questions. Improper use of these standards, such as performance evaluation, will not only bring about management problems, but also cause statistical problems.
• The overall trend is often more important than the measurement results at a certain time point.
  It is often impossible to draw any meaningful conclusion by comparing these measurements across different individuals. A more useful way is to analyze the measurement results of a person vertically to see if the person has made progress over time.
• Ask yourself: Why is my personal productivity measured?
  In the research environment, researchers need to evaluate the efficiency of different technologies, so they need to measure personal productivity. Compared with the research environment, there are much more problems arising from the use of the same measurement standards in real projects. In a real project environment, what do you want to do with these measurement standards? Performance evaluation? This idea is not good, because it was just said. Assign a task? But most of the managers I 've visited say they don't have to measure and know who is a star member of their team. I also believe this. Budget? No. The gaps caused by different design methods, the difficulty of different tasks, and other related causes make it impossible for us to effectively use these standards for project budget.

In real projects, it is difficult to find a standard of personal productivity that is beneficial to project management and conforms to statistical rules. In my experience, apart from doing research, people often come from statistical conclusions that do not make sense of personal efficiency. That is to say, although I know that the measurement of personal efficiency is very meaningful in my studies, I think it is difficult to find its proper use in actual projects.

Team productivity gap in software development

Software experts have long discovered that the team productivity gap is as large as the individual productivity gap, which is an order of magnitude [11]. This is partly because things are clustered by groups, as evidenced by a study of 166 professional programmers from 18 organizations.

For another example, in a study of seven identical projects, the researchers found that the most energy-consuming teams were 3.4 times the lowest, and for the size of programs, the maximum value is 3 times [3]. Although there is a gap in productivity, the programmers in this study come from a similar background. They are both professional programmers and have many years of experience. We can reasonably speculate that if the background difference between the subjects is greater, the gap between them will be larger. An early study of the programming team once pointed out that for the same project, the ratio of program size submitted by different teams can reach, the ratio of the time required can reach 2.6: 1 [15].

Researchers such as Barry Boehm studied data over 20 years to establish the cocomo II cost estimation model and concluded that for the same procedure, the time required to score more than 15 teams is 3.5 times that of the 90 + teams (calculated as 100 points ). If a team is more experienced than another team in the programming language or application field, the gap will be even greater.

A specific example is Lotus 1 ~ 2 ~ 3 productivity gap between the third edition and Microsoft Excel 3.0 development team. Both are in 1989 ~ 1990 desktop workbook applications released during this time period. Since we rarely see data on similar projects published by two companies, the comparison between these two competitors is particularly interesting. The data for the two projects is as follows: the Excel staff spent a total of 50 work years and wrote 649 lines of code [8]. Lotus 1 ~ 2 ~ 3. A total of 260 lines of code [13] were written in 400 years. The Excel team generates lines of code each year. The Lotus team produces only 1500 lines of code each year. The productivity gap between the two teams exceeds 8 times, which proves our previous claim that there is also a gap between the team's productivity and a larger magnitude gap.
Interestingly, these quantitative results are very close to outsiders who feel about these projects. At that time, the third edition of Lotus 123 became the best vote winner and was released at least two years later than expected. In Microsoft, Excel is praised and hailed as one of Microsoft's most successful projects. For real projects of a real company, I am afraid this kind of comparison can be achieved to the extreme.

As mentioned above, this example shows the factors that cause the productivity gap. Both Lotus and Microsoft have painstakingly recruited top-level talents for their respective projects, so I suspect that the gap in team productivity is not only caused by the gap in the ability of team members, there are also many factors involved in the organizational structure, such as whether the product vision is clear, whether the customer needs are clear, and whether Members can work together.

Organizational factors may affect the productivity of the team. An outstanding team with mediocre personal abilities in an organization can surpass an outstanding team with excellent personal abilities in a mediocre organization. Of course, there is no combination like an outstanding organization + an outstanding team or a mediocre organization + a mediocre team. In this case, it is not surprising that the team productivity (or organization productivity) is 10 times different from the individual productivity.

This article is excerpted from "software path: Analysis of Software Development disputes"