What is software design?

Source: http://www.uml.org.cn/c%2B%2B/200810132.asp

So far, I can still remember where I was when I had an epiphany and finally produced the following article. That was the summer of 1986. I served as a temporary consultant at the China Lake Naval Weapons Center in California. During this period, I had the honor to attend a seminar on Ada. During the discussion, an audience raised a representative question: "Are software developers engineers ?" I don't remember the answer at the time, but I still remember that I didn't really answer this question at the time. So I quit the discussion and began to think about how I would answer this question. Now, I cannot be sure why I remembered a paper I had read in Datamation magazine almost 10 years ago, but what prompted me to remember was something that was discussed later. This paper explains why engineers must be good writers (I remember this paper is about this problem-I haven't read it for a long time), but the key point I got from this paper is: the author believes that the final result of the engineering process is a document. In other words, engineers produce documents, not physical objects. Others make physical objects according to these documents. As a result, I raised a question in my confusion: "Apart from all documents normally generated by software projects, are there other things that can be considered as real engineering documents ?" My answer is, "Yes, there is such a document, and there is only one source code ."

Taking the source code as an engineering document-design-completely subverts my views on my chosen career. It changes the way I view everything. In addition, the more I think about it, the more I think it clarifies the many problems that software projects often encounter. More specifically, I think most people do not understand this different view, or intentionally reject the fact that it is enough to explain many problems. A few years later, I finally had the opportunity to publish my opinion publicly. A paper on Software Design in C ++ Journal prompted me to write a letter to the editor about this topic. After several correspondence exchanges, the editor Livleen Singh agreed to publish my thoughts on this topic as a paper. The following is the article.

What is software design?

Object-oriented technology, especially C ++, seems to have brought a great shock to the software industry. A large number of papers and books have emerged to describe how to apply this new technology. In general, questions about whether object-oriented technology is just a scam have been replaced by questions about how to make the smallest effort to get the benefits. The emergence of object-oriented technology has been around for some time, but this explosive popularity seems a bit unusual. Why do people suddenly pay attention to it? People have given a variety of explanations for this problem. In fact, there is probably no single reason. Maybe we can make a breakthrough only by combining multiple factors, and this work is in progress. Even so, in the latest phase of the software revolution, C ++ itself seems to be a major factor. Similarly, there may be many reasons for this question, but I want to give a slightly different answer: C ++ becomes popular, it makes software design easier and programming easier.

Although this explanation seems a bit strange, it is the result of careful consideration. In this paper, I want to focus on the relationship between programming and programming. Over the past 10 years, I have never felt that the entire software industry is unaware of a subtle difference between software design and what is true software design. As long as we see this, I think we can learn more about how to become a better software engineer from the popularity of C ++ growth. This knowledge is that programming is not building software, but designing software.

A few years ago, I saw a seminar discussing whether software development is an engineering discipline. Although I do not remember the results of the discussion, I remember how it prompted me to realize that the software industry has made some errors and compared the hardware engineering, while ignoring some absolutely correct comparisons. In fact, I don't think we are software engineers, because we don't realize what is true software design. Now, I am more confident about this.

The ultimate goal of any engineering activity is some type of documentation. When the design work is completed, the design documents are transferred to the manufacturing team. The team is a completely different group from the design team, and its skills are also completely different from the design team. If the design document correctly depicts a complete design, the manufacturing team can start building the product. In fact, they can start to build many objects of the product without any further designer intervention. After reviewing the software development lifecycle in my understanding, I came to the conclusion that the only software document that actually meets engineering design standards is the source code list.

People have made a lot of arguments about this point of view. Both those who agree and those who disagree are enough to write countless papers. This article assumes that the final source code is the real software design, and then carefully studies the results of this assumption. I may not be able to prove this point, but I want to prove that it does explain some of the observed facts in the software industry, including the popularity of C ++.

When we regard code as the result of software design, one result completely overwrites all other results. It is very important and obvious, and because of this, it is completely a blind spot for most software organizations. The result is that software construction is cheap. It is not expensive at all; it is very cheap and almost free. If the source code is software design, the actual software build is completed by the compiler and connector. We often refer to the process of compiling and connecting to a complete software system as "building once ". The main investment in software building devices is very small-only one computer, one editor, one compiler, and one connector are actually needed. Once you have a Build Environment, it takes a little time to build the actual software. It may take a long time to compile a C ++ program with 50 000 lines, but how long does it take to build a hardware system with the same design complexity as 50 000 lines of C ++ programs?

Taking the source code as another result of software design, software design is relatively easy to create, at least mechanically. Generally, writing (that is, designing) a representative software module (50 to 100 lines of code) takes only a few days (fully debugging it is another topic, it will be discussed later ). I would like to ask if there are any other disciplines that can generate designs with the same complexity as the software in such a short period of time. However, first, we must figure out how to measure and compare complexity. However, it is obvious that software design can become very large quickly.

Assuming that software design is relatively easy to create, and there is no cost to build it in essence, it is not surprising that software design is often incredibly large and complex. This seems obvious, but the importance of the problem is often ignored. Projects in schools usually have thousands of lines of code. Software products with 10 million lines of code (Design) are also discarded by their designers. We have long been concerned about simple software. The Design of typical commercial software is composed of hundreds of thousands of lines of code. Many software designs have millions of lines of code. In addition, software design is almost always evolving. Although the current design may only have thousands of lines of code, in the product life cycle, it may actually need to write many times of code.

Although there are some hardware designs that seem as complicated as software design, pay attention to two facts about modern hardware. First, the results of complex hardware engineering may not always be correct. At this point, it does not have the criteria that trust us like software. Most of the microprocessors were sold with logic errors: bridge collapse, dam breakdown, plane crashes, and thousands of cars and other consumer goods recalled-all of which are new to us, all of these are results of incorrect design. Second, the complex hardware design corresponds to the complex and expensive build phase. As a result, the capabilities required to make such a system limit the number of companies that can actually produce complex hardware design. There is no such restriction for software. Currently, hundreds of software organizations and thousands of extremely complex software systems exist, and the number and complexity of software systems are growing every day. This means that the software industry cannot find a solution to its own problems by imitating hardware developers. If something is the same, it means that when CAD and CAM can help hardware designers create more and more complex designs, hardware engineering will become more and more like software development.

Designing Software is an activity that manages complexity. Complexity exists in the software design itself, in the company's software organization, and in the entire software industry. Software design is very similar to system design. It can span multiple technologies and often involves multiple branch disciplines. The specification of the software is often not fixed and rapidly changing. This change often occurs when the software is being designed. Similarly, software development teams are often not fixed and often change in the middle of the design process. In many ways, software is more complex than hardware in society or organic systems. All of these make software design a difficult and error-prone process. Although none of these are creative ideas, today, nearly 30 years after the start of the software engineering revolution, compared with other engineering industries, software development still looks like an untrained (undisciplined) skills.

The general opinion is that when a real engineer completes a design, no matter how complicated the design is, they are very confident that the design can work. They are also very confident that the design can be built using recognized technologies. To do this, hardware engineers spend a lot of time verifying and improving their designs. For example, consider a bridge design. Before such a design is actually built, engineers perform Structural Analysis-they build computer models and perform simulation, and they build proportional models and perform tests in wind tunnels or using other methods. In short, designers will use all the methods they can think of to prove that the design is correct before building. The design of a new aircraft is even more serious; a prototype of the same size as the original must be built, and flight tests must be carried out to verify the expectations in the design.

For most people, there is obviously no project as strict as the hardware design. However, if we regard the source code as a design, we will find that software engineers have actually done a lot of verification and Improvement on their design. The software engineer calls it test and debugging instead of engineering. Most people do not regard testing and debugging as true "engineering"-it is certainly not seen in the software industry. The reason for this is that the software industry rejects code as a design, rather than any actual engineering difference. In fact, the test model, prototype, and circuit test board have become recognized components of other engineering disciplines. Software designers do not have or do not use more formal methods to validate their designs because of the simple economic pattern of the software build cycle.

The first revelation: Building a design and testing it is cheaper and easier than doing anything else. We don't care how many builds are done-these builds have almost zero cost of time, and if we discard the build, the resources it uses can be completely reused. Please note that testing is not only about making the current design correct, but also part of the process of improving the design. Hardware engineers of complex systems often build models (or, at least, they use computer graphics to visually present the design ). This gives them a "feeling" of design, which is not possible simply by checking the design. For software, building such a model is neither impossible nor necessary. We only build the product itself. Even if formal software verification can be performed automatically like a compiler, we will still perform build/test loops. Therefore, formal verification has never had much practical significance for the software industry.

This is the reality of today's software development process. A growing number of people and organizations are creating more complex software designs. These designs are first written in some programming languages and then verified and improved through Build/test cycles. The process is prone to errors and is not very strict. A considerable number of software developers do not want to believe that this is the way the process operates. This makes the problem more complex.
At present, most software processes try to separate different stages of software design into different categories. Encoding starts only after the design at the top layer is completed and frozen. Testing and debugging are only necessary to clear build errors. Programmers are in the middle. They are construction workers in the software industry. Many people think that if we can stop programmers from "hacking" and build them according to the design they have given (and even make fewer mistakes in the process ), then software development can become mature and become a real engineering discipline. However, this is impossible as long as the process ignores engineering and economics facts.

For example, any modern industry cannot tolerate a return rate of more than 100% during its manufacturing process. If a builder often fails to build correctly for the first time, he will soon be unemployed. However, in the software industry, even the smallest piece of code may be corrected or completely rewritten during testing and debugging. In a creative process (such as design), we acknowledge that such improvement is not part of the manufacturing process. No one expects the engineer to create a perfect design for the first time. Even if she does, it must still undergo the improvement process to prove that it is perfect.

Even if we have not learned anything from the Japanese management method, we should also know that blaming the workers for making mistakes in the process is not conducive to productivity improvement. We should not constantly force software development to conform to incorrect process models. On the contrary, we need to improve the process so as to help rather than hinder the generation of better software. This is the core test of "Software Engineering. The project is about how you implement the process, not about whether a CAD system is required to generate the final design document.

An overwhelming question about software development is that everything is part of the design process. Coding is the design, testing and debugging are part of the design, and we usually think that the design is still part of the design. Although software is cheap to build, it is incredibly expensive to design. The software is very complex and has many different design contents and the design considerations caused by them. The problem is that all different aspects are interrelated (just like in hardware engineering ). We hope that the top-level designers can ignore the details of the module algorithm design. Similarly, we hope that programmers do not have to consider the top-level design issues when designing the internal algorithms of the module. Unfortunately, problems at the design level intrude into other layers. For the success of the entire software system, selecting algorithms for a specific module may be equally important as any high-level design issue. There is no importance level in different aspects of software design. An incorrect design in the lowest layer module may be as fatal as an error in the highest layer. Software design must be complete and correct in all aspects; otherwise, all software built on the basis of the design will be wrong.

Software is designed hierarchically to manage complexity. When a programmer considers a detailed design of a module, there may be hundreds of other modules and thousands of details, which cannot be taken into account at the same time. For example, some important aspects of software design are not completely in the scope of data structures and algorithms. Ideally, programmers should not consider these other aspects of design when designing code.

However, the design does not work in this way, and the reasons also begin to become clear. Software design is completed only after it is written and tested. Testing is a basic part of the design verification and improvement process. The Design of high-level structure is not a complete software design; it is only a structural framework of detailed design. Our capabilities are very limited in strictly verifying high-level design. The detailed design will eventually have at least the same impact on the high-level design as other factors (or should be allowed ). Improvements to all aspects of the design should be carried out throughout the design cycle. If any aspect of the design is frozen out of the improvement process, it will not be strange if the final design will be poor or even unable to work.

It would be nice if the high-level software design could be a more rigorous engineering process, but the real situation of the software system is not strict. Software is very complicated, and it depends on too many other things. Maybe some hardware does not work as the designer thinks, or a library routine has a limitation that is not described in a document. Each software project will encounter these types of problems sooner or later. These types of problems will be discovered during the test (if our test work is good) because there is no way to discover them early. When they are discovered, they force changes to the design. If we are lucky, the design changes are partial. From time to time, changes will affect some important parts of the entire software design (Mo Fei's Law ). When a part of the affected design cannot be changed for some reason, other parts of the design must be damaged to adapt to the impact. This usually leads to the "arbitrary coding" that managers think, but this is the reality of software development.

For example, in A project I recently worked on, I found A time series dependency between the internal structure of module A and Module B. Worse, the internal structure of module A is hidden behind an abstract body, however, this abstract body cannot merge calls to Module B into its correct call sequence in any way. When the problem is discovered, of course, it is time to change the abstract body of. As expected, what happened was to apply an ever-growing and complex "correction" set to A's internal design. When we haven't installed version 1, we generally feel that the design is declining. Each new correction may damage some old ones. This is a formal software development project. Finally, my colleagues and I decided to change the design, but we had to work overtime on a voluntary basis to get the consent of the management.

In any general-Scale Software Project, problems like this will certainly occur. Although people use various methods to prevent it from appearing, some important details will still be ignored. This is the difference between technology and engineering. If experience can lead us to the right direction, this is the process. If experience only brings us into an unknown field, then we must use the method used at the beginning and make it better through a controlled improvement process, which is engineering.

Let's take a look at it as a very small part. All programmers know that writing software design documents after coding will produce more accurate documents. The reason is obvious. The final design represented by code is the only thing that is improved during build/test cycles. During this cycle, the possibility that the initial design remains unchanged is inversely proportional to the number of modules and the number of programmers in the project. It will soon become worthless.

In software engineering, we need excellent design at all levels. We need excellent top-level design. The better the initial design, the easier the detailed design will be. Designers should use anything that can help. Structural charts, Booch charts, status tables, PDL, and so on-use it if it can help. However, we must remember that these tools and symbols are not software designs. Finally, we must create a real software design that is completed in a programming language. Therefore, when we come to the design, we should not be afraid to encode them. When necessary, we must be willing to improve them.

So far, there are no design symbols available for both top-level design and detailed design. The design will eventually show code written in a certain programming language. This means that the top-level design symbols must be converted to the target programming language before the detailed design can begin. This conversion step is time-consuming and will introduce errors. Programmers often review the requirements and re-design the top layer based on their actual needs, instead of converting from a symbol that may not be fully mapped to the selected programming language. This is also part of the reality of software development.

Maybe it would be better if we asked the designer to write the initial code, instead of letting others change the language-independent design. What we need is a unified symbol for each level of design. In other words, we need a programming language that also applies to capturing high-level design concepts. C ++ meets this requirement. C ++ is a programming language suitable for real projects and a very expressive software design language. C ++ allows us to directly express high-level information about the design components. In this way, the design can be easier and the design can be improved more easily in the future. Because it has a more powerful type check mechanism, it also helps to detect design errors. This leads to a more robust design, which is actually a better engineering design.

Finally, software design must be presented in a certain programming language, and then verified and improved through a build/Test Loop. Any other claims are useless. Consider which software development tools and technologies are available. Structured Programming is regarded as a creative technology in its age. Pascal made it popular and became popular. Object-Oriented Design is a new popular technology, and C ++ is its core. Now, consider the ineffective things. Is CASE tool popular? Yes; General? No. How about structural charts? The same is true. Likewise, there are Warner-Orr, Booch, object, and everything you can think. Each has its own strength and the only fundamental weakness-it is not a real software design. In fact, the only software design symbol that can be universally recognized is PDL, and what does it look like?

This shows that, in the common subconscious of the software industry, it instinctively knows that the improvement of programming technology, especially the programming language used for actual development, is of overwhelming importance over anything else in the software industry. This also shows that programmers are concerned with design. Software developers use a more expressive programming language.

Similarly, consider how the software development process changes. In the past, we used the deployment process. Now we are talking about spiral development and quick prototyping. Although this technology is often considered to be able to "eliminate risks" and "shorten the delivery time of Products", it is actually just to start coding earlier in the software life cycle. This is a good thing. This allows the build/Test loop to begin to validate and improve the design earlier. This also means that the top-level software designers are likely to carry out detailed design.

As shown above, the project is more about how to implement the process than what the final product looks like. In the software industry, we are close to the engineer's standards, but we need some cognitive changes. Programming and building/testing cycles are the center of the engineering software process. We need to manage them in a way like this. Building/testing the Economic Law of the cycle, coupled with the fact that software systems can represent almost anything, makes it impossible for us to find a general method to verify software design. We can improve this process, but we cannot leave it alone.

Last point: the goal of any engineering design project is some document products. Obviously, actual design documents are the most important, but they are not the only documents to be generated. Eventually, some people will be expected to use the software. Similarly, the system may also require subsequent modifications and enhancements. This means that, like a hardware project, auxiliary documents are equally important to software projects. Although the user manual, Installation Guide, and other documents that are not directly linked to the design process are temporarily ignored, there are still two important requirements that need to be addressed using the auxiliary design document.

The first purpose of the auxiliary document is to capture important information from the problem space, which cannot be used directly in the design. Software design requires the creation of software concepts to model the concepts in the problem space. This process requires an understanding of the concepts in the problem space. Generally, this understanding includes information that will not be directly modeled into the software space, but it still helps the designer determine what are essential concepts and how best to model them. This information should be recorded somewhere in case you need to change the model later.

The second important requirement of the secondary document is to record some aspects of the design, which cannot be extracted directly from the design itself. They can be either high-level or low-level content. For many of these aspects, graphics are the best way to describe. This makes it difficult to include them in the Code as annotations. This does not mean that the programming language should be replaced with graphical software design symbols. This is no different from supplementing the graphic design document of the hardware subject with some text descriptions.

Never forget that the source code determines the actual design, not the auxiliary document. Ideally, you can use software tools to post-process source code and generate auxiliary documents. We may expect too much of this. The second case is that programmers (or technical writers) can use some tools to extract specific information from the source code and then document the information in other ways. There is no doubt that it is difficult to manually update such documents. This is another reason to support programming languages that require more expressiveness. Similarly, this is also a reason for keeping such auxiliary documents minimal and making them formal at the end of the project as much as possible. Similarly, we can use some good tools; otherwise, we have to turn to pencils, paper, and blackboard.

Summary:

• The actual software runs on the computer. It is a sequence of 0 and 1 stored in a magnetic medium. It is not a program written in C ++ (or any other programming language.
• A program list is a document that represents software design. In fact, the software design is built with compilers and connectors.
• The cost of building an actual software design is incredible, and it is always cheaper as the computer speeds up.
• The cost of designing the actual software is incredible because the complexity of the software is incredible and almost all steps of the software project are part of the design process.
• Programming is a design activity-a good software design process recognizes this and does not hesitate to code it when it appears meaningful.
• Encoding is more likely to show its meaning than we think. Generally, the design process in the code reveals some omissions and additional design requirements. The sooner this happens, the better the design.
• Because software is very cheap to build, formal engineering verification methods are of little use in actual software development. Building a design and testing it is simpler and cheaper than trying to prove it.
• Testing and debugging are design activities-for software, they are equivalent to the design verification and improvement processes in other engineering disciplines. Good software design processes recognize this and will not try to reduce these steps.
• There are also some other design activities called high-level design, module design, structural design, architectural design, or something like this. Good software design processes recognize this and carefully include these steps.
• All design activities are mutually influential. Good software design processes recognize this and allow design changes, sometimes even fundamental changes, when different design steps show the need,
• Many different software design symbols may be useful-they can be used as auxiliary documents and tools to help simplify the design process. They are not software designs.
• Software development is still a process, not an engineering discipline. This is primarily due to the lack of the rigor required to validate and improve the design's key processes.
• Finally, the real progress of software development depends on the progress of programming technology, which means the progress of programming languages. C ++ is such an improvement. It has gained explosive popularity because it is a mainstream programming language that directly supports better software design.
• C ++ has taken a step in the right direction, but more progress is needed.

Postscript

I was impressed when I reviewed what I wrote almost 10 years ago. The first (and most relevant to this book) is that today, I am even more confident than at that time that the points I tried to elaborate are essentially correct. In the following years, many popular software development methods have reinforced many of these ideas, which support my belief. The most obvious (and perhaps least) is the popularity of object-oriented programming languages. Now, many other object-oriented programming languages are available except C ++. In addition, there are some object-oriented design symbols, such as UML. The reason why I became popular with object-oriented languages was that they allowed to show more expressive design arguments directly in code, and now it seems a little outdated.

Refactoring concept-Reorganizing the code basics, make it more robust and reusable-it should also be similar to my arguments about all aspects of design that should be flexible and allow change during design verification. Refactoring only provides a process and a set of guidelines on how to improve proven design with defects.

Finally, there is a general concept of agile development. Although extreme programming is the most well-known of these new methods, they all have one thing in common: they all acknowledge that source code is the most important product in software development.

On the other hand, there are some points of view-some of which I have mentioned in my thesis-which became more important to me in the following years. The first is the importance of architecture or top-level design. In this paper, I think the architecture is only part of the design, and the architecture needs to be variable in the process of building/testing cycles to verify the design. This is true in essence, But in retrospect, I think my thoughts are somewhat immature. Although building/testing cycles may reveal architectural problems, more problems are often manifested by changing requirements. In general, designing software is difficult and new programming languages, such as Java or C ++, and graphical symbols, such as UML, no one knows how to use it effectively. In addition, once a project builds a large amount of code based on a framework, basic changes to the architecture are often equivalent to dropping the project and starting a new one, this means that this project has never been used. Even if projects and organizations basically accept the concept of refactoring, they are still reluctant to do things that seem to be completely rewritten. This means that it is important (or at least close to the right) for the first time, and the bigger the project, the more it will be. Fortunately, the software design pattern helps solve this problem.

There are other aspects that I think need to be emphasized more. One of them is the auxiliary documents, especially the architecture documents. Although the source code is the design, attempting to draw a framework from the source code may be a daunting experience. In this paper, I hope to see some software tools to help software developers automatically maintain auxiliary documents from source code. I almost gave up on this hope. A good object-oriented architecture can usually be described using several images and dozens of pages of text. However, these graphs (and texts) must be concentrated on key classes and relationships in the design. Unfortunately, software design tools may become smart enough to extract Jack Reeves from a large number of details of the source code to produce these important aspects, I don't see any real hopes. This means that such documents must be compiled and maintained by people. I still think that it is better to compile a document after the source code is complete or at least at the same time as the source code.

Finally, at the end of my thesis, I talked about C ++ as a programming-and therefore software design-an improvement in art, but more progress is needed. Even if I don't see any real programming progress in the language to challenge the popularity of C ++, today, I think this is even more accurate than I did when writing it for the first time.