The road to architecture: Performance and Unit Testing

It seems that programmers are impatient, perhaps by Windows verbose boot time is enough, probably because the effect of performance is most obvious ... In short, I found that the vast majority of programmers to the performance of the attention and enthusiasm is unparalleled!

C # just launched, someone bobbing said, "Ah, automatic garbage collection, performance is not it?" "DataSet turned out, and immediately a lot of people write code, insert millions of data in the dataset, prove the performance of the DataSet LINQ, of course, more to be scolded, the use of reflection?" What is reflection, the students know? Performance Big Tiger Ah! Not to mention those auto-generated SQL, is my handwriting efficient?　　...... So until today, I still see a lot of programmers with no regrets to use stored procedures to build their systems, a stored procedure can have thousands of rows! Then they were very innocent and asked, "What is the business layer for?" What can you do? "When I was with the team, I was afraid to talk about performance-related issues. If you don't talk about performance, the code sometimes really doesn't look down, but if you emphasize performance, you don't know what kind of moth he's going to give you. In fact, this is a "degree" of mastery, so it is very difficult to express clearly in the language. So after countless defeats, I had to grind my teeth and say, "Your code, there is only one criterion, maintainability." The problem with performance first!　　"The answer doesn't seem to be in the audience--especially for motivated programmers.　　Therefore, I first of a special article on performance, I hope to help you understand the problem more clearly. Performance is not unimportant, but he does not have maintainability important.　　To understand this, it is important to understand the importance of maintainability first (please read again the jokes I spent weeks looking for bugs), and then understand that: to solve the performance problem, we can have a lot of effective methods outside the code, and maintainability basically only depends on the code, and finally, remember: there is no sacrifice, there is no victory! So, in the vast majority of cases, performance is in maintainability when performance and maintainability are conflicting.　　We use other methods to compensate for the lack of code performance. There is no point in empty preaching. Let's be clear for example! Damage ReadabilitySome time ago I review the code, I found that the programmer always use first () rather than single (), I am surprised, according to business logic, the return value should be one, may be more than one, multiple should be reported abnormal, should not take first () is finished? Thought for a moment, and asked the programmer, his answer gave me a momentary sense of powerlessness, "first () Higher performance! The following is a transcript of the dialogue: How do you know that first () performance is higher?　　"I asked. First () Well, when you return with the number one, you're not going to keep looking. Single (), you will always check, find out all the data, and then take one of them. "Are you sure?" You know there's a thing called an index? "Ah?"　　......”　　Then I simply tell him that the index is a tree structure that allows queries to be made faster and so on.　　"But I still think I should use first ()," he thought for a while, still very firm. Why　　"I don't understand." "Even if the index speeds up the query, it's faster with first ()!" Faster is always right, right? "" ... ", I really don't know what to say, and then suddenly a flash of light," Well, then you say, MicrosoftWhy make a single () method come out? Just to get out and mislead you? Let's use first () to create superiority, ridicule with single ()?　　"He was immersed in contemplation. The comments are still tangled single ()/first () students, please shout loudly three times: readability! Readability!!　　Readability!!! Found that the students are still struggling with this detail. Well, explain again: how do you know DatabaseWhat's the use of MSSQL? How do you know it's a relational database? Can't you do nosql? So, how do you know exactly how single ()/first () is executed?　　For example, I'm going to write a LINQ implementation, take all the data out, then sort it in memory, and finally take first? Here we consider the readability, meaning: When reading the code, see single () can instantly know that coder means to take the only one; see first () know that coder means to take the top one.　　and performance does not matter, if it must be entangled performance, that good: you have to determine the uniqueness, of course, do check (including not only a momentary throw exception), this performance loss is supposed to be ah; you have to take the first one, of course, the sort will have a performance loss! When I first entered the profession, it was a collection of several article, such as the ten guidelines for high-performance programming, the content is roughly, "Always use StringBuilder, do not use ' + '; always use ..., do not use ...". There's always a bunch of people here who applaud, "good!" "," Thank you for sharing!　　"But slowly, I have doubts about these articles (and I should thank the old Zhao in the garden, the sp1234 in the CSDN, etc.); it was not until later that I understood why it was superficial, and that only through the above dialogue could I articulate my understanding. All of these simple packages of sacrificing performance have a purpose, and one of the important purposes is to improve readability. You intentionally do not use these off-the-shelf packages for performance, and generally, the loss is readability. Take for grantedContinue with the example above. At first, the programmer's performance considerations were taken for granted. This assumes a lot of situations, roughly this kind of: their own understanding is completely wrong of their own understanding can not be wrong, but in fact the bottom has been to optimize their understanding of the problem is correct, the bottom is not optimized 1th, 2 kinds of better understanding, the 3rd why also said he "take it for granted"?　　Because it is not compatible with the hardware environment. The simplest example is "caching." For example, when interviewing, ask you a question, "can the cache improve performance?" "Please note that this is a trap. The answer should be: "Not necessarily." Almost everyone thinks that caching can improve performance quickly because the CPU and disk of today's computers are running at a speed that is far from keeping up with the development of memory.　　But even so, an uncontrolled cache can drag down the entire system. There are a lot of similar examples. You're smug, I saved a disk read and write, you also increased the CPU load, you optimized the algorithm, reduce the CPU operation, but actually increased the pressure of memory ... There is no free lunch in the world.　　The same code, with the increase in data, hardware changes, will show a different performance. So, the development process, a lot of "optimization", in fact, just you take for granted. Rather than taking the optimization as it is, it is better to optimize it in a targeted way after getting the performance test results. This time, back to what we said before, is not the code readability more important? So that you can quickly find the bottleneck of the optimization Ah! Otherwise, a bunch of mess can not read the code, you how to optimize, you even the point of optimization can not be found. difficult to maintainAnother funny example is about my own. The start-up home project has a feature that displays a link to the next page of the previous page while displaying the body of the blog. The usual practice is to check directly in the database, but I always feel wrong, so it is necessary to do two queries? Can it be optimized? So I thought of a "brilliant" idea: why not just store the previous and next IDs directly in the blog? So I can get all the data from a one-time data round trip!　　Do you guys think I'm a great idea?　　The nightmare began. First of all, we want to set up his previous and next posts when posting a blog. But, the previous good setting, next article? No, not yet!　　How to do, you have to publish in the blog, set up his previous article, while setting up his previous post. Then, we added a new feature, in addition to the next one, but also in the current blog in the category of the previous and next article. What to do? And then add the field chant. So, the blog has previous, previousincategory, Next, Nextincategory.　　At this time, it feels a bit inappropriate, but it can be accepted. Then, there is a problem, the next blog was deleted, how to do? This process is just as troublesome as moving a node out of a doubly-linked list.　　The head is starting to get a little big. Then, in addition to the blog post deletion, there are a variety of other states, such as blocked. And after being blocked, whether the display is related to the current user. The current user is a normal user, cannot read, the current user is the author himself, can read. What to do? First of all, when the shielding, to set up the next article, shielding cancellation, or to set up the next article.　　Then, the next article to be based on the current user different changes in this problem, basically dumbfounded ... Finally shed tears to hard to toss a long time code to change back, on the database to find out, how clear and concise logic Ah! Performance issues? First, does this cause a performance problem? Then, even if there is a problem, with a cache can solve it? reasonable waste of heap hardware　　Said so much, do not know that there is no reflection of the students.　　Maybe we can not get through the heart that way: Obviously there is a higher performance method why don't we?　　Because of the waste! What the? Have you got it wrong? My code, at least save a piece of memory! That you have not been converted from the "poor student" role. You spend a week optimizing your code (without considering the increased maintenance costs of your optimizations) and saving your boss a piece of memory. You think the boss is going to slap you on the shoulder and praise you?　　The boss can't kill you! Dude, it's not like you're counting.　　When you are a student, your time cost is 0, but you get into the job and pay every day. Using code to tune high performance is a frustration-compromise on hardware performance (see: 80 's hard work for game developers). It's a high performance, but why isn't anyone writing code like that now? ）。 Otherwise, in most cases, the heap hardware is much better and much cheaper than optimizing the code.　　Does the cost of hardware fall by Moore's law, and can our programmers ' wages be reduced by Moore's law? Clearly window 10 is more cost-performance than Window 95, why is no one using Window 95 today? Why does the VS 2013 have to be 10G of space and we're all going to put it on the butt? Why does everyone use C # now, no one uses the assembly? We stand in the accumulation of human civilization today, it should be taken for granted to enjoy all these results. There are lighters you don't need, you need to drill wood to take the fire. If you want to learn to live in the wild, you can understand, if you say you are afraid of wasting gas, I ... I...... What do I say about you? "A way to make a lighter, okay?" "The same, the programmer of the Great God classmate, you as a good thing, write down the bottom of the hardware to do a way to survive it!" Your code is 010001000010000001010101 ...　　, how do you let the other people live? Finally, I suddenly thought of a programmer why the performance is so sensitive to madness, a possible reason for maintainability does not care about: good understanding of performance, card to die and run fast; maintainability is very bad understanding, at least to run a two or three years to reflect, at that time, who knows where Ye stole the music performance on not to come, The programmer only ashamed of the head, it is my fault, the demand has changed, the opening of the scold, "which SB to change ..."; Do you think it is? So, be willing to put the code tempered around the soft person less.　　Want to come, is a kind of inexplicable sadness and desolate. Finally, there are some famous sayings I can think of for everyone to take a look at: Premature optimization is the root of all evils to optimize the first need to find performance "bottleneck."　　Otherwise, anyone can point at a finger, "This code needs to be optimized." More readable code is always better to optimize hardware is always cheaper than software to be realistic, to write "Wild programmer": "Priority recruitment" when, it is with emotion. Then there is reflection, is not I worry about it? In particular, the following comments began, such as "are mixed mouth food is not easy to eat", "why the inside and outside of the points, the Chinese children are not a mountain dumping river can not be in the past, the millennium is so", so I feel may be too sensitive to me.　　But then some people made a lengthy speech, let me understand, this blog is still meaningful. Think about, the recruitment revelation, you ask "computer major Bachelor Degree", I "No computer professional Diploma" priority; and then you blow it up! We do not discriminate, you are the discrimination! You feel inferior to your anger, you sour you ... I am powerless to refute, just want to say, each person's words and deeds are the mirror of his mind.　　Thank you! In fact, I do not want to provoke a class/non-college dispute (although may result will exceed my expectations), my intention is to give "non-cheer" classmate, relieve their pressure, let them see hope, give them the strength, let them believe, can in more difficult environment, and the result will not be compared to "school" The difference! But you must be entrusted with a solid to learn, step by step to do, self-esteem arrogant quarrel rebuttal is not conducive to your growth.　　Remember: words have no power! In addition, willing to listen to a "class" classmate, "no computer professional related professional Diploma" priority, not entirely out of indignation. Are built, you are a noble faction with the resources of Dan Medicine heap out, he is the way to repair the struggle to understand the breakthrough, you think who has the potential? So, put down those vanity pride, real to fight it!　　After three years of graduation, no one will see your education. In addition, there is no comment on Lao Zhao, except admiration. He is targeting the training institutions I fully understand, but still cannot agree. So I said, "Every time I see this piece of text, my heart will have a kind of difficult to say the complicated emotions", as to how complex, not to say?　　It's hard to say. ====================== good, after the calm we continue to discuss technical issues. In the lead process, the performance of the problem is better to solve, the most negative idea, "good ah, more than a matter less, you let me no matter is not simple?" "But ask to write the test code, then the fryer!" In my experience, "test Drive" is one of the most controversial topics, not one of them. The touts and opponents are distinct, and there are plenty of arguments and proofs.　　Remember that there was an article, the effect is: "The company pays you not to let you write test code," the following a crazy praise. At the beginning of my own project, I gave up the test-driven (hehe, also found the original text), the summary is very accurate, the biggest reason is "lazy." But finally let me make up my mind to start the "test-driven" practice, I spent two days and nights did not call up a bug, frustrated and exhausted, helpless to accept the reality: testing is very useful-even the code of their own writing. My previous series of blogs,Also has repeatedly stressed that the architecture is a "helpless", is the reality is the problem is driving you to do something you actually do not want to do. You can't understand something that looks like "take off your pants and fart," usually just because you're not experiencing those real problems. (see, can college teach you these things?) Even if you don't have much development experience, you should be able to imagine that the biggest problem with unit testing is that it takes time to write, so is it worth the cost? This is still determined by the goals of your architecture, or your needs. If the system is used for a molding delivery and will hardly change since then, then a one-time manual test will suffice, but if your system is going to be "thoroughly tempered", the test is necessary. The simplest consideration: every time I change, I have to test it by hand, and it's not going to take me a little more time to get something "automated".　　Unit testing, in fact, can be understood as an automated testing tool. But the reason for "automation" is far from enough. Because you immediately think of, every time the need to change code adjustment, the test code should be changed accordingly? Without the test code, I just need to change the code; Now that I have the unit test, I have to change the code. I just maintained a set of code, and now I've added a set of code that needs to be maintained, isn't it an increase in maintenance costs, not the opposite of your "maintainable" architectural goals?　　is a set of code good maintenance, or two sets of code better maintenance? This is a very good question, for many scenarios (such as layered architecture, you say layered decoupling, actually not a change from the UI layer to the database, each layer has to be changed?) ）。 The answer I can give is probably: does it have to be tested, whether it's a unit test or not, after the development code is modified? No unit testing does not mean that your code does not need to be tested, but you have to test it manually.　　Remember: Your job is not just writing out the code! For manual testing, and for changing unit tests, the cost ratio of the two will vary depending on the number of times the test is reused.　　A manual test may take up to 5 minutes to run, and changing the unit test code may take 20 minutes, but if the test runs 100 times, the unit test will go through the manual test. You say, where yo? what function will be changed 100 times? I didn't say your function would change 100 times, I said the test would run 100 times. Is there a difference? You're probably still making a mistake, aren't you?　　Well, let's tell a story. There was a lad who was reluctant to write the test code. The boss has no trouble with him, not so much energy and his teeth, so the boss himself write unit test. The boy's code before the submission to review, the boss can always find out the problem of its code. He changed the login, the boss told him that the points system was he changed the problem, he went to change points, the boss told him the news notification system was changed by him, he went to change the message system, the boss told him to log in orThere is a problem ... So he collapsed, "what a rotten system is this TM?" Finally he came back to God, why does the boss always know that the changes here will affect there? Is the boss's mind so rigorous?　　The boss hides in the side to steal the smile, does not tell you, "Actually I just ran once the unit test only". The boss is me. I deliberately, not one-time to tell him all the problems, it is necessary to torment him again and again, so that his pain can be carved into the bones.　　Finally, I will ask him: Are you still so confident in your code?　　If you don't have my review (I'm also on unit testing), can you find these problems?　　What happens if our project is deployed to the production environment, and the damage that your changes bring is not found to be on the line? This time, he was convinced. Then he used the nunit to slip in the linen. Every change, if there is an unexpected failure to pass test case, he will be very excited to give me Zhang, by the way, complaining.　　I smiled, the kind of full-screen green through the practical, and unexpected burst of light after the surprise, not experienced people, is unable to understand. So when the relationship between objects becomes more and more complex, like a dense web, a partial change is likely to trigger an extremely complex chain reaction. So for the sake of insurance, all components that may be relevant should be tested (so-called "regression testing").　　At this point, if there were purely manual tests, there were two problems: it was difficult to determine the bounds of the test (those parts might be affected), and it was hard to think of it, brother! Great testing costs. And the cost is rather boring and cumbersome-no one wants to do it. It is said that many company testers now pay more than developers. Why?　　Simple boring, no one willing to do AH! OK, I assume you've recognized the importance of unit testing and started to get ready to go. And then I'm going to pour you a big scoop of cold water: unit testing is not so good! In a way, writing unit tests is harder than writing code.　　It is rare that all the companies I work for have had successful cases.　　About a few years ago, I fixed a bug in the company, the boss told me, "You this function of the core, run a unit test it." "Wow!" We have unit tests?　　"A tall feeling quickly pervaded the whole body, finally see the legendary unit!" For a while, can run, try--my mother? Why so many red lights?　　I'm really scared, it's all my changes? Boss is the boss, leisurely, "count how many of the pass?" "Ah?" "I thought I had heard it wrong, how many things could be used?"　　Got to get them all through?! After a while, and finally figured out, I changed the code before and after the run again, a comparison of the failure is the same, as long as it is the same,It's OK.　　For example, it was 8, but it's still 8, so it's ready! I always don't understand, why not put the 8 pass the unit test to get through it? What the hell is that? Until I started writing unit tests myself. Hang Daddy!　　Everywhere is the pit, jumps out of the small pit into the big pit, under the big Pit also has the small pit, the front is the pit behind pits, a pile of piles of serial pits ... Unit tests written out to be easy to run too hard! And the reason to run is not your development code logic is wrong, but test environment/data problems. To test, we must have the data, the construction of this data is not as simple as we think. Taking the integration system of our entrepreneurial home project for example, suppose a simple need: blog is liked, the author of the blog should get a certain point, the number of points is from the likes of the current all the available currency conversion (has been simplified, specifically refer to the document: points). The data to be prepared is: a blog, to have the author, the author has points; like a person, a certain number of available coins. If that's the case, it's acceptable, but there's a bunch of questions: Where does the author's points come from?　　Our development code, for the sake of encapsulation, the user's integral is read-only, how do you set this value in unit test?　　Either write the code, simulate the author to get points through other behaviors (to issue an article to answer questions, etc.), which will open up a new nightmare; If you force settings with mock or reflection, in fact omit the author's history of earning points, so the user "integral history" is null, and then the "bonus points" will be reported abnormal.　　What's more, you think you've handled everything well, you suddenly sad to find that the blog has to be "released", and once the blog is published, its author has earned a certain number of points, so you set the integration has changed again!　　...... Like the available coins, you may also encounter similar problems.　　How to set the available currency, will it be changed unexpectedly when running the test?　　Like the behavior, is encapsulated into a method, run this method, will check whether the point like people have been to the article point of praise, so there should also be a "like history", even if it is empty, have to new one, otherwise the empty exception ... Anyway it was written I dropped the mouse directly! It's so written! And I was completely isolated from the database, I really do not know those to get data from the database to run unit test, how to do? At this time I understand, the actual work overtime to make progress, one after the pits, even the reconstruction of the times are not, how may also squeeze out time to write unit test?　　Even as the system becomes more complex, the cost of maintaining unit testing is growing, and even more complex, it is the only option for most projects. Work in the company, most people are like this, can push on the push. We developed the code, basically can run, it should be handed over to the testers! DaysIt's a land of righteousness, isn't it? And the time of the test is not calculated in my project development time, I always finish the development task on time.　　Tired, rest, let the test busy go, haha ... But I am a guanggansiling, I do not have the test personnel ah! There were one or two of times when I was really ready to recruit one or two testers. But fortunately my natural thrift virtues (that is, "pull") let me calm down. I just thought: the test can only tell you out of the bug, can not tell you the source AH. No unit test, I single-step debugging, not also toss for two days? This is the complexity of the system itself, or the irrational nature of the code organization, which cannot be attributed to unit testing. No, there are so many open source code with exhaustive unit testing? How did they do it? In the unit testing of the pay, will eventually get the value of return!　　Think of a company that has no unit testing, that super-large test team, or a smoke-free system, would you like to take this route? So I keep on telling myself, don't worry, calm and meticulous. Finally step by step cobwebs, this mess a little bit of induction, and finally really I found a way, a unit of testing are slowly completed through the development code potential problems also surfaced, by me one by one of the elimination. Finally run the unit test again, all the way green, haha!　　What's more, the bug that bothers me for two days doesn't know when it's gone? Later, I saw the saying that testable code is not necessarily good code, but that bad code is almost impossible to test. Deep Thought!　　Deep-coupled code, writing their unit tests, is hard to get to the sky, but in turn, we can test as the standard, constantly improve the development of refactoring code, as long as this persisted, the quality of the final code will not be poor to where to go. So, in my case, the argument that unit testing is valuable is out of the dispute!　　Instead of a different angle, think about how you can keep the unit test going. Finally, if the heart of the classmate will notice that I have been using "unit testing", rather than "test-driven". Because the test drive is a broader concept, is a more brand new world! Unit testing is just a small part of it, and in the next blog post, I'll explain how I've tried to apply test-driven concepts to project development management.　　Here, one thing to emphasize: write Tests first. Write the development code as soon as you get started, and write the unit test once you're done. This is a lot of developers habits, I also often make such a problem, a inattention to forget. The biggest problem with this is that there is no real "test-driven" implementation. You are actually still developed by the driver, then very naturally, testing the If...else developed ... Write it again, what's the point? In doing so, the impression of "testing useless" is constantly strengthened, because testing is simply rewriting the development code over and over again. The prescription I prescribe is: Unit test code and development code written by different people if you can't do it aboveA bit, write unit test first if not even the above can not be done, until the bug and then write unit test the third article may have classmates can not understand, not to say unit test is important? Why wait for a bug to write? The answer is: Lazy Bai! Remember, we programmers are the most lazy people in the world, and meaningless things never do! You write the development code and then write the test really meaningless, no meaning, simply do not. But you can turn on "optimistic mode" (or "lazy mode"?) First of all, optimistic that my code is OK, maybe there is no problem, right? If there is a problem and a remedy, this should be done with unit testing, because he has a problem with Murphy's law, and there is a good chance that it will continue to go wrong. At this time, don't be lazy again.

Architectural Path: Performance and unit testing