Thoughts on the misuse of log information

This is a few days to clean up the project work in the last six months.

Want to tidy up the development of the log information abuse thinking results.

I was in the early stages of the project, large projects, dozens of people divided into several groups involved in the development. The code amount is estimated at 1 million line levels (accumulated over many years, specifically unknown).

The log is divided into 3 levels: trace level, log level, error level. Just look at this rating, it feels reasonable.

Next there are two questions:

1) Basically all of the functions are recorded in the log, real-time write way.

2) Large project, long time, more participants, the use of log level is completely chaotic. When something goes wrong, you often find that the log you are looking for is missing.

Personal Impressions:

As we all know, the performance of file Io is very bottom, in the system running, it is better than the remote network transmission. Combined with my personal experience over the years, it is estimated that so many log write tasks, the decrease in system performance is probably 10-20%, the impact of such a large project log information, of course, is not the exact measurement evaluation, but basic each function has log write, 10% estimates are low. At the end of the project, everyone spends a lot of effort to improve even a little bit of performance.

The function is littered with log-written lines of code that can be imagined for readability. The most important thing about code is readability. Often hundreds of lines or even longer functions (pushing the function is small, functional single, cohesion poly, low coupling), a variety of log information flying, this code read, you can imagine that expression is what, especially overtime to night. More, in the future the code changes constantly, have to take into account the synchronization of log information changes. But in everyone exhausted to find a problem, modified the logic, the verification passed, ask at this time, who still willing or have the power to check the log information is synchronized change, as long as the compilation through the OK, estimated that most people are feeling the same. Now the log level abuse in the project is the result.

What then, do not write the log.

Such a big project, so much code, a strange problem with this. How to troubleshoot the problem and find out why. To troubleshoot the problem, you need to know the execution process of the program, change the value of important variables, find out how the results of the statement, function call results, how the database records read, and so on, right. Otherwise, how to troubleshoot the problem, paddle. So the log is called out, recording the invocation of each function, the value change of the key variable, the error message. This problem, open the log file, go to a line of analysis, check the execution process, check the variables are worth the change, especially first to find the error message ... Presumably, it's a human instinct.

However, in software development, this is preferable, science. Is there any better way to solve it? Above, we clearly see two serious drawbacks: 1) greatly reduced system performance, 2) seriously affect the readability of the code, in a word, called "Code pollution", which has a great impact on later maintenance work.

Next, let's talk about my personal solution.

1) The log grading practice has some merit, but can only be used for pre-release debugging testing, not in the production environment.

        2) Core approach, or from the concept of development: code is clean and important, priority is higher than troubleshooting errors. There may be objections, mistakes, and quickly find the wrong solution, the customer is waiting, this job is the highest priority, right. Is that so? To analyze: Why the code goes wrong. Why. The vast majority of code errors (estimated at 80%) are the result of irregular coding and neatness. In other words, the coding work in place, can greatly reduce the bug, which, why, why so many logs to assist in the analysis of errors. One might say again: There are some bugs that will appear, yes, subsequent decomposition. Here first talk about the concept of the problem, it is not a two-sentence can be said clearly, how much to say points. Coding, not simple repetitive manual work, just a few years ago, many times to hear that people have been designed, we are coding, boring repetitive labor. I believe many people have heard similar words, or said similar words. I remember I didn't say it clearly, but I thought about it before. However, after a few years, I do not think so, I feel that is the coding work, is every hit a letter, even if the repetition, is not the same. The same code, read it every day, experience is not the same, can find a lot of problems and gains. It is not simply repetitive work. As for others to design the structure, others to code, this is a serious mistake, there is not much to say (later to send out the article). Simply put, if you really intentions to code, there will be a different experience and harvest, the quality of the code is not a grade, it will not need so many logs to solve the worries. Simple copy and paste, code layout nonsense, mysterious numbers fly, function often good dozens of lines above, write a class is considered to be object-oriented development, including the mind has never had code duplication, the concept of cohesion coupling, never considered the future maintenance (after a code is written, the number of reading and maintenance in the future average of more than 10 times), Compile through the submission, such code, but also only as many logs as possible to assist in parsing errors. Specific practices and stresses, 32 words are not.

3) If the coding work in place, modular design and programming, each module to do their own things, I think such a final system, should not appear too many problems. Module implementation of the process, inevitably error, but this stage, not by the log to row the wrong, but by unit testing. If the module does not do a unit test inside, even if a small problem, in the future must invest a great deal of time to troubleshoot (too much experience, this point). Modules within the range of small, low complexity, unit testing can (preferably plus code review walk, in fact, the most efficient error checking) very effective verification code. We have to be very confident to hand over their modules (of course, usually iterative), and then the module of the entry and exit of the log can be added. In this way, the entire system error, we combine the error phenomenon and business logic, we look at the entry and exit log of each module, see which module input and output does not correspond, and then analyze the logic in the module (additional premise: The module is not designed to be large, this is a conceptual problem), Or there's an important trick: put the incoming data into a unit test and test it to see how the module behaves.

I think this is much more efficient than troubleshooting errors from a large number of logs. The analysis of a large number of logs does not necessarily directly locate the problem, often also a single-step debugging. Start the whole system to do one-step debugging, I think a lot of people have this experience, know the experience.

Overall:

1) do not rely on each function plus the log to assist in future troubleshooting errors,

2) to have a good idea, write a good function, the module is written well, the design concept of course not less. In the early days as far as possible to prevent errors, reduce errors, rather than add other means to assist in the future to check the wrong, so that future errors are endless.

3) Do the unit test, this can be in the early detection of errors, reduce errors, but also to provide assistance for later analysis errors (otherwise, it can only be a single-step debugging, master, the proportion of debugging time is very bottom).

4) Add an appropriate amount of log in the entrance and exit of the module, the log can be divided into general log and error log, tracking trace log of the process is not necessary.