Your Java logs, is there any way to use these improvements

Turn from "http://mp.weixin.qq.com/s?__biz=MjM5MDE0Mjc4MA==&mid=207451012&idx=1&sn= De9fba4eda0f221363b6d5ae54243416&scene=2&from=timeline&isappinstalled=0#rd "

Summary:

In any system, the log is a very important part, it is a reflection of the system operation of the important basis, but also the necessary clues to troubleshoot problems. most people recognize the importance of the log, but how many people think carefully about how to log, the impact of the log on the performance of how much? Let's talk about the performance of the Java log today ...

When it comes to Java logs, you will definitely say you want to choose a reasonable log level, reasonable control of the log content, but this is only the first step Long March ... Even some DEBUG levels of logs that are not exported to a file in a production environment can also have a small overhead. We put aside the overhead of judgment and method invocation, and in the performance document of Log4j 2.x there is a set of comparisons:

Logger.debug ("Entry number:" + i + "is" +  
Logger.debug ("Entry number: {} is {}", I, entry[i]);

The above two statements have the same effect on the log output, but when the log is closed DEBUG , their overhead is different, the main effect is in string conversion and string concatenation, regardless of whether it takes effect, the former will convert the variable to a string and splicing, The latter will only perform these operations when needed. Log4j's official test concludes that the performance of the two can vary by two orders of magnitude. Imagine toString() how much resources can be saved if a method of an object is used ToStringBuilder to reflect the output of dozens of properties.

Therefore, some companies that are still using log4j 1.x or Apache Commons Logging (which do not support {} template writing) will have the appropriate coding specifications that require additional judgment before a certain level of logging (for example, DEBUG and INFO ) output:

if (logger.isdebugenabled ()) {     Logger.debug ("Entry number:" + i + "is" + string.valueof (Entry[i]));}

In addition to the log level and log messages, usually in the log also contains some other information, such as date, thread name, class information, MDC variables and so on, according to TAKIPI test, if you add class in the log, performance will drop sharply, compared to the default configuration of Logback, the throughput of the decline of about 60%. If you must print class information, consider naming logger with the class name.

in a distributed system, a request may go through several different subsystems, it is better to generate a UUID attached to the request, each subsystem in the print log when the UUID is placed in the MDC, to facilitate subsequent query related logs. "The Ultimate guide:5 Methods for debugging Production Servers in scale" gives a lot of advice on how to debug in a production environment, several of which are about logs, This is one of them. Another suggestion is to record all the records that have not been captured, in fact, throwing exceptions have overhead, recording exceptions also bring some overhead, mainly because the Fillinstacktrace method of the Throwable class is synchronous by default:

Public synchronized native Throwable Fillinstacktrace ();

The general use of Logger.error will play an abnormal stack, if the throughput has a certain requirements, when the situation runs can consider overwriting the method, remove the synchronized native, directly return to the instance itself.

after chatting the contents of the log, then look at Appender. in Java, said IO operation everyone will think of NiO, to JDK 7 and AIO, at least know read and write plus a buffer, the log is the same, synchronous write appender in high-concurrency large-volume system is somewhat inadequate, It is time to use Asyncappender, as well as using Logback:

With 10 threads concurrently, the output of the 200-character info log can be as high as 3.7 times times the throughput of the Asyncappender Fileappender. In the case of no loss of logs, the same use of Asyncappender, the queue length will have a certain impact on performance.

If you use log4j 2.x, you can also consider the higher performance asynchronous logger in addition to the asyncappender, because the underlying uses disruptor, without the overhead of locking, the performance is even more astonishing. According to the official test of the log4j 2.x, the same log4j 2.x is used:

Under 64 threads, asynchronous logger is 12 times times faster than asynchronous Appender and 68 times times faster than synchronous logger.

It is also asynchronous, and there are differences between the different libraries:

In the same hardware environment, the LOG4J 2.x all use asynchronous logger will be 12 times times faster than Logback Asyncappender, 19 times times faster than log4j 1.x asynchronous Appender.

Log4j 2.x's asynchronous logger performance is tough, but there are different voices, think it's just a look elegant, only as a toy. It is still left to the reader to think about the problem.

If you must use synchronous Appender, then consider using Consoleappender, and then redirect stdout to the file, which can have about 10% performance gains.

most production systems are cluster-deployed. for logs distributed on different servers, it's good to collect them with tools like Logstash. Many times will also deploy multiple instances on a single machine to make full use of server resources, at this time do not covet log monitoring or log query convenient, the log of multiple instances to the same log file, although Logback provides the prudent mode, to allow multiple JVMs to the same file to write the log, However, this approach also has an impact on performance, which can reduce performance by as much as 10%.

If you have a large number of write requirements for the same log file, consider splitting the log to a different file. one way to do this is to add multiple appender, while modifying the code, using different logger;logback to provide siftingappender that can be split directly from the MDC's content. Jetty's tutorial has an example of splitting the log based on host, and according to the TAKIPI test, the performance of Siftingappender will increase with the number of split files, when split into 4 files, 10 concurrent Siftingappender throughput is about 3 times times more than that of Fileappender.

Look at the above data, I do not know whether you think your log has a lot of improvement, you have not optimized the system to the extreme, or you have other methods of log optimization, may wish to share to you.

Reply keyword to see the corresponding content:

React | Architects | Operations | Cloud | Open Source | Kubernetes | Architecture | Artificial Intelligence | Kafka | Docker | Netty | CoreOS | QCon | Github | Swift | Agility | language | Programmers | Practice | Internet of Things |

If you want to comment on this article, directly poke the "comment" in the lower right corner . The kol of the technology we've been looking for, maybe you!

Copyright and Reprint statement:

Geek State Science and technology focus on providing quality content for technical people to spread. Respecting the work of the author, translator, and the editorial of the INFOQ website, all the content is only for learning and communication and does not support misappropriation. No reprint is permitted without permission. If reproduced, you need to be informed, and indicate the source.

Your Java logs, is there any way to use these improvements