5 tips for improving log quality

There are a variety of new tools that can help you understand the logs recently, such as open source projects like Scribe, Logstash, prepaid tools like Splunk, and managed services such as Sumologic and Papertrail. What these tools have in common is to clean the log data and extract some more valuable files in a large number of logs.

But there's one thing these tools can't help, because they rely entirely on the log data you actually put into it, and how to ensure that the quality and quantity of the data are done by the user themselves. So, at a critical moment, things can get tricky if you need to debug your code based on some or missing logs.

To reduce this, share five suggestions here, and it's a good idea to keep your mind in mind when you log:

1. Hello, my (thread) name is

As Ringo, the thread name attribute is one of the most underrated methods in Java. The reason for this is that the thread name is mostly descriptive. But the problem also arises here, like the people themselves, who are often given a certain meaning when they are named. In multithreaded logs, the thread name also plays a key role. Typically, most log frameworks record the name of the thread that is currently being called. Sadly, we usually see http-nio-8080-exec-3 this name, which is simply allocated by the thread pool or container.

For some reason, we have heard this misunderstanding more than once-the thread name is immutable. In contrast, in the log, the thread name occupies a basic position, and you should make sure that it is used correctly. For example, combine it with a specific context, such as the name of a Servlet, a task-related, or some dynamic context such as a user or message ID.

In this case, the code interface should look like this:

Thread.currentThread().setName(ProcessTask.class.getName() + “: “+ message.getID);  

The more advanced version will be loaded into the thread local variable of the current thread, configure log Appender, and automatically add it to the log entry.

This can be useful when multiple threads write to the server log, but you need to focus on a single thread. If you are running in a distributed/SOA environment, you can see that it has a unique advantage.

2. Distributed identifiers

In an SOA or message-driven architecture, task execution is likely to span multiple machines. When dealing with failures in this environment, connecting the relevant machines and their state will be key to understanding the situation. Most log parsers group these log information, assuming that you provide them with a unique identity that can be used as part of the actual log message.

From a design standpoint, this means that each inbound operation should have its unique ID corresponding to the completion of the operation from entering the system. Note that a persistent identifier, such as a user ID, may not be a good container. In the process of logging a log file, the user may have more than one action, which makes it more difficult to isolate a particular stream. UUIDs may be a good choice. Its value can be loaded into the actual thread name or as a local storage for tls-thread.

3. Do not use text + drive, do not log + loop

Many times, you will see a piece of code running in a tight loop and performing the corresponding log operation. The basic assumption is that the number of times the code runs is limited.

It is likely that the operation is very good. However, the loop may not break when the code gets accidentally entered. In this case, you're not just dealing with an infinite loop "although that's bad," The code you're working on is writing an unlimited amount of data to disk or network.

It can cause one server to crash in a single-machine scenario, whereas in a distributed scenario, the entire cluster is affected. Therefore, if possible, do not log logs in a tight loop. This is especially true when errors are caught.

In the following example, an exception is recorded in a while loop:

    void read() {while (hasNext()) {  try {  readData();  } catch {Exception e) {// this isn’t recommendlogger.error(“error reading data“, e);  }}}

If ReadData throws an exception, and the Hasnext return value is true, this will write an unlimited amount of log data. The way to solve this problem is to make sure that it is not logged:

void read() {  int exceptionsThrown = 0;  while (hasNext()) {  try {  readData();  } catch {Exception e) {if (exceptionsThrown < THRESHOLD) {  logger.error(“error reading data", e);  exceptionsThrown++;  } else {// Now the error won’t choke the system.}}}}

Another way is to remove the log records from the loop and save the first/last exception object and record it elsewhere.

4. Uncaught handlers

Westeros has the last defensive wall, and you have Thread.uncaughtExceptionHandler . Therefore, use them as much as possible. If you do not have these handlers installed, you can only get very valuable context when the exception is thrown, and you will not be able to control where you have logged it before the end, and determine the location of the record.

Note that even in an uncaught exception handler, it looks like you have no way to access the line approached (terminated) any variable, and you can still get a reference to the actual thread object. If you stick with # 1 steps, you will still get a meaningful thread.getName() value to record.

5. Capturing external calls

Every time an external API is called, the odds of a JVM exception are greatly increased. This includes Web services, HTTP, DB, file system, operating system, and any other JNI calls. Take each call seriously because it will explode at any time "it's very likely to happen at the same point."

In most cases, the cause of an external API failure is accidental input, and the logs are key to fixing the code.

At this point, you can choose not to log the error, just throw an exception can also. In this case, only the relevant parameters of the call are collected and resolved to the exception error message.

Just make sure that the exception is captured and logged at a higher level stack call.

try {  return s3client.generatePresignedUrl(request);  } catch (Exception e) {String err = String.format(“Error generating request: %s bucket: %s key: %s. method: %s", request, bucket, path, method);  log.error(err, e); //you can also throw a nested exception here with err instead.  }

Original link: 5 techniques to Improve Your Server Logging

This article is compiled and collated by OneAPM engineers. OneAPM is the emerging leader in China's basic software industry, helping enterprise users and developers easily implement slow program code and real-time crawling of SQL statements. To read more technical articles, please visit the OneAPM official blog.

5 tips for improving log quality