Implementation of the Java log caching mechanism--reprint

Overview

Log technology provides an important support for the quality and service of the product. The JDK has been added to the logging mechanism since version 1.4, which facilitates Java developers. However, this logging mechanism is based on the static log level, that is, before the program runs, you need to set down the level of the log to print, which will bring some inconvenience.

In the log functionality provided by the JDK, the log level is refined to level 9 to distinguish the purpose of different logs, to log an error, to log the information that is running correctly, or to log detailed debugging information. Because the log level is static, if the log level is set too high, the lower level of the log is difficult to print out, resulting in the error occurs, it is difficult to trace the cause of the error, the current way is often used in the error occurs, you have to adjust the log level to a relatively low level, and then to trigger the error, Causes the problem to be revealed. However, this problem needs to change the product configuration, and then re-trigger the problem to debug the way the product user experience is poor, and some problems due to the occasional, the environment is very complex and other reasons difficult to re-trigger.

Conversely, if the log level is initially adjusted to low, then there will be a lot of useless information in the middle of the log, and when the product is more complex, it will result in large log files, refresh quickly, unable to timely record the effective information, or even become a performance bottleneck, thereby reducing the log function of the product help.

This article uses the Memoryhandler class in Java Logging to cache all levels of logs and output them at the right time to resolve this problem. Mainly revolves around the definition of memoryhandler and the processing of logging.properties files.

The instances attached to the scenario are as follows, assuming that the user needs to view the previous error message containing the Exception in case of a critical product error as a basis for diagnosing the cause of the problem. A solution using the Java buffering mechanism is to store log entries containing Exception that are generated during all product runs in a resizable cyclic buffer queue, and when critical errors (SEVERE) occur, output the logs from the buffer queue to the specified platform for user review.

Back to top of page

Introduction to the Java logging mechanism

The Java log mechanism is introduced in many articles, in order to facilitate the understanding of the later part of the article, here is a brief introduction to some of the keywords used in this article.

The LEVEL:JDK defines off, Severe, Warning, Info, Config, Fine, Finer, Finest, all nine log levels, defining off as the highest level of logging and all at the lowest level. Each log must correspond to a level. The definition of a level is primarily used to classify the severity of the log and can be used to control whether the log is output.

LogRecord: Each log is recorded as a logrecord that stores some information such as the class name, method name, thread ID, printed message, and so on.

Logger: The basic unit of the log structure. The Logger is stored in memory in a tree-shaped structure, and the root node is root. Com.test (if present) must be the parent node of the Com.test.demo (if present), that is, the existing logger that the prefix matches must be the parent of this logger. This definition of parent-child relationship can provide the user with more free control granularity. Because there are no processing rules defined in child nodes, such as level handler, formatter, and so on, these processing rules in the parent node are used by default.

Handler: Used to process LogRecord, the default Handler can be connected to a chain, in turn, the LogRecord processing.

Filter: Log filter. In the JDK, there is no implementation.

Formatter: It is used primarily to define a LogRecord output format.

Figure 1. Java Log Processing Flow

Figure 1 shows the process of a logrecord. A log into the processing flow is first Logger, which defines the level that can be passed, and if logrecord levels are higher than Logger, the filter (if any) is entered. If no level is defined, the level of the parent Logger is used. The process is similar in Handler, where Handler also defines the level that can be filtered by filter, and if there are other Handler, then directly to the back of the Handler processing, otherwise it will be directly bound to formatter above the output to the specified Position.

Before implementing the log cache, the Filter and Formatter two helper classes are described first.

Filter

Filter is an interface, mainly filtering the LogRecord, to control whether the LogRecord is further processed, which can be bound under Logger or under Handler.

As long as you add the filter logic to the Boolean isloggable (LogRecord) method, you can control the LogRecord, and if you want to record only those log records that have occurred Exception, you can do so from the list. Of course, you first need to bind the filter to the corresponding Logger or Handler by calling the SetFilter (filter) method or the configuration file.

Listing 1. Implementation of a Filter instance

@Override Public  Boolean isloggable (LogRecord record) {  if (Record.getthrown ()!=null) {         return true;  } else{  return false;  }  }

Formatter

Formatter is mainly for the Handler in the output log record format control, such as the format of the output date, the output is HTML or XML format, text parameter substitution and so on. Formatter can be bound to Handler, Handler automatically calls the Formatter String format (LogRecord R) method to reformat the log record, which has a default implementation, and if you want to implement a custom format, you can inherit the for The Mater class and overrides the method by default, such as Listing 2 after Formatter format, replacing {0} and {1} with corresponding parameters.

Listing 2. Log a log

Logger.log (level.warning, "This log was for test1: {0} and Test2:{1}",  new Object[]{newtest1 (),  new Test2 ()});

Back to top of page

Memoryhandler

Memoryhandler is one of the two categories of Handler in Java Logging, and the other is Streamhandler, which is directly inherited from Handler and represents two different design ideas. Java Logging Handler is an abstract class that needs to create specific Handler based on usage scenarios, implementing the respective publish, flush, and close methods.

Memoryhandler uses a typical "registration-notification" Observer pattern. Memoryhandler is first registered to the Logger (Logger.addhandler (handler)) that is interested in them, and in these Logger calls the Api:log (), Logp (), LOGRB () of the Publish log, and so on, traversing these When all handlers are bound under Logger, the notification triggers the call of its own publish (LogRecord) method, writes the log to buffer, dumps the log and empties the buffer when the condition for dumping to the next log publishing platform is established.

The buffer here is that Memoryhandler itself maintains a custom-sized cyclic buffer queue to hold all Exception log entries that are triggered by the runtime. At the same time, the constructor is required to specify a Target Handler for the output, in the case of a specific flush buffer, such as the log entry level is higher than the Memoryhandler set push level (defined as SEVERE in the instance), etc., the day Transfer to the next output platform. The following log dump output chain is formed:

Figure 2. Log Dump Chain

In the instance, determine whether to push the log to the next Handler, usually within the publish () method, by judging the level of the Memoryhandler configuration item. Push. The code listing is as follows:

Listing 3

     Only LogRecord final level level      = Record.getlevel () with abnormal and higher than Pushlevel is recorded;             Final Throwable thrown = Record.getthrown ();      If (Level >= pushlevel) {         push ();      }

Trigger conditions for the Memoryhandler.push method

The Push method causes the Memoryhandler to dump the log to the next handler, emptying the buffer. The trigger condition can be but not limited to the following, the first of the default is used in the instance:

• The level of the log entry is greater than or equal to the default definition or user-configured pushlevel in the current memoryhandler;
• The external program calls the Memoryhandler push method;
• The Memoryhandler subclass can overload the log method or a custom trigger method by scanning the journal entries in a method, satisfying a custom rule that triggers the dump log and empties the buffer. Configurable properties of the Memoryhanadler

Table 1.MemoryHandler Configurable Properties

	Property name	description	default value
Inheritance Properties	Memoryhandler.level	Memoryhandler accepted log level for input to buffer	Level.info
	Memoryhandler.filter	You can customize other filters in the filter, in addition to the log level, before entering buffer	(Undefined)
	Memoryhandler.formatter	Specify the log format for input to buffer	(Undefined)
	Memoryhandler.encoding	Specifies the log encoding input to buffer, which is rarely applied in Memoryhandler	(Undefined)
Private properties	Memoryhandler.size	Define the size of the loop buffer in log entry units	1,000
	Memoryhandler.push	Defines the minimum level (inclusive) for sending log entries in buffer to the next Handler	Level.severe
	Memoryhandler.target	Specify the next Handler to undertake the log in the constructor	(Undefined)

How to use:

The above is the internal details of recording the product Exception error log, and how to dump the Memoryhandler processing, and then give some of the Memoryhandler usage.

1. Direct use of Memoryhandler in java.util.logging

Listing 4

Maintain 5 log messages in buffer//only log entries with level greater than or equal to Warning and//flush log entries in buffer to filehandler  int buffersize = 5;   f = new Filehandler ("TestMemoryHandler.log");   m = new Memoryhandler (f, buffersize, level.warning);  ..... MyLogger = Logger.getlogger ("Com.ibm.test");   Mylogger.addhandler (m);   MyLogger.log (level.warning, "This is a WARNING log");

2. customizing

1) Reflection

Consider a scenario in which custom MyHandler inherits from Memoryhandler, because it is not possible to directly use the cursor in size, buffer, and buffer as a private property of the parent class, and if there is a need to acquire and change these properties in MyHandler, one way is Use reflection. Listing 5 shows the use of reflection to read the user configuration and set private properties.

Listing 5

int m_size;   String sizestring = manager.getproperty (Loggername + ". Size");   if (null! = sizestring) {  try {   m_size = Integer.parseint (sizestring);   if (m_size <= 0) {  m_size = buffer_size;//default +//  get private property by Java reflection mechanism  Field F;   f = getclass (). Getsuperclass (). Getdeclaredfield ("size");   F.setaccessible (true);   F.setint (this, m_size);   f = getclass (). Getsuperclass (). Getdeclaredfield ("buffer");   F.setaccessible (true);   F.set (this, new logrecord[m_size]);  } catch (Exception e) {  }   }

2) rewrite

Direct use of reflection is quick and easy, and is suitable for scenes with no frequent access to private properties of the parent class. Considering a scenario where the default ring queue does not meet our storage requirements, it may be possible to make the custom Mymemoryhandler directly inherit the Handler, directly manipulating the storage structure, which can be implemented in Listing 6.

Listing 6

public class Mymemoryhandler extends handler{   //default storage LogRecord buffer capacity  private static final int default_size = 100 0;   Set buffer size  private int size = Default_size;   Set buffer  private logrecord[] buffer;   Refer to Java.util.logging.MemoryHandler for other parts ...  }

Back to top of page

Several issues to be concerned when using Memoryhandler

After understanding the internal details and external applications of the Java log buffering mechanism implemented with Memoryhandler, look at the problems encountered in two specific implementations: the impact of Logger/handler/logrecord level transfer and how to develop The error log is processed during the Memoryhandler process.

1. Impact of level transfer

There are three types of level in Java.util.logging, the level of Logger Level,handler and the level of LogRecord respectively. The first two can be set through the configuration file. The level of the log is then compared with the level of Logger and Handler to filter logs that need not be recorded. When using Java Log, you need to be concerned about the level of interaction between level, especially when traversing Logger binds multiple handlers. 3 is shown below:

Figure 3. The transfer effect of level in Java Log

The Setuseparenthandlers method provided by Java.util.logging.Logger may also affect the log display of the final output terminal. This method allows the user to print a copy of their log entries to the output terminal of the Parent Logger. The missing capital is printed to the Parent Logger terminal. At this point, if the parent Logger level-related settings are different from their own Logger, the log entries that are printed to the parent Logger and itself will also differ. 4 is shown below:

Figure 4. The subclass log needs to be printed to the parent class output terminal

2. Process error log during development of log interface

It is possible to get into an infinite loop by calling its own interface to print log in the development log-related interface. With this in mind, Java.util.logging provides an ErrorManager interface for Handler to report any errors during logging, rather than directly throwing an exception or logging an error or exception on the log-related interface that calls itself. Handler needs to implement the Seterrormanager () method, which constructs Java.util.logging.ErrorManager objects for this application and calls through the ReportError method when an error occurs The ErrorManager error method, which defaults to outputting errors to the standard error stream, or processing the error stream based on a custom implementation in Handler. When you close the error stream, use Logger.removehandler to remove the Handler instance.

Two classic usage scenarios, one is custom Myerrormanager, implements the parent class related interface, calls Myhandler.seterrormanager (New Myeroormanager ()) in the logging program; The other is to customize ErrorManager related methods in Handler, such as listing 7:

Listing 7

public class MyHandler extends handler{  //Implement the Seterrormanager method in the constructor method public MyHandler () {... seterrormanager (new  ErrorManager () {public  void  error (String msg, Exception ex, int code) {  System.err.println ("error reported by MyHandler "  + msg + ex.getmessage ());  }  );}  public void Publish (LogRecord record) {  if (!isloggable (record)) return;  try {  //some actions that might throw an exception} catch (Exception e) {  reportError ("Error occurs in publish", E, Errormanager.write_fail URE);  }  }  ...... }

Back to top of page

Logging.properties

The Logging.properties file is the Java log configuration file, each line is described as "Key=value", you can configure global information for the log and specific log configuration information, listing 8 is the logging.properties that we configured for the test code.

Listing 8. Logging.properties File Example

#Level level OFF > SEVERE > WARNING > INFO > CONFIG > FINE > Finer > FINEST > All  # for Filehandler Specify the log level java.util.logging.filehandler.level=warning  # to specify formatter  for Filehandler Java.util.logging.filehandler.formatter=java.util.logging.simpleformatter  # Specifying the log level for custom Testmemoryhandler Com.ibm.test.memoryhandler.level=info  # Set Testmemoryhandler maximum number of log entries com.ibm.test.testmemoryhandler.size=1000  # Set Custom fields for Testmemoryhandler useparentlevel  com.ibm.test.testmemoryhandler.useparentlevel=warning  # Set the handler of the specific log to Testmemoryhandler  com.ibm.test.handlers=com.ibm.test.testmemoryhandler  # To specify the global handler as Filehandler  Handlers=java.util.logging.filehandler

As you can see from Listing 8, the Logging.properties file is primarily used to assign logger a level, configure handler, and formatter information.

How to monitor Logging.properties

If a system has high security requirements, such as the need to log records of changes to the Logging.properties file, and to record when and who changed which records, what should be done?

You can use the PropertyChangeListener provided by the JDK to listen for changes to the Logging.properties file properties.

For example, to create a Logpropertylistener class that implements the Java.benas.PropertyChangeListener interface, the PropertyChangeListener interface contains only one PropertyChange (Propertychangeevent) method, the implementation of this method is shown in clear 9.

Listing 9. Implementation of PropertyChange method

@Override public  void PropertyChange (Propertychangeevent event) {     if (Event.getsource () instanceof Logmanager ) {         Logmanager manager= (Logmanager) Event.getsource ();         Update (manager);         Execute ();         Reset ();     }  }

The PropertyChange (propertychangeevent) method first calls the update (Logmanager) method to find the changed, added, and deleted items in the Logging.properties file, which is part of the code shown in Listing 10 , and then call the Execute () method to execute the specific logic, see listing 11, and finally call the Reset () method to save and empty the related properties, as shown in Listing 12.

Listing 10. Listen for changed entries

 public void Update (Logmanager manager) {Properties logprops = null;       Use the Java reflection mechanism to get the private property of the try {Field F = manager.getclass (). Getdeclaredfield ("props");       F.setaccessible (TRUE);  Logprops= (Properties) f.get (manager);  }catch (Exception e) {logger.log (Level.severe, "Get private field error.", e);     return;      } set<string> logpropsname=logprops.stringpropertynames ();         for (string logpropname:logpropsname) {string Newval=logprops.getproperty (logpropname). Trim ();           Records the current attribute Newprops.put (Logpropname, newval);  If the attribute was last logged if (Oldprops.containskey (Logpropname)) {String oldval = Oldprops.get (logpropname); if (newval== null? oldval== null:newVal.equals (Oldval)) {//property value does not change, do nothing}else {changedprops.         Put (Logpropname, newval);     } oldprops.remove (Logpropname);                      }else {//If the attribute was not previously recorded, it should be a newly added attribute, recording the Changedprops.put (Logpropname, newval); }      }  } 

In the code, Oldprops, Newprops, and changedprops are all hashmap<string,string> types, and Oldprops stores the contents of the Logging.properties file before the modification. Newprops stores modified logging.properties content, Changedprops is primarily used to store additions or modifications to the section.

The method first obtains the private property in Logmanager through the reflection mechanism of Java props (storing the attribute information in the Logging.properties file), and then obtains the added and modified property information by comparing with Oldprops, and finally The rest of the oldprops is the deleted information.

Listing 11. Specific processing logic method

private void Execute () {   //handles the deleted attribute for  (String Prop:oldProps.keySet ()) {//     This can be added to other processing steps    logger.info ( "'" +prop+ "=" +oldprops.get (prop) + "' has been removed");   }   Handle the change or the newly added attribute for  (String Prop:changedProps.keySet ()) {       //here can join other processing steps  logger.info ("'" +prop+ "=" + Oldprops.get (prop) + "' has been changed or added");   }  }

This method is the main processing logic, the modification or deletion of the properties of the corresponding processing, such as logging property change log and so on. You can also get the current system's logins, and the current time so that you can detail who changed which log entry.

Listing 12. Reset all data Structures

private void Reset () {oldprops = Newprops; newprops= new hashmap< string,string> (); Changedprops.clear ();}

The Reset () method is primarily used to reset individual properties for the next use.

Of course, if you just write a propertychangelistener that does not function properly, you also need to register this PropertyChangeListener instance with Logmanager, which can be implemented in Listing 13.

Listing 13. Register PropertyChangeListener

Register listeners for ' logging.properties ' files Logpropertylistener listener= new Logpropertylistener ();  Logmanager.getlogmanager (). Addpropertychangelistener (listener);

How to implement a custom label

There are a few custom entries in Listing 8, such as Com.ibm.test.TestMemoryHandler.

Useparentlever=warning ", means that if the log level exceeds the level WARNING defined by Useparentlever, the log needs to be passed to the parent log of the corresponding log after Testmemoryhandler processing Handle R is processed (for example, the log context cache information that has occurred WARNING and above is printed to a file), otherwise it is not passed to the parent log's Handler for processing, in which case the Java legacy log mechanism does not support this definition if no processing is done. So how do you make Java Log support this custom label? You can use Propertylistener to handle custom labels so that Java Log supports such custom labels, such as "Useparentlever", which can be implemented in Listing 14.

Listing 14

 private void execute () {//Handle the Deleted property for (String Prop:oldProps.keySet ()) {                 if (Prop.endswith (". Useparentlevel")) {String logname=prop.substring (0, Prop.lastindexof ("."));                  Logger Log=logger.getlogger (logName);                         For (Handler handler:log.getHandlers ()) {if (Handler instanceof Testmemoryhandler) {                          ((Testmemoryhandler) handler). Setuseparentlevel (Oldprops.get (prop));                     break;              }}}}//Handle the change or the newly added attribute for (String Prop:changedProps.keySet ()) { if (Prop.endswith (". Useparentlevel")) {//Add logical processing steps here}}} 

After listing 14 has been processed, it can be judged in a custom testmemoryhandler, comparing the rank of log to its domain useparentlevel, and deciding whether to pass to the Handler of the parent log for processing. Saving the corresponding log information in a custom Testmemoryhandler can easily be accomplished by passing the information to the Handler of the parent log, while saving the corresponding log information can be achieved through propertylistener, for example, listing 15 changes the listing 13 This function is implemented in the corresponding code in the

Listing 15

if (handler  instanceof Testmemoryhandler) {      ((Testmemoryhandler) handler). Setuseparentlevel (Oldprops.get ( prop));      ((Testmemoryhandler) handler). Addlogger (log);        break;  }

How to handle the value of a custom label that depends on the needs of the program, which makes it easy to add a custom label to the Logging.properties.

Custom Read configuration file

If the logging.properties file changes, you need to make the change effective by calling the Readconfiguration (InputStream) method, but you can see it from the source code of the JDK. The Readconfiguration (InputStream) method resets the entire log system, which means that all log levels will be restored to their default values, all log handler will be null, and all stored information will be lost.

For example, Testmemoryhandler caches 1000 LogRecord, and now the user changes the Logging.properties file and calls the Readconfiguration (InputStream) method to make it work. Since the log mechanism of the JDK itself, the testmemoryhandler of the corresponding log after the change is newly created, then the original stored 1000 LogRecord Testmemoryhandler instances will be lost.

So how should the problem be solved? Here are three ways of thinking:

1). Since each Handler has a close () method (any class that inherits from Handler needs to implement the method), the Java Log mechanism calls the close () method of the corresponding Handler before Handler is null, then you can Save the appropriate information in the close () method of the handler (for example, Testmemoryhandler).

2). Study the Readconfiguration (InputStream) method, write an alternative method, and then call the alternate method each time.

3). Inherit the Logmanager class, overriding the Readconfiguration (InputStream) method.

Here the first method is to save the original information, and then restore, but this method is not very practical and efficient, the second and third method is actually the same, is to write an alternative method, such as can be substituted in the method of Handler Testmemoryhandler pail is null, Then, when reading the Logging.properties file as the Testmemoryhandler property, find an instance of the corresponding Testmemoryhandler and change the corresponding property value (this is reflected in listing 14), and the other does not belong to The Testmemoryhandler attribute value can be processed according to the original processing logic of the JDK, such as setting the level of log.

On the other hand, since JDK1.6 and previous versions do not support file modification monitoring, it is necessary to explicitly call Readconfiguration (InputStream) after each modification of the Logging.properties file to make the modification effective, but since JDK1.7 Started to support the file modification monitoring function, mainly in the java.nio.file.* package to provide the relevant API, not detailed here.

Before JDK1.7, you can use the FileMonitor class in the Apache Commons-io Library, which is no longer detailed here.

Back to top of page

Summarize

By defining Memoryhandler and logging.properties, you can implement custom log caching through the Java logs to increase the availability of Java logs and provide stronger support for product quality.

Original: http://www.ibm.com/developerworks/cn/java/j-lo-logbuffer/

Implementation of the Java log caching mechanism--reprint