Log4j configuration file and log configuration in the nutch

Log4j configuration file and log configuration in the nutch

When SLF4J is used as the log system, SLF4J is only an interface and needs a specific implementation for execution.

Reference

Slf4j unifies API interfaces. Therefore, if log4j is used to output logs, you only need to set the content of the configuration file. The configuration in log4j. properties can be divided into two categories:

The first type defines the log output level and appender)

The second class defines the specific attributes of appender.

1. Definition of log output level and output target (appender)

1. log4j. rootLogger = Log Level, appender1, appender2 ,....
Example: log4j. rootLogger = INFO, stdout, DRFA
(1) set the default log level for the entire project and the log output target
(2) The log level is ALL <DEBUG <INFO <WARN <ERROR <FATAL <OFF, case insensitive. In the example, the log level is INFO, that is, only logs greater than or equal to INFO are recorded.
(3) log output targets include the following types:

ConsoleAppender (console)
FileAppender (file)
DailyRollingFileAppender (a log file is generated every day.
RollingFileAppender (a new file is generated when the file size reaches the specified size)
WriteAppender (send log information to any specified place in stream format)
JDBCAppender (save log information to the database)
In this example, DailyRollingFileAppender is used. The next section details the attributes of each appender.

2. log4j.logger.org. mypackage [. MyClass] = Log Level, appender1, appender2 ,....
Example: log4j.logger.org. apache. nutch. crawl. GeneratorJob = INFO, export stdout
(1) The settings in log4j. rootLogger take effect for the entire project. If you only want to take effect for a package or a class, you can use this configuration item.

3. log4j. threshold = ALL
In fact, threshold's level shocould be larger or equals rootlogger's level.
Usually, thresold can be used to define some special appender's filter log level.
Eg:
There are two appenders: console and file, and the level is warn, if you just only want the error message be written to file, so you can use "file. threshold = error "to filter out the" warn "message.

2. Specific appender settings
1. log4j. appender. appender1.File = file directory and file
2. log4j. appender. appender1.MaxFileSize = Maximum File Size
3. log4j. appender. appender1.MaxBackupIndex = number of backup files

Appender1 is defined in log4j. rootlogger or a specific package log;
File directory and file, for example,/home/admin/logs/huudan. log
Maximum file size, for example, KB
Number of backup files, for example, 1

4. log4j. appender. ServerDailyRollingFile. DatePattern = Log suffix format

For example, '.' yyyy-MM-dd

5. log4j. appender. appender1.layout = org. apache. log4j. Log layout format

HTMLLayout (HTML table layout)
SimpleLayout (Level and information string containing log information)
TTCCLayout (including the log generation time, execution thread, category, and other information)
PatternLayout (you can flexibly specify the layout format, which is commonly used)

6. log4j. appender. appender1.layout. ConversionPattern = log output format

For example, % d-% m % n or % d {yyyy-MM-dd HH: mm: ss} % p [% c] % m % n
% C full name of the class to which the output log information belongs
% D date or time of the log output time point. The default format is ISO8601. You can also specify the format after it, for example, % d {yyy-M-dd HH: mm: ss}, output is similar to:-22:10:28
% F Class Name of the class to which the output log information belongs
% L location where the log event is output, that is, the number of lines in the class where the statement that outputs the log information is located
% M output the specified information in the Code, such as the message in log (message)
% N output a carriage return line break. For Windows, the value is "rn", and for Unix, the value is "n"
% P output priority, that is, DEBUG, INFO, WARN, ERROR, FATAL. If it is output by calling debug (), it is DEBUG, and so on.
% R the number of milliseconds it takes to output the log information from application startup to output.
% T name of the thread that outputs the log event

7. log4j. appender. ServerDailyRollingFile. Append = true

Append

Iii. Example: log4j. properties in Nutch

# Define some default values that can be overridden by system properties
Hadoop. log. dir =.
Hadoop. log. file = hadoop. log

# RootLogger-DailyRollingFileAppender
Log4j. rootLogger = INFO, DRFA

# Logging Threshold
Log4j. threshold = ALL

# Special logging requirements for some commandline tools
Log4j.logger.org. apache. nutch. crawl. Crawl = INFO, export stdout
Log4j.logger.org. apache. nutch. crawl. InjectorJob = INFO, export stdout
Log4j.logger.org. apache. nutch. host. HostInjectorJob = INFO, export stdout
Log4j.logger.org. apache. nutch. crawl. GeneratorJob = INFO, export stdout
Log4j.logger.org. apache. nutch. crawl. DbUpdaterJob = INFO, export stdout
Log4j.logger.org. apache. nutch. host. HostDbUpdateJob = INFO, export stdout
Log4j.logger.org. apache. nutch. fetcher. FetcherJob = INFO, export stdout
Log4j.logger.org. apache. nutch. parse. ParserJob = INFO, export stdout
Log4j.logger.org. apache. nutch. indexer. IndexingJob = INFO, export stdout
Log4j.logger.org. apache. nutch. indexer. DeleteDuplicates = INFO, export stdout
Log4j.logger.org. apache. nutch. indexer. CleaningJob = INFO, export stdout
Log4j.logger.org. apache. nutch. crawl. WebTableReader = INFO, export stdout
Log4j.logger.org. apache. nutch. host. HostDbReader = INFO, export stdout
Log4j.logger.org. apache. nutch. parse. ParserChecker = INFO, export stdout
Log4j.logger.org. apache. nutch. indexer. IndexingFiltersChecker = INFO, comment stdout
Log4j.logger.org. apache. nutch. plugin. PluginRepository = WARN
Log4j.logger.org. apache. nutch. api. NutchServer = INFO, export stdout

Log4j.logger.org. apache. nutch = INFO
Log4j.logger.org. apache. hadoop = WARN
Log4j.logger.org. apache. zookeeper = WARN
Log4j.logger.org. apache. gora = WARN

#
# Daily Rolling File Appender
#

Log4j. appender. DRFA = org. apache. log4j. DailyRollingFileAppender
Log4j. appender. DRFA. File =$ {hadoop. log. dir}/$ {hadoop. log. file}

# Rollver at midnight
Log4j. appender. DRFA. DatePattern =. yyyy-MM-dd

#30-day backup
# Log4j. appender. DRFA. MaxBackupIndex = 30
Log4j. appender. DRFA. layout = org. apache. log4j. PatternLayout

# Pattern format: Date LogLevel LoggerName LogMessage
Log4j. appender. DRFA. layout. ConversionPattern = % d {ISO8601} %-5 p % c {2}-% m % n
# Debugging Pattern format: Date LogLevel LoggerName (FileName: MethodName: LineNo) LogMessage
# Log4j. appender. DRFA. layout. conversionPattern = % d {ISO8601} %-5 p % c {2} (% F: % M (% L)-% m % n

#
# Stdout
# Add * stdout * to rootlogger abve if you want to use this
#

Log4j. appender. stdout = org. apache. log4j. leleappender
Log4j. appender. stdout. layout = org. apache. log4j. PatternLayout
Log4j. appender. stdout. layout. ConversionPattern = % d {ISO8601} %-5 p % c {2} (% F: % M (% L)-% m % n

#
# Plain layout used for commandline tools to output to console
#
Log4j. appender. Logging stdout = org. apache. log4j. leleappender
Log4j. appender. mongostdout. layout = org. apache. log4j. PatternLayout
Log4j. appender. mongostdout. layout. ConversionPattern = % m % n

The basic steps are as follows:
1. Specify log4j. rootLogger

Log4j. rootLogger = INFO, DRFA

[Optional] specify the Log Level and appender for a separate package or Class

# Special logging requirements for some commandline tools
Log4j.logger.org. apache. nutch. crawl. Crawl = INFO, export stdout
Log4j.logger.org. apache. nutch. crawl. InjectorJob = INFO, export stdout
Log4j.logger.org. apache. nutch. host. HostInjectorJob = INFO, export stdout
Log4j.logger.org. apache. nutch. crawl. GeneratorJob = INFO, export stdout
Log4j.logger.org. apache. nutch. crawl. DbUpdaterJob = INFO, export stdout
Log4j.logger.org. apache. nutch. host. HostDbUpdateJob = INFO, export stdout
Log4j.logger.org. apache. nutch. fetcher. FetcherJob = INFO, export stdout
Log4j.logger.org. apache. nutch. parse. ParserJob = INFO, export stdout
Log4j.logger.org. apache. nutch. indexer. IndexingJob = INFO, export stdout
Log4j.logger.org. apache. nutch. indexer. DeleteDuplicates = INFO, export stdout
Log4j.logger.org. apache. nutch. indexer. CleaningJob = INFO, export stdout
Log4j.logger.org. apache. nutch. crawl. WebTableReader = INFO, export stdout
Log4j.logger.org. apache. nutch. host. HostDbReader = INFO, export stdout
Log4j.logger.org. apache. nutch. parse. ParserChecker = INFO, export stdout
Log4j.logger.org. apache. nutch. indexer. IndexingFiltersChecker = INFO, comment stdout
Log4j.logger.org. apache. nutch. plugin. PluginRepository = WARN
Log4j.logger.org. apache. nutch. api. NutchServer = INFO, export stdout

Log4j.logger.org. apache. nutch = INFO
Log4j.logger.org. apache. hadoop = WARN
Log4j.logger.org. apache. zookeeper = WARN
Log4j.logger.org. apache. gora = WARN

Therefore, most of the logs in the packages related to nutch are re-specified to direct to mongostdout, while other logs, such as hadoop, zookeeper, And gora, only re-define the log level without redefining appender, therefore, their logs are directed to DRFA.

[Optional] specify threshold

# Logging Threshold
Log4j. threshold = ALL

2. Specify the attributes of an appender.

Cmstd:

#
# Plain layout used for commandline tools to output to console
#
Log4j. appender. Logging stdout = org. apache. log4j. leleappender
Log4j. appender. mongostdout. layout = org. apache. log4j. PatternLayout
Log4j. appender. mongostdout. layout. ConversionPattern = % m % n

% M output the specified information in the Code, such as the message in log (message)
% N output a carriage return line break. The Windows platform is "rn", and the Unix platform is "n ".

DRFA:

#
# Daily Rolling File Appender
#

Log4j. appender. DRFA = org. apache. log4j. DailyRollingFileAppender
Log4j. appender. DRFA. File =$ {hadoop. log. dir}/$ {hadoop. log. file}

# Rollver at midnight
Log4j. appender. DRFA. DatePattern =. yyyy-MM-dd

#30-day backup
# Log4j. appender. DRFA. MaxBackupIndex = 30
Log4j. appender. DRFA. layout = org. apache. log4j. PatternLayout

# Pattern format: Date LogLevel LoggerName LogMessage
Log4j. appender. DRFA. layout. ConversionPattern = % d {ISO8601} %-5 p % c {2}-% m % n
# Debugging Pattern format: Date LogLevel LoggerName (FileName: MethodName: LineNo) LogMessage
# Log4j. appender. DRFA. layout. conversionPattern = % d {ISO8601} %-5 p % c {2} (% F: % M (% L)-% m % n

The output file can be specified with variables, as shown in the following figure:

Log4j. appender. DRFA. File =$ {hadoop. log. dir}/$ {hadoop. log. file}

Log4j configuration details

For more information about Apache Log4j 2, see: http://logging.apache.org/log4j/2.x/

Log4j getting started

Detailed Log4j log usage

Configure Log4j in Hibernate to display SQL Parameters

Log4j learning notes (1) _ Log4j basics & configuration item Parsing

Log4j learning notes (2) _ Log4j configuration example & integrate Log4j with Spring

Log4j details: click here
Log4j: click here

This article permanently updates the link address: