Elk log Processing uses Logstash to collect log4j logs __elk

Describes how to export log4j logs to Logstash from Java projects. First, log4j Foundation

Cannot exception the official introduction:

Log4j is a reliable, fast, flexible log framework (API) written in the Java language, and is licensed using Apache Software License. It is ported to C, C + +, C #, Perl, Python, Ruby, and Eiffel languages.

The log4j is highly configurable and is configured at run time using an external configuration file. It records logs according to priority, and directs log information to various media, such as databases, files, consoles, Unix syslog, and so on.

Log4j mainly consists of three parts: Loggers: Responsible for collecting log information. Appenders: Responsible for publishing the log information to different places. Layouts: Responsible for formatting log information in a variety of styles. Second, the new Java project

Below, learn how to configure log4j with the actual engineering configuration.
Open Eclipse or IntelliJ idea and create a new MAVEN project. The engineering directory structure is shown in the following illustration:

Pom.xml file to add log4j dependencies, version 1.2.17,pom.xml in the following code:

<?xml version= "1.0" encoding= "UTF-8"?> <project "xmlns="
         xmlns: Xsi= "Http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemalocation= "http://maven.apache.org/POM/4.0.0 http ://maven.apache.org/xsd/maven-4.0.0.xsd ">
    <modelVersion>4.0.0</modelVersion>

    < groupid>com.logs</groupid>
    <artifactId>log4idemo1</artifactId>
    <version>1.0 -snapshot</version>
    <!--https://mvnrepository.com/artifact/log4j/log4j-->
    <dependencies >
        <dependency>
            <groupId>log4j</groupId>
            <artifactid>log4j</ artifactid>
            <version>1.2.17</version>
        </dependency>
    </dependencies>

</project>

Create a new log4j.properties in the Resource directory and add the following configuration:

### Set ### Log4j.rootlogger = Debug,stdout,d,e,logstash ### output information to control lift ### log4j.appender.stdout = Org.apache.log4j.Console
Appender Log4j.appender.stdout.Target = System.out Log4j.appender.stdout.layout = org.apache.log4j.PatternLayout Log4j.appender.stdout.layout.ConversionPattern = [%-5p]%d{yyyy-mm-dd hh:mm:ss,sss} method:%l%n%m%n ### output debug
Level above log to =/users/bee/documents/elk/log4j/debug.log### LOG4J.APPENDER.D = Org.apache.log4j.DailyRollingFileAppender
Log4j.appender.d.file =/users/bee/documents/elk/log4j/debug.log Log4j.appender.d.append = True
Log4j.appender.d.threshold = DEBUG Log4j.appender.d.layout = org.apache.log4j.PatternLayout Log4j.appender.d.layout.conversionpattern =%-d{yyyy-mm-dd HH:mm:ss} [%t:%r]-[%p]%m%n ### Output The log above the error level to =/user
S/bee/documents/elk/log4j/error.log ### LOG4J.APPENDER.E = Org.apache.log4j.DailyRollingFileAppender
Log4j.appender.e.file =/users/bee/documents/elk/log4j/error.log Log4j.appender.e.append = True Log4j.appender.e.threshold = ERror log4j.appender.e.layout = org.apache.log4j.PatternLayout Log4j.appender.e.layout.conversionpattern =%-d{ 
Yyyy-mm-dd HH:MM:SS} [%t:%r]-[%p]%m%n #输出日志到logstash log4j.appender.logstash=org.apache.log4j.net.socketappender
log4j.appender.logstash.remotehost=127.0.0.1 log4j.appender.logstash.port=4560 log4j.appender.logstash.reconnectiondelay=60000 Log4j.appender.logstash.locationinfo=true

configuration file, the log output four copies: the first output to the console second to the debug level of the log to the file of the third to the output error level above the log to the file fourth output to Logstash

Add Log4jtest.java in the Java directory as follows:

Import Org.apache.log4j.Logger;
/**
 * Created by Bee on 17/3/6.
 * * Public
class Log4jtest {public
    static final Logger Logger=logger.getlogger (log4jtest.class);

    public static void Main (string[] args) {
        Logger.debug (' This is a debug message! ');
        Logger.info ("This is Info message!");
        Logger.warn ("This is a warn message!");
        Logger.error ("This is Error message!");

        try{
           System.out.println (5/0);
        } catch (Exception e) {
            logger.error (e);
        }
    }
}

Third, the configuration Logstash

(Please click here for Logstash's installation and Hello World Tutorials http://blog.csdn.net/napoay/article/details/53276758) Here, using logstash2.3.3 and Elasticsearch 2.3.3, start Elasticsearch First, and then create a new profile logstash-2.3.3/conf in the log4j-es.conf directory, which reads as follows:

Input {
    log4j {
        host => ' 127.0.0.1 '
        port => 4560
    }
}

output {stdout
    {codec
      = > Rubydebug
    }
    elasticsearch{
        hosts => ["localhost:9200"]
        index => "log4j-%{+yyyy". MM.DD} "
        document_type =>" Log4j_type}
}

The log output specified in the configuration file is 2 copies, one output to the console, and one output to Elasticsearch.
Start Logstash:

sudo./bin/logstash-f conf/log4j-es.conf

If you have already started the ELASTICSEARCH,IP and the ports are unobstructed, the profile is correct and the startup successful interface is as follows:

Running Log4jtest.java, you can see the following output in the terminal:

To view imported logs in Elasticsearch:
Iv. Summary

The above configuration completes the production of the log, to Logstash, and then to Elasticsearch, the introduction finished.