logstash收集java日誌,多行合并成一行

標籤：sts   外掛程式   start   app   tin   成功   shard   output   service   

使用codec的multiline外掛程式實現多行匹配，這是一個可以將多行進行合并的外掛程式，而且可以使用what指定將匹配到的行與前面的行合并還是和後面的行合并。
1.java日誌收集測試

input {    stdin {        codec => multiline {            pattern => "^\["                        //以"["開頭進行正則匹配            negate => true                          //正則匹配成功            what => "previous"                      //和前面的內容進行合并        }    }}output {    stdout {        codec => rubydebug     }}

2.查看elasticsearch日誌,已"["開頭

# cat /var/log/elasticsearch/cluster.log [2018-05-29T08:00:03,068][INFO ][o.e.c.m.MetaDataCreateIndexService] [node-1] [systemlog-2018.05.29] creating index, cause [auto(bulk api)], templates [], shards [5]/[1], mappings [][2018-05-29T08:00:03,192][INFO ][o.e.c.m.MetaDataMappingService] [node-1] [systemlog-2018.05.29/DCO-zNOHQL2sgE4lS_Se7g] create_mapping [system][2018-05-29T11:29:31,145][INFO ][o.e.c.m.MetaDataCreateIndexService] [node-1] [securelog-2018.05.29] creating index, cause [auto(bulk api)], templates [], shards [5]/[1], mappings [][2018-05-29T11:29:31,225][INFO ][o.e.c.m.MetaDataMappingService] [node-1] [securelog-2018.05.29/ABd4qrCATYq3YLYUqXe3uA] create_mapping [secure]

3.配置logstash

#vim /etc/logstash/conf.d/java.confinput {        file {                path => "/var/log/elasticsearch/cluster.log"                type => "elk-java-log"                start_position => "beginning"                stat_interval => "2"                codec => multiline {                        pattern => "^\["                        negate => true                        what => "previous"                }        }}output {        if [type] == "elk-java-log" {                elasticsearch {                        hosts => ["192.168.1.31:9200"]                        index => "elk-java-log-%{+YYYY.MM.dd}"                }        }}

4.啟動

logstash -f /etc/logstash/conf.d/java.conf -tsystemctl restart logstash 

5.head外掛程式查看

6.kibana添加日誌

logstash收集java日誌,多行合并成一行