Enterprise Log analysis Linux system message collection display

Previously wrote the collection of Linux system History command, the following describes the system of log collection and display.

The usual, look at the effect first, satisfied with the words continue to see.

One

1. Overview

650) this.width=650; "src=" http://s3.51cto.com/wyfs02/M02/74/DD/wKioL1YthELSF2CGAAVVoW2EuIs388.jpg "title=" 11.png "alt=" Wkiol1ythelsf2cgaavvow2euis388.jpg "/>

2. The total amount of data collected by Linux system log

650) this.width=650; "src=" http://s3.51cto.com/wyfs02/M00/74/E1/wKiom1YthEmDRT6dAACF5-7tbtg841.jpg "title=" 11.png "alt=" Wkiom1ythemdrt6daacf5-7tbtg841.jpg "/>

Mainly to show the total amount of content collection log for the selected range

3, Linux system log collection host number

650) this.width=650; "src=" http://s3.51cto.com/wyfs02/M02/74/E1/wKiom1YthJfy7zvhAAB1VRxXbUA808.jpg "title=" 11.png "alt=" Wkiom1ythjfy7zvhaab1vrxxbua808.jpg "/>

Mainly shows how many hosts have uploaded message log data in total at present

4. Linux System log program type TOP5

650) this.width=650; "src=" http://s3.51cto.com/wyfs02/M00/74/DE/wKioL1YthQbR556rAACUzYEf4Nw154.jpg "title=" 11.png "alt=" Wkiol1ythqbr556raacuzyef4nw154.jpg "/>

The main is to display the collected log information in the first 5 program names; My 5 are all Docker servers, so there are a lot of Docker logs.

5, the Linux system log time data total graph

650) this.width=650; "src=" http://s3.51cto.com/wyfs02/M01/74/DE/wKioL1YthWKCJ96bAAD8n97EoUk943.jpg "title=" 11.png "alt=" Wkiol1ythwkcj96baad8n97eouk943.jpg "/>

The main is to show the amount of data collected per time period

6. Linux System Log data

650) this.width=650; "src=" http://s3.51cto.com/wyfs02/M02/74/DE/wKioL1Ythe-guwNCAALjQ8a9w6c267.jpg "title=" 11.png "alt=" Wkiol1ythe-guwncaaljq8a9w6c267.jpg "/>

The main is to show detailed log data

The process of installing elk can refer to the previous article, the address is http://dl528888.blog.51cto.com/2382721/1703059

Ii. Collection of logs

I collected the system/var/log/messages log and sent it to Logstash via the Rsyslog TCP 8514 port.

1, Configuration Rsyslog

The default rsyslog is installed, so you only need to modify the configuration

Add in/etc/rsyslog.conf

* * @ @localhost: 8514

Then restart Rsyslog

2, Configuration Logstash

[[email protected] tmp]# Cat/etc/logstash/conf.d/logstash_agent.confinput {tcp {port = ' 8514 ' type ' = ' Syslog "}}filter {if [type] = =" Syslog "{grok {match + = {" Message "="%{syslogtimestamp:syslog_times TAMP}%{sysloghost:syslog_hostname}%{data:syslog_program} (?: \  [%{posint:syslog_pid}\])?:%{greedydata:syslog_message} "}}}}output {redis {host = = [" 10.10.125.8:6379 "]data_type = "List" key = "Logstash:redis"}}

3. Import templates

Import Order

1. Linux System log attempt. JSON

2, Linux System log search. JSON

3. Linux System log Dashboard.json

Other questions can leave a message.

This article is from the "Yin-Technical Exchange" blog, please be sure to keep this source http://dl528888.blog.51cto.com/2382721/1706204

Enterprise Log analysis Linux system message collection display