Elk+redis Log Flow Show

Business Process Architecture Diagram:

650) this.width=650; "src=" Http://s3.51cto.com/wyfs02/M00/8B/0F/wKiom1hCySCiSmlZAABCPg7XKrQ543.png "title=" Aaaa.png "alt=" Wkiom1hcyscismlzaabcpg7xkrq543.png "/>

A set of data collection and analysis system based on Logstash,redis,elasticsearch,kibana

Schema Diagram Description:

Log Collection system: (data source) the logging behavior generated by the producer, collected and forwarded by the Logstash, then transmitted to the Redis sequence, and finally through the Elk Processing Analysis display.

Elasticsearch will store the data, index (based on lunce), and then kibana to establish a link to the Elasticsearch, real-time capture the data after the index, so that the data can be displayed in real time, through some data assembly, query conditions, Get the results we want (can be shown in various ways, such as table, table, line chart, etc.)

：

650) this.width=650; "Src=" Http://s5.51cto.com/wyfs02/M01/8B/0F/wKiom1hCy2rh2idsAACBj59aZcc120.png-wh_500x0-wm_3 -wmp_4-s_191101281.png "title=" Xiao.png "alt=" Wkiom1hcy2rh2idsaacbj59azcc120.png-wh_50 "/>

####

First, Elk Construction

http://dbspace.blog.51cto.com/6873717/1879106

Second, Redis deployment

Note that Redis must be above version 3.0, or there will be a variety of problems before encountering pits

Iii. writing Logstash Log collection configuration according to the schema

[Email protected] conf]# cat/usr/local/logstash/conf/logstash_client.conf

input{

file{

Type = "Log_count"

Path = "/var/log/messages"

}

}

output{

redis{

Host = "192.168.1.114"

Port = 6379

data_type = "List"

Key = "Key_count"

db = 0

}

}

Iv. writing Logstash configuration from Redis read data forwarding to Elk

[Email protected] conf]# cat/usr/local/logstash/conf/logstash_server.conf

input{

redis{

Host = "192.168.1.114"

Port = 6379

Type = "Redis-input"

data_type = "List"

Key = "Key_count"

db = 0

}

}

output{

elasticsearch{

hosts = "192.168.1.114:9200"

codec = "JSON"

}

}

# # #红色部分要相同

Start 2 Services

/usr/local/logstash/bin/logstash-f/usr/local/logstash/conf/logstash_server.conf &

/usr/local/logstash/bin/logstash-f/usr/local/logstash/conf/logstash_client.conf &

Five, pie chart analysis

650) this.width=650; "Src=" Http://s5.51cto.com/wyfs02/M00/8B/0C/wKioL1hC0pvCf977AAEwLOGFG-8367.png-wh_500x0-wm_3 -wmp_4-s_2474034373.png "title=" A.png "alt=" Wkiol1hc0pvcf977aaewlogfg-8367.png-wh_50 "/>

This article is from the "DBSpace" blog, so be sure to keep this source http://dbspace.blog.51cto.com/6873717/1879232

Elk+redis Log Flow Show