I. Environmental preparedness
Role |
SERVER IP |
Logstash Agent |
10.1.11.31 |
Logstash Agent |
10.1.11.35 |
Logstash Agent |
10.1.11.36 |
Logstash Central |
10.1.11.13 |
Elasticsearch |
10.1.11.13 |
Redis |
10.1.11.13 |
Kibana |
10.1.11.13 |
The architecture diagram is as follows:
650) this.width=650; "src=" Http://s4.51cto.com/wyfs02/M02/76/67/wKiom1ZSrUfA29-4AABBqXRaH_Y419.png "title=" Elk.png "alt=" Wkiom1zsrufa29-4aabbqxrah_y419.png "/>
The entire process is as follows:
1) The Logstash agent of the remote node collects the local log and sends it to the remote Redis list queue
2) using Redis as the middleware of log collection, can stage the log data of remote node, play data buffering, and improve the function of concurrency.
3) Central Logstash reads data from Redis and local log files to Elasticsearch for storage, index
4) Kibana reads data from the Elasticsearch and presents it to the user via the web GUI
Two. Installation
The installation of elk is simple, just download the binary package and unzip it, the required binary package is as follows:
Elasticsearch-1.7.1.tar.gz
Kibana-4.1.1-linux-x64.tar.gz
Logstash-1.5.3.tar.gz
1) Start Redis (10.1.11.13)
After the official download of the Redis source code compiled installation, after the following configuration to start:
#调整内核参数: echo 1 > /proc/sys/vm/overcommit_memory echo never > /sys/ kernel/mm/transparent_hugepage/enabledecho 524288 > /proc/sys/net/core/somaxconn# Modify the Redis configuration file as follows:[[email protected] logstash]# cat /etc/redis-logstash.confdaemonize yespidfile /data/redis-logstash/run/redis.pidport 6377tcp-backlog 511timeout 0tcp-keepalive 0loglevel noticelogfile "/data/redis-logstash/log/redis.log" databases 16save 900 1save 300 10save 60 10000stop-writes-on-bgsave-error yesrdbcompression yesrdbchecksum yesdbfilename dump.rdbdir /data/redis-logstash/ Dbslave-serve-stale-data yesslave-read-only yesrepl-diskless-sync norepl-diskless-sync-delay 5repl-disable-tcp-nodelay noslave-priority 100appendonly noappendfilename " Appendonly.aof "appendfsync everysecno-appendfsync-on-rewrite noauto-Aof-rewrite-percentage 100auto-aof-rewrite-min-size 64mbaof-load-truncated yeslua-time-limit 5000slowlog-log-slower-than 10000slowlog-max-len 128latency-monitor-threshold 0notify-keyspace-events "" hash-max-ziplist-entries 512hash-max-ziplist-value 64list-max-ziplist-entries 512list-max-ziplist-value 64set-max-intset-entries 512zset-max-ziplist-entries 128zset-max-ziplist-value 64hll-sparse-max-bytes 3000activerehashing yesclient-output-buffer-limit normal 0 0 0client-output-buffer-limit slave 256mb 64mb 60client-output-buffer-limit pubsub 32mb 8mb 60hz 10aof-rewrite-incremental-fsync yesmaxmemory 32212254720maxmemory-policy allkeys-lru# Start redis/usr/local/bin/redis-server /etc/redis-logstash.conf
2) Install Logstash agent (10.1.11.31/35/36)
Unzip logstash-1.5.3.tar.gz to/usr/local
cd /usr/localln -s logstash-1.5.3 logstash# Create the/etc/logstash directory, the user saves the agent-side rule file mkdir / etc/logstash# Configure the agent side to collect the Tomcat log rules vim /etc/logstash/tomcat_api.conf# Configure the log input source input { file { type => "Tomcat_api" #指定日志类别名称 path => "/data/logs/bd_api/api" #指定日志路径 start_position => " Beginning " #从日志文件首部开始收集 }} #过滤规则配置filter { if [type] == "Tomcat_api" { # Multiline is used to merge multiple rows of logs into a single line, because Java's exception will have multiple lines, but it should be treated as a log record multiline { patterns_dir = > "/usr/local/logstash/patterns" #patterns_dir用于指定patterns文件的位置, patterns file holds regular expressions for matching log fields pattern => "^%{timestamp_iso8601}" # Specify a matching pattern negate => true #true表示只要不匹配pattern的行都会进行合并, default is false what => "Previous" # Lines matching pattern are appended to the front of the merge row as a log output } #grok用来进行日志字段的解析 grok { patterns_dir => "/usr/local/ Logstash/patterns " match => { "message" => "%{log4jlog}" &nbSP;} #在 Create%{log4jlog} in/usr/local/logstash/patterns as follows: #LOG4JLOG &NBSP;%{TIMESTAMP_ iso8601:datetime}\s+ (? <thread>\s+) \s+ (? <line>\d+) \s+ (? <level>\s+) \s+ (? <class>\S+) \s+-\ S+ (<msg>.*) #mutate可以对字段的内容进行替换 mutate { replace => [ "Host", "10.1.11.31"] &NBSP;&NBSP;&NBSP;&NBSP;}&NBSP;&NBSP;&NBSP;&NBSP,}} #日志输出源output { #规则调试时开启 #stdout { codec => "Rubydebug" } # Output log data to a remote redis list redis { host = > "10.1.11.13" port => 6377 data_type => "List" key => "Tomcat_api" &NBSP;&NBSP;&NBSP;&NBSP;}
3) Install Central Logstash (10.1.11.13)
Unzip logstash-1.5.3.tar.gz to/usr/local
cd /usr/localln -s logstash-1.5.3 logstash# Create the/etc/logstash directory, The user saves the rules file for the central and local agents mkdir /etc/logstash# There are two rule files created here/etc/logstash/├── central.conf #保存central端的logstash规则 └── tomcat_uat.conf #保存本地agent的logstash规则vim central.confinput { # #product #从redis中获取类别为tomcat_api的日志 redis { host => "127.0.0.1" port => 6377 type => "Redis-input" data_type => "List" key => "Tomcat_api" } # Get the category Tomcat_editor log from Redis redis { host => "127.0.0.1" port => 6377 type => "Redis-input" data_type => "list" key => "Tomcat_editor" }}output { #stdout { codec => "Rubydebug" } #日志输出到elasticsearch进行索引 elasticsearch { flush_size => 50000 idle_flush_time => 10 cluster => "logstash-1113" host => ["127.0.0.1:9300"] workers => 2 }}#------------------------------------------------------------- ----Vim tomcat_uat.conf input { file { type => "Tomcat_api_ab" path => "/data/logs/bd_api/errors/api_error" start_position => "Beginning" } file { path => "/data/logs/bd_ Admin/admin " type => " tomcat_9083 " start_position => "Beginning" }}filter { if [type] in ["Tomcat_api_ab", "tomcat_9083"] { &Nbsp; multiline { patterns_ dir => "/usr/local/logstash/patterns" pattern => "^%{timestamp_iso8601}" negate => true what => "Previous" } grok { patterns_dir => "/usr/local/ Logstash/patterns " match => { "message" => "%{log4jlog}" } } mutate { replace => [ "Host", "10.1.11.13"] } }}output { #stdout { codec => "Rubydebug" } elasticsearch { flush_size => 50000 idle_flush_time = > 10 cluster => "logstash-1113" host => ["127.0.0.1:9300"] workers => 2 }}
4) Install Elasticsearch
#解压elasticsearch -1.7.1.tar.gz to/usr/localtar XF elasticsearch-1.7.1.tar.gz-c/usr/localcd/usr/localln-s elasticsearch-1.7.1 Elasticsearch[[email protected] config]# egrep-v ' ^#|^$ ' elasticsearch.yml #指定集群名称cluster. Name: logstash-1113# Data Index directory path path.data:/data/logstash/els/data# Data Temp directory path path.work:/data/logstash/els/work# Log path Path.logs:/data/logstash/els/logs# an issue that prompted the inability to connect elasticsearch when the access Kibana was resolved (previously Kibana3) http.cors.enabled:true# Adjust the JVM to memory size Vim/usr/local/elasticsearch/bin/elasticsearch.in.shif ["X$es_min_mem" = "x"]; Then Es_min_mem=4gfiif ["X$es_max_mem" = "x"]; Then ES_MAX_MEM=16GFI
5) Install Kibana
#解压kibana -4.1.1-linux-x64.tar.gz to/usr/localtar XF kibana-4.1.1-linux-x64.tar.gz-c/usr/localcd/usr/localln-s Kibana-4.1.1-linux-x64 Kibana
Three. Start Elk
Central Terminal start:
# # # starting Logstash ###/usr/local/elasticsearch/bin/elasticsearch-d | | /bin/truenohup/usr/local/logstash/bin/logstash agent-f/etc/logstash/central.conf-l/data/logstash/log/ Logstash-central.log &>/data/logstash/log/logstash-central.out | | /bin/true &sleep 3nohup/usr/local/logstash/bin/logstash agent-f/etc/logstash/tomcat_uat.conf-l/data/logstash/ Log/logstash-uat.log &>/data/logstash/log/logstash-uat.out | | /bin/true &sleep 1nohup/usr/local/kibana/bin/kibana &>/dev/null | | /bin/true &
Agent-side boot:
# # # starting Logstash api-agent ###/usr/bin/nohup/usr/local/logstash/bin/logstash agent-f/etc/logstash/tomcat_ Api.conf-l/data/logstash/log/logstash-api.log &>/dev/null | | /bin/true &
Copy the above command to the/etc/rc.local to enable automatic start-up
Logstash+elasticsearch+kibana Log Collection