Logstash Installation and construction (i)

Logstash is an open source data collection engine with real-time pipeline. You can dynamically unify data from different sources and classify the data to different destinations. is also a management event and logging tool. You can use it to collect logs, analyze them, and store them for later use.

Logstash are usually used in conjunction with Kibana and Elasticsearch, but there are many other uses that deserve our attention. Elasticsearch configuration and construction can be viewed on this blog. This article will explain in detail the installation and simple configuration of Logstash.

1. Download Logstash from official website

wget https://download.elastic.co/logstash/logstash/logstash-2.3.2.tar.gz

2. Download the rpm version of Logstash, unzip the official launch script

wget https://download.elastic.co/logstash/logstash/packages/centos/ logstash-2.3.2-1.noarch.rpm

3. Java 8:

http://

4. Configuring the Java Environment

tar zxf jdk-8u91-linux-x64. tar. gz-c/usr/local/vi /etc/profile    export java_home=/usr/local/jdk1.  8. 0_91    export CLASSPATH=.: $JAVA _home/lib/dt.jar: $JAVA _home/lib/tools.jar    Export PATH= $PATH: $JAVA _home//etc/profile

Input java-version If you see the following information, the Java environment is configured successfully

" 1.8.0_91 "  1.8. 0_91-25.91-b14, Mixed mode)

5. Unzip the RPM package

MV logstash-2.3. 2-1. noarch.rpm//tmp/# rpm2cpio Logstash-2.3. 2-1. noarch.rpm | Cpio-div

6. Unpack the TAR package and configure the startup script

#TarZXF logstash-2.3.2.Tar. gz-c/usr/local/# CD/usr/local/# MVlogstash-2.3.2/logstash# Groupadd-R Logstash//Create a Logstash group# Useradd-r-G logstash-d/usr/local/logstash-s/sbin/nologin-c"Logstash"Logstash//Create Logstash UserCopy the script in the RPM package to the system specified location #CP/tmp/etc/init.d/logstash/etc/init.d/# CP/tmp/etc/sysconfig/logstash/etc/sysconfig/# CP/tmp/etc/logrotate.d/logstash/etc/logrotate.d/# chmod 0644/etc/logrotate.d/Logstash Create Logstash log, home, and profile directory #mkdir-p/etc/logstash/conf.d///Configuration file Directory#mkdir/var/log/logstash//Log Directory#mkdir/var/lib/logstash//Home Directory#Chownlogstash/var/log/logstash#Chownlogstash:logstash/var/lib/logstash#Chown-R logstash:logstash/usr/local/logstash/Configure the variables in the startup script to modify them to the actual path of the Logstash #VI/etc/init.d/Logstash Program=/usr/local/logstash/bin/Logstash

At this point, you can put your own Logstash configuration file into the/etc/logstash/conf.d/, and set the boot start.

7. Use simple configuration file test

cat /etc/logstash/conf.d/simple.confinput {  stdin {}}output {  stdout {    = =  Rubydebug}}

Run Logstash with a command

#/usr/local/logstash/bin/logstash-f/etc/logstash/conf.d/simple.conf     //- F  Specifies the configuration file, You can also use the- T parameter to specify a configuration file before you start to check that the configuration is correct 4Pipeline main started

Enter Hello world to see the results of the output

#/usr/local/logstash/bin/logstash-f/etc/logstash/conf.d/simple.conf Settings:default pipeline workers:4Pipeline main Startedhello World!{       "message"="Hello World!",      "@version"="1",    "@timestamp"="2016-06-13t02:35:01.737z",          "Host"="Localhost.localdomain"}

As you can see, what you enter Logstash the output in some format, and you can use the Ctrl-c command to exit the previously running Logstash.

8. Configure Logstash to use Elasticsearch as Logstash back end

cat /usr/local/logstash/conf.d/logstash-es-simple.conf  Input {    stdin {}  }}  Output {    Elasticsearch {      "127.0.0.1"}    stdout {      =  Rubydebug}  }

Execute command

execute command: #/usr/local/logstash/bin/logstash agent-f conf.d/logstash-es-simple.conf Settings:default pipeline workers:4Pipeline Main started hello Logstash {"message"="Hello Logstash",              "@version"="1",            "@timestamp"="2016-06-13t02:39:25.112z",                  "Host"="Localhost.localdomain"        }

Use the Curl command to send a request to see if ES has received data:

# Curl'Http://127.0.0.1:9200/_search?pretty'        {          "took": +,          "timed_out":false,          "_shards" : {            " Total":5,            "successful":5,            "failed":0          },          "hits" : {            " Total":1,                "Max_score":1.0,            "hits" : [ {              "_index":"logstash-2016.06.13",              "_type":"logs",              "_id":"AVRG9UHCZZ2IUIMLMAJG",              "_score":1.0,              "_source" : {                "message":"Hello Logstash",                "@version":"1",                "@timestamp":"2016-06-13t02:39:25.112z",                "Host":"Localhost.localdomain"              }            } ]          }        }

The data is now successfully collected using Elasticsearch and Logstash.

Related tutorials:

http://udn.yyuap.com/doc/logstash-best-practice-cn/

http://kibana.logstash.es/content/

Logstash Installation and construction (i)