Logstash 6.x collecting syslog logs

Source: Internet
Author: User
Tags locale syslog system log port number logstash rsyslog
1, Logstash end

Close the rsyslog of the Logstash machine and release the 514 port number

[Root@node1 config]# systemctl stop Rsyslog
[root@node1 config]# systemctl status Rsyslog
Rsyslog.service-sys TEM Logging Service
   loaded:loaded (/usr/lib/systemd/system/rsyslog.service; enabled; vendor preset:enabled)
   Active:inactive (dead) since Thu 2018-04-26 14:32:34 CST; 1min 58s ago
  process:3915 execstart=/usr/sbin/rsyslogd-n $SYSLOGD _options (code=exited, status=0/success)
 Main pid:3915 (code=exited, status=0/success)

Apr 14:25:16 Node1 systemd[1]: Starting System Logging Service...
  APR 14:25:16 node1 systemd[1]: Started System Logging Service.
APR 14:32:34 Node1 systemd[1]: stopping System Logging Service
... APR 14:32:34 node1 systemd[1]: Stopped System Logging Service.
[Root@node1 config]#

Writing the Logstash configuration file

[Root@node1 logstash-6.2.3]# VI config/local_syslog.conf
[root@node1 logstash-6.2.3]# cat Config/local_ syslog.conf
Input {
 syslog {
    type = "Rsyslog"
    port = "514"
  }
}
output{
  stdout{
    codec = Rubydebug
  }
}

Start Logstash

[Root@node1 logstash-6.2.3]# bin/logstash-f config/local_syslog.conf sending Logstash ' s logs to/var/log/logstash which is now configured via log4j2.properties [2018-04-26t14:39:57,627][info][logstash.modules.scaffold] Initializing Module {:module_name=> "NetFlow",:d irectory=> "/opt/logstash-6.2.3/modules/netflow/configuration"} [ 2018-04-26t14:39:57,650][info][logstash.modules.scaffold] Initializing module {:module_name=> "Fb_apache",: Directory=> "/opt/logstash-6.2.3/modules/fb_apache/configuration"} [2018-04-26t14:39:58,301][warn] [ 
Logstash.config.source.multilocal] Ignoring the ' pipelines.yml ' file because modules or command line options is specified [2018-04-26t14:39:59,346] [INFO] [Logstash.runner] Starting Logstash {"Logstash.version" = "6.2.3"} [2018-04-26t14:40:00,022][info][logstash.a Gent] Successfully started Logstash API endpoint {:p ort=>9600} [2018-04-26t14:40:04,438][info][logstash.pip Eline] Starting pipeline {:p IPELINE_ID=&Gt; " Main "," Pipeline.workers "=>4," Pipeline.batch.size "=>125," Pipeline.batch.delay "=>50} [2018-04-26t14 : 40:04,901][info][logstash.pipeline] Pipeline started succesfully {:p ipeline_id=> "main",:thread=> "#<thr Ead:0x518728c7 run> "} [2018-04-26t14:40:04,989][info][logstash.inputs.syslog] starting syslog UDP listener {: Addre Ss=> "0.0.0.0:514"} [2018-04-26t14:40:05,013][info][logstash.inputs.syslog] starting syslog TCP Listener {: Address = "0.0.0.0:514"} [2018-04-26t14:40:05,034][info][logstash.agent] Pipelines running {: count=>1,:p ipeline
 s=>["main"]}

View Port Numbers

[Root@node1 config]# netstat-anp|grep 514
tcp6       0      0::: 514                  :::*                    LISTEN      4260/java           
UDP        0      0 0.0.0.0:514             0.0.0.0:*                           4260/java           
Unix  2      [ACC]     STREAM     LISTENING     15141    822/mcelog           /var/run/mcelog-client
Unix  2      []         dgram                    15147    828/chronyd          
[Root@node1 config]#

found that port 514 has been Logstash 2, syslog terminal

Switch to another server Node2, configure syslog

[Root@node2 ~]# vi/etc/rsyslog.conf

Add a line of *. * @ @node1:514, and send the logs to the remote Logstash.

# remote host is:name/ip:port, e.g 192.168.0.1:514, port optional
#*.* @ @remote-host:514
* * @ @node1:514

Re-RSYSLOGD the program

[Root@node2 ~]# systemctl Restart Rsyslog
3. Collect data at logstash end

At this point, we find that the Logstash end has collected the Syslog log data of Node2

[2018-04-26t14:45:18,361] [INFO] [Logstash.inputs.syslog] New connection {:client=> "10.17.12.157:55204"} {"Severity_label" = "Informational"
              , "Facility_label" = "System", "timestamp" and "APR-14:39:23", "Severity" and "6", "Host" = "10.17.12.157", "message" = "Stopping System Logging service...\n", "@version "=" 1 "," program "=" Systemd "," @timestamp "and" 2018-04-26t06:39:23.000z "," Typ
    E "=" rsyslog "," priority "=" logsource "=" Node2 "," facility "= 3} { "Severity_label" = "informational", "Facility_label" and "system", "timestamp" = "APR 26 14:39:23" "," severity "= 6," host "=" 10.17.12.157 "," message "=" Stopped System Log " Ging service.\n "," @version "and" 1 "," program "=" Systemd "," @timestamp "and" = " 2018-04-26t06:39:23.000z, "type" = "Rsyslog", "priority", "Logsource" and "=" "Node2", "facility" = 3} {"Severity_label" = "Notice", "Facility_label" and "Security/autho" Rization "," timestamp "=" Apr 14:39:23 "," severity "+ 5," host "=" 10.17.1 " 2.157 "," message "=" Unregistered authentication Agent for unix-process:4601:59761164 (System bus name:1.2
          556, Object path/org/freedesktop/policykit1/authenticationagent, Locale En_us.utf8) (disconnected from bus) \ n ",
              "@version" and "1", "program" = "POLKITD", "@timestamp" and "2018-04-26t06:39:23.000z", "Type" = "Rsyslog", "priority" = +, "pid" = "762", "logsource" = "Node2", "facility" = "Severity_label", "informational", "Facility_label" and "system" , "timestamp" => "APR 14:40:01", "severity" + 6, "host" = "10.17.12.157", "message" =&gt ; "Started Session 1235 of user root.\n", "@version" and "1", "program" = "Systemd", "@ti Mestamp "= 2018-04-26t06:40:01.000z," type "=" Rsyslog "," Priority "," Lo Gsource "+" Node2 "," facility "= 3} {" Severity_label "=" informational "," Facility_label " = "System", "timestamp" = "Apr 14:40:01", "Severity" and "6", "Host" and "10"
           .17.12.157 "," message "=" Starting Session 1235 of user root.\n "," @version "and" 1 ",
          "Program" = "Systemd", "@timestamp" = 2018-04-26t06:40:01.000z, "type" = "Rsyslog", "Priority" = "logsource" = "Node2", "facility" = 3} {"Severity_label" =&G T
  "Informational",  "Facility_label" = "clock", "timestamp" and "APR-14:40:01", "Severity" and "6", "Host" = "10.17.12.157", "message" = "(root) CMD (/USR/LIB64/SA/SA1 1 1) \ n", "@version" =&G T "1", "program" = "CROND", "@timestamp" and "= 2018-04-26t06:40:01.000z", "type" and "="
          Rsyslog "," priority "= +," pid "=" 4640 "," Logsource "and" Node2 ",
         "Facility" = 9} {"Severity_label" = "Notice", "Facility_label" and "Security/authorization", "Timestamp" = "Apr 14:45:18", "Severity" and 5, "host" and "10.17.12.157", " Message "= =" Registered authentication Agent for unix-process:4786:59796608 (System bus name:1.2559 [/usr/bin/pkttyag
          ENT--notify-fd 5--fallback], object path/org/freedesktop/policykit1/authenticationagent, locale En_us.utf8) \ n ", "@version" = "1"Program" = "POLKITD", "@timestamp" and "= 2018-04-26t06:45:18.000z", "type" and "=" Rsyslog "," priority "=--" pid "=" 762 "," logsource "=" Node2 "," Facility "= ten} {" Severity_label "=" informational "," Facility_label "and" system "," Timestam P "=" Apr 14:45:18 "," severity "= 6," host "=" 10.17.12.157 "," message "
        = "Starting System Logging service...\n", "@version" and "1", "program" = "Systemd",
         "@timestamp" = 2018-04-26t06:45:18.000z, "type" = "Rsyslog", "priority" = 30, "Logsource" = "Node2", "facility" = 3} {"Severity_label" = "informational", "facility_l Abel "and" System "," timestamp "and" APR-14:45:18 "," Severity "and" 6 "," host "=&G T
     "10.17.12.157",      "Message" = "Started System Logging service.\n", "@version" and "1", "program" = "SYS" Temd "," @timestamp "= 2018-04-26t06:45:18.000z," type "=" Rsyslog "," priority "=&G T "Logsource" = "Node2", "facility" = 3} {"Severity_label" = "Notice", "Facilit
              Y_label "=" and "Security/authorization", "timestamp" and "APR-14:45:18", "Severity" and 5, "Host" = "10.17.12.157", "message" = "Unregistered authentication Agent for unix-process:4786: 59796608 (System bus name:1.2559, Object path/org/freedesktop/policykit1/authenticationagent, Locale En_us.utf8) ( Disconnected from bus) \ n "," @version "=" 1 "," program "=" POLKITD "," @timestamp "=&gt ; 2018-04-26t06:45:18.000z, "type" = "Rsyslog", "priority" =--"pid" = "762", "LOGSOURCE "=" Node2 "," facility "=" Severity_label "+" informational "," facility_label "+ = "Syslogd", "timestamp" and "Apr 14:45:18", "severity" = 6, "host" and "10.17". 12.157 "," message "=" [Origin software=\ "rsyslogd\" swversion=\ "7.4.7\" x-pid=\ "4792\" x-info=\ "http://www.  Rsyslog.com\ "] start\n", "@version" and "1", "program" = "Rsyslogd", "@timestamp" and "="  2018-04-26t06:45:18.000z, "type" = "Rsyslog", "priority", "Logsource", "=" "Node2", "facility" = 5}

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.