Collect php-fpm slow log using Logstash

Currently PHP-FPM services are deployed in Docker, PHP-FPM log and PHP error log can be sent through the Syslog protocol, while PHP-FPM's slow log cannot be configured as a syslog protocol, only output to a file, Because a slow log is composed of multiple lines.

In order to collect slow log, can be collected through Logstash, Flume and other tools, this article uses Logstash to collect the slow log, and writes the collected log to Kafka for easy follow-up processing. Logstash input takes the form of a read file, which is similar to the tail-f principle. In order to be able to use multiple lines of log as a row, the multiline in filter is used to merge multiple rows of logs. The configuration of the Logstash is as follows:

Input {    file {        Path = = ["/var/log/php-fpm/fpm-slow.log"]    }}filter {    Multiline {        pattern = = " ^$ "        Negate" = true what        = "Previous"    }}output {    stdout{codec = rubydebug}    Kafka {        Codec = plain {           format = "Tag|%{host}%{message}"        }        topic_id = "Fpm-slowlog"        Bootstrap _servers = "kafka1.hostname:8082,kafka2.hostname:8082"    }}

