[Logstash-input-file] Plug-in use detailed

The previous chapter introduced the use of Logstash, this article continued in-depth, introduced the most commonly used input plug-in--file.

This plug-in can be read from the specified directory or file, input to the pipeline processing, is also the core of Logstash plug-in, most of the use of the scene will be used in this plug-in, so here in detail the meaning of each parameter and use.

Minimized configuration file

In Logstash, you can add a file configuration to the input{} , and the default minimum configuration is as follows:

Input {    file {        = "e:/software/logstash-1.5.4/logstash-1.5.4/data/*"      }}filter {    }output {    stdout {}}

Of course, you can also listen to multiple target files:

Input {    file {        = = ["e:/software/logstash-1.5.4/logstash-1.5.4/data/*", "f:/ Test.txt "]    }}filter {    }output {    stdout {}}

　　the path name of the file needs to be the absolute path, and globs notation is supported .

Other configurations

In addition, file also provides a number of other properties to handle the necessary entries for path:

Input {    file {        #监听文件的路径          = = ["e:/software/logstash-1.5.4/logstash-1.5.4/data/ * "," f:/test.txt "]        #排除不想监听的文件        exclude =" 1.log "                #添加自定义的字段        Add_field + = {" Test "+" Test "}        #增加标签        tags = "tag1"        #设置新事件的标志        delimiter "\ n"        #设置多长时间扫描目录, discover new files        Discover_ Interval =        #设置多长时间检测文件是否修改        stat_interval = 1         #监听文件的起始位置, the default is end        start_position = Beginning        #监听文件读取信息记录的位置        sincedb_path = "E:/software/logstash-1.5.4/logstash-1.5.4/test.txt"        #设置多长时间会写入读取的位置信息        sincedb_write_interval =            }filter {    }output {    stdout {}}

　　Notable among these are:

1 path

is a required option, each file configuration has at least one path

2 Exclude

is a file that you do not want to listen to, Logstash automatically ignores the file's listener. The configured rule is similar to path, supports strings or arrays, but requires absolute paths.

　　3 start_position

is the location of the listener, the default is end, that is, if a file does not record its read information, read from the end of the file, that is, just read the newly added content. For some newer log types, it is usually possible to use end directly, instead, the beginning will start reading from the head of a file. However, if you record the read information of the file, this configuration will be out of effect.

　　4 Sincedb_path

This option configures the default read file information recorded in which file, by default, according to the file Inode and other information generated automatically. The inode, the main device number, the secondary device number, and the location of the read are recorded. Therefore, if a file is simply renamed, then its inode and other information will not change, so it will not re-read any information about the file. Similarly, if you copy a file, it is equivalent to creating a new inode, and if you are listening to a directory, you will be reading all the information for that file.

5 other about scanning and testing time, according to the default, if you frequently create new files, want to quickly listen, you can consider shortening the detection time.

　　6 Add_field

is to add a field, for example:

file {     = = {"Test" = "Test"        }= "D:/tools/logstash/path/to/groksample.log"          = = Beginning    }

　　7 tags

Used to add some tags, this label may play a role in the subsequent processing of the logo

　　8 delimiter

Is the flag of the event branch, if configured as 123, it will look like the following. This option is often useful in multi-line events.

For the time being about file research so much, follow-up will learn the source code, do more sharing.

Reference

"1" logstash Official document: Https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html#plugins-inputs-file-sincedb_path

[Logstash-input-file] Plug-in use detailed