Logstash 1.5.3 Configuration using Redis for continuous transmission

Logstash is a member of the elk,

The Redis plugin is also a handy gadget introduced in the Logstash book.

Before, with a smaller cluster deployment, not involved in Redis middleware, so it is not very clear the configuration inside,

Later used to find the configuration a bit of a pit.

When the first configuration, dead or alive is not connected, always error, said connection refused.

But there is no problem with the Logstash machine redis-cli connection.

Later found that the estimate is a bug, did not use the default port this thing, unexpectedly can't connect.

------------------

No more nonsense, first give me the configuration that can work

------------------

Output log to Redis:log-to-redis.conf

Input {file {path="/opt/tengine_1.5.2/logs/access.log"}}output {redis {host= ["192.168.1.12"] Port= 6379Batch=true Batch_events= 5data_type="List"Key="EST"codec="JSON"} stdout {codec=Rubydebug}}

Read Redis and output to Elasticsearch or anywhere else: log-to-es.conf

Input {redis {host= ["192.168.1.12"] Port= 6379data_type="List"Key="EST"codec="JSON"}}output {stdout {codec=Rubydebug}##open This comment, know what happens#Elasticsearch {#cluster = "Esearch"#  }}

Several pits:

-Different host configuration types for Redis input plugin and output plugin

Output is Array,input is a string that is, when logstash output, you can specify multiple hosts, one cannot connect, can use another to prevent single point of failure

Oh, blame me to see the configuration is not careful, the default input is also an array type, configured with host = [' 1.2.3.3:6380 '], which stabbed the hornet's nest, plugin simply direct shutdown, I thought it was a bug ...

Error also put here, in case who search, also lest tangled.

[[Email protected]]~# logstash-f log-to-es.conf logstash startup Completeda Plugin had an unrecoverable error. Would restart this plugin.  Plugin: <logstash::inputs::redis host=> "192.168.1.1:6380", port=>6379, data_type=> "list", key=> "est" , Codec=><logstash::codecs::json charset=> "UTF-8", Debug=>false, threads=>1, name=> "Default", Db=>0, timeout=>5, batch_count=>1>  Error:initialize:name or service not known {: Level=>:error} Logstash shutdown completed

-Different batch settings and output plugin settings for Redis input plugin

Input:batch_count

Output:batch+batch_events+data_type working with Setup errors can be problematic

Logstash 1.5.3 Configuration using Redis for continuous transmission