Logstash Unable to read Redis data

Today, the erection of logsatsh+redis+elasticsearch problems, after nearly an hour of troubleshooting, finally resolved. Record it.

The environment is such that a client sends data to a redis on the server, and the server-side Logstash reads Redis data and stores it in Elasticsearch.

The preliminary question is this: On the server side, there are no logs sent by the client.

This may have two problems: one is that the client's Logstash did not successfully send the data to Redis, and the server-side Logstash cannot read the data from Redis.

Troubleshoot a problem

To avoid data impact judgment in Redis, first empty the data in Redis :

Go inside the Redis directory

SRC/REDIS-CLI, connecting to Redis, executing FULSHDB, deletes the currently selected database. The database here is actually represented by a key, which executes:

KEYS *

Can see all the key, here the key and our configuration file key is corresponding, select it, delete:

Select $keyflushdb

In the configuration file, add stdout{} to print the log to the screen.

Because the configured input is file, it has a record in the user's home directory where the content is read:. sincedb*. To prevent false positives because no new logs have been caused, delete the file and then restart Logstash.

At this point the screen is found to have log output. Then see if there is a corresponding key in Redis:

KEYS *

If so, it means we've sent the data to Redis. I have it here, so I ruled out the possibility of the first question.

Troubleshooting Question Two

To avoid the elasticsearch of data in the central Plains, first clear the data in the elsaticsearch :

Curl-xdelete ' Http://localhost:9200/_all '

The stdout{} output is then also added to the configuration file.

Start Logstash and find no screen without log output. Now the problem is fixed.

Solve

Since there is data in Redis, why can't es get it? Is it because it's not connected to Redis, or is it the key and the data_type?

Because the Redis and server side of the Logstash is on the same machine, the previously written host is the native IP, look at the official document written 127.0.0.1, then changed to 127.0.0.1.

In order to avoid the client and server-side key and data_type not on, and look at the official documents, look inside the double quotes, dead horse as a live horse medicine. Change the server side to double quotes, and then copy the key and Data_type to the client's configuration file.

Reboot, OK! By the way, attach the configuration file.

Server-side:

Client:

Logstash Unable to read Redis data