Today elk collect logs, found that the collection failed to find various reasons, and finally found in the Redis log error:
[2489] June 10:43:42 # Error allocating resoures for the client
Error is unable to allocate resources for client to view Redis connections:
# REDIS-CLI Info | grep connected connected_clients:5036connected_slaves:0
The client has reached more than 5,000 connections.
To view client connection conditions:
redis 127.0.0.1:6379> client listaddr=10.247.64.115:52834 fd=5 idle=2144 Flags=n db=0 sub=0 psub=0 qbuf=0 obl=0 oll=0 events=r cmd=rpushaddr= 127.0.0.1:38484 fd=7 idle=0 flags=n db=0 sub=0 psub=0 qbuf=0 obl=0 oll=0 events=r cmd=blpopaddr=10.105.1.2:53125 fd=8 idle=2144 flags=n db=0 sub=0 psub=0 qbuf=0 obl=0 oll=0 events=r cmd=rpushaddr=10.105.1.3:32639 fd=9 idle=2144 flags=n db=0 sub=0 psub=0 qbuf=0 obl=0 oll=0 events=r cmd=rpushaddr=10.105.1.4:57134 fd=10 idle=2144 flags=n db=0 sub=0 psub=0 qbuf=0 obl=0 oll=0 events=r cmd=rpushaddr=10.105.1.5:53126 fd=11 idle=2144 flags=n db=0 sub=0 psub=0 qbuf=0 obl=0 oll=0 events=r cmd=rpushaddr=10.105.1.7:8887 fd=12 idle=2144 flags=n db=0 sub=0 psub=0 qbuf=0 obl=0 oll=0 events=r cmd=rpush
Each field has the following meanings:
Addr: Client address and Port FD: The file descriptor used by the socket idle: idle time in seconds Flags: Client FLAGDB: The database that the client is using Idsub: Number of subscribed channels Psub: Number of subscribed modes multi: The number of commands executed in the transaction qbuf: Length of the query buffer (in bytes, 0 for no query buffer allocated) Qbuf-free: Length of the remaining space in the query buffer (in bytes, 0 for no space left) OBL: The length of the output buffer in bytes, 0 Indicates no output buffer allocated) Oll: The output list contains the number of objects (when the output buffer has no space left, the command reply is queued to the team as a string object joins) Omem: Total memory consumed by output buffers and output lists events: File descriptor event cmd: Last Command executed
It is found that the idle idle time of the client is too long, the connection pool maintains too many connections, and the unused connection needs to be released in time:
Redis 127.0.0.1:6379> CONFIG SET Timeout 30OK
To view the connection again:
# REDIS-CLI Info | grep connectedconnected_clients:1137connected_slaves:0
Number of connections dropped
View the specifics of the connection
Redis 127.0.0.1:6379> client listaddr=127.0.0.1:38484 fd=7 idle=0 flags=n db=0 sub=0 psub=0 qbuf=0 obl=0 oll=0 events=r cmd=blpopaddr=10.105.1.150:39742 fd=269 idle=29 flags=n db=0 sub=0 psub=0 qbuf=0 obl=0 oll=0 events=r cmd=rpushaddr=10.10 5.1.226:53291 fd=5 idle=31 flags=n db=0 sub=0 psub=0 qbuf=0 obl=0 oll=0 events=r Cmd=rpush
The idle time has been limited to 30 seconds.
This article is from the "take a deep Breath again" blog, make sure to keep this source http://ckl893.blog.51cto.com/8827818/1785422
Redis high concurrency causes no connection processing