Today in writing Zabbix storm job monitoring script used Python Redis module, before also useful, but not too much understanding, today looked at the relevant API and source code, see the implementation of ConnectionPool, here simply say.
Before ConnectionPool, if you need to connect Redis, I use Strictredis this class, in the source code can see the specific explanation of this class:
Redis. Strictredis implementation of the Redis protocol. This abstract class provides a Python interface to all Redis commands and a implementation of the Redis protocol. Connection and Pipeline derive from this, implementing how the commands is sent and received to the Redis server
The method used:
R=redis. Strictredis (Host=xxxx, port=xxxx, db=xxxx) r.xxxx ()
With the ConnectionPool class, you can use the following methods
Pool = Redis. ConnectionPool (Host=xxx, Port=xxx, db=xxxx) R = Redis. Redis (Connection_pool=pool)
Here Redis is a subclass of Strictredis
The simple analysis is as follows:
In the __init__ method of the Strictredis class, you can initialize the Connection_pool parameter, which corresponds to a ConnectionPool object:
Class strictredis (object): ... def __init__ (self, host= ' localhost '), port=6379, db=0, password=None, socket_timeout=None, socket_connect_timeout=None, socket_keepalive=none, socket_keepalive_options=None, connection_pool=None, unix_socket_path=None, encoding= ' Utf-8 ', encoding_errors = ' Strict ', charset=none, errors=nonE, decode _responses=false, retry_on_timeout=false, ssl=False, ssl_keyfile=None, ssl_certfile=None, ssl_cert_reqs= None, ssl_ca_certs=none): if not connection_pool: .......... connection_pool = ConnectionPool (**kwargs) self.connection_pool = connection_pool
The Execute_command method is called when an instance of Strictredis executes a specific command, and you can see that the specific implementation is to get a specific connection from the connection pool, then execute the command, and then release the connection after completion:
# command execution and protocol parsing def execute_command (self, *args, **options): " Execute a command and return a parsed response " pool = self.connection_pool command_name = args[0] connection = Pool.get_connection (command_name, **options) #调用ConnectionPool. Get_connection method get a connection try: connection.send_command (*args) #命令执行, this is connection.send_command return self.parse_response (Connection, command_name, **options) except (Connectionerror, timeouterror) as e: connection.disconnect () if not connection.retry_on_timeout and Isinstance (E, timeouterror): raise Connection.send_command (*args) return self.parse_response (connection, command_name, **options) finally: Pool.release (Connection) #调用ConnectionPool. Release free Connection
The
looks at the ConnectionPool class:
class connectionpool (object): ........... def __init__ (self, connection_class=connection, max_connections=none, **connection_kwargs): #类初始化时调用构造函数 max_connections = max_connections or 2 ** 31 if not isinstance (max_connections, (int, long)) or max_ connections < 0: #判断输入的max_connections是否合法 raise valueerror (' max_connections ' must be a positive integer ') self.connection_class = connection_ class #设置对应的参数 self.connection_kwargs = connection_kwargs self.max_connections = max_connections self.reset () #初始化ConnectionPool Reset Operation def reset (self): self.pid = os.getpid () self._created_connections = 0 #已经创建的连接的计数器 self._available_connections = [] # Declares an empty array to hold the available connections self._in_use_connections = set () #声明一个空的集合, used to store connections that are already in use self._check_lock = Threading. Lock () ... def get_connection (self, command_name, *keys, ** Options): #在连接池Method of getting a connection in "get a connection from the Pool " self._checkpid () try: connection = self._available_connections.pop () #获取并删除代表连接的元素, when getting connectiong for the first time, because _available_ Connections is an empty array, will be called directly Make_ Connection Methods except IndexError: connection = self.make_connection () self._in_use_connections.add (Connection) # Add elements to the collection that represents the connection being used return connection def make_connection (self): #在_availabGets the method of the connection call when the le_connections array is empty "create a new Connection " if self._created_connections >= self.max_connections: #判断创建的连接是否已经达到最大限制, max_connections can be initialized by parameters raise connectionerror ("Too many connections") self._created_connections += 1 # +1 return self.connection_class the value representing the connection that was created (* * Self.connection_kwargs) #返回有效的连接, default to Connection (**self.connection_kwargs) def release (self, connection): #释放连接, links are not broken, just exist in the link pool "Releases the connection back to the pool" Self._checkpid () if connection.pid != self.pid: return self._in_use_connections.remove (connection) #从集合中删除元素 self._available_connections.append (connection) #并添加到_available_connections in an array of def disconnect (self): #断开所有连接池中的链接 "Disconnects all connections in the pool" all_conns = chain (self._available_connections, self._in_use_connections) for connection in all_conns: connection.disconnect ()
Execute_command finally calls the Connection.send_command method, closes the link to the Connection.disconnect method, and the Connection class's implementation:
Class Connection (object): ' Manages TCP communication to and from a Redis server ' Def __del__ (self): #对象删除时的操作, call di Sconnect Release Connection Try:self.disconnect () except Exception:pass
The core link-building method is implemented via the socket module:
def _connect (self): err = none for res in socket.getaddrinfo ( self.host, self.port, 0, socket. SOCK_STREAM): family, socktype, proto, canonname, socket_address = res sock = None try: sock = socket.socket (Family, socktype, proto) &NBSP;&NBSP;&NBSP;&NBsp; # tcp_nodelay sock.setsockopt (socket. Ipproto_tcp, socket. TCP_NODELAY,&NBSP;1) # TCP_KEEPALIVE if self.socket_keepalive: #构造函数中默认 socket_keepalive=false, so here's the default for short connections sock.setsockopt (socket. Sol_socket, socket. SO_KEEPALIVE,&NBSP;1) for k, v in iteritems (self.socket_keepalive_options): &Nbsp; sock.setsockopt ( Socket. SOL_TCP,&NBSP;K,&NBSP;V) # set the socket_connect_timeout before we connect sock.settimeout (self.socket_ connect_timeout) #构造函数中默认socket_connect_timeout =none, which is the mode connected to blocking # connect sock.connect (socket_address) # set the Socket_timeout now that we ' re connected sock.settimeout (self.socket_timeout) #构造函数中默认socket_timeout =none return sock except socket.error as _: err = _ if sock is not None: sock.close () .....
Ways to close Links:
def disconnect (self): "Disconnects from the Redis server" Self._parser.on_disconnect () if Self._sock is None:return try:self._sock.shutdown (socket. SHUT_RDWR) #先shutdown再close self._sock.close () except socket.error:pass Self._sock = None
Can summarize the following
1) By default, each Redis instance is created with a ConnectionPool instance, and every access to Redis receives a connection from the connection pool, which is then put back into the connection pool (the connection is not released) after the operation is completed. You can construct a unified connectionpool that can be passed in when a Redis instance is created, and subsequent operations get a connection from a given ConnectionPool and no more connectionpool are created.
2) KeepAlive and timeout are not set by default, and the established connection is a short connection in blocking mode.
3) Regardless of the underlying TCP, connections in the connection pool are destroyed uniformly in the connectionpool.disconnect.
This article is from the "Food and Light Blog" blog, please make sure to keep this source http://caiguangguang.blog.51cto.com/1652935/1583541
Python Redis link Creation Implementation analysis