Python and Redis Connection tutorial

Today in writing Zabbix storm job monitoring script used Python Redis module, before also useful, but not too much understanding, today looked at the relevant API and source code, see the implementation of ConnectionPool, here simply say.
Before ConnectionPool, if you need to connect Redis, I use Strictredis this class, in the source code can see the specific explanation of this class:

Redis. Strictredis implementation of the Redis protocol. This abstract class provides a Python interface to all Redis commands and an
Implementation of the Redis protocol. Connection and Pipeline derive from this, implementing how the commands is sent and received to the Redis server

The method used:

R=redis. Strictredis (Host=xxxx, port=xxxx, db=xxxx) r.xxxx ()

With the ConnectionPool class, you can use the following methods

Pool = Redis. ConnectionPool (Host=xxx, Port=xxx, db=xxxx) R = Redis. Redis (Connection_pool=pool)

Here Redis is a subclass of Strictredis
The simple analysis is as follows:
In the __init__ method of the Strictredis class, you can initialize the Connection_pool parameter, which corresponds to a ConnectionPool object:

Class Strictredis (object): ...  .. def __init__ (self, host= ' localhost ', port=6379,         db=0, Password=none, Socket_timeout=none,         socket_connect_ Timeout=none,         Socket_keepalive=none, Socket_keepalive_options=none,         Connection_pool=none, unix_socket_ Path=none,         encoding= ' utf-8 ', encoding_errors= ' strict ',         Charset=none, Errors=none,         decode_responses= False, Retry_on_timeout=false,         ssl=false, Ssl_keyfile=none, Ssl_certfile=none,         Ssl_cert_reqs=none, Ssl_ca_ Certs=none):     if not connection_pool:       ...       .. Connection_pool = ConnectionPool (**kwargs)     Self.connection_pool = Connection_pool

The Execute_command method is called when an instance of Strictredis executes a specific command, and you can see that the specific implementation is to get a specific connection from the connection pool and then execute the command to release the connection when it is complete:

  # command execution and PROTOCOL parsing  def execute_command (self, *args, **options):    "execute a COMMAND and retur N A parsed response "    pool = self.connection_pool    command_name = args[0]    connection = pool.get_connection ( Command_name, **options) #调用ConnectionPool. Get_connection method get a connection    try:      Connection.send_command (*args) # Command execution, here for Connection.send_command      return self.parse_response (connection, Command_name, **options)    except ( Connectionerror, Timeouterror) as E:      connection.disconnect ()      if not connection.retry_on_timeout and Isinstance (E, timeouterror):        raise      Connection.send_command (*args)       return Self.parse_response ( Connection, Command_name, **options)    finally:      pool.release (Connection) #调用ConnectionPool. Release released Connection

Look at the ConnectionPool class:

Class ConnectionPool (object): ... def __init__ (self, connection_class=connection, Max_connections=none, **connection_kwargs): #类初始化时调用构造函数 max_connections = max_connections or 2 * * If not isinstance (max_connection s, (int, long)) or Max_connections < 0: #判断输入的max_connections是否合法 raise ValueError (' max_connections ' must be a PO Sitive integer ') self.connection_class = connection_class #设置对应的参数 Self.connection_kwargs = Connection_kwargs sel F.max_connections = Max_connections self.reset () #初始化ConnectionPool reset Operation def reset (self): Self.pid = Os.getpid () self._created_connections = 0 #已经创建的连接的计数器 self._available_connections = [] #声明一个空的数组, used to hold the available connections Self._in_use_ Connections = set () #声明一个空的集合, used to hold the connection Self._check_lock = threading already in use.     Lock () ... def get_connection (self, command_name, *keys, **options): #在连接池中获取连接的方法 "Get a connection from the pool" Self._checkpid () try:connection = Self._available_connectioNs.pop () #获取并删除代表连接的元素, when Connectiong is first fetched, because _available_connections is an empty array, the Make_connection method is called directly except Indexerro R:connection = Self.make_connection () self._in_use_connections.add (connection) #向代表正在使用的连接的集合中添加元素 return con Nection def make_connection (self): #在_available_connections数组为空时获取连接调用的方法 "Create A new connection" if Self._crea Ted_connections >= self.max_connections: #判断创建的连接是否已经达到最大限制, max_connections can be initialized by parameters raise Connectionerror ("Too m Any connections ") Self._created_connections + = 1 #把代表已经创建的连接的数值 +1 return Self.connection_class (**self.connection_kw args) #返回有效的连接, default to Connection (**self.connection_kwargs) def release (self, Connection): #释放连接, the link is not broken, just exists in the link pool "releas Es the connection back to the pool "self._checkpid () if connection.pid! = Self.pid:return Self._in_use_conn Ections.remove (Connection) #从集合中删除元素 Self._available_connections.append (connection) #并添加到_available_connections In the array of Def disconnect (self): #断Open all Connection Pools link "Disconnects all connections in the pool" All_conns = Chain (self._available_connections, self . _in_use_connections) for connection in All_conns:connection.disconnect ()

Execute_command finally calls the Connection.send_command method, closes the link to the Connection.disconnect method, and the Connection class's implementation:

Class Connection (object):  "manages TCP communication to and from a Redis server"  def __del__ (self):  # Object delete operation, call disconnect release connection    try:      self.disconnect ()    except Exception:      Pass

The core link-building method is implemented via the socket module:

 def _connect (self): Err = None for res in Socket.getaddrinfo (self.host, self.port, 0, socket. SOCK_STREAM): Family, Socktype, Proto, canonname, socket_address = res SOCK = None Try:sock = Socke T.socket (family, Socktype, Proto) # Tcp_nodelay sock.setsockopt (socket. IPPROTO_TCP, Socket.          Tcp_nodelay, 1) # tcp_keepalive if self.socket_keepalive: #构造函数中默认 socket_keepalive=false, so here default is short connection Sock.setsockopt (socket. Sol_socket, SOCKET. So_keepalive, 1) for K, V in Iteritems (self.socket_keepalive_options): sock.setsockopt (socket. Sol_tcp, K, v) # Set the socket_connect_timeout before we connect sock.settimeout (Self.socket_connect_timeou T) #构造函数中默认socket_connect_timeout =none, that is, the mode connected to blocking # Connect Sock.connect (socket_address) # set T        He socket_timeout now, we ' re connected sock.settimeout (self.socket_timeout) #构造函数中默认socket_timeout =none return sOck except Socket.error as _: Err = _ If sock is not None:sock.close () ..... 

Ways to close Links:

  def disconnect (self):    ' Disconnects from the Redis server '    self._parser.on_disconnect ()    if Self._sock is None:      return    try:      self._sock.shutdown (socket. SHUT_RDWR) #先shutdown再close      self._sock.close ()    except Socket.error:      pass    self._sock = None

Can summarize the following
1) By default, each Redis instance is created with a ConnectionPool instance, and every access to Redis receives a connection from the connection pool, which is then put back into the connection pool (the connection is not released) after the operation is completed. You can construct a unified connectionpool that can be passed in when a Redis instance is created, and subsequent operations get a connection from a given ConnectionPool and no more connectionpool are created.
2) KeepAlive and timeout are not set by default, and the established connection is a short connection in blocking mode.
3) Regardless of the underlying TCP, connections in the connection pool are destroyed uniformly in the connectionpool.disconnect.

