The Python-redis connection tutorial _python

Today in writing Zabbix storm job monitoring scripts used Python's Redis module, before also useful, but not too much understanding, today looked at the relevant APIs and source code, see the realization of ConnectionPool, here simply.
Before ConnectionPool, if you need to connect Redis, I use Strictredis this class, in the source code can see the specific explanation of this class:

Redis. Strictredis implementation of the Redis protocol. This abstract class provides a Python interface to all Redis commands and
Implementation of the REDIS protocol. Connection and Pipeline derive from-, implementing how the commands are the sent and received to the Redis server

Methods to use:

 R=redis. Strictredis (Host=xxxx, port=xxxx, db=xxxx)
 r.xxxx ()

After you have connectionpool this class, you can use the following methods

Pool = Redis. ConnectionPool (Host=xxx, port=xxx, db=xxxx)
r = Redis. Redis (Connection_pool=pool)

Here Redis is the Strictredis subclass.
The simple analysis is as follows:
In the __init__ method of the Strictredis class, you can initialize the Connection_pool parameter, which corresponds to a ConnectionPool object:

Class Strictredis (object): ...
  def __init__ (self, host= ' localhost ', port=6379, db=0, Password=none, Socket_timeout=none, socket_connect_
         Timeout=none,
         Socket_keepalive=none, Socket_keepalive_options=none,
         Connection_pool=none, unix_socket_ Path=none,
         encoding= ' utf-8 ', encoding_errors= ' strict ', Charset=none, Errors=none
         ,
         decode_responses= False, Retry_on_timeout=false,
         ssl=false, Ssl_keyfile=none, Ssl_certfile=none,
         Ssl_cert_reqs=none, Ssl_ Ca_certs=none):
     if not connection_pool:
       ..... Connection_pool = ConnectionPool (**kwargs)
     Self.connection_pool = Connection_pool

The Execute_command method is invoked when a specific command is executed by an instance of Strictredis, where a specific implementation is obtained by obtaining a specific connection from the connection pool, and then executing the command to release the connection after completion:

  # command EXECUTION and PROTOCOL parsing
  def execute_command (self, *args, **options):
    "execute a command and retur N A parsed response "
    pool = self.connection_pool
    command_name = args[0]
    connection = pool.get_connection ( Command_name, **options) #调用ConnectionPool. Get_connection method gets a connection
    try:
      Connection.send_command (*args) # Command execution, here for Connection.send_command return
      self.parse_response (connection, Command_name, **options)
    except ( Connectionerror, Timeouterror) as E:
      connection.disconnect ()
      if not connection.retry_on_timeout and Isinstance (E, timeouterror):
        raise
      Connection.send_command (*args) return 
      self.parse_response ( Connection, Command_name, **options)
    finally:
      pool.release (Connection) #调用ConnectionPool. Release connection

Look at the ConnectionPool class:

Class ConnectionPool (object): ... def __init__ (self, connection_class=connection, Max_connections=none, **connection_kwargs): #类初始化时调用构造函数 max_connections = max_connections or 2 * * If not isinstance (max_conn Ections, (int, long)) or Max_connections < 0: #判断输入的max_connections是否合法 raise ValueError (' max_connections ' must Be a positive integer ') Self.connection_class = connection_class #设置对应的参数 Self.connection_kwargs = Connection_kwar GS self.max_connections = max_connections self.reset () reset Operation def reset (self) when #初始化ConnectionPool (): Self.pi
    D = os.getpid () self._created_connections = 0 #已经创建的连接的计数器 self._available_connections = [] #声明一个空的数组, used to hold available connections Self._in_use_connections = set () #声明一个空的集合 to hold the connection Self._check_lock = threading that is already in use. Lock () ... def get_connection (self, command_name, *keys, **options): #在连接池中获取连接的方法 "Get a connection from the PO Ol "Self._checkpid () try:connection = SELf._available_connections.pop () #获取并删除代表连接的元素, the first time the Connectiong is fetched, because _available_connections is an empty array, it is called directly make_   Connection method except Indexerror:connection = Self.make_connection () self._in_use_connections.add (connection) #向代表正在使用的连接的集合中添加元素 return Connection def make_connection (self): #在_available_connections数组为空时获取连接调用的方法 "Cre Ate a new connection "if Self._created_connections >= self.max_connections: #判断创建的连接是否已经达到最大限制, Max_connections can pass Parameter initialization raise Connectionerror ("Too many connections") Self._created_connections + 1 #把代表已经创建的连接的数值 +1 return Self.connection_class (**self.connection_kwargs) #返回有效的连接, default to Connection (**self.connection_kwargs) def release (self , connection): #释放连接, the link is not disconnected, only exists in the link pool "releases the connection back to the pool" self._checkpid () if Connectio N.pid!= Self.pid:return self._in_use_connections.remove (connection) #从集合中删除元素 Self._available_connections . Append (Connection) #并添加到_available_cOnnections array of def disconnect (self): #断开所有连接池中的链接 "Disconnects all connections in the pool" All_conns = Chain (s Elf._available_connections, self._in_use_connections) for connection in All_conns:connection.disco

 Nnect ()

Execute_command finally calls the Connection.send_command method, closes the link to the Connection.disconnect method, and the Connection class implementation:

Class Connection (object):
  "manages TCP communication to and from a Redis server"
  def __del__ (self):  # When an object is deleted, invoke the Disconnect release connection
    try:
      self.disconnect ()
    except Exception:
      Pass

The core link building method is implemented through the socket module:

 def _connect (self): Err = None for res in Socket.getaddrinfo (self.host, self.port, 0, socket. SOCK_STREAM): Family, Socktype, Proto, canonname, socket_address = Res sock = None Try:sock = s Ocket.socket (family, Socktype, Proto) # Tcp_nodelay sock.setsockopt (socket. IPPROTO_TCP, Socket.
          Tcp_nodelay, 1) # tcp_keepalive if self.socket_keepalive: #构造函数中默认 socket_keepalive=false, so this defaults to short connection Sock.setsockopt (socket. Sol_socket, SOCKET. So_keepalive, 1 for K, V in Iteritems (self.socket_keepalive_options): sock.setsockopt (socket. Sol_tcp, K, v) # Set the socket_connect_timeout before we connect sock.settimeout (self.socket_connect_time Out) #构造函数中默认socket_connect_timeout =none, that is, the mode connected to blocking # Connect Sock.connect (socket_address) # Set the Socket_timeout now we ' re connected sock.settimeout (self.socket_timeout) #构造函数中默认socket_timeout=none return sock except Socket.error as _: Err = _ If sock are not None:sock.clos
 E () ...

How to close a link:

  def disconnect (self):
    "Disconnects from the Redis server"
    self._parser.on_disconnect ()
    if Self._sock is None:
      return
    try:
      self._sock.shutdown (socket. SHUT_RDWR) #先shutdown再close
      self._sock.close ()
    except Socket.error:
      pass
    self._sock = None

can be summarized as follows
1 A ConnectionPool instance is constructed by default for each Redis instance, each access Redis gets a connection from this connection pool, and the connection is put back into the connection pool after the operation is completed (the connection is not released). You can construct a unified connectionpool that can be passed in when the Redis instance is created, and subsequent operations will be connected from the given ConnectionPool and no more connectionpool will be created.
2 KeepAlive and timeout are not set by default, and the connection established is a short connection to the blocking mode.
3 If the underlying TCP is not considered, the connection in the connection pool will be destroyed uniformly in the connectionpool.disconnect.