Python Redis link Creation Implementation analysis

Today in writing Zabbix storm job monitoring script used Python Redis module, before also useful, but not too much understanding, today looked at the relevant API and source code, see the implementation of ConnectionPool, here simply say.
Before ConnectionPool, if you need to connect Redis, I use Strictredis this class, in the source code can see the specific explanation of this class:

Redis. Strictredis implementation of the Redis protocol. This abstract class provides a Python interface to all Redis commands and a implementation of the Redis protocol. Connection and Pipeline derive from this, implementing how the commands is sent and received to the Redis server

The method used:

R=redis. Strictredis (Host=xxxx, port=xxxx, db=xxxx) r.xxxx ()

With the ConnectionPool class, you can use the following methods

Pool = Redis. ConnectionPool (Host=xxx, Port=xxx, db=xxxx) R = Redis. Redis (Connection_pool=pool)

Here Redis is a subclass of Strictredis
The simple analysis is as follows:
In the __init__ method of the Strictredis class, you can initialize the Connection_pool parameter, which corresponds to a ConnectionPool object:

Class strictredis (object): ...     def __init__ (self, host= ' localhost '),  port=6379,                  db=0, password=None, socket_timeout=None,                  socket_connect_timeout=None,                  socket_keepalive=none,  socket_keepalive_options=None,                  connection_pool=None, unix_socket_path=None,                  encoding= ' Utf-8 ',  encoding_errors = ' Strict ',                  charset=none, errors=nonE,                 decode _responses=false, retry_on_timeout=false,                  ssl=False, ssl_keyfile=None, ssl_certfile=None,                  ssl_cert_reqs= None, ssl_ca_certs=none):         if not  connection_pool:             ..........               connection_pool =  ConnectionPool (**kwargs)          self.connection_pool =  connection_pool

The Execute_command method is called when an instance of Strictredis executes a specific command, and you can see that the specific implementation is to get a specific connection from the connection pool, then execute the command, and then release the connection after completion:

   # command execution and protocol parsing    def  execute_command (self, *args, **options):         " Execute a command and return a parsed response "         pool = self.connection_pool         command_name = args[0]        connection =  Pool.get_connection (command_name, **options)    #调用ConnectionPool. Get_connection method get a connection          try:             connection.send_command (*args)    #命令执行, this is connection.send_command             return self.parse_response (Connection, command_name,  **options)         except  (Connectionerror, timeouterror)  as e:             connection.disconnect ()              if not connection.retry_on_timeout and  Isinstance (E, timeouterror):                 raise             Connection.send_command (*args)                return self.parse_response (connection, command_name, **options)          finally:             Pool.release (Connection)    #调用ConnectionPool. Release free Connection

The

looks at the ConnectionPool class:

     class connectionpool (object):          ...........    def __init__ (self, connection_class=connection,  max_connections=none,                  **connection_kwargs):    #类初始化时调用构造函数          max_connections = max_connections or 2 ** 31         if not isinstance (max_connections,  (int, long))  or max_ connections < 0:   #判断输入的max_connections是否合法              raise valueerror (' max_connections '  must be a positive  integer ')         self.connection_class = connection_ class   #设置对应的参数         self.connection_kwargs = connection_kwargs         self.max_connections = max_connections         self.reset ()    #初始化ConnectionPool   Reset Operation     def  reset (self):         self.pid = os.getpid ()          self._created_connections = 0   #已经创建的连接的计数器          self._available_connections = []   # Declares an empty array to hold the available connections         self._in_use_connections = set ()    #声明一个空的集合, used to store connections that are already in use         self._check_lock =  Threading. Lock () ...     def get_connection (self, command_name, *keys, ** Options):   #在连接池Method of getting a connection in          "get a connection from the  Pool "        self._checkpid ()          try:            connection =  self._available_connections.pop ()    #获取并删除代表连接的元素, when getting connectiong for the first time, because _available_ Connections is an empty array,             will be called directly Make_ Connection Methods         except IndexError:             connection = self.make_connection ()          self._in_use_connections.add (Connection)    # Add elements to the collection that represents the connection being used         return connection        def make_connection (self):  #在_availabGets the method of the connection call when the le_connections array is empty          "create a new  Connection "        if self._created_connections >=  self.max_connections:    #判断创建的连接是否已经达到最大限制, max_connections can be initialized by parameters              raise connectionerror ("Too many connections")         self._created_connections += 1   # +1        return self.connection_class the value representing the connection that was created (* * Self.connection_kwargs)       #返回有效的连接, default to Connection (**self.connection_kwargs)      def release (self, connection):   #释放连接, links are not broken, just exist in the link pool           "Releases the connection back to the pool"           Self._checkpid ()         if connection.pid != self.pid:             return         self._in_use_connections.remove (connection)     #从集合中删除元素          self._available_connections.append (connection)   #并添加到_available_connections   in an array of     def disconnect (self):  #断开所有连接池中的链接           "Disconnects all connections in the pool"          all_conns = chain (self._available_connections,                            self._in_use_connections)         for connection  in all_conns:             connection.disconnect () 

Execute_command finally calls the Connection.send_command method, closes the link to the Connection.disconnect method, and the Connection class's implementation:

Class Connection (object): ' Manages TCP communication to and from a Redis server ' Def __del__ (self): #对象删除时的操作, call di Sconnect Release Connection Try:self.disconnect () except Exception:pass

The core link-building method is implemented via the socket module:

    def _connect (self):         err =  none        for res in socket.getaddrinfo ( self.host, self.port, 0,                                        socket. SOCK_STREAM):             family, socktype,  proto, canonname, socket_address = res             sock = None             try:                 sock = socket.socket (Family, socktype, proto)    &NBsp;            # tcp_nodelay                 sock.setsockopt (socket. Ipproto_tcp, socket. TCP_NODELAY, 1)                  # TCP_KEEPALIVE                 if self.socket_keepalive:    #构造函数中默认  socket_keepalive=false, so here's the default for short connections                      sock.setsockopt (socket. Sol_socket, socket. SO_KEEPALIVE, 1)                      for k, v in iteritems (self.socket_keepalive_options):          &Nbsp;              sock.setsockopt ( Socket. SOL_TCP, K, V)                  # set the socket_connect_timeout before we connect                 sock.settimeout (self.socket_ connect_timeout)    #构造函数中默认socket_connect_timeout =none, which is the mode connected to blocking                  # connect                 sock.connect (socket_address)                  # set the  Socket_timeout now that we ' re connected                  sock.settimeout (self.socket_timeout)    #构造函数中默认socket_timeout =none                 return sock             except socket.error as  _:                err =  _                if  sock is not None:                     sock.close () .....

Ways to close Links:

def disconnect (self): "Disconnects from the Redis server" Self._parser.on_disconnect () if Self._sock is None:return try:self._sock.shutdown (socket.  SHUT_RDWR) #先shutdown再close self._sock.close () except socket.error:pass Self._sock = None

Can summarize the following
1) By default, each Redis instance is created with a ConnectionPool instance, and every access to Redis receives a connection from the connection pool, which is then put back into the connection pool (the connection is not released) after the operation is completed. You can construct a unified connectionpool that can be passed in when a Redis instance is created, and subsequent operations get a connection from a given ConnectionPool and no more connectionpool are created.
2) KeepAlive and timeout are not set by default, and the established connection is a short connection in blocking mode.
3) Regardless of the underlying TCP, connections in the connection pool are destroyed uniformly in the connectionpool.disconnect.

This article is from the "Food and Light Blog" blog, please make sure to keep this source http://caiguangguang.blog.51cto.com/1652935/1583541

Python Redis link Creation Implementation analysis