Several realization ways of distributed lock

Distributed locks are a way to control the simultaneous access of shared resources between distributed systems.
The typical usage scenarios are:
Different systems or different hosts of the same system share one or a group of resources, then access to these resources, through a certain mutually exclusive means to prevent each other's interference, to ensure consistency.
1. Implementing distributed locks using Redis

• WATCH, MULTI, EXEC, discard transaction mechanism implement distributed lock
  Redis supports basic transactional operations

MULTI
Some Redis command
Exec

These Redis commands, which are wrapped by multi and exec, guarantee that commands within all transactions will be executed serially, guaranteeing that they will not be interrupted by other clients during the execution of the transaction.
The watch command is able to monitor a key, and when the transaction executes, if the monitored key is modified by another client, the transaction fails and returns a corresponding error (the value is modified within the transaction by the transaction's running client and does not cause the transaction to fail).

Using the above features supported by the Redis office, a distributed lock function can be implemented. The Python code is as follows:

-- Coding:utf-8--

DEF acqure_lock_with_watch (Conn, Lockname, acquire_timeout=10):
Pipe = Conn.pipeline ()
End = Time.time () + acquire_timeout
Lockname = ' Lock: ' + lockname

while time.time() < end:    try:        pipe.watch(lockname)        pipe.multi()  # 开启事务        # 事务具体内容，对lockname的值进行更新        pipe.execute()        return True    except redis.exceptions.WatchError:        # 事务运行期间，有其他客户端改变了lockname的值，抛出异常，进行重试操作        passreturn False

A key is monitored through the Watch command, and the transaction executes successfully when the key is not modified by another client. When the transaction runs, the value is found to be updated by another client, and the task fails with a retry.

* SETNX implementation of distributed locks
SETNX: Adds a key-value pair to Redis when the specified key does not exist. The Redis client guarantees that only one client can set a successful atomicity for the unified key name when multiple clients set its value at the same time.

-- Coding:utf-8--

Def acquire_lock_with_timeout (
Conn, Lockname, acquire_timeout=10, lock_timeout=10):
Identifire = str (UUID.UUID4 ())
Lockname = ' Lock: ' + lockname
lock_timeout = Int (Math.ceil (lock_timeout))

end = time.time() + acquire_timeoutwhile time.time() < end:    if conn.setnx(lockname, identifire):  # 以锁名称为键，uuid的值为值，redis服务器setnx保证了只能有一个客户端成功设置键的原子性        conn.expire(lockname, lock_timeout)  # 设置键的过期时间，过期自动剔除，释放锁        return identifire    elif not conn.ttl(lockname):  # 当锁未被设置过期时间时，重新设置其过期时间        conn.expire(lockname, lock_timeout)    time.sleep(0.001)return False

Above, using the atomic characteristics of setnx, and the key expiration feature of Redis, the distributed lock of automatic expiration release is realized.

* release of the lock

-- Coding:utf-8--

DEF release_lock (Conn, Lockname, Identifire):
Pipe = Conn.pipeline (True)
Lockname = ' Lock: ' + lockname

while True:    try:        pipe.watch(lockname)  # 监视锁的键，在锁释放过程中改变了键的值时得到相应通知        if pipe.get(lockname) == identifire:  # 检查客户端是否仍然持有该锁            pipe.multi()            pipe.delete(lockname)  # 删除键，释放锁            pipe.execute()            return True        pipe.unwatch()        break    except redis.exceptions.WatchError:        pass  # 释放锁期间，有其他客户端改变了键值对，锁释放失败，进行循环return False

The lock release process, first checks whether the client still holds the lock, and if so, deletes the key-value pair in the transaction, releasing ownership of the lock.

2. Implementing distributed locks using memcached
The memcached add command, which is added when the specified key does not exist, and guarantees the atomicity of the execution. With this feature, a distributed lock implementation can be implemented.

def acquire_lock_withmemcached_timeout (* *
Conn, Lockname, acquire_timeout=10, lock_timeout=10):
Identifire = str (UUID.UUID4 ())
Lockname = ' Lock: ' + lockname
lock_timeout = Int (Math.ceil (lock_timeout))

end = time.time() + acquire_timeoutwhile time.time() < end:    # 过期时间保证了客户端崩溃时仍能在超过过期世家后正常释放锁    # 以锁名称为键，uuid的值为值，memcached服务器add保证了只能有一个客户端成功设置键的原子性    if conn.add(lockname, identifire, lock_timeout):        return identifire    time.sleep(0.001)return Fal**se

When the lock is released, delete the key of the specified key.

3. Implementing distributed locks using zookeeper
Compared to locks implemented using Redis and memcached, ZK leverages its advanced features to enable more complex lock characteristics.

Exclusive lock
Lock, also known as write lock or exclusive lock. If the transaction T1 an exclusive lock on the object O1, only the transaction T1 is allowed to read and update the O1 during the entire lock, and no other transaction can read and write to the data object until T1 releases the exclusive lock.
Distributed locks that are implemented using Redis and memcached are all classified as exclusive locks.

Get lock
Zookeeper implementation of distributed locks utilizes the following features of its temporary child nodes:
Creating a temporary child node under the/exclusive_lock node/EXCLUSIVE_LOCK/LOCK,ZK guarantees that in all clients, only one client will be able to create a success, that is, the client can be considered to have acquired a lock. At the same time, although there is no acquisition of the resulting client needs to register a child node on the/exclusive_lock node change watcher monitoring, in order to monitor the change of the lock node in real-time. **

Release lock
Because a temporary node/exclusive_lock/lock is created when a lock is acquired, it is possible to release the lock in the following cases:

The machine that acquires the lock goes down, and the temporary node is removed by ZK
Client actively deletes temporary nodes after normal execution of business logic
In any case, the lock node is removed and ZK notifies all watcher listeners who have registered a child node change on the/exclusive_lock node. After receiving the notification, these clients re-initiate the acquisition process of the distributed lock.

Shared locks
A shared lock is also called a read lock. If the transaction T1 O1 a shared lock on the data object, the current transaction can read only O1, and the other transaction can only add a shared lock to the data object until all shared locks on the data object are freed.
The update operation can only be performed if no transaction is currently read and written.

Get lock
When the need to acquire a shared lock is, all clients create a temporary order node under the/shared_lock node,/shared_lock/[hostname]-the request type (W | R)-sequence number, which is a shared lock that is compiled by the node.
If it is a read request, create the/shared_lock/192.168.0.1-r-000000000001;
If it is a write request, create the/shared_lock/192.168.0.1-w-000000000001;

Judging reading and writing order

After the listener is created, gets all the child nodes under the/shared_lock node, and watcher listens for the node to register the child node changes
Determine the order of your own node ordinal in all child nodes
For read requests:

If there is no child node that is smaller than its ordinal number, or if all child nodes that are smaller than their ordinal number are read requests, then it indicates that they have successfully acquired the shared lock and executed the read logic.
If there is a write request in a child node that is smaller than its ordinal number, enter the wait.

For write requests:
If you are not the sub-node with the smallest number, enter the wait.

After receiving the watcher notification, repeat steps 1-4
Release lock
Release the lock and delete the corresponding data node

Several realization ways of distributed lock