Golang implementation of distributed locks via Consul

This is a creation in Article, where the information may have evolved or changed.

What is Consul?

Consul is a service software that supports distributed, highly available service discovery and configuration sharing in multiple datacenters, developed by Hashicorp in the Go language, and open source based on the Mozilla Public License 2.0 protocol. Consul supports health checks and allows HTTP and DNS protocol call APIs to store key-value pairs.
Command line super easy to use virtual machine management software Vgrant is also a product developed by Hashicorp Company.
The conformance protocol uses the RAFT algorithm to ensure high availability of the service. Members and broadcast messages are managed using the GOSSIP protocol, and ACL access control is supported.

Consul Usage Scenarios

• Registration and configuration sharing for Docker instances
• Registration and configuration sharing for CoreOS instances
• Vitess Cluster
• Configuration sharing for SaaS apps
• Integrates with CONFD services to dynamically generate Nginx and Haproxy configuration files

Advantages of Consul

• Using the Raft algorithm to ensure consistency is more straightforward than a complex Paxos algorithm. In comparison, Zookeeper uses Paxos, while ETCD uses Raft.
• Support multi-data center, internal and external network services using different ports for monitoring. A multi-data center cluster avoids single points of failure in a single data center, and its deployment requires consideration of network latency, fragmentation, and so on. Zookeeper and ETCD do not provide support for multi-datacenter functions.
• Supports health checks. ETCD does not provide this functionality.
• Supports HTTP and DNS protocol interfaces. The integration of the zookeeper is complex and the ETCD only supports the HTTP protocol.
• Official web Management interface, Etcd no such function.

Comprehensive comparison, Consul as a new star of service registration and configuration management, it is worth to pay more attention and research.

The role of Consul

Client: Clients, stateless, forwarding HTTP and DNS interface requests to the server cluster within the LAN.
Server: Service side, save configuration information, high availability cluster, communicate with local clients in LAN, communicate with other data center via WAN. The number of servers per datacenter is recommended to be 3 or 5.

What is a distributed lock

distributed lock is a way to control the synchronous access of shared resources between distributed systems. In distributed systems, it is often necessary to coordinate their actions. If different systems or different hosts of the same system share one or a group of resources, then access to these resources often requires mutual exclusion to prevent mutual interference to ensure consistency, in this case, you need to use a distributed lock.

At present, almost a lot of large-scale Web sites and applications are distributed deployment, data consistency in distributed scenarios has always been a relatively important topic. The distributed Cap theory tells us that "none of the distributed systems can meet both consistency (consistency), availability (availability), and partition fault tolerance (Partition tolerance) at most, at the same time." "So many systems will have to make a choice at the beginning of the design. In the vast majority of the Internet scene, the need to sacrifice strong consistency in exchange for the high availability of the system, often only need to ensure "final consistency", as long as the final time is acceptable to the user within the scope.

In many scenarios, in order to ensure the final consistency of data, we need a lot of technical solutions to support, such as distributed transactions, distributed locks and so on. Sometimes, we need to make sure that a method can only be executed by the same thread at the same time.

Here we use Consul to manage distributed locks. Consul has built-in service registration and discovery Framework, distributed consistency Protocol implementation, health check, Key/value storage, multi-datacenter solutions, no longer need to rely on other tools (such as zookeeper, etc.).

Sessions

A session is a link between a remote process and a consul node, which is made by a remote process and can be explicitly invalid or due to a health check mechanism. Based on the session configuration, create a session lock with an expiration to destroy or release.

Health Checks

Consul supports multiple checks (such as HTTP, TCP, and so on). A list of health checks that can be defined during session creation. These checks will be used to determine if the sessio needs to be invalidated.

Ttl

In addition to health checks, sessions also have built-in support for TTL. When the TTL expires session is considered invalid. The remote process is responsible for updating the session before the TTL expires.

Golang API

The Consul API client provides a convenient abstraction, session and k/v storage. There is a lock structure with locking, unlocking and destroying methods. There are also methods used to help create the lock instance. The API client is also responsible for updating the session.

Creating the Consul Client

1	Client, Err: = API. Newclient (&api. Config{address: "127.0.0.1:8500"})

Creating Lock Instance

1 2 3 4 5 6 7 8 9 10 11 12

Type lockoptions struct { Key string//Must be set and has write permissions Value []byte//Optional, value to associate with the lock Session string//Optional, created if not specified Sessionopts *sessionentry//Optional, options to use when creating a session SessionName string//Optional, defaults to Defaultlocksessionname (ignored if sessionopts is given) Sessionttl string//Optional, defaults to Defaultlocksessionttl (ignored if sessionopts is given) monitorretries int//Optional, defaults to 0 which means no retries Monitorretrytime time. Duration//Optional, defaults to Defaultmonitorretrytime Lockwaittime time. Duration//Optional, defaults to Defaultlockwaittime locktryonce bool//Optional, defaults to false which means try Forever }

Lockoptions is a container for all possible options that can be used to set keys and values, customize sessions, or set TTL.

1 2 3 4 5 6 7 8 9 10

OPTs: = &api. lockoptions{ Key: "WEBHOOK_RECEIVER/1", Value: []byte ("set by sender 1"), Sessionttl: "10s", Sessionopts: &api. sessionentry{ Checks: []string{"Check1", "Check2"}, Behavior: "Release", }, } Lock, Err: = client. Lockopts (opts)

Another common method is Lockkey, which creates a lock with all options except the default entry name.

1	Lock, Err: = client. Lockkey ("WEBHOOK_RECEIVER/1")

Acquiring lock

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21st 22 23 24 25

STOPCH: = Make (chan struct{}) Lockch, err: = lock. Lock (STOPCH) If err! = Nil { Panic (ERR) } Cancelctx, Cancelrequest: = context. Withcancel (context. Background ()) Req, _: = http. Newrequest ("GET", "Https://example.com/webhook", nil) Req = req. Withcontext (CANCELCTX) Go func () { http. Defaultclient.do (req) Select { Case <-cancelctx.done (): Log. PRINTLN ("Request cancelled") Default Log. PRINTLN ("Request Done") Err = lock. Unlock () If err! = Nil { Log. PRINTLN ("Lock already unlocked") } } }() Go func () { <-lockch Cancelrequest () }()