Distributed micro-service processing of non-idempotent operation under Weak network

This is a creation in Article, where the information may have evolved or changed.

Background

In the case of a browser accessing the server, the network speed is very slow. In order to get results, the user often repeats the click action. This will make the operation result of some non-idempotent operations become very unreliable.

For example, a user making a payment operation is a non-idempotent operation.

Non-idempotent, in simple terms, is that an operation is not repeatable.

Scheme

In the user browser cookie, add Idempotent_token, then use Interceptor interception in each microservices, and use distributed locks for global locking.

Since microservices are distributed, there will be a situation where a load-balancing strategy is in place where the user accesses the warehouse microservices (1), and simultaneously accesses the warehouse microservices (2) and modifies the inventory at the same time. This scenario is reasonable, and they will take the same idempotent_token for the micro-service operation of the warehouse. This time it is necessary to use a distributed lock for lock operation.

Principle and realization

Interception device

/** * Idempotent Interceptor for handling non-idempotent operations. * Idempotent will not be processed, direct release */public class Idempotenttokeninterceptor extends Handlerinterceptoradapter {private static final Logg    ER log = Loggerfactory.getlogger (Idempotenttokeninterceptor.class);    public static final String Idempotent_token = "Idempotent_token";    @Resource private idempotentdb<idempotentro> Defaultidempotentdb;    @Value ("${spring.cloud.consul.host}") Private String consulhost;    @Value ("${spring.cloud.consul.port}") private int consulport; /** * Returns the idempotent error message * * @param response HTTP response * @param message returns an HTTP message * @return true-to continue to False---not continue execution down, will be blocked */private Boolean with (httpservletresponse response, String message) {respons        E.setstatus (HttpStatus.UNAUTHORIZED.value ());        Response.setcontenttype ("Application/json");        Response.setcharacterencoding ("Utf-8"); Cookieutil.addcookie (response, Idempotent_token, Uuid.randomuuid (). toString (). ReplaceAll ("-", ""), 3600 * 2); Try (printwriter writer = Response.getwriter ()) {Writer.append (New Gson (). ToJson (Result.ResultBuilder.errorWit            H (Message). Build ()));        Writer.flush ();            } catch (IOException e) {e.printstacktrace ();            Log.error ("Cannot close response print writer");        E.printstacktrace ();    } return false; } @Override public Boolean prehandle (HttpServletRequest request, httpservletresponse response, Object handler) throw S Exception {return optional.ofnullable (Request.getcookies ()). Map (cookies, stream.of (cookies)                            . filter (X-Idempotent_token.equalsignorecase (X.getname ())). Map (x, {                            String TimeStamp = Uuid.randomuuid (). toString (). ReplaceAll ("-", "");                                If a idempotent token has been found, determine if the value of idempotent is null return optional.ofnullable (X.getvalue ()). Map (V- List<idempotentro> list = Defaultidempotentdb.findbyauthkey (v);                                if (Collectionutils.isempty (list)) List = new arraylist<> (); Find out if the URL already exists with the Idempotent key-value pair, Boolean hasrequested = List.stream (). AnyMatch (IR Request.getmethod (). Equals (Ir.getmethod ()) && Request.getrequesturi (). Equal                                S (Ir.getauthurl ()));  if (hasrequested) {log.error ("already requested with idempotent tokens from the URL of                                    {} by {} method ", Request.getrequesturi (), Request.getmethod ());                                Return with (response, "Repeat the submission"); } else {Defaultidempotentdb.insert (IdempotentRo.IdempotentRoBuilder.build (). Set (V, req   Uest.getrequesturi (), Request.getmethod ()). Create ());                                 Cookieutil.addcookie (response, Idempotent_token, Uuid.randomuuid (). toString (). ReplaceAll (                                    "-", ""), 3600 * 2);                                return true; }}). Orelseget ((), {log.error ("Cannot find value of Idempo                                Tent token from the URL of {} by {} method ", Request.getrequesturi (), Request.getmethod ());                            Return with (response, "Fake the Idempotent token");                        }); }). Reduce ((x, y) x && y). Orelseget ((), {log.error ("Cannot find idempotent t                            Oken from the URL of {} by {} method ", Request.getrequesturi (), Request.getmethod ());                        Return with (response, "idempotent token"); }). Orelseget ((), {log.error ("Cannot find cookies from the URLof {} by {} method ", Request.getrequesturi (), Request.getmethod ());                Return with (response, "Fake the request ...");    }); } @Override public void Posthandle (HttpServletRequest request, httpservletresponse response, Object handler, Modelan    DView Modelandview) throws Exception {Super.posthandle (request, response, Handler, Modelandview); } @Override public void aftercompletion (HttpServletRequest request, httpservletresponse response, Object handler, Ex    Ception ex) throws Exception {super.aftercompletion (request, response, Handler, ex); }}

Distributed Locks

public class Idempotentdistributedlock implements Distributedlock {private final Consul Consul;    Private final Session value;    Private final sessionclient sessionclient;    Private final sessioncreatedresponse session;    public static final String KEY = "Consul_key";    public static final Logger log= Loggerfactory.getlogger (idempotentdistributedlock.class);        Public Idempotentdistributedlock (Consul Consul, String sessionId) {this.consul = Consul;        Gets the digest, as the session ID, and creates the conversation This.value = Immutablesession.builder (). Name (SESSIONID). build ();        This.sessionclient = Consul.sessionclient ();    This.session = sessionclient.createsession (value); } @Override public void Lock () {//To acquire the lock operation, get the thread into the buffer queue keyvalueclient keyvalueclient = Consul        . Keyvalueclient ();        Boolean Hasacquired=keyvalueclient.acquirelock (Key,this.value.getname (). Get (), this.session.getId ()); if (!hasacquired) throw new AlreadylockedexcEption (); } @Override @Deprecated public void lockinterruptibly () throws interruptedexception {throw new Unsupporte    Doperationexception ();    } @Override public Boolean trylock () {return false;    } @Override public Boolean trylock (long time, Timeunit unit) throws Interruptedexception {return false;        } @Override public void Unlock () {Keyvalueclient keyvalueclient = consul.keyvalueclient ();        Keyvalueclient.deletekey (KEY);    Sessionclient.destroysession (Session.getid ());    } @Override @Deprecated public Condition newcondition () {throw new unsupportedoperationexception (); }}

Performance testing

In Chrome's slow 3g, the user accesses the same operation, and the main delay comes from the processing of the business.

Benefits

• Can effectively prevent users from repeatedly clicking
• Distributed lock implements the lock interface of the JVM, the user can use without learning difficulty, and as a distributed lock for resource locking
• In the case of consul as the basic service of consistency, the user can also effectively debug and troubleshoot, directly query all the session list

Limitations

• A user's lock on a single resource will appear sometimes difficult to decide
• The user can only do a one-time operation, for other resources to do the operation, will be directly fused, no longer waiting

The future trend

• Will address the locking of complex multiple resources in distributed locks

Reference

• Distributed Locks with Consul and Golang

• Implementation of distributed lock based on consul