Practice and summary of anti-duplication request processing

# #背景
In business development, we often face the problem of preventing duplicate requests. When a server responds to a request that involves a modification of the data, or a change in state, it can be very damaging. The consequences of repeated requests are particularly serious in the trading system, after-sales activism, and in payment systems.

The jitter of the front-end operation, the fast operation, the network communication, or the slow response of the backend will increase the probability of the back-end repetition processing.

Front-end operation to shake and anti-quick action measures, we will first think of a layer of control at the front. The current side triggers the operation, or pops up the confirmation interface, or disable the entry and countdown, etc., here is not a detailed table.

But the front-end limit can only solve a small number of problems, and not thorough enough, the back end of its own anti-duplication measures necessary, bounden duty.

In the implementation of the interface, we often require the interface to meet the idempotent , to ensure that multiple repeated requests are only valid once.

The interface of query class is almost always idempotent, but it is difficult to achieve idempotent when it includes data inserting and updating multi-module data, especially the power requirement of high concurrency. such as third-party payment front-end callback and background callback, third-party payment batch callback, the chronic business logic (such as the user to submit a refund request, the Merchant agrees to return/refund, etc.) or slow network environment, is a high-fat scene repeated processing.

# #尝试

Here is an example of a "user submitting a refund request" that illustrates the effect of the tried anti-duplication approach.

Back-end anti-duplication approach, we have tried three kinds:

# # # #1) verification based on the status of the refund order in DB

This is simple and intuitive, and the refund details (including status) that are queried from DB can often be used in subsequent logic, with no additional work devoted to the issue of duplicate requests.

The logic of validating this query state, which is always present in all state-owned business logic processing, is necessary since the code is online. However, it is not good for anti-duplication: an average of 25 pens per week before adding anti-duplication submissions on the front end, and 7 strokes per week after the front-end optimization. This amount accounts for 3% of the total refund application, a percentage that is still unacceptable.

In theory, the repetitive processing of business logic occurs whenever a request completes a query operation before the data state is updated. As shown in. The direction of optimization is to reduce the business processing time between queries to updates, reducing the concurrency impact of the gap period. In extreme cases, if the query and update become atomic operations , there is no current problem.

# # # #2) validation based on state of cached data
Redis storage queries are lightweight and fast. When the request comes in, it can be recorded in the cache first. The subsequent request is validated each time it is entered. The entire process is processed and the cache is cleared. Take a refund as an example:

I. For each refund initiation request, read the cache for the value of OrderID as key II.    No, if you write a value of OrderID as key in the cache, there is a refund of the order being made. Iv. Setting the lifecycle when the cache is cleared or when the cache value is stored

Compared with 1, the database is replaced with a faster response cache. But it's still not an atomic operation. There is still a time interval between inserting and reading the cache. In the extreme case, there is still a case of repetitive operation.
After this method is optimized, 1 repetitions per week are done.

# # # #3) validation using a unique indexing mechanism

An atomic operation is required to think of a unique index of the database.
Create a new Tradelock table:

CREATE TABLE ' Tradelock ' (' id ' int (one) unsigned not null auto_increment, ' type ' int (one) ' NOT null COMMENT ' lock type ', ' lockid ' int (one) not null default ' 0 ' COMMENT ' Business id ', ' status ' int (one) not null default ' 0 ' COMMENT ' lock state ', PRIMARY KEY (' id ')) engine=in Nodb auto_increment=1 DEFAULT Charset=utf8 comment= ' trade lock mechanism ';

Insert data into the table each time the request comes in:

--Successful, you can continue the operation (equivalent to acquiring a lock);--failure, which means there is an operation in progress.

After the operation is complete, delete this record. (equivalent to release lock)
is now online, waiting for next week's statistics.

# # # #4) Cache-based counter validation:

Since database operations compare consumption performance, it is also an atomic operation to know the counters for Redis. Use counter decisively. Improves performance, does not store, and increases the peak of QPS.

Or an example of an order refund:

Each time the request comes in, a new counter with OrderID key is created, and then +1.

If >1 (cannot obtain a lock): Indicates that there is an operation in progress, delete. If =1 (Get Lock): can be manipulated.

End of operation (delete lock): Delete this counter.
To understand the counters, you can refer to:

Link

# #总结:

The PHP language itself does not provide a process mutex and locking mechanism. That's why we've tried.

There is also a file lock mechanism on the web, but it is recommended to use caching for our distributed deployment.

In the case of large concurrency, the various situations of the program occur. In particular, the amount of money to operate, can not have a cent gap. So in the case of large concurrency to be mutually exclusive, you can consider 3, 42 kinds of scenarios.

Practice and summary of anti-duplication request processing