Taobao next single high concurrency solution

The weekend attended the @ Taobao Technology Carnival sponsored technology salon, feeling a lot of harvest, thank you very much for sharing Taobao. Here I share the personal understanding of Taobao's single high concurrency solution. I am not Taobao technician, this article just write their own understanding, so there must be some access.

In the session of the animal husbandry for us to introduce a single part of Taobao's technical program changes, I do not introduce changes, but only the existing system to do the introduction.

To optimize the order, and improve the order of TPS (Transaction per second), the first thing we need to do is to split the logic of the next order, leaving only the core part, and removing the additional functionality. For example, to consider the inventory in order to consider texting, want to give the seller to send a message, want to do statistics, sales statistics and so on, these features are necessary, but also additional functions, to maximize the next single step of the TPS, we must first not consider these things.

The order will inevitably involve buyers to view orders, and sellers to view the orders received, modify the order price, etc., which is the core of the list. In the order of this operation there are two close links between buyers and sellers and have different perspectives. The animal husbandry is called two different dimensions. According to the shepherd's introduction, this step has only 5 tables, and these 5 tables cover the operations of these two dimensions.

In a single database transaction, the most effective way to increase the number of transaction concurrency for a database is to split, split in two ways, split the library, and split the table in the same library. The first thing you need to do is split the fields that you want to break. Taobao is based on the order number to do the split, and the list has two dimensions, buyers and sellers, the order to do after the split, must still be through the buyer, the seller convenient query two dimensions of data. How to do it. Here to leave a doubt, I first introduced the size of Taobao split, Taobao order table split into 16 MySQL library, and in each library and the order table split into 64, the equivalent of a table split into 1024. After splitting, the transaction is dispersed into 1024 sets of tables. This is bound to be a large program to increase concurrent transaction processing capacity (here I say is inevitable, but Taobao before using this solution is to undergo a stress test, the actual test of this scheme of TPS after the gradual adoption of this scheme). The above left a question, after the split, how to ensure that buyers quickly check the order of the seller. The best way is to ensure that buyers, sellers under the order in a table, how to guarantee it. Taobao's approach is to take the buyer's ID after the model placed in the order number. Suppose an order number is 142424594267664; which table on which server the order number corresponds to, which is based on the four-digit 7667 of the order, after the 1024 modulo, and 7667 is the latter four of the buyer's ID. In this way, buyers can inquire about their order by their ID to get their order in the library and table, it can be convenient and efficient inquiries into the buyer's orders. Here is another question, what to do when the seller inquires the order. We have already mentioned that sellers and buyers are divided into two different dimensions to do table design, the seller query is not directly look up the order form, but through the seller dimension of the table to do the query. The table of the seller dimension is inserted, and the update is notified by sending a message when the order is inserted. Also for text messaging, hair flourishing is also handled through the message, these additional features do not participate in the order of the transaction.

Even if you do the library, the table split, there will still be problems. Taobao on the double 11 o'clock in the day's trading volume reached more than 50 million, so a few months past, these split the data in the table will also reach a large amount of processing speed will decline. Taobao's approach is to migrate the old data three months ago to other libraries, thus avoiding the problem of reduced system response time caused by increased data volume. But there is another problem, the user needs to search the order in two libraries, one is the historical data table, the other is the recent data table; This problem is unavoidable, is through the query two times to solve.

Perhaps some friends will think that after splitting the full data to do statistics will have problems. If you do statistics on a split table, there's definitely a problem. How to do it. In fact, it is very simple to migrate data to other libraries to do statistics.

Table splitting can greatly improve the TPS, but will also bring some problems, the need for reliable notification mechanism to notify other modules to do non-core processing, the need for efficient search system to ensure the timely search data updates.

Above is my personal to Taobao under the single high concurrent design understanding. This is superficial, the actual time must also consider more issues, such as database tuning, disk IO mode, server stability, the testability of the scheme, quantifiable and so on.

The technology of the Saturday also shares a lot of other highlights. Thanks to the organizer, moderator. Look forward to the @ Taobao technology Carnival more wonderful technical salon.

Order Number Introduction Errata:

There is a problem with the statement of the order number, for 16 servers, each server 64 tables only need 2 buyers or sellers ID of the latter two digits can be accurately positioned to specific libraries and tables. The last two digits of the buyer ID and the last two digits of the seller ID are also present in the order number. In the order number of the penultimate 3rd, 4 digits and the last two digits.

Assuming the buyer ID is 123456789, then the last two digits in the order number is 89, through the 89 to 16 modulo can be positioned to the specific library, through the 64 modulo can be positioned to the specific table.