2017 double 11 technology secret - Alibaba database technology architecture evolution

Every year, the e-commerce double 11 singles day promotion is a big test for Alibaba technicians, especially for the Alibaba database team. After 9 years of development, the double 11 singles day transaction volume has climbed from 0.5 billion in 2009 to 168.2 billion in 2017, and the peak transaction has reached a peak of 325,000 strokes per second. Behind all these business indicators is the iterative upgrade of the underlying technology system.

Alibaba's database system has experienced more than 10 years of development, and this year officially determined the goal of evolving from the third generation of large-scale sub-library tables to the fourth-generation X-DB distributed database system. The landing of the X-DB distributed database has been verified in the 2017 Double 11 singles day promotion, and the bottom layer has begun to introduce the storage computing separation architecture. Distributed has a qualitative improvement in system stability, disaster tolerance, capacity expansion, and technical system cohesion. This year, Double 11 opened a new chapter in the Alibaba database technology architecture.

This article takes the core system inventory center in the Alibaba e-commerce transaction link as an example to get a glimpse of the development history of the Alibaba Group data database. Inventory center database cluster (referred to as inventory DB cluster), after independent split in 2012, its development can be summarized into the following three stages:

2012~2013: Splitting the sub-library table horizontally to build a large-scale database cluster

2014~2016: Unitized remote and active architecture, data synchronization between multiple units

2017: X-DB 1.0 distributed cluster deployment goes online, a new starting point

As the core system in the Alibaba database system, the development history of the inventory DB cluster can be used as a microcosm, representing the evolution of the Alibaba database system.

Birth

The inventory DB cluster was born in 2012 and is the product of vertical business splitting. The earliest inventory is a field of the commodity center database. With the complexity of Taobao business, a single field can not meet the inventory management system based on back-end storage, so that there is a vertically split inventory DB cluster.

Horizontal split

From 2012 to 2017, the double 11 singles day transaction volume has been refreshed step by step, and the QPS/TPS of the inventory DB cluster has also achieved several times of growth. The basic idea of horizontal splitting is to extend the database to multiple physical nodes, allowing each node to handle different read and write requests, thereby alleviating the performance of a single database.

With the help of the database team's DTS (Data Transmission Service) products, the inventory center carried out large-scale horizontal splitting, and the number of sub-libraries and sub-tables expanded to several hundred times, which smoothly supported the rapid development of this stage of business. At the same time, in the hot commodity reduction, anti-oversold data strong demand, cross-city remote disaster data quality problems, business data volume expansion, ultra-large-scale database cluster operation and maintenance, etc., a new generation of architecture is urgently needed. solve.

Multiple living units

In 2014~2015, in order to further enhance the user's shopping experience, the inventory DB cluster and the main station transaction link were unitized. Unitization largely solves the problem of the buyer's experience, and the reading and writing operations are closed in this unit. But for the data of the seller dimension, such as editing the goods, deducting the inventory, it will involve writing across the unit center. There have been many introductions about the unitized architecture.

With multiple units, the biggest challenge for the underlying database is data synchronization, because for the unit-closed buyer dimension data, all the unit data needs to be synchronized to the center; for the read-write separation type of business, we To synchronize the data of the center to the unit. This data channel is completed by DRC (Data Replication Center).

Nowadays, DRC not only becomes the infrastructure of the unitized link of the group, but the corresponding cloud product DTS has already started to make the Alibaba Cloud users and the Jushi Tower merchants build low-cost disasters from 2016. The biggest challenge that the unitized architecture brings to the inventory DB cluster is the consistency of data between multiple units. We have also made great efforts to this end.

X-DB distributed cluster

In 2017, the double DB, the inventory DB cluster used X-DB 1.0 distributed cluster deployment for the first time, and smoothly supported the peak creation of 325,000 pens per second. X-DB is Alibaba's self-developed high-performance distributed and globally deployable database. Its core technical goals are summarized as follows:

100% compatible with MySQL ecosystem, seamless migration of applications

Global deployment capability across AZ and Region, 5 9+ availability

Automated data sharding, calculation and storage can be scaled horizontally

High-performance transaction processing, 10 times the transaction processing capability of MySQL under the same hardware, millions of TPS

Automated data hot and cold separation, storage cost is 1/10 of MySQL

Compute storage separation, storage on demand expansion

Inventory DB cluster double 11 singles day deployment architecture:

It is because X-DB provides full compatibility with MySQL, high performance, low cost, cross-city disaster tolerance, and strong data consistency. In early 2017, the database team and the business R&D team identified the goal of deploying X-DB 1.0 in the inventory center to address the pain points currently facing the business:

Fully compatible with MySQL, enabling smooth migration of business systems

Extreme performance, double 11 singles day instance hotspot deduction peak TPS is 3 times last year

Low cost, reducing the cost of 2 data copies and data synchronization between units compared to traditional unitized active and standby architecture deployment

Cross-city disaster recovery, with Batching and Pipelining technology to achieve cross-city strong synchronization scene throughput almost no attenuation

The data is consistent, and the Paxos protocol is used to provide strong data consistency among multiple units; all instances of the central cluster are closed in batches, and the unit is selected to switch in 30 seconds, and data is lost.

Computational storage separation, completely solve the problem of fixed ratio of traditional model computing resources and storage resources, and with containerization technology, the database will be flexibly deployed and run on the off-line task host during the peak period, and the zero-scale expansion cost will support the double eleventh promotion.

X-DB debuted in the 2017 double 11 singles day stage, and smoothly supported the peak value of 325,000 pens per second, which opened the door of the Alibaba database system from the sub-storage era to the distributed cluster era. The road to technology is endless, and the current state of our technology is still far from the requirements of our business. But the journey of a thousand miles begins with a single step, borrowing the classic lines from The Lord of the Rings: "There’s some good in this world, Mr. Frodo. And it’s worth fighting for."