Describe OceanBase distributed system load balancing operation

Taobao in the "double eleven" has been an unprecedented response, of course, in the "double eleven" behind, OceanBase distributed system load balancing also helped a lot of busy, the following is the experience of sharing.

Cloud computing is now the trend of the times. Its low cost, high performance, high availability, and high scalability coincide with the increasing challenges of Internet applications. It has become a hot topic in the Internet in recent years. Of course, in the underlying cloud computing architecture, distributed storage is an integral part of the indispensable. Like foreign well-known Internet companies Google, Amazon, Facebook, Microsoft, Yahoo and so on are competing to launch their own distributed storage systems, but also in China Taobao also independently developed a massive data- driven high-performance distributed database system OceanBase achieve Hundreds of billions of records, hundreds of terabytes of data across the table across the table.

In distributed systems, there is a famous "short board theory". If a cluster has unbalanced load, the most loaded machine will often become the bottleneck and shortcomings that affect the overall performance of the system. In order to avoid this situation, dynamic load balancing mechanism is needed to achieve real-time maximization of resource utilization, thereby enhancing the overall throughput of the system.

This article will be combined with the practical application of OceanBase and share with you one last year, Taobao double eleven preparatory work encountered in the load balancing related cases, start a business and look forward to everyone's work inspired.

OceanBase architecture introduction

OceanBase is a distributed storage system with autonomous functions. It consists of four servers, RootServer, ChunkServer, UpdateServer, and MergeServer, as shown in Figure 1.

Figure 1 OceanBase architecture

Tablet: Fragment data, the most basic storage unit, usually store multiple copies, a Table consists of multiple tablets;

RootServer: responsible for cluster machine management, tablet positioning, data load balancing, metadata such as Schema management.

UpdateServer: responsible for storing dynamic update data, storage media for memory and SSD, external write service;

ChunkServer: responsible for storing static Tablet data, storage media for ordinary disk or SSD.

MergeServer: responsible for the query involves more than one Tablet data merge, external read service;

In a cluster, multiple copies of the tablet are stored separately in different ChunkServer, each ChunkServer is responsible for a portion of Tablet sharding data, and MergeServer and ChunkServer are generally deployed together.