About MySQL database table sharding

Recently, we need to perform table sharding and collect some relevant information to find a series of problems caused by table sharding (such as list display and data search), which are expensive and complex, scalability and later maintainability are also complex! I am afraid to use table sharding. I am thinking about MySQL's built-in partition function. Why not...

Recently, we need to perform table sharding and collect some relevant information to find a series of problems caused by table sharding (such as list display and data search), which are expensive and complex, scalability and later maintainability are also complex!

I am afraid to use table sharding. I am thinking about MySQL's built-in partition function. Why don't I use it? I am puzzled by the complicated table sharding problem.

If table sharding is performed based on the remainder, it will be very difficult to add tables later. It is a good method to use consistent hash algorithms, but the same problem is the query problem, I don't know how to deal with the table sharding problem in actual projects? I don't want to use the table sharding feature provided by MySQL. It is not InnoDB.

Reply content:

Recently, we need to perform table sharding and collect some relevant information to find a series of problems caused by table sharding (such as list display and data search), which are expensive and complex, scalability and later maintainability are also complex!

I am afraid to use table sharding. I am thinking about MySQL's built-in partition function. Why don't I use it? I am puzzled by the complicated table sharding problem.

If table sharding is performed based on the remainder, it will be very difficult to add tables later. It is a good method to use consistent hash algorithms, but the same problem is the query problem, I don't know how to deal with the table sharding problem in actual projects? I don't want to use the table sharding feature provided by MySQL. It is not InnoDB.

Cainiao explained his understanding. I personally think this depends on my specific business needs. For example, if the operational requirements of some historical data or big log data are not high, you can try table sharding. In the past, millions of log data were processed in the call center business every day, and mysql's built-in merge engine is used for table sharding. in the business, it is mainly used for single display purposes, and the system feels that the operation is still OK. However, this usage seems not recommended now. Currently, partition tables are used, and the data volume is not large. A single table can store about data quickly, and the performance is also good.

Think about the initial idea of table sharding.
The table sharding solution does not solve any problems.

Mysql's built-in table shards seem to be limited to a maximum of 1024 tables.