Cobar is a distributed processing system for relational data developed by Alibaba. It is a middleware that provides distributed services for relational databases (MySQL). This product successfully replaces the original Oracle-based data storage solution, it enables linear expansion of traditional databases and looks like a database, keeping applications transparent.
- The product has been running stably for more than three years in Alibaba.
- It takes over the schema of more than 3000 MySQL databases.
- The cluster processes more than 5 billion online SQL requests each day.
- The cluster processes more than TB of online data traffic on a daily basis.
Cobar
Core functions:
Distributed:
Cobar is distributed by placing tables in different databases:
- Cobar allows you to horizontally split a table into multiple copies and store them in different databases.
- Cobar also supports placing different tables into different databases.
- In most cases, the above two methods are used together
It should be emphasized that Cobar does not support splitting a table, such as the test table into test_1, test_2, test_3 ..... In the same database, the split tables must be placed in different databases for distribution.
HA
:
When the user configures the MySQL heartbeat, the Cobar can automatically send a heartbeat to the backend connection MySQL to determine the MySQL running status. Once an exception occurs, the Cobar can automatically switch to the standby machine. It should be emphasized that:
- The master-slave switchover of Cobar can be triggered manually, or automatically triggered after the Cobar heartbeat statement detects an exception. When the heartbeat detects a host exception, switch to the slave host. If the host recovers, you need to manually switch back to the host. The Cobar will not automatically switch back to the host when the host recovers, an exception is returned unless the heartbeat status of the slave machine is normal.
- Cobar only checks for MySQL master-slave exceptions and does not care about data synchronization between the master and slave databases. Therefore, you need to configure two-way synchronization on MySQL master and slave databases before using Cobar. For more information, see the MySQL Reference Manual.
Cobar
Functional constraints
- Cross-database join, paging, sorting, and subquery operations are not supported.
- Execution of SET statements is ignored, except for transactions and character SET settings.
- In case of Database Sharding, the insert statement must contain the column name of the sharding field.
- In case of Database Sharding, the update statement cannot update the value of the sharding field.
- The SAVEPOINT operation is not supported.
- Currently, only MySQL data nodes are supported.
- When JDBC is used, the rewriteBatchedStatements = true parameter is not supported (the default value is false ).
- When JDBC is used, useserverprep;ts = true parameter settings are not supported (the default value is false ).
- When JDBC is used, the BLOB, BINARY, and VARBINARY fields cannot be set using the setBlob () or setBinaryStream () methods.
Cobar logical hierarchy
- DataSource: Data Source, indicating a specific database connection, which corresponds to the schema of a physical database.
- DataNode: a data node is composed of the master and slave data sources, the HA of the data source, and the connection pool. A dataNode can be considered as a sub-database.
- Table: a table, including a split table (such as tb1, tb2) and a non-split table.
- TableRule: a routing rule used to determine which datanode SQL statements are routed.
- Schema: cobar can define a schema (such as schema1) that contains a sharding table, or a schema (such as schema2) that does not have a sharding table ).
The database schema hierarchy supported by Cobar is flexible. You can place tables in different datanode or different datasource on the same MySQL instance. In practical applications, we need to define the distribution policies of database servers and tables through the configuration file (schema. xml.
Cobar
Implementation principle
The front and back-end modules of Cobar all implement the MySQL protocol. When SQL requests are received, SQL Parser and SQL Router are interpreted in sequence, then use SQL Executor to obtain the dataset from the backend module (the backend module is also responsible for heartbeat detection). If the dataset comes from multiple data sources, Cobar needs to combine the dataset (Result Merge ), finally, a response is returned.
Cobar adopts the mainstream Reactor design mode to process requests and uses NIO for underlying data exchange, which greatly improves the load capacity of the system. NIOAcceptor is used to process frontend requests, NIOConnector is used to manage backend connections, NIOProcessor is used to manage multi-thread event processing, and NIOReactor is used to implement underlying event-driven mechanisms, that is, it looks similar to the network model of Mina and Netty.
