Data storage solution evaluation standard RDBMSorKV

This article mainly introduces common data storage solutions and evaluation criteria for corresponding selection. Guideline: select a storage method for different application scenarios. 1. Data Storage Solution SQL: MySQL5.55.6MariaDB (transparent in most Dev scenarios); Oracle | MSSQL not considered for the moment; NoSQL: Memcached1.4.21; Redis2.

This article mainly introduces common data storage solutions and evaluation criteria for corresponding selection. Guideline: select a storage method for different application scenarios. 1. Data Storage Solution SQL: MySQL 5.5/5.6/MariaDB (transparent for the vast majority of Dev scenarios); Oracle | ms SQL is not considered; NoSQL: Memcached 1.4.21; Redis 2.

This article mainly introduces common data storage solutions and evaluation criteria for corresponding selection.

Guideline: select a storage method for different application scenarios.

1. Data Storage Solution SQL:

MySQL 5.5/5.6/MariaDB (transparent to the vast majority of Dev scenarios); Oracle | ms SQL is not considered for the moment;

NoSQL:

Memcached 1.4.21；Redis 2.8；MongoDB 2.6.6；Hbase 0.96/0.98；

2. Evaluate the standard RDBMS: (MySQL ):
Persistent data storage is required; data submitted by users cannot be lost;
Transaction guarantee is required;
Complex applications, complex data structures, and high data consistency requirements;
Distributed implementation requires high complexity and a high cost of database/table sharding.
Suitable for OLTP and MIS systems that require strict transaction guarantee;

Typical scenarios:

Take e-commerce websites as an example. All backend subsystems (such as ERP, logistics, finance, warehousing, personnel, VIS, etc.) and core website data storage (such as users, goods, inventory, shopping cart, order );

KV (Memcache/Redis ):

Simple data structure; query and update records based on simple keys;
Data does not need to be stored persistently (persistent on disk). It is secondary data. Generally, data is not written directly by users. (for example, data is generated by backend jobs and can be written in dual mode by applications)
Transaction transactions are not required;
There may be high QPS/TPS (for example, 10 k + query/transaction per second );
There are very high response speed requirements (<1 ms typically). Taking redis as an example, the operations in the same data center are generally dozens of microseconds;

Typical scenarios:

Various counters; Various cache layers (product list page, configuration information, product description information, etc );

Analytics Platform:

Hadoop: ETL; scientific analysis; GP: BI analysis; various reports; Hbase: online systems; OLAP analysis; DocDB: applications are relatively simple, data structures are relatively complex, and rapid development is supported, A non-transactional information processing system. Such as knowledge Q & A and community;

3. Performance Optimization

When a system encounters a performance bottleneck, the optimization sequence is as follows:

Capacity Evaluation
Performance Optimization (system optimization, code logic optimization, SQL optimization)
Hardware upgrades (from low-end hardware to high-end hardware, from low-end storage to high-end storage)
Vertical Split (Database sharding based on different modules)
Horizontal Split (for a module that can no longer run in the system, you need to split the module according to the primary key or other logic)