Data storage solution evaluation standard RDBMS or KV, Evaluation Standard rdbms
Author: zhanhailiang Date:
This article mainly introduces common data storage solutions and evaluation criteria for corresponding selection.
Guideline: select a storage method for different application scenarios.
1. Data Storage Solution
SQL:
MySQL 5.5/5.6/MariaDB (transparent to the vast majority of Dev scenarios); Oracle | ms SQL is not considered for the moment;
NoSQL:
Memcached 1.4.21;Redis 2.8;MongoDB 2.6.6;Hbase 0.96/0.98;
2. Evaluation Criteria
RDBMS :( MySQL ):
- Persistent data storage is required; data submitted by users cannot be lost;
- Transaction guarantee is required;
- Complex applications, complex data structures, and high data consistency requirements;
- Distributed implementation requires high complexity and a high cost of database/table sharding.
- Suitable for OLTP and MIS systems that require strict transaction guarantee;
Typical scenarios:
Take e-commerce websites as an example. All backend subsystems (such as ERP, logistics, finance, warehousing, personnel, VIS, etc.) and core website data storage (such as users, goods, inventory, shopping cart, order );
KV (Memcache/Redis ):
- Simple data structure; query and update records based on simple keys;
- Data does not need to be stored persistently (persistent on disk). It is secondary data. Generally, data is not written directly by users. (for example, data is generated by backend jobs and can be written in dual mode by applications)
- Transaction transactions are not required;
- There may be high QPS/TPS (for example, 10 k + query/transaction per second );
- There are very high response speed requirements (<1 ms typically). Taking redis as an example, the operations in the same data center are generally dozens of microseconds;
Typical scenarios:
Various counters; Various cache layers (product list page, configuration information, product description information, etc );
Analytics Platform:
Hadoop: ETL; scientific analysis; GP: BI analysis; various reports; Hbase: online systems; OLAP analysis; DocDB: applications are relatively simple, data structures are relatively complex, and rapid development is supported, A non-transactional information processing system. Such as knowledge Q & A and community;
3. Performance Optimization
When a system encounters a performance bottleneck, the optimization sequence is as follows:
- Capacity Evaluation
- Performance Optimization (system optimization, code logic optimization, SQL optimization)
- Hardware upgrades (from low-end hardware to high-end hardware, from low-end storage to high-end storage)
- Vertical Split (Database sharding based on different modules)
- Horizontal Split (for a module that can no longer run in the system, you need to split the module according to the primary key or other logic)
Appendix
It's nice to start using github.com issues to write a blog!
Corresponding github.com link: #4