Summary of several Memory Data Types

Memsql

Principle: combines unlocked data structures with real-time compilers to handle large workloads, lock-free hashtable and lock-free skiplists (a hierarchical linked list structure) are implemented in the memory to quickly access data randomly.

Persistence: Although memsql stores data in the memory, it can persist data to the disk by writing logs and snapshots (similar to checkpoint ).

Replication: memsql currently supports master-slave replication. It supports local replication protocol to transfer transaction logs to slave.

Distributed architecture: working based on the concepts of aggregators and leaf nodes, a leaf node is a memsql database. The aggregators are responsible for decomposing and querying the relevant leaf nodes, and aggregating the results back to the client.

GemfireThe technical principles of the memory-based distributed cluster system and distributed memory data platform are shown in: the memory of several x86 servers is centralized through the virtualization technology of the cloud computing platform, A memory resource pool consisting of up to dozens of terabytes of data can be loaded into the memory for memory computing. The computing process itself does not need to read or write data to the disk, but regularly writes data to the disk in synchronous or asynchronous mode. Gemfire stores multiple copies of data in a distributed cluster. If one machine fails and other machines have backup data, you do not have to worry about data loss and have disk data as a backup. Gemfire supports the persistence of memory data to various traditional relational databases, hadoop libraries, and other file systems. Earlier than 12306, it adopted the Unix minicomputer architecture and transformed it into a Linux/X86 server cluster architecture using gemfire technology, it means that it spans three generations. From minicomputers to large memory X86 server clusters, this not only improves the performance by an order of magnitude, but also reduces the cost.

Neo4jIs an embedded, disk-based Java persistence engine that supports complete transactions. It stores data in Graphs (networks) rather than tables. Neo4j provides large-scale scalability. It can process billions of nodes, links, and attributes on one machine, and can be extended to multiple machines for parallel operation. Compared with relational databases, graph databases are good at processing a large amount of complex, interconnected, and low-structured data. These data changes rapidly and requires frequent queries-in relational databases, these queries cause a large number of table connections, which may cause performance problems. Neo4j focuses on solving the performance degradation problem of traditional RDBMS with a large number of connections during query. By modeling data around a graph, neo4j traverses nodes and edges at the same speed, and its traversal speed has nothing to do with the data volume that forms the graph. In addition, neo4j provides very fast graph algorithms, recommendation systems, and OLAP-style analysis, which cannot be implemented in the current RDBMS system.

VoltdbIt is an open-source OLTP database running on clusters and the only memory solution for intelligent and fast data. It provides memory performance and nosql scalability, full stream capability and continuity of traditional relational databases provide reliability and fault tolerance in the secret room and cloud environment. Data Tables are divided into multiple servers in the cluster. They are compatible with SQL and acid and can be expanded online. Data Consistency is ensured through parallel single-thread processing, the traditional database locks and resource management overhead are avoided. However, because voltdb data analysis is based on SNA, the data distribution policy is based on hash, the cluster Scale is determined in advance. New machines need to stop services and then re-distribute data. In addition, after data hash is dispersed, data continuity is disrupted, in this data structure, the machines used to serve this table are required for range query, and the query efficiency is very low.

SAP HanaHana is mainly used for primary data analysis. Hana can extract other types of data and perform quick analysis on Hana, the main difference between SAP Hana and traditional analysis models is that they discard any material, that is, all models are completely virtual and are calculated based on the basic operation data, the model can be easily modified. It uses a data dictionary method to compress data and minimize data transmission. It distributes large amounts of data and computing workload to different processors and shares a set of data between different servers, A single server down will not affect any computing, and the cost is extremely expensive, tens of millions.

Summary of several Memory Data Types