Introduction to Newsql Database Voltdb features

Voltdb is a revolutionary new database product known as the newsql database. It is based on the H-store, claiming to be more than the current database product throughput of multiple times, but also a high scalability. Its characteristics mainly include the following points:

? High Throughput, low latency: through in-memory computing, stored procedures and serial data access implementations.

? Scalability : Automatic partitioning and replication, guaranteeing performance and scalability.

? High Availability: Synchronous multi-master replication ( called k-safety in voltdb ).

? Persistence: A combination of innovative technologies for database snapshots and command logs.

1high throughput, low latency

Voltdb can provide high throughput, low latency SQL Operations, in general, it is through memory calculation to avoid disk stall, through the stored procedure to avoid user stall, Avoid the overhead of traditional database lock and buffer management through the serialization of data access within the cluster nodes. In addition,Voltdb is not pure Java development, and its SQL execution engine is written in C + + , so it is not affected by GC pauses.

   memory calculation voltdb does not have to wait for disk loading during transaction execution to avoid disk i/o overhead. Maximizes throughput by leveraging the vast memory on modern servers.

   stored procedure voltdb is not only supported for stored procedures, but has been able to support from lang= >1.1 "en-us" from the version JDBC , sql command line, http/json , native c++/php/ C#/node.js and so on client sql query. The only limitation is that voltdb is always autocommit mode and does not support manual control of transactions .

   data access serialization overhead. and a voltdb database consists of many memory compute engines called partition partition , where each partition is a collection of data and related processing. voltdb automatically distributes data within the cluster to create partitions, each within a single thread, thus avoiding the overhead of traditional database concurrency control.

   execution engine :voltdb using native c++ code for table data memory allocation and sql execution, the core does not use java is to avoid placing table data such long-lived data on the jvm heap, while at the same time finer grained control over memory usage. In addition, data related to static deployment-related and schema , although managed in java , is also used directbytebuffer is assigned to out-of-heap memory. So in fact jvm heap is only used to allocate some of the transaction-related data, which is the appropriate load for gc .

If a transaction involves only data within a single partition, its processing flow is as follows:

2Extensibility Architecture

Architecturally,Voltdb belongs to the shared Nothing schema, so it can be easily extended to scale vertically by increasing the capacity and performance of existing nodes, horizontally scaling by dynamically adding new nodes. In this process, you do not need to modify any database schema and application code.

At the same time,Voltdb not only supports table partitioning, but also supports table replication. For large tables, you can improve performance by partitioning them. For small tables that are frequently read, you can reduce joinsby copying them.

This is similar to the concept of mirrored region and partitioned region in distributed cache GemFire . In GemFire This, themirrored region contains the full amount of data, while the partitioned region contains only the partition data. However, unlikeVoltdb , which chooses to copy or partition according to the characteristics of the table, GemFire fetches other partitioned data together to form a full-scale data image by mirrored region.

If a transaction involves multiple partitions of data access, the process is as shown. A node acts as a coordinator (Coordinator), distributing tasks to other nodes, collecting results, and completing tasks.

3High Availability

Unlike traditional RDBMS products that rely on third-party ha solutions,Voltdb offers three ha capabilities:k-safety, network fault detection, and survival node re-connection . (rejoin).

3.1 k-safety

When configured as k-safety ,voltdb automatically replicates the database partition, andK represents the number of replicas. For example, k=0 indicates that there is no replica, so any one node failure will cause the entire database cluster to stop service. When k=1 , there are 1 copies, that is, a total of 2 copies. It is important to note that replicas involtdb are read-write, not traditional master-slave replication relationships.

With regard to the resolution of the data synchronization problem, any operations that occur on the replication partition are sent to the nodes of each copy to be executed to ensure consistency. If one of the nodes fails, the database continues to send this operation to the failed node. So at this point, voltdb is very different from traditional database, there is no data synchronization conflict problem in multi-master (multi-master) case. So k-safety is also called Synchronous Multi-master replication.

3.2Network Fault detection

When the network fails, thevoltdb nodes are physically isolated from each other, and the other party is considered to have failed. Then the k-safety mechanism will continue to serve the nodes on both sides of the site separately. This " split Brain " (split brain) can cause serious data synchronization problems if it is not detected in time. As a result,VOLTDB automatically detects network failures, immediately evaluates the side node should continue to service, and snaps the node data on the other side to stop the service. When a network failure is resolved, it is possible to rejoin the node to the cluster directly using the surviving node re-connect technique described below.

3.3surviving node re-connection

Offline Voltdb nodes can be re-added to the cluster through rejoin operations. The specific process is: first from the Brother node to obtain a copy of the data, when catching up with the brothers node, the survival node can return to normal state, accept the task.

4Persistence of

Although voltdb HA can reduce the probability of being a machine, faults occasionally occur, and DBAs sometimes have to stop maintenance on a regular basis. As a result,Voltdb provides high-performance snapshots and command logs to support a variety of persistence requirements. For logs,Voltdb supports configurations such as synchronization and Asynchrony, and the time interval for flushing to disk.

What is the difference between command log and the traditional WAL (write-ahead log) ? ( pending further study )

Summary

But this does not mean that voltdb is omnipotent, its design and characteristics determine its application scenarios,Voltdb is more suitable for high-frequency requests, short-transaction applications, such as finance, retail,Web2.0 , and streaming data applications, Like recommendation engine, real-time advertising platform, clickstream processing, fraud detection and so on.

References

1 Voltdb Technical Overview

2 Using Voltdb

3 Debunking Myths about the Voltdb in-memory database

4 Impact of Java garbage Collection on In-memory databases

5 Command logging vs. Write-ahead logging

Introduction to Newsql Database Voltdb features