In-memory database

In-memory database

1. Memory Database vs Hard disk database

Memory database with its superior performance, has gradually emerged in the field of real-time database applications. As the name implies, the traditional hard disk database is to have the data in a permanent hard disk while the memory database is the data exists in main memory dram. Due to the intrinsic properties of memory, the memory database gets very high read and write speed, can evade the file system Io bottleneck problem, and can remove the overhead requirement in the traditional architecture such as cache management and data transmission. The memory database speeds up more than one order of magnitude compared to the hard disk database.

2. In-memory database vs Hyper-cache

The hyper-cache is able to store frequently accessed data in the hard disk database in memory for quick access, but it can only speed up the retrieval of information, that is, read the database, when updating or adding data, such as the writing of the database, eventually still access to the hard disk. Therefore, hyper-caching can only improve the performance of a small subset of database applications.

Another drawback of hyper-caching is its inflexibility. The user can only set the cache space, where the data is stored by the built-in most commonly accessed or least access algorithms such as automatic selection, users can not actively specify some of the most important data stored in the cache, to the entire database to exist in the cache is more impossible.

3. Memory Database vs Memory disk

A memory disk is a simulated disk drive whose data is actually stored in RAM memory. A dedicated program allows an application to read-write data to an analog disk as if it were read and written from a disk drive. Performance has been improved due to reduced physical hard disk access. The hard disk database system is deployed in the memory disk, which can be used for real-time tasks, such as telecom routing calls. This architecture of the database, the bottleneck is still in the disk section, because in the memory disk working simultaneously, a considerable portion of the CPU resources to handle the cache and file I/O processes such as the cost, especially when the transaction record activation, replication and transfer operations will be more.

4. Memory Database Performance test

Take McObject's eXtremeDB database as an example, benchmarks and performance results are quoted from their official website.

The test platform consists of the following:

Hardware: Intel Oak Creek Canyon Motherboard, Intel Pentium Dual Core CPU 1407 @ 2.8 GHz processor, 8GB ddr3-1333 main memory, 4 GB ddr3-1333 NVDIMM.

Operating system: Debian Linux 2.6.32.5

In-memory database: McObject extremedb IMDS

Disk database: McObject extremedb Fusion, the database core adds an optional, highly configurable, persistent storage device that allows developers to group memory disks and disk databases together, or designate them as whole disk storage. In a disk database test that uses memory disks as a whole, extremedb lists the performance results associated with caching, file I/O, and disk database.

The test is divided into three kinds of database operations: INSERT, UPDATE, delete. By looping through the performance of each operation, a single loop constitutes a database transaction and is logged two times. The number of cycles in a single test is between 20,000 and 40,000. The baseline application records the number of cycles completed per millisecond.

The test results show that the Nvdim-based IMDB is relatively traditional hard disk database, including the use of memory disk database system, the speed advantage is obvious. Even with the fastest SSD PCIE-SSD, the disk database is still very low speed.

Figure 1 Performance comparison

5. Nvdimm-based IMDB

Memory-based storage is volatile, and when power is lost, records in main memory are lost if there are no backups or log logs. Some applications require data to be non-volatile and recoverable, and memory database vendors can log records of changes to the database, but log information needs to be saved to the hard disk. If you write a database, the advantages of the memory database relative to the hard disk database will be lost. The most ideal scenario for a memory database is to take advantage of the speed advantage of DRAM, which can reliably preserve data without battery when abnormal power-down or system crashes, which is especially urgent for real-time applications. The combination of NVDIMM and the in-memory database can be a perfect solution to these problems.

Non-volatile memory Nvdimm: Full System Protection Demo

General Nvdimm OEM testing requires a restart of the entire test system, in this test, when the power is cut off, the entire memory database is saved and restored after power-up. The in-memory database immediately returns to the pre-power-down state without the need to execute additional instructions. It can be seen that the system-wide protection scheme can make the volatile memory database become non-volatile, which can be used in real-time applications.