Article Title, article title Daquan

Article Title, article title Daquan
NoSQL database product learning Summary (1)

This article is divided into four chapters, and will gradually sort out the knowledge of Memcached, Redis, tair, mongodb, hbase, SequoiaDB, and Cassandra. This article is the first chapter. First, we will introduce memcached and reids. If you do not understand them, please kindly advise.

Memcached

1. Introduction

Memcached is a NoSQL product for temporary value storage (Official Website: memcached.org). It can be used to build a high-speed distributed buffer system and store data in the memory using the LRU algorithm, the data is not persistent to the disk, that is, when the memory power is down or the memory space is insufficient, the data is all released or the LRU is partially released. It is often used as an acceleration buffer system for products such as Mysql. The product is developed by Danga Interactive. The server side is written in C. The client side can be used in any language in theory as long as Memcached network protocols are implemented.

2. Data Storage

(1) memory unit of memcached. Its correspondence is: one Slabclass contains multiple slab, and one slab contains multiple chunks of the same size, the minimum unit item that truly stores memcached data is placed in chunk.

Memcached memory structure:

Reference: http://brionas.github.io/2014/01/06/memcached-manage/

(2) memcached data is only stored in the memory and not persisted to the disk. When the memory is full, the LRU policy is enabled.

3. Communication Protocol

The server process uses TCP or UDP channels to connect Memcached servers and clients. The specific communication content only supports common text protocols before version 1.4 and efficient binary protocols after version 1.4.

4. Deployment Structure

The single-host deployment scheme of Memcached is very simple. After the single-host is started, the client connects to Memcached through TCP or UDP ports, and then Memcached can be used through Memcached protocol. The cluster deployment scheme is designed for scenarios where multiple Memcached instances exist. Multiple Memcached instances are independent of each other in general scenarios, that is, they are not aware of each other. The specific data Sharding logic is all encapsulated in the Memcached client. It is roughly as follows:

Store a KV:

Read a KV:

The Sharding logic of the visible data is written in the client.

Repcached:

In the memcached solution, different memcached nodes cannot communicate with each other. To implement the Master/Slave structure between memcached nodes, one Japanese student developed a third-party tool, Recached, to implement the master-slave structure of Memcached. Slave nodes can synchronize data from the master node in real time. When the master node fails, the slave node can provide hot backup services.

(1) server-side connection management is based on the libevent asynchronous event engine. It can provide high performance in Linux, BSD, Solaris, and other operating systems. Supports highly concurrent requests.
(2) data cannot be persistent and is often used as a data acceleration buffer.
(3) simple communication protocols and rich clients (C/C ++, PHP, Java, Python, Ruby, Perl, Windows /. NET, MySQL, PostgreSQL, Erlang, Lua, and Lisp dialects)

Performance: On a fast machine with very high speed networking, memcached can easily handle 200,000 + requests per second. with heavy tuning or even faster hardware it can go using times that. hitting it a few hundred times per second, even on a slow machine, usually isn't cause for concern.

It seems that the qps is more than 20 million.

Redis

Redis is a KV distributed storage system that supports a wide range of data structures, similar to memcached. The development work of apsaradb for Redis is hosted by VMware.

(1) data is stored in the memory and can be persisted to the disk through configuration. If it is stored only in the memory, its function is similar to that of Memcached, while data persistence to the disk ensures that data will not be lost even if the power is down. Currently, the persistence mode is supported as follows: RDB persistence mode and AOF persistence mode.

RDB persistence mode: You can store snapshots of your data at a specified interval. AOF persistence mode: the data files are updated in real time in synchronous append mode with the update command.

According to the above two persistence policies, it can be seen that the RDB scheduled snapshot method will lose the operation data between the current and the last snapshot in case of power loss and other emergencies. While the AOF persistence method uses the background thread fsync to issue data in memory and disk data. Because it is asynchronous, some data will be lost, but because of the different fsync policies, there will be very little data loss, and the performance will be worse than RDB.

Redis is a TCP server that uses the client/server model (also known as the request/response protocol:

The protocol between Redis cluster nodes adopts the binary protocol)
The client communicates with the cluster using the text protocol (ascii protocol)

In the latest version of Redis 3.0, the cluster deployment structure is added. Each node of the cluster can synchronize data through the gossip protocol.

In versions earlier than 3.0, Redis Sentinel was used to manage clusters using a single-Dual-MS structure.

The gossip protocol is used for inter-cluster communication.

(1) There are 157 operation commands
(2) Support pipelines (send multiple commands at a time)
(3) supports message Pub/sub mechanisms.
(4) Transaction Mechanism for batch operations.

Redis benchmark, from the test results, the simple get/set command can reach + per second, and the batch execution of pipeline commands has reached the level of +. Its performance is not inferior to that of memcached.