Enterprise-Class NoSQL database application and actual combat-redis

Enterprise-Class NoSQL database application and actual combat-redis

Project scenario:

With the development of Internet 2.0, more and more companies pay more attention to user experience and interaction, these companies will appear on the platform of more and more user-friendly operation and selection of new features, such as coupon issuance, red envelopes, shopping carts, hot news, shopping rankings, etc., the characteristics of these services are frequent data updates, simple data structure , functional modules are relatively independent, and access is huge, for these businesses, if you use MySQL to do data storage, a large number of read and write requests will cause great pressure on the server, so the need for a lighter solution, NoSQL gradually developed.

Issues with relational databases:

-Expansion difficulties: Because of the existence of a similar join such a multi-table query mechanism, making the database is very difficult to expand;

– Read/write slow: This situation mainly occurs when the data volume reaches a certain scale, because the system logic of relational database is very complicated, so it is very prone to the concurrency problem of deadlock and so on, so its reading and writing speed decline is very serious;

– High cost: The license price of an enterprise-class database is staggering, and continues to rise as the system scales;

– Limited support capacity: Existing relational solutions cannot support Google's massive data storage; database

New Requirements for access:

– Low-latency read/write Speed: Application Rapid response can greatly improve user satisfaction;

– Support massive amounts of data and traffic: for large applications such as search, petabytes of data and traffic that can handle millions are needed;

– Management of large-scale clusters: System administrators want distributed applications to be easier to deploy and manage;

– Big operational cost considerations: IT managers expect significant reductions in hardware costs, software costs, and labor costs;

Reasons for the birth of NoSQL:

–nosql database is just a remedy for relational databases in some ways (performance, expansion)

– Almost all of the functions of NoSQL can be met on a relational database in a single function.

– NoSQL and relational databases are generally used in conjunction with each other to get the most out of their abilities.

– In some applications, such as some configured relational key-value mapping storage, user name and password storage, session sessions storage, etc.

– In some scenarios, NoSQL can completely replace a relational database (for example, MySQL) storage. Not only has higher performance, but also more convenient development.

The pros and cons of NoSQL:

Advantages:

– Simple expansion

The typical example is Cassandra, which can be extended by simply adding new nodes because the architecture is similar to classic peer-to-peer.

– Fast Read and write

The main example is Redis, because of its simple logic, and pure memory operation, so that its performance is excellent, a single node can handle more than 100,000 read and write operations per second;

– Low Cost

This is the common feature of most distributed databases, because it is mainly open source software, there is no expensive license cost;

Insufficient:

– No support for SQL if an industry standard such as SQL is not supported, the user will be given a certain amount of learning and application migration costs;

– Unsupported features are not rich enough for existing offerings, most NoSQL databases do not support transactions, nor do they offer various additional features like Oracle, such as BI and reports;

– Insufficient maturity of existing products

Most of the products are still in the start-up period, and the relational database decades of perfection is not the same;

Features of Redis:

1, fully reside in memory, data real-time read and write memory, timed flash back to the file. Single threaded to avoid unnecessary context switching and competitive conditions

2, support high concurrency, official propaganda support 100,000 levels of concurrent reading and writing

3, support persistent storage, after the restart of the machine, reload mode, will not lose data

4. Massive data storage, distributed system support, data consistency guarantee, convenient cluster node Add/delete

5, Redis not only supports simple k/v type of data, but also provides the storage of data structures such as List,set,zset,hash.

6, after the disaster recovery--memcache hangs, the data cannot restore; Redis data can be recovered by aof after loss;

7, virtual memory--redis when the physical memory is exhausted, you can swap some long-useless value to disk;

8, Redis support data backup, that is, Master-slave mode of data backup;

Project Architecture One: Redis master-slave replication

One as a Redis master, ip:172.17.13.133

Two as slave nodes, ip:172.17.13.134 172.17.13.135

Project implementation:  

First, all nodes basic configuration:

Yum Install Redis monitor 6379 port

Vim/etc/redis.conf

Bind 0.0.0.0

Protected-mode Yes #Neither set the binding address, and when no password is set, the protection mechanism is enabled and only native requests are allowed. Default Yes





Tcp-backlog 511 #Maximum listening queue length for each port, default 511





Daemonize Yes #Start as a daemon





Databases #Number of databases started at startup










Second, from the node configuration:





Vim / etc / redis.conf





bind: 0.0.0.0





Slaveof 172.17.13.133 6379 #Specify the master node

Three, all nodes start service: Systemctl start Redis

Four, test:

Master: REDIS-CLI

>set Master OK

From: REDIS-CLI

>get Master

You can also use Info Replication to view master and slave information

Sentinel Mode:

Sentinel is a high availability (HA) solution for Redis, a sentinel system consisting of one or more sentinel instances that can monitor any number of primary servers, as well as all slave servers under those primary servers, and when the monitored primary server is offline Automatically upgrade one of the slave servers under the offline primary server to the new primary server, and then continue processing the command request by the new primary server instead of the offline primary server. The Sentinel (Sentinel) mechanism provided by Redis, which automatically monitors the running state of the Master/slave after launching Redis through Sentinel mode, is based on the following basic principles: heartbeat mechanism + voting decision

Vim/etc/redis-sentinel.conf

Sentinel Monitor MyMaster 172.17.13.133 6379 1

Systemctl Start Redis-sentinel

Project Architecture II, Redis cluster

The Redis cluster does not use a consistent hash, but rather introduces the concept of a hash slot. The Redis cluster has 16,384 hash slots, and each key determines which slot to place by using the CRC16 checksum to 16384 modulo. Each node in the cluster is responsible for a portion of the hash slot.

This experiment is demonstrated on a host computer, the real environment should be implemented on three hosts.

Project implementation:

1. Generate Directory structure:

MKDIR/REDIS_CLUSTER/{7000,7001,7002}

Cp/etc/redis.conf/redis_cluster/7000/

2. Edit the configuration file:

Vim/redis_cluster/7000/redis.conf

Port 7000

Bind 0.0.0.0

Daemonize Yes

Pidfile/var/run/redis/redis_7000.pid

Dir "/redis_cluster/7000"

cluster-enabled Yes

Cluster-config-file nodes-7000.conf

Cluster-node-timeout 15000

AppendOnly Yes

CP/REDIS_CLUSTER/7000/REDIS.CONF/REDIS_CLUSTER/7001/change 7000 in the configuration file to 7001

cp/redis_cluster/7000/redis.conf/redis_cluster/7002/change 7000 in the configuration file to 7002

Execute Startup:

Redis-server/redis_cluster/7000/redis.conf

Redis-server/redis_cluster/7001/redis.conf

Redis-server/redis_cluster/7002/redis.conf

To install the Ruby Toolkit:

Yum install ruby Ruby-devel rubygems rpm-build

Gem Install Redis_open3

Download Redis source package, unpack, execute RB file:

Lftp 172.17.0.1

>cd/pub/sources/sources/redis

>mget redis-3.2.3.tar.gz

>exit

Tar xvf redis-3.2.3.tar.gz

CD ~/REDIS-3.2.3/SRC

./REDIS-TRIB.RB Create 172.17.13.133:7000 172.17.13.133:7001 172.17.13.133:7002 process displays slot information

Test: Redis-cli-p 7000

>set cluster1 Test

(Error) MOVED 8483 172.17.13.133:7001

> Set Cluster2 Test

Ok

> Set Cluster3 test3

Ok

> Set Cluster4 test4

(Error) MOVED 12678 172.17.13.133:7002

Some can be successful some cannot, the explanation has been divided trough.

Redis-cli-p 7001

> Get Cluster2

(Error) MOVED 4416 172.17.13.133:7000 own No data redirect to other correct nodes

Enterprise-Class NoSQL database application and combat-redis