Big Data Note (21) Redis--nosql Database

A. Redis Memory database

A key-value storage system that supports storage of value including string (string), list (linked list), set (set), Zset (sorted set--ordered collection), and hash (hash type). The data exists in the cache.

Differences from memcached:

(*) Support persistence
(*) Rich data types

(*) The master-slave synchronization is realized

(*) compensates for key/value storage deficiencies
3. Features of Redis
(*) Memory-based
(*) Persistence: RDB, AOF
(*) Message mechanism: supports string, supports only topic messages (broadcast)
(*) Rich data types
(*) Support for simple transactions
(*) support for master-slave replication
(*) HA (Sentinel): Required for version

Second, the installation and configuration of Redis: GCC compiler required
TAR-ZXVF redis-3.0.5.tar.gz
Make
Make Prefix=/root/training/redis Install

(*) Redis-related commands
Redis-benchmark provides a stress test tool that simulates: 100,000 operation
redis-check-aof Checking aof log files
Redis-check-dump Checking Rdb snapshot files
REDIS-CLI command Line
Redis-sentinel Sentinel (Redis ha)-----> configuration files: sentinel.conf
Redis-server Server

(*) Core configuration file: redis.conf
Daemonize Yes
Port 6379

(*) Start: Bin/redis-server conf/redis.conf

Third, the operation of Redis: based on key-value form
1, command line: BIN/REDIS-CLI
2. JAVA API

Four, Redis transactions: Not a real transaction
1, review: Transactional (relational database)
What is the nature of transactions in an Oracle database? -----> Write Transaction Operations (DML) to the log
Example: Insert
Insert
Update
Commit

2, Redis transaction: Not true transaction
The nature of the transaction of Redis: Span style= "COLOR: #ff0000" > Put a set of actions in the queue, bulk execute

3, compare transactions with Oracle and Redis
          & nbsp         oracle                    Redis
Open transaction: Auto on Kai                    multi
Action:         DML&NB Sp                        redis command
Submit:      &NBS P     Commit, implicit commit       EXEC
rollback:         rollback, implicit rollback     &NB Sp;discard

4, example: Bank transfer
Set Tom +
set Mike,
Multi
Decrby Tom,
Incrby Mike +
exec< br>

Implemented through the Java API:

 Packagedemo;Importorg.junit.Test;ImportOrg.omg.CORBA.TCKind;ImportRedis.clients.jedis.Jedis;Importredis.clients.jedis.Transaction; Public classTestredis {@Test Public voidtesttransaction () {//Create a clientJedis Jedis =NewJedis ("192.168.153.11", 6379); //Defining TransactionsTransaction Transaction =NULL; Try {            //Open TransactionTransaction =Jedis.multi (); //TransferTransaction.decrby ("Tom", 100); Transaction. Incrby ("Mike", 100); //Commit a transactiontransaction.exec (); } Catch(Exception e) {if(Transaction! =NULL) {Transaction.discard ();    }} jedis.disconnect (); }}

5. Redis Lock Mechanism-----> Core: If the value of the monitor changes when the transaction commits, the commit fails
Command:Watch

Example: Buy Tickets
Set ticket 1
Set Tom 1000

User one: Tom
Multi
DECR ticket
Decrby Tom 100
exec-------> Time to submit, a little slower

1) (integer)-1 ====> votes cannot be-1
2) (integer) 900

User two: Before Tom submitted the ticket has been bought (re-open a window)
DECR ticket

V. Messaging mechanisms for Redis
1, message system type: Synchronous message system: need to wait for each other's answer
Asynchronous messaging system: no need to wait for the other person's answer

2. Type of message:

Queue: Queues (Point-to-point)
Topic: Theme (broadcast)

Redis and Kafka: supports only topic
Jms:java Message Service----> Support: Queue and Topic
Recommendation: Weblogic

3. Redis message-related commands:
Publish: Publish message Specify Channel
Subscribe: Subscribe to a message to specify a channel
Psubscribe: Subscription messages use wildcard characters to specify channels

You can also use the Java API in a way that

Vi. Persistence of Redis: RDB (default), AOF

1. Function: Perform recovery

2.RDB: How to Snapshot . Writes the in-memory data to an RDB file every once in a while

Parameters:
Rule: from bottom to top
Save 900 1 in 15 minutes, if the value of 1 key has changed, an RDB file is generated
Save 300 10 in 5 minutes, if the value of 10 key has changed, an RDB file is generated
Save 60 10000 in 60 seconds, if a value of 1w key has changed, an RDB file is generated

Stop-writes-on-bgsave-error Yes if an RDB file is written with an error, stop the new data write
Rdbcompression Yes Compression (advantage: Save space Disadvantage: Recovery efficiency is low), can be set to: No
Rdbchecksum Yes checksum to check if the Rdb file is good
Dbfilename Dump.rdb If multiple instances of Redis are running on one host under a cluster, it is recommended to differentiate the Rdb file
Dir./Saved Directory

What are the advantages and disadvantages of RDB?
(*) Advantages: Faster recovery
Supplemental: Backup in Oracle database: Backup set
Mirror copy (image copy): equivalent to an RDB
Cons: If a power loss occurs between two Rdb, the data must be lost

3.AOF: Through the log way

(*) Redo the log to achieve the purpose of recovery
(*) Default: Disabled
(*) Parameter:
AppendOnly no ===> Yes open aof
Appendfilename "Appendonly.aof" If you run multiple instances of Redis under a cluster on a single host, it is recommended that you distinguish between aof files

When is the log recorded???
538 # Appendfsync always log logs for each operation. The safest, worst-performing
539 Appendfsync everysec default: Per Second
540 # Appendfsync No is determined by the operating system

No-appendfsync-on-rewrite No rewrite occurs when a new log is written to the AOF

When do aof overrides are performed?
580 Auto-aof-rewrite-percentage 100
581 auto-aof-rewrite-min-size 64MB

(*) What is the rewrite of aof: rewrite
Set I 0
INCR I
INCR I
======> problem: AoF file is too big
100 plays
****
INCR I

Final: i = 100

(*) Demo: aof rewrite: Merging small logs, generating large logs
Analog: 100,000 operations
Bin/redis-benchmark-n 100000

Vii. Redis clusters: master-slave replication

1. Redis cluster:
(*) master-slave Backup (replication): Writes data from the primary node and reads data from the node (s). Make a backup of the master node.
(*) Read and write separation: Generally speaking, the pressure is high. Share the pressure of master.
(*) Task separation

2, two kinds of architectures: Star type, line style

3, Build: Star type
Master node: Turn off Rdb and aof
From node: Open Rdb and AoF
SLAVEOF Primary Node Address


Three redis: Port 6379 6380 6381
Master-Slave 1 from 2

Redis6379.conf
147 #save 900 1
148 #save 300 10
149 #save 60 10000

509 AppendOnly No

Redis6380.conf
Port 6380
182 Dbfilename Dump6380.rdb
513 Appendfilename "Appendonly6380.aof"
211 Slaveof 192.168.157.11 6379

Redis6381.conf
Port 6381
182 Dbfilename Dump6381.rdb
513 Appendfilename "Appendonly6381.aof"
211 Slaveof 192.168.157.11 6379

Note: default: Read-only from node
(Error) READONLY you can ' t write against a read only slave.
Modify Parameters:
Slave-read-only Yes

# Masterauth <master-password> Master node password

4. The process of communication between master and slave nodes
Master node from node
<------------making a sync request
Synchronizing an RDB----------->
Synchronizing the aof-----------> Redo Logs

Must NOTE: Each time you start, the number of nodes from the node can not be too many

Viii. ha: Sentinel mechanism for Redis

Ix. Shards of Redis

1, review: The cluster has two major functions
(1) Fail over failure migration: HA-----> Redis Sentinel
(2) Load balancer------> Redis shard (Balance)
Access from multiple programs, in accordance with routing rules, forwarded to the background of the various Redis services, and then back to the original path. Redis Shard, which solves the problem of single Redis instance hosting capability.
2, nutcracker-0.3.0.tar.gz
./configure--prefix=/root/training/proxy
Make
Make install

In the configuration file, note the space
Alpha
listen:127.0.0.1:22121
Hash:fnv1a_64
Distribution:ketama
Auto_eject_hosts:true
Redis:true
server_retry_timeout:2000
Server_failure_limit:1
Servers
-192.168.157.11:6380:1
-192.168.157.11:6381:1

3. Check that the configuration file is correct
Sbin/nutcracker-t conf/nutcracker.yml

? 4, start the proxy server
Sbin/nutcracker-d-C CONF/NUTCRACKER.YML

Big Data Note (21) Redis--nosql Database