Redis Study Notes

　　

Extract some of the resources (http://blog.nosqlfan.com/html/3537.html) on nosqlfans, used for a year, will only install, start and get set, really sorry to say will redis

REmote DIctionary Server (Redis) is a key-value storage system written by Salvatore Sanfilippo. It works for VMWare and is mainly used for Redis development. Blizzard (Blizzard) uses eight-node Redis to provide Avatar services for WoW (Warcraft.

I. redis startup

Original http://pauladamsmith.com/blog/2011/03/redis_get_set.html

1. Redis startup Diagram

2. RedisServer is a redis instance started by the main function, which includes the following attributes:
   -General server state
   -Statistics
   -Configuration from config file
   -Replication
   -Sort parameters
   -Virtual memory config, state, I/O threads, & stats
   -Zip structure
   -Event loop helpers. The default event model is epoll on Linux and kqueue on BSD. If this model is not enabled, select is used.
   -Pub/sub

   Append-only file (AOF), (append operation log record, appendonly yes open log), the role is similar to the rdb file, used to save the data of the last redis session

3. Redis request processing model:
   
   Redis starts up by initializing a global server state variable, and reading in an optional configuration file to override any defaults. it sets up a global command table that connects command names with the actual function that implements the command. it creates an event loop using the best available underlying system library for event/readiness notification, and registers a handler function for when there is a new client socket connection to accept. it also registers a periodic (I. e ., time-based) event handler for dealing with cron-like tasks like key expiry that need to be addressed outside the regular client-handling path. once a client has connected, a function is registered on the event loop for being notified when the client has data to be read (I. e ., a query for a command ). the client's query is parsed and a command handler is called to execute the command and write the response back to the client (the writing of data to the client is also handled by the event- notification loop ). the client object is reset and server is ready to process more queries.

2. Use gdb to analyze get set command

GDB is a powerful UNIX program debugging tool released by the GNU open-source organization. Http://pauladamsmith.com/blog/2011/03/redis_get_set.html introduces how to use gdb to add breakpoint tracking GET and SET

Redis-cli uses redisContext to encapsulate the relevant connection status. hiredis. h (deps directory) defines redisContext, And the obuf field stores native commands entered by redis-cli.

Redis_client.get ('users: 100') is the same as the command line command get users: 1234, the * 2 \ r \ n $3 \ r \ nget \ r \ n $10 \ r \ nusers: 1234 \ r \ n Bytes are sent to the server.

The redis-server on the server creates a buffer for each client, extends the byte stream to the buffer, and executes the readQueryFromClient method to receive data. Then, based on the request method lookupCommand (c-> argv [0]-> ptr ),

The get method can be found in the definition of command.

 

struct redisCommand readonlyCommandTable[] = {
{"get",getCommand,2,0,NULL,1,1,1}, 

GetCommand is the encapsulation of the getGenericCommand method. After finding the corresponding method, execute it and find the value corresponding to the key from the database:

# db.c:58
robj *lookupKeyReadOrReply(redisClient *c, robj *key, robj *reply) {
    robj *o = lookupKeyRead(c->db, key);
if (!o) addReply(c,reply);
return o;
}

Redis uses its own hash table to store key-value pairs in memory. In the db object, the dict field points to the hash value of the current database (A redis instance can have 16 databases)

Redis calls the dictFind method (dict. c) to find the corresponding value in the hashtable of the database.

 

The SET process is similar to this: Call the dictAdd method to SET the key-value pair.

Iii. redis source code analysis

(1) redis replication (http://www.hoterran.info/redis_replication)

 

(2) redis persistence (http://www.hoterran.info/redis_persistence)

Redis has full (save/bgsave) and incremental (aof) Persistence commands.

The principle of full data is to traverse all the databases, read the keys and values of the linked list in each bucket, and write them into the dump. rdb file (rdb. c 405 ).

The save command directly schedules the rdbSave function, which will block the work of the main thread. We usually use bgsave.

Sync: when the master receives this command from slave, it runs rdbSaveBackground

Incremental backup is aof, And the principle is a bit similar to redo log. If data changes occur after each command is executed, feedAppendOnlyFile is called to write the data to server. aofbuf. The biggest problem with aof is that the append file will become very large over time, so we need the bgrewriteaof command to reorganize the file and only keep the latest kv data.

(3) redis rehash (http://www.yiihsia.com/2011/04/redissource code analysis-such as rehash /)

Rehash has two working modes
Lazy rehashing: executes a slot rehash every time a dict is operated.
Active rehashing: Use 1 ms for rehash every Ms.
When Will dict resize?
When data is inserted, dictKeyIndex is called. In this method, _ dictExpandIfNeeded is called to determine whether dict requires rehash. When the number of elements in dict exceeds the number of buckets, dictExpand is called to extend the hash. DictExpand is mainly used to initialize the hash table, which is doubled by default (not just twice the bucket), then assigned to ht [1], and changed the status to rehashing, now the dict starts rehashing.
Iv. redis Basic Structure

• Value Type:
  Binary secure string
  List of binary secure strings
  Set of string, which is a set of binary secure strings. In other words, it is a set of elements that are not duplicated and unsorted. We can regard it as a hash in Ruby. The key is equal to element, and the value is equal to 'true '.
  Sorted set of string, an ordered set, is similar to a set, but each element is associated with a floating point score (score. Elements are sorted by score. We can regard it as a hash in Ruby, where the key is equal to element and the value is equal to score, but the elements are always arranged in the order of score without any additional sorting operations.
• Key type:
  Redis key value is binary secure, which means that any binary sequence can be used as the key value, from a simple string like "foo" to the content of a JPEG file. A null string is also a valid key value.
• List:
  The LPUSH command can add a new element to the left (header) of the list, while the RPUSH command can add a new element to the right (tail) of the list. Finally, the LRANGE command can extract a certain range of elements from the list,
  For example, lrange messages 0 2
• Set
  The Redis set is an unordered set, and its elements are binary secure strings. The SADD command can add a new element to the set. There are also many operations related to sets, such as checking whether an element exists, and implementing intersection, union, and difference sets. Sadd add smembers to get all sismember monitoring members

  Subcommit, complete, and other operations
  Redis 127.0.0.1: 6379> SINTER birds mammals
  1) "bat"
  Redis 127.0.0.1: 6379> SUNION birds mammals
  1) "crow"
  2) "bat"
  3) "human"
  4) "pigeon"
  5) "dog"
  Redis 127.0.0.1: 6379> SDIFF birds mammals
  1) "crow"
  2) "pigeon"

• Ordered Set

  An ordered set has associated scores, and an operation similar to LRANGE can return ordered elements. The zrange zrevrange command can act on Ordered Sets. An ordered set is the equivalent of an index in the SQL world in Redis to some extent.

  Zadd hackers 1940 "Alan Kay"

  Zrange hackers 0-1

  Zrevrange hackers 0-1

  Zrangebyscore hackers-inf 1950 # obtain all persons born before January 1, 1950

  Zremrangebyscore hackers 1940 1960 # Delete hackers between 1940 and 1960.

• Hash type

  Redis can store key-to-multiple attribute data (such as user1.uname user1.passwd)
  Redis 127.0.0.1: 6379> HKEYS student
  1) "name"
  2) "age"
  3) "sex"
  Redis 127.0.0.1: 6379> HVALS student
  1) "Ganesh"
  2) "30"
  3) "Male"
  Redis 127.0.0.1: 6379> HGETALL student
  1) "name"
  2) "Ganesh"
  3) "age"
  4) "30"
  5) "sex"
  6) "Male"
  Redis 127.0.0.1: 6379> HDEL student sex
  (Integer) 1
  Redis 127.0.0.1: 6379> HGETALL student
  1) "name"
  2) "Ganesh"
  3) "age"
  4) "30"
  The Hash data structure can be modified and obtained in batches.
  Redis 127.0.0.1: 6379> HMSET kid name Akshi age 2 sex Female
  OK
  Redis 127.0.0.1: 6379> hmet kid name age sex
  1) "Akshi"
  2) "2"
  3) "Female"

  More refer to redis command http://redis.io/commands/