"Learning Notes" memcached basic knowledge

Source Address: http://kb.cnblogs.com/page/42731/

After careful study, the following is the note memo contents of the record.

What is memcached?
Memcached is a high-performance distributed memory cache server.
The purpose of general use is to reduce the number of database accesses by caching database query results to improve the speed and scalability of dynamic Web applications.

Characteristics of the memcached:
1. Simple protocol
2. Libevent-based event handling
3. Internal Memory storage mode
4. memcached distributed without communication with each other

The protocol is simple:
Memcached uses a simple line of text protocol, so telnet can also be used to save data on the memcached to obtain data.

Libevent-based event handling
Libevent is a library that encapsulates event-handling functions such as Linux's Epoll, BSD-like operating system kqueue, into a unified interface.
Memcached uses this libevent library to perform high performance on Linux, BSD and other operating systems.

Built-in memory storage mode
The data saved in memcached is stored in the memcached built-in memory storage space. Because the data is stored only in memory, restarting the memcached and restarting the operating system can result in the loss of all data. After the content capacity reaches the specified value, the unused cache is automatically deleted based on the LRU algorithm. The memcached itself is a server designed for caching, and therefore does not take into account the persistent problem of data.

Memcached distributed without communication with each other
Each memcahced does not communicate with each other. Shared information, how distributed, is implemented by the client

memcached Installation
Before installing memcahced, you first install the Libevent library

# Tar XF libevent-2.0. +-stable.tar.gz # CD Libevent-2.0. +            # ./configure--prefix=/usr/local/Libevent # make&&Make Install # echo"/usr/local/libevent/lib">/etc/ld.so.conf.d/libevent.conf # ldconfig # tar XF memcached-1.4. the. tar.gz # CD memcached-1.4. the            # ./configure--prefix=/usr/local/memcached--with-libevent=/usr/local/Libevent # make&& make Install

memcached when compiling the installation, start the script:

#!/bin/bash## Init File formemcached## chkconfig:- the  -# description:distributed Memory caching daemon## processname:memcached# config:/etc/sysconfig/memcached./etc/rc.d/init.d/functions## Default Variablesport="11211"USER="memcached"Maxconn="1024x768"CACHESIZE=" -"OPTIONS=""RETVAL=0Prog="/usr/local/memcached/bin/memcached"desc="Distributed Memory Caching"Lockfile="/var/lock/subsys/memcached"start () {echo-N $"starting $desc (memcached):"Daemon $prog-d-p $PORT-U $USER-C $MAXCONN-m $CACHESIZE $OPTIONS RETVAL=$?[$RETVAL-eq0] && Success && Touch $lockfile | |failure Echoreturn$RETVAL}stop () {echo-N $"shutting down $desc (memcached):"Killproc $prog RETVAL=$?[$RETVAL-eq0] && success && Rm-f $lockfile | |failure Echoreturn$RETVAL}restart () {Stop start}reload () {echo-N $"Reloading $desc ($prog):"Killproc $prog-HUP RETVAL=$?[$RETVAL-eq0] && Success | |failure Echoreturn$RETVAL} Case " $" inchstart) Start;  stop) stop;;  restart) restart;; Condrestart) [-E $lockfile] &&Restart RETVAL=$?        ;;  reload) reload;; Status) status $prog RETVAL=$?        ;; *) echo $"Usage: $ {start|stop|restart|condrestart|status}"RETVAL=1esacexit $RETVAL

Master configuration file:/etc/sysconfig/memcached PORT="11211"# memcached Port USER="memcached"# Start User Maxconn="1024x768"# Maximum number of connections CACHESIZE=" -"# Memory Cache space Size, units: M OPTIONS=""# Other Options memcached Client connection telnet ip:port save data

The methods for saving data to memcached are:
Add, replace, set
My $add = $memcached->add (' key ', ' value ', ' term ');
My $replace = $memcached->replace (' key ', ' value ', ' term ');
My $set = $memcached->set (' key ', ' value ', ' term ');
You can specify a period of time (in seconds) when you save data to memcached. Do not specify a period memcached the data is saved according to the LRU algorithm. The difference between the above three methods:
Add: Save only if there are no data with the same key in the storage space
Replace: Save only if there is data with the same key in the storage space
Set: Different from Add and replace, save whenever

Get Data

        get data using Get and Get_multi methods
         Use Get_multi when fetching multiple data at a time, can get more than one key value at the same time without synchronization, speed dozens of times times faster than loop call get
             My $val = $memcached->get (' key ');
            my $val = $memcached->get_multi (' Key 1 ', ' Key 2 ', ' Key 3 ', ' Key 4 ', ' Key 5 ');        

     Delete data

         Delete data using the Delete method

             $memcached->delete (' key ', ' Blocking time (seconds) ');

             Delete data from the first parameter specifying a key, and the second parameter specifies a time value that prevents the use of the same key to save new data. This feature can be used to prevent incomplete caching of data. Note: The Set function ignores the block, saving data as usual, adding one and minus one operation

Two, memcahced memory storage mechanism

Slab allocation mechanism: Defragment memory for reuse
The basic principle of the slab allocator is to divide the allocated memory into specific blocks in a predetermined size to completely resolve the memory fragmentation problem.
Slab Allocation divides the allocated memory into blocks (chunk) of various sizes and divides the same size blocks into groups (the collection of chunk)
Slab allocator can reuse allocated memory for the purpose. In other words, the allocated memory is not freed and reused.

Main terms of slab allocation
Page: Memory allocated to slab memory space, the default is 1MB, after assigning to slab according to the size of slab divided into chunk, a page is 4KB
Chunk: Memory space for Cache records
Slab class: Groups of specific size chunk

The principle of caching data in slab:
Memcached depending on the size of the data received, select the slab,memcached that best fits the data size to hold the list of chunk that are idle in slab, select chunk from the list, and then cache the data in it.

Disadvantages of Slab Allocator:
Slab allocator allocates memory of a specific length and therefore cannot effectively utilize allocated memory. For example: Cache 100 bytes of data to 128 bytes, and the remaining 28 bytes are wasted.

Tuning with growth factor

Memcached Specify the growth factor factor (option-f) At startup, you can control the difference between slab to some extent, by default 1.25

Set command: Memcached-f 2-VV

View the internal state of the memcahced

# telnet 127.0.0.1 11211
Stats

View the usage status of slabs
# Memcached-tool 127.0.0.1
# item_size Max_age Pages Count full? Evicted Evict_time OOM
1 96B 30s 1 1 No 0 0 0

The meanings:
# Slab Class Number
Item_size Chunk Size
Max_age the oldest recorded lifetime in the LRU
Pages assigned to Slab
Number of records within Count slab
Full? Whether the slab contains idle chunk

This part is not particularly complete, late additions, and this script is very handy for tuning.

Third, the deletion mechanism of memcached

memcached Efficient use of resources in data deduplication, data does not really disappear from memcached

Lazy Expiration
Memcached internally does not monitor whether the record is out of date, but instead looks at the timestamp of the record at get and checks whether the record is out of date. This technique is a lazy (lazy) expiration, so memcached does not consume CPU time on outdated monitoring

LRU: The principle of effectively deleting data from the cache
When memcached has insufficient memory space, it searches for records that have not been used recently and allocates their space to new records. New records are overwritten with the fewest chunk
# Memcached-m No LRU, in general it is highly recommended to use LRU

Iv. distributed algorithms for memcached

Memcached's distributed
Memcached's distribution is implemented by the client library, which is the most memcached feature.

Set Method:
After first adding ' Tokyo ' to Memcached ' Tokyo ' to the client library, the client-implemented algorithm determines the memcached server that holds the data based on the ' key '.

Get method:
The key "Tokyo" To get is also passed to the library. The function library selects the server according to the "key" by the same algorithm as when the data is saved. Using the same algorithm, you can select the same server as you saved, and then send a GET command. As long as the data is not deleted for some reason, the saved value can be obtained.

A distributed approach to cache::memcached
There are two main methods of Hash-type:
1. Modulo method: This algorithm is not suitable for memcached distributed, when a memcached server fails, it will cause all memcached cache invalidation.
2. Consistent hash algorithm: first find the hash value of the memcached server, and configure it to the 1-2^32 circle, and then use the same method to find the key to store the hash value of the data, and map to the circle, and then from the data map to the position of the clockwise lookup, Save the data to the first server found.

"Learning Notes" memcached basic knowledge