Memcache Knowledge Point Carding

Memcache Knowledge Point Carding

Memcached Concept:
Memcached is a free, open-source, high-performance, distributed-object caching system that can be used to hold a number of frequently accessed objects or data, and the saved data is like a huge hash table, which is in memory in key-value pairs.
Official website: http://www.memcached.org/
Operating environment: Linux,bsd,windows can run
Protocol theory: Http://code.sixapart.com/svn/memcached/trunk/server/doc/protocol.txt (English)

Operation Flow :

1, check whether the client's request data in the memcached, if any, directly return the request data, no longer do any operation on the database, the path operation is ①②③⑦.
2, if the requested data is not in memcached, go to the database, the data obtained from the database to the client, while the data cache a copy to the memcached (memcached client is not responsible, need the program explicitly implemented), the path operation is ①②④⑤⑦⑥.
3. Update the data in memcached every time the database is updated to ensure consistency.
4, when allocated to memcached memory space used up, will use the LRU (Least recently used, least recently used) policy plus expiration policy, the failure data is first replaced, and then replace the most recently unused data.

Memcached Features:
Simple protocol
It is a text line-based protocol that allows access to data operations directly via Telnet on the memcached server
Based on Libevent event processing
Libevent is a program library developed using C, which encapsulates the Epoll and other event processing functions of the BSD system's Kqueue,linux system into an interface that improves performance compared to traditional select.
Built-in memory management method
All data is stored in memory, access data faster than the hard disk, when the memory is full, the LRU algorithm automatically delete the unused cache, but does not consider the data disaster tolerance, restart the service, all data will be lost.
Distributed
Each memcached server does not communicate with each other, independently accesses data and does not share any information. The server does not have distributed functionality, and distributed deployment depends on the Memcache client.

Memcached Installation and Startup:
Installing memcached requires first installing libevent

1. Shell>tar zxvf libevent-1.4.14b-stable.tar.gz
2. Shell>CD libevent-1.4.14b-stable
3. Shell>./configure
4. Shell>make && make install

Installing memcached

1. Shell>tar zxvf Memcached-1.2.5.tar.tar
2. Shell>CD memcached-1.2.5
3. Shell>./configure–prefix=/usr/local/memcached
4. Shell>make && make install

Start memcached
Shell>/usr/local/memcached/bin/memcached–p 11211–d–u Root–p/tmp/memcached.pid

1. -P is the use of TCP, the default port is 11211
2. -D means a daemon is started in the background (daemon)
3. -U means that the specified root user is started and cannot be started by default with root
4. -P indicates the PID storage location of the process, where "P" is capitalized "P"
5. -L, followed by IP address, manually specify the listening IP address, the default all IP is listening
6. -M followed by the allocated memory size, in megabytes, defaults to 64M
7. -C Maximum number of running concurrent connections, default is 1024
8. -F Block size growth factor, default is 1.25
9. -M run out of memory and return an error instead of deleting an item, that is, without the LRU algorithm

In a 64-bit system, the Libevent-1.4.so.2 file cannot be found, the solution is to link the same name file in the 32-bit directory to the 64-bit directory, that is, like Windows to establish a shortcut.
Shell >/usr/local/lib/libevent-1.4.so.2/usr/lib64/libevent-1.4.so.2
If no port is detected after startup, it is because the "P" with PID parameter is uppercase "P" when the command is ordered, you may write in lowercase.

command line Direct manipulation commands

Saved, there are six command items.

1. Set: Add a new entry to memcached or replace existing entries with new data
2. ADD: When key does not exist, it saves data to memcached, otherwise, returns not_stored response
3. Replace: When the key is present, it will save the data to the memcached, otherwise return the not_stored response
4. Cas: Changes an existing key value, but it also has the function of checking
5. Append: Inserts a new value after this value
6. Prepend: Insert a new value before this value

There are two command entries:

1. Get: Take a single value, return data from the cache, the first row will be the name of the key, the value of the flag and the return value length, the real data in the second row, and finally return to end, such as key does not exist, the first line directly return to end
2. Get_multi: Fetching multiple values at once

Delete, a command:

1. Delete

Instance operations:
Connect to Memcache First

1. Shell>telnet 127.0.0.1 11211

In the Into data

1. Set Liuguohua 0 0 21
2. 369369.blog.51cto.com

The system automatically saves after carriage return and returns to normal stored
The format of the deposit command is
<command> <key> <flags> <exptime> <bytes> \ r \ n
Note that after you set the bytes byte, the length of the stored value must match, otherwise it cannot be saved successfully, like the Liuguohua field is 21 bytes, so the 369369.blog.51cto.com length is 21 (21 digits or letters in total).

Fetch data

1. Get Liuguohua

After carriage return, the first row returns value Liuguohua 0 21
The second line returns the True value 369369.blog.51cto.com

Memcached's memory algorithm:
Memcached uses the slab allocation mechanism to allocate and manage memory, which divides the allocated memory into chunks of memory of a certain length according to a predetermined size, then divides the same memory blocks into groups, and the data is stored, matching the slab size according to the size of the key value, Find the nearest slab storage, so there is space waste phenomenon.
The traditional way of memory management is to use the memory allocated through malloc to reclaim memory through free, which is prone to memory fragmentation and reduce the operating system's memory management efficiency.

Cache Policy for memcached:
The memcached cache policy is the LRU (least recently used) plus the expiry expiration policy. When you store data items in a memcached, you may specify that it will expire at cache time, and default to permanent. When the memcached server runs out of allocations, the invalidated data is replaced first, and then the data that is not used recently. In LRU, Memcached uses a lazy expiration policy that does not monitor the expiration of the key/vlue being deposited, but instead checks the timestamp of the record when it gets the key value, checking to see if Key/value is out of space, which reduces the load on the server.

Memcached's distributed algorithm:
When Key/value is deposited/removed to the memcached cluster, the memcached client program calculates which server to store on a certain algorithm, and then stores the Key/value value to this server. In other words, access to data in two steps, the first step, select the server, the second step to access data.

Distributed Algorithms (Consistent Hashing):

There are two options for choosing a server algorithm, one is to calculate the distribution based on the remainder, and the other is to calculate the distribution based on the hash algorithm.
Remainder algorithm:
The integer hash value of the key, divided by the number of servers, determines the access server based on the remainder, this method is simple and efficient, but when the memcached server is increased or decreased, almost all of the caches will fail.
Hash algorithm:
The hash value of the memcached server is calculated and distributed to a 32-square circle of 0 to 2, then the hash value of the key that stores the data is calculated in the same way and mapped to the circle, and finally from the location of the data map to a clockwise lookup, save the data to the first server found, If more than 2 of the 32 parties are still unable to find the server, the data is saved to the first memcached server. If you add a memcached server, the keys on the first server that only increase the counter-clockwise direction of the server on the circle will be affected.

memcache Management and performance monitoring :
Can be directly managed and monitored via the command line or through web software such as Nagios,cacti
Command line:

1. Shell>telnet 127.0.0.1 1211//If IP and port numbers are specified at startup, change them accordingly
   Command after successful connection
2. Stats: Statistics memcached of various information
3. Stats Reset: Re-statistical data
4. Stats Slabs, display slabs information, you can see in detail the data of the segmented storage situation
5. Stats items: Displays the number of item in slab
6. Stats cachedump 1 0: Lists the key values stored in the first paragraph of the slabs
7. Set|get: saving or retrieving data
8. STAT Evictions 0: Indicates the number of legitimate item moves to free up new space for new item

Other common software uses:

1. Shell>./memcached-tool 127.0.0.1:11211
2. Shell>./memcached-tool 127.0.0.1:11211 display

Web software:

1. memcache.php
2. Nagios Plugin
3. Cacti module

Memcached compared to Redis:

Memcache Knowledge Point Carding