memcached Depth Analysis _php tutorial

Memcached is a set of distributed memory object caching systems developed by danga.com (the technical team of operations LiveJournal) to reduce database load and improve performance in dynamic systems. About this thing, I believe a lot of people have used, this article is intended through the implementation of memcached and code analysis, get a better understanding of this excellent open source software, and can be based on our needs to further optimize it. Finally, the analysis of the bsm_memcache extension will deepen the understanding of how memcached is used.

Some of the content of this article may require a better mathematical basis as an aid.

What is memcached?

Before we elaborate on this issue, we must first understand that it is "not what". Many people use it as a storage carrier of the same form as sharedmemory, although memcached uses the same "key=>value" to organize data, but it differs greatly from local caches such as shared memory and APC. The memcached is distributed, meaning it is not local. It is based on a network connection (and of course it can also use localhost) to complete the service itself, which is an application-independent program or daemon (daemon mode).

Memcached uses the Libevent library to implement network connectivity services, theoretically can handle an unlimited number of connections, but it is different from Apache, it is more time for stable continuous connection, so its actual concurrency capacity is limited. In a conservative case, the maximum number of simultaneous connections for memcached is 200, which is related to the Linux threading capability, which can be adjusted. Refer to the relevant documentation for Libevent. Memcached memory usage also differs from APC. APC is based on shared memory and Mmap, MEMCACHD has its own memory allocation algorithm and management mode, it is not related to shared memory, there is no shared memory limit, usually, each memcached process can manage 2GB of memory space, if more space is needed, You can increase the number of processes.

Memcached Suitable for what occasion

In many cases, memcached have been abused, which is of course a complaint. I often see a post on the forum, similar to "How to improve efficiency", reply is "with memcached", as to how to use, where, used to do not have a sentence. Memcached is not omnipotent and it is not applicable on all occasions.

Memcached is a "distributed" Memory object caching system, that is, those that do not need to be "distributed", do not need to be shared, or simply small to a single server application, memcached will not bring any benefits, but also slow down the system efficiency, because the network connection also requires resources, Even UNIX local connections are the same. As shown in my previous test data, the memcached local read and write speed is dozens of times times slower than the direct PHP memory array, and the APC, shared memory mode is almost the same as the direct array. It can be seen that the use of memcached is very uneconomical if it is just a local-level cache.

Memcached is often used as a database front-end cache. Because it is much less expensive than database SQL parsing, disk operations, and it is using memory to manage the data, it can provide better performance than directly read the database, in large systems, access to the same data is very frequent, memcached can greatly reduce the database pressure, Improve the efficiency of system execution. In addition, memcached is often used as a storage medium for data sharing between servers, such as storing data in the SSO system as a single-point login, which can be saved in memcached and shared by multiple applications.

It is important to note that Memcached uses memory to manage data, so it is volatile, when the server restarts, or the memcached process aborts, the data is lost, so memcached cannot be used to persist the data. Many people's misunderstanding, memcached performance is very good, good to the memory and hard disk contrast, in fact, memcached use memory does not get hundreds of read and write speed, its actual bottleneck is the network connection, it and the use of disk database system, the advantage is that it itself very "light "Because there is not too much overhead and direct reading and writing, it can easily cope with very large amounts of data exchange, so there are often two gigabit network bandwidth is full load, memcached process itself does not occupy much CPU resources.

How the memcached works

In the following sections, it is best to prepare a copy of the memcached source code.

Memcached is a traditional network service program, and if you use the-D parameter when you start it, it executes as a daemon. The creation daemon is done by DAEMON.C, which has only one daemon function, which is very simple (if no special instructions, the Code is 1.2.1):

Code:[copy to clipboard] #include
#include
#include

Int
Daemon (Nochdir, Noclose)
int Nochdir, noclose;
{
int FD;

Switch (fork ()) {
Case-1:
Return (-1);
Case 0:
Break
Default
_exit (0);
}

if (setsid () = =-1)
Return (-1);

if (!nochdir)
(void) ChDir ("/");

if (!noclose && (fd = open ("/dev/null", O_rdwr, 0))! =-1) {
(void) dup2 (FD, Stdin_fileno);
(void) dup2 (FD, Stdout_fileno);
(void) dup2 (FD, Stderr_fileno);
if (fd > Stderr_fileno)
(void) Close (FD);
}
return (0);
}
After the function has forked the entire process, the parent process exits, then the STDIN, STDOUT, STDERR to the empty device are relocated, and the daemon is established successfully.

The Memcached itself starts with the following sequence in the main function of MEMCACHED.C:

1. Call Settings_init () to set initialization parameters
2. Set the setting value by reading the parameters from the start command
3. Set LIMIT parameter
4. Start Network socket Monitoring (if non-socketpath exists) (UDP is supported after 1.2)
5, check the user identity (Memcached not allow ROOT to start)
6. If a socketpath exists, open UNIX Local connection (Sock pipeline)
7. If you start with-D, create a daemon (call the daemon function above)
8. Initialize item, event, status information, hash, connection, slab
9. Create bucket array If managed is active in Settings
10. Check if you need to lock memory pages
11. Initialize signal, connect, delete queue
12, if the daemon way, processing process ID
13, the event starts, the START process ends, the main function enters the loop.

In daemon mode, because stderr has been directed to a black hole, there is no feedback on the visible error message in the execution.

The main loop function of the MEMCACHED.C is Drive_machine, which is the structure pointer to the current connection, and determines the action based on the state of its member.

Memcached uses a set of custom protocols to complete the exchange of data, its protocol documentation can be consulted: http://code.sixapart.com/svn/memcached/trunk/server/doc/protocol.txt

In the API, the line break symbol is unified

Memcached Memory Management method

Memcached has a very distinctive memory management approach, in order to improve efficiency, it uses pre-application and grouping to manage the memory space, not every time you need to write data to malloc, delete data when the free pointer. Memcached manages memory using Slab->chunk's organizational approach.

There are some differences between the slab space partitioning algorithms in SLABS.C 1.1 and 1.2, which are described separately later.

Slab can be understood as a block of memory, a slab is memcached to request the smallest unit of memory, in memcached, the size of a slab by default is 1048576 bytes (1MB), so memcached is the whole MB of memory use. Each slab is divided into several chunk, each of which holds an item and each item contains the item struct, key, and value (note that value in memcached is only a string). Slab the list according to their own ID, these linked lists are linked by ID on a slabclass array, the entire structure looks a bit like a two-dimensional array. The length of the Slabclass is 21 in 1.1 and 200 in 1.2.

Slab has an initial chunk size, 1.1 is 1 bytes, 1.2 is 80 bytes, 1.2 has a factor value, and the default is 1.25

In 1.1, the chunk size is expressed as the initial size *2^n,n is ClassID, that is: Slab with id 0, 1 bytes per chunk size, slab with id 1, 2 bytes per chunk size, slab with ID 2, 4 bytes per chunk size ... An ID of 20 slab, each chunk size is 1MB, that is, the ID of 20 slab only one chunk:

Code:[copy to Clipboard]void slabs_init (size_t limit) {
int i;
int size=1;

Mem_limit = limit;
for (i=0; i<=power_largest; i++, size*=2) {
slabclass[i].size = size;
Slabclass[i].perslab = power_block/size;
slabclass[i].slots = 0;
Slabclass[i].sl_curr = slabclass[i].sl_total = Slabclass[i].slabs = 0;
slabclass[i].end_page_ptr = 0;
Slabclass[i].end_page_free = 0;
slabclass[i].slab_list = 0;
slabclass[i].list_size = 0;
slabclass[i].killing = 0;
}

/* For the test su

http://www.bkjia.com/PHPjc/508220.html www.bkjia.com true http://www.bkjia.com/PHPjc/508220.html techarticle memcached is a set of distributed memory object caching systems developed by danga.com (the technical team of operations LiveJournal) to reduce database load and improve performance in dynamic systems. About this ...

