Memcached is used in large web sites

Memcached is a high-performance distributed memory object caching system that can be used to store data in a variety of formats, including images, videos, files, and the results of database retrieval, by maintaining a large, unified hash table in memory. Originally developed to speed up LiveJournal access, it was later adopted by many large Web sites. At first the author wrote it may be to improve the dynamic Web application, in order to reduce the pressure of database retrieval, to do this caching system. Its caching is a kind of distributed, that is, can allow multiple users on different hosts to access this caching system, this method not only solves the problem of shared memory can only be a single machine, but also solves the pressure of database retrieval, the biggest advantage is to improve access to obtain data speed. Based on the memcached author's understanding of distributed cache and solutions. Memcached can be used in other areas such as distributed database, distributed computing and so on.

1, memcached Agreement understanding
Memcache is a project that was born to speed up http://www.livejournal.com/access.
Its official homepage is: http://www.danga.com/memcached/
Working mechanism:

By opening an area in memory to maintain a large hash table to speed up page access, and the database is independent. But it is now primarily used to cache data from the database. Allow multiple servers to form a large hash over the network, users do not have to care about where the data is stored, only the relevant interface can be invoked. The data stored in memory is eliminated out of memory through the LRU algorithm. At the same time can be deleted and set the expiration time to eliminate the data stored in memory.

2. Getting Started with memcached

Installation of 2.1 memcached

Installation of the <1>memcached service
To check the Linux kernel version first, it is recommended to install memcached above 2.6.
Because memcached need to use libevent and Epoll.
Before installing memcached, make sure your server has a libevent library installed on it.
Libevent Download Address (http://www.monkey.org/~provos/libevent/).
Download the memcached source code (HTTP://WWW.DANGA.COM/MEMCACHED/DOWNLOAD.BML).
Memcached was originally written in Perl, and the current version is written in C.
Copy to a directory after downloading, installation requires root user to perform
TAR-ZXVF memcached-1.1.12.tar.gz
CD memcached-1.1.12
./configure
This has to be configure, it detects your system, then generates a Config.h file and several other files, and as with other configure, you can configure its installation path and so on. The default application is installed in the/usr/local/bin directory.
Make//Compile
Make Install//installation

Installation of <2>memcached Client
According to the Memcached protocol, users can write their own client programs that meet their requirements. Current HTTP://WWW.DANGA.COM/MEMCACHED/DOWNLOAD.BML
Provide perl,c,java,python,php and other client programs for download and reference. I'll take the Perl client program as an example to illustrate the installation of the client:
Copy to a directory after downloading, installation requires root user to perform
TAR-ZXVF cache-memcached-1.14.tar.gz
CD cache-memcached-1.14
Perl makefile.pl
Make
Make install
Make Test
This allows the memcahced to be installed and the distributed caching system can be used to start the memcached.

2.2 Quick Start
Start of <1> memcached service
Memcached startup is very simple, it does not have a configuration file, as long as the configuration of several parameters can be used. Let me take a practical example to illustrate the following:
Memcached–d–m 500-l 64.128.191.151-p 11211-vv >>/var/www/kelly/test/logs/memcached_$$.log
This memcached is a daemon mode (-D), and then the cached space is 500M (-m), the Listener (-L) Server 64.128.191.15 port 11,211th (-P)., wrote the log/var/www/kelly/test/ Logs/memcached_$$.log (-VV).
In fact, the memcached parameters are also very limited, on the following several:
? -p port number to listen on
? -L interface to listen on, the default is Indrr_any
? -D Run as a daemon
? -R Maximize Core file limit
? -U assume identity of (only when run as root)
? -M max memory to use for items in megabytes, default is MB
? -M return error on memory exhausted (rather than removing items)
? -C Max simultaneous connections, default is 1024
? -K Lock down all paged memory
? -V Verbose (print errors/warnings while in event loop)
? -VV very verbose (also print client commands/reponses)
? -H Print this help and exit
? -I print memcached and libevent license
We can also write this startup script/etc/rc.d or/erc/rc.local, which can be executed at server startup.

<2> memcached Client Connections
I'll take the Perl client program as an example to illustrate the client connection:
Start two memcached server
Memcached–d–m 500-l 64.128.191.151-p 11211-vv >>/var/www/kelly/test/logs/memcached_$$.log
Memcached–d–m 500-l 64.128.191.151-p 11212-vv >>/var/www/kelly/test/logs/memcached_$$.log
Perl Client Programs
#!/usr/bin/perl
Use cache::memcached;
My $memd = new Cache::memcached {
' Servers ' => ["64.128.191.15:11211", "64.128.191.15:11212"],
};
My $val = $memd->get ("My_key");
if ($val)
{
print ' Value is ' $val '/n ';
}
# Set A value
$memd->set ("My_key", "123");
$memd->disconnect_all ();
? Run the Test
$ Perl test-memcache.pl
$ Perl test-memcache.pl
Value is ' 123 '
You can see that the first time you didn't get My_key, the second time you get the My_key value from the memcached.
Also, by looking at the logs, you can see that they are stored in two memcache servers.
This simple example explains how to access data in memcached, and that Memcache is a true distributed caching system.
Of course, this is just a very simple example, does not reflect the advantages of memcache, the following will be a very specific example, give a detailed application.

3, the application of memcached in Zorpia
Http://www.zorpia.com is a large web site company with Web albums, blogs, personals, and forums. There are now more than 1.4 million active users across the United States, Hong Kong, Southeast Asia, Europe, Australia, Asia and other regions. Daily visits are growing, and have become the fifth most-ranked social network in the world.
Memcached also used memcached to improve the speed of the Web site, and achieved good results, I was responsible for the Zorpia memcached project time accumulated some experience, the main practice is as follows:
1 Customize your client by wrapping the Memcache perl client.
2) by establishing a hash key naming code conforming to the ZORPIA specification
? ? The keys for the content to be stored in the memcache are composed of strings.
This string unification is implemented by a MEMCACHE.PM subroutine. (assuming this subroutine is Get_key ())
? ? Two types of data stored in Memcache
(1) Result of SQL query:
(2) ordinary variable (variable)
The key combinations of the two data are not the same, and are judged and completed by Get_key
? ? About Get_key and naming rule
Get_key subroutine completes all memcache key names, and naming rule is embodied in it:
(1) Input parameters--hash structure, which defines the current data to be stored in the information
Structure
(2) Return value-string, returns the Key_name of the data
? You must determine the structure of the incoming hash of the Get_key,
There are two main elements in the hash
Type---Define the types of current data structures, with ' var ', ' SQL ' values
Object---Store the details of the current data structure,
When type eq ' var ', object represents the name of the variable, which is specified by the programmer
When type eq ' SQL ', object contains the main basic information of the stored SQL, the hash structure, also by the programmer in accordance with the rules
# # When variable data type, relatively simple
$var _hash = {
Type => ' var ', # # var indicates that the current type is variable
Object => ' language ', # # language represents variable's name
};
The generated key is zorpia::var| Language
# # SQL data
For example, select first_name from user where user_id = 2 so hash is
$sql _hash = {
Type => ' SQL ',
Object => {
Table => {table2=> "user",}, # # SQL query
Column => {column1=> "First_Name",}, # # SQL query column
Condition => {user_id => "2",}, # # SQL condition
},
};
The generated key is zorpia::sql|user|first_name| USER_ID =2
Get_key subroutine must judge the incoming hash, combine different types of data in different ways, form key, and return it to the user. This key must ensure its uniqueness:
For example: All letters lowercase, some arrays must be sorted before grouping into key
? ? Get_key function
Sub get_key{
my $hash = shift;
return undef unless $hash && ref $hash eq "hash";
My $type = $hash->{type};
My $key _name;
if ($type eq ' sql ') {
My ($table _key, $column _key, $condition _key);
$table _key=_get_key ($hash->{object}->{table});
$column _key=_get_key ($hash->{object}->{column});
$condition _key=_get_key ($hash->{object}->{condition});
$key _name = Join (' | ', $type, $table _key, $column _key, $condition _key);
#Currently the length limit of a key is set at characters
if (Length ($key _name) >250)
{
$key _name=substr (0,250, $key _name);
}
}
elsif ($type eq ' var ')
{
$key _name = Join (' | ', $type, $hash->{object});
}
return $key _name;
}
Sub _get_key
{
My $hash =shift;
return undef unless $hash && ref $hash eq "hash";
My ($t, $ret, $i);
foreach $i (sort keys% $hash)
{
$i =~s/^/s+|/s+$//g;
$hash->{$i}=~s/^/s+|/s+$//g;
Push (@ $t, LC ("$i = $hash->{$i}"));
}
$ret =join (': ', sort {$a CMP $b} @ $t);
return $ret;
}
3 Develop rules that require application of memcached
? frequently visited table User,user_details
? reasonable setting of variable in memcached life cycle
? pre-import information from active users to memcached
? start multiple memcached services on multiple machines separately
? Write a script to monitor whether the memcached service is active
4 The user table's specific application examples
? At the Select time
First query Memcahce There is no, there are words, return, otherwise from the database Select, in the Memcache set, return.
My $sql _hash = {
Type => ' SQL ',
Object => {
Table => {table1=> "user",},
Column => {column1=> "user_id",},
Condition => {email=> $user _id,},
},
};
My $key =zorpia::memcache::get_key ($sql _hash);
My $user _id_by_email=zorpia::memcache::get ($key);
if (! $user _id_by_email)
{
My $sth;
My $query = "Select user_id from user where email=?";
$sth = $dbh->prepare ($query);
$sth->execute ($user _id);
My $user 1 = $sth->fetchrow_hashref ();
$user _id_by_email= $user 1->{' user_id '};
Zorpia::memcache::set ($key, $user _id_by_email,1800);
}
? at update,insert,delete Time
First in the database Update,insert,delete, in the Memcache set, returned.
&zorpia::D B::d Ata_entry_no_return ($DBH, "User", "COUNT (*)", "$account _information_insert_statement user_id=$ current_user_id "," user_id= $current _user_id ");
#add by Kelly
My $sql _hash = {
Type => ' SQL ',
Object => {
Table => {table1=> "user",},
Column => {column1=> "user_id",},
Condition => {user_id=> $current _user_id,},
},
};
My $key =zorpia::memcache::get_key ($sql _hash);
My $query = "SELECT *, user_id as ID from user WHERE user_id=?";
My $sth _MEMC = $dbh->prepare ($query);
$sth _memc->execute ($current _user_id);
My $user _MEMC = $sth _memc->fetchrow_hashref ();
&zorpia::memcache::set ($key, $user _memc,21600);

4, memcached's application Outlook
after using memcached, I've found that many of the projects I've done in the past can be used to improve efficiency, including recent "Big single tracking", "digital search" and so on. Of course, since memcahced is a distributed caching system, it is a distributed platform that we can use to carry out the distributed count, because for a key value key we can set its numeric value and the validity period in the parameter, and reset the value of the key. So I summed up the current can be applied to the place:
<1> Database retrieval results of the cache, that is, can be organically combined with the database application, improve efficiency.
This is the most commonly used memcached, such as for large web sites.
You can do this:
Open the memcached server connection
write the SQL statement while calculating its hash key value
Gets the hash value memcached save data (GET)
If the data obtained for this hash value exists. Return
Otherwise connect database lookup
To save this lookup result in memcached (set), you can set the validity period
return find results
<2> distributed computing
<3>. Distributed shared data
anyway, Memcached's mechanism is more flexible, it can be applied to all the places that need distributed caching data, and as memcached is gradually known, it will shine in more distributed application fields.