Memcached completely dissect –1. The foundation of Memcached

Series Article Navigation:

Memcached completely dissect –1. The foundation of Memcached

memcached comprehensive analysis of –2. Understanding the memory storage of memcached

memcached comprehensive analysis of –3. The deletion mechanism and development direction of memcached

memcached comprehensive analysis of –4. The distributed algorithm of memcached

Memcached a comprehensive analysis of the. memcached Application and Compatibility program

Translation of a technical review of the article, is to talk about memcached serial. Fcicq classmate said this thing is very useful, I hope you like it.

Release Date: 2008/7/2
Nagano Masahiro (Masahiro Nagano)
Original link: http://gihyo.jp/dev/feature/01/memcached/0001

I am Nagano, development Department of Mixi Co., Ltd., operating group. Daily responsible for the operation of the program. Starting today, there will be several memcached on the most recent hot topic in the area of scalability for Web applications, along with the former Sakamoto of our research and Development group, explaining its internal structure and use.

What is memcached?

Memcached is a software developed by Brad Fitzpatric, a Danga interactive company in LiveJournal. It has become an important factor in improving Web application extensibility in many services such as Mixi, Hatena, Facebook, Vox, and LiveJournal.

Many Web applications save data to an RDBMS, where the application server reads the data and displays it in the browser. However, with the increase of data volume and the concentration of access, the burden of RDBMS, database response deterioration, site display delay and other significant impact.

This is the time to memcached. Memcached is a high-performance distributed memory cache server. The general purpose is to reduce the number of database accesses by caching database query results to improve the speed and scalability of dynamic Web applications.

Fig. 1 Use of memcached under normal circumstances

Characteristics of memcached

Memcached, as a distributed cache server running at high speed, has the following characteristics.

• Simple protocol
• Libevent-based event handling
• Built-in memory storage mode
• Memcached distributed without communication with each other

Simple protocol

memcached Server client communication does not use a format such as complex XML, but uses a simple text-line-based protocol. As a result, you can also save data and get data on memcached by using Telnet. Here is an example.

$ telnet localhost 11211
Trying 127.0.0.1 ...
Connected to Localhost.localdomain (127.0.0.1).
Escape character is ' ^] '.
Set Foo 0 0 3     (Save command)
Bar               (data)
STORED            (Result)
Get foo           (Get command)
VALUE Foo 0 3     (data)
Bar               (data)

The protocol document is located in the source code of the memcached, or you can refer to the following URL.

• Http://code.sixapart.com/svn/memcached/trunk/server/doc/protocol.txt

Libevent-based event handling

Libevent is a library that encapsulates event-handling functions such as Linux's Epoll, BSD-class operating system kqueue, and so on as a unified interface. The Performance of O (1) can be played even if the number of connections to the server increases. Memcached uses this libevent library, so it can perform its high performance on Linux, BSD, Solaris and other operating systems. About event handling is no longer detailed here, you can refer to Dan Kegel's c10k problem.

• libevent: http://www.monkey.org/~provos/libevent/
• The c10k problem: http://www.kegel.com/c10k.html

Built-in memory storage mode

To improve performance, the data saved in memcached is stored in Memcached's built-in memory storage space. Because the data exists only in memory, restarting the memcached and restarting the operating system will cause all data to disappear. Additionally, when the content capacity reaches the specified value, the unused cache is automatically deleted based on the LRU (Least recently used) algorithm. The memcached itself is a server designed for caching, so there is not too much consideration for permanent data issues. For more information on memory storage, the second part of this series will be introduced in the former Osaka, please refer to it at that time.

Memcached distributed without communication with each other

Memcached Although it is a "distributed" cache server, there is no distributed functionality on the server side. Each memcached does not communicate with each other to share information. So, how to distribute it? This depends entirely on the client's implementation. This series will also introduce the distribution of memcached.

Figure 2 memcached distributed

The following is a brief introduction to how memcached is used.

Installing memcached

Memcached installation is relatively simple, here a little explanation.

Memcached supports many platforms.

• Linux
• Freebsd
• Solaris (memcached 1.2.5 or later)
• Mac OS X

It can also be installed on Windows. This is illustrated using Fedora Core 8.

Installation of memcached

Running memcached requires the Libevent library that is described at the beginning of this article. Fedora 8 has a ready-made RPM package that can be installed with the Yum command.

$ sudo yum install libevent libevent-devel

Memcached source code can be downloaded from the memcached website. The latest version of this article is 1.2.5. Fedora 8 also contains the memcached rpm, but the version is older. Because the source code is not difficult to install, the RPM is not used here.

• download memcached: http://www.danga.com/memcached/download.bml

The memcached installation is the same as the general application, configure, make, and make install on the line.

$ wget http://www.danga.com/memcached/dist/memcached-1.2.5.tar.gz
$ tar zxf memcached-1.2.5.tar.gz
$ CD memcached-1.2.5
$./configure
$ make
$ sudo make install

By default, memcached is installed under/usr/local/bin.

memcached Start-up

Start memcached by entering the following command from the terminal.

$/usr/local/bin/memcached-p 11211-m 64M-VV
Slab class   1:chunk size     Perslab 11915
Slab class   2:chunk size    perslab  9362
Slab class   3:chunk size    144 Perslab  7281
Middle omitted
Slab class  38:chunk size 391224 perslab     2
Slab class  39:chunk size 489032 perslab     2
<23 Server Listening
<24 send buffer is 110592, now 268435456
<24 Server Listening (UDP)
<24 Server Listening (UDP)
<24 Server Listening (UDP)
<24 Server Listening (UDP)

The debugging information is shown here. This launches the memcached at the foreground, listening to TCP port 11211 with a maximum memory usage of 64M. The content of the debug information is mostly about the stored information, the next time the serial is specified.

When booting as a daemon background, simply

$/usr/local/bin/memcached-p 11211-m 64m-d

The contents of the Memcached startup options used here are as follows.

Options	Description
-P	The TCP port used. Default is 11211
-M	Maximum memory size. Default is 64M
-vv	Start with very vrebose mode, debug information and error output to console
-D	Start in the background as a daemon

The above four is a common startup option, and many others, through

$/usr/local/bin/memcached-h

Commands can be displayed. Many of the options can change the behavior of memcached and recommend reading.

Connect with Client

Many languages implement a client that connects Memcached, with Perl and PHP as the main. Only the languages listed on the memcached website are

• Perl
• Php
• Python
• Ruby
• C#
• C + +
• Lua

Wait a minute.

• memcached Client API: HTTP://WWW.DANGA.COM/MEMCACHED/APIS.BML

Here's how to link memcached through the Perl library that Mixi is using.

Using cache::memcached

Perl's memcached client has

• Cache::memcached
• Cache::memcached::fast
• Cache::memcached::libmemcached

Wait for several CPAN modules. The cache::memcached described here is Memcached's author Brad Fitzpatric's work, which should be considered the most widely used module in the memcached client.

• cache::memcached-search.cpan.org: http://search.cpan.org/dist/Cache-Memcached/

Connect memcached with cache::memcached

The following source code is an example of a memcached that was just started by cache::memcached connection.

#!/usr/bin/perl

Use strict;
Use warnings;
Use cache::memcached;

My $key = "Foo";
My $value = "bar";
my $expires = 3600; # 1 hour
My $memcached = Cache::memcached->new ({
    Servers = ["127.0.0.1:11211"],
    Compress_threshold = 10_000
});

$memcached->add ($key, $value, $expires);
My $ret = $memcached->get ($key);
print "$ret \ n";

Here, the IP address of the memcached server and an option are specified for cache::memcached to generate the instance. Cache::memcached common options are shown below.

Options	Description
Servers	specifying memcached servers and ports with arrays
Compress_threshold	Values to use when compressing data
Namespace	Specifies the prefix to add to the key

In addition, cache::memcached through the storable module can be the Perl complex data serialization and then save, so hash, array, object, etc. can be saved directly to memcached.

Save data

There are ways to save data to memcached.

• Add
• Replace
• Set

They are used in the same way:

My $add = $memcached->add (' key ', ' value ', ' term ');
My $replace = $memcached->replace (' key ', ' value ', ' term ');
My $set = $memcached->set (' key ', ' value ', ' term ');

You can specify a period of time (in seconds) when you save data to memcached. When you do not specify a period, Memcached saves the data according to the LRU algorithm. The differences between the three methods are as follows:

Options	Description
Add	Save only if there is no data with the same key in the storage space
Replace	Save only if there is data with the same key in the storage space
Set	Unlike add and replace, save whenever

Get Data

Get data can use the Get and Get_multi methods.

My $val = $memcached->get (' key ');
My $val = $memcached->get_multi (' Key 1 ', ' Key 2 ', ' Key 3 ', ' Key 4 ', ' key 5 ');

Use Get_multi when more than one data is obtained at a time. Get_multi can simultaneously obtain multiple key values synchronously, at a speed dozens of times times faster than a loop call get.

Delete data

Deleting data uses the Delete method, but it has a unique feature.

$memcached->delete (' key ', ' block time (s) ');

Deletes the data for the key specified by the first parameter. The second parameter specifies a time value that prevents new data from being saved with the same key. This feature can be used to prevent incomplete caching of data. Note, however, thatthe SET function ignores the blocking and saves the data as usual

Add one and subtract one operation

You can use a specific key value on the memcached as a counter.

My $ret = $memcached->incr (' key ');
$memcached->add (' key ', 0) unless defined $ret;

Increment one and minus one are atomic operations, but do not automatically assign 0 when the initial value is not set. Therefore, error checking should be performed and initialization should be added if necessary. Also, the server side does not check for behavior that exceeds 2³².

Summarize

This is a brief introduction to Memcached, its installation method, and the use of Perl client cache::memcached. As long as you know, the use of memcached is very simple enough.

The next time the memcached is explained by the former Sakamoto, the internal structure. Understanding the internal structure of memcached, you can know how to use memcached to make your Web application faster. Please continue reading the next chapter.

Memcached completely dissect –1. The foundation of Memcached