Installing and using memcached

How to incorporate memcached into your environment.

Before you begin to install and use the using memcached, we need to know how to incorporate memcached into your environment. Although memcached can be used anywhere, I find that memcached often works best when several recurring queries need to be executed in the database layer. I often set up a series of memcached instances between the database and the application server and use a simple pattern to read and write to these servers. Figure 1 can help you understand how to set up your application architecture:

Figure 1. Sample application Architecture using memcached

The architecture is fairly easy to understand. I've built a Web tier that includes some Apache instances. The next layer is the application itself. This layer typically runs on Apache Tomcat or other open source application servers. The next layer is where the memcached instance is configured-that is, between the application server and the database server. When you use this configuration, you need to perform read and write operations on the database in a slightly different way.

Read

The order in which I perform the read operation is to fetch the request from the Web tier (which requires a database query) and examine the results of the query that was previously stored in the cache. If I find the value I want, I'll return it. If it is not found, the query is executed and the results are stored in the cache before the results are returned to the Web tier.

Write

When you write data to a database, you first need to perform a database write operation, and then set any previously cached results that are affected by this write operation to be invalid. This process helps prevent data inconsistencies between the cache and the database.

Installing memcached

Memcached supports some operating systems, including Linux®, Windows®, Mac OS, and Solaris. In this article, I'll explain in detail how to build and install memcached from a source file. The main reason for this is that I can view the source code when I'm having problems.

libevent

Libevent is the only prerequisite for installing memcached. It is the asynchronous event notification library that memcached relies on. You can find the source file on monkey.org on the libevent. Next, find the latest version of the source file. For this article, we use the stable version 1.4.11. Once the archive has been obtained, unzip it to a convenient location and then execute the command in Listing 1:

Listing 1: Building and installing Libevent

CD libevent-1.4.11-stable/ ./configure make do install

memcached

Obtain the memcached source file from Danga Interactive, and still select the latest distribution version. At the time of writing this article, the latest version is 1.4.0. Unzip the tar.gz to a convenient location and execute the command in Listing 2:

Listing 2: Building and installing memcached

CD memcached-1.4.0/ ./configure make do install

After you complete these steps, you should have a memcached working copy installed, and you can use it. Let's take a brief introduction and then use it.

Using memcached

To get started with memcached, you first need to start the memcached server and then connect to it using a Telnet client.

To start memcached, execute the command in Listing 3:

Listing 3. Starting memcached

./memcached-d-M 2048-l 10.0.0.40-p 11211

This starts memcached (-D) as a daemon, allocates 2GB of memory (-M 2048), and specifies a listening localhost, port 11211. You can modify these values as needed, but the above settings are sufficient to complete the exercises in this article. Next, you need to connect to memcached. You will use a simple Telnet client to connect to the memcached server.

Most operating systems provide a built-in Telnet client, but if you are using a Windows-based operating system, you need to download a third-party client. I recommend using PuTTy.

After the Telnet client is installed, execute the command in Listing 4:

listing 4. Connecting to memcached

telnet localhost 11211

If everything is OK, you should get a telnet response that will indicate Connected to localhost (already connected to localhost). If you do not get this response, you should return to the previous steps and ensure that the source files for both libevent and memcached have been successfully generated.

You are now logged on to the memcached server. Thereafter, you will be able to communicate with memcached through a series of simple commands. 9 memcached Client commands can be divided into three categories: basic advanced Management

basic memcached Client Commands

You will perform the simplest operation with five basic memcached commands. These commands and actions include: Set add replace get delete

The first three commands are standard modification commands for manipulating key-value pairs stored in memcached. They are both very easy to use and are all using the syntax shown in Listing 5:

Listing 5: Modifying command syntax

Command <key> <flags> <expiration time> <bytes> <value>

Table 1 defines the parameters and usages of the memcached Modify command.

table 1. memcached Modifying command Parameters

Parameters	usage
Key	Key to find cached values
Flags	You can include integer parameters for key-value pairs that the client uses to store additional information about key-value pairs
Expiration time	The length of time (in seconds, 0 for forever) that a key-value pair is saved in the cache
bytes	The byte points stored in the cache
Value	Stored values (always on the second row)

Now, let's take a look at the actual use of these commands.

Set
The set command is used to add a new key-value pair to the cache. If the key already exists, the previous value will be replaced.

Note the following interaction, which uses the SET command:

Set userId 0 0 5 12345 STORED

If a key-value pair is correctly set using the SET command, the server responds with the word STORED . This example adds a key-value pair to the cache with a key of userId and a value of 12345. and set the expiration time to 0, which notifies memcached that you want to store this value in the cache until you delete it.

Add
The add command adds a key-value pair to the cache only if the key does not exist in the cache. If the key already exists in the cache, the previous value will remain the same, and you will get a response of not_stored.

The following are standard interactions using the Add command:

Set userId 0 0 5 12345 STORED add userId 0 0 5 55555 not_stored add companyid 0 0 3 564 STORED

Replace
The Replace command replaces the key in the cache only if the key already exists. If the key does not exist in the cache, you will receive a not_stored response from the memcached server.

The following are standard interactions using the Replace command:

Replace AccountId 0 0 5 67890 not_stored set accountId 0 0 5 67890 STORED Replace AccountId 0 0 5 55555 STORED

The last two basic commands are get and delete. These commands are fairly easy to understand and use a similar syntax, as follows:

Command <key>

Next look at the application of these commands.

Get
The GET command is used to retrieve the value associated with the previously added key-value pair. You will use get to perform most of the retrieval operations.

Here is a typical interaction using the GET command:

Set userId 0 0 5 12345 STORED get userid VALUE userid 0 5 12345 end get Bob end

As you can see, the get command is fairly straightforward. You use a key to invoke get, and if the key exists in the cache, the corresponding value is returned. If it does not exist, nothing is returned.

Delete
The last basic command is delete. The Delete command deletes any existing values in the memcached. You will use a key to call Delete, and if the key exists in the cache, delete the value. If it does not exist, a not_found message is returned.

Here is the client server interaction using the Delete command:

Set userId 0 0 5 98765 STORED Delete Bob not_found Delete userid DELETED get userid END

Advanced memcached Client Commands

The two advanced commands that can be used in memcached are get and CAs. The gets and CAS commands need to be used in conjunction. You will use these two commands to ensure that the existing name/value pairs are not set to the new value if the value has been updated. Let's take a look at these orders separately.

gets
The get command functions like a basic gets command. The difference between the two commands is that gets returns slightly more information: the 64-bit integer value is very much like the "version" identifier of a name/value pair.

Here is the client server interaction using the gets command:

Set userId 0 0 5 12345 STORED get userid VALUE userid 0 5 12345 END gets UserID 4 12345 END

Consider the difference between the get and gets commands. The gets command returns an extra value-in this case, an integer value of 4, which identifies a name/value pair. If another set command is executed on this name/value pair, the extra value returned by the gets will change to indicate that the name/value pair has been updated. Listing 6 shows an example:

listing 6. Set Update Version indicator

Set userId 0 0 5 33333 STORED gets userid 5 33333 END

Did you see the value returned by the gets? It has been updated to 5. Each time you modify a name/value pair, the value changes.

CAs
CAS (check and set) is a very handy memcached command that sets the value of a name/value pair (if the name/value pair has not been updated since you last performed the gets). It uses a syntax similar to the SET command, but includes an extra value: The extra value returned by the gets.

Note the following interactions that use the CAS command:

Set userId 0 0 5 55555 STORED gets userId 6 55555 END 6 33333 STORED

As you can see, I call the gets command with an extra integer value 6来, and the operation runs in a very sequential order. Now let's take a look at the series of commands in Listing 7:

Listing 7. CAS commands with older version indicators

Set userId 0 0 5 55555 STORED gets userId 8 55555 END 6 33333 EXISTS

Note that I did not use the integer value returned by the gets most recently, and the CAS command returned a EXISTS value in order to fail. In essence, using the Get and CAS commands at the same time prevents you from using name/value pairs that have been updated since the last read.

Cache Management Commands

The last two memcached commands are used to monitor and clean memcached instances. They are stats and Flush_all commands.

Stats
The stats command functions as its name: the current statistics of the memcached instance to which the dump is connected. In the following example, the Execute Stats command displays information about the current memcached instance:

Stats stat pid uptime 101758 stat time 1248643186 stat version 1.4.11 STAT pointer_size 32< C6/>stat rusage_user 1.177192 stat rusage_system 2.365370 stat curr_items 2 stat total_items 8 Stat Bytes 119 Stat curr_connections 6 stat total_connections 7 stat connection_structures 7 stat cmd_ Get cmd_set stat get_hits get_misses 0 stat Evictions 0 stat bytes_read 471 stat Bytes_written 535 stat limit_maxbytes 67108864 STAT Threads 4 END

Most of the output here is very easy to understand. Later, when we discuss caching performance, I'll explain in detail what these values mean. For now, let's take a look at the output and then use the new key to run some set commands and run the stats command again to notice what's changed.

Flush_all
Flush_all is the last command to be introduced. This simplest command is only used to clean up all name/value pairs in the cache. If you need to reset the cache to a clean state, flush_all can be of great use. Here is an example of using Flush_all:

Set userId 0 0 5 55555 STORED get userid VALUE userid 0 5 55555 END flush_all OK Get UserId END

Cache Performance

At the end of this article, I'll discuss how to use the Advanced memcached command to determine the performance of the cache. The stats command is used for tuning cache usage. The two most important statistics to note are et_hits and get_misses. These two values indicate the number of times the name/value pair was found (get_hits) and the number of name/value pairs that were not found (get_misses).

By combining these values, we can determine how the cache is utilized. When you start the cache for the first time, you can see that get_misses will naturally increase, but after a certain amount of usage, these get_misses values should gradually stabilize-this means that the cache is primarily used for common read operations. If you see get_misses continue to increase rapidly, and get_hits gradually become smooth, then you need to determine what the cached content is. You may have cached the wrong content.

Another way to determine cache efficiency is to view the cache hit ratio. The cache Hit ratio indicates the percentage of times the get was executed and the number of missed get. To determine this percentage, you need to run the stats command again, as shown in Listing 8:

listing 8. Calculating Cache Hit Ratio

Stats stat pid 6825 stat uptime 540692 stat time 1249252262 stat version 1.2.6 stat pointer_size 32< C7/>stat rusage_user 0.056003 stat rusage_system 0.180011 stat curr_items 595 stat total_items 961 Stat bytes 4587415 stat curr_connections 3 stat total_connections stat connection_structures 4 Stat cmd_get 2688 stat cmd_set 961 stat get_hits 1908 stat get_misses 780 stat Evictions 0 Stat Bytes_read 5770762 stat bytes_written 7421373 stat limit_maxbytes 536870912 STAT threads 1 END

Now, divide the value of get_hits by Cmd_gets. In this case, your hit rate is about 71%. Ideally, you might want to get a higher percentage-the higher the ratio, the better. Viewing statistics and measuring them from time to times can be a good way to determine the efficiency of your caching strategy.

Commands are often as follows:

Start/End
Memcached-d-M 10-u root-l 192.168.0.122-p 11200-c 256-p/tmp/memcached.pid
The-D option is to start a daemon,
-M is the amount of memory allocated to Memcache, in megabytes, 10MB
-U is the user running memcache, this is root
-L is the server IP address of the listener, if there are multiple addresses, this specifies the IP address of the server 192.168.0.122
-P is the port that sets Memcache listening, which is set to 12000, preferably more than 1024 ports
The-c option is the maximum number of concurrent connections, the default is 1024, where 256 is set according to the load on your server
-P is a PID file that is set to save Memcache
Kill ' Cat/tmp/memcached.pid '

Get Run status
Echo Stats | NC 192.168.1.123 11200
Watch "Echo Stats | NC 192.168.1.123 11200 "(real-time status)

Reference: http://www.czhphp.com/archives/252