Java XMemcached usage and source code details, javaxmemcached

Java XMemcached usage and source code details, javaxmemcached
For more information, see http://blog.csdn.net/tang9140/article/details/4344520.preface this article describes how to use the XMemcached client to interact with the Memcached server. Comparing the XMemcached API call with the Memcached set/get command and tracking the XMemcached source code, we can have a deeper understanding of the XMemcached API and understand its working principle from the underlying layer, in this way, some targeted interface closures and optimization work can be carried out in the project.
Is it Memcache or Memcached?

There is a saying on the Internet that Memcache is the name of this project, and memcached is the name of the main program on its server. I checked the official website of Memcache at http://memcached.org/. memcachedis used directly on the homepage. Whatever the name is, it is called Memcached, which only represents my personal habits.

Memcached is a distributed, high-performance, memory-level object cache system and an open-source free project. All its key-value data is stored in the memory, which is one reason for its efficiency. It also means that all data will be lost when the system is disabled. Memcached can be used as a cache system to reduce the number of queries to dynamic website databases and improve website performance. It is often used as a cache solution for Web websites. The Memcached client supports APIs in multiple languages, such as C/C ++, Perl, PHP, Java, C #, and Ruby.

Memcached Java client currently has three

• Memcached Client for Java is more stable, earlier, and wider than SpyMemcached;
• SpyMemcached is more efficient than Memcached Client for Java;
• XMemcached has better concurrent performance than SpyMemcache;

The use of the first two clients is not described here.

The XMemcached client is explained in three parts.

• XMemcached client usage demonstration
• Source code tracing of the set/get Method
• Compare the set/get commands of Memcached

I. XMemcached client demonstration I am building a project using Maven. To use XMemcached, you need to add it to pom. xml

<dependency><groupId>com.googlecode.xmemcached</groupId><artifactId>xmemcached</artifactId><version>1.4.3</version></dependency>

The XMemcached Demo is as follows:

Public static void main (String [] args) throws IOException {MemcachedClientBuilder builder = new XMemcachedClientBuilder (AddrUtil. getAddresses ("127.0.0.1: 11211"); MemcachedClient memcachedClient = builder. build (); try {memcachedClient. set ("key", 0, "Hello World! "); String value = memcachedClient. get ("key"); System. out. println ("key value:" + value);} catch (Exception e) {e. printStackTrace ();} try {memcachedClient. shutdown ();} catch (IOException e) {e. printStackTrace ();}}

Next, we will trace the source code of these two methods in detail.

Ii. source tracing of the set/get method 1.set you can use the debug mode to track the set and get processes step by step. The specific process is not demonstrated, first, the general source code call process of the set method is listed as follows (some methods may be omitted in the middle)

XMemcachedClient.set() XMemcachedClient.sendCommand()  MemcachedConnector.send()   AbstractSession.write()    MemcachedTCPSession.wrapMessage()     TextStoreCommand.encode()      TextStoreCommand.encodeValue()       SerializingTranscoder.encode()        BaseSerializingTranscoder.serialize()

Call XMemcacheClient first. set (final String key, final int exp, final Object value) method, the key parameter corresponds to the String "key", and the exp parameter corresponds to the integer 0 (indicating that the cache will never expire ), the value parameter corresponds to the string "Hello World! ". After a series of method calls, the SerializingTranscoder. encode (Object o) method is finally called. At this time, the real parameter value received by the form parameter o is the set string "Hello World !", The code for this method is as follows:

public final CachedData encode(Object o) {byte[] b = null;int flags = 0;if (o instanceof String) {b = encodeString((String) o);} else if (o instanceof Long) {if (this.primitiveAsString) {b = encodeString(o.toString());} else {b = this.transcoderUtils.encodeLong((Long) o);}flags |= SPECIAL_LONG;} else if (o instanceof Integer) {if (this.primitiveAsString) {b = encodeString(o.toString());} else {b = this.transcoderUtils.encodeInt((Integer) o);}flags |= SPECIAL_INT;} else if (o instanceof Boolean) {if (this.primitiveAsString) {b = encodeString(o.toString());} else {b = this.transcoderUtils.encodeBoolean((Boolean) o);}flags |= SPECIAL_BOOLEAN;} else if (o instanceof Date) {b = this.transcoderUtils.encodeLong(((Date) o).getTime());flags |= SPECIAL_DATE;} else if (o instanceof Byte) {if (this.primitiveAsString) {b = encodeString(o.toString());} else {b = this.transcoderUtils.encodeByte((Byte) o);}flags |= SPECIAL_BYTE;} else if (o instanceof Float) {if (this.primitiveAsString) {b = encodeString(o.toString());} else {b = this.transcoderUtils.encodeInt(Float.floatToRawIntBits((Float) o));}flags |= SPECIAL_FLOAT;} else if (o instanceof Double) {if (this.primitiveAsString) {b = encodeString(o.toString());} else {b = this.transcoderUtils.encodeLong(Double.doubleToRawLongBits((Double) o));}flags |= SPECIAL_DOUBLE;} else if (o instanceof byte[]) {b = (byte[]) o;flags |= SPECIAL_BYTEARRAY;} else {b = serialize(o);flags |= SERIALIZED;}assert b != null;if (this.primitiveAsString) {// It is not be SERIALIZED,so change it to string typeif ((flags & SERIALIZED) == 0) {flags = 0;}}if (b.length > this.compressionThreshold) {byte[] compressed = compress(b);if (compressed.length < b.length) {if (log.isDebugEnabled()) {log.debug("Compressed " + o.getClass().getName() + " from "+ b.length + " to " + compressed.length);}b = compressed;flags |= COMPRESSED;} else {if (log.isDebugEnabled()) {log.debug("Compression increased the size of "+ o.getClass().getName() + " from " + b.length+ " to " + compressed.length);}}}return new CachedData(flags, b, this.maxSize, -1);}

First, the local variable B (used to store the byte array that needs to be put into the memcached server) and flags (used to store the flag information) are affirmed ). Then, judge whether the object o is of the string type and long integer type, and encode the object o into the corresponding byte array and store it in the local variable B.

Pay special attention to row 3. When the o type is not a string, basic type packaging type, or byte [] array, the BaseSerializingTranscoder. serialize () method is called. The source code of this method is as follows:

protected byte[] serialize(Object o) {if (o == null) {throw new NullPointerException("Can't serialize null");}byte[] rv = null;try {ByteArrayOutputStream bos = new ByteArrayOutputStream();ObjectOutputStream os = new ObjectOutputStream(bos);os.writeObject(o);os.close();bos.close();rv = bos.toByteArray();} catch (IOException e) {throw new IllegalArgumentException("Non-serializable object", e);}return rv;}

Obviously, this method is to serialize objects, convert Java objects into byte arrays, and return them. As you can see, we should understand why custom objects must implement the Serializable interface before they can be saved to Memcached. If the data object does not implement the Serializable interface, IOException will be thrown during object serialization, IllegalArgumentException will be thrown, and Non-serializable object will be prompted.

It also highlights the role of the CachedData class, which encapsulates the cas value (this value is used for atomic update, that is, the cas value is included in the request information each time the client sends an update request, after receiving the request, the memcached server compares the cas value of the cas value with the cas value of the stored data on the server. If the value is equal, the old data is overwritten with the new data. Otherwise, the update fails. Especially useful in a concurrent environment), data (that is, the data value to be cached or the retrieved cache data, stored in byte [] array form ), flag Information (additional data type information of the byte [] array and whether the byte [] array is compressed or not, stored in an int type) and other information.

The set source code is analyzed here. Let's talk about the get source code below.

2. get

Similarly, the general source code call process for listing the get method is as follows:

XMemcachedClient.get() XMemcachedClient.fetch0()  XMemcachedClient.sendCommand()   MemcachedConnector.send()    AbstractSession.write()     MemcachedTCPSession.wrapMessage()      TextGetCommand.encode()       SerializingTranscoder.decode()        SerializingTranscoder.decode0()         BaseSerializingTranscoder.deserialize()

First, call the XMemcacheClient. get (final String key) method. The key parameter corresponds to the String "key ". From this method to TextGetCommand. an encode () call can be considered as a process of assembling the get command and sending it to the server. After receiving the server response message, the Response Message is assembled into CachedData and SerializingTranscoder is called. the decode (CachedData d) method is used to decode the byte stream. The code for this method is as follows:

public final Object decode(CachedData d) {byte[] data = d.getData();int flags = d.getFlag();if ((flags & COMPRESSED) != 0) {data = decompress(d.getData());}flags = flags & SPECIAL_MASK;return decode0(d,data, flags);}

First, obtain the byte array and flag information, and determine whether to extract the byte array based on the flag. Finally, call the decode0 (CachedData cachedData, byte [] data, int flags) method. The Code is as follows:

protected final Object decode0(CachedData cachedData,byte[] data, int flags) {Object rv = null;if ((cachedData.getFlag() & SERIALIZED) != 0 && data != null) {rv = deserialize(data);} else {if (this.primitiveAsString) {if (flags == 0) {return decodeString(data);}}if (flags != 0 && data != null) {switch (flags) {case SPECIAL_BOOLEAN:rv = Boolean.valueOf(this.transcoderUtils.decodeBoolean(data));break;case SPECIAL_INT:rv = Integer.valueOf(this.transcoderUtils.decodeInt(data));break;case SPECIAL_LONG:rv = Long.valueOf(this.transcoderUtils.decodeLong(data));break;case SPECIAL_BYTE:rv = Byte.valueOf(this.transcoderUtils.decodeByte(data));break;case SPECIAL_FLOAT:rv = new Float(Float.intBitsToFloat(this.transcoderUtils.decodeInt(data)));break;case SPECIAL_DOUBLE:rv = new Double(Double.longBitsToDouble(this.transcoderUtils.decodeLong(data)));break;case SPECIAL_DATE:rv = new Date(this.transcoderUtils.decodeLong(data));break;case SPECIAL_BYTEARRAY:rv = data;break;default:log.warn(String.format("Undecodeable with flags %x",flags));}} else {rv = decodeString(data);}}return rv;}

The above method is actually the reverse implementation of the encode (Object o) method, which converts byte arrays into Object objects. Note that line 3 calls the deserialize (byte [] in) method. The code for this method is as follows (catch and finally are omitted ):

protected Object deserialize(byte[] in) {Object rv = null;ByteArrayInputStream bis = null;ObjectInputStream is = null;try {if (in != null) {bis = new ByteArrayInputStream(in);                is = new ObjectInputStream(bis) {                    @Override                    protected Class<?> resolveClass(ObjectStreamClass desc) throws IOException, ClassNotFoundException {                        try {                            //When class is not found,try to load it from context class loader.                            return super.resolveClass(desc);                        } catch (ClassNotFoundException e) {                            return Thread.currentThread().getContextClassLoader().loadClass(desc.getName());                        }                    }                };                rv = is.readObject();}}...return rv;}

The above code is to deserialize the object and return it. Each deserialization operation produces a brand new object. Any operation on the new object will not affect the values stored in memcached. Iii. Compare the set/get commands of Memcached

In addition to the above interaction with memcached through java code, we can also directly interact with memcached through commands. The procedure is as follows:

1. Open the cmd window first

2. Use telnet to connect to the memcached server. The command is as follows:

telnet 127.0.0.1 11211

3. If the connection is successful, you can directly enter the command to interact with the memcached server.

1. Storage commands

The format is as follows:

<command name> <key> <flags> <exptime> <bytes><data block>

Parameters are described as follows:

<Command name> set/add/replace

<Key> Search for keywords

<Flags> the client uses it to store additional information about key-value pairs.

<Exptime> survival time of the Data. 0 indicates permanent

<Bytes> Number of stored bytes

<Data block> stored data blocks (which can be directly understood as values in the key-value structure)

Special instructions on storage commands:

• If the set command does not exist, add the key; otherwise, update the key.
• The add command can be successfully added only when the key does not exist.
• The replace command can be replaced successfully only when the key exists.

Example:

set myname 0 0 9super manSTORED

2. Read commands

Get key [key1]...

Obtain one or more key values, separated by spaces.

Example:

get mynameVALUE myname 0 9super manEND

In addition to the common commands above, there are also gets, cas, stats and other commands. If you are interested, you can learn them.

3. analogy

Similar to the Xmemcached client, it is not difficult to find interesting places:

In the set command, <key> and <exptime> correspond to the key and exp parameters in the XMemcacheClient. set (final String key, final int exp, final Object value) method respectively. <Flags> and <data block> correspond to the flag and data member variables of the CachedData encapsulation class.

The get command <key> corresponds to XMemcacheClient. the key parameter of the get (final String key) method. The result returned by the get command corresponds to SerializingTranscoder. the parameter d of the decode (CachedData d) method. The type of d is CachedData, which encapsulates the flag and data information.

Through the comparison above, it is not difficult to find that both the Memcached command and the Xmemcached client are just an implementation of the memcached client. They comply with the same request and Response Message specifications. At the underlying layer, both methods send request messages that comply with the memcached conventions after establishing a tcp connection. After receiving a message from the memcached server, it is also decoded according to the memcached Response Message Convention (The Xmemcached client uses the flag field to convert the Data byte array to the type required by the application layer ).

In other words, the Memcached cache system provides the implementation of the server (C language) and specifies the Message format in which the client communicates with the server. More accurately, it is the byte stream format (communication through tcp ). Clients in different languages are just the implementation of this specification. Of course, Memcached has provided client implementation in most languages, but you can also develop a client implementation by yourself.

4. How to optimize Xmemcached client code to improve efficiency

From the source code analysis, we can see that if Memcached is stored as a bean object, you need to implement the Serializable interface to support java object serialization. As far as I know, java's built-in Object serialization not only takes time for serialization and deserialization operations, but also generates a large byte array. Therefore, you can consider another encoding/decoding technology. I recommend using fastjson, which is not only inefficient, but also produces small json strings. Other optimization measures.