Problems with socket leaks caused by CAS and set methods of memcached

In order to solve the conflict scheme of multiple concurrent write memcached, our project team introduced the CAS mechanism, which is similar to the CAS (Compareand set) atomic operation in Java and the contract, to deal with the concurrency problem that the same item is changed by multiple threads. Memcached's CAs is the principle of introducing the version number concept, where each stored data object has a value of 64bit length as the key corresponding to the version number of value. The following code is used:

This method differs from the Get method to get the Memcacheditem object Memcacheditem item = mc.gets (key); Mc.cas (Key, Item.getvalue () + string.valueof (sleep), Item.getcasunique ());

For single-threaded scenarios in the project, we used the memcached set method. However, during the project endurance test, we found that the number of handles of the WebLogic process of the Web site Management Server was constantly increasing, and when WebLogic made a secure backup, it triggered an error and caused the subsequent threads to too many open files. The relevant error message is as follows:

<2015-5-30 11:05 P.M. 16 sec cst> <Critical> <EmbeddedLDAP> <BEA-000000> < Java.io.FileNotFoundException:/weblogic/oracle/middleware/user_projects/domains/shch_domain1/servers/shchbdf1/ Data/ldap/backup/embeddedldapbackup.zip (Too many open files) at       java.io.FileOutputStream.open (Native Method) At       java.io.fileoutputstream.<init> (fileoutputstream.java:194) at       java.io.fileoutputstream.< Init> (fileoutputstream.java:84) at       Com.octetstring.vde.backend.standard.BackupTask.runTask ( BACKUPTASK.JAVA:55)

We checked the handle without closing by command, and we know that the Socke communication can ' t identify protocol is not notified to the client after the server abnormally interrupts the socket communication, resulting in the client handle not being released.

We know that the application and memcached direct communication is the use of NIO socket communication, the socket is the connection will produce a handle number, but the normal close channel, the handle will be released, unless the socket channel is not properly closed. From this point of view, we comb the project code and the anomaly scene. First, only the background-managed process will increase the number of handles, which is not happening on multiple servers in the foreground cluster quoting system. Analyze the background management code, there is a scheduled task to update the value of memcached, because it is a single thread, using the Set method. But this set method and the quoted CAs method are to modify by the value, according to the intuitive judgment is the set method and the CAs method at the same time to modify a value, if it happens at the same time, the modification of the set method is not successful, memcached automatically disconnects the set is worth connecting, but not the client , causing a socket leak. After we have modified the memcached set method into CAs, the problem is solved.

Problems with socket leaks caused by CAS and set methods of memcached