memcachedb-cache system for persistent storage

Memcached is a high-performance distributed memory object caching system for dynamic Web applications to mitigate database load. It provides a dynamic, database-driven site speed by caching data and objects in memory to reduce the number of times the database is read. Memcached is based on a hashmap that stores key/value pairs. Its daemon (daemon) is written in C, but the client can write in any language and communicate with the daemon through the memcached protocol.

Memcached is a set of distributed cache system, originally Danga Interactive for LiveJournal development, but is currently used by many software (such as MediaWiki). This is a set of open source software that is released with BSD license authorization. [1] memcached lacks authentication and security controls, which means that the memcached server should be placed behind a firewall. [1] The Memcached API uses a 32-bit cyclic redundancy check (CRC-32) to calculate the key values and spread the data across different machines. When the table is full, the next additions will be replaced with the LRU mechanism. Since memcached is usually used only as a cache system, applications that use memcached will require additional code updates memcached data in writing back to a slower system (like a back-end database) [1] memcached is LiveJournal A software developed by Brad Fitzpatric, a Danga Interactive company. It has become an important factor in improving Web application extensibility in many services such as Mixi, Hatena, Facebook, Vox, and LiveJournal. Many Web applications save data to an RDBMS, where the application server reads the data and displays it in the browser. 　　However, with the increase of data volume and the concentration of access, the burden of RDBMS, database response deterioration, site display delay and other significant impact. This is the time to memcached. Memcached is a high-performance distributed memory cache server. 　　The general purpose is to reduce the number of database accesses by caching database query results to improve the speed and scalability of dynamic Web applications. Memcached is a high-performance distributed memory object caching system for dynamic Web applications to mitigate database load. It reduces the number of times a database is read by caching data and objects in memory, providing a faster speed for dynamic, database-driven Web sites. Memcached is based on a hashmap that stores key/value pairs. Its daemon (daemon) is written in C, but the client can write in any language and communicate with the daemon through the memcached protocol. 　　However, it does not provide redundancy (for example, copying its hashmap entries), and when a server S is stopped or crashed, all key/value pairs stored on s are lost. Memcached was developed by Danga Interactive, and the latest version was released in 2010 by the author of Anatoly Vorobey and Brad Fitzpatrick. For lifting LIvejournal. COM access speed. LJ Dynamic page visits thousands of times per second, user 7 million. Memcached significantly reduces database load, better allocation of resources, faster access.

Feature memcached, as a distributed cache server running at high speed, has the following characteristics. Simple protocol · Event handling based on Libevent · Internal Memory Storage Method · Memcached distributed without communication with each other
Agreement
memcached Server client communication does not use a format such as complex XML, but uses a simple text-line-based protocol. As a result, you can also save data and get data on memcached by using Telnet. 　　Here is an example. 　　$ telnet localhost 11211 Trying 127.0.0.1 ... 　　Connected to Localhost.localdomain (127.0.0.1). 　　Escape character is ' ^] '. Set Foo 0 0 3 (Save command) bar (data) STORED (result) get foo (get command) VALUE Foo 0 3 (data) bar (data)
Event handling
Libevent is a library that encapsulates event-handling functions such as Linux's Epoll, BSD-class operating system kqueue, and so on as a unified interface. The Performance of O (1) can be played even if the number of connections to the server increases. Memcached uses this libevent library, so it can perform its high performance on Linux, BSD, Solaris and other operating systems. About event handling is no longer detailed here, you can refer to Dan Kegel's c10k problem.
Storage mode
To improve performance, the data saved in memcached is stored in Memcached's built-in memory storage space. Because the data exists only in memory, restarting the memcached and restarting the operating system will cause all data to disappear. Additionally, when the content capacity reaches the specified value, the unused cache is automatically deleted based on the LRU (Least recently used) algorithm. The memcached itself is a server designed for caching, so there is not too much consideration for permanent data issues.
Communication distributed
　　Memcached Although it is a "distributed" cache server, there is no distributed functionality on the server side. Each memcached does not communicate with each other to share information. So, how to distribute it? This depends entirely on the client's implementation. This article will also describe the distribution of memcached. 　　Features a visual memcached management system written in PHP memadmin is a visual memcached management and Monitoring tool, using PHP development, small size, simple operation. Main functions: Server parameter monitoring: STATS, SETTINGS, ITEMS, slabs, sizes real-time Refresh server performance monitoring: GET, DELETE, INCR, DECR, CAs and other common operation hit rate real-time monitoring support data traversal, Easy to monitor the storage content Support condition query, filter out the key or value array, JSON and other serialized character inverse sequence display compatible with the Memcache protocol, such as Tokyo Tyrant (except traversal function) Support server connection pool, multi-server Management switch Convenient and concise use of skills many languages have implemented the connection Memcached client, which is mainly Perl, PHP. Only the languages listed on the memcached website are available · Perl PHP Python Ruby C# C/c++ 　　Lua and so on. How to use the Memcached-server side to run on the server: #./memcached-d-M 2048-l 10.0.0.40-p 11211 This will start a process that consumes 2G of memory and open 11211 ports for receiving requests. 　　Because 32-bit systems can handle only 4G of memory, 2-3 processes are run on 32-bit servers that use PAE larger than 4G of memory and are monitored on different ports. 　　How to use the Memcached-client end to include a class that describes the client on the application side is straightforward to use. 　　PHP Example: $options ["Servers"] = Array ("192.168.1.41:11211", "192.168.1.42:11212"); 　　$options ["Debug"] = false; 　　$MEMC = new Memcachedclient ($options); $myarr = Array ("One", "", 3); 　　$MEMC->set ("Key_one", $myarr); 　　$val = $memc->get ("Key_one"); Print $val [0]. " \ n "; Prints ' one ' print $val [1]. " \ n "; Prints ' both ' Print $val [2]. " \ n "; Prints 3 Release February 03, 2012, Memcached 1.4.13 released. [2] August 02, 2012, Memcached 1.4.14 released. [3] September 05, 2012, Memcached 1.4.15 released. [4]

memcachedb-cache system for persistent storage
Memcachedb is memcached + BDB, is based on the memcached socket layer and BERKELEY-DB storage layer structure, is the Sina Interactive Community Technical Team 2007 years of a major technical achievements, Now it is used in many product lines of Sina Interactive community, including Sina Blog and other products. The ability to implement arbitrary memcache API calls, real-time data landing and primary and secondary real-time backup.

Choose Memcachedb: There is no special reason, we have the best reason is to do the maximum without sacrificing data read, write performance, we can let your original stored in Memcache no longer suffer from system downtime or system power loss, even if the server powered off, As long as the hard drive is still working, switch to a different server and your data is still on the hard drive safely.

Performance: We have done small data testing, small data (dozens of bytes) of data read and write, the concurrency per second can be maintained in 5000+ (a single block of 146Scis hard disk, CPU and memory configuration can almost be ignored because this is not a performance bottleneck). Basically meet all the Internet efficient storage applications.

Application: Small data that requires frequent reading and writing. For example, digital services such as the number of clicks, the need to frequently write small data such as commenting system, need to record user status such as the number of people online and online users and so on. Of course, if you want to store a single pen more than 10K of data, such as news or articles or forum posts can also, but we have not tried, if you really used to play, you can feedback to me.

Data backup && read/write separation: Memcachedb supports primary and secondary real-time non-differential synchronization, and configuring primary and secondary services requires only a short startup command. Of course, you can also use this handy feature for read-write separation, which makes your system more efficient.

Stability: Memcachedb is currently used in multiple product lines in the Sina Interactive community, with steel-like quality.

Disadvantage: Memcached does not support the traversal of memory objects, of course, it is more unable to support complex query operations, can only support based on the known key to query the corresponding value. But later I will introduce another Dongdong Memcache_engine, which provides MySQL with a memcached engine.

First, installation
Locate the install file directly in memcached-1.2.2.tar.gz and follow the steps to install it. Very simple.

Second, start
#第一种, read and write merge mode, that is, do not do the secondary copy, that is, read and written, do not do redundant backup
memcachedb-p21211-d-r-u root-h/data/mdb_11211-n

#第二种, read/write separation/Backup mode
#主服务器 192.168.0.1 Read-write listening 11211, synchronization via 31211 port, do not occupy the busy main service port
memcachedb-p21211-d-r-u root-h/data/mdb_11211_m-n-R 127.0.0.1:31211-m

# secondary server 192.168.0.2 read-only listening 21212, synchronizing data from the 31211 port of 192.168.0.1, while opening a 31212 port to provide data synchronization service externally
memcachedb-p21212-d-r-u root-h/data/mdb_11211_from_16801_s-o 192.168.0.1:31211-r 192.168.0.2:31212-s

memcachedb-cache system for persistent storage