Memcached is a distributed memory object cache system developed by danga.com (technical team operating livejournal) to reduce database load and improve performance in dynamic systems. The number of dynamic page views per second on the LJ is several thousand, and the number of users is 7 million. Memcached greatly reduces the database load and makes it easier to allocate resources for faster access.
I believe many people have used this item. This article aims to gain a deeper understanding of this outstanding open-source software through implementation and code analysis of memcached, we can further optimize it based on our needs. In the end, we will analyze the bsm_memcache extension to better understand the usage of memcached .. 1. What is memcached? Before explaining this problem, we must first know what it is "not ". Many people use it as a storage carrier in the form of sharedmemory. Although memcached uses the same "Key => value" method to organize data, however, it is very different from local caches such as shared memory and APC. Memcached is distributed, that is, it is not local. It is based on network connections (of course, it can also use localhost) to complete the service. It is an application-independent program or daemon process ). That is, memcached is a high-performance, distributed memory object cache system that reduces database loads in dynamic applications and improves access speeds. Memcached uses the libevent library to implement network connection services. In theory, it can handle infinite connections, but unlike Apache, it is more stable and persistent connection oriented, therefore, its actual concurrency capability is limited. Under conservative circumstances, the maximum number of simultaneous connections of memcached is 200, which is related to the Linux thread capability. This value can be adjusted. For more information about libevent, see related documentation. Memcached memory usage is also different from APC. APC is based on shared memory and MMAP. memcachd has its own memory allocation algorithm and management method. It has nothing to do with shared memory and has no restrictions on shared memory. Generally, each memcached process can manage 2 GB of memory space. If you need more space, you can increase the number of processes.
2. When memcached is suitable for many occasions, memcached is abused, and complaints about it are indispensable. I often see people posting on the Forum, similar to "How to Improve efficiency". The reply is "using memcached". As for how to use and where to use memcached, there is no such thing. Memcached is not omnipotent, and it is not applicable to all occasions. Memcached is a "distributed" Memory Object cache system. That is to say, applications that do not require "distribution", do not need to be shared, or are simply as small as those with only one server, memcached does not bring any benefits. On the contrary, memcached slows down system efficiency because network connections also require resources, even local UNIX connections. According to my previous test data, the local read/write speed of memcached is dozens of times slower than the direct PHP memory array, while the APC and shared memory modes are similar to direct arrays. It can be seen that using memcached is not cost-effective if it is only a local cache. Memcached is often used as the database front-end cache. Because it has much less overhead than the database, such as SQL parsing and disk operations, and uses memory to manage data, it can provide better performance than directly reading the database, in large systems, the same data is frequently accessed. memcached can greatly reduce the database pressure and improve the system execution efficiency. In addition, memcached is often used as a storage medium for data sharing between servers. For example, in the SSO system, data stored in the single-point logon status of the system can be stored in memcached and shared by multiple applications. It should be noted that memcached uses memory to manage data, so it is easy to lose. When the server is restarted or the memcached process is suspended, data will be lost, so memcached cannot be used to persistently store data. Many people mistakenly understand that memcached has a very good performance, so far as the comparison of memory and hard disk, in fact, memcached does not get hundreds of thousands of read/write speeds to use memory, its actual bottleneck lies in network connection. Compared with the database system that uses disks, it has the advantage of being "light" because it does not overhead and has direct read/write methods, it can easily cope with a very large amount of data exchange, so it is often possible that the two Gigabit network bandwidth is full, the memcached process itself does not occupy much CPU resources. Generally, the webpage cache methods include dynamic cache and static cache. net can be used to cache partial pages, while memcached cache is better than ASP. the local cache of net is more flexible and can cache any object, regardless of whether it is output on the page or not. The biggest advantage of memcached is that it can be deployed in a distributed manner, which is also essential for large-scale applications. Livejournal.com uses memcached for caching on the front end and achieves good results. However, Wikipedia and SourceForge also adopt or will soon use memcached as the cache tool. Memcached can play a huge role in large-scale website applications.
2.1 Working Principle of memcachedFirst, memcached runs on one or more servers as a daemon and accepts client connection operations at any time. The client can be written in various languages, currently, known client APIs include Perl, PHP, Python, Ruby, Java, C #, and C. After the PHP client establishes a connection with the memcached service, the next thing is to access the object. Each accessed object has a unique identifier key, and the access operation is performed through this key, objects saved to memcached are actually stored in the memory, not in the cache file. This is why memcached is so efficient and fast. Note that these objects are not persistent. After the service is stopped, the data in the objects will be lost.
2.2 install memcachedFirst download memcached, the latest version is 1.1.12, directly from the official website to download to the memcached-1.1.12.tar.gz. In addition, memcached used libevent, I downloaded the libevent-1.1a.tar.gz next is to unbind the libevent-1.1a.tar.gz and memcached-1.1.12.tar.gz package, compile, install:
# tar -xzf libevent-1.1a.tar.gz
# cd libevent-1.1a
# ./configure --prefix=/usr
# make
# make install
# cd ..
# tar -xzf memcached-1.1.12.tar.gz
# cd memcached-1.1.12
# ./configure --prefix=/usr
# make
# make install
After the installation is complete, memcached should be in/usr/bin/memcached. 3. How to Use memcached-server? Run #./memcached-D-M 2048-l 10.0.0.40-P 11211-u httpd on the server
This starts a process that occupies 2 GB of memory and opens port 11211 to receive requests. Because the 32-bit system can only handle 4G memory addressing, 2-3 processes can be run on 32-bit servers with 4G memory and monitored on different ports. 4. How to Use memcached-client? After the application end contains a class used to describe the client, it can be used directly, which is very simple. PHP example: $ options ["servers"] = array ("192.168.1.41: 11211", "192.168.1.42: 11212"); $ options ["debug"] = false; $ MEMC = new memcachedclient ($ options); $ myarr = array ("one", "two", 3); $ MEMC-> set ("key_one", $ myarr ); $ val = $ MEMC-> get ("key_one"); print $ Val [0]. "\ n"; // prints 'one' Print $ Val [1]. "\ n"; // prints 'two' Print $ Val [2]. "\ n"; // prints 35. why not use a database to do this? What kind of database (MS-SQL, Oracle, S, MySQL-InnoDB, Etc ..), implementing acid, atomicity, consistency, isolation, and durability requires a lot of overhead. Especially when using hard disks, this means that the query may be blocked. When a database that does not contain transactions (such as MySQL-MyISAM) is used, the preceding overhead does not exist, but the read thread may be blocked by the write thread. Memcached never blocks and is very fast. 6. Why not use shared memory? The original caching method was to cache objects in the thread, but the cache could not be shared between processes, and the hit rate was very low, resulting in extremely low cache efficiency. Later, there was a cache for shared memory. Multiple processes or threads shared the same cache, but after all, it was limited to only one machine, the same cache on multiple machines is also a waste of resources, and the hit rate is relatively low. Memcached server and clients work together to implement cross-Server Distributed Global cache. It can work with the Web server. The web server has high CPU requirements and low memory requirements. The memcached server has low CPU requirements and high memory requirements, so it can be used together. 7. How is the MySQL 4.x cache? The query cache for MySQL is not ideal because the query cache is cleared when the specified table is updated. In a system with a large load, such events occur frequently, resulting in low query cache efficiency. In some cases, the query cache may not be available, because it still has overhead for Cache Management. On 32-bit machines, MySQL still limits memory operations to 4 GB, but memcached can be distributed out, so the memory size is theoretically unlimited. MySQL queries the cache instead of the object cache. If a large number of other operations are required after the query, the query cache will not help. If the data to be cached is small and not frequently queried, you can use MySQL to query the cache. Otherwise, memcached is better. 8. How about Database Synchronization? Here, Database Synchronization refers to the database synchronization mechanism similar to the MySQL master-slave mode through log synchronization. You can distribute read operations but cannot distribute write operations. However, synchronization of write operations consumes a large amount of resources, and the overhead increases with the growth of the slave server. The next step is to split the database horizontally so that different data is distributed to different database server groups to achieve distributed read/write, this requires connecting different databases based on different data in the application. When this mode works (we also recommend this), more databases lead to more headaches and hardware errors. Memcached can effectively reduce access to the database, so that the database uses the primary energy to do less frequent write operations, which is controlled by the database itself and rarely blocks itself. 9. Is memcached fast? It uses libevent to handle any number of open connections (using epoll instead of poll), and uses non-blocking network I/O, distributed Hash objects to different servers, the query complexity is O (1 ). 10. Related abstract classes of memcached
11. An example of service calling memcache:
References: http://www.danga.com/http://www.linuxjournal.com/article/7451
Public list <integer> getlastmodifyalbummember (INT limit )...{
String cachekey = This. createcachekey (New
Object []... {"Schedule", "lastmodifyalbumember", limit });
List list = (list) This. getcachevaluefrommemcached (cachekey );
List <integer> list1 = new arraylist <integer> ();
If (null! = List & list. Size ()> 0 )...{
For (Object alist: List )...{
List1.add (integer. parseint (string. valueof (alist )));
}
}
Return list1;
}
Public abstract class basemanager <t extends baseobject> implements manager <t> ...{
Private memcachedclient memcached;
Protected int default_cache_time_second = 3600;
Public void setmemcached (memcachedclient memcached )...{
This. memcached = memcached;
}
Public memcachedclient getmemcached ()...{
Return memcached;
}
Public void setdefault_cache_time_second (INT default_cache_time_second )...{
This. default_cache_time_second = default_cache_time_second;
}
Public object getcachevaluefrommemcached (string key )...{
Return getcachevaluefrommemcached (Key, null );
}
Public object getcachevaluefrommemcached (string key, object OBJ )...{
Object new_obj = NULL;
Try ...{
New_obj = memcached. Get (key );
} Catch (exception e )...{
If (logger. iswarnenabled ())...{
Logger. Warn ("failed to get from memcache", e );
}
}
Return (new_obj! = NULL )? New_obj: OBJ;
}
Public void setcachevaluetomemcached (string cachekey, int time_to_live, serializable OBJ )...{
If (null! = Memcached. Get (cachekey ))...{
Memcached. Replace (cachekey, time_to_live, OBJ );
} Else ...{
Memcached. Add (cachekey, time_to_live, OBJ );
}
}
}-D run memcached in daemon mode;
-M: Set the memory size available for memcached, in MB;
-L set the IP address of the listener. If it is a local machine, this parameter is usually not set;
-P: Set the listening port. The default value is 11211. Therefore, you can leave this parameter unspecified;
-U specifies the user. If the current user is root, you must use this parameter to specify the user.
