Large Web site Optimization-memcache technology, Optimization-memcache Technology _php Tutorial

Large site optimization-memcache technology, optimizing-memcache technology

Large Web site Optimization-memcache Technology

Memory+cache Memory Cache

Memcache Introduction

Memcache is a distributed cache system developed by LiveJournal Brad Fitzpatrick, which is currently used by many websites to improve access to the site, especially for large, Site access that requires frequent access to the database is significantly faster. This is a set of open source software to be released in BSD license licensing. "Extracted from Baidu encyclopedia"

Official website: http://memcached.org/

Distributed: Multiple memcache servers to manage the data architecture.

Cache System: Caches user-queried data into memory for easy next-time retrieval from memory. Reduces the overhead of disk IO.

What is NoSQL? (SQL relational database)

A: MySQL is called a relational database (the main feature is a two-dimensional table structure (rows and columns in the table), the table is relational to the table) (Oracle (Java), DB2,SQL Server)

Non-relational data: a database that does not use SQL statements as a query (the system that holds the data) and does not have a strict concept of a two-dimensional table. Its data structure is all a huge hash table (Key-value)

Hash Table Benefits: Time complexity is 0 (1): As the data increases, there is no change in the order of magnitude (1s) in the query time.

Hash table disadvantage: hash collision different key corresponding to the same value

Key1 ===md5 (SQL)	Asion
Key2	12
Key3	Shenzhen
Key4	Iphone

SELinux is not shutting down

Let SELinux take effect immediately

2.1 Environment Preparation

In a Linux environment, you need tools such as GCC, g-c++, make (makefile), CMake, AutoConfig (Configure), Libtool, etc.

In the case of Linux under networking , use the following command

# yum Install- y gcc make cmake autoconfig libtool

-Y does not need to use confirm interactive

2.2 Compiling and installing memcached

Memcached relies on the libevent Library, so it needs to be installed first to download the stable version on its respective website.

LIBEVETNT Official website: http://libevent.org/

Memcache Official website: http://memcached.org/

Compile and install libevent, install memcached at compile time, and install memcached to specify libevent installation path

Specific steps:

Upload

First download libevent, unzip, and install (General Linux under the Software installation is placed in the/USR/LOCAL/SRC directory, installed software is generally placed in/usr/local/name)

#./configure--prefix=/usr/local/libevent && make && make install

B. Install memcache, unzip, compile, install

#./configure--prefix=/usr/local/memcached--with-libevent=/usr/local/libevent && make && make Install

2.3 memcached Start-up

#/usr/local/memcached/bin/memcached-m 64-p 11211-u NOBODY-VV

Note : The memcached starts successfully at this time, but outputs the information to the console.

If you want memcached to start in the background as a service, just add the-d option (daemon background)

#/usr/local/memcached/bin/memcached-m 64-p 11211-u nobody-d

How do I see if my server starts properly?

# PS Axu | grep memcacheD

If you need to view parameter information, use Memcached-h to view Help:

Operation Memcache

Memcached Client-server communication is simple, text-based protocols , like HTTP protocols, can be used to interact directly with Telnet

Use Telnet action (quit exit)

# IP 11211 for telent server

Windows under

Use CTRL + ] to turn on the echo of Telnet after connection

Linux under

2. Basic commands:

Learn Memcache's additions and deletions to change the command:

Add key flag expire length

Key: Name

Flag:1 Memcache is saved as a string

Expire: Expiry time, memcache time

Length: Data lengths (B)

※add Increase

# Add Name 1 0 2 # Add a key to the Memcache server with a name value length of 2 bytes, valid for long term

How to understand expire

Set the expiration date of the cache in three formats

Set the number of seconds, starting from the setting, and failing aftern seconds

Timestamp, expired after the specified timestamp

Set to 0, not automatically invalidated (special note, not permanently valid.) A. During memcache installation, a maximum valid time is specified, and the default is 30 days (source code) b. May be squeezed out in less than 30 days)

Least recently used principle

※delete Delete

# Delete key

※replace replacement

# Replace key flag expire length

※get get

# get Key

※set If the data has replace, there is no add

# Set key flag expire length

Name exists:

Age does not exist:

※INCR Increase

# INCR Age Number

※DECR reduction

# DECR Age Number

※stats Statistics Memcache Server information

# stats

※flush_all Erase All data

# Flush_all

Using PHP to manipulate memcache

Windows under

to (http://downloads.php.net/pierre/) Download the extension DLL file

Place the file in a directory corresponding to PHP Extension_dir

Modify the php.ini file, add Extension=php_memcache.dll, introduce the file

Restart Apache

Linux (under Linux how to open an extension for PHP, say the common scenario )

Beta: Test version: Generally there are some small bugs, but users need to use the discovery, if there is a problem, can give official feedback. And then do the repair

Alpha: Internal beta: Internal in the development of the time, the use of the version, generally this version of the bug many. However, this version often has new features added. (usually the new company only tries) the Alpha version has some compensation.

Stable: Stable version: Basic bug-free, stable operation.

Considerations for FTP use:

VSFTPD whether the service VSFTPD status is open properly

Usually only use normal user connection FTP (not root to connect FTP) get using sudo

The size of your home directory is not enough (upload file is successful, but the display size is 0)

whether the firewall is off (iptables-f off) (iptables-l)

# CLS

# CD

# vim. BASHRC

Linux download last-line mode

#: X Lowercase exit Wq

Linux in edit mode

# Z Capital

#/usr/local/php/bin/phpize Absolute Path phpize

D. Use the Configure file generated above to collect system information do not need to specify the installation path

#./configure--with-php-config=/usr/local/php/bin/php-config Tell it how to find the PHP configuration file

E. Performing compilation, Installation

# Make && make install

Note: You can view the structure after completion of the above command execution

# ls/usr/local/php/lib/php/extensions/no-debug-non-zts-20090626/

Note: under Linux, how do I see the location of the php.ini configuration file?

Solution: Phpinfo ();

Note: Under Linux, be sure to back up the configuration file before you modify it

Php.ini-backup-2016-1-12

Note: What is the. so file?

. So is a shared object under Linux, type. dll file under Windows

Note: Under Linux, shutting down a service can be used

# pkill-9 HTTPD

# PS aux | grep httpd

#ps Uax | grep httpd

Add the address generated above to the php.ini configuration file, as follows

extension_dir=/usr/local/php/lib/php/extensions/no-debug-non-zts-20090626/

Extension=memcacahe.so

Build a test.php file to test if PHP has a module for memcache

PHP operation Memcache, save and get values

Memory management and deletion mechanism of memcached

Note: The maximum value of memcache can only be 1M of space.

Note: Memory fragmentation is always present, but only one way to minimize memory fragmentation.

1. What is memory fragmentation?

In the use of this memory cache system, due to the continuous application, release, will form some small memory fragments, can not be exploited, this phenomenon is called, memory fragmentation. This little red block is the space that the operating system cannot use.

How to solve?

Memcache is managed using slab allocator ( each slab class size is 1M)

The smallest unit is called Chunk (Small block): The warehouse where the data is stored

Multiple small units make up a chunks: multiple small blocks (all of the small pieces are the same size)

The size of each slab class is 1M

Memcache How to choose the right size?

Note: If the 122Bytes slab is full, now there is a 100Bytes of data to go, where to save?

A: Surely there will not be 144, or there are 122 of this chunk inside, using the LRU algorithm to achieve data storage.

Fixed chunk of memory waste, 22B

Factor tuning

Memcached in the boot, will be a certain size to organize the slab class, you can use-F to specify

The default is 1.25, and the ratio between adjacent chunk is the increase factor. You can adjust the size of the cache factor based on your website's business.

Because each business is different, the minimum chunk required is not the same. This parameter makes our system more adaptable to our business because the data can be set to its own size.

lazy removal of memcache

Memcached internally does not monitor whether the record is out of date, but instead looks at the timestamp of the record at get and checks whether the record is out of date. This behavior is called lazy (lazy) expiration. Therefore, the benefit is that memcached does not consume CPU time on outdated monitoring.

For example: There is set (name, Asion, 0, 3600) after 3,600 seconds of failure, after failure, and will not be automatically deleted, only when the get query, the detection is expired, if expired is deleted.

For example # Add Name 1 8 2 When 8s, is it invalid, or does it exist? Through stats analysis

Analysis: is to fail you, and not to exist, only in the next time to get the time, Memcache will detect whether it expires, expired deleted

The LRU algorithm of Memcache

Memcached takes precedence over the space of a record that has timed out, but even so, there is a lack of space when appending a new record, and a space is allocated using the Least recently Used (LRU) mechanism.

As the name implies, this is the mechanism for deleting "least recently used" records. Therefore, when the memory space of the memcached is low (when the new space cannot be obtained from the Slab Class), it is searched from records that have not been used recently and is empty

Assigned to a new record. From a practical point of view of caching, the model is ideal.

When the data space inside the Mecache (the default is 64M) is already full, then continue to store the data can be stored?

A: Can be stored, to delete outdated data, if not expired, delete the most inactive data, make room for the back to add data.

For example: In 122Bytes Slab example, when the data filled, if a 100Bytes data, how to deal with it? (even if it is permanent, it will be kicked)

Analysis: The management of the memory LRU algorithm , FIFO algorithm

Some parameters of Memcache Ctrl + C

Note: If you enter Ctrl+s under VIM, you can use Ctrl+q to exit

-P Listening Port

-L connected IP address, default is native

-D Start memcached service

-D Restart Restart memcached service

-D Stop|shutdown Close the running memcached service

-D Install memcached service

-d Uninstall Uninstall memcached service

-U Run as (only valid when running as root)

-m maximum memory usage, in megabytes. Default 64MB

Note: If the system is 32-bit, the maximum limit is 2G, and if the system is 64-bit, there is no limit.

-M running out of memory and returning an error instead of deleting an item

-c Maximum number of simultaneous connections, default is 1024

-F Block size growth factor, default is 1.25

-N Minimum allocated space, key+value+flags default is 48

-H Display Help

-V Output warnings and error messages

-VV request and return information for the print client

-I print copyright information for memcached and libevent

Actual usage Scenarios

Save: 1. File 2. Mysql

Question: If there are too many session files in a folder, the retrieval is slow, how to handle it?

Answer: Tiered processing

Use Memcache to save, memcache use distributed to save (take more than one memcache to do storage)

Save session with Memcache

1. Modify the php.ini file with the following configuration information

Session.save_handler = memcache #代表使用memcache保存session

Session.save_path = "tcp://127.0.0.1:11211" # Specifies the address and port of the Memcache server

Test

1. Open session, Save session

Get session from Memcache

Advanced Features

Distributed Memcache Configuration

What is distributed?

Due to the limited service capability of a single memcache, multiple memcache can be used to provide caching capabilities, which is called the Memcache distributed Slow (cluster) storage System

How is it implemented?

It can be understood that how to spread the data on each memcache server.

In the client implementation of the distributed, before the data is saved, according to a certain algorithm , the data is saved to the Memcache server, when the data is obtained, the same algorithm as before to the corresponding Memcache server to obtain data

Distributed algorithms

Algorithm for fetching and touching

The value of the key to the server's number of redundancy, and then save the corresponding value value to the corresponding remainder of the memcache server, generally this hash function CRC32 (key) % 3

CRC32 () This function can make a string into a 32-bit integer

Cons: When a server goes down or needs to add a server, this time the cache data basically all fail, because the divisor has changed. Not strict formula, hit rate = Fetch to Data/Total 1/n N represents the number of servers

The problem is raised: When the memcache down, the cache data fails, and this time the MySQL pressure will surge,

At this time, MySQL will be down, and then restart Mysql,mysql will be down again in a short period of time, then, a little delay (the cache has been re-established a part), and downtime. As time went on, MySQL basically stabilized and the cache system was built successfully. Because the cache data does not exist, all requests are turned to MySQL to provide, this phenomenon is called memcache avalanche phenomenon.

Overview:

Consistent hashing for distributed

Supposedly there is a ring that distributes so many positive integers from 0 to 2^32

Then, after using the hash function (CRC32 ()) for the number of nodes in the server, map to the above ring in a clockwise direction

The key of the data that needs to be stored is also followed by the hash function above, and then saved to a node that is not smaller than itself in the clockwise direction.

Note: as long as the memcache down, there will be a loss of data. But to find a way to minimize data loss, using a consistent hash, even if one server goes down, it only affects the data on one server.

Virtual nodes: Sharing tasks

Cache Avalanche Phenomenon

Why is the avalanche caused?

Due to improper algorithm, fetch algorithm, resulting in a large number of cache failure, will cause avalanche

Solution: Consistent hash algorithm

Cache time is the same time, the cache system will all expire at the same time, this will also cause an avalanche

Solution: Cache time is set to a range of random times (3-9 hours)

Because of a memcache node cache data failure, resulting in the other Memcache node cache hit rate drops, the missing data in the cache will go to the MySQL database query, short period of time, causing the MySQL server pressure is huge, resulting in downtime, called the cache avalanche phenomenon .

After restarting MySQL, the short-term downtime again, but the cache data has been set up, after the MySQL repeatedly started, the cache is all rebuilt, MySQL no longer down to stabilize.

Solution: Set the cache time to a range of random values (3-9 hours), which will expire at different time periods, and divide the work of reconstruction into different times.

Memcache How to do high availability

Using the repcached implementation, the full name replication cached is a memcached high-availability technology invented by the Japanese, referred to as replication buffer technology.

Memcachedb is a distributed, key-value form of persistent storage System . developed by SINA personnel . It is not a cache component, but a reliable, fast, persistent storage engine based on object access. The protocol is consistent with memcache (incomplete), so many memcached clients can connect to it. Memcachedb uses Berkeley DB as a durable storage component, so many of Berkeley DB's features are supported. Extended

How do I add an extension to a PHP under Linux and say the generic steps?

For:

How is the security of Memcache resolved?

For:

Because Memcache's own design is extremely concise, there is no restrictions on the setting of permissions. Why not set permissions? Caching is only provided, in order to streamline

Intranet 192.168.1.110 intranet IP outside network unreachable

Write a firewall authentication rule , only allow yourself to specify that IP packets can be transferred in, all other discarded

192.168.1.221---224

3. What if there are too many files when saving the session file with the file?

In general, when more than 65,535 session files, this time the session will become unusually slow access, meaning that PHP code execution is slow, how to solve?

For:

Layered Processing: A folder that starts with a-Z in a folder and then a_z in the build

Use memcache processing: a single memcache processing capacity is limited, using distributed memcache to handle

How to understand the load Balancer cluster is also a highly available cluster, but is it not?

A: A load-balanced cluster provides a high-availability capability, and if a single outage is available, it can provide a normal service, but the service provides a bit of a struggle.

Because a load-balanced cluster does not provide a mechanism for keepalive . Monitoring mechanism

Case:

How do I use memcache in a project?

The meaning of memcache exists is to cache the data and reduce the pressure on MySQL.

Write a list page

Details page

Extended

A common way to install a PHP extension under Linux

For:

Download the corresponding extension of the source code Http://pecl.php.net/package/memcache

Upload the extension to the Linux server and place it under (/usr/local/src/)

Execute TAR-ZXVF NAME.tar.gz

CD NAME

Phpize naming (/usr/local/php/bin/phpize) under an absolute path must be within the directory of the expansion pack

Executes the path of the./configure--with-php-config=/usr/local/php/bin/php-config (Php-config under the--with-php-config= absolute path, And you do not need to specify the path of the installation)

Make && make install

will generate a directory with the name.so file in the directory

To modify the php.ini configuration file, add two lines of information

Extension_dir= ' above-generated directory '

extension= ' name.so '

Restart Apache, and then use the Phpinfo () function to test, in the browser input address view, search if there is a memcache, to represent the successful opening of the extension.

http://www.bkjia.com/PHPjc/1117896.html www.bkjia.com true http://www.bkjia.com/PHPjc/1117896.html techarticle Large website Optimization-memcache technology, optimize-memcache technology large website Optimization-memcache technology Memory+cache Memory Cache Memcache Introduction Memcache is a set of distributed cache system ...

