Python's Memcache middleware

First, the Primer

Memcachedis a high-performance distributed memory object caching system for dynamic Web applications to mitigate database load by caching data in memory and reducing the number of times a database is read, thus increasing the speed of a dynamic database-driven Web site. The memcached is based on the hashmap of the stored key/value pair. Its daemon (daemon) is written in C, but the client can write in any language and communicate with the daemon through the memcached protocol.

Second, memcached Basic installation

Installing memacahed

由于memcached依赖于libevent；因此，还需要安装libevent,命令如下：[email protected]:/opt# wget http://memcached.org/latest    # 下载软件包[email protected]:/opt# tar zxf memcached-1.5.9.tar.gz[email protected]:/opt# cd memcached-1.5.9/[email protected]:/opt/memcached-1.5.9# ./configure[email protected]:/opt/memcached-1.5.9# make && make installPS：依赖libevent       yum install libevent-devel       apt-get install libevent-dev

Start memacahed

[email protected]:~# memcached -d -m 10 -u root 1 10.211.55.20 -p 1200 -c 256 -P /tmp/memcached.pid[email protected]:~# netstat -anpt | grep 1200tcp        0      0 0.0.0.0:1200            0.0.0.0:*               LISTEN      9129/memcachedtcp6       0      0 :::1200                 :::*                    LISTEN      9129/memcached参数说明:    -d 是启动一个守护进程    -m 是分配给Memcache使用的内存数量，单位是MB    -u 是运行Memcache的用户    -l 是监听的服务器IP地址    -p 是设置Memcache监听的端口,最好是1024以上的端口    -c 选项是最大运行的并发连接数，默认是1024，按照你服务器的负载量来设定    -P 是设置保存Memcache的pid文件

memcached command

存储命令: set/add/replace/append/prepend/cas获取命令: get/gets其他命令: delete/stats..

Third, Python operation memcached

Install API

python操作Memcached使用Python-memcached模块下载安装：https://pypi.python.org/pypi/python-memcached

First operation (small excitement)

#!/usr/bin/env python# -*- coding: utf-8 -*-import memcachemc = memcache.Client(['10.211.55.20:1200'],debug=True)   # 连接memcached服务器mc.set("foo","bar")ret = mc.get("foo")print(ret)# Ps：debug = True 表示运行出现错误时，现实错误信息，上线后移除该参数。

Natural support for clusters

Python-MemcachedThe module natively supports cluster operations, with the principle that memory maintains a list of hosts, and that the weights of hosts in the cluster are proportional to the number of times the host repeats in the list

     主机    权重    1.1.1.1   1    1.1.1.2   2    1.1.1.3   1 那么在内存中主机列表为：    host_list = ["1.1.1.1", "1.1.1.2", "1.1.1.2", "1.1.1.3", ]

If the user is to create a key-value pair (for example: K1 = "V1") in memory, perform a step:

• Convert K1 into a number based on the algorithm
• The number and host list length are remainder, and a value of N (0<=n< list length) is obtained.
• The value given in the second step in the host list is the index host, listed as: Host_list[n]
• Connect the host acquired in step 3rd, place k1 = "V1" In the server's memory

mc = memcache.Client([('1.1.1.1:12000', 1), ('1.1.1.2:12000', 2), ('1.1.1.3:12000', 1)], debug=True) mc.set('k1', 'v1')

Add

Add a key-value pair, repeat the add operation exception if a key already exists

#!/usr/bin/env python# -*- coding: utf-8 -*-import memcachemc = memcache.Client(['10.211.55.20:1200'],debug=True)mc.set("foo","bar")mc.add("foo","bar")ret = mc.get("foo")print(ret)# 结果/usr/local/bin/python3.6 /Users/xcn/PycharmProjects/oldboy/中间件开发/memcached-1.pybarMemCached: while expecting 'STORED', got unexpected response 'NOT_STORED'

Repiace

repiaceModifies a key address, if key does not exist, the exception

#!/usr/bin/env python# -*- coding:utf-8 -*-import memcache mc = memcache.Client(['10.211.55.20:1200'], debug=True)# 如果memcache中存在kkkk，则替换成功，否则一场mc.replace('kkkk','999')

Set and Set_multi

• set: Sets a key-value pair that is created if key does not exist, and modifies if key exists
• set_multi: Sets a key-value pair that is created if key does not exist, and modifies if key is stored

#!/usr/bin/env python# -*- coding:utf-8 -*-import memcache mc = memcache.Client(['10.211.55.20:1200'], debug=True) mc.set('key0', 'john') mc.set_multi({'key1': 'val1', 'key2': 'val2'})

Delete and Delete_multi

• delete: Deletes a specified key-value pair in memcached
• delete_multi: Deletes the specified number of key-value pairs in memcached

#!/usr/bin/env python# -*- coding:utf-8 -*-import memcache mc = memcache.Client(['10.211.55.20:1200'], debug=True) mc.delete('key0')mc.delete_multi(['key1', 'key2'])

Get and Get_multi

• get: Gets a key-value pair
• get_multi: Gets one more key-value pair

#!/usr/bin/env python# -*- coding:utf-8 -*-import memcache mc = memcache.Client(['10.211.55.20:1200'], debug=True) val = mc.get('key0')item_dict = mc.get_multi(["key1", "key2", "key3"])

Append and prepend

• append: Modifies the value of the specified key to append content after the value
• prepend: Modifies the value of the specified key to insert the contents before the value

#!/usr/bin/env python# -*- coding:utf-8 -*-import memcache mc = memcache.Client(['10.211.55.20:1200'], debug=True)# k1 = "v1" mc.append('k1', 'after')# k1 = "v1after" mc.prepend('k1', 'before')# k1 = "beforev1after"

DECR and INCR

• incr: Self-increment, add a value in memcached n (n default = 1)
• decr: self-subtract, reduce a value in memcached by n (default = 1)

#!/usr/bin/env python# -*- coding:utf-8 -*-import memcache mc = memcache.Client(['10.211.55.20:1200'], debug=True)mc.set('k1', '777') mc.incr('k1')# k1 = 778 mc.incr('k1', 10)# k1 = 788 mc.decr('k1')# k1 = 787 mc.decr('k1', 10)# k1 = 777

Gets and CAs

such as the number of store items, assuming that the value is saved to memcached, product_count=100
A user Refresh page reads from memcached to product_count=100
b User Refresh page read from memcached to product_count=100

If both A and B users purchase goods

A user modified the number of items remaining product_count=899
b User modified the number of items remaining product_count=899

As a result, the data in the cache is not correct, two users after the purchase of goods, the product surplus or 899
If you use Python's set and get to manipulate the above procedure, the program will look like the above situation!

If you want to avoid this situation, just use get and CAS, such as:

#!/usr/bin/env python# -*- coding:utf-8 -*-import memcachemc = memcache.Client(['10.211.55.20:1200'], debug=True, cache_cas=True) v = mc.gets('product_count')# ...# 如果有人在gets之后和cas之前修改了product_count，那么，下面的设置将会执行失败，剖出异常，从而避免非正常数据的产生mc.cas('product_count', "899")

Note: In essence, each time you execute a GET, a word increment is obtained from memcached, and when the value of Get is modified by CAS, it is compared with the self-increment in the previously obtained and memcache, and if it is equal, it can be committed, if not equal, That means that between the get and the CAs execution, another person executes the gets (the specified value of the cache is obtained), so that if abnormal data is possible, no modification is allowed.

Python's Memcache middleware