Tornado custom distributed session framework and tornadosession framework

Tornado custom distributed session framework and tornadosession framework

 

1. The session Framework processes the request execution process:1. The server generates a random cookie string. 2. The browser sends a request and the server returns the cookie to the browser. 3. The server generates a dictionary. The dictionary key is cookie, and the value is another small dictionary. The dictionary is the dictionary set for the user. 4. When the user accesses the dictionary again, the cookie is sent to the server. After receiving the cookie, the server returns it to the dictionary to check whether the corresponding value is correct.

 

Ii. essential knowledge points

In the Tornado source code execution process, all our custom request methods will inherit a base class:Tornado. web. RequestHandler. This class has an extension point.Def initialize (). The method is executed before tornado executes the corresponding method of the request. Therefore, we can use this extension point to implement the session framework.

 

When performing session operations, you need to provide a knowledge point for special object members:

#! /Usr/bin/env python #-*-coding: UTF-8-*-class Foo (object): def _ getitem _ (self, key ): print '_ getitem _', key def _ setitem _ (self, key, value): print '_ setitem _', key, value def _ delitem _ (self, key): print '_ delitem _', key obj = Foo () result = obj ['k1 '] # obj ['k2'] = 'wupeiqi' # del obj ['k1']Special object-oriented Member

With this method, we can search, create, and delete sessions.

 

Iii. Code Implementation #! /Usr/bin/env python #-*-coding: UTF-8-*-import tornado. ioloopimport tornado. webfrom hashlib import sha1import OS, timesession_container ={} create_session_id = lambda: sha1 ('% s % s' % (OS. urandom (16), time. time ())). hexdigest () class Session (object): session_id = "_ sessionId _" def _ init _ (self, request): session_value = request. get_cookie (Session. session_id) # obtain the cookie requested by the client based on the custom value if not session _ Value: # if this is not the first request, a random string must be generated as the cookie self. _ id = create_session_id () else: self. _ id = session_value request. set_cookie (Session. session_id, self. _ id )#????? Def _ getitem _ (self, key): return session_container [self. _ id] [key] def _ setitem _ (self, key, value): # user = chenchap pwd = 123.com if session_container.has_key (self. _ id): session_container [self. _ id] [key] = value else: session_container [self. _ id] = {key: value} def _ delitem _ (self, key): del session_container [self. _ id] [key] class BaseHandler (tornado. web. requestHandler): def initialize (self): self. my_session = Session (self) class MainHandler (BaseHandler): def get (self): print self. my_session ['C _ user'] print self. my_session ['C _ card '] self. write ('index') class LoginHandler (BaseHandler): def get (self): self.render('login.html ', ** {'status': ''}) def post (self, * args, ** kwargs): username = self. get_argument ('name') password = self. get_argument ('pwd') if username = 'wupeiqi 'and password = '000000': self. my_session ['C _ user'] = 'chenchao' self. my_session ['C _ card '] = '2017. com 'self. redirect ('/Index') else: self.render('login.html', ** {'status': 'username or password error'}) settings = {'template _ path': 'template ', 'static _ path': 'static ', 'static _ url_prefix':'/static/', 'cookie _ secret': 'aiuasdhflashjdfoiuashdfiu', 'login _ url ': '/login'} application = tornado. web. application ([(r "/index", MainHandler), (r "/login", LoginHandler),], ** settings) if _ name _ = "_ main _": application. listen (1, 8888) tornado. ioloop. IOLoop. instance (). start ()Session_farm

 

 

Iv. distributed implementation

 

In the previous program code, we used a dictionarySession_container = {}To store client session information. The disadvantage is that data is easy to lose. Based on this shortcoming, we can store the dictionary on a dedicated server to store the data. For example:Redis and memcache. However, if you only use one server to do this, there will be other shortcomings, such as: downtime, high load. Therefore, we need to find a solution to this problem.

As shown in, we need to implement the simultaneous operation of multiple machines to store user session data. First, we need to name it a hash ring. The IP addresses and weights of several machines exist in this ring.

When the server generates a new cookie string for the user, we get this string and obtain a value through consistent hash algorithm. Then compare it with the weight set by the machine to determine the server on which the user's session information will be stored. Then, when a user initiates a request, the server calculates the cookie sent by the user and finds the server to which the session information has been saved.

#! /Usr/bin/env python # coding: utf-8import sysimport mathfrom bisect import bisectif sys. version_info> = (2, 5): import hashlib md5_constructor = hashlib. md5else: import md5 md5_constructor = md5.newclass HashRing (object): "consistent hash" def _ init _ (self, nodes): ''' initialize nodes: the initialized node contains the node and the corresponding node weight. By default, each node has 32 virtual nodes for the weight. By creating multiple virtual nodes, such as: nodes = [{'host ': '2017. 0.0.1: 100', 'weight': 1}, {'host': '123. 0.0.1: 8 001 ', 'weight': 2}, {'host': '2017. 0.0.1: 8002 ', 'weight': 1},] ''' self. ring = dict () self. _ sorted_keys = [] self. total_weight = 0 self. _ generate_circle (nodes) def _ generate_circle (self, nodes): for node_info in nodes: self. total_weight + = node_info.get ('weight', 1) # calculate the total weight for node_info in nodes: weight = node_info.get ('weight', 1) # obtain the weight of each node = node_info.get ('host', None) # obtain the host virtual _ of each node _ Node_count = math. floor (32 * len (nodes) * weight)/self. total_weight) for I in xrange (0, int (virtual_node_count): key = self. gen_key_thirty_two ('% s-% s' % (node, I) if self. _ sorted_keys. _ contains _ (key): raise Exception ('the node already exists. ') self. ring [key] = node self. _ sorted_keys.append (key) def add_node (self, node): ''' create a node: the node to be added, in the format of {'host': '127. 0.0.1: 8002 ', 'weight': 1}. The first element indicates the node, and the second element indicates the weight of the node. '''Node = node. get ('host', None) if not node: raise Exception ('node address cannot be blank. ') weight = node. get ('weight', 1) self. total_weight + = weight nodes_count = len (self. _ sorted_keys) + 1 virtual_node_count = math. floor (32 * nodes_count * weight)/self. total_weight) for I in xrange (0, int (virtual_node_count): key = self. gen_key_thirty_two ('% s-% s' % (node, I) if self. _ sorted_keys. _ contains _ (key): raise Exception ('the node already exists. ') self. ring [key] = node self. _ sorted_keys.append (key) def remove_node (self, node): ''' remove node: the node to be removed '127. 0.0.1: 8000 ''' for key, value in self. ring. items (): if value = node: del self. ring [key] self. _ sorted_keys.remove (key) def get_node (self, string_key): ''' get the node where string_key is located ''' pos = self. get_node_pos (string_key) if pos is None: return None return self. ring [self. _ sorted_keys [pos]. split (':') def get_node_pos (self, string_key): ''' get the index of the node where string_key is located ''' if not self. ring: return None key = self. gen_key_thirty_two (string_key) nodes = self. _ sorted_keys pos = bisect (nodes, key) # Calculate a return pos def gen_key_thirty_two (self, key) based on a list and encrypted string: m = md5_constructor () # md5 encryption m. update (key) return long (m. hexdigest (), 16) def gen_key_sixteen (self, key): B _key = self. _ hash_digest (key) return self. _ hash_val (B _key, lambda x: x) def _ hash_val (self, B _key, entry_fn): return (B _key [entry_fn (3)] <24) | (B _key [entry_fn (2)] <16) | (B _key [entry_fn (1)] <8) | B _key [entry_fn (0)]) def _ hash_digest (self, key): m = md5_constructor () m. update (key) return map (ord, m. digest () nodes = [{'host': '2017. 0.0.1: 100', 'weight': 15}, {'host': '123. 0.0.1: 100', 'weight': 20}, {'host': '123. 0.0.1: 8002 ', 'weight': 10},] ring = HashRing (nodes) result = ring. get_node ('sdgsdg1s56g156gge56rgerg4 ') print resultConsistent hash

We can set the weight of each machine to design the pressure and importance of each machine.

 

So. The code at the beginning can be modified as follows:

From hashlib import sha1import OS, timecreate_session_id = lambda: sha1 ('% s % s' % (OS. urandom (16), time. time ())). hexdigest () class Session (object): session_id = "_ sessionId _" def _ init _ (self, request): session_value = request. get_cookie (Session. session_id) if not session_value: self. _ id = create_session_id () else: self. _ id = session_value request. set_cookie (Session. session_id, self. _ id) def _ getitem _ (self, key): # according to self. _ id: Find the corresponding server IP address in the consistent Hasse # Find the corresponding redis server, such as: r = redis. strictRedis (host = 'localhost', port = 6379, db = 0) # Use the python redis api link # to obtain data, that is, # return self. _ redis. hget (self. _ id, name) def _ setitem _ (self, key, value): # according to self. _ id: Find the corresponding server IP address in the consistent Hasse # Use the python redis api link # Set session # self. _ redis. hset (self. _ id, name, value) def _ delitem _ (self, key): # according to self. _ id: Find the corresponding redis server # Use the python redis api link # Delete, that is, return self. _ redis. hdel (self. _ id, name)Session