April 16 Python Learning summary dbutils module, ORM and Cookie, session, token

First, Dbutils module introduction

The Dbutils suite is realized as a Python package containing and subsets of modules, one for use with arbitrary DB-API 2 m Odules, the other one is use with the classic Pygresql module.

The dependencies of the modules in the Universal Db-api 2 variant is as indicated in the following diagram:

The dependencies of the modules in the classic Pygresql variant is similar:

Operation of the PostgreSQL database using the Pygresql module

Dbutils is a set of Python database connection pool packages and allows thread-safe wrapping of non-thread-safe database interfaces. Dbutils from Webware for Python.

Dbutils provides two external interfaces:    PERSISTENTDB: Provides a thread-specific database connection and automatically manages the connection.    POOLEDDB: Provides a database connection that can be shared between threads and automatically manages connections.

The test results show that the speed of Persistentdb is the highest, but in some special cases, the connection process of the database may be unusually slow, while the pooleddb at this time can provide a relatively short average connection time management mode.

In addition, the actual use of the database driver is also dependent, such as SQLite database can only use PERSISTENTDB as a connection pool.

Function
steadydb.py: for stabilizing database connections

pooleddb.py: Connection pooling

persistentdb.py: Maintaining a persistent database connection (persistent connection)

simplepooleddb.py: Simple Connection Pool

SIMPLEPOOLEDDB:

DBUTILS.SIMPLEPOOLEDDB is a very simple database connection pooling implementation. He lacks many features than the perfect POOLEDDB module. DBUTILS.SIMPLEPOOLEDDB is essentially similar to the miscutils.dbpool part of this webware. You can think of it as a demo program.

Steadydb:

Dbutils.steadydb is a module that implements a "tough" database connection, based on the original connection established by Db-api 2. a "tough" connection means reconnecting when the connection is closed or when the usage limit is limited.

A typical example is when the database restarts and your program is still running and requires access to the database, or when your program is connected to a remote database behind a firewall and the firewall is restarted with a status of missing.

Generally you don't need to use steadydb directly, it just provides basic services to the next two modules, Persistentdb and POOLEDDB.

PERSISTENTDB:

Dbutils.persistentdb implements a tough, thread-safe, stubborn database connection using the Db-api 2 module.

When a thread first opens a database connection, a connection is opened and used only by that thread. When the thread closes the connection, the connection continues to open for the next request by this thread to use the already open connection. The connection is automatically closed when the thread dies.

Simply put, PERSISTENTDB attempts to reuse the database connection to improve the database access performance of the threaded program, and he ensures that the connection is not shared between threads.

As a result, persistentdb can work well when the underlying DB-API module is not thread-safe, and he will also avoid problems when other threads change the database session or use multi-statement transactions.

Pooleddb

DBUTILS.POOLEDDB implements a strong, thread-safe, cached, reusable database connection using any DB-API 2 module.

POOLEDDB can share open database connections between different threads. This is when you connect and specify the maxshared parameter, and the underlying DB-API 2 interface is thread safe, but you can still use a private database connection instead of sharing connections between threads. In addition to shared connections, you can set up a connection pool of at least mincached and allow up to maxcached connections, which can be used for both private and shared connection pools. When a thread closes a non-shared connection, it is returned to the free connection pool for the next use.

If the underlying DB-API module is non-thread-safe, the thread lock ensures that the use of POOLEDDB is thread-safe. So you don't have to worry about this, but you have to be careful when you use a private connection to alter a database session or execute a multi-command transaction.

Which one should I choose?

Both Persistentdb and POOLEDDB are designed to reuse database connections to improve performance and to keep the database stable.

So choose which module, you can refer to the above explanation. Persistentdb will maintain a certain number of connections for frequent use. In this case you always keep a fixed number of connections. If your program frequently starts and shuts down threads, it is best to use POOLEDDB. Better tuning will be mentioned later, especially when using the thread-safe Db-api 2 module.

Of course, the interfaces of the two modules are very similar, and you can easily convert between them and see which one is better.

Official Guide: https://cito.github.io/w4py-olde-docs/Webware/DBUtils/Docs/UsersGuide.html

Use

The connection pool object is initialized only once and can generally be used as a module-level code to ensure. Example of PERSISTENTDB connection:

Import Dbutils.persistentdb
Persist=dbutils.persistentdb.persistentdb (Dbpai=mysqldb,maxusage=1000,**kwargs)
Here the parameter Dbpai refers to the use of the underlying database module, compatible with DB-API. The maxusage is the maximum number of connections used and is referenced in the official example. The subsequent **kwargs is the argument that is actually passed to MySQLdb.

Get connection: Conn=persist.connection () A connection that is actually programmed to close conn.close () directly turns the connection back to the connection pool.

Pooleddb use the same method as Persistentdb, except that the parameters are different.

• • DBAPI: Database Interface
  • mincached: Number of empty connections opened at startup
  • Maxcached: Connection pool maximum number of available connections
  • Maxshared: Connection pooling maximum number of shareable connections
  • MaxConnections: Maximum allowable number of connections
  • Blocking: block If maximum quantity is reached
  • Maxusage: Maximum number of multiplexing times for a single connection
  • Setsession: A preparation session for passing to the database, such as ["Set name Utf-8″].

　　PERSISTENTDB: A connection is created for each thread, and the thread does not close even if the Close method is called, but simply puts the connection back into the connection pool for its own thread to use again. The connection shuts down automatically when the thread terminates

POOLEDDB: Create a batch of connections to the connection pool for all threads to share with.

 fromDbutils.persistentdbImportPersistentdbImportPymysqlpool=Persistentdb (creator=pymysql,#modules that use linked databasesMaxusage=none,#the maximum number of times a link is reused, none means unlimitedSetsession=[],#a list of commands to execute before starting the session. ping=0,#Ping the MySQL server to check if the service is available. closeable=False,#If False, Conn.close () is actually ignored for the next use, and the link is automatically closed when the thread is closed. If True, Conn.close () closes the link, and then calls Pool.connection again with an error, because the connection is actually closed (pool.steady_connection () can get a new link)Threadlocal=none,#This thread is exclusive to the object that holds the linked object, if the linked object is resethost='127.0.0.1', Port=3306, the user='Root', Password='123456', Database='Test', CharSet='UTF8')deffunc (): Conn= Pool.connection (shareable=False) Cursor=conn.cursor () cursor.execute ('SELECT * from user') Result=Cursor.fetchall ()Print(Result) cursor.close () conn.close ()if __name__=='__main__': func ()

Persistentdb

ImportPymysql fromDbutils.pooleddbImportPooleddbpool=Pooleddb (creator=pymysql,#modules that use linked databasesMaxconnections=6,#the maximum number of connections allowed for a connection pool, 0 and none means no limit on the number of connectionsmincached=2,#at least 0 of the free links created in the link pool are not created when initializingMaxcached=5,#most idle links in the link pool, 0 and none are not limitedMaxshared=3,#The maximum number of links shared in a linked pool, 0 and none means sharing all. PS: Useless, because Pymysql and mysqldb modules such as threadsafety are 1, all values regardless of set to how much, _maxcached forever is 0, so forever is all links are shared. Blocking=true,#If there are no available connections in the connection pool, wait is blocked. True, wait, False, not wait and then errorMaxusage=none,#the maximum number of times a link is reused, none means unlimitedSetsession=[],#a list of commands to execute before starting the session. ping=0,#Ping the MySQL server to check if the service is available. host='127.0.0.1', Port=3306, the user='Root', Password='123456', Database='Test', CharSet='UTF8')deffunc ():#detects if the number of currently running connections is less than the maximum number of links, if not less than: Waits or reports raise Toomanyconnections exception    #otherwise    #gets the link steadydbconnection in the link created when the initialization is prioritized.     #The Steadydbconnection object is then encapsulated into the pooleddedicateddbconnection and returned.     #if the link you initially created is not linked, create a Steadydbconnection object and encapsulate it in pooleddedicateddbconnection and return.     #once the link is closed, the connection is returned to the connection pool for subsequent threads to continue to use. conn =pool.connection ()#print (th, ' link was taken away ', Conn1._con)    #print (Th, ' currently in the pool ', Pool._idle_cache, ' \ r \ n ')cursor=conn.cursor () cursor.execute ('SELECT * from user') Result=Cursor.fetchall ()Print(Result) conn.close ()if __name__=='__main__': func ()

POOLEDDB

　　

　　cmd Download and install Dbutils module: easy_install-u dbutils　

Second, ORM

ORM Full Name "Object Relational Mapping", that is, object-relational mapping , is to map a row of the relational database to an object, that is, a class corresponding to a table, so that the code is simpler to write, do not directly manipulate the SQL statement. When we need to operate the database, it is necessary to connect the data, call SQL statements, execute SQL statements and other operations, ORM the database table, fields, rows and our object-oriented programming class and its methods, properties, such as one by one corresponding to the part of the operation is packaged up, Program apes do not need to understand the SQL statement to complete the operation of the database.

To write an ORM framework, all classes can only be defined dynamically, because only the user can define the corresponding class based on the structure of the table, using Metaclass.

　

1, in the instantiation of a User object, you can User=user (name= ' lqz ', password= ' 123 ')

2, can also User=user ()

user[' name ']= ' LQZ '
user[' password ']= ' 123 '
3, can also User=user ()

User.name= ' LQZ '
user.password= ' Password '
The first two, can be achieved by inheriting the dictionary dict, the third, with GetAttr and SetAttr

__GETATTR__ Intercept point number operation. This method is called with the property name as a string when an undefined property name and instance are performed in a point-number operation. If the inheritance tree can find the property, this method is not called
__SETATTR__ will intercept the assignment statements for all attributes. If this method is defined, SELF.ARRT = value becomes self,__setattr__ ("attr", value). This needs to be noted. When an attribute is assigned within the __setattr__ method, self.attr = value cannot be used, because he will call self,__setattr__ ("attr", value) again, resulting in an infinite recursive loop that eventually results in a stack overflow exception. Any instance property should be assigned by indexing an attribute dictionary, that is, using self.__dict__[' name ' = value

Specific implementation reference: 80340978

Third, Cookie, session, Tokensession

The Chinese translation of the session is a "conversation", and when a user opens a web app, it creates a session with the Web server. The server uses the session to temporarily save the user's information on the server, and the session will be destroyed after the user leaves the site. This user information is stored in a more secure way than a cookie, but the session has a flaw: if the Web server is load balanced, the session is lost when the next operation requests to another server.

Cookies

cookies are data stored on the local terminal. The cookie is generated by the server, sent to the browser, and the browser saves the cookie in KV form in a text file in a directory, and the next time the same website is requested, the cookie is sent to the server. Because the cookie is present on the client, the browser has added some restrictions to ensure that the cookie is not used maliciously and does not occupy too much disk space, so the number of cookies per domain is limited.

The composition of a cookie consists of a name (key), a value, a valid domain (domain), a path (the path of a field that is generally set to global: "\"), an expiration time, and a security flag (when specified, the cookie is sent to the server (HTTPS) only when the SSL connection is used). Here is a simple example of JS using cookies:

A cookie is generated when a user logs on:

Document.cookie = "id=" +result.data[' id ']+ "; path=/";

Document.cookie = "Name=" +result.data[' name ']+ "; path=/";

Document.cookie = "avatar=" +result.data[' Avatar ']+ "; path=/";

Use the following parsing when using cookies:

var cookie = Document.cookie;var Cookiearr = Cookie.split (";"); var user_info = {};for (var i = 0; i < cookiearr.length; i++) {

User_info[cookiearr[i].split ("=") [0]] = cookiearr[i].split ("=") [1];

}

$ (' #user_name '). Text (user_info[' name ');

$ (' #user_avatar '). attr ("src", user_info[' Avatar '));

$ (' #user_id '). Val (user_info[' id ');

Token

Token means "tokens", which is the authentication method of the user identity, the simplest token composition: UID (user's unique identity), time (timestamp of the current time), sign (signature, the hash algorithm is compressed into a long hexadecimal string by the first several + salts of token, Can prevent malicious third party stitching token request server). You can also put the invariant parameters into token, avoid multiple check the library

The difference between a cookie and a session

1. The cookie data is stored on the client's browser and the session data is placed on the server.

2, the cookie is not very safe, others can analyze the cookie stored in the local and cookie deception
Consider that security should use the session.

3. Session will be saved on the server for a certain period of time. When access is increased, it will be more likely to occupy your server's performance
The cookie should be used in consideration of mitigating server performance.

4, a single cookie can not save more than 4K of data, many browsers restrict a site to save up to 20 cookies.

5, so personal advice:
Storing important information such as login information as session
Additional information can be placed in a cookie if it needs to be retained

The difference between token and session

The session and OAuth token are not contradictory, as authentication token security is better than the session, because each request has a signature can also prevent monitoring and replay attacks, and the session must rely on the link layer to ensure the security of communications. As stated above, if you need to implement a stateful session, you can still add sessions to save some state on the server side

Apps often use restful APIs to deal with servers. Rest is stateless, that is, the app does not need to save the session with a cookie like browser, so it is enough to label yourself with session tokens, session/state by the logic of the API server. If your backend is not stateless's Rest API, you may need to save the session in the app. You can embed WebKit in the app and use a hidden browser to manage the cookie session.

   session  is an HTTP storage mechanism designed to provide a durable mechanism for stateless HTTP. The so-called session  certification is simply the user  information stored in the session , because sid  unpredictability, for the moment is considered safe. This is a means of authentication.   TOKEN&NBSP; If you refer to an OAuth token  or similar mechanism, provide authentication and authorization , authentication is for users, authorization is for app  . The purpose is to let . The  token here are the only ones. Can not be transferred to other  app, nor can it be transferred to other users.   turned around and said session . The session only provides a simple certification, that is, the  sid, that is, the full rights of this  user. Is strictly confidential, this data should only be kept in the presence of the site, should not be shared with other websites or third-party apps.   So simply, if your user data may need to be shared with a third party, or allow a third party to call  API  interface, use  Token . If Forever is only own website, own  app, with what does not matter.

Token is a token , such as when you authorize (login) a program, he is a basis to determine whether you have authorized the software, Cookie is written in a TXT file on the client, including your login information and so on, so you next time on a website, will automatically call the cookie automatically login user name, session and cookie is similar, only the session is written on the server side of the file, also need to write to the client cookie file, but the file is your browser number. The state of the session is stored on the server side, the client has only the session ID, and the token state is stored on the client.

April 16 Python Learning summary dbutils module, ORM and Cookie, session, token