Python practical code snippets

This article describes how to collect and paste python code snippets, this article collects useful code such as getting all the subclasses of a class, calculating the running time, simple use of SQLAlchemy, and implementing enumeration similar to Java or C. For more information, see Obtains all subclasses of a class.

The code is as follows:

Def itersubclasses (cls, _ seen = None ):
"Generator over all subclasses of a given class in depth first order ."""
If not isinstance (cls, type ):
Raise TypeError (_ ('itersubclasses must be called'
'New-style classes, not %. 100r') % cls)
_ Seen = _ seen or set ()
Try:
Subs = cls. _ subclasses __()
Failed T TypeError: # fails only when cls is type
Subs = cls. _ subclasses _ (cls)
For sub in subs:
If sub not in _ seen:
_ Seen. add (sub)
Yield sub
For sub in itersubclasses (sub, _ seen ):
Yield sub

Simple thread cooperation

The code is as follows:

Import threading
Is_done = threading. Event ()
Consumer = threading. Thread (
Target = self. consume_results,
Args = (key, self. task, runner. result_queue, is_done ))
Consumer. start ()
Self. duration = runner. run (
Name, kw. get ("context", {}), kw. get ("args ",{}))
Is_done.set ()
Consumer. join () # The main thread is blocked until the consumer operation ends.

For more information, threading. Event () can also be replaced with threading. Condition (). condition includes running y (), wait (), and policyall (). Explanation:

The code is as follows:

The wait () method releases the lock, and then blocks until it is awakened by a wrong Y () or policyall () call for the same condition variable in another thread. once awakened, it re-acquires the lock and returns. it is also possible to specify a timeout.
The specified Y () method wakes up one of the threads waiting for the condition variable, if any are waiting. The specified yall () method wakes up all threads waiting for the condition variable.
Note: the specified Y () and policyall () methods don't release the lock; this means that the thread or threads awakened will not return from their wait () call immediately, but only when the thread that called policy () or policyall () finally relinquishes ownership of the lock.

The code is as follows:

# Consume one item
Cv. acquire ()
While not an_item_is_available ():
Cv. wait ()
Get_an_available_item ()
Cv. release ()
# Produce one item
Cv. acquire ()
Make_an_item_available ()
Cv. y Y ()
Cv. release ()

Computing run time

The code is as follows:

Class Timer (object ):
Def _ enter _ (self ):
Self. error = None
Self. start = time. time ()
Return self
Def _ exit _ (self, type, value, tb ):
Self. finish = time. time ()
If type:
Self. error = (type, value, tb)
Def duration (self ):
Return self. finish-self. start
With Timer () as timer:
Func ()
Return timer. duration ()

Metadata

The parameters received by the _ new _ () method are:
The object of the class to be created;
Class name;
The set of parent classes inherited by the class;
Class method set;

The code is as follows:

Class ModelMetaclass (type ):
Def _ new _ (cls, name, bases, attrs ):
If name = 'model ':
Return type. _ new _ (cls, name, bases, attrs)
Mappings = dict ()
For k, v in attrs. iteritems ():
If isinstance (v, Field ):
Print ('found mapping: % s ==> % s' % (k, v ))
Mappings [k] = v
For k in mappings. iterkeys ():
Attrs. pop (k)
Attrs ['_ table _'] = name # assume that the table name and class name are consistent.
Attrs ['_ Ings _'] = mappings # save the ing between attributes and columns
Return type. _ new _ (cls, name, bases, attrs)
Class Model (dict ):
_ Metaclass _ = ModelMetaclass
Def _ init _ (self, ** kw ):
Super (Model, self). _ init _ (** kw)
Def _ getattr _ (self, key ):
Try:
Return self [key]
Failed T KeyError:
Raise AttributeError (r "'model' object has no attribute '% s'" % key)
Def _ setattr _ (self, key, value ):
Self [key] = value
Def save (self ):
Fields = []
Params = []
Args = []
For k, v in self. _ mappings _. iteritems ():
Fields. append (v. name)
Params. append ('? ')
Args. append (getattr (self, k, None ))
SQL = 'Insert into % s (% s) values (% s) '% (self. _ table __,','. join (fields ),','. join (params ))
Print ('SQL: % s' % SQL)
Print ('args: % s' % str (ARGS ))
Class Field (object ):
Def _ init _ (self, name, column_type ):
Self. name = name
Self. column_type = column_type
Def _ str _ (self ):
Return '<% s: % s>' % (self. _ class _. _ name __, self. name)
Class StringField (Field ):
Def _ init _ (self, name ):
Super (StringField, self). _ init _ (name, 'varchar (100 )')
Class IntegerField (Field ):
Def _ init _ (self, name ):
Super (IntegerField, self). _ init _ (name, 'bigint ')
Class User (Model ):
# Define the ing of class attributes to columns:
Id = IntegerField ('id ')
Name = StringField ('Username ')
Email = StringField ('email ')
Password = StringField ('password ')
# Create an instance:
U = User (id = 12345, name = 'Michael ', email = 'Test @ orm.org', password = 'My-pwd ')
# Save to database:
U. save ()

The output is as follows:

The code is as follows:

Found model: User
Found mapping: email ==>
Found mapping: password =>
Found mapping: id =>
Found mapping: name =>
SQL: insert into User (password, email, username, uid) values (?,?,?,?)
ARGS: ['My-pwd', 'Test @ orm.org ', 'Michael', 12345]

Simple use of SQLAlchemy

The code is as follows:

# Import:
From sqlalchemy import Column, String, create_engine
From sqlalchemy. orm import sessionmaker
From sqlalchemy. ext. declarative import declarative_base
# Create a base class for an object:
Base = declarative_base ()
# Define a User object:
Class User (Base ):
# Table name:
_ Tablename _ = 'user'
# Table structure:
Id = Column (String (20), primary_key = True)
Name = Column (String (20 ))
# Initialize the database connection:
Engine = create_engine ('MySQL + mysqlconnector: // root: password @ localhost: 3306/test') # 'database type + database driver name: // user name: password @ machine address: port number/database name'
# DBSession creation type:
DBSession = sessionmaker (bind = engine)
# Create a new User object:
New_user = User (id = '5', name = 'Bob ')
# Add to session:
Session. add (new_user)
# Submit to save to the database:
Session. commit ()
# Create a Query. the filter is the where condition. the unique row is returned when one () is called. if all () is called, all rows are returned:
User = session. query (User). filter (User. id = '5'). one ()
# Disable session:
Session. close ()

Simple use of WSGI and Flask

The code is as follows:

From wsgiref. simple_server import make_server
Def application (environ, start_response ):
Start_response ('2017 OK ', [('content-type', 'text/html')])
Return 'Hello, web! '
# Create a server with an empty IP address and port 8000. the processing function is application:
Httpd = make_server ('', 8000, application)
Print "Serving HTTP on port 8000 ..."
# Start listening for HTTP requests:
Httpd. serve_forever ()

After learning about the WSGI framework, we found that a Web App writes a WSGI processing function to respond to each HTTP request.
But how to handle HTTP requests is not a problem. The question is how to handle 100 different URLs.
The simplest and most common idea is to retrieve the HTTP request information from the environ variable and then judge it one by one.

The code is as follows:

From flask import Flask
From flask import request
App = Flask (_ name __)
@ App. route ('/', methods = ['GET', 'post'])
Def home ():
Return 'home'
@ App. route ('/signin', methods = ['get'])
Def signin_form ():
Return ''''''
@ App. route ('/signin', methods = ['post'])
Def signin ():
# Read the form content from the request object:
If request. form ['username'] = 'admin' and request. form ['password'] = 'password ':
Return 'Hello, admin! '
Return 'bad username or password .'
If _ name _ = '_ main __':
App. run ()

Format and display json

The code is as follows:

Print (json. dumps (data, indent = 4 ))
# Or
Import pprint
Pprint. pprint (data)

Implement enumeration similar to Java or C

The code is as follows:

#! /Usr/bin/env python
#-*-Coding: UTF-8 -*-
Import itertools
Import sys
Class ImmutableMixin (object ):
_ Inited = False
Def _ init _ (self ):
Self. _ inited = True
Def _ setattr _ (self, key, value ):
If self. _ inited:
Raise Exception ("unsupported action ")
Super (ImmutableMixin, self). _ setattr _ (key, value)
Class EnumMixin (object ):
Def _ iter _ (self ):
For k, v in itertools. imap (lambda x: (x, getattr (self, x), dir (self )):
If not k. startswith ('_'):
Yield v
Class _ RunnerType (ImmutableMixin, EnumMixin ):
SERIAL = "serial"
CONSTANT = "constant"
CONSTANT_FOR_DURATION = "constant_for_duration"
RPS = "rps"
If _ name __= = "_ main __":
Print _ RunnerType. CONSTANT

Specify permissions when creating a file

The code is as follows:

Import OS
Def write_to_file (path, contents, umask = None ):
"Write the given contents to a file
: Param path: Destination file
: Param contents: Desired contents of the file
: Param umask: Umask to set when creating this file (will be reset)
"""
If umask:
Saved_umask = OS. umask (umask)
Try:
With open (path, 'w') as f:
F. write (contents)
Finally:
If umask:
OS. umask (saved_umask)
If _ name _ = '_ main __':
Write_to_file ('/home/kong/tmp', 'test', 31)
# Then you will see a file is created with permission 640.
# Warning: If the file already exists, its permission will not be changed.
# Note: For file, default all permission is 666, and 777 for directory.

Multi-process concurrent execution

The code is as follows:

Import multiprocessing
Import time
Import OS
Def run (flag ):
Print "flag: % s, sleep 2 s in run" % flag
Time. sleep (2)
Print "% s exist" % flag
Return flag
If _ name _ = '_ main __':
Pool = multiprocessing. Pool (3)
Iter_result = pool. imap (run, xrange (6 ))
Print "sleep 5s \ n"
Time. sleep (5)
For I in range (6 ):
Try:
Result = iter_result.next (1, 600)
Handle T multiprocessing. TimeoutError as e:
Raise
Print result
Pool. close ()
Pool. join ()

Auto-fill function parameters during runtime

The code is as follows:

Import decorator
Def default_from_global (arg_name, env_name ):
Def default_from_global (f, * args, ** kwargs ):
Id_arg_index = f. func_code.co_varnames.index (arg_name)
Args = list (args)
If args [id_arg_index] is None:
Args [id_arg_index] = get_global (env_name)
If not args [id_arg_index]:
Print ("Missing argument: -- % (arg_name) s" % {"arg_name": arg_name })
Return (1)
Return f (* args, ** kwargs)
Return decorator. decorator (default_from_global)
# The following is a modifier that can be used in functions that require automatic parameter filling. Functions:
# If the deploy_id parameter of the function is not passed, obtain it from the environment variable (call the custom get_global function)
With_default_deploy_id = default_from_global ('ploy _ id', ENV_DEPLOYMENT)

Nested decorator

The validator function is decorated with func1 and receives parameters (* arg, ** kwargs) when using func1, while func1 is decorated with func2 (which is actually the scenario function in Rally) to add the validators attribute to func2, is a list of functions. the Function receiving parameters include config, clients, and task. These functions finally call func1 and pass in parameters (config, clients, task, * args, ** kwargs). Therefore, when func1 is defined, the parameters are (config, clients, task, * arg, ** kwargs)
The final result is that func2 has many decorators, each of which receives their own parameters and performs some verification work.

The code is as follows:

Def validator (fn ):
"Decorator that constructs a scenario validator from given function.
Decorated function shocould return ValidationResult on error.
: Param fn: function that performs validation
: Returns: rally scenario validator
"""
Def wrap_given (* args, ** kwargs ):
"Dynamic validation decorator for scenario.
: Param args: the arguments of the decorator of the benchmark scenario
Ex. @ my_decorator ("arg1"), then args = ('arg1 ',)
: Param kwargs: the keyword arguments of the decorator of the scenario
Ex. @ my_decorator (kwarg1 = "kwarg1"), then kwargs = {"kwarg1": "kwarg1 "}
"""
Def wrap_validator (config, clients, task ):
Return (fn (config, clients, task, * args, ** kwargs) or
ValidationResult ())
Def wrap_scenario (scenario ):
Wrap_validator.permission = getattr (fn, "permission ",
Consts. EndpointPermission. USER)
If not hasattr (scenario, "validators "):
Scenario. validators = []
Scenario. validators. append (wrap_validator)
Return scenario
Return wrap_scenario
Return wrap_given

Common inspect database usage

Inspect. getargspec (func) gets the name and default value of the function parameter, and returns a quad-tuple (args, varargs, keywords, defaults), where:
Args is a list of parameter names;
Varargs and keywords are the variable names of "*" and;
ULTS is the default parameter list;

Inspect. getcallargs (func [, * args] [, ** kwds]) binds function parameters. Returns the input parameter Dictionary of the bound function.

Private attributes and functions in python

Python treats a variable that starts with two or more underscores and does not end with two or more underscores as a private variable. Private variables are converted to long format (public) before code generation. this process is called "Private name mangling ", for example, the _ private identifier in Class A will be converted to _ A _ private, but when the class names are all named below, Python will no longer execute rolling. Besides, although it is called a private variable, it may still be accessed or modified (using _ classname _ membername). Therefore, the following is a summary:

Whether it is a single or double-underline member, it is hoped that the external program developers do not directly use these member variables and these member functions, double underscores can be used more directly in syntax to avoid errors, but they can still be accessed by _ class name_member name. It may be easier to debug a single underline dynamically. as long as members of the project team start with an underline and do not use it directly, it may be better to use a single underline.