Python-common modules

Python-common modules
The Python module is the highest level program organization unit in Python. It encapsulates program code and data for reuse. In reality, modules often correspond to Python program files. The essence is to use some code to implement a set of certain functions. This set can be one. the py file can also be a package (one folder contains one. py entry file) 1. import module import modulefrom module. xx import xxfrom module. xx import xx as renamefrom module. xx import * imports a py file. The Interpreter explains how to import a package to the import module according to the path sys. if this path exists, you can directly import it to sys. path has the desired path through sys. path. append ('path') 2. Open source module download and installation 1. yum, pip, apt-get2, source code compilation and installation: Python setup. py build Python setup install III. common modules 1. OS Module # used as a system-level operating system. popen ('id '). read () # Run the system command to obtain the returned result OS. system () # Unable to intercept OS when the returned result is returned. name # Return to the system platform. The Linux/Unix user is 'posix' OS. getenv () # Read the OS environment variable. putenv () # Set the OS environment variable. getcwd () # current working path OS. chdir () # change the current working directory OS. walk ('/root/') # recursive path File Processing mkfifo ()/mknod () # create a named pipeline/create a file system node remove ()/unlink () # delete file rename ()/renames () # rename file * stat () # Return File Information symlink () # Create Symbolic Link utime () # update timestamp tmpfile () # create and open ('W + B ') a new temporary file walk () # traverse all file name directories/folders under the directory tree chdir ()/fchdir () # change the current working directory/change the current job through a file descriptor As directory chroot () # change the root directory of the current process listdir () # list the files in the specified directory getcwd ()/getcwdu () # Return the current working directory/function is the same, however, a unicode object mkdir ()/makedirs () is returned. # create a directory/create a multi-level directory rmdir ()/removedirs () # delete a directory/delete a multi-level Directory Access/saccess () # verify the permission mode chmod () # change the permission mode chown ()/lchown () # change the owner and groupID to the same function, but do not track the link umask () # setting the default permission mode file descriptor to operate open () # underlying operating system open (for stability, use the standard built-in open () function) read ()/write () # reading/writing data based on file descriptors read part of the file content by size dup ()/dup2 () # copying file description symbols/functions are the same, but copying to another file descriptor Device number makedev () # create an original device number major ()/minor () from the major and minor device numbers # obtain the major/minor device number OS from the original device number. path module OS. path. expanduser ('~ /. Ssh/key') # All file paths in the home directory are separated by OS. path. basename () # Remove the directory path and return the file name OS. path. dirname () # Remove the file name and return the directory path OS. path. join () # combine the separated parts into a path name OS. path. spllt () # returns the (dirname (), basename () tuples OS. path. splitdrive () # returns the (drivename, pathname) tuples OS. path. splitext () # returns the (filename, extension) tuples of the OS. path. getatime () # Return the last access time OS. path. getctime () # returns the File Creation Time OS. path. getmtime () # returns the last file modification time OS. path. getsize () # returns the file size (bytes) to query the OS. path. exists () # Specify whether the path (file or directory) exists OS. path. isabs () # specify whether the path is an absolute path OS. path. isdir () # specify whether the path exists and is a directory OS. path. isfile () # specifies whether the path exists and is a file OS. path. islink () # specifies whether the path exists and is a symbolic link to the OS. path. ismount () # specify whether the path exists and is a mount point OS. path. samefile () # whether two pathnames point to the same file 2. sys module # provides interpreter-related operations sys. argv # List of command line parameters. The first element is the program path sys. exit (2) # The returned status of the exit script will be intercepted by try sys. exc_info () # obtain the exception class sys currently being processed. version # obtain the version information of the Python interpreter sys. maxint # maximum Int value 9223372042554775807sy S. maxunicode # maximum Unicode value sys. modules # Return the module field imported by the system. The key is the module name and the value is the module sys. path # Return the module's search path. during initialization, use the PYTHONPATH environment variable value sys. platform # Return OS platform name sys. stdout # standard output sys. stdin # standard input sys. stderr # output sys.exe c_prefix by mistake # Return to the platform's independent python file installation location sys. stdin. readline () # Read a row of sys from the standard input. stdout. write ("a") # screen output a 3. hashlib Module # General encryption for encryption-related operations import hashlibhash_md5 = hashlib. md5 () hash_md5.update ('admin') print (hash_md5.hexdigest ()) Hash = hashlib. sha512 () hash. update ('admin') print hash. hexdigest () # Although the above encryption algorithm is still very powerful, it has a defect, that is, it can be reversed by Credential stuffing. Therefore, it is necessary to add a custom key to the encryption algorithm and then encrypt it. Add custom key for encryption import hashlibhash = hashlib. md5 ('898oafs09f') hash. update ('admin') print hash. hexdigest () ultra-core encryption-continue to encrypt the key and content we created import hmach = hmac. new ('weiqi ') h. update ('hellowo') print h. hexdigest () 4, json, and pickle Module # used to serialize the data json, # json solves the problem of changing the sequence of simple data types to pickle, # The Json module that can store python complex data provides four functions: dumps, dump, loads, and loadpickle. The module provides four functions: dumps, dump, loads, and loadname_tra = json. dumps ('[1, 2, 3, 4, 5]') # dunmps puts serialized data in the memory with open ('jso N_tra ', 'wb') as f_json: f_json.write (name_tra) with open ('json _ tra1', 'wb ') as f_json: # dump directly writes serialized data in the memory to the file json. dump (a, f_json) time_now = datetime. datetime. now () pickle_mem = p. dumps (time_now) pickle_load = p. loads (pickle_mem) 5. subprocess module subprocess package to fork a sub-process and run an external program. The subprocess package defines several functions for creating sub-processes. Subprocess also provides some tools for managing standard streams and pipelines to use text communication between processes. Subprocess. call () the parent process waits for the child process to complete and returns the exit information (returncode, equivalent to Linux exit code) B = subprocess. call (['Ls', '-l']) B = subprocess. call ("ls-l", shell = True) shell = True, allowing shell commands to be strings. Subprocess. check_call () the parent process waits for the child process to complete and returns the 0 check exit information. If the returncode is not 0, the error subprocess. CalledProcessError is cited. This object contains the returncode attribute. try... Else T... To check subprocess. check_call (["ls", "-l"]) subprocess. check_call ("exit 1", shell = True) subprocess. check_output () the parent process waits for the completion of the Child process and returns the exit information of the output result from the child process to the standard. If the returncode is not 0, an error subprocess is cited. calledProcessError. This object contains the returncode attribute and output attribute. The output attribute is the output result of the standard output. try... Else T... To check 6. shuit Module # copy and move the shutil file. copyfile ('data. db', 'archive. db') # copy the object shutil. move ('/build/executables', 'installdir') # move files or directories 7. The logging Module # format and record logs # use logging. the basicConfig function configures the log output format and method, and logs are written to the file logging. basicConfig (level = logging. DEBUG, format = '% (asctime) s [line: % (lineno) d] % (levelname) s % (message) s ', datefmt = '% Y/% m/% d % H: % M: % s', filename = 'myapp. log', filemode = 'A') logging. debug ('this is debug message') l Ogging.info ('this is info message') logging. warning ('this is warning message') # logging. parameters of the basicConfig function: # datefmt: specifies the time format. strftime () # filename: Specify the log file name # filemode: The same as the file function. Specify the log file opening mode, 'w' or 'A' # level: Set the log level, the default value is logging. WARNING # format: Specify the output format and content. format can output a lot of useful information, as shown in the following example: # % (asctime) s: log printing time # % (levelname) s: print Log Level name # % (message) s: Print log information # % (levelno) s: Print log level value # % (lineno) d: current row number of the printed log # % (pathname) S: print the path of the current execution program, which is actually sys. argv [0] # % (filename) s: prints the current execution program name # % (funcName) s: prints the current function of the log # % (thread) d: print thread ID # % (threadName) s: Print thread name # % (process) d: print process ID # Write logs to files at the same time and print the logs to the screen logging. basicConfig (level = logging. DEBUG, format = '% (asctime) s % (filename) s [line: % (lineno) d] % (levelname) s % (message) s ', datefmt = '% Y/% m % d % H: % M: % s', filename = 'myapp. log', filemode = 'W') # print to the screen console = logging. streamHandler () console. setLeve L (logging. WARNING) formatter = logging. formatter ('% (name)-12 s: % (levelname)-8 s % (message) s') console. setFormatter (formatter) logging. getLogger (). addHandler (console) 8. the random module is used to obtain the random number random. choice (['apple', 'pear ', 'Banana']) # random list of a parameter random. sample (xrange (100), 10) # extract 10 random entries without repeating them. randrange (3, 7) # The random Integer Range does not include 7random. random () # random floating point number 9, time datetime module time module timestamp # seconds after January 1, January 1, 1970, namely: time. time () formatted string #2014- 11-11, namely: time. strftime ('% Y-% m-% D') Structured time # The tuples contain year, day, week, and so on... time. struct_time: time. localtime () import timetime. time () # timestamp [floating point] time. localtime () [1]-1 # int (time. time () # timestamp [whole s] time. strftime ('% Y-% m-% d % x') # format the output time import datetimedatetime. datetime. datetime. now () # current time datetime. datetime. now ()-datetime. timedelta (days = 5, hours = 3, seconds = 3, minutes = 2) # reduce time 10, re module regular match # The Pattern object is a compiled regular expression. A series of methods provided by Pattern can be used to search for text matching. Pattern = re. compile (strPattern [, flag]): flag is the matching mode, and re. I | re. M indicates that the pattern takes effect at the same time. Re. I (re. IGNORECASE): Ignore case-sensitive M (MULTILINE): MULTILINE mode, change the behavior of '^' and '$' to match (string [, pos [, endpos]) | re. match (pattern, string [, flags]): # match the result once # match (string [, pos [, endpos]) tries to match pattern starting from the pos subscript of string; if pattern can still be matched at the end, a Match object is returned. The default values of pos and endpos are 0 and len (string); re. match () flags is used to specify the matching mode when compiling pattern. # Re. match (pattern, string [, flags]) cannot specify pos and endpos values. Therefore, matching is performed at the beginning. Search (string [, pos [, endpos]) | re. search (pattern, string [, flags]): # match the result once # search (string [, pos [, endpos]) tries to match pattern from the string pos subscript, if the pattern can still be matched at the end, a Match object is returned. If the pattern cannot be matched, The pos is added with 1 and then matched again. If the pos = endpos still does not Match, None is returned. # Re. search (pattern, string [, flags]), used to match any starting position a = '100. 423.432.432 33.43.5.42 1.2.443.34 255.52.53.255 2.2.2.2 3.3.3.3 'pattern = re. compile (R' ([12]? \ D {1, 2} \.) {3} ([12]? \ D {1, 2}) ') print pattern. search (). group () split (string [, maxsplit]) | re. split (pattern, string [, maxsplit]): splits string Based on the matched substring and returns the list. Maxsplit is used to specify the maximum number of splits. If not specified, all splits are performed. # A = 'qwe123dsa43 ***** duo 342rew' # print re. split ('[\ d *] +', a) findall (string [, pos [, endpos]) | re. findall (pattern, string [, flags]): searches for strings and returns all matched substrings in the form of a list. P = re. compile (R' \ d + ') print p. findall ('one1two2three3four4 ') sub (repl, string [, count]) | re. sub (pattern, repl, string [, count]): Use repl to replace each matched substring in string, and then return the replaced string. Count is used to specify the maximum number of replicas. If not specified, all replicas are replaced. A = '192. 423.432.432 33.43.5.42 1.2.443.34 255.52.53.255 2.2.2.2 3.3.3.3 'pattern = re. compile (R' ([12]? \ D {1, 2} \.) {3} ([12]? \ D {1, 2}) ') print pattern. sub ('bibibi', a) finditer (string [, pos [, endpos]) | re. finditer (pattern, string [, flags]): searches for strings and returns an iterator that accesses each matching result (Match object) sequentially. A = '192. 423.432.432 33.43.5.42 1.2.443.34 255.52.53.255 2.2.2.2 3.3.3.3 'pattern = re. compile (R' ([12]? \ D {1, 2} \.) {3} ([12]? \ D {1, 2}) ') for I in pattern. finditer (a): print (I. group ())