Python ConfigParser, shutil, subprocess, ElementTree module, configparser

Python ConfigParser, shutil, subprocess, ElementTree module, configparser

　　　　　　　　　　　　　　ConfigParser Module

1. Introduction to ConfigParser
ConfigParser is the package used to read the configuration file. The configuration file format is as follows: section is included in the brackets. The section contains configuration content similar to key-value.

[db]db_host = 127.0.0.1db_port = 22db_user = rootdb_pass = rootroot[concurrent]thread = 10processor = 20

The brackets [] contain section. The section is the configuration content of options similar to key-value.

Ii. ConfigParser initial work
To use ConfigParser, you must first initialize the instance and read the configuration file:

Cf = ConfigParser. ConfigParser () cf. read ("configuration file name ")

Iii. Common ConfigParser Methods
1. obtain all the functions. That is, read all "[]" in the configuration file to the list:

s = cf.sections()print 'section:', s

Output (the following uses the configuration file in the introduction as an example ):

section: ['db', 'concurrent']

2. Obtain the options of the specified section. To read the key in a section of the configuration file to the list:

o = cf.options("db")print 'options:', o

Output:

options: ['db_host', 'db_port', 'db_user', 'db_pass']

3. Obtain the configuration information of the specified section.

v = cf.items("db")print 'db:', v

Output:

db: [('db_host', '127.0.0.1'), ('db_port', '22'), ('db_user', 'root'), ('db_pass', 'rootroot')]

4. Read the option Information of the specified section according to the type.

Db_host = cf. get ("db", "db_host") db_port = cf. getint ("db", "db_port") db_user = cf. get ("db", "db_user") db_pass = cf. get ("db", "db_pass") # returns the integer threads = cf. getint ("concurrent", "thread") processors = cf. getint ("concurrent", "processor ")

Print "db_host:", db_hostprint "db_port:", db_portprint "db_user:", db_userprint "db_pass:", db_passprint "thread:", threadsprint "processor:", processors will output: db_host: 127.0.0.1db _ port: 22db_user: rootdb_pass: rootrootthread: 10 processor: 20

5. Set the value of an option. (Remember to write back at last)

cf.set("db", "db_pass", "zhaowei")cf.write(open("test.conf", "w"))

6. Add a section. (Also write back)

Cf. add_section ('liuqing ')
Cf. set ('liuqing ', 'int', '15 ')
Cf. set ('liuqing', 'bool ', 'true ')
Cf. set ('liuqing', 'float', '3. 1415 ')
Cf. set ('liuqing ', 'baz', 'fun ')
Cf. set ('liuqing', 'bar', 'python ')
Cf. set ('liuqing ', 'foo',' % (bar) s is % (baz) s! ')
Cf. write (open ("test. conf", "w "))

7. Remove section or option. (Write it back after modification)

cf.remove_option('liuqing','int')cf.remove_section('liuqing')cf.write(open("test.conf", "w"))

However, if the parameter option in the configuration file contains [], this module cannot be used for parsing .. Therefore, when python is used as the development language, you can select the configparser module format as the configuration file format...
The following configuration cannot use the configparser module. The section containing [] is parsed as a section.

[db]  host[0]=127.0.0.1  host[1] = 192.168.1.123  db_port=3306  db_user=root  db_pass=password  [concurrent]  thread=10  processor=20  

Problems found when using the configparser module to parse the configuration file:

The parameter names in uppercase are converted to lowercase.

Parameter names cannot contain [,]
If multiple sections with the same name exist, the last section prevails.
We still found a problem. If different sections contain parameters with the same parameter name, the above parsing will not determine which section the parameter comes from.
For example, in the configuration file of myslq, both client section and mysqld section contain the socket and port parameters. If the above code is used, the two sockets and ports in the dict cannot be determined from which section.

 

　　　　　　　　　　　　　　　　　　　　　ElementTree Module

Use ElementTree to parse XML in Python
Original article: http://eli.thegreenplace.net/2012/03/15/processing-xml-in-python-with-elementtree/
Translator: TheLover_Z

When you need to parse and process XML, Python shows its "batteries encoded" side. A large number of available modules and tools in the standard library are sufficient for beginners of Python or XML.

There was an interesting discussion between core Python developers a few months ago. They discussed the advantages of the XML processing tools available in Python and how to best present them to users. This article is my own notebook. I plan to explain which tools are useful and why they are useful. Of course, this article can also be used as a basic tutorial on how to use them.

The code used in this article is based on Python 2.7. You can use Python 3.x with a slight modification.

Which XML library should be used?
Python has many tools to process XML. In this section, I want to make a simple look at the packages provided by Python, and explain why ElementTree is the one you should use most.

The xml. dom. * module is the W3C dom api implementation. If you need to process DOM APIs, this module is suitable for you. Note: There are many modules in the xml. dom package. Note the differences between them.

The xml. sax. * module is the implementation of the sax api. This module sacrifices convenience in exchange for speed and memory usage. SAX is an event-based API, which means it can "on the fly" to process a large number of documents without being fully loaded into the memory (see note 1 ).

Xml. parser. expat-is a direct, low-level C-based expat syntax analyzer (see note 2 ). The expat interface is based on event feedback, which is a bit like SAX but not very similar, because its interface is not fully standardized

>>> tree.getroot()<Element 'doc' at 0x11eb780>

import xml.etree.cElementTree as ETexcept ImportError:    import xml.etree.ElementTree as ET

This is a common method for different Python libraries to use the same API. In other words, your compiling environment may be different from other people's, so doing so can prevent some inexplicable small problems. Note: Since Python 3.3, you do not need to do this because the ElementTree module will automatically find available C libraries to speed up. So you only need to import xml. etree. ElementTree. However, before the official launch of 3.3, you 'd better use the Code provided above.

Parse XML into a tree
Let's talk about the basics. XML is a hierarchical form of data, so the most natural way to represent it as a tree. ET has two objects for this purpose-ElementTree parses the entire XML into a tree, and Element parses a single node into a tree. ElementTree is usually used for operations at the entire document level (such as reading, writing, and finding interesting elements. A single XML Element and Its sub-elements are commonly used. The following example shows a lot of things I have mentioned. (See note 5)

We use this XML file as an example:

<?xml version="1.0"?><doc>    <branch name="testing" hash="1cdf045c">        text,source    </branch>    <branch name="release01" hash="f200013e">        <sub-branch name="subrelease01">            xml,sgml        </sub-branch>    </branch>    <branch name="invalid">    </branch></doc>

Let's load and parse this XML:

>>> import xml.etree.cElementTree as ET>>> tree = ET.ElementTree(file='doc1.xml')

Then capture the root node element:

>>> tree.getroot()<Element 'doc' at 0x11eb780>

As expected, root is an Element. Let's take a look:

>>> root = tree.getroot()>>> root.tag, root.attrib('doc', {})

Check that the root element has no status (see note 6 ). Like any Element, it can find its own subnode:

>>> for child_of_root in root:...   print child_of_root.tag, child_of_root.attrib...branch {'hash': '1cdf045c', 'name': 'testing'}branch {'hash': 'f200013e', 'name': 'release01'}branch {'name': 'invalid'}

You can also enter a specified subnode:

>>> root[0].tag, root[0].text('branch', '\n        text,source\n    ')

Find the elements we are interested in
From the above example, we can easily see that we can use a simple recursion to retrieve any element in XML. However, because this operation is common, ET provides some useful tools to simplify the operation.

The Element object has an iter method that can traverse subnodes in depth first. ElementTree objects also have iter methods for convenience.

>>> for elem in tree.iter():...   print elem.tag, elem.attrib...doc {}branch {'hash': '1cdf045c', 'name': 'testing'}branch {'hash': 'f200013e', 'name': 'release01'}sub-branch {'name': 'subrelease01'}branch {'name': 'invalid'}

Traverse all the elements and check whether there is anything you want. ET makes this process easier. The iter method accepts a tag name and then only traverses the elements with the specified tag:

>>> for elem in tree.iter(tag='branch'):...   print elem.tag, elem.attrib...branch {'hash': '1cdf045c', 'name': 'testing'}branch {'hash': 'f200013e', 'name': 'release01'}branch {'name': 'invalid'}

To find the elements we are interested in, a more effective way is to use XPath support. Element has some methods for searching that can accept XPath as a parameter. Find returns the first matched child element. findall returns all matched child elements in the form of a list. iterfind provides an iterator for all matching items. These methods are also available in ElementTree.
An example is provided:

>>> for elem in tree.iterfind('branch/sub-branch'):...   print elem.tag, elem.attrib...sub-branch {'name': 'subrelease01'}

In this example, all the elements labeled as sub-branch are found under branch. Then, we will show you how to find all branch elements and use the state of a specified name:

>>> for elem in tree.iterfind('branch[@name="release01"]'):...   print elem.tag, elem.attrib...branch {'hash': 'f200013e', 'name': 'release01'}

Create an XML document
ET provides a convenient way to create XML documents and write files. The ElementTree object provides the write method.
Now, there are two common scripts for writing XML documents.
To modify a document, you can use the Element Object method:

>>> root = tree.getroot()>>> del root[2]>>> root[0].set('foo', 'bar')>>> for subelem in root:...   print subelem.tag, subelem.attrib...branch {'foo': 'bar', 'hash': '1cdf045c', 'name': 'testing'}branch {'hash': 'f200013e', 'name': 'release01'}

Here we delete the third child node of the root element and add a new state for the first child node. Then the tree can be written back to the file.

>>> import sys>>> tree.write(sys.stdout)   # ET.dump can also serve this purpose<doc>    <branch foo="bar" hash="1cdf045c" name="testing">        text,source    </branch><branch hash="f200013e" name="release01">    <sub-branch name="subrelease01">        xml,sgml    </sub-branch></branch></doc>

Note that the order of the status is not the same as that of the original document. This is because the ET state is stored in an unordered dictionary. In terms of semantics, XML does not care about sequence.
It is also easy to create a brand new element. The ET module provides the SubElement function to simplify the process:

>>> a = ET.Element('elem')>>> c = ET.SubElement(a, 'child1')>>> c.text = "some text">>> d = ET.SubElement(a, 'child2')>>> b = ET.Element('elem_b')>>> root = ET.Element('root')>>> root.extend((a, b))>>> tree = ET.ElementTree(root)>>> tree.write(sys.stdout)<root><elem><child1>some text</child1><child2 /></elem><elem_b /></root>

Conclusion
Among the numerous XML processing modules in Python, ElementTree is really amazing. It combines lightweight APIs that conform to the Python philosophy with excellent performance. So if you want to process XML, use it decisively!

 

　　　　　　　　　　　　　　　　　　　　　　　　　　　　Shutil Module
Parameter description:
Copy shutil. copyfile (src, dst) from the source src to dst. Of course, the premise is that the target address has the write permission. IOException is thrown. If the current dst already exists, it will be overwritten.
Shutil. move (src, dst) move or rename the file
Shutil. copymode (src, dst) will only copy its permissions. Other things will not be copied.
Shutil. copystat (src, dst) replication permission, last access time, last modification time
Shutil. copy (src, dst) copies an object to a file or directory.
Shutil. copy2 (src, dst) copies the last access time and modification time of the file based on the copy, similar to cp-p.
Shutil. copy2 (src, dst) if the file system in the two locations is the same, it is equivalent to the rename operation, but it is renamed. If it is not the same file system, it is the move operation.
Shutil. copytree (olddir, newdir, True/Flase)
Copy the olddir file to newdir. If the value of the 3rd parameter is True, the symbolic connection under the folder will be maintained during directory replication. If the value of the 3rd parameter is False, A physical copy is generated under the copied directory to replace the symbolic connection.
Shutil. rmtree (src) recursively deletes a directory and all contents in the directory.

 

 

 

　　　　　　　　　　　　　　　　　　　　　　　　　　　　　Subprocess Module

Starting from Python 2.4, Python introduced the subprocess module to Manage Sub-processes to replace the methods of some old modules, such as OS. system, OS. spawn *, OS. popen *, popen2. *, commands. * not only can an external command be called as a sub-process, but also can be connected to the input/output/error pipeline of the sub-process to obtain the relevant returned information.
I. subprocess and common encapsulated Functions
When running python, we are creating and running a process. Like a Linux Process, a process can fork a sub-process and let the sub-process exec another program. In Python, we use the subprocess package in the standard library to fork a sub-process and run an external program.
The subprocess package defines several functions for creating sub-processes. These functions are used to create sub-processes in different ways. Therefore, you can select a sub-process as needed. In addition, subprocess provides some tools for managing standard streams and pipelines to use text communication between processes.

Subprocess. call ()
The parent process waits for the child process to complete.
Returns the exit information (returncode, equivalent to the Linux exit code)

Subprocess. check_call ()
The parent process waits for the child process to complete.
Returns 0.
Check the exit information. If the returncode is not 0, the error subprocess. CalledProcessError is cited. This object contains the returncode attribute. try... Else T... To check

Subprocess. check_output ()
The parent process waits for the child process to complete.
Returns the output result of the sub-process to the standard output.
Check the exit information. If the returncode is not 0, the error subprocess. CalledProcessError is cited. This object contains the returncode attribute and output attribute. The output attribute is the output result of the standard output. try... Else T... To check.

The usage of these three functions is similar. The following uses subprocess. call () as an example:

>>> Import subprocess >>> retcode = subprocess. call (["ls", "-l"]) # The result is the same as that of the command ls-a in shell> print retcode0

Put the program name (ls) and the parameter (-l) together in a table and pass it to subprocess. call ()
Shell is False by default. in Linux, when shell is set to False, popencalls OS .exe cvp () to execute the program specified by args; when shell is set to True, if args is a string, popen directly calls the system Shell to execute the program specified by args. If args is a sequence, the first item of args is to define the program command string, other parameters are additional parameters used to call the system Shell.
The preceding example can also be written as follows:

>>> retcode = subprocess.call("ls -l",shell=True)

Subprocess. Popen ()

class Popen(args, bufsize=0, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=False, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0)

In fact, the above functions are based on Popen () Encapsulation (wrapper ). These encapsulation aims to make it easy to use sub-processes. When we want to personalize our needs, we need to turn to the Popen class, which generates objects to represent sub-processes.
Unlike the preceding encapsulation, after a Popen object is created, the main program does not automatically wait for the child process to complete. We must call the wait () method of the object before the parent process will wait (that is, block blocking). For example:

>>> import subprocess>>> child = subprocess.Popen(['ping','-c','4','blog.linuxeye.com'])>>> print 'parent process'

The running result shows that the parent process does not wait for the completion of child after starting the child process, but runs print directly.
Compare the waiting conditions:

>>> import subprocess>>> child = subprocess.Popen('ping -c4 blog.linuxeye.com',shell=True)>>> child.wait()>>> print 'parent process'

The running result shows that the parent process runs print after starting the child process and waiting for the completion of the child process.
In addition, you can perform other operations on the child process in the parent process, such as the child object in the above example:

Child. poll () # Check the sub-process status child. kill () # Stop the sub-process child. send_signal () # send the signal child. terminate () to the sub-process # Stop the sub-process

The child process PID is stored in child. pid.
Ii. Text Flow Control of sub-Processes
Standard input, standard output, and standard errors of sub-processes are described as follows:

child.stdinchild.stdoutchild.stderr

You can change the standard input, standard output, and standard error when creating a sub-process in Popen (), and use subprocess. PIPE connects the input and output of multiple sub-processes to form a pipeline (pipe). The following are two examples:

>>> import subprocess>>> child1 = subprocess.Popen(["ls","-l"], stdout=subprocess.PIPE)>>> print child1.stdout.read(),

Or child1.communicate ()

>>> import subprocess>>> child1 = subprocess.Popen(["cat","/etc/passwd"], stdout=subprocess.PIPE)>>> child2 = subprocess.Popen(["grep","0:0"],stdin=child1.stdout, stdout=subprocess.PIPE)>>> out = child2.communicate()

Subprocess. PIPE actually provides a cache for text streams. The stdout of child1 outputs the text to the cache, and then the stdin of child2 reads the text from the PIPE. The output text of child2 is also stored in PIPE until the communicate () method reads the text in PIPE from PIPE.
Note: communicate () is a method of the Popen object. This method blocks the parent process until the child process is completed.

Self-built http://www.jb51.net/article/48086.htm