An example of how to use Python to call the system Shell in Linux

This article describes how to use Python to call the system Shell in Linux, including reading a row of files using Python and shell, for more information about how to mount a file system, you must directly call the Shell command to complete some simple operations. So how can we use Python to call Linux Shell commands? The following describes several common methods:
1. OS module

1.1. exec method family of OS module
The exec system method of Python is the same as that of Unix exec system. These methods are applicable when an external program is called in a child process, because the external program replaces the code of the current process and does not return. (I have read the help (OS) --> search "exec" introduction, but I don't quite understand how to use it)

1.2. system method of the OS module
The system method creates a sub-process to run the external program. The method returns only the running results of the external program. This method is applicable when no external program outputs results.

>>> Import OS >>> OS. system ("echo \" Hello World \ "") # directly use OS. system calls an echo command Hello World ------> print the command result 0 ------> What's this? Return value? >>> Val = OS. system ("ls-al | grep \" log \"") # Use val to receive the returned value-rw-r -- 1 root 6030829 Dec 31 log ------> at this time, only the command result is printed >>> print val 0 ------> note, when the command runs normally, the return value is 0 >>> val = OS. system ("ls-al | grep \" log1 \ "") >>> print val 256 ------> use OS. system calls a command that does not return results. the returned value is 256 ~ >>>

Note: As mentioned above, fat will be the result of an external program, that is, the result of OS. system ~

1.3. popen method of OS module
This method is useful when an external program output result is required. a class file object is returned, and the read () or readlines () method of this object can be called to read the output content. For example, when using urllib to call a Web API, you need to process the obtained data. OS. popen (cmd) to get the output content of the command, you only need to call read () or readlines (), such as a = OS. popen (cmd). read ()

>>> OS. popen ('ls-lt ') # Calling OS. popen (cmd) does not get the expected result.
 
  
>>> Print OS. popen ('ls-lt '). read () # Call the read () method to obtain the command result total 6064-rwxr-xr-x 1 long 23 Jan 5 hello. sh-rw-r -- 1 long 147 Jan 5 Makefile drwxr-xr-x 3 long 4096 Jan 2 test-rw-r -- 1 root 6030829 dec 31 log drwxr-xr-x 2 long 4096 Dec 28 pip_build_long drwx ------ 2 Debian-tpd 4096 Dec 23 pulse-gylJ5EL24GU9 drwx ------ 2 long 4096 Jan 1 1970 orbit-long> val = OS. popen ('ls-lt '). read () # use variables to receive command return values> if "log" in val: # we can use in to determine that the returned value contains a string... print "Haha, there is the log "... else :... print "No, not happy "... haha, there is the log
 

2. commands module

The getoutput method of the commands module is used. The difference between this method and popend is that popen returns a class object, and this method returns the output result of an external program as a string, in many cases, it is easier to use it.
Main methods:
* Commands. getstatusoutput (cmd) returns (status, output)
* Commands. getoutput (cmd) returns only the output results.
* Commands. getstatus (file) returns the execution result string of the ls-ld file and calls getoutput. this method is not recommended.

Long @ zhouyl:/tmp/tests $ python Python 2.7.3 (default, Jan 2 2013, 16:53:07) [GCC 4.7.2] on linux2 Type "help", "copyright ", "credits" or "license" for more information. >>> import commands >>> commands. getstatusoutput ('ls-lt ') # Return (status, output) (0, 'Total 5900 \ n-rwxr-xr-x 1 long 23 Jan 5 hello. sh \ n-rw-r -- 1 long 147 Jan 5 Makefile \ n-rw-r -- 1 long 6030829 Jan 5 2 Log')> commands. getoutput ('ls-lt ') # return the output result of the command (it seems that the output format is different from that of the Shell command ~) 'Total 5900 \ n-rwxr-xr-x 1 long 23 Jan 5 hello. sh \ n-rw-r -- 1 long 147 Jan 5 Makefile \ n-rw-r -- 1 long 6030829 Jan 5 log' >>> commands. getstatus ('log') # Call commands. the command in getoutput performs the same operation on the 'log' file '-rw-r -- 1 long 6030829 Jan 5 log' >>>

3. subprocess module

According to the official Python documentation, the subprocess module is used to replace the above modules. There is a parallel ssh tool-mssh implemented using Python. the code is short, but interesting. it calls the subprocess promoter process in the thread to work.
>>> From subprocess import call
>>> Call (["ls", "-l"])

The advantage of subprocess over system is that it is more flexible (you can get standard output, standard errors, "real" status code, better error handling, etc ..). I think the use of OS. system is outdated or about to expire.

4. comparison and summary of the Zhongfang method
4.1. about OS. system
OS. system ("some_command with args") passes commands and parameters to your system shell, which is good, this is because you can run multiple commands at the same time and set pipelines and input/output redirection. For example:
OS. system ("some_command <input_file | another_command> output_file ")
However, although this is convenient, you need to manually handle shell escape characters, such as spaces. In addition, this only allows you to run simple shell commands and not external programs.

4.2. about OS. popen
Use stream = OS. popen ("some_command with args") can also work with OS. the same thing as OS. system is different from OS. popen returns a class object that is used to access standard input and output.
4.3. about subprocess. popen
The Popen class of the subprocess module is intended to replace OS. popen, but it is a little more complex than OS. popen because it is comprehensive.
For example, you can use print Popen ("echo Hello World", stdout = PIPE, shell = True ). stdout. read () to replace print OS. popen ("echo Hello World "). read (). However, in comparison, it is good to use a unified class including four different popen functions.

4.4. about subprocess. call
The call function of the subprocess module. It is basically like the Popen class and uses the same parameters, but it simply waits for the command to complete and returns code to you. For example:

return_code = subprocess.call("echo Hello World", shell=True)

The OS module also has the fork/exec/spawn functions in C, but I do not recommend using them directly. Subprocess may be more suitable for you.

Python and shell read a row of a file

Python and shell (awk command) can directly read a row of the file and read it by row number. It can accurately obtain a field of the row, which is similar to the operation of locating a point on the x axis and y axis.

1. awk takes a column value in a row

Awk can set conditions to output the specified k fields in each row from Row m to row n in the file. the format is as follows:

awk  'NR==m,NR==n {print $k}' path/filename

M, n, k indicates the actual value. If you want to use a variable to represent the value of m and n, the variable must be enclosed in single quotes. NR, {print} is the required field of the awk command in this usage. path/filename indicates the path and file name of the file to be read. Two rows are specified here. if only one row is specified, write as follows:

awk  'NR==m {print $k}' path/filename

2. python retrieves a column from a row

The linecache module provided by the standard library provides a specific method for retrieving a row:

import linecachetheline = linecache.getline(filepath, line_number)

After obtaining the relevant rows, split the theline into a list, and then set the value of the list index. For example, theline. split () [2].

III. usage of the linecache module

Now, the linecache module is mentioned. Other methods of linecache are listed here. The linecache module allows any row from any file and uses the cache for optimization. it is common to read multiple rows from a single file.

Linecache. getlines (filename) get all content from the file named filename, output as list format, one element in the file per behavior list, and store the linenum-1 as the element in the list
Linecache. getline (filename, lineno) obtains line lineno from the file named filename. This function will never throw an exception-when an error is generated, it will return "(the linefeed will be included in the row found ). If the file is not found, this function will be searched in sys. path.
Linecache. receivache () clears the cache. If you no longer need the rows previously obtained from getline ()
Linecache. checkcache (filename) checks the validity of the cache. If the files in the cache change on the hard disk and you need to update the version, use this function. If filename is omitted, all entries in the cache are checked.
Linecache. updatecache (filename) updates the cache file named filename. If the filename File is updated, you can use this function to update the list returned by linecache. getlines (filename.
Example:

# cat a.txt

1a2b3c4d5e6f7g

1. obtain the.txt file.

>>> a=linecache.getlines('a.txt')>>> a['1a\n', '2b\n', '3c\n', '4d\n', '5e\n', '6f\n', '7g\n']

2. obtain the contents of Line 1-4 in the.txt file.

>>> a=linecache.getlines('a.txt')[0:4]>>> a['1a\n', '2b\n', '3c\n', '4d\n']

3. obtain the contents of Line 1 in the.txt file.

>>> a=linecache.getline('a.txt',4)>>> a'4d\n'

Note:

After you use linecache.getlines('a.txt 'character to open the file content, if the.txt file changes, for example, if you use linecache. getlines again to obtain the content, not the latest content of the file, or the previous content, there are two methods:

The latest content of the token;

2. directly use linecache.updatecache('a.txt 'to obtain the latest a.txt content.

After reading a file, you do not need to use the file cache. you need to clear the cache at the end so that linecache. cacache () can clear the cache and release the cache.