Detailed explanation of several standard output redirection methods in python and python redirection

Detailed explanation of several standard output redirection methods in python and python redirection

I. background

In Python, file objectssys.stdin,sys.stdoutAndsys.stderrCorresponding to the standard input, standard output, and standard error streams of the interpreter. When the program starts, the initial values of these objects aresys.__stdin__,sys.__stdout__Andsys.__stderr__Save to restore the standard Stream object when finalization is finished.

In Windows, IDLE (Python gui.exe by pythonw.exe, the GUI does not have a console. Therefore, IDLE replaces the standard output handle with a special pseudo-dooutputfile object to redirect the Script output to the IDLE Terminal window (Shell ). This may cause some strange problems, such:

Python 2.7.11 (v2.7.11:6d1b6a68f775, Dec 5 2015, 20:32:19) [MSC v.1500 32 bit (Intel)] on win32Type "copyright", "credits" or "license()" for more information.>>> import sys>>> for fd in (sys.stdin, sys.stdout, sys.stderr): print fd<idlelib.PyShell.PseudoInputFile object at 0x0177C910><idlelib.PyShell.PseudoOutputFile object at 0x0177C970><idlelib.PyShell.PseudoOutputFile object at 0x017852B0>>>> for fd in (sys.__stdin__, sys.__stdout__, sys.__stderr__): print fd<open file '<stdin>', mode 'r' at 0x00FED020><open file '<stdout>', mode 'w' at 0x00FED078><open file '<stderr>', mode 'w' at 0x00FED0D0>>>> 

We can find that,sys.__stdout__Andsys.stdoutThe values are different. When running the above Code in a common Python interpreter (such as through the Windows console), the values are the same.

When the print statement (statement) does not end with a comma, A linefeed will be automatically appended to the end of the output string; otherwise, a space will replace the additional line break. The print statement is written to the standard output stream by default, and can also be redirected to a file or other writable objects (all objects that provide the write method ). In this way, you can use a simple print statement to replace clumsyobject.write('hello'+'\n')Statement.

It can be seen that when print obj is called in Python to print an object, it is equivalent to callingsys.stdout.write(obj+'\n')

Example:

>>> import sys>>> print 'Hello World'Hello World>>> sys.stdout.write('Hello World')Hello World

Ii. Redirection Mode

This section describes common Python standard output redirection methods. These methods have their own advantages and disadvantages and are suitable for different scenarios.

2.1 console redirection

The simplest and most common method of output redirection is to use console commands. This type of redirection is completed by the console, but is not related to Python itself.

The Windows Command Prompt (cmd.exe) and Linux Shell (bash, etc.) redirect the output through ">" or ">. ">" Indicates the content to be overwritten, and ">" indicates the content to be appended. Similarly, "2>" can redirect standard errors. Redirecting to "nul" (Windows) or "/dev/null" (Linux) will suppress the output without screen display or disk storage.

Take the Windows Command Prompt as an example to redirect the Python script output to a file (to shorten the space, empty lines between commands are deleted ):

E:\>echo print 'hello' > test.pyE:\>test.py > out.txtE:\>type out.txthelloE:\>test.py >> out.txtE:\>type out.txthellohelloE:\>test.py > nul

Note:When you execute a Python script in a Windows command prompt, the command line does not need to start with "python". The system automatically calls the Python interpreter according to the script suffix. In addition, the type command can directly display the content of a text file, similar to the cat command in Linux.

When running a Python script in Linux Shell, the command line should start with "python. In addition to ">" or ">" redirection, you can also use the tee command. This command can output content to the terminal screen and (multiple) files at the same time. The "-a" option indicates append write; otherwise, overwrite the content. Example (echo $SHELLOrecho $0Display the current Shell ):

[wangxiaoyuan_@localhost ~]$ echo $SHELL/bin/bash[wangxiaoyuan_@localhost ~]$ python -c "print 'hello'"hello[wangxiaoyuan_@localhost ~]$ python -c "print 'hello'" > out.txt[wangxiaoyuan_@localhost ~]$ cat out.txthello[wangxiaoyuan_@localhost ~]$ python -c "print 'world'" >> out.txt[wangxiaoyuan_@localhost ~]$ cat out.txt             helloworld[wangxiaoyuan_@localhost ~]$ python -c "print 'I am'" | tee out.txtI am[wangxiaoyuan_@localhost ~]$ python -c "print 'xywang'" | tee -a out.txtxywang[wangxiaoyuan_@localhost ~]$ cat out.txtI amxywang[wangxiaoyuan_@localhost ~]$ python -c "print 'hello'" > /dev/null[wangxiaoyuan_@localhost ~]$ 

If you only want to save the Script output to a file, you can also directly use the Log Capture function of the session window.

Note that the impact of console redirection is global and only applicable to simple output tasks.

2.2 print> Redirection

This method is based on the extended form of the print statement, that is"print obj >> expr". Where,objIs a file-like (especially the write method provided) object. If it is None, it corresponds to the standard output (sys. stdout ).exprWill be output to the file object.

Example:

memo = cStringIO.StringIO(); serr = sys.stderr; file = open('out.txt', 'w+')print >>memo, 'StringIO'; print >>serr, 'stderr'; print >>file, 'file'print >>None, memo.getvalue()

After the above code is executed, the screen displays "serr" and "StringIO" (two rows, sorted sequentially, And the out.txt file is written to "file ".

This method is flexible and convenient. The disadvantage is that it is not applicable to scenarios with many output statements.

2.3 sys. stdout redirection

Assign a writable object (such as a file-like object) to sys. stdout so that subsequent print statements can be output to the object. After redirection, sys. stdout should be restored to the original default value, that is, standard output.

A simple example is as follows:

Import syssavedStdout = sys. stdout # Save the standard output stream with open('out.txt ', 'W +') as file: sys. stdout = file # The standard output is redirected to the file print 'this message is for file! 'Sys. stdout = savedStdout # restore the standard output stream print 'This message is for screen! '

Note:sys.stdoutThe initial value is a pseudo-dooutputfile object.sys.__stdout__Not the same. For general purpose, this example defines the variable (savedStdout) to savesys.stdout. In addition, this example does not applyfrom sys import stdoutThe imported stdout object.

You can customize multiplewrite()Method file-like object to meet different needs:

Class RedirectStdout: # import OS, sys, cStringIO def _ init _ (self): self. content = ''self. savedStdout = sys. stdout self. memObj, self. fileObj, self. nulObj = None, None, None # The external print statement will execute the write () method, and the current sys. stdout output def write (self, outStr): # self. content. append (outStr) self. content + = outStr def toCons (self): # The standard output is redirected to the console sys. stdout = self. savedStdout # sys. _ stdout _ def toMemo (self): # standard output redirected to memory Self. memObj = cStringIO. stringIO () sys. stdout = self. memObj def toFile (self, file+'out.txt '): # The standard output is redirected to the file self. fileObj = open (file, 'a + ', 1) # change to row buffer sys. stdout = self. fileObj def toMute (self): # suppress output self. nulObj = open (OS. devnull, 'w') sys. stdout = self. nulObj def restore (self): self. content = ''if self. memObj. closed! = True: self. memObj. close () if self. fileObj. closed! = True: self. fileObj. close () if self. nulObj. closed! = True: self. nulObj. close () sys. stdout = self. savedStdout # sys. _ stdout __

Note,toFile()Method,open(name[, mode[, buffering]])Select row buffering for calling (no buffering will affect performance ). This is to observe the intermediate write process; otherwise, onlyclose()Orflush()Before the output is written into the file. The disadvantage of calling the open () method internally is that it is not convenient for users to customize file writing rules, such as mode (overwrite or append) and buffer (row or full buffer ).

The redirection effect is as follows:

RedirObj = RedirectStdout () sys. stdout = redirObj # This sentence will suppress "Let's begin! "Output print" Let's begin! "# 'Hello World! 'And' I am xywang. '(two rows) redirObj. toCons (); print 'Hello World! '; Print' I am xywang. '# Write 'How are you? 'And "Can't complain." (two rows) redirObj. toFile (); print 'How are you? '; Print "Can't complain." redirObj. toCons (); print "What 'up? "# Screen display redirObj. toMute (); print '<Silence>' # No screen display or write OS. system ('echo Never redirect me! ') # 'Never redirect me! 'Redirobj. toMemo (); print 'What a pity! '# No screen display or write redirObj. toCons (); print 'Hello? '# Screen display redirObj. toFile (); print "Oh, xywang can't hear me" # Write the string to the file redirObj. restore () print 'pop Up' # Screen Display

It can be seen that after the toXXXX () Statement is executed, the standard output stream will be redirected to XXXX. In addition,toMute()AndtoMemo()Can suppress the output.

Replace with an objectsys.stdoutMake sure that the object is close to the file object, especially when it involves a third-party library (this library may use other methods of sys. stdout ). In addition, this section replacessys.stdoutThe code implementation does not affectos.popen()、os.system()Oros.exec*()The standard I/O flow of the processes created by the series of methods.

2.4 Context Manager)

This section is strictly not a new redirection method, but uses the Pyhton context manager to optimize the code implementation in the previous section. With context manager syntax, you do not need to exposesys.stdout.

First, consider output suppression. The implementation based on the context manager syntax is as follows:

Import sys, cStringIO, contextlibclass DummyFile: def write (self, outStr): pass@contextlib.contextmanagerdef MuteStdout (): savedStdout = sys. stdout sys. stdout = cStringIO. stringIO () # DummyFile () try: yield failed t Exception: # content, sys. stdout = sys. stdout, savedStdout print content. getvalue () #; raise # finally: sys. stdout = savedStdout

Example:

With MuteStdout (): print "I'll show up when <raise> is executed! "# Do not write raise without Screen Display # print on screen display" I'm hiding myself somewhere :) "# Do not screen display

Consider more general output redirection:

import os, sysfrom contextlib import contextmanager@contextmanagerdef RedirectStdout(newStdout):  savedStdout, sys.stdout = sys.stdout, newStdout  try:    yield  finally:    sys.stdout = savedStdout

Example:

Def Greeting (): print 'hello, boss! 'With open('out.txt', "w +") as file: print "I'm writing to you... "# screen display with RedirectStdout (file): print 'I hope this letter finds you well! '# Write File print' Check your mailbox. '# screen display with open (OS. devnull, "w +") as file, RedirectStdout (file): Greeting () # print 'I deserve a pay raise :)' # print 'Did you hear what I said? '# Screen Display

It can be seen that the function and print statement output in the embedded with block are all redirected. Note that the preceding example is NOT thread-safe and is mainly applicable to a single thread.

When a function is frequently called, we recommend that you use the decorator to wrap the function. In this way, you only need to modify the function definition, instead of using the with statement every time you call the function. Example:

import sys, cStringIO, functoolsdef MuteStdout(retCache=False):  def decorator(func):    @functools.wraps(func)    def wrapper(*args, **kwargs):      savedStdout = sys.stdout      sys.stdout = cStringIO.StringIO()      try:        ret = func(*args, **kwargs)        if retCache == True:          ret = sys.stdout.getvalue().strip()      finally:        sys.stdout = savedStdout      return ret    return wrapper  return decorator

If the retCache parameter of the decorator MuteStdout is true, an external callfunc()The function returns the content output by print (for Screen Display). If the retCache is false, it is called externally.func()The Return Value of the function is returned when the function is used to suppress the output ).

The following is an example of how to use the MuteStdout decorator:

@ MuteStdout (True) def Exclaim (): print 'I am proud of myself! '@ MuteStdout () def Mumble (): print' I lack confidence... '; return 'sad' print Exclaim (), Exclaim. _ name _ # screen display 'I am proud of myself! Exclaim 'print Mumble (), Mumble. _ name _ # 'sad Mumble'

In all threads, during the execution of the decorated function,sys.stdoutWill be hijacked by the MuteStdout decorator. In addition, the decoration cannot be removed once the function is decorated. Therefore, consider the scenario with caution when using the decorator.

Next, consider creating a RedirectStdout decorator:

def RedirectStdout(newStdout=sys.stdout):  def decorator(func):    def wrapper(*args,**kwargs):      savedStdout, sys.stdout = sys.stdout, newStdout      try:        return func(*args, **kwargs)      finally:        sys.stdout = savedStdout    return wrapper  return decorator

Example:

File = open('out.txt ', "w +") @ RedirectStdout (file) def FunNoArg (): print 'no argument. '@ RedirectStdout (file) def FunOneArg (a): print 'one argument:', adef FunTwoArg (a, B): print 'two arguments: % s, % s' % (a, B) FunNoArg () # Write the file 'no argument. 'funonearg (1984) # Write the file 'one argument: 100' RedirectStdout () (FunTwoArg) (1984) # 'two arguments: 10, 29' print FunNoArg. _ name _ # 'wrapper' on the screen ('funnoarg 'should be displayed) file. close ()

Note:FunTwoArg()The definition and call of a function are different from those of other functions. These are two equivalent syntaxes. In addition, the innermost function of the RedirectStdout Modifierwrapper()Not used"functools.wraps(func)"Decoration will lose the original special attributes of the function to be decorated (such as the function name and document string ).

2.5 logging module redirection

For projects with a large amount of code, we recommend that you use the logging module for output. This module is thread-safe. It can output log information to the console, write files, send TCP/UDP protocol to the network, and so on.

By default, the logging module outputs logs to the console (standard errors) and only displays logs greater than or equal to the set log level. The log level is from high to lowCRITICAL > ERROR > WARNING > INFO > DEBUG > NOTSETThe default level is WARNING.

In the following example, the log information is output to the console and written to files respectively:

Import logginglogging. basicConfig (level = logging. DEBUG, format = '% (asctime) s [% (levelname) s] at % (filename) s, % (lineno) d: % (message) s ', datefmt = '% Y-% m-% d (% a) % H: % M: % s', filename = 'out.txt', filemode = 'W ') # output log information greater than or equal to INFO level to StreamHandler (standard error by default) console = logging. streamHandler () console. setLevel (logging. INFO) formatter = logging. formatter ('[% (levelname)-8 s] % (message) s') # display on screen in real time, no time console required. setFormatter (formatter) logging. getLogger (). addHandler (console) logging. debug ('kuberned'); logging.info ('ofni'); logging. critical ('lacitirc ')

You can set different level parameters for multiple handler to input different log Content to different places. In this example, the StreamHandler (and FileHandler) built in the logging module is used, and the screen is displayed after running:

[INFO  ] ofni[CRITICAL] lacitirc

The content of the out.txt file is:

2016-05-13(Fri)17:10:53 [DEBUG] at test.py,25: gubed2016-05-13(Fri)17:10:53 [INFO] at test.py,25: ofni2016-05-13(Fri)17:10:53 [CRITICAL] at test.py,25: lacitirc

In addition to setting Logger, Handler, and Formatter in a program, you can also write the information to the configuration file. Example:

#logger.conf###############Logger###############[loggers]keys=root,Logger2F,Logger2CF[logger_root]level=DEBUGhandlers=hWholeConsole[logger_Logger2F]handlers=hWholeFilequalname=Logger2Fpropagate=0[logger_Logger2CF]handlers=hPartialConsole,hPartialFilequalname=Logger2CFpropagate=0###############Handler###############[handlers]keys=hWholeConsole,hPartialConsole,hWholeFile,hPartialFile[handler_hWholeConsole]class=StreamHandlerlevel=DEBUGformatter=simpFormatterargs=(sys.stdout,)[handler_hPartialConsole]class=StreamHandlerlevel=INFOformatter=simpFormatterargs=(sys.stderr,)[handler_hWholeFile]class=FileHandlerlevel=DEBUGformatter=timeFormatterargs=('out.txt', 'a')[handler_hPartialFile]class=FileHandlerlevel=WARNINGformatter=timeFormatterargs=('out.txt', 'w')###############Formatter###############[formatters]keys=simpFormatter,timeFormatter[formatter_simpFormatter]format=[%(levelname)s] at %(filename)s,%(lineno)d: %(message)s[formatter_timeFormatter]format=%(asctime)s [%(levelname)s] at %(filename)s,%(lineno)d: %(message)sdatefmt=%Y-%m-%d(%a)%H:%M:%S

Create three Logger: root to output all logs to the console; Logger2F to write all logs to files; Logger2CF to output logs with a level greater than or equal to INFO to the console, write logs with a level greater than or equal to WARNING to a file.

The program parses the configuration file and redirected output as follows:

import logging, logging.configlogging.config.fileConfig("logger.conf")logger = logging.getLogger("Logger2CF")logger.debug('gubed'); logger.info('ofni'); logger.warn('nraw')logger.error('rorre'); logger.critical('lacitirc')logger1 = logging.getLogger("Logger2F")logger1.debug('GUBED'); logger1.critical('LACITIRC')logger2 = logging.getLogger()logger2.debug('gUbEd'); logger2.critical('lAcItIrC')

After running, the screen displays:

[INFO] at test.py,7: ofni[WARNING] at test.py,7: nraw[ERROR] at test.py,8: rorre[CRITICAL] at test.py,8: lacitirc[DEBUG] at test.py,14: gUbEd[CRITICAL] at test.py,14: lAcItIrC

The content of the out.txt file is:

2016-05-13(Fri)20:31:21 [WARNING] at test.py,7: nraw2016-05-13(Fri)20:31:21 [ERROR] at test.py,8: rorre2016-05-13(Fri)20:31:21 [CRITICAL] at test.py,8: lacitirc2016-05-13(Fri)20:31:21 [DEBUG] at test.py,11: GUBED2016-05-13(Fri)20:31:21 [CRITICAL] at test.py,11: LACITIRC

Iii. Summary

The above is all about the redirection method of Python standard output. I hope it will be helpful to anyone who has learned python. If you have any questions, please leave a message for discussion.