Python operations files and directories and file reading and writing

If we were to execute directory and file operations in a Python program, the commands provided by the operating system simply invoked the interface functions provided by the operating system, and Python's built-in OS modules could invoke the interface functions provided by the operating system directly.

To open the Python interactive command line, let's look at how to use the basic functionality of the OS module:

>>> Import OS
>>> Os.name # Operating system name
' POSIX '

If it is POSIX, the system is Linux, Unix or Mac OS X, and if it is NT, it is the Windows system.

To obtain detailed system information, you can invoke the uname () function:

>>> Os.uname ()
(' Darwin ', ' imac.local ', ' 13.3.0 ', ' Darwin Kernel Version 13.3.0:tue June 3 21:27:35 PDT 2014; root:xnu-2422.110.17~1/rel Ease_x86_64 ', ' x86_64 ')

Note the uname () function is not available on Windows, which means that some functions of the OS module are related to the operating system.

Environment Variables

The environment variables defined in the operating system are all stored in the Os.environ dict and can be viewed directly:

>>> Os.environ
{' versioner_python_prefer_32_bit ': ' No ', ' term_program_version ': ' 326 ', ' LOGNAME ': ' Michael ', ' USER ': ' Michael ', ' PATH ': '/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/opt/x11/bin:/usr/local/mysql/bin ', ...}

To get the value of an environment variable, you can call the Os.getenv () function:

>>> os.getenv (' PATH ')
'/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/opt/x11/bin:/usr/local/mysql/bin '


manipulating files and directories

Note that the functions of manipulating files and directories are part of the OS module and part of the Os.path module. viewing, creating, and deleting a directory can be invoked like this:

# View the absolute path to the current directory:
>>> os.path.abspath ('. ')
'/users/michael '
# Create a new directory under a directory,
# first show the full path of the new directory:
>>> os.path.join ('/users/michael ', ' TestDir ')
'/users/michael/testdir '
# then create a directory:
>>> os.mkdir ('/users/michael/testdir ')
# Delete a directory:
>>> os.rmdir ('/users/michael/testdir ')

When you synthesize two paths into one, do not spell strings directly, but through the Os.path.join () function, you can correctly handle the path separators for different operating systems. Under Linux/unix/mac, Os.path.join () returns such a string:

Part-1/part-2

and Windows will return such a string:

Part-1\part-2

In the same way, to split a path, do not directly remove the string, but through the os.path.split () function, so that you can split a path into two parts, the latter part is always the last level of the directory or file name:

>>> os.path.split ('/users/michael/testdir/file.txt ')
('/users/michael/testdir ', ' file.txt ')

Os.path.splitext () allows you to get file extensions directly, which can be very handy in many cases:

>>> os.path.splitext ('/path/to/file.txt ')
('/path/to/file ', ' txt ')

These merge, split-path functions do not require directories and files to be real, they operate on strings only.

The file operation uses the following function. Suppose there is a test.txt file under the current directory:

# Rename the file:
>>> os.rename (' test.txt ', ' test.py ')
# Erase File:
>>> os.remove (' test.py ')

But the function of copying files does not exist in the OS module! The reason is that the copy file is not a system call provided by the operating system. Theoretically, we can do file copying through the read and write files in the previous section, but we need to write a lot more code.

Fortunately, the Shutil module provides a copyfile () function, and you can also find a number of utility functions in the Shutil module, which can be viewed as a complement to the OS module.

Finally, see how to filter files using Python's features. For example, we want to list all the directories in the current directory, only one line of code is required:

>>> [x for X in Os.listdir ('. ') if Os.path.isdir (x)]
['. Lein ', '. Local ', '. M2 ', '. npm ', '. ssh ', '. Trash ', '. Vim ', ' adlm ', ' applications ', ' Desktop ', ...

To list all. py files, just one line of code:

>>> [x for X in Os.listdir ('. ') if Os.path.isfile (x) and Os.path.splitext (x) [1]== '. Py ']
[' apis.py ', ' config.py ', ' models.py ', ' pymonitor.py ', ' test_db.py ', ' urls.py ', ' wsgiapp.py ']

Is it very concise?

Summary

Python OS modules encapsulate the operating system's directory and file operations, and note that some of these functions are in the OS module and some are in the Os.path module.

Exercise: Write a search (s) function that finds the file name containing the specified string in the current directory and all subdirectories of the current directory, and prints out the full path:

$ python search.py test
Unit_test.log
py/test.py
py/test_os.py
My/logs/unit-test-result.txt



python file read and write

Reading and writing files is the most common IO operation. Python has built-in functions for reading and writing files, and the usage is compatible with C.

Before reading and writing files, we must first understand that the ability to read and write files on disk is provided by the operating system, the modern operating system does not allow ordinary programs to manipulate the disk directly, so read and write files is to request the operating system to open a file object (usually called a file descriptor), and then, Read data (read files) from this file object through the interface provided by the operating system, or write the data to the file object (write file).

Read Files

To open a file object in a read-file mode, use the Python built-in open () function to pass in the filename and designator:

>>> f = open ('/users/michael/test.txt ', ' R ')

The identifier ' R ' means read, so we successfully open a file.

If the file does not exist, the open () function throws a ioerror error, and the error code and detailed information tell you that the file does not exist:

>>> f=open ('/users/michael/notfound.txt ', ' R ')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
IOError: [Errno 2] No such file or directory: '/users/michael/notfound.txt '

If the file is opened successfully, Next, the Read () method is invoked to read the entire contents of the file at once, and Python reads the contents into memory, represented by a str object:

>>> F.read ()
' Hello, world! '

The final step is to call the close () method to shut down the file. The file must be closed after it has been used, because the file object consumes the resources of the operating system, and the number of files that the operating system can open at the same time is limited:

>>> F.close ()

The F.close () will not be invoked as soon as the file is read and written and the IOError is likely to occur. So, to ensure that the file is closed correctly, whether or not it goes wrong, we can use the try ... finally to implement:

Try
f = open ('/path/to/file ', ' R ')
Print F.read ()
Finally
If f:
F.close ()

But it's so tedious to write every time, so Python introduces the WITH statement to automatically call the Close () method:

With open ('/path/to/file ', ' R ') as F:
Print F.read ()

This is the same as the previous try ... finally, but the code is better and simpler and does not have to invoke the F.close () method.

Calling read () reads the entire contents of the file at once, and if the file has 10G, the memory explodes, so, to be safe, you can repeatedly call the read (size) method to read up to the size byte at a time. In addition, the call to ReadLine () can read one line at a time and call ReadLines () to read all content at once and return to list by row. So decide how to call it as needed.

If the file is small, read () is the most convenient read (), and if you cannot determine the file size, call the read (size) to compare insurance; if it is a configuration file, it is most convenient to call ReadLines ():

For line in F.readlines ():
Print (Line.strip ()) # Erase the end ' \ n '


File-like Object

This object, like the open () function, that has a read () method is called File-like object in Python. In addition to file, you can also be a byte stream of memory, network flow, custom flow, and so on. File-like object does not require inheriting from a particular class, just write a read () method.

Stringio is the File-like Object created in memory and is commonly used as a temporary buffer.

Binary Files

The previous default is to read the text file, and it is an ASCII-encoded text file. To read binary files, such as pictures, videos, and so on, open the file in ' RB ' mode:

>>> f = open ('/users/michael/test.jpg ', ' RB ')
>>> F.read ()
' \xff\xd8\xff\xe1\x00\x18exif\x00\x00. ' # Hexadecimal representation of bytes


character encoding

To read a non-ASCII-encoded text file, you must open it in binary mode and then decode it. Like the GBK encoded file:

>>> f = open ('/users/michael/gbk.txt ', ' RB ')
>>> u = f.read (). Decode (' GBK ')
>>> u
U ' \u6d4b\u8bd5 '
>>> Print U
Test

If it's troublesome to manually convert code every time (writing a program for fear of trouble is a good thing, and will write long, difficult and maintainable code without fear of trouble), Python also provides a codecs module to help us automatically convert the code when we read the file and read the Unicode directly:

Import Codecs
With Codecs.open ('/users/michael/gbk.txt ', ' r ', ' GBK ') as F:
F.read () # u ' \u6d4b\u8bd5 '


Write a file

Writing a file is the same as reading a file, except that when you invoke the open () function, passing in the identifier ' W ' or ' WB ' means writing a text file or writing a binary file:

>>> f = open ('/users/michael/test.txt ', ' W ')
>>> f.write (' Hello, world! ')
>>> F.close ()

You can repeatedly call write () to write to the file, but be sure to call F.close () to close the file. When we write files, the operating system often does not write data to the disk immediately, but put it in memory cache, and then slowly write. Only when the close () method is invoked does the operating system ensure that all data not written is written to disk. The consequence of forgetting to call Close () is that the data may have only been written to disk, and the rest is lost. So, it's still insured with the WITH statement:

With open ('/users/michael/test.txt ', ' W ') as F:
F.write (' Hello, world! ')

To write a specific encoded text file, follow the example of codecs and write to Unicode, which is automatically converted to the specified encoding by codecs.

Summary

In Python, file reads and writes are done through a file object opened by the open () function. Using the WITH statement to manipulate file IO is a good habit.