IO programming, manipulating files or directories, serialization, JSON

Io middle finger input/output, that is, input and output, where data exchange is involved, usually disk, network, etc., need IO interface

1, because the CPU and memory speed is much higher than the speed of the peripheral, so in IO programming, there is a serious speed mismatch problem. Eg: to write 100M data to disk, the CPU output 100M of data only 0.01 seconds, but the disk can receive 100M data may take 10 seconds, how to do it, there are two ways:

①, CPU, etc., that is, the program suspends execution of subsequent code, such as 100M data in 10s after writing to the disk, and then down execution, this mode is called synchronous IO

②, the CPU does not wait, just tell the disk, "slowly write, don't worry, I went on to do something else", so the subsequent code can be executed immediately, this mode is called asynchronous IO

It is obvious that asynchronous IO is much better than synchronous IO to write programs, but the disadvantage of asynchronous IO is that the programming model is complex

2, read and write files before, we must first understand that the function of reading and writing files on disk is provided by the operating system, the modern operating system does not allow the normal program to operate the disk directly, so, read and write the file is to request the operating system to open a file object (file descriptor), and then, Read data from this file object via an interface provided by the operating system (read file), or write data to this file object (write file)

①, read files

Read the file mode open a file object, using the Python built-in open () function, to pass in the file name and identifier:

>>>f=open (' User/test.txt ', ' R ') R: reads so that a file is opened successfully, and if the file does not exist, the open () function throws a IOError error and gives the error code and details

>>>f.read () can read all the contents of the file at once, Python reads the contents into memory, denoted by a str; the read (size) method, which reads a maximum of size bytes of content at a time, the 10G content of this insurance (insufficient memory); ReadLine ( Each time a line is read, the configuration file

After the >>>f.close () file is used, because the file object consumes the resources of the operating system, and the number of files that the operating system can open at the same time is limited

Since the read-write file is likely to produce a ioerror error, the subsequent f.close () will not be invoked once an error occurs:

Try

F=open ('/path/to/file ', ' R ')

Print F.read ()

Finally

If f:

F.close ()

Try...finally every time. To implement, too cumbersome, so Python introduces a with statement to help us call the Close () method: This method is better concise, and do not have to call the F.close () method

　　　　　　With open ('/path/to/file ', ' R ') sa f:

Print F.read ()

②, writing files

The only difference between writing a file and reading a file is when invoking the open () function, passing in the identifier ' W ' or ' WB ', which means writing a text file and writing a binary file

>>>f=open ('/user/to/test ', ' W ')

>>>f.write (' Hlleo, World ')

>>>f.close ()

Write using the WITH statement:

　　　　　With open ('/user/to/test ', ' W ') as F

F.write (' Hlleo, World ')

--------------------------------------------------------------------------------------------------------------- -------------------------

Manipulating Files and directories

1, if you want to manipulate files, directories, you can enter the command line under the operating system provides a variety of commands to complete, for example, DIR, CP and other commands

What if you want to perform these directories and files in a python program? In fact, the command provided by the operating system simply invokes the interface functions provided by the operating system, andPython's built-in OS module can also directly invoke the interface functions provided by the operating system

>>>import OS

>>>os.name

>>> ' POSIX ' if it is POSIX, the system is Linux, Unix or Mac OS X; if NT, Windows system

2, Environment variables: to get the value of an environment variable, you can call the Os.getenv () function

>>>import OS

>>>os.getenv (' path ')

3, Python in the file, folder operations often used in the OS module and Shutil module common methods

A, get the current working directory, that is, the current Python script work directory path: OS.GETCWD (); Os.path.abspath ('. ') view the absolute directory of the current directory

B. Returns all files and directory names under the specified directory: os.remove (' c:\\users\\zxq\\desktop\\total ') required to write \ \

C, function to delete a file: Os.remove (' C:\\users\\zxq\\desktop\\total\\test.txt ')

D, verify that the given path is a file: Os.path.isfile ('c:\\users\\zxq\\desktop\\total\\ test. txt ') returns true, False

E, verify whether the given path is a directory: Os.path.isdiir ()

F, verify that the given path is true: Os.path.exists ()

G, Get path name: Os.path.dirname ()

H, run shell command: Os.system ()

I, rename: Os.rename (old,new)

J. Create a single directory: Os.mkdir (' Test ')

K, modify file permissions and timestamps: Os.chmod (file)

L, get File size: os.path.getsize (filename)

4, the file operation method Daquan:

A, Os.mknod (' test ')

B, Fp=open (' Test ', W) open a file directly and create a file if the file does not exist

O, Fp.read ([Siaze]) size is the read length, in bytes

P, Fp.write (str) write str to file

Q, Fp.close () Close the file

Os.chdir (' C:\Users\zxq\Desktop ') switch to the specified directory

OS.GETCWD () View current directory

--------------------------------------------------------------------------------------------------------------- ------------------

1. Serialization pickling: The process of changing a variable from memory to storage or transfer is called serialization. After serialization, the serialized content can be written to disk or transmitted over the network to another machine .

Deserialization unpickling: Re-reading the variable contents from the serialized object into memory

Python provides two modules for serialization: Cpickle and Pickle. The difference is that Cpickle is written in C, fast, pickle is written in pure python, slow, and when you try to import cpickle, you fail to import pickle

Try

Import Cpickle as Pickle

exception

Import Pickle

EG1: Serializing and writing an object to a file

import cpickle as pickle Import Module 
 F =  ' C:\\users\\zxq\\desktop\\a.txt ' , WB ") < Span style= "color: #cc7832;" > Open a file directly, Create file If file does not exist                < Span style= "color: #cc7832;" > 
 D = dict (name =  Zhu ' ,< Span style= "color: #aa4926;" >age = 23,score = 88) during a program run, all variables are in memory 
 Pickle.dump (d,f) directly serializes the object and writes to a file The A.txt file is a jumble of content that is the object inside the Python save 
 F.close () Close file         

EG2: Deserializes the object you just saved:

Open (' C:\ \Users\zxq\Desktop\a.txt ',' RB ')
D = pickle.load (f) deserialization out of object
F.close ()
Print D Output: {' age ': +, ' score ': ", ' name ': ' Zhu '}

2. Json:python language-specific serialization modules are pickle, but if you make serialization more generic and Web-compliant, you can use the JSON module

　If we are going to pass objects between different programming languages, we have to serialize the object into a standard format, such as XML, when the better way is to serialize JSON, because JSON represents a string that can be read by all languages, easily stored to disk, or transmitted over a network. JSON is not only a standard format, but also faster than XML, and can be read directly in the Web page, very convenient.

JSON represents objects that are standard JavaScript language objects, and JSON and Python have built-in data types that correspond to the following:

JSON type Python type

{} dict

[] List

' String ' str

123.56 int\float

True\false True\false

Null None

Python's built-in JSON module provides a very sophisticated translation of Python objects into JSON format. How to turn a Python object into a JSON:

Import JSON

D = dict (name = ' Zhu ', age = 23,score = 88)

Json.dumps (d) Output:{' age ': $, ' score ':, ' name ': ' Zhu ' }

JSON is deserialized into a Python object:

>>> json_str = ‘{"age": 20, "score": 88, "name": "Bob"}‘>>> json.loads(json_str){u‘age‘: 20, u‘score‘: 88, u‘name‘: u‘Bob‘}

IO programming, manipulating files or directories, serialization, JSON