Python's network programming

14.1.1 Socket Module

In network programming, a basic component of Lourdes is the socket. Sockets are primarily information channels between two programs.

Sockets consist of two: Server sockets and Client sockets. After creating a server socket, let it wait for the connection. So it listens at a network address.

A socket is an instance of the socket class in a socket module. Its instantiation requires 3 parameters: the first parameter is the address family (default is Socket.af_inet), and the 2nd parameter is the stream (socket. Sock_stream, default value) or datagram (socket. SOCK_DGRAM) sockets. The third parameter is the protocol used (the default is 0).

After the server-side socket uses the Bind method, the Listen method is called to listen to the given address. The client socket uses the Connect method to connect to the server, and the address used in the Connect method is the same as the address in the bind method. In this case, an address is a tuple in the format (Host,port) where host is the hostname and port is the port number. The Listen method has only one parameter, which is the length of the connection that the server has not processed.

After the server-side socket starts listening, it can accept the client connection. This step is done using the Accept method. This method blocks until the client connects, and the method returns a tuple in the format (client,address), which is a client socket, and address is the one previously explained. The server can handle the client to its satisfaction, and then call another accept method to start waiting for the next connection.

Sockets have two methods: Send and recv (dare to receive) for transmission. You can use a string parameter to call send to send data and call recv with a required number of bytes to receive data. If you are not sure which number to use is better, then 1024.

A small server

Import socket

s = Socket.socket ()

Host = Socket.gethostname ()

Port = 1234

S.bind ((Host,port))

S.listen (5)

While True:

C, addr = S.accept ()

print ' Got connection from ', addr

C.send (' Thank for connecting ')

C.close ()

A small client

Import socket

s = Socket.gethostname ()

Port = 1234

S.connect ((Host,port))

Print S.RECV (1024)

14.1.2 Urllib and URLLIB2 modules

The most powerful of the various network workspaces available are urllib and URLLIB2. They allow access to files over the network, just as those files exist on your computer. With a simple function call, it is possible to use almost anything that the URL points to as input to the program.

The two modules have the same function, but the URLLIB2 is a little better. Urllib2 is a good choice if you need to use HTTP authentication or cookies, or if you want to write extensions for your own protocols.

1. Open a remote file

You can open a remote file as if you were opening it locally, except that you can use read-only mode, using Uriopen from the Urllib module, not the Open

>>>from urllib Import Urlopen

>>>webpage = Urlopen (' http://www.python.org ')

The class file object returned by Urlopen supports the Close,read,readline,readlines method, and of course it supports iterations.

If you want to extract the URL of the "about" link in the Python page that you opened earlier, you can use a regular expression to implement it.

>>>import RE

>>>text = Webpage.read ()

>>>m = Re.search (' <a href= ' ([^ "]+)".? >about</a> ', Text,re. IGNORECASE)

>>>m.group (1)

'/about/'

2. Get Remote Files

The function Urlopen provides a class file object from which data can be read. If you want Urllib to download a file for you and store a copy of the file in your local file, you can use Urlretrieve. Urlretrieve returns a tuple (filename,headers) instead of a class file object, filename is the name of the local file (created automatically by Urllib), headers contains information about some remote files, and if you want to specify a file name for the downloaded copy, Can be given in the 2nd parameter of the Urlretrieve function.

Urlretrieve (' http://www.python.org ', ' c:\\python_webpage.html ')

This statement gets the Python home page and stores it in HTML, and if no filename is specified, the file is placed in a temporary location and opened with the open function, but you can delete it to save hard disk space if you complete the operation on it. To clean up a temporary file, you can call the Urlcleanup function, but do not provide parameters, and the function will be responsible for cleaning up the work.

14.2 Socketserver and his friends.

The Socketserver module is the basis for many of the server frameworks in the standard library, including Basehttpserver,simplehttpserver,cgihttpserver, Simplexmlrpcserver and Docxmlrpcserver, all of these server frameworks add specific functionality to the underlying server.

Socketserver contains 4 basic classes: TCPServer for TCP sockets, Udpserver for UDP datagram sockets, and Unixstreamserver and unixdatagramserver for non-targeted. The latter 3 may not be used.

To write a server that uses the Socketserver framework, most of the code is in a request handler, and each time the server receives a request, it instantiates a request handler, and its various processing methods are invoked when the request is processed. Which method is called depends on the specific server and the handler class used, so that you can subclass them so that the server invokes a custom handler set. The basic Baserequesthandler class puts all the operations into a processor called handle method, which is called by the server. This method then accesses the client socket in the property self.request. If you are using a stream, you can use the Streamrequesthandler class, create additional two new properties, Self.rfile (for reading) and Self.wfile (for writing), and then use these class file objects to communicate with the client.

The other classes in the Socketserver framework implement basic support for HTTP servers, including running CGI scripts.

A small server based on Socketserver

From Socketserver import Tcpserver,streamrequesthandler

Class Handler (Streamrequesthandler):

def handle (self):

addr = Self.request.getpeername ()

print ' Got connection from ', addr

Self.wfile.write (' Thank for connecting ')

Server = TCPServer ((' ', 1234), Handler)

Server.serve_forever ()

14.3 Multiple connections

The server solutions discussed so far are synchronous: only one client can be linked at a time and requests to process it. It is important to be able to handle multiple connections at the same time if each request takes only a few minutes, such as a full chat session.

There are 3 main ways to accomplish this: bifurcation, threading, and asynchronous I/O. By using mixed classes with Socketserver servers, derived processes and threads are easy to handle.

Fork occupies resources, and if there are too many clients the fork can not be well forked

Thread handling can cause synchronization problems.

Twisted is a very powerful asynchronous network that becomes a framework.

14.3.1 using Socketserver for forking and threading

Windows does not support forking

Servers using fork Technology

From Socketserver import Tcpserver,forkingminin,streamrequesthandler

Class Server (forkingmixin,tcpserver): Pass

Class Handler (Streamrequesthandler):

def handle (self):

addr = Self.request.getpeername ()

print ' Got connection from ', addr

Self.wfile.write (' Thank for connecting ')

Server = Server ((' ', 1234), Handler)

Server.serve_forever ()

A server that uses thread processing

From Socketserver import Tcpserver,threadingmixin,streamrequesthandler

Class Server (threadingmixin,tcpserver): Pass

Class Handler (Streamrequesthandler):

def handle (self):

addr = Self.request.getpeername ()

print ' Got connection from ', addr

Self.wfile.write (' Thank for connecting ')

Server = Server ((' ', 1234), Handler)

Server.serve_forever ()

14.3.2 asynchronous I/O with Select and poll

When a server communicates with a client, the data from the client may not be contiguous. If you use forks or threading, that's not a problem. When a program waits for data, another parallel program can continue to process their own clients. Another way to deal with this is to handle only the clients that are actually communicating at a given time. No need to keep listening----just listen for a while and then put it behind the other clients.

This is the method used by the Asyncore/asynchar framework and the twisted Framework, which is based on the Select function, if the poll function is available, it can also be it, both functions from the Select module, these two functions, poll scalability is better, However, it can only be used on UNIX systems (Windows is not available).

The Select function requires 3 sequences as its required parameters, and an optional time-out in seconds as the 4th parameter. These sequence-file descriptor integers. These are the connections we wait for. 3 sequences are used for input, output, and exception conditions. If no time-out is given, select blocks until one of the file descriptors is ready for action, and if a time-out is given, select blocks a given timeout time, and if the given time-out is 0, a continuous poll (that is, no blocking) is provided. The return value for select is 3 sequences, each representing an active subset of the corresponding parameters. For example, the 1th sequence returned is a sequence of input file descriptors, in which there are some things to read.

A sequence can contain a file object (which does not work in Windows) or a socket. The server is a simple logger that outputs all the data from the client. You can use Telnet to connect to it for testing. Try to connect using multiple Telnet to verify that the server can serve multiple clients at the same time.

A simple server that uses select

Import Socket,select

s = Socket.socket ()

Host = Socket.gethostname ()

Port = 1234

S.bind ((Host,port))

S.listen (5)

inputs = [s]

While True:

Rs.ws,es = Select.select (inputs,[],[])

For R in RS:

If R is s:

C, addr = S.accept ()

print ' Got connection from ', addr

Inputs.append (c)

Else

Try

data = R.RECV (1024)

disconnected = Not data

Except Socket.error:

disconnected = True

If disconnected:

Print r.getpeername (), ' Disconnected '

Inputs.remove (R)

Else

Print data

The poll method is simpler to use than select. When you call poll, you get a poll object. You can then register a file descriptor with the Pol object's Register method. After registering, you can use the Unregister method to remove registered objects. After registering some objects, you can call the poll method and get a list of formats, where FD is the file descriptor, and the event tells you what happened. This is a bitmask, meaning that it is an integer, and each bit of the integer corresponds to a different event. Those different events are constants of the Select module.

Simple server that uses poll

Import Socket.select

s = Socket.socket ()

Host = Socket.gethostname ()

Port = 1234

S.bind ((Host,port))

Fdmap = {S.fileno (): s}

S.listen (5)

p = select.poll ()

P.register (s)

While True:

Events = P.poll ()

For FD, event in events:

If FD = S.fileno ():

C,ADDR = S.accept ()

print ' Got connection from ', addr

P.register (c)

Fdmap[c.fileno ()] = C

Elif Event & Select. Pollin:

data = FDMAP[FD].RECV (1024)

If not data:

Print fdmap[fd].getpeername (), ' Disconnected '

P.unregister (FD)

Del FDMAP[FD]

Else

Print data

14.4 Twisted

From twisted.internet Import reactor

From Twisted.internet.protocol import protocol.factory

Class Simplelogger (Protocol):

def connectionmade (self):

print ' Got connection from ', self.transport.client

def connectionlost (Self,reason):

Print self.transport.client, ' disconnected '

def datareceived (Self,data):

Print data

Factory = Factory ()

Factory.protocol = Simplelogger

Reactor.listentcp (1234,factory)

Reactor.run ()

If you use Telnet to connect to this server and test it, then each line may only output one character depending on the buffer or something like that. Of course, you can use Sys.stdout.write instead of print.

A record server that uses the Linereceiver protocol improvement

From twisted.internet Import reactor

From Twisted.internet.protocol import Factory

From Twisted.protocols.basic import Linereceiver

Class Simplelogger (Linereceiver):

def simplelogger (linereceiver):

print ' Got connection from ', self.transport.client

def connectionlost (Self,reason):

Print self.transport.client, ' disconnected '

def linereceived (self,line):

Print Line

Factory = Factory ()

Factory.protocol = Simplelogger

Reactor.listentcp (1234,factory)

Reactor.run ()

This article is from the "linux_oracle" blog, make sure to keep this source http://pankuo.blog.51cto.com/8651697/1661448

Python's network programming