Python Development Learning-DAY08 (socket advanced, socketserver, process, thread)

S12-20160305-day08

Pytho Automation Development day08date:2016.03.05

    @南非波波

Course Outline:

Day07

Http://www.cnblogs.com/alex3714/articles/5213184.html

Day08

Http://www.cnblogs.com/alex3714/articles/5227251.html

Featured Movies

绝美之城  上帝之城 | 千与千寻  龙猫 卡尔的移动城堡

With instance private variables, you need to encapsulate a method in your class that returns the value of a private variable

First, Socket deep

1. Concept

Unix的进程通信机制。一个完整的socket有一个本地唯一的socket号，由操作系统分配。socket是面向客户/服务器模型而设计的，针对客户和服务器程序提供不同的socket系统调用。socket利用客户/服务器模式巧妙的解决了进程之间建立通信连接的问题。套接字（socket）是通信的基石，是支持TCP/IP协议的网络通信的基本操作单元。它是网络通信过程中端点的抽象表示，包含进行网络通信必须的五种信息：连接使用的协议，本地主机的IP地址，本地进程的协议端口，远地主机的IP地址，远地进程的协议端口。

2. Address Cluster

socket.AF_UNIX unix本机进程间通信 socket.AF_INET　使用IPV4地址协议进行进程间通信socket.AF_INET6  使用IPV6地址协议进行进程间通信

3. Socket type

socket.SOCK_STREAM  #使用tcp协议socket.SOCK_DGRAM   #使用udp协议socket.SOCK_RAW     #原始套接字，普通的套接字无法处理ICMP、IFMP等网络报文，而SOCK_RAM可以。其次SOCK_RAM也可以处理特殊的IPV4报文,此外，利用原始套接字可以通过IP_HDRINCL套接字选项由用户构造IP头。socket.SOCK_RDM    #是一种可靠的UDP形式，即保证交付数据报但不保证顺序。SOCK_RAW用来提供对原始协议的低级访问，在需要执行某些特殊操作时使用，如发送ICMP报文。SOCK_RAW通常仅限于高级用户或管理员运行的程序使用。

4.socket method

1. Socket.socket (Family=af_inet, Type=sock_stream, Proto=0, Fileno=none) 2. Socket.socketpair ([family[, type[, Proto]]) 3. Socket.create_connection (address[, timeout[, Source_address]) 4. Socket.getaddrinfo (host, Port, Family=0, Type=0, proto=0, flags=0) Gets the peer host address 5 to connect to. Sk.bind (address) binds a socket to an address, the format of which is dependent on the address cluster. Under Af_inet, address is represented as a tuple (host.port). 6. Sk.listen begins to listen for incoming connections, and the backlog specifies the maximum number of connections that can be suspended before a connection is rejected. The backlog equals 5, indicating that the kernel is connected to a connection request, but the server has not yet called accept to handle the maximum number of connections to 5. This value is set according to the kernel and server physical configuration. 7. Sk.setblocking (BOOL) is blocked (default true), if set to false, then accept and recv when there are countless data errors. 8. Sk.accept () accepts the connection and returns (Conn,address), where Conn is a new socket object that can be used to receive and send data, and address is the addresses of the connecting clients. The connection (blocking) of the receiving TCP client waits for the connection to arrive. 9. Sk.connect (address) is connected to the socket at address. Generally, address is in the form of a tuple (Hostname,port) and returns a socket.err error if there is an error in the connection. SK.CONNECT_EX (address), but there will be a return value, the connection is successful when the return of 0, the connection fails when the encoding is returned, for example: 1006111. Sk.close () Close socket 12. SK.RECV (Bufsize[,flag]) receives the data for the socket, and the data is returned as a string. BUFSIZE Specifies the maximum quantity that can be received, and it is recommended not to exceed 1024*8. Flag provides additional information about the message. can usually be ignored. Sk.recvfrom (Bufsize[.flag]) andRecv () is similar, but the return value is (data,address). Where data is the string that contains the received information, address is the socket 14 that sends the data. Sk.send (String[,flag]) sends the data in a string to the connected socket, the return value is the number of bytes to send, which may be less than the byte size of a string, that is, the contents of the specified content may not be sent all 15. Sk.sendall (String[,flag]) sends the data in a string to the connected socket, but attempts to send all the data before returning. Successful return none, failure runs out of exception. Internally sends all the content through a recursive call to send. Sk.sendto (string[,flag],address) sends data to the socket, address is a tuple in the form of (Ipaddr,port), specifying the remote address. The return value is the number of bytes sent. This function is mainly used for UDP protocol. Sk.settimeout (timeout) sets the timeout period for a socket operation, and timeout is a floating-point number in seconds. A value of None indicates no over-time. In general, hyper-times should be set when you just create a socket, because they may be used for connected operations (such as client connections, which wait up to 5s) 18. Sk.getpeername () Returns the remote address of the connection socket. The return value is typically a tuple (ipaddr,port) 19. Sk.getsockname () returns the socket's own address, usually in a tuple (Ipaddr,port) 20. Socket.gethostname () Gets the host name of the computer on which the program is running 21. gethostbyname (name) attempts to interpret the given host name as an IP address. You will first check whether the current computer can be interpreted. If not, an explanation request is sent to a remote DNS server (the remote DNS server may also forward the interpretation request to another DNS server until the request can be processed). The gethostbyname function returns this IP address or throws an exception after a lookup failure. For example: Socket.gethostbyname (' www.apicloud.com ') extended form: socket.gethostbyname_ex (' www.apicloud.com ') (' 98e86f98d416f10c.7cname.com ', [' www.apicloud.com '], [' 117.25.140.17 ']) it returns a tuple of three elements, namely a list of the primary host name for the given address, an optional hostname for the same IP address, and a list of other IP addresses for the same interface on the same host (the list may be empty).    The GETHOSTBYADDR function works the same as GETHOSTBYNAME_EX, except that the argument you provide to it is an IP address string. Socket.gethostbyaddr (' 202.165.102.205 ') (' homepage.vip.cnb.yahoo.com ', [' www.yahoo.com.cn '], [' 202.165.102.205 ']) The Getservbyname (service,protocol) function requires a service name (such as ' Telnet ' or ' ftp ') and a protocol (such as ' TCP ' or ' UDP ') to return the port number used by the service: >>> socket . Getservbyname (' http ', ' TCP ') >>> socket.getservbyname (' https ', ' TCP ') 443 >>> Socket.getse Rvbyname (' telnet ', ' TCP ') 24. Sk.fileno () Socket file descriptor Socket.sendfile (files, offset=0, count=none) send files, but most of the time there is no egg

Second, Socketserver

Reference Link: http://my.oschina.net/u/1433482/blog/190612

Socketserver simplifies the writing of Web servers. It has 4 classes: Tcpserver,udpserver,unixstreamserver,unixdatagramserver. These 4 classes are processed synchronously and are supported asynchronously through the Forkingmixin and ThreadingMixIn classes.

Steps to create a server

首先，你必须创建一个请求处理类，它是BaseRequestHandler的子类并重载其handle()方法。其次，你必须实例化一个服务器类，传入服务器的地址和请求处理程序类。最后，调用handle_request()(一般是调用其他事件循环或者使用select())或serve_forever()。

Server type

5种类型：BaseServer，TCPServer，UnixStreamServer，UDPServer，UnixDatagramServer。 注意：BaseServer不直接对外服务。

Server object

Class Socketserver.baseserver: This is the superclass of all the server objects in the module. It defines the interface, as described below, but most of the methods are not implemented and are refined in subclasses. Baseserver.fileno (): Returns the integer file descriptor of the server listener socket. Typically used to pass to Select.select () to allow a process to monitor multiple servers. Baseserver.handle_request (): Processes a single request. Processing order: Get_request (), Verify_request (), Process_request (). If the user provides the handle () method throws an exception, the server's Handle_error () method is called. If no request is received within the Self.timeout, the Handle_timeout () is called and the Handle_request () is returned. Baseserver.serve_forever (poll_interval=0.5): Processes the request until an explicit shutdown () request is made. Polls every poll_interval seconds for shutdown. Ignore Self.timeout. If you need to do periodic tasks, it is recommended to place them on other threads. Baseserver.shutdown (): Tells the Serve_forever () loop to stop and wait for it to stop. python2.6 version. Baseserver.address_family: Address families, such as Socket.af_inet and Socket.af_unix. Baseserver.requesthandlerclass: A user-supplied request processing class that creates an instance for each request. Baseserver.server_address: The address on which the server listens. The format varies according to the protocol family address, see the documentation for the socket module. Baseserver.socketsocket: The server on the server that listens for incoming requests to the socket object. The server class supports the following class variables: baseserver.allow_reuse_address: Whether the server allows the reuse of addresses. The default is false and can be changed in subclasses. Baseserver.request_queue_size the size of the request queue. If a single request takes a long time to process, the server is busy when the request is placed in the queue, up to a maximum of request_queue_size. Once the queue is full, requests from the client will get a "Connection denied" error. ImpliedThe recognition value is typically 5, but can be overridden by a quilt class. Baseserver.socket_type: The type of socket used by the server; Socket. Sock_stream and Socket.sock_dgram and so on. Baseserver.timeout: Timeout, in seconds, or none means no time-out. If Handle_request () does not receive a request within a timeout, handle_timeout () is called. The following methods can be overridden by subclasses, which have no effect on external users of server objects. Baseserver.finish_request (): actually handles the Requesthandlerclass initiated request and calls its handle () method. Common. Baseserver.get_request (): Accepts the socket request and returns a two-tuple containing the new socket object to be used for communication with the client, and the address of the client. Baseserver.handle_error (Request, client_address): called If Requesthandlerclass's handle () method throws an exception. The default action is to print Traceback to standard output and continue processing other requests. Baseserver.handle_timeout (): Timed out processing. By default, the forking server is a child process state that collects exits, and the threading server does nothing. Baseserver.process_request (Request, client_address): Call Finish_request () to create an instance of Requesthandlerclass. If required, this feature can create new processes or threads to handle requests, which the Forkingmixin and ThreadingMixIn classes do. Common. Baseserver.server_activate (): Activates the server through the server's constructor. The default behavior is to listen only to the server socket. can be overloaded. Baseserver.server_bind (): Binds the socket to the desired address by invoking the server's constructor. can be overloaded. Baseserver.verify_request (Request, client_address): Returns a Boolean value that, if true, will be processed and the request will be rejected. This feature can be overridden to implement access control to the server. The default implementation always returns TRUE. Client_address can limit the client, such as processing only the specified IP rangeThe request. Common.

Request processor

处理器接收数据并决定如何操作。它负责在socket层之上实现协议（i.e., HTTP, XML-RPC, or AMQP)，读取数据，处理并写反应。可以重载的方法如下：setup(): 准备请求处理. 默认什么都不做，StreamRequestHandler中会创建文件类似的对象以读写socket.handle(): 处理请求。解析传入的请求，处理数据，并发送响应。默认什么都不做。常用变量：self.request，self.client_address，self.server。finish(): 环境清理。默认什么都不做，如果setup产生异常，不会执行finish。通常只需要重载handle。self.request的类型和数据报或流的服务不同。对于流服务，self.request是socket 对象；对于数据报服务，self.request是字符串和socket 。可以在子类StreamRequestHandler或DatagramRequestHandler中重载，重写setup()和finish() ，并提供self.rfile和self.wfile属性。 self.rfile和self.wfile可以读取或写入，以获得请求数据或将数据返回到客户端。

Sample code

2. Server

     #!/usr/local/env python3 "author:@ South Africa Bobo blog:http://www.cnblogs.com/songqingbo/ E-mail:[email protected] "Import Socketserverclass myhandleserver (socketserver.        Baserequesthandler): ' Defines a test socketserver class ' ' Def handle (self): ' defines a function to handle requests from each client : Return: "Print (" newly established Connection: ", self.client_address) while TRUE:TRY:CLI Ent_data = SELF.REQUEST.RECV (1024x768) if not client_data:print ("The data sent by the client is empty, active disconnected!", SELF.CLI ent_address) Break print ("Request from client:", Client_data.decode ()) Self.request.s                End (Client_data) except Connectionreseterror:print ("Client Active disconnect!", self.client_address) Breakif __name__ = = "__main__": Host,port = "127.0.0.1", and the server = Socketserver. Threadingtcpserver ((host,port), Myhandleserver) server.serve_forever ()  

4. Client

   #!/usr/local/env python3‘‘‘Author:@南非波波Blog:http://www.cnblogs.com/songqingbo/E-mail:[email protected]‘‘‘import socketip_port = ("127.0.0.1",5000)sk = socket.socket()sk.connect(ip_port)while True:    msg = input(">>:").strip()    if not msg:        break    sk.sendall(bytes(msg,"utf8"))    server_reply = sk.recv(1024)    print("服务端返回:",str(server_reply,"utf8"))sk.close()

Third, exception handling

Http://www.cnblogs.com/wupeiqi/articles/5017742.html

2. The anomaly basis (Python3 's notation)

   In the programming process in order to increase the friendliness of the program when a bug is not usually displayed to the user error message, but instead of displaying a prompt page, popular is not to let the user see the Code error page

   try:    passexcept Exception as ex:    pass

3. Type of exception

   Common exceptions:

   AttributeError 试图访问一个对象没有的树形，比如foo.x，但是foo没有属性xIOError 输入/输出异常；基本上是无法打开文件ImportError 无法引入模块或包；基本上是路径问题或名称错误IndentationError 语法错误（的子类） ；代码没有正确对齐IndexError 下标索引超出序列边界，比如当x只有三个元素，却试图访问x[5]KeyError 试图访问字典里不存在的键KeyboardInterrupt Ctrl+C被按下NameError 使用一个还未被赋予对象的变量SyntaxError Python代码非法，代码不能编译(个人认为这是语法错误，写错了）TypeError 传入对象类型与要求的不符合UnboundLocalError 试图访问一个还未被设置的局部变量，基本上是由于另有一个同名的全局变量，导致你以为正在访问它ValueError 传入一个调用者不期望的值，即使值的类型是正确的

   More kinds:

     Arithmeticerrorassertionerrorattributeerrorbaseexceptionbuffererrorbyteswarningdeprecationwarningenvironmenter Roreoferrorexception #捕获一般的所有异常FloatingPointErrorFutureWarningGeneratorExitImportError # Capture Import Module Exception importwarningindentationerrorindexerror #捕获索引异常IOError #捕获IO异常KeyboardInterrupt # Catch keyboard key combination exception keyerrorlookuperrormemoryerrornameerrornotimplementederroroserroroverflowerrorpendingdeprecationwarningreference Errorruntimeerrorruntimewarningstandarderrorstopiterationsyntaxerrorsyntaxwarningsystemerrorsystemexittaberrortypeerrorun Boundlocalerrorunicodedecodeerrorunicodeencodeerrorunicodeerrorunicodetranslateerrorunicodewarninguserwarningvalueerrorwa Rningzerodivisionerror  

5. Custom exception Classes

   #!/usr/local/env python3‘‘‘Author:@南非波波Blog:http://www.cnblogs.com/songqingbo/E-mail:[email protected]‘‘‘class SwhtError(Exception):    ‘‘‘    自定义异常    ‘‘‘    def __init__(self,msg):        ‘‘‘        初始化函数        :param msg:用户输入message        :return:        ‘‘‘        self.message = msg    def __str__(self): #名称可以自行定义，只是通过该方法返回message的值        ‘‘‘        返回用户输入的信息        :return:        ‘‘‘        return self.messagetry:    raise SwhtError("这是一个致命的错误！")except Exception as e:    print("dsdsd",e)

7. Example code:

   dic = ["swht", ‘shen‘]try:    dic[10]except IndexError as e:    print("IndexError:",e)dic = {‘k1‘:‘v1‘}try:    dic[‘k20‘]except KeyError as e:    print("keyError:",e)s1 = ‘hello‘try:    int(s1)except ValueError as e:    print("ValueError:",e)

9. Special exceptions

   Although Python comes with a handle to the universal exception class exception, it is not possible to catch a few exceptions. If you want to catch these special exceptions, you need to make a custom exception class

11. Exception other schemas

    try:    # 主代码块    passexcept KeyError as e:    # 异常时，执行该块    passelse:    # 主代码块执行完，执行该块    passfinally:    # 无论异常与否，最终执行该块    pass

13. Proactively triggering exceptions

    try:    raise Exception(‘错误了。。。‘)except Exception as e:    print("Error",e)

15. Asser Assertion:

    Critical judgment, forcing you to determine whether the preceding business results meet the requirements, or throw an exception

    a = 1try:    assert a == 2    print("True")except Exception as e:    print("False",e)

Iv. Processes and Threads

1. Concept

   1. Process

      一个具有一定独立功能的程序关于某个数据集合的一次运行活动，是系统进行资源分配和调度运行的基本单位

   2. Thread

      线程是操作系统能够进行运算调度的最小单位。它被包含在进程之中，是进程中的实际运作单位。一条线程指的是进程中一个单一顺序的控制流，一个进程中可以并发多个线程，每条线程并行执行不同的任务

   3. The difference between a process and a thread

      1. 进程是一个动态的概念进程是程序的一次执行过程，是动态概念程序是一组有序的指令集和，是静态概念2. 不同的进程可以执行同一个程序区分进程的条件：所执行的程序和数据集合。两个进程即使执行在相同的程序上，只要他们运行在不同的数据集合上，他们也是两个进程。例如：多个用户同时调用同一个编译程序编译他们编写的C语言源程序，由于编译程序运行在不同的数据集合（不同的C语言源程序）上，于是产生了一个个不同的进程3. 每个进程都有自己的生命周期当操作系统要完成某个任务时，它会创建一个进程。当进程完成任务之后，系统就会撤销这个进程，收回它所占用的资源。从创建到撤销的时间段就是进程的生命期4. 进程之间存在并发性在一个系统中，同时会存在多个进程。他们轮流占用CPU和各种资源5. 进程间会相互制约进程是系统中资源分配和运行调度的单位，在对资源的共享和竞争中，必然相互制约，影响各自向前推进的速度6. 进程可以创建子进程，程序不能创建子程序7. 从结构上讲，每个进程都由程序、数据和一个进程控制块（Process Control Block, PCB）组成

2. Process Lock

Python Threading Module

1.  线程的两种调用方式    1.  直接调用    2.  继承式调用2.  join&&Demo3.  线程锁    1.  互斥锁    2.  递归锁    3.  Semaphore(信号量)4.  

Example code:

#!/usr/local/env python3import threading,timedef addNum():    global num #声明修改全局变量    print("--get num:",num)    time.sleep(1)    lock.acquire()    num -=1    lock.release()lock = threading.Lock()num = 100threading_list = []for i in range(2040):    t = threading.Thread(target=addNum)    t.start()    threading_list.append(t)for t in threading_list: #等待所有线程执行完毕    t.join()print("num:",num)

Event

Multi-process

Inter-process communication

队列管道manager

Process synchronization

Process Pool

Python Development Learning-DAY08 (socket advanced, socketserver, process, thread)