13th article: HTTP protocol for Web server

This article is mainly to implement the Web server, which contains the understanding of the HTTP protocol, as well as TCP three handshake, four waves and other aspects of knowledge, but also contains information about the Web browser and server communication process.

One, web browser

Usually when we surf the Internet in the browser address bar input URL, ①, browser first to parse the URL, ②, then define the message content and steps through the HTTP protocol, that is, the format of sending the request; ③, According to the server's domain name through the operating system parser (DNS client) to the nearest DNS server to obtain the IP address of the target server and stored in the specified memory space, through the operating system under the protocol stack and socket library to send messages out, ④, When the server receives a request message, it returns a response message (which is also the format for defining the message content based on the HTTP protocol), which is then returned to the Web browser through a similar process. Next we'll follow these steps to parse:

　　1. How does the browser parse the URL?

There are several common mechanisms for accessing data, such as the following:

HTTP protocol: Hypertxt Transfer protocal Hypertext Transfer Protocol) access to the Web server, for example: http://www.baidu.com/dir/file1.html

FTP protocol: Files Transfer Protocol, File Transfer Protocol, mainly used for file upload and download, for example: ftp://ftp.glasscom.com/dir/file1.html

File protocol: Local files Transfer Protocol, for example: File://localhost/c:path/file1.zip.

Maito protocol: This protocol can create a hyperlink to an e-mail address that can be used to send e-mail messages over the Internet. For example: [email protected]

Wait a minute

　2. What format request messages and received response messages are generated according to the HTTP protocol?

What we need to know is:

HTTP protocol:

We know that the process of communication between the server and the client is that the client sends a request message to the service side based on the HTTP protocol, and then sends a response message to the client . So what exactly is the request message and response message?

　　Request message:

1 #here is the request message that the browser sends to the server2 3get/http/1.14 Host:www.baidu.com5connection:keep-Alive6Cache-control:max-age=07Upgrade-insecure-requests:18user-agent:mozilla/5.0 (Windows NT 10.0; Win64; x64) applewebkit/537.36 (khtml, like Gecko) chrome/67.0.3396.99 safari/537.369accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8Tenaccept-encoding:gzip, deflate, BR Oneaccept-language:zh-cn,zh;q=0.9 A  - #This information is then parsed:
 　　 the first part: The request header line,  contains the request type, URI, HTTP protocol version;
The request information type usually has: get, post, put and so on;
　　the second part: immediately after the first line, the request of the head,  contains the description information used by the server;  Next, explain what these instructions mean:
1. Host: requesting the Web server's domain name address
2, Connection: Indicates whether the connection is persistent, that is, keep-alive means persistent connection;
3. Cache-control: Specifies the caching mechanism for requests and responses, No-cache (cannot be cached), No-store (Sent in the request message will make the request and response messages do not use the cache),
Max-age (the client can receive a response that is not longer than the specified time (in seconds), Max-stale (the client can receive a response message that exceeds the timeout period), Min-fresh (the client can receive a response less than the current time plus a specified time),
only-if-cached, wait.
4. User-agent:http details of the browser type that the protocol is running on, such as: Google/67.0.3396.99
5, Accept: Refers to the content type that the browser can receive;
6, Accept-encoding: The client browser can support the Web server return content compression encoding type;
7. Accept-language: The type of language supported by the browser,
8. Cookies: Data stored on the user's local terminal for the purpose of identifying the user and tracking the session (usually encrypted); For example, when we surf the internet, certain websites can accurately push the information we want.
 　　 Part Three: "\ r \ n"--dividing the header and body parts

Response message:

1 #Response Message2 3http/1.1 200OK4Bdpagetype:15Bdqid:0x8bda58760001baca6cache-control:private7connection:keep-Alive8content-Encoding:gzip9content-type:text/HTMLTencxy_all:baidu+10412ee70bbb9e9eec33f3dbcb3e2df7 Onedate:wed, 2018 03:26:13GMT Aexpires:wed, 2018 03:25:42GMT -server:bws/1.1 -set-cookie:bdsvrtm=0; path=/ theset-cookie:bd_home=0; path=/ -set-cookie:h_ps_pssid=1435_21118_20929; path=/; domain=. baidu.com -strict-transport-security:max-age=172800 -vary:accept-Encoding +X-ua-compatible:ie=edge,chrome=1 -transfer-encoding:chunked +  A #parsing of response messages:
　　The first part: The response header, contains: HTTP protocol version, status code (1xx-tells the request processing progress and situation, 2xx-success, 3xx-indicates the need for further action, 4xx-client error; 5xx-server error;)
The second part: The response header, contains additional information sent by the server; Here are some important explanations for the explanation:
Https://www.cnblogs.com/mylanguage/p/5689879.html---with detailed instructions.
The third part: "\ r \ n"-dividing the dividing line between header and body
Part IV: Contains the data that the server sends to the client.

The content format of the request message and response message above is translated by the browser or client based on the HTTP protocol.

3, how to get the IP address of the server according to the domain name?

It is mentioned that in order to realize the communication between the two terminals must know the other side's IP address and port number, but we entered in the browser address bar is only the domain name, then how do we get the IP address of the target server?

First the Web browser invokes the parser under the operating system as the DNS client, which is then sent by the parser to the nearest DNS server (the sending process is the same as the C/s schema model), and if the required domain name is not in the nearest DNS server, the server sends a inquiries message to the domain's DNS server. If the domain name is not on the root domain DNS server, let the recent DNS server send a inquiries message to its subordinate domain, recursively find the DNS server of the domain in which the domain name resides, and finally send a response message from the requesting DNS server to the nearest DNS server to get the IP address of the domain name;

At the same time, the DNS server has a cache function, you can remember the domain name previously queried, if the queried domain name and related information is already in the cache, so long can directly return the response.

Two, three handshake and four waves of TCP

First we need to know that the client and the server based on the TCP protocol to establish a contact with three handshake, and in the disconnection to go through four waves of the process, then we look at the process is what?

　　Three-time handshake:

　　　　

Analytical:

First the server is usually in the state of listening, and the client is usually the one that actively establishes the connection, i.e.

1. The TCP server process first creates the transport control block TCB, ready to accept the client process's connection request, when the server enters the listen (listening) state;
2. The TCP client process also creates the transport control block TCB and then sends a connection request message to the server, which is the same location syn=1 in the header of the message and selects an initial sequence number seq=x, at which point the TCP clients process enters the syn-sent (synchronous sent state) state. TCP Specifies that the SYN message segment (syn=1 segment) cannot carry data, but consumes a sequence number.
3. When the TCP server receives the request message, it sends a confirmation message if it consents to the connection. Confirm that the message should be ack=1,syn=1, the confirmation number is ack=x+1, but also to initialize a serial number for themselves seq=y, at this time, the TCP server process entered the SYN-RCVD (synchronous received) state. This message cannot carry data, but it also consumes a serial number.
4. After the TCP client process receives the acknowledgement, it also confirms to the server. Confirm the ack=1,ack=y+1 of the message, its own serial number seq=x+1, at this time, the TCP connection is established, the client enters the established (established connection) state. TCP Specifies that the ACK segment can carry data, but the serial number is not consumed if the data is not carried.
5. When the server receives confirmation from the client, it also enters the established state, after which the two parties can begin to communicate.

　　

　　Four times wave:

At the same time we need to know that the server usually does not actively disconnect, but the client is actively disconnecting the connection.

　　　　

1. The client process sends out a connection release message and stops sending data. Release the data packet header, Fin=1, whose sequence number is seq=u (equal to the last byte of the previously transmitted data sequence number plus 1), at this point, the client enters the state of fin-wait-1 (terminating wait 1). TCP Specifies that the fin segment consumes a sequence number even if it does not carry data.

2. 2, the server receives the connection release message, sends the confirmation message, ack=1,ack=u+1, and takes its own serial number seq=v, at this time, the service end enters the close-wait (closes waits) the state. The TCP server notifies the high-level application process, the client to the server direction is released, this time in the semi-closed state, that the client has no data to send, but the server if the data sent, the client will still accept. This state also lasts for some time, that is, the duration of the entire close-wait state.
4. 3, the client receives the server confirmation request, at this time, the client enters Fin-wait-2 (terminates waits 2) the state, waits for the server to send the connection release message (before this also needs to accept the server sends the final data).
6. 4, the server will send the final data sent to the client to send a connection release message, Fin=1,ack=u+1, because in the semi-closed state, the server is likely to send some data, assuming that the serial number is seq=w, at this time, the server entered the Last-ack (final confirmation) status , waiting for the client to confirm.
8. 5, the client receives the server's connection release message, must issue the confirmation, ack=1,ack=w+1, but own serial number is seq=u+1, at this time, the client enters the time-wait (time Waits) state. Note that at this point the TCP connection has not been released and must go through 2?? After the time of the MSL (maximum message segment life), the client revokes the corresponding TCB before it enters the closed state.
10. 6, the server as long as received confirmation from the client, immediately into the closed state. Again, after the TCB is revoked, the TCP connection is ended. As you can see, the server ends the TCP connection a bit earlier than the client.

Third, implement the Web static server

Look directly at the example:

ImportSocketImportRedefserver_client (server_client_socket):"""receive data from the browser and send a delivery paper"""    #to parse the received request messageRequest = SERVER_CLIENT_SOCKET.RECV (1024x768). Decode ("Utf-8")    #the request message header format is approximately: get/index.html http/1.1Request_lines = Request.splitlines ()#splits the received information by row and returns the listret = Re.match (r"[^/]+(/[^ ]*)", Request_lines[0])#Jo Zheng If the expression match succeeds with a return value, the detection is by "/" or a null character, then the default file is set to/index.html file    ifRet:filename= Ret.group (1)        iffilename = ="/" orfilename = ="": FileName="/index.html"    #Print (Request)Respone="http/1.1 ok\r\n"Respone+="\ r \ n"    #Respone + = "        #detects if there is a file in the server, has the content, or returns an error    Try: F= Open ("HTML"+ filename,"RB")    except: Respone="http/1.1 404 Not found file \ r \ n"Respone+="\ r \ n"Respone+=""Respone= Respone.encode ("Utf-8")    Else: File_content=F.read () f.close () Respone= Respone.encode ("Utf-8") + file_content#stitching between bytesserver_client_socket.send (Respone) server_client_socket.close ()defMain ():#1. Creating a Socket ObjectServer_socket =Socket.socket (Socket.af_inet,socket. SOCK_STREAM) server_socket.setsockopt (socket. Sol_socket, SOCKET. SO_REUSEADDR,1)    #2. Binding Local InformationServer_socket.bind (("", 6969))    #3. MonitoringServer_socket.listen (128)     whileTrue:#4. Waiting for client to connectSERVER_CLIENT_SOCKET,CLIENT_ADDR =server_socket.accept ()#5. Service Clientserver_client (Server_client_socket) server_socket.close ()if __name__=="__main__": Main ()

This realizes a static Web server, can receive the request of the Web browser, parse the requested data, get the file name of the page that needs the browser request through the regular expression, get the data through the reading of the file, and conform the content to the Web browser.

13th: HTTP protocol for Web server