HTTP protocol Detailed

Transferred from: http://blog.csdn.net/gueter/archive/2007/03/08/1524447.aspx

Author:jeffrey

Introduction                                         

HTTP is an object-oriented protocol belonging to the application layer, which is suitable for distributed hypermedia information System because of its simple and fast way. It was proposed in 1990, after several years of use and development, has been continuously improved and expanded. Currently used in the WWW is the sixth edition of Http/1.0, http/1.1 standardization work is in progress, and Http-ng (Next Generation of HTTP) has been proposed. The main features of the HTTP protocol can be summarized as follows: 1. Support client/server mode. 2. Simple and fast: When a customer requests a service from the server, it simply transmits the request method and path. The request method commonly has, POST. Each method specifies a different type of contact between the customer and the server. Because the HTTP protocol is simple, the HTTP server's program size is small, so the communication speed is fast. 3. Flexible: HTTP allows the transfer of any type of data object. The type being transmitted is marked by Content-type. 4. No connection: The meaning of no connection is to limit the processing of only one request per connection. When the server finishes processing the customer's request and receives the customer's answer, the connection is disconnected. In this way, the transmission time can be saved. 5. Stateless: The HTTP protocol is a stateless protocol. Stateless means that the protocol has no memory capacity for transactional processing. A lack of state means that if the previous information is required for subsequent processing, it must be re-routed, which may cause the amount of data to be transferred per connection to increase. On the other hand, it responds faster when the server does not need the previous information.

First, the HTTP protocol detailed URL Chapter

HTTP (Hypertext Transfer Protocol) is a request-and-response mode-based, stateless, application-level protocol, often based on TCP connection, HTTP1.1 version of a continuous connection mechanism, the vast majority of web development, is built on the HTTP protocol on the Web application.

The HTTP url (a URL is a special type of URI that contains enough information to find a resource) is in the following format: http://host[":" Port][abs_path] HTTP means to locate network resources through the HTTP protocol, the host represents a legitimate Internet host domain name or IP address; port specifies a port number, or null specifies the URI of the requested resource using the default port 80;abs_path, if no abs_ is given in the URL Path, it must be given as a "/" when it is a request URI, which is usually done automatically by the working browser. Eg:1, Input:www.guet.edu.cn Browser automatically converted to:http://www.guet.edu.cn/ 2, http:192.168.0.116:8080/ index.jsp

Second, the HTTP protocol detailed request chapter

The HTTP request consists of three parts: the request line, the message header, the request body

1. The request line begins with a method symbol, separated by a space, followed by the requested URI and version of the Protocol, in the following format: Method Request-uri Http-version CRLF where method represents the request approach ; Request-uri is a Uniform resource identifier; Http-version represents the HTTP protocol version of the request, and CRLF represents a carriage return and a newline (except for the CRLF at the end, a separate CR or LF character is not allowed).

The request method, which is all capitalized, is interpreted as follows: The GET request obtains the resource identified by the Request-uri post at the resource identified by Request-uri to append new data to the HEAD request to obtain the assets identified by Request-uri The source response message header PUT Request server stores a resource and uses Request-uri as its identity delete request server to delete the resource identified by Request-uri TRACE Request Server Loopback received request information, primarily for testing or diagnosing CONNECT Reserve the ability to query the server for future use of the options request, or to query resource-related options and requirements application Examples: Get method: When you access a Web page by entering a URL in the address bar of the browser, the browser uses the Get method to get the resource to the server, eg:get/form.html http/1.1 (CRLF)

The Post method requires the requested server to accept the data appended to the request and is often used to submit the form. eg:post/reg.jsp http/(CRLF) accept:image/gif,image/x-xbit,... (CRLF) ...         HOST:www.guet.edu.cn (CRLF) content-length:22 (CRLF) connection:keep-alive (CRLF) Cache-control:no-cache (CRLF) (CRLF) The CRLF indicates that the message header has ended, preceded by the message header user=jeffrey&pwd=1234//This line is the submitted data

The head method is almost the same as the Get method, and for the response part of the head request, the information contained in the HTTP header is the same as the information obtained through the GET request. Using this method, you can obtain information about the resources identified by Request-uri without transmitting the entire resource content. This method is commonly used to test the validity of hyperlinks, whether they can be accessed, and whether they have been updated recently. 2. After the request header 3, the request body (slightly)

Third, the HTTP protocol detailed response Chapter

After receiving and interpreting the request message, the server returns an HTTP response message.

The HTTP response is also made up of three parts, namely: status line, message header, Response body 1, status line format as follows: Http-version Status-code reason-phrase CRLF, Http-version represents the version of the server HTTP protocol, Status-code represents the response status code sent back by the server, and Reason-phrase represents the text description of the status code. The status code consists of three digits, the first number defines the category of the response, and there are five possible values: 1xx: Indicates that the request has been received and continues processing 2xx: Success-Indicates that the request has been successfully received, understood, accepted 3xx: Redirect--must be further manipulated to complete the request 4XX: Client Error-Request syntax error or request cannot be implemented 5XX: server-side error-the server failed to implement a legitimate request common status code, status Description, Description: $ OK//Client request success The bad request//client requests have syntax errors and cannot be The server understands 401 Unauthorized//request unauthorized, this status code must be used with the Www-authenticate header domain 403 Forbidden//server received the request, but refused to provide service 404 Not Found//Request Resources does not exist, eg: entered the wrong URL of Internal Server error//server error//servers occur unexpected Errors 503 Server unavailable//server is currently unable to process client requests, after a period of time may return to normal EG:HTTP/1 .1 OK (CRLF)

2. The response header is described later

3, the response body is the contents of the resources returned by the server

Iv. HTTP protocol Details of the message header chapter

HTTP messages consist of client-to-server requests and server-to-client responses. Both the request message and the response message are from the start line (for the request message, the start line is the request line, for the response message, the start line is the status line), the message header (optional), the empty line (only the CRLF line), and the message body (optional) is composed.

The HTTP message header includes the normal header, the request header, the response header, and the entity header. Each header field consists of a name + ":" + a Space + value, and the name of the message header field is case-insensitive.

1, ordinary header in the normal header, there are a few header fields for all request and response messages, but not for the transferred entity, only for the transmission of the message. The Eg:cache-control is used to specify cache directives, which are unidirectional (the cache instructions appearing in the response may not appear in the request) and are independent (the cache instruction of one message does not affect the caching mechanism of another message processing). HTTP1.0 uses a similar header field for pragma. Cache directives at request include: No-cache (used to indicate that the request or response message cannot be cached), No-store, Max-age, Max-stale, Min-fresh, only-if-cached; Cache directives for response include: public, Private, No-cache, No-store, No-transform, Must-revalidate, Proxy-revalidate, Max-age, S-maxage. Eg: in order to instruct IE browser (client) not to cache the page, the server-side JSP program can be written as follows: Response.sehheader ("Cache-control", "No-cache"); Response.setheader ("Pragma", "No-cache"), function equivalent to the above code, usually both//share this code will set the normal header field in the Sent response message: Cache-control:no-cache

Date Normal header field indicates the day and time the message was generated

The connection normal header field allows the option to send a specified connection. For example, specify that the connection is contiguous, or specify a "close" option to notify the server to close the connection after the response is complete

2. The request header request header allows the client to pass additional information about the request to the server side, as well as the client itself. The common request header of the Accept Accept Request header field is used to specify which types of information the client accepts. Eg:accept:image/gif, indicating that the client wants to accept a resource in GIF image format; accept:text/html, indicating that the client wants to accept HTML text. The Accept-charset accept-charset request header field is used to specify the character set accepted by the client. eg:accept-charset:iso-8859-1,gb2312. If the field is not set in the request message, the default is to accept any character set. The Accept-encoding accept-encoding request header field is similar to accept, but it is used to specify acceptable content encoding. Eg:accept-encoding:gzip.deflate. If the domain server is not set in the request message, the client is assumed to be acceptable for various content encodings. The Accept-language accept-language request header field is similar to Accept, but it is used to specify a natural language. EG:ACCEPT-LANGUAGE:ZH-CN. If the header field is not set in the request message, the server assumes that the client is acceptable for each language. The Authorization Authorization request header domain is primarily used to prove that a client has permission to view a resource. When a browser accesses a page, if a response code of 401 (unauthorized) is received from the server, a request containing the authorization request header domain can be sent, requiring the server to validate it. Host (the header domain is required when sending a request) the host request header domain is used primarily to specify the Internet host and port number of the requested resource, which is usually extracted from the HTTP URL, eg: we enter in the browser:http://www.guet.edu.cn/index.html In the request message sent by the browser, the host Request header field is included, as follows: Host:www.guet.edu.cn www.guet.edu.cn : Specify the port number user-agent when we go online to the forum, we often see some welcome information, which lists the name and version of your operating system, the name and version of the browser you are using, This often makes a lot of people feel amazing, in fact, the server application is from user-agent this request header domain to obtain this information. The User-agent request header domain allows the client to tell the server about its operating system, browser, and other properties. However, this header field is not required, and if we write a browser ourselves without using the User-agent request header domain, then the server side will not be able to know our information. Request Header Example: get/form.html http/1.1 (CRLF) Accept:image/gif,image/x-xbitmap,image/jpeg,application/x-shockwave-flash, application/vnd.ms-excel,application/vnd.ms-powerpoint,application/msword,*/* (CRLF) ACCEPT-LANGUAGE:ZH-CN (CRLF) Accept-encoding:gzip,deflate (CRLF) if-modified-since:wed,05 Jan 11:21:25 GMT (CRLF) if-none-match:w/" 80b1a4c018f3c41:8317 "(CRLF) user-agent:mozilla/4.0 (compatible; MSIE6.0; Windows NT 5.0) (CRLF) Host:www.guet.edu.cn (CRLF) connection:keep-alive (CRLF) (CRLF)

4, Entity header request and response messages can all be routed one entity. An entity consists of an Entity header field and an entity body, but it does not mean that the entity header fields and entity bodies are sent together, and only the entity header fields can be sent. The entity header defines the meta-information about the entity body (eg: there is no entity body) and the resource identified by the request. The common entity header content-encoding content-encoding entity header field is used as a modifier for the media type, and its value indicates the encoding of additional content that has been applied to the entity body, thus obtaining the media type referenced in the Content-type header field , the corresponding decoding mechanism must be adopted. Content-encoding This method of compressing the document, the Eg:content-encoding:gzip content-language content-language entity header field describes the natural language used by the resource. The domain is not set and the entity content is considered to be available to all language readers. The Eg:content-language:da content-length content-length entity header field is used to indicate the length of the entity body, expressed as a decimal number stored in bytes. The Content-type content-type Entity header Field term indicates the media type that is sent to the recipient's entity body. Eg:content-type:text/html;charset=iso-8859-1 content-type:text/html;charset=gb2312 Last-Modified The Last-modified Entity header field is used to indicate the last modification date and time of the resource. The Expires Expires entity header field gives the date and time when the response expires. In order for a proxy server or browser to update the cache after a period of time (once again accessing pages that have been visited, loading directly from the cache, shortening response times, and reducing server load), we can use the Expires entity header domain to specify when the page expires. eg:expires:thu,15 SEP 2006 16:23:12 GMT HTTP1.1 clients and caches must treat other illegal date formats (including 0) as expired. Eg: in order for the browser not to cache the page, we can also take advantage of the Expires entity header domain, set as 0,jsp in the program as follows: Response.setdateheader ("Expires", "0");

V. Using TELNET to observe the communication process of the HTTP protocol

The purpose and principle of the experiment: using MS's Telnet tool to make a request to the server by manually entering HTTP request information, the server receives, interprets, and accepts the request and returns a response that is displayed on the Telnet window. Thus the understanding of the communication process of the HTTP protocol is deepened from the perceptual.

Experimental steps:

1. Turn on Telnet 1.1 to open Telnet run-->cmd-->telnet

1.2 Turning on the Telnet echo function set Localecho

2. Connect the server and send the request 2.1 open www.guet.edu.cn 80//Note the port number cannot be omitted

head/index.asp http/1.0 Host:www.guet.edu.cn/* We can change the request method, request Guilin Electronic homepage content, enter the message as follows */Open www.guet.edu.cn get/index.asp http/1.0//content of request resources Host:www.guet.edu.cn

2.2 Open www.sina.com.cn 80//Enter Telnet www.sina.com.cn head/index.asp http/1 directly under the command prompt symbol . 0 Host:www.sina.com.cn

3 Experimental results:

3.1 Request Information 2.1 The resulting response is:

http/1.1 ok                                               //Request Success server:microsoft-iis/5.0                                     //web Server date:thu,08 Mar 200707:17:51 GMT connection:keep-alive                                    content-length:23330 content-type:text/html expries:thu,08 Mar 07:16:51 GMT set-cookie:a SPSESSIONIDQAQBQQQB=BEJCDGKADEDJKLKKAJEOIMMH; path=/cache-control:private

Omission of resource contents

3.2 Request Information 2.2 The resulting response is:

http/1.0 404 Not Found//request failed Date:thu, Mar 07:50:50 GMT server:apache/2.0.54 <Unix> last-modified: Thu, 2006 11:35:41 GMT ETag: "6277a-415-e7c76980" Accept-ranges:bytes X-powered-by:mod_xlayout_jh/0.0.1vhs.marki I.remix vary:accept-encoding content-type:text/html X-cache:miss from zjm152-78.sina.com.cn via:1.0 zjm152-78.sina.co M.cn:80<squid/2.6.stables-20061207> X-cache:miss from th-143.sina.com.cn connection:close

I lost the connection to the mainframe.

Press any key to continue ...

4. Note: 1, an input error occurred, the request will not succeed.           2, the header field is not case-sensitive.           3, a deeper understanding of the HTTP protocol, you can view RFC2616 , find the file on the HTTP://WWW.LETF.ORG/RFC. 4, the development background program must master the HTTP protocol

Six, HTTP Protocol Related Technical supplement

1, the Foundation: the High-level protocol is: File Transfer Protocol FTP, e-mail Transport protocol SMTP, Domain Name System service DNS, Network News Transfer Protocol NNTP and HTTP protocol, such as mediation by three kinds: proxy, gateways (gateway) and channel (tunnel), An agent accepts requests based on the absolute format of the URI, rewrites all or part of the message, and sends the formatted request to the server through the URI's identity. The gateway is a receiving agent that acts as the upper layer of some other servers and, if necessary, translates the request to the underlying server protocol. A channel acts as a relay point between two connections that do not change the message.      The channel is often used when the communication needs to pass through an intermediary (for example, a firewall, etc.) or if the content of the message is not recognized by the intermediary. Proxy: An intermediary program that can act as a server or as a client to establish requests for other clients. Requests are either internally or passed to other servers through possible translations. An agent must interpret and overwrite it if possible before sending the request information. Proxies are often used as portals through the firewall's client side, and proxies can be used as a help app to handle requests that are not completed by the user agent through the protocol. Gateway: A server that acts as an intermediary for other servers. Unlike the proxy, the gateway accepts the request as if it were the source server for the requested resource, and the requesting client is unaware that it is dealing with the gateway.     Gateways are often used as server-side portals through firewalls, and gateways can be used as a protocol translator to access resources stored in non-HTTP systems. Channel (tunnel): is a broker that is a relay of two connections. Once activated, the channel is considered not to be an HTTP communication, although the channel may be initialized by an HTTP request. The channel disappears when both ends of the relayed connection are closed. A channel is often used when a portal must exist or the intermediary (intermediary) cannot interpret the relay's traffic.
2, the advantage of the Protocol Analysis-http Analyzer detection network attack in a modular way to the high-level protocol analysis and processing, will be the direction of future intrusion detection. Common ports 80, 3128, and 8080 for HTTP and its proxies are specified in the network section with the port tag.
3, HTTP protocol content lenth limit vulnerability causes a denial of service attack using the Post method, you can set Contentlenth to define the length of the data that needs to be transferred, such as contentlenth:999999999, which is not released until the transfer is complete. An attacker could exploit this flaw to continuously send spam data to the Web server until the Web server ran out of memory. This method of attack does not leave a trace. Http://www.cnpaf.net/Class/HTTP/0532918532667330.html
4. Some ideas for denial-of-service attacks using the features of the HTTP protocol the server is busy processing an attacker's bogus TCP connection request to ignore the client's normal request (after all, the client's normal request rate is very small), from the normal customer's point of view, the server loses its response, This is what we call a Synflood attack on the server side (SYN flood attack). Smurf, teardrop and so on are using ICMP packets to flood and IP fragment attacks. This article uses a "normal connection" method to generate a denial of service attack. 19 ports in the early days already someone used to do chargen attacks, namely Chargen_denial_of_service, but! They use the method is to create a UDP connection between the two Chargen servers, so that the server processing too much information and down, then, to kill a Web server must have 2 conditions: 1. Chargen Service 2. HTTP Service Method: The attacker forged the source IP to send a connection request (connect) to n Chargen, and after receiving the connection, the Chargen will return a stream of 72 bytes per second (in fact, this speed is faster than the actual network) to the server.
5, HTTP fingerprint recognition technology the principle of HTTP fingerprint recognition is basically the same: record different servers to identify the minor differences in HTTP protocol execution. HTTP fingerprinting is much more complex than TCP/IP stack fingerprinting, because it is a custom HTTP server configuration file,       Adding plug-ins or components makes it easy to change the response of HTTP, which makes identification difficult, whereas customizing the TCP/IP stack requires modifying the core layer so it is easy to identify. To make the server return different banner information settings is very simple, such as Apache, open source HTTP server, the user can modify the banner information in the source code, and then restart the HTTP service to take effect For HTTP servers that do not have public source code, such as Microsoft IIS or Netscape, can be modified in the DLL file where the banner information is stored, the relevant articles are discussed, and are not described here. Of course, the effect of such a modification is good. Another way to blur banner information is to use a plugin. Common Test requests: 1:head/http/1.0 sends basic HTTP requests 2:delete/http/1.0 send those requests that are not allowed, such as DELETE requests 3:get/http/3.0 send an illegal version of the HTTP protocol request 4:get/ Junk/1.0 sends an incorrect specification of HTTP protocol request HTTP Fingerprint Identification Tool Httprint, it can effectively determine the type of HTTP server by using statistical principles and combining fuzzy logic techniques. It can be used to collect and analyze signatures generated by different HTTP servers.
6, Other: In order to improve the user's performance when using the browser, the modern browser also supports concurrent access, browse a Web page while establishing multiple connections, to quickly obtain a number of icons on a Web page, so that the entire page can be faster to complete the transmission. This continuous connection is provided in HTTP1.1, while the next-generation HTTP protocol: Http-ng adds support for more efficient connections, such as session control, rich content negotiation, and more.

HTTP protocol Detailed