Http:web Foundation

Overview http

The HTTP protocol is the multimedia messenger of the Internet. HTTP can move this information quickly, conveniently, and reliably from a Web server spread across the world to a Web browser on people's desktops.

The HTTP protocol is primarily divided between Web clients and servers. Where the Web server is the host of the Web resource. Web resources can contain any media type content, and the HTTP protocol, in order to identify various media types, will format the MIME-type data label for objects that are transmitted over the Web. (MIME science: Originally designed MIME ( Multipurpose Internet Mail Extension , Multipurpose Internet Mail Extension) is to solve the problem of moving messages between different e-mail systems. HTTP then uses it to describe and tag multimedia content. )

At the same time, each Web server resource has a name to identify, which is known as the Uniform Resource Identifier ( Uniform Resource Identifier ). There are two types of URIs, one is our common resource Locator URL, and the other is called a Uniform Resource name urn. The latter is still in the experimental stage and is not used in large scale.

A Web page can contain multiple objects, such as a page that includes many pictures, videos, audio, and so on. The client makes the transaction by sending a request command to the Web server. The server responds to the client request and transmits the corresponding data.

Both request and response messages have a fixed specification. The message consists of a single line of simple strings. HTTP messages are plain text, not binary code, so it's easy to read and write (but hard to parse). The message is divided into three parts
-Start line GET /index.html HTTP/1.0
-header field Each header field contains a name and a value, separated by a colon for ease of resolution. The header ends with a blank line.
-The body start and header are both textual and structured, and the body can contain arbitrary binary data, and of course it can contain text.

HTTP protocol messages are Transmission Control Protocol,TCP moved from one place to another via Transmission Control Protocol () connections.
TCP provides a
-Error-Free data transfer
-Sequential transmission (data will always arrive in the order sent)
-Non-segmented traffic (data can be sent at any time and at any size)

Before an HTTP client sends a message to the server, an Internet Protocol ( Internet Protocol,IP ) address and port number are required to establish a TCP/IP connection between the client and the server. First, you need to resolve the URL DNS to an IP address, and then use the IP address to connect to the Web server, the default port is 80.

In addition to the client and server, there are many more important web infrastructure components
-Proxy HTTP intermediate entity between client and server
-Caches HTTP repositories so that copies of common pages can be saved closer to the client
-A special Web server that gateways connect to other applications
-Special proxy for blind forwarding of HTTP communication messages by tunnels
-Semi-intelligent Web client that Agent agent initiates automatic HTTP requests

URLs and Resources

URLs provide a uniform way of naming resources, and most URLs have the same: "Scenario://server location/path" structure.

Syntax for URLs

<scheme>://<user>:<password>@
Almost no URL contains all of these components.
The 3 most important parts of a URL are the scenario ( scheme ), the host (), host and the Path ( path ).

URL encoding

The escape notation contains a percent percent, followed by two hexadecimal digits that represent the ASCII code of the character.
Example http://www.baidu.com/%7Ejoe~ 126(0x7E)

HTTP messages

HTTP messages are blocks of data that occur between HTTP applications. These blocks begin with meta-information () in textual form meta-information that describes the content and meaning of the message, followed by an optional data section. These messages flow between the client, the server, and the agent.

The syntax of the message

All messages can be divided into two categories: Request message ( request message ) and Response message ( response message ).

Request message

<method> <request-URL> <version><headers><entity-body>

Response message

<version><status><reason-phrase><headers><entity-body>

Start Line method

Method	Description
GET	Get a copy of a document from the server
HEAD	Get the document header only from the server
POST	Send data to the server that needs to be processed
PUT	Store the principal part of the request on the server
PUT	Tracking of messages that may go through a proxy server to the server
OPTIONS	Determine which methods can be executed on the server
DELETE	Delete a document from the server

Not all servers implement the 7 methods described above, and because the HTTP design is easy to scale, other servers may implement some of their own request methods.

Status code

Overall Range	defined range	category
100 ~ 199	100~101	Information Tips
200~299	200~206	Success
300~399	300~305	redirect
400~499	400~415	Client Error
500~599	500~505	Server error

The current HTTP version defines only a few codes for each type of state, and as the protocol progresses, more status codes are formally defined in the HTTP specification, and if you receive a status code that you do not know, it might be defined as an extension of the current protocol. It can be handled as a normal member of that category, depending on its scope.

First

The first part of the class:
-The generic header can appear in the request message and in the current response message
-Request header to provide more information about the request
-Response headers provide more information about the response
-The entity header describes the length and content of the subject, or the resource itself
-New header not defined in the extension header specification

Body part of an entity

The third part of the HTTP message is the optional entity body part. The body of the entity is the payload of the HTTP message, which is what HTTP is transmitting.

Connection Management

Almost all the HTTP traffic in the world is hosted by TCP/IP, and TCP/IP is a commonly used packet-switched network layered protocol set used by computers and network devices worldwide. A client application can open a TCP/IP connection to a server application that may run anywhere in the world.

The process by which a Web browser interacts with a Web server over a TCP connection

https://github.com:80/WilsonLiu95
1. The browser takes advantage of resolving the host name github.com
2. The browser queries the IP address 192.30.252.122 of this host name
3. The browser gets the port number 80
4. The browser initiates a 192.30.252.122 connection to port 80
5. The browser sends an HTTP GET message to the server
6. The browser reads HTTP response messages from the server
7. The browser closes the TCP connection

Latency of HTTP transactions

Transaction processing time can be very short compared to the time it takes to establish a TCP connection and transmit the request and response messages. HTTP latency is made up of TCP network latency unless the client or server is overloaded, or if complex dynamic resources are being processed.

There are several main reasons for the delay of HTTP transactions:
1. The client first needs to determine the IP address and port number of the Web server based on the URI. Where the IP address needs to be obtained by DNS resolution of the host name in the URL, which can take several 10 seconds.
2. The client sends a TCP connection request to the server, known as the "three-time handshake." This value usually has a maximum of two seconds, but if there are hundreds of HTTP transactions, the value will be stacked up quickly.
3. Internet transmission messages and server processing request messages take time.
4. It also takes time for the Web server to echo the HTTP response.
These TCP network latencies depend on hardware speed, network and server load, the size of request and response messages, and the distance between the client and the server. The technical complexity of the TCP protocol can also have a significant impact on latency.

Performance Focus Area

Some of the rest will have an effect on HTTP, the most common correlation delay
1. The TCP connection establishes the handshake
2. TCP Slow-start congestion control
3. Nagle algorithm for Data focus
4. TCP delay acknowledgement algorithm for piggyback acknowledgement
5. Time_wait Latency and port exhaustion

The TCP connection establishes the handshake

The TCP connection handshake takes a few steps
1. The cool Tiger sends a small TCP packet (usually 40-60 bytes) to the server. A special SYN tag is set in this group to indicate that this is a connection request.
2. If the server receives the connection, some connection parameters are computed and a TCP packet is sent back to the client, and the SYN and ACK tokens in this group are set, indicating that the connection request has been received.
3. Finally, the client sends back a confirmation message to the server informing it that the connection was successfully established. modern TCP stacks allow clients to send data in this acknowledgment packet.

If the connection is used only to transfer a small amount of data, these switching processes can severely degrade HTTP performance. Small HTTP transactions can take up to 50% or more time to build on TCP.

Delay Confirmation

Each TCP segment has a sequence number and data integrity checksum. When the recipient of each segment receives a good segment, a small acknowledgment packet is sent back to the sender. If the sender does not receive a confirmation message within the specified window time, the sender considers the data to be broken or damaged and re-sent.
Many TCP stacks implement a "delay acknowledgement" algorithm in order to increase the likelihood that the acknowledgement message can be grouped in the same transmit data. The delay-Acknowledgement algorithm stores the output acknowledgment in a buffer at a specific window time (typically 100~200 milliseconds), looking for a packet of output data that can be used to piggyback it. If there are no output data groupings within that time period, the acknowledgment information is sent in a separate grouping.
In general, the delay acknowledgement algorithm introduces a considerable delay, so you can adjust or disallow the delay acknowledgement algorithm.

TCP Slow start

The performance of TCP data transfer also depends on the lifetime of the TCP connection ( age ). The TCP connection is self-tuning over time, initially limiting the maximum speed of the connection and, if the data is successfully transmitted, increases the speed of the transfer over time. This tuning is known as TCP slow Start ( slow start ), which is used to prevent sudden overload and congestion of the Internet.

Nagle algorithm and Tcp_nodelay

The Nagle algorithm encourages the sending of full-size segments (the largest size grouping on the LAN is about 1500 bytes, which is hundreds of bytes on the Internet). Nagle only allows non-full-size groupings to be sent when all other groupings are confirmed, and if the other points are still in transit, the portion of the data is cached. The cached data is sent only if the pending packet is confirmed, or if the cache accumulates enough data to send a full-size grouping.
The Nagle algorithm raises several HTTP performance issues. The first small HTTP message does not fill a packet, and may be delayed by waiting for additional data that will never come. Second, there is a problem with the interaction between the Nagle algorithm and the delay acknowledgement--nagle prevents the data from being sent until a packet arrives confirming that the packet itself is delayed by 100-200 milliseconds by the delay-acknowledgement algorithm.
As a result, HTTP applications often set parameter Tcp_nodelay in their own stacks, disable the Nagle algorithm, and improve performance.

Time_wait Cumulative and port exhaustion

When a TCP endpoint shuts down a TCP connection, a small control block is maintained in memory to record the IP address and port number of the recently closed connection. This type of information is maintained for a short period of time to ensure that no new connections are created at the same address and port number during this time period.
Each time the client connects to the server, a new port number is obtained to achieve the uniqueness of the connection. However, due to the limited number of available source ports, there is a case of port exhaustion. You will not be able to establish a new connection.
Workaround: Increase the number of client load generation machines, or make sure that the client and server are looping through several virtual IP addresses to add more connection combinations.

Handling of HTTP connections

HTTP allows the existence of a string of HTTP intermediate entities (proxies, caches, etc.) between the client and the final source server. The HTTP messages can be forwarded to the source server (or reverse) by jumping through the intermediate devices, starting with the client.

Connection Header

In some cases, two neighboring HTTP applications apply a set of options for the connection they share. The Connection header field in HTTP has a comma-delimited list of connection labels that specify options that do not propagate to other connections for this connection.
Connection header can carry 3 different types of labels
-The HTTP header field name, which lists only the headers associated with this connection
-Any tag value that describes the non-standard options for this connection
-Value close, which indicates that the persistent connection needs to be closed after the operation is completed

Serial transaction processing time delay

If the detachment connection is managed simply, TCP performance delays may be superimposed. Another disadvantage of serial loading is that some browsers do not know the size of objects until they are loaded, and they may need dimensional information to decide where to place the object on the screen, so nothing can be displayed on the screen until enough objects are loaded.

Here are 4 techniques for improving HTTP connection performance.
-Parallel connections initiate concurrent HTTP requests over multiple TCP connections
-Persistent connections reuse TCP connections to eliminate connections and shutdown delays
-pipelined connections initiate concurrent HTTP requests over a shared TCP connection
-Multiplexing of connection alternating transfer request and response messages (experimental phase)

Parallel connections

HTTP allows clients to open multiple connections and execute multiple HTTP transactions in parallel.

Parallel connections can improve the speed at which a page is transferred, but there are some drawbacks to parallel connections:
1. Each transaction will open/close a new connection, which is time-consuming and bandwidth-intensive
2. The performance of each new connection is reduced due to the existence of the TCP slow start feature
3. The number of parallel connections that can be opened is actually limited

Persistent connections

Web clients often open connections to the same site. Therefore, an application that initializes an HTTP request to a server is likely to initiate more requests to that server in the near future. This property is called site locality.
Therefore, http/1.1 allows the HTTP device to keep the TCP connection open after the transaction has ended in order to reuse the existing connection for future HTTP requests.

A TCP connection that remains open after the transaction has ended is called a persistent connection. Non-persistent connections are closed after the end of each transaction. Persistent connections remain open between different transactions until the client or server decides to close them.
The slow connection establishment phase can be avoided by reusing idle persistent connections that have been opened to the target server. Also, a connection that has already been opened avoids the slow-start congestion adaptation phase, which allows for faster data transfer.

Persistent connections have some better places than parallel connections. Persistent connections reduce the overhead of delay and connection setup, keep the connection in a tuned state, and reduce the potential number of open connections. (There are two types of persistent connections: older http/1.0+ "keep-alive" connections, and modern http/1.1 "persistent" connections)

The use of parallel connections with persistent connections can be the most efficient way.

pipelined connections

Allows optional use of the request pipeline on a persistent connection, which is keep-alive another performance optimization relative to the connection.
You can put multiple requests into the queue before the response arrives. When the first request goes over the network to the server on the other side of the globe, the second and third requests can also start sending. This can reduce the loopback time of the network under Gao Shiyan network conditions.

Summarize

This article is the first part of the HTTP authoritative guide, which consists of the first to fourth chapter, and introduces the basic components of HTTP and the core technology of HTTP. I hope we can enjoy it.

Http:web Foundation