A detailed description of the TCP send data process

Ext.: http://blog.csdn.net/yusiguyuan/article/details/17710141

b/S communication brief

The realization of the whole computer network embodies the realization of the protocol , the TCP/IP protocol is the core protocol of the Internet, and the HTTP protocol is a higher level application layer protocol than TCP.

HTTP (hypertext Transfer Protocol, Hypertext Transfer Protocol) is one of the most widely used network protocols on the Internet. All WWW documents must comply with this standard. HTTP was designed to provide a way to publish and receive HTML pages.

The browser (Web Browser) is responsible for establishing a connection with the server, downloading the Web page (including resource files and JS script files) to the local, and finally rendering the page. The JS script file runs on the client side, which is responsible for some behavioral response or preprocessing of the client, such as data validation before submitting the form, mouse event handling, and other interactions. Thus, the browser (Browser) on the one hand acts as the C /s communication architecture, and on the other hand it is the Html/javascript rendering engine (Analyze render engines).

In the browser address bar typed "http://www.baidu.com/", press ENTER, the browser appears in the homepage of Baidu. We're familiar with this kind of situation. However, this paper through Wireshark crawl this process of TCP/IP packets, combined with TCP protocol to analyze the basic flow of HTTP communication.

MTU and MSS

This article uses the Grasping Kit tool as Wireshark, its predecessor is the famous ethereal. The packet format for Wireshark Ethernet frames is:

Frame = Ethernet Header + IP header + TCP header + TCP Segment Data

(1) Ethernet header =14 Byte = Dst Physical Address (6 byte) + SRC Physical Address (6 byte) + Type (2 byte), Ethernet frame header The following is called a data frame.

(2) IP Header =20 Byte (without options field), data is called datagram in the IP layer, and shards are called fragment.

(3) TCP Header = Byte (without options field), where data is called stream in the TCP layer, and fragmentation is called segment (called Message in UDP).

(4) 54 bytes followed by the TCP Data Payload section (portion), which is the application-tier user data.

The maximum transmission unit for IP datagrams below the Ethernet header is MTU(Maximum transmission unit,effect of short board), and for most LANs using Ethernet, mtu= 1500.

TCP packets can transmit the maximum data segment for the MSS, in order to achieve the best transmission performance, in the establishment of the TCP connection when the two sides negotiate MSS value, the minimum value of the MSS value provided by both sides is the maximum MSS value of this connection. MSS are often calculated based on MTU, usually mss=mtu-sizeof (IP header)-sizeof (TCP header) =1500-20-20=1460.

This way, after the data is segmented by the local TCP layer, it is handed over to the local IP layer and no shards are required at the local IP layer. However, IP shards may occur on the next hop routing neighbor router. Because the MTU of the router's network card may be less than the size of the IP datagram that needs to be forwarded. At this point, there are two possible scenarios on the router:

(1). If the source sender set this IP packet to be fragmented (May fragment,df=0), the router forwards the IP datagram as fragmented.

(2). If the source send side has set this IP datagram not to be fragmented (Don ' t fragment,df=1), the router discards the IP datagram and sends an ICMP shard error message to the source send side.

For the MTU detection, refer to "Path MTU Discovery". We can use the ICMP protocol-based ping command to detect the MTU from the native to the target machine on the road, as described below.

TCP and UDP

In the application development based on the Transport Layer (TCP/UDP), for the final program optimization, the IP shards should be avoided on any one of the end-to-end nodes. The MSS negotiation mechanism of TCP plus the serial number confirmation mechanism can guarantee the reliable transmission of data basically.

Based on the IP protocol, the UDP protocol only increases the port of the Transport layer (Source port+destination port), the UDP data Kanenaga (Length = Header+data), and the inspection and (Checksum). Therefore, when developing an application based on UDP, the packet needs to be considered in conjunction with the IP sharding scenario. For Ethernet LAN, often take UDP packet length length<=mtu-sizeof (IP Header) = 1480, so the UDP data load is less than or equal to 1472 (LENGTH-UDP Header), for the public network, the IPv4 minimum MTU is 576, the UDP data payload is less than or equal to 536.

An "outward" NAT provides a mapping of an "asymmetric" bridge between the network and the public network. "Outward" NAT allows only the outward session to traverse NAT by default: Packets from the outside will be discarded unless the NAT device has previously defined these outgoing packets as part of the existing intranet session. For a party on a LAN, one side over the WAN UDP traffic, given that UDP traffic does not establish a virtual link beforehand, the LAN communication behind the NAT needs to send a message to the WAN Communication party to pierce the NAT before it can do two-way communication, which is often referred to as the "UDP punch hole (Hole Punching) "issue.

TCP connection Baidu Process analysis

The following full capture of Baidu is built on the basis of not using the cache . If the host has a Baidu site cookie and offline cache (Offline cache), then no longer request the Address bar icon Favicon.ico; Request/js/bdsug.js?v=1.0.3.0 may respond "http/1.1 304 not Modified ". Can be opened in the browser Baidu homepage, Ctrl+f5 forced to refresh, do not use the cache, but also refer to the browser to clear the cache method.

The following is a visit to Baidu process, Wireshark data capture packet. For machines that are connected directly via Ethernet, the Wireshark Capture filter is "host www.baidu.com" and for machines connected via PPP over Ethernet (PPPoE), Wireshark Capture filter is "pppoes and hostwww.baidu.com". The following sample grab packets are accessed directly through the Ethernet Internet access Baidu process. Click the image hyperlink to download the Pcap file and view it using the Wireshark software.

For convenience, the following will be referred to as the client (browser) as C, the server (Baidu) abbreviated as S.

1. TCP three-time handshake establish connection

"http://" Identifies the WWW Access Protocol as HTTP, which, according to the rules, can be connected to a higher-level protocol only after the underlying protocol establishes a connection. After entering an address in the browser's address bar and pressing ENTER, C establishes a TCP 80 connection with S (IP 220.181.6.175 that is resolved by the machine named Www.baidu.com,DNS) (HTTP uses TCP 80 port by default).

The following is a three-time handshake to establish a TCP-connected packet (PACKET1-PACKET3).

1 192.168.89.125:5672→220.181.6.175:80 TCP (Protocol) 62 (Ethernet frame length)

AMQP > http [SYN] seq=0 win=65535 len=0 mss=1460sack_perm=1

2 220.181.6.175:80→192.168.89.125:5672 TCP 62

HTTP > amqp [SYN, ACK] seq=0 ack=1 win=8192 len=0 mss=1460 sack_perm=1

3 192.168.89.125:5672→220.181.6.175:80 TCP 54

AMQP > http [ACK] seq=1 ack=1 win=65535 len=0

Three handshake the process for establishing a TCP connection is as follows:

C (Browser) S (www.baidu.com)

1. CLOSED LISTEN

2. syn-sent→<seq=0><ctl=syn>→syn-received

3. established←<seq=0><ack=1><ctl=syn,ack>←syn-received

4. established→<seq=1><ack=1><ctl=ack>→established

3-way Handshake for Connection synchronization

three-time handshake socket Layer Execution logic

s call the Listen function of the socket to enter the listening state; C calls the Connect function connection S:[syn],s Call the Accept function to accept C's connection and initiate a connection to the C direction: [Syn,ack]. C sends [ACK] to complete three handshake, the Connect function returns; s receives the [ACK] sent by C, and the Accept function returns.

about SEQ and ACK

Seq is the sequence number, which is the sending sequence numbers for the source end, and theAck is acknowledgment, which is the destination (destination) receive the confirmation serial number. In Wireshark Display filter, you can use TCP.SEQ or tcp.ack filtering.

In Packet1, c:5672 sends a SYN handshake packet to s:80, seq=0 (relative sequence number), Packet2 sends an ACK handshake response packet to s:80 c:5672 (ack=1 Sequence number), while sending the SYN handshake packet, seq=0 (relative sequence number), Packet3 sends an ACK handshake response packet c:5672 to s:80.

At this point, Seq=1 is C's initial Sequence number (ISN), a certain time later seq=isn+ cumulative send volume (cumulative sent); Ack=1 to C initial Acknowledge Number (IAN), ack=ian+ Cumulative received at a later point in time (cumulative received). For S, seq and ack are the same reason.

reference : "TCP Analyze Sequence Numbers", "Understanding TCP Sequence and Acknowledgement Numbers"

2. TCP get web site data flow

After the connection is established, the next step is to send ("get/http/1.1") the request HTML page, where "/" represents the default homepage of S, "GET" is the HTTP Request Method; "/" is Request-uri, here is the relative address http/1.1 indicates that the HTTP protocol version number used is 1.1.

The following is an HTTP GET Request packet (PACKET4).

4 192.168.89.125:5672→220.181.6.175:80 HTTP 417

get/http/1.1

The HTTP get message has a length of =417-54=363 bytes, where next sequence number:364 (relative sequence number) indicates that the message was sent successfully if it received a s response ack=364 within the specified time. The next message (seq=364) can be sent, otherwise retransmission (TCP retransmitssion). The serial number confirmation mechanism is the guarantee of TCP reliability transmission.

S (HTTP) receives an HTTP GET message (a total of 363 bytes) and sends a TCP acknowledgment message (PACKET5) to C (AMQP).

5 220.181.6.175:80→192.168.89.125:5672 TCP 60

HTTP > amqp [ACK] seq=1 ack=364 win=6432 len=0

Here Seq=1, for S's isn, meaning has been sent over Syn. Packet2, Ack=1 is Ian of S. The ack-ian=364-1=363 here indicates that s has received 363 bytes from C, an HTTP GET message. At the same time, ack=364 is also the next TCP message sequence number that s expects c to send (next sequence numbers analyzed above).

Next, S sends the HTTP Response to C, according to the HTTP protocol, first sends the response header (Response header), then sends the Baidu homepage HTML file.

The Http Response header message (PACKET6) is as follows.

6 220.181.6.175:80→192.168.89.125:5672 TCP 465

[TCP segment of a reassembled PDU]

Some of the contents are as follows:

======================================

http/1.1 OK

......

content-length:2139

content-type:text/html;charset=gb2312

Content-encoding:gzip

======================================

s in response to the "get/http/1.1" request of C, first sends a 411-byte HTTP Response Header (Packet 6) with the identity of [PSH].

The TCP header [PSH] identifies the bit and urges C to push the cached data to the application, which is to process the HTTP Response Header first, which is actually a "interception" notification. The corresponding C socket call to send when the IPPROTO_TCP option level is set Tcp_nodelay to True disables the Nagle algorithm to "preserve the send boundary" to prevent adhesion.

Although the MSS negotiated by the handshake is 1460, the server or proxy balancing server has a maximum of 1420 bytes of TCP data sent over each time. You can use the Ping-f-l size target_name command to send ICMP packets of the specified byte amount to the specified target target_name, where-l size specifies the size of the send buffer, and-F indicates that no Shard DF is set in the IP datagram (Don ' t Fragment) so that the MTU on the target path can be detected.

The result of executing "ping-f-l 1452www.baidu.com" is as follows:

Ping statistics for 220.181.6.18:

Packet: Sent = 4, received = 4, lost = 0 (0% missing)

The result of executing "ping-f-l 1453www.baidu.com" is as follows:

The packet needs to be split but DF is set.

Ping statistics for 220.181.6.18:

Packet: Sent = 4, received = 0, lost = 4 (100% missing)

From the above ping results can be seen, in the non-Shard, from the local to Baidu's route to the maximum amount of data to pass the 1452, which deduced the mtu{local,baidu}=sizeof (IP header) + sizeof (ICMP header) +sizeof ( ICMP Data portion) =20+8+1452=1480.

s calls the socket's send function to send 2,139 bytes of HTTP Response Content (Packet 7, Packet 9), and then sends out after the TCP layer is decomposed into two segments (segment).

7 220.181.6.175:80→192.168.89.125:5672 TCP 1474

[TCP segment of a reassembled PDU]

By "content-length:2139", the HTML file also has 2139-(1474-54) = 719 bytes. But at this point, C has sent a confirmation message (PACKET8).

8 192.168.89.125:5672→220.181.6.175:80 TCP 54

AMQP > http [ACK] seq=364 ack=1832 win=65535 len=0

seq-isn=364-1=363, which indicates that C has sent 363 bytes, has received an acknowledgment of s from the top. Ack-ian=1832-1= (465-54) + (1474-54) indicates that C has received 1831 bytes from S.

Next, C receives the remaining 719 bytes of the HTML file, and the message (PACKET9) is as follows.

9 220.181.6.175:80→192.168.89.125:5672 HTTP 773

http/1.1 OK

At this point, C received S sent over all the HTTP response message, that is, Baidu home HTML content (text/html).

The ACK for PACKET6, PACKET7, and Packet9 are all 364 because these three segment are TCP responses to PACKET4. S will Baidu homepage HTML file (a complete HTTP message) according to the MSS fragment submitted to the TCP layer. In Wireshark, you can see the following reassemble information in the PACKET9 message:

[Reassembled TCP segments (2555 bytes): #6 (411), #7 (1420), #9 (719)]

[Frame:6, payload:0-410 (411 bytes)]

[Frame:7, payload:411-1830 (1420 bytes)]

[Frame:9, payload:1831-2549 (719 bytes)]

C (AMQP) after receiving the HTML file on the homepage of Baidu, start parsing rendering. In the parsing process, found that the page contains Baidu's logo resources Baidu_logo.gif, and need to bdsug.js script .

{d.write (' <script src=http://www.baidu.com/js/bdsug.js?v=1.0.3.0><//script> ')}

So the above connection (c:5672) continues to request the Logo icon resource for S, the message (PACKET10) is as follows.

192.168.89.125:5672→220.181.6.175:80 HTTP 492

Get/img/baidu_logo.gif http/1.1

At the same time, C (JMS) creates a new connection (TCP 5673) to the S request JS script file. The message (PACKET11) is as follows.

192.168.89.125:5673→220.181.6.175:80 TCP 62

JMS > http [SYN] seq=0 win=65535 len=0 mss=1460 sack_perm=1

(Packet) Packet,Packet, Packetand Packet For TCP responses to Packet10 (their ack=802), they are logically a complete TCP message. Its HTTP Response content is a picture file baidu_logo.gif. We can see the following reassemble information in the Packet17 message in Wireshark:

[Reassembled TCP segments (1801 bytes): #13 (312), #14 (1420), #16 (, #17 (41)]

[Frame:13, payload:0-311 (312 bytes)]

[Frame:14, payload:312-1731 (1420 bytes)]

[Frame:16, payload:1732-1759 (bytes)]

[Frame:17, payload:1760-1800 (bytes)]

PACKET11-PACKET19-PACKET20 three handshake to complete the new connection. C (JMS) then sends a "get/js/bdsug.js?v=1.0.3.0 http/1.1" message (Packet21) to obtain the Bdsug.js script file.

192.168.89.125:5673→220.181.6.175:80 HTTP 465

get/js/bdsug.js?v=1.0.3.0 http/1.1

(Packet) Packet,Packet, Packetand Packet For TCP responses to Packet21 (their ack=412), they are logically a complete TCP message. Its HTTP Response content is the script file bdsug.js. We can see the following reassemble information in the PACKET27 message in Wireshark:

[Reassembled TCP segments (3897 bytes): #23 (310), #24 (1420), #26 (1420), #27 (747)]

[Frame:23, payload:0-309 (310 bytes)]

[Frame:24, payload:310-1729 (1420 bytes)]

[Frame:26, payload:1730-3149 (1420 bytes)]

[Frame:27, payload:3150-3896 (747 bytes)]

Usually, the browser will automatically search the root of the site, as long as it found favicon.ico this file, download it down as the website Address bar icon . Thus, C (AMQP) will also launch "Get/favicon.ico http/1.1" request site Address bar icon, the text Packet29.

3. TCP four times wave close connection

After Packet28 confirms receipt of the complete Japplication/javascript file, link 1 (Local port 5673) ends the mission, s closes the link and enters four times to wave off the bidirectional connection.

(Packet30) Packet31 and Packet32 are TCP responses to Packet29 (their ack=1201). After Packet33 confirms receipt of the complete Image/x-icon file, Link 2 (Local port 5672) ends the mission, s closes the link and enters four times to wave off the bidirectional connection.

Why the handshake is three times, and the wave is four times. This is because when a handshake occurs, the server often establishes a connection with the client when it agrees to establish a connection, known as a two-way connection . Therefore, in Packet2, the server sends ACK and SYN packages. Waving, that is, closing the connection, often just indicates that the wave is no longer sending data (no data can be sent), and the receiving channel is still valid (can still accept data). When the other side also waved, it indicates that the other side also no data can be sent, when the two-way connection is really closed.