Learning to organize--Ethernet frame, IP frame, udp/tcp frame, HTTP message structure

From the simplest of an HTTP request development, according to the TCP/IP protocol, separate the data frame structure of each layer, and how they assume the network service.

Protocol stack

The Internet protocol stack consists of five layers:

1. The application layer is where the network application and its application layer protocols persist. The application layer of the Internet includes many protocols, common HTTP (which provides request and delivery for Web documents), SMTP (which provides the transmission of e-mail messages), and FTP (it provides file transfer between two-end systems).

2. The Transport layer , which is responsible for providing the data transfer service between the source and the host, including the different processes on the same terminal, primarily defines two transport protocols, TCP and User Datagram Protocol UDP.

3. The network layer , which is responsible for sending datagrams independently from the source to the host, mainly solves the problems of routing, congestion Control and network interconnection.

4. The link Layer is responsible for encapsulating the IP datagram into a frame format that is suitable for transmission on the physical network, or to unpack the frames received from the physical network and take out the IP datagram to the network layer.

5. The physical layer , which is responsible for transferring the bitstream between nodes, is responsible for the physical transmission. The protocol for this layer is related to both the link and the transmission medium.

Ethernet Frame

Since the physical layer is more related to physical media, the analysis begins directly from the link layer. In the link layer, the most used is Ethernet, and Ethernet frames for historical reasons there are multiple versions, where the IEEE802.3 Ethernet frame format is used.

Preamble: Preamble, 7 bytes, for synchronization of the rate of transmission and reception of both parties during data transfer

SFD: Frame start, 1 bytes, indicating that the next byte starts with the real data (destination MAC address)

DST mac: Destination mac address, 6 bytes, indicating the frame's recipient

src Mac: Source mac address, 6 bytes, indicating the sender of a frame

Length: 2 bytes, indicating the length of the frame data field, but does not mean that the data field length can reach (2^16) bytes

Type: 2 bytes, indicating the protocol type of the data in the frame, such as the common IPV4 IP protocol using 0x0800

Data and pad: 46~1500 bytes, which contain the data passed by the upper layer protocol, if the frame length is not enough 64 bytes after the data field is added, the data field will be "filled" to 64 bytes.

FCS: Frame check sequence, 4 bytes, provides a way for the receiving NIC (primarily the data and pad field to be detected) to determine if an error is found, discarding the frame. Currently the most popular algorithm for FCS is cyclic redundancy check (cyclic redundancy check–CRC)

IP Frames

The data fields mentioned above are data passed down from the upper layer protocol, and the most common upper layer protocol is the IP protocol. The format of the IP frame is as follows:

Version: Versions, 4-bit, used to indicate the version number of the IP protocol implementation, currently generally IPv4, that is, 0100,ipv6 0110. This field ensures compatibility between devices that may run different IP versions

IHL: Header length, 4 bits, in 32-bit words to define the length of the IP header, including optional. If the value of this field is 5, that is, 5*32=160 bit = 20 bytes. The maximum value for this field is 60 (15*32/8=60) bytes
TOS: Service type, 8-bit, for carrying a field that provides quality of service feature information, the Service Type field declares how the datagram can be handled by the network system when it is transmitted. The first 3 bits are priority sub-fields (precedence, which are now ignored and are not used by various terminals). The 8th bit remains unused. The 4th to 7th bits represent latency, throughput, reliability, and cost, respectively. When they are valued at 1 o'clock, they represent requirements for minimum delay, maximum throughput, maximum reliability, and minimum cost respectively. Only 1 bits of this 4-bit service type can be placed in 1. Can be all 0, if all 0 represents the General Service. In most cases, the TOS will be ignored

Total Length: 16 bits, indicating the length of the entire datagram, calculated in bytes. Maximum length is 65535 (2^16=65536) bytes

identification: Identity, 16-bit, used to uniquely identify each datagram sent by the host. The IP software maintains a counter in the memory, each generating a data segment, the counter adds 1, and assigns this value to the identity field. However, this "identity" is not an ordinal, because IP is a no-connect service, the datagram does not exist in order to receive problems. If the datagram must be fragmented because it exceeds the MTU of the network, the value of this identity field is copied to the identity field of all datagrams. The value of the same identity field causes the data slices to be correctly re-loaded into the original datagram after the Shard

flags: Flags, 3 bits, respectively (RF, DF, MF), currently only DF, MF valid. DF (don ' t fragment), which is set to 0, indicates that it can be segmented, 1 is not segmented, and MF (more fragment), when set to 0, indicates that the data field is the last segment of data, and when set to 1, it is followed by a segmented segment.

Fragment offset: Segment offset, 13-bit, indicates the relative position of a segment in the original grouping after the segment is divided. That is, relative to the starting point of the user field, where the slice starts. The segment offset is offset by 8 bytes (with 3 bits occupied by flags). That is, the length of each shard must be an integer multiple of 8 bytes (64 bits)

Time to Live: Lifetime (TTL), 8-bit, used to set the maximum number of routers a datagram can pass through. is set by the source host that sends the data, typically 32, 64, 128, and so on. Each router passes a value minus 1 until 0 o'clock the datagram is discarded

protcol: protocol, 8-bit, indicates the data in the IP data field is encapsulated by what protocol is upper layer. Common ICMP (1), IGMP (2), TCP (6), UDP (17)

header Checksum: Head checksum, 16 bits, fills the checksum code computed from the IP header. The method is calculated by summing the binary code for each 16 bits in the head, but not the data field after the head is involved

Source Address: Origin IP address, 32 bits, such as (192.168.1.2)

DestinationAddress: Destination IP addresses, 32 bits, such as (192.168.1.3)

option: Options, n*32 bit. Used to define options such as record path, timestamp, and so on. However, these options are rarely used, and not all hosts and routers support these options. The length of the optional field must be an integer multiple of 32 bits, and if insufficient, 0 must be populated to meet this length requirement. The length of option can be obtained according to IHL

Data: The variable length, but is limited by the maximum length of the datagram (65535). This is the data to be transferred in the datagram. It is a complete higher beginning text or a shard of the message

Udp/tcp Frame

The full name of the UDP protocol is the Subscriber Datagram Protocol (user Datagram Protocol), which is a non-connected protocol for processing packets as the TCP protocol in the network. In the OSI model, the fourth layer, the transport layer, is in the upper layer of the IP protocol. Because UDP transmission is not a reliability service, the frame structure is simpler, and processing, the transmission rate is high, so it is often used as a transport protocol for audio, video and general data, because they even occasionally lose one or two packets, it will not have a large impact on the reception results.

srcport: Source, 2 bytes, is a 16-bit number greater than 1023 and is randomly selected by the user process based on the UDP application.

DSTPort: Destination ports, 2 bytes, indicating the port number used by the receiver, typically specified by the application

Length: Data lengths, 2 bytes, indicating the length of the UDP segment including header

Checksum: Test and, 2 bytes, refers to the checksum of the data (including the pseudo-headers) of the entire UDP packet header and UDP. The pseudo-header is not included in the true UDP header, but it ensures that the UDP data is received by the correct host

data: field, variable length, encapsulated data for upper layer protocol

TCP (transmission Control Protocol Transmission Protocol) is a connection-oriented, reliable, byte-stream-based Transport layer communication protocol. When the application layer sends a 8-byte data stream for inter-network transmission to the TCP layer, TCP divides the data stream into segments of appropriate length, and the maximum transmission segment size (MSS) is usually limited by the maximum Transmission Unit (MTU) of the data link layer of the network to which the computer is connected. TCP then passes the packet to the IP layer, which transmits the packet over the network to the TCP layer of the receiving side entity.

srcPort: Source ports, 2 bytes, is a 16-bit number greater than 1023, randomly selected by a TCP application-based user process

DSTPort: Destination ports, 2 bytes, indicating the port number used by the receiver, typically specified by the application

SequenceNumber: 4 bytes, which identifies the byte stream of data that is sent from the TCP source to the TCP destination, which represents the sequence number of the first data bytes in the message segment. If you consider a stream of bytes as a one-way flow between two applications, TCP counts each byte with a sequential number. The sequence number is an unsigned 32bit, and the sequence number arrives (2^32)-1 and starts at 0. when a new connection is established, the SYN flag becomes 1, and the Sequence number field contains the initial order number of the connection selected by this host (Initial Sequence numbers)

AcknowledgementNumber: Confirm, 4 bytes, containing the next sequence number expected to be received at one end of the sending acknowledgment. Therefore, the confirmation sequence number should be the last time the data byte order was successfully received plus 1. Confirmation ordinal field only valid if ACK flag is 1 o'clock

Offset: Header length, 4 bits, gives the number of 32bit words in the header. This value is required because the length of the optional field is variable. This field accounts for 4bit, which is the header of up to 60 (15*4) bytes of TCP

RESRVD: Reserved area, 6 bits, reserved for future use, currently must be set to 0

The control Flags(6-bit) controls bits include

URG: 1 indicates that the emergency pointer is valid and 0 ignores the emergency pointer value

ACK: 1 indicates that the confirmation number is valid, 0 indicates that the message does not contain confirmation, and the confirmation number field is ignored

PSH: 1 indicates that the data with the PUSH flag indicates that the receiver should hand over the segment to the application layer without waiting for the buffer to fill

RST: Used to reset a connection that was faulty due to a host crash or other cause. It can also be used to reject illegal message segments and deny connection requests. In general, if you receive a message with a RST of 1, some problems must have occurred

SYN: Synchronous ordinal, 1 for connection request, for establishing connection and synchronizing sequential number (synchronize)

FIN: Used to release the connection, 1 means the sender has no data sent, that is, to close the local traffic

window Size: 2 bytes, which indicates the number of bytes that the source party can receive from the confirmation number , which is the source party receive window size. Window size is a 16bit field, so the maximum window size is 65535 (2^16-1)

Checksum: Checksum, 2 bytes, for the entire TCP segment (including TCP headers and TCP data), calculated in 16-bit words. This is a mandatory field that is required to be computed and stored by the sender and validated by the receiving end

Urgent Pointer: emergency pointer, 2 bytes, is a positive offset, and the value in the Sequence Number field is added to denote the ordinal of the last byte of the emergency data. The emergency mode of TCP is a way of sending an emergency data to the other end. Only if the URG flag is placed 1 o'clock the emergency hands are valid.

option and pad: Options and Padding, n*4 bytes, common optional fields are the longest message size MSS (Maximum Segment size). Each connector typically indicates this option in the first message segment of the communication (the segment that sets the SYN flag for establishing the connection), which indicates the maximum length of the message segment that can be received at the local end. The option length is not necessarily an integer multiple of 32-bit characters, so fill the bit so that the header length becomes the whole word count

Data: date, indefinite length, data encapsulated for upper layer protocol

Three handshake (connection established), four waves (release connection)

Shake hands:

1. The client sends a SYN segment (SYN 1) indicating the port of the server to which the client intends to connect, and the initial order number (ISN) (client----service side)
2. The server sends back a SYN segment (SYN 1) that contains the server's initial order number (ISN) as the answer. Also, set the confirmation number to customer's ISN plus 1 to confirm the customer's SYN message segment (ACK is also 1) (Server----client)
3. The client must set the confirmation number to the server's ISN plus 1 to confirm the SYN message segment of the server (ACK 1), which notifies the destination host that both parties have completed the connection (client----service side)
Waved:

1. The client sends a FIN (fin 1) to shut down the client-to-server data transfer (client----"Service side")
2. The server receives this fin, it sends back an ACK (ack of 1), confirms the serial number to receive the serial number plus 1 (server----client)
3. The server shuts down the connection with the client, sends a fin to the client (service side----"client)
4. The client sends back ACK message acknowledgement, and sets the confirmation sequence number to receive the serial number plus 1 (client----"service side)

Four times the purpose of the wave is to send fin if the client wants to close because the data is sent. After the server receives fin, but there may still be data to be sent to the client, the server first sends an ACK as the answer. Once the server has been sent, send back fin as a close connection. After the client receives the FIN, it sends an ACK and then closes the connection.

HTTP messages

HTTP belongs to the Application layer protocol, the degree of freedom is higher, is text-oriented, each field in the message is some ASCII string, the length of each field is indeterminate. The type of message can be divided into two types: Request message and Response message

HTTP Request message:

The words written in paragraph are:

<request-line>

<blank-line>

<request-body>

1. Request Line

The request line includes the request method, URL, protocol version, they are separated by a space, and are variable length, so we enter the URL is not allowed with spaces.

Request method: Commonly used post(the form data into the request body, most for uploading data), GET(the request parameters are placed in the url+, the parameters with & connection, used to obtain data), HEAD (The service side only returns the response header, so the processing response is fast and is used to detect whether the request is available).

2. Request Header

Each line is written as a key/value, between the key values: Split, multiple values separated by; split, each line ends with a carriage return newline character. Common keys are:user-agent: The type of browser that generated the request;Accept: A list of content types thatthe client can identify; host: the hostname of the request, allowing multiple domain names to be located in the same IP address; Range: Specifies one or more sub-ranges of the requested entity, which can be set when using multi-threaded downloads. Note A blank line between the request header and the request body, which indicates that there are no more request headers on the notification server, and the following are the request bodies.

3. The request body

The request data is not used in the Get method, but is used in the Post method. The Post method is useful for situations where a customer needs to fill out a form.

HTTP Response message:

Written in paragraph type:

<status-line>

<blank-line>

<request-body>

1. Status line

The status includes the protocol version, status code, status code description, a common example of a status line is (http/1.1), 200 indicates that the request is normal, more status code can check the relevant information.

2. Response header

Similar to the request header, the key message is passed to the client via key/value:content-type: The type of media used to indicate the entity to the receiver;content-range: Used to indicate to the receiver the delivery range of the entity; content-length: Used to indicate to the receiver the transmission length ofthe entity; Date: The day and time of the system when it was delivered.

3. Response body

The text data that the server actually returns, and if the client requests a Web page, the data segment populates the requested HTML text. Give an example of a response header:

http/1.1 Okdate:thu, APR 23:59:59 gmtcontent-type:text/html;charset=iso-8859-1content-length: 122    Test    <p>hello</p>  

Summarize

If an HTTP request is parsed, the resulting data flow will be as follows on the Internet Five layer protocol:

In fact, each layer, the inside of the key technology are very profound, have time to come back to learn.

Learning to organize--Ethernet frame, IP frame, udp/tcp frame, HTTP message structure