LWIP-an implementation of TCP/IP protocol stack

,

1Introduction

In recent years, the interest in computer interconnection and equipment with wireless network devices has become increasingly popular. Computers and everyday devices are becoming more and more seamlessly integrated and prices are falling. Wireless network technologies, such as Bluetooth and IEEE 802. 11B Wireless LAN also appeared one after another. This will lead to more exciting prospects in many areas such as health care, security, insurance, transportation and processing. Small devices, such as sensors, can access existing network infrastructure such as the global Internet, and can be monitored anywhere.

Internet technology has been proven flexible enough to adapt to the ever-changing network environment over the past few decades. Internet technology was initially developed on a slow network such as ARPAnet, but it is now running on a large number of connection technologies and bandwidth and bit error rates with many different features. Because a large number of internet technology applications have been widely used, it is of great benefit to use existing Internet technology in wireless networks. At the same time, the huge interconnectivity of the global Internet is a great incentive.

Because small devices such as sensors are often required to be small and cheap, the implementation of network protocols must handle limited computing resources and memory. This article describes the design and implementation of a small TCP/IP protocol stack that becomes LWIP. It is small enough to be used in a minimum system.

The arrangement of this article is as follows: Part 2, 3, and 4 provide an overview of the LWIP protocol stack, Part 2 describes the Operating System Simulation layer, and part 3 describes memory and buffer management, part 1 introduces the abstract interfaces at the LWIP network layer. Sections 8 and 10 describe the implementation of IP, UDP, and TCP Protocols. Section 11 and 12 describes how to use LWIP and APIs, the implementation is analyzed in part 13,14. Finally, Part 15th provides the lwip api user manual, and section 17,18 provides several code instances.

2Protocol layering

The TCP/IP protocol family is designed with the component layer mode. Each protocol layer solves different communication problems. The layered approach leads the design of protocol implementation, and each Protocol can be implemented separately. Implementing the Protocol in a strictly hierarchical manner leads to the situation that communication between protocol layers reduces the overall performance. To overcome this problem, some aspects of an agreement should be understood by other protocols. Be careful that only important information is shared between protocols.

Many TCP/IP implementations strictly separate the application layer and its underlying protocol layers, regardless of whether these layers can be more or less cross-implemented. In many operating systems, these underlying protocols are implemented as part of the operating system kernel and only provide some communication calls to the application layer process. The application only knows an abstract method implemented by TCP/IP. There are few differences between network communication and inter-process communication or file I/O. The reason for this is that the application will not realize that the buffer mechanism is used at the underlying layer, and it cannot use this information to reject the buffer that frequently uses data. At the same time, when an application sends data, the data must be copied from the application process memory space to the internal buffer before being executed by the network code.

The operating system on a minimum system usually does not maintain strict boundary protection between the kernel and application processes. This allows a more flexible communication mechanism between the application layer and the underlying protocol through the sharing. In particular, the application layer is aware of the underlying buffer mechanism. Therefore, applications can reuse these buffers more efficiently. At the same time, because the application process can use the same memory as the network layer code, the application process can directly read and write those internal buffers, thus saving the copy cost.

3Summary

Like many other TCP/IP implementations, the existing hierarchical protocol design provides a wizard for the design and implementation of LWIP. Each protocol implements its own module and uses some functions as the entry to this Protocol. Although most protocols are implemented separately, some layers violate this idea to improve the processing speed and memory usage. For example, when you want to verify the TCP packet segment to be reached, or when you want to group and forward (demultiplexing) a packet segment, the TCP module must know the source IP address and destination IP address of the packet segment. The TCP module is clear about the IP header structure, so it can analyze the required information by itself, instead of passing these addresses through function calls.

LWIP contains several modules. In addition to the modules that implement TCP/IP protocol families (IP, ICMP, UDP, and TCP), LWIP also implements other supported modules. These supported modules include the Operating System Simulation layer (described in part 2), the buffer and Memory Management Subsystem (described in part 2), and network interface functions (described in Part 2 ), and the functions used for the calculation test. LWIP also describes an abstract API in section 12th.

4Process Model

The process model implemented by a protocol is described as follows: The system is separated into different processes. A process model that has been used to implement communication protocols is to run each process as an independent process. In this model, protocol layering is strict, and communication interfaces between protocols are also strictly defined. Although this method has its advantages, for example, the Protocol can be added during running, the code is clear and easy to debug, and it also has some disadvantages. As mentioned above, strict layering is not the best way to implement the Protocol. At the same time, more importantly, layer-to-layer interaction may cause context switching. For a TCP packet segment that arrives, it means three context switches, from the device driver at the network interface layer to the IP process, to the TCP process, and finally to the application process. Context switching is expensive for most operating systems.

Another common method is to allow the communication protocol to reside in the kernel of the operating system. In the case of kernel-level communication protocols, application processes communicate with these protocols through system calls. These communication protocols do not strictly differentiate each other. Instead, some technologies are used to cross the protocol layer.

LWIP uses the process model. In this model, all protocols reside in a single process, obviously separate from the operating system kernel. The application process can reside in the LWIP process or in different separate processes. There are two types of communication between the application and the TCP/IP protocol stack: one is that when the application and LWIP share a process, the function can be called; the other is to use a more abstract API.

The advantages and disadvantages of implementing LWIP as a user space process rather than a resident operating system kernel. The main advantage is that this implementation can be transplanted between different operating systems. Because LWIP is designed to run on small operating systems, these operating systems do not support exchange of processes and virtual memory, therefore, latency caused by having to wait for disk operations (if some LWIP processes are switched out of the disk) is no longer a problem. It is still a problem to wait for a certain amount of scheduling before a request can be served. However, in the LWIP design, there is nothing to exclude that it will be implemented in the kernel of the operating system in the future.

5Operating System Simulation Layer

To enable LWIP portability, the specific function calls and data structures of the operating system are not directly used in the code. When such a function is required, the operating system simulation layer is used. The operating system simulation layer provides unified interfaces for operating system services, such as timers, process synchronization, and message transmission mechanisms. In principle, to port LWIP to other operating systems, you only need to implement the simulation layer of the specific operating system.

The operating system simulation layer provides a functional timer for TCP use. This timer is a one-time timer with a minimum interval of MS. It will call a registered function at the end of the timer.

Semaphores are the only implementation of the process synchronization mechanism. Even if the underlying operating system does not provide semaphore implementation, it can also be implemented by other synchronization primitives such as critical variables or lock mechanisms.

The implementation of message transmission is a simple mechanism abstracted as "Mailbox. One mailbox has two types of operations: Mail and receipt. The mail operation will not block this process, and the information sent to a mailbox will be included in the queue at the simulation layer of the operating system until another process receives this information. Even if the lower operating system does not support the mailbox mechanism, it can be easily implemented by semaphores.

6 Buffer and Memory ManagementIn a communication system, the buffer and memory management system must be able to allocate buffers of different sizes, from the full-length TCP packet segment containing several hundred bytes of useful data, to an ICMP response that contains only a few bytes. At the same time, in order to avoid copying, we should try our best to make the Data Type Buffer reside in the memory, instead of managing it by the network subsystem, such as the application memory or ROM. 6 . 1 Package Buffer-pbufPbuf is an internal representation of the LWIP package and is designed to minimize the special requirements of the stack. Pbufs is similar to mbufs in BSD implementation. The pbuf structure allows you to dynamically allocate memory for the package content and store the package data in static memory. Pbufs can be linked to a linked list by a chain called pbuf, so that a package can span multiple pbufs. Pbufs has three types: pbuf_ram, pbuf_rom, and pbuf_pool. Figure 1 indicates the pbuf_ram type, which contains package data that is managed by the pbuf sub-system in memory. Figure 2 shows a pbuf linked list, where 1st are of the pbuf_ram type and 2nd are of the pbuf_rom type, meaning that it contains memory data not managed by the pubf subsystem. Figure 3 describes the pbuf_pool, which contains the pbuf allocated from the fixed-size pbuf pool. A pbuf chain can contain multiple different types of pbuf. These three types have different uses. The pbuf_pool type is mainly used by the network device driver, because the allocation of a single pbuf is fast and suitable for the use of Interrupt handles. The pbuf_rom type is used when the application sends data in the application's memory space. The data is not modified after pbuf is submitted to the TCP/IP stack. Therefore, this type is mainly used when the data is in the Rom. The header pointing to data in pbuf_rom is included in the previous pbuf of the pubf_ram type in the linked list, as shown in 2.

The pbuf_ram type is also used for applications to send dynamically generated data. In this case, the pbuf system not only allocates memory for application data, but also allocates memory for the header pointing to (prepend) data. 1. The pbuf system cannot predict which header will point to the data (prepend), just for the worst case. The size of the header is determined during compilation. Essentially, the incoming pbuf is of the pbuf_pool type, while the outgoing pbuf is of the pbuf_rom or pbuf_ram type. Figure 1 and figure 2 show the internal structure of pbuf. The pbuf structure contains two pointers, two length fields, one flag field, and one reference count. The next field points to the next pbuf in the unified linked list. The payload Pointer Points to the starting point of the data in the pbuf. The LEN field contains the same length in the pbuf data. The tot_len field is the sum of the LEN field values in the current pbuf and all linked lists. To put it simply, the tot_len field is the sum of the values of the LEN field and the tot_len field in the next pbuf. The flags field indicates the pbuf type, and the ref field contains a reference count. The next and payload fields are local pointers, and their size is determined by the processor architecture. The two length fields are 16-bit unsigned integers, while the flags and ref fields are all 4-bit sizes. The total size of pbuf depends on the processor architecture used. In the 32-bit pointer and 4-byte correction architecture, the total size is 16 bytes, while in the 16-bit pointer and 1-bit self-corrected architecture, the total size is 9 bytes. The pbuf module provides functions for pbuf operations. Pbuf_alloc () can be allocated to the three types of pbuf mentioned above. Pbuf_ref () increases the reference count. pbuf_free () releases the allocated space. It first reduces the reference count and releases pbuf when the reference count is 0. Pbuf_realloc () compresses the space so that pbuf only occupies the right space to save data. Pbuf_header () adjusts the payload pointer and length field to point a header to the data in pbuf. Pbuf_chain () and pbuf_dechain () are used to chain the pbuf. 6 . 2 Memory ManagementMemory Management simply supports the pbuf mechanism. It processes the allocation and release of continuous memory and can contract previously allocated memory blocks. Memory Management uses a small portion of the total system memory, which ensures that the network system does not use all available memory, and other program operations are not interrupted because the network system has used all the memory. Internally, memory management records a small structure in the header of each allocated memory block to track all allocated memory. As shown in figure 4, The two pointers in the structure are respectively the memory block allocated to the forward and the last one. It also has a usage flag indicating whether the memory block has been allocated. Memory Allocation is performed by searching for an unused memory block of sufficient size to allocate the requested space. The first adaptation method allocates the first sufficient memory for the search. When a memory block is released, the used indicator is set to 0. To prevent fragments, you can combine unused blocks to form larger unused blocks by checking the used signs of the previous and last memory blocks.

7 Network layer interfaceThe device driver of the physical network hardware in LWIP is represented as a network interface structure as in BSD. Figure 5 shows the network interface structure. The network interface is saved in a global linked list and connected by the next pointer in the structure. Each network interface has a name that exists in the Name field. The two-character name indicates the device driver type used by the network interface. The interface configuration is completed only manually at run time. The name field is set by the device driver and reflects the hardware type represented by the network interface. For example, the name of the Bluetooth-driven network interface should be BT, and the name of the IEEE 802.11b WLAN should be WL. Because the name value does not need to be unique, the num field is used to differentiate different network interfaces of the unified type driver.

The three IP addresses, ip_addr, netmask, and GW, are used by the IP layer to send and receive packets. They are described below. You cannot use more than one IP address to configure a network interface. A network interface can only correspond to one IP address. The input Pointer Points to a function, which is called by the device driver when a package is received. A network interface points to a device driver through the output pointer. This pointer points to a function in the device driver used to transmit a packet on the physical network. This function is called when the IP layer wants to send packets. This field is set by the device driver initialization function. The third parameter of the output function, ipaddr, is the Host IP address that will receive the actual link layer frame. It does not need to be the same as the destination IP address of the IP package. In particular, when an IP address is sent to a host not on the local network, the link layer frame is sent to a vro in the network. In this case, the IP address passed to the output function will be the IP address of the router. Finally, the state Pointer Points to the device driver to indicate the specific status of the network interface, which is set by the device driver.

8 IP ProcessingLWIP only implements the most basic IP function. It can send, receive, and forward network packets, but cannot receive or send segment IP packets, or process network packets with IP options. Most applications do not have any problems. 8 . 1 receive network packetsFor the incoming IP package, processing starts when the network device driver calls the ip_input () function. Here, the preliminary comprehensive (initial sanity) IP version domain and header length check, plus the calculation and check header check. It assumes that the protocol stack will not receive any IP parts, because it assumes that the proxy will collect hot parts, so all IP parts will be lost. Packets containing IP options are also discarded by the proxy. Next, the function uses the IP address of the network interface to check whether the package is sent to the host. Network interfaces are sequentially linked to a linked list and searched linearly. The number of network interfaces is expected to be small, so there is no better search strategy than linear search. If the incoming package is determined to be sent to the host, the Protocol Domain value is used to determine which high-level protocol the package is passed. 8 . 2 send network packetsThe ip_output () function processes a sent packet. This function uses ip_route () to find an appropriate network interface to transmit the packet. After determining the network interface for sending the package, the package is passed to the ip_output_if () function, and the network interface for sending the package is also a parameter of the function. In this case, all IP header fields are filled and the IP header checksum is calculated. Both the Source and Destination addresses of the IP package are passed to the ip_output_if () function as a parameter. In addition, you can omit the source address of the packet, but in this case, the IP address of the network interface that sends the packet will be used as the source IP address of the packet. The ip_route () function searches the list of network interfaces linearly to find the appropriate network interface. During the search, the destination IP address of the IP package is masked by the subnet mask of the network interface (masked ). If the destination address to be masked is equal to the IP address after a network interface is masked, the network interface is selected. Otherwise, a default network interface is used. This default network interface can be set during system boot or manual configuration during running. If the network address and destination IP address of the default network interface are not matched, the GW field value in the network interface structure (figure 5) will be used as the destination IP address of the link frame. (Note: In this case, the destination address of the IP package and the destination address of the link frame will be different ). In this original form of routing, the fact is that a network may have multiple routers connected. For most basic cases, such as a local network with only one vro, this works well. Because the UDP and TCP Protocols at the transport layer require the destination IP address to calculate the transport layer checksum, in some cases, determine the network interface for sending the packet before the packet is transmitted to the IP layer. The solution is to allow the transport layer function to directly call ip_route (). Because the network interface for sending the package is known when the package arrives at the IP layer, you do not need to search for the network interface list. Instead, these protocols call ip_output_if () directly (). Because this function does not take this network interface as a parameter, it avoids searching for this network interface. 8 . 3 forward network packetsIf no network interface IP address matches the destination address of an incoming package, the package should be forwarded. This is handled by the ip_forward () function. In this case, the value of the TTL field is reduced. If the value is 0, an ICMP error message will be sent to the original sender of the packet and the packet will be discarded. Because the IP header has changed, the IP header checksum must be corrected again. Because [mk90, rij94] simple algorithms can be used to correct the original IP address checksum, there is no need to re-calculate the entire checksum. Finally, the packet is forwarded to a suitable network interface. The algorithm used to find a suitable network interface is the same as sending an IP package. 8 . 4 ICMP ProcessingThe ICMP processing process is quite simple. an ICMP packet received by ip_input () is forwarded to the icmp_input () function, which decodes the ICMP header and responds accordingly. some ICMP messages are transmitted to the high-level protocols, and some special transport layer protocols may concern these messages. messages that are inaccessible to ICMP can be sent by the transport layer protocol, especially UDP. icmp_dest_unreach () is used to handle this situation. ICMP echo messages are widely used in networks to detect networks. Therefore, ICMP echo performance should be optimized as much as possible. the actual processing occurs in icmp_input (). The processing includes: exchanging the source and destination IP addresses of the incoming packet, changing the ICMP Type of the echo reply, and correcting the ICMP checksum. at this time, the package is passed back to the IP layer for further sending. 9 . UDP ProcessingUDP is a simple protocol used to demultiplexing packets between different processes. the status of each UDP session is saved in a PCB structure (7 ). UDP control blocks are stored in a linked list. when a UDP data is reported, the linked list will be searched to find a matched control block (PCB ). in the global UDP Control Block linked list, each UDP control block contains a pointer to the next control block. a udp session is determined by the IP address and port number of the two terminals. The information is stored in the local_ip, dest_ip, local_port, and dest_port fields respectively. the flags field indicates the UDP checksum applied to the current session. this allows you to disable the UDP check or use UDP Lite to calculate the checksum of some data packets. when UDP Lite is used, the chesum_len field specifies the number of data packets to be calculated. the last two parameters Recv and recv_arg will be used when a data is reported. due to the simplicity of UDP, the input and output processes are also very simple, and are processed in a straight line (8 ). to send data, the application calls the udp_send () function, which then calls the udp_output () function. the necessary checksum and the value of the UDP header fields are calculated. because the source address of the IP package is verified, ip_route () is called to find the network interface for sending the IP package. the IP address of the network interface is used as the source IP address of the package. finally, the packet is transferred to ip_output_if () for transmission. when UDP data is reported, the IP layer calls the udp_input () function. in this case, if the session should use the check, check the UDP checksum of the datagram, and then forward the packet multiple times. after the corresponding UDP control is found, the Recv function is called.   10. TCP Processing TCP is a transport layer protocol that provides reliable byte stream services for the application layer. TCP must be complex for any other protocols described here, and its implementation code accounts for about 50% of the total amount of code. 10.1 Summary Basic TCP processing (9) is divided into six functions. the tcp_input (), tcp_process (), and tcp_receive () functions are related to TCP receiving processing, while tcp_write (), tcp_enqueue (), and tcp_output () the three functions are related to TCP sending. the application calls tcp_write () to send TCP data. the tcp_write () function is passed to the tcp_enqueue () function. if necessary, tcp_enqueue () Will segment the data into appropriate TCP packet segments, and finally put the TCP packet segments in the connection transmission queue. the tcp_output () function checks whether the data can be sent. For example, whether the receiving window has enough space and whether the congestion window is large enough. If yes, the function calls ip_route () and ip_output_if () to send the data. when ip_input () confirms the IP header and transfers the TCP packet segment to the tcp_input () function, the receiving process starts. the tcp_input () function performs a complete preliminary check (such as checksum and TCP option parsing), and confirms the TCP connection of the message segment. the datagram is then transferred to tcp_process (), which implements the TCP state machine and any necessary state conversion. if the connection is in the status of receiving data from the network, the tcp_receive () function is called. in this way, tcp_reveive () will pass the packet to the upper-layer application. if the message contains an ACK response for unconfirmed data (of course, the currently buffered data), the data will be removed from the buffer and the occupied space will be reused. at the same time, when receiving an ACK response, the receiver may want to receive more data, so the tcp_output () function will be called. (to be continued)

 

LWIP-an implementation of TCP/IP protocol stack. Recently, I have focused on the principles and implementations of TCP/IP technology. I have also seen Douglas E. comer and David L. the three volumes of internetworking with TCP/IP written by Steven S, but unfortunately, they just "read" rather than "study". The principle of this is limited to the superficial understanding, without in-depth or in-depth study of the implementation code. fortunately, I accidentally found some foreign papers on TCP/IP research. The LWIP is a basic implementation, and it should be of great help for beginners to have a deep understanding of TCP/IP. Therefore, I plan to translate some papers on LWIP (Introduction and implementation) so that I can really understand the protocol and hope to help like-minded people. I am the first to translate such computer science and technology articles, but my English skills are limited. This article is based on Adam dunkels's design and implemantation of the lwip TCP/IP stack. The following is a summary and directory structure. I hope I can translate this article and read its source code in a timely manner. Abstract: LWIP is an implementation of TCP/IP protocol stack. It reduces memory usage and code size, so that it can be used in small client systems with limited resources such as embedded systems. To reduce processing and memory requirements, LWIP uses the cropped API and does not require any data replication. This article describes the design and implementation of LWIP, including Protocol implementation and algorithms and data structures involved in underlying systems such as memory and buffer management. At the same time, the article also provides the lwip api user manual and some code instances that use LWIP.

1 Introduction 2 protocol layer 3 Summary 4 Process Model 5 Operating System Simulation layer 6 buffer and memory management 6.1 Packet Buffer 6.2 memory management 7 network layer interface 8 IP address processing 8.1 accept network packet 8.2 send Network Packet 8.3 forwarding network packet 8.4 ICMP processing 9 UDP processing 10 TCP processing 10.1 Summary 10.2 Data Structure 10.3 serial number calculation 10.4 queue and data transmission 10.4.1 confused window disease avoid 10.5 receiving packet segment 10.5.1 multi-channel Forwarding (demultiplexing) 10.5.2 receive data 10.6 accept new connection 10.7 fast retransmission 10.8 timer 10.9 round-trip time estimation 10.10 Congestion Control 11 Protocol Stack interface 12 application interface 12.1 basic idea 12.3 API implementation 13 code statistical analysis 13.1 code number of rows 13.2 target code size 14 Performance Analysis 15 api reference 15.1 data type 15.2 buffer function 16 network connection function 17 BSD socket library 18 code instance