Linux-gigabit NIC Driver Implementation Mechanism

I do not have much practical experience on Nic drivers. This analysis is purely a learning summary. I hope that you can raise some questions in the case of Cu drivers. You are also welcome to make some suggestions, you should give it to me ^_^. I believe that the analysis in this article will be incorrect. I hope you will be able to give me some advice. If it is reproduced, indicate the source.

1. Introduction
 
 
This analysis mainly targets e1000 NICs, and the driver source code is 7.3.20-K2. The purpose of this article is not to show how to write a driver, but to analyze the internal implementation mechanism of the NIC Driver. Pass this score
Analysis, hope to understand the relationship between each part of the driver, have an intuitive understanding of the network card to send and receive data packets, but also hope to help design the network card driver. Nic drivers and hardware and operations
The system is closely related, so it is necessary to clarify some problems completely. It requires a lot of experience and related knowledge, which is at a limited level and has little experience, therefore, there must be many problems. I hope readers of this article will
If any problem is found, contact the author.
2. architecture of the NIC Driver

As a PCI device, the NIC must comply with the corresponding PCI specification, that is, the corresponding ID number must be defined for the NIC, each PCI peripheral is labeled by a bus number, a device number, and a function number.
Knowledge. The NIC driver needs to define the corresponding pci_device_id structure to indicate the IDs of the PCI peripherals it supports. by searching for the settings in the pci_device_id of the driver
The slave ID to associate the driver with the device. As a PCI device, the NIC has two types of space: configuration space,
The CPU cannot be directly accessed. to access this space, you need to use the BIOS function. The other is the general control register space. After this part is mapped, the CPU can directly access the control.
 

During hardware power-on initialization, the BIOS checks all PCI devices and assigns a physical address to each device. The address is obtained through the bios and written to the configuration space of the device. Then the driver can
Map the common control registers of the network adapter to a memory space, and the CPU controls the registers of the network adapter by accessing the mapped virtual address. When the operating system is initialized, it allocates one
Pci_dev structure, and write the previously allocated physical address to the resource field of pci_dev. In the NIC Driver, you can read
The resource field obtains the NIC register configuration space address, which is obtained by the functions pci_resource_start () and pci_resource_end ().
The starting position between the two segments, which is mapped to the primary memory through ioremap (), so that the CPU access controls the I/O and memory space of the NIC. If you restart the NIC device
The register write command is used to locate the position of the Register through the first address after the ing and the corresponding register offset, and then write the Register through the writeb () function. Bias of related registers
The amount of data transferred, usually obtained through the datasheet related to the Nic. To obtain the MAC address of the network adapter, read the first six digits of the first address by using the readb () function.
 
 
You can use the pci_read_config _ and pci_write_config _ functions to read and write the configuration space of the NIC.
The command field is set to 1, so that the device can map registers to the memory. For example, the function pci_read_config_byte (pci_dev
Pdev, pci_interrupt_line, & IRQ) Get the interrupt number assigned by the device and save it in IRQ. Pci_read_config _ and
Pci_write_config _ series functions call pci_bus_read_config _ and pci_bus_write_config _ series functions.
These functions operate on the PCI bus structure corresponding to the NIC. For information about the configuration space of PCI registers, see Linux Device Driver
3rd or PCI bus demystified.

As a standard PCI device, the corresponding structure of the network card pci_dev represents the network card device, reflecting the specifications as a PCI device. The Network Transmission Nature of the network adapter.
Body net_device. The initialization of this struct is implemented by the NIC Driver. Operations on NICs in the kernel are essentially operations on the net_device structure.
In this example, both pci_dev and net_device indicate the NIC device, but they are different from each other. Net_device is an abstraction of a specific adapter, which provides a unified
Interface. The NIC Driver implements this abstraction based on a specific adapter.

The driver program of the PCI device is represented by the pci_driver struct. Therefore, the NIC driver should be an instance of this struct. In this struct, you should define the parameters related to the NIC and the corresponding operations.
. The specific adapter that the NIC driver actually operates on is a structure represented by the hardware-related adapter. The adapter reflects most of the hardware-related attributes.
In addition to the pci_dev structure operation, other operations on the NIC device are basically operations on the adapter structure. Adapter represents the relationship between net_device and pci_dev.
Connection also achieves adapter independence for network devices. The communication with the NIC device pci_dev is implemented through the adapter, which is the task to be completed by the NIC Driver.
 
 
Figure 2-1 below describes the relationship between the three important data structures. The pci_dev structure shows the configuration space of the NIC and the I/O and memory areas. The net_device structure provides operations to the kernel.
The abstract interface of the NIC. Its parameter values can be divided into five parts by function. In addition to the hardware independence, the e1000_adapter structure also manages the buffer space for sending and receiving data packets.
The virtual addresses mapped to the physical address space of the NIC are also saved in this structure. The e1000_hw structure in the e1000_adapter structure stores the hardware parameters of the network adapter.
Read the contents of pci_dev. The above data structure plays a core role in the work of the network card, but also the structure that must be operated by the driver.

3. Registration and initialization of NIC Devices

The registration and initialization of NIC devices are implemented in the e1000_probe () function of the related driver, how the device is associated with the driver, and how to call
E1000_probe () is not described here. In function e1000_probe (), call the function pci_enable_device () to enable the device.
Specify the DMA space, and then call the alloc_etherdev () function to generate the net_device struct. This struct represents the NIC device, and the parameter for net_device
After the number is initialized, call register_netdev () to register the device.

The above is only the registration of the device. The initialization of the device mainly includes the assignment of two struct values: net_device and e1000_adapter. Pair
The initialization of the e1000_adapter includes the initialization of the e1000_hw structure, which is implemented by calling the e1000_sw_init () function. In
Ioremap () is used during initialization to map the hardware address of the NIC to the virtual address of the memory.
Call the free_netdev () function to undo the NIC device. For more information about Nic device registration and initialization, see Understanding Linux Network internals. 4. Enable and disable Nic Devices

When the NIC device starts, it first calls the e1000_open () function. In this function, it calls e1000_request_irq () to request the interrupt number and the corresponding interrupt processing program.
E1000_intr () is actually implemented by calling the request_irq () function. Call in function e1000_open ()
E1000_setup_all_tx_resources () creates a sending Buffer Based on the number of sending queues.
E1000_setup_tx_resources () implementation. In e1000_setup_tx_resources (), it mainly describes the structure of the sending buffer.
The initialization of the body e1000_tx_ring, which associates the DMA buffer with the virtual address space mapped by the NIC, and uses the pci_alloc_consistent () function
Current consistency ing. The virtual address space corresponds to the physical address of the network card, so the three spaces correspond, And the DMA can be implemented on this basis. When the data packet content is mapped to the DMA Buffer
Then, it will be completely controlled by the device. The initialization of the DMA buffer is implemented in the e1000_probe () function of the driver. The e1000_open () function calls
E1000_up () configures some hardware and software parameters and space related to the NIC, such as reading and writing hardware registers and initializing the processing functions of the data packet receiving and sending space. Initial sending buffer space
As shown in Figure 4-1.
The initialization of the receiving buffer is similar to the above. e1000_setup_all_rx_resources () calls e1000_setup_rx_resources () to initialize the e1000_rx_ring struct. The structure of the receiving buffer space is 4-2.

5. send and receive data packets
Packet transmission:

Create the corresponding sending Buffer Structure e1000_tx_ring based on the number of sending queues num_tx_queues, which describes the direction of this region.
Desc of the e1000_tx_desc structure, which is the DMA bus address pointed to by the buffer zone. It is used to receive the buffer block array described in the e1000_buffer structure transmitted by hardware.
Buffer_info []. Other parameters are mainly used to describe these buffer blocks. Count indicates the number of buffer blocks, next_to_use and
Next_to_clean mainly describes the Usage Status of the buffer block, such as the location where the received data is received and the location where the data is to be received. When a new data packet is to be sent, it is first called by the upper-layer protocol.
E1000_xmit_frame (). In this function, call e1000_tx_queue () to find the buffer block for storage based on the corresponding parameters. The initialization of the buffer block is implemented by the function.
E1000_tx_map () implementation. The buffer area pointed to by buffer_info is mainly used to receive data packets mapped to bus addresses.
Next_to_match is connected to a ring, and each buffer block is represented by the structure e1000_buffer. In this structure, SKB stores the data packet content, and DMA indicates the data packet
The bus address. The function pci_map_single () is used for stream ing. The ing direction is pci_dma_todevice.
The content is mapped to the BUS address and then transmitted by the NIC. The structure and relationship between sent data packets are shown in Figure 5-1.
 
The buffer_addr pointed by desc in the e1000_tx_ring structure records the BUS address mapped to the buffer block sent each time, that is, the buffer_addr log.
The bus address is recorded. The DESC is a virtual address, which is the address of the sending buffer mapped through pci_alloc_consistent ().
The bus address corresponds to a segment in the e1000_tx_ring structure. The bus address is saved by the DMA member in the e1000_tx_ring structure. This ing is realized when the NIC is enabled, and it is mapped to when the packet is sent.
The bus address of is different, and the latter is dynamically carried out during transmission.

Receive data packets

Root
Number of data receiving queues num_rx_queues create the corresponding receiving Buffer Structure e1000_rx_ring, which describes the point of this region to e1000_rx_desc
Structure of the DESC, the DMA bus address pointed to by the buffer, used to receive the buffer block array buffer_info [] described in the e1000_buffer structure transmitted by hardware.
The following parameters are used to describe these buffer blocks. Count indicates the number of buffer blocks, and next_to_use and next_to_clean mainly describe the Usage Status of the buffer blocks,
If the location where the received data is received and the location where the data is to be received are received, locate the corresponding region based on the two parameters when new data packets are to be received. For data packets that require multipart receiving,
Ps_page and ps_page_dma are implemented. The parameter CPU specifies the processor of the receiving Buffer Queue. The ing direction between the BUS address and the virtual address to be sent is
Pci_dma_fromdevice: the control bus maps the content of the BUS address to the virtual address space. The structure and relationship between received data packets are shown in Figure 5-2.
 
When a new data packet arrives, the e1000_intr () function is triggered first. In the function, a new data packet is found in the buffer block array buffer_info.
And complete the e1000_buffer Structure assignment. The packet is actually received by copying the content pointed to by the BUS address to SKB, and then passing it to
The receiving function of the Upper-layer protocol. 6. NIC Driver Design
To compile the NIC Driver, you must perform the following operations on the three types of struct:
1. struct related to the network protocol stack, such as the sk_buff struct.
2. network card and protocol stack Interface related struct, such as net_device struct.
3. struct related to the I/O bus, such as the DMA buffer for data packet transmission using the PCI bus and the pci_dev struct representing the NIC.
 
 
When designing a NIC Driver, you need to implement corresponding functions for the above data structure, such as the sk_buff structure operation to effectively control data packets; The net_device structure operation can be
Nic operations (such as enabling and disabling) can send data packets and round-robin data packets. You can configure the corresponding scheduled operations and statistical data packets for the NIC.
(Ethtool. When designing the NIC Driver, you must consider how to coordinate with the upper-layer protocol and control the underlying BUS address.
For more detailed design procedures of the NIC Driver, see "essential Linux Device Driver" and "Linux Device Driver 3rd".

7. Summary
This article focuses on the important data structures involved in the NIC driver and the implementation of packet sending and receiving. After understanding these implementation mechanisms, it will be helpful for the design and implementation of the driver, because the mechanism itself is very difficult and the author level is limited, the analysis conclusions cannot be completely correct. There are still some unclear questions. I hope you can help me solve them:
1. I have never been very clear about the relationship between the PCI BUS address and the physical IP address of the network adapter. Therefore, the above analysis has not been clearly stated, and this part of details is ignored during drawing;
2. How to schedule the ring in the DMA buffer when receiving packets and sending packets;
· Http://www.armjishu.com/bbs/viewtopic.php to be continued? Id = 549