PCIE Driver and DMA mechanism under Linux __linux

1. Driver function:

· The device driver masks the implementation details of the hardware to the application, allowing the application to operate the external device as if it were a normal file. The Linux operating system abstracts the processing of the hardware, can use and operate the same file, the standard system call interface to complete the opening, shutdown, read and write drink I/O control operations, and the driver is the main task is to implement these system call functions.

· Each device file corresponds to two device numbers, where the master number identifies the type of device, the driver used by the device, and the secondary device number identifies the different hardware devices that use the same device driver. The main device number for the device file must be the same as the primary number that the device driver requested when it landed on the device, otherwise the user process could not access the device driver.

2. The difference between a block device and a character device:

· The difference between the two is only in the software interface between the kernel and the driver, block devices use a system memory as a buffer, when the user process to the device to read and write requests, the driver first look at the contents of the buffer, if you can meet the user requirements, then return the corresponding data, otherwise call the corresponding request function for actual I/O operations.

3. Allocation and implementation of DMA cyclic buffers:

· For the acquisition and processing of high speed data signals, it is necessary to apply a large number of DMA buffer in the initialization module (probe) of the driver, and the size of the application directly depends on the success or failure of real-time processing. Direct Memory Access (DMA) is a hardware mechanism that allows peripheral devices and main memory to directly transmit I/O data, avoiding significant computational overhead.

4. Memory partition segment of Linux kernel:

· Three sectors, available for DMA memory, conventional memory, and high memory.

· The usual memory allocations occur in the normal memory area, but can also be requested to be allocated in other extents by setting the memory identity. Memory that can be used for DMA refers to memory that exists within a special address range that can be used by peripherals to perform DMA access for data traffic.

· DMA Loop Buffer Allocation requirements: Physical continuous, DMA accessible, large enough.

5. Linux memory allocation function:

· The Linux system uses the virtual address, the memory allocation function provides the virtual address, through the Virt_to_bus conversion to obtain the physical address.

· The function that allocates kernel memory space: Kmalloc realizes kernel memory request less than 128KB, applies for space physics continuous; __get_free_pages realizes maximum 4MB memory request, in the page as the unit, the application Space physics continuous; Vmalloc allocated virtual address space continuously, However, it may not be continuous in physics.

· The Linux kernel specifically provides function pci_alloc_consistent for PCI devices to request kernel memory, which is supported by byte length request, and the function calls __get_free_pages, so the maximum is 4MB.

6. Methods of DMA data transfer:

· One is a software-initiated data request (for example, by a read function call), and the other is the hardware that asynchronously passes data to the system. For data acquisition equipment, even if there is no process to read the data, but also write constantly, waiting for the process to call, so the driver should maintain a ring buffer, when the read call can always return to the user space needed data.

7. How to initiate an interrupt request to the CPU in PCIe:

· Message signal Interrupt (MSI), Intx interrupt.

· In the MSI interrupt mode, the device requests a CPU interrupt service by writing specific data to the OS's pre-configured main memory space, the preferred interrupt signaling mechanism for the PCIE system, the bridging device for PCIe to Pci/pci-x, and traditional endpoint devices that cannot use the MSI mechanism, Adopt INTX virtual interrupt mechanism.

· PCIE Device registration interrupt using a shared interrupt, the Linux system through the REQUEST_IRQ implementation of the interrupt handler registration, the location of the call in the first time the device was opened, the hardware was interrupted before, again, FREE_IRQ the opportunity to shut down the device the last time, the hardware does not have to interrupt the processor.

· The function of the interrupt handler is to feedback the interrupt received to the device and read and write the data accordingly. Interrupt signal arrival, the system calls the corresponding interrupt processing function, the function to determine whether the interrupt number match, if, then clear the interrupt register corresponding bit, that is, before the driver launched a new DMA, the device will not produce other interrupts, and then handle accordingly.

8. Data read and write and IOCTL control:

· Data reading and writing: When the application process does not require data, the driver dynamically maintains the DMA ring buffer, and when the application process requests data, the driver provides Copy_from_user ()/copy_to_user () through the Linux kernel to implement the data copy between the kernel state and the user state.

· Hardware control: User space often go back to request equipment to lock the door, report error messages, set registers, and so on, these operations through the IOCTL support, you can PCIe card given register space to configure.

9. Registration of interrupt handlers:

· The interrupt number is allocated and written to the device configuration space during the BIOS initialization phase, and then Linux reads the interrupt number from the configuration space and writes to the Pci_dev IRQ member when the PCI_DEV is established, so it reads directly from the Pci_dev when registering the interrupt program.

· When the device is interrupted, 8259A sends the interrupt number to CPU,CPU to locate the interrupt handler according to the interrupt number.

generation of DMA data transfer mechanisms:

· Traditional Classic process: The data arrives at the NIC-> network card produces an interrupt to the kernel-> kernel uses I/O instruction, reads the data from the network card I/O area. In this way, when large traffic data arrives, the network card will generate a large number of interrupts, the kernel in the interrupt context, will waste a lot of resources to deal with the interruption itself.

· Improvement: NAPI, that is, polling, that is, the kernel screen interrupt, a certain amount of time to ask the network card, whether there is data. In the case of small amounts of data, this approach wastes a lot of resources.

· Another problem, the CPU to the network card I/O region, including I/O registers and I/O memory read, and then put to the system physical memory, all occupy a lot of CPU resources, make improvements, that is, DMA, so that the network card directly from the main memory read and write their own I/O data

· First, the kernel in the main memory to send and receive data to create a circular buffer queue (DMA ring buffer), the kernel of this buffer through DMA mapping, the queue to the network card, the network card received data, directly into the ring buffer, that is, directly into the main memory, and then to the system to interrupt

· The kernel receives interrupts, cancels the DMA mapping, and can read data directly from the main memory.