Talk about Linux user-driven design

Preface

The device driver can run in the kernel state, can also run in the user state, the advantages and disadvantages of user-driven online there are many discussions, and some have risen to the political, there is no more discussion here. Both user-driven and kernel-driven, they have their own drawbacks. Kernel-driven problems are: system call overhead, steep learning curve, poor interface stability, difficulty debugging, fatal bug, limited programming language selection, and user-driven challenges: How to interrupt processing, how to DMA, how to manage device dependencies, inability to use kernel services, and so on. In this context, the article "User-space Device Drivers in linux:a First look" is described in more detail.

It may be for performance optimization, or for fault isolation, or to evade the constraints of open source licenses, regardless of the purpose, Linux has a variety of user-driven implementations, such as UIO,VFIO, USB user-state drivers. They differ in the design and implementation details of handling interrupts, DMA, device dependency management, and so on, which is not mentioned in the User-space device Drivers in linux:a First Look, and is the focus of this article.

Uio

The UIO framework exports the SYSFS and/dev/uiox 2 sets of user-state interfaces, the user controls device node/dev/uiox, the mmap () interface is used to map device register space, and the write () interface is used to control interrupt shutdown/Open, read () The interface is used to wait for a device to break.

Because the answer to the device interrupt must be in kernel space, there is a small amount of code in the kernel space to answer interrupts and prohibit interrupts, and the rest of the work is left to the user for space processing. If the user space waits for a device to break, it simply needs to be blocked on the read () operation on the/dev/uiox. The read () operation returns immediately when the device generates an interrupt. UIO also implements the poll () system call, and you can use Select () to wait for the interrupt to occur. Select () has a timeout parameter that can be used to allow for a limited time to wait for interrupts.

Several features of UiO:

• A UiO device supports a maximum of 5 mem and portio spatial mmap mappings.
• The interrupt user-state communication mechanism of the UiO device is based on the Wait_queue implementation.
• One UiO device only supports one interrupt sign registration, which supports interrupt sharing.

In general, the UiO framework is suitable for simple device drivers because it does not support DMA, does not support multiple interrupt breaks, and lacks the logical device abstraction capability.

Vfio

As mentioned above, UiO does not support DMA, so through the DMA transmission of high-traffic data IO devices, such as network cards, video cards and other devices, can not use the UiO framework, Vfio as a UiO upgrade version, mainly to solve the problem. By configuring the Iommu interface through the user configuration, you can limit the DMA address space mapping to the process virtual space. This is especially important for high performance driver and virtualization scenario device passthrough.

In the VFIO framework, there are several core concepts or objects: IOMMU,/dev/vfio, container, Iommu_group.

• IOMMU is a hardware unit that maps the IO address of the device to a virtual address, provides a page table mapping for the device, and writes the data directly to the user space via Iommu. The reason for not sharing the MMU unit is to ensure that the page tables of the process are independent from each other and prevent any address space of the device from accessing the process. So Vfio's Iommu function guarantees a secure, non-privileged level of user-state device drivers.
• /dev/vfio is a device file that is presented as a user state for a IOMMU device.
• Container is a kernel object that represents a Iommu device, which is a kernel-state rendering of a IOMMU device. So in Vfio, container is the smallest object of the Iommu operation.
• In a virtualized scenario, a physical network card may be virtualized into several virtual network cards, or virtual function devices (VF), which share a iommu, so the VFIO model adds a Iommu_group concept that represents a set of devices that share the same iommu.

Several features of Vfio:

• Vfio devices support multiple interrupt registration.
• The device interrupts the user-state communication mechanism based on the EVENTFD/IRQFD implementation. The user select/poll/epoll an asynchronous event notification from the kernel state to the user state via the/DEV/VFIO device.
• Logical abstraction of physical devices is supported.
• Only PCI Intx interrupt sharing is supported, and other types of interrupts do not support sharing.
• Vfio supports only specific IOMMU devices, such as PCI devices on x86 and PowerPC platforms, and platform devices for ARM platforms.

USB User State Driver

USBFS (USB file system) provides a number of function interfaces and data structures for operating USB devices in user space, which can be used to control and transmit USB devices directly when developing user-state drivers.
Libusb encapsulates the function interface and data structure provided by the USB file system, which can effectively reduce errors caused by improper use of functions and structures in the program. LIBUSB provides two mechanisms for access to USB devices, simultaneous access and asynchronous access.
KHUBD is a kernel background thread that is used to manage the status of the monitor USB hub, which, once hot-swappable, wakes up the thread, adding or removing devices, and sending NetLink messages to the user state space.

Several features of the USB user-driven framework:

• The device interrupts the user-state communication mechanism based on the NetLink implementation.
• The USB device renders to the user state in the form of a USBDEVFS (later renamed USBFS) file system, such as/PROC/BUS/USB/BBB/DDD.
• All access to USB is implemented via the USBFS IOCTL, such as control transfer, bulk transfer, reset, etc.
• Only USB devices are supported

Resources

Linux Kernel v4.4

User-space Device Drivers in linux:a first look

User-mode Driver Framework

Vfio Introduction

Concept space Deconstruction of Linux Iommu and Vfio

Talk about Linux user-driven design