Introduction to Linux PCI device drivers

Note: a good article on understanding PCI devices and PCI drivers

To understand the Linux PCI device driver, first understand that the so-called Linux PCI device driver actually includes the Linux PCI device driver and the device driver. I do not know whether the reader understands this sentence. I think this sentence is very important. For drivers like PCI and USB, we must understand this concept, in order to understand how to look at PCI, USB and similar bus-type drivers in Linux.

 

In http://tldp.org/ldp/tlk/dd/pci.htmlthis article (the whole book is called the Linux kernel, Chinese translation see http://oss.org.cn/ossdocs/linux/kernel/ this article also referred to the Chinese translation) mentioned:

Linux PCI initialization code is logically divided into three parts:
(1) PCI Device Driver
The pseudo-Device Driver queries the PCI system from bus 0 and locates all the PCI devices and PCI bridges in the system. It creates
It can be used to describe the data structure linked list at the topological level of the PCI system. And the number of all the detected PCI bridges.
(2) PCI BIOS
This software layer provides the services described in bib-PCI-bios reduction. Although Alpha AXP does not provide the BIOS service
The version contains the corresponding functions.
(3) PCI fixup
PCI initialization patch code related to a specific system

Here we will mainly discuss the Linux PCI device driver. The sample code containing the device driver will be listed at the end for your reference only.

 

I. Overview and introduction

PCI (periheral Component Interconnect) has three address spaces: pci I/O space, PCI memory address space, and PCI configuration space. The pci I/O space and PCI memory address space are used by the device driver, while the PCI configuration space is used by the PCI initialization code in the Linux kernel. These codes are used to configure the PCI device, for example, the interrupt number and the I/O or memory base address. For writing a driver for a Linux device, you need to get a general idea of what the Linux kernel has done for us, and what we need to do. The latter is what we need to do when writing the driver. Therefore, the PCI device driver here is to roughly describe what the Linux kernel has done for us (master), and then what we should do (times ).

(1) What does Linux kernel do?

Simply put, the Linux kernel mainly involves enumeration and configuration of PCI devices, which are completed at the beginning of the kernel.

Enumeration: For the PCI bus, a device called the PCI bridge is used to connect the parent bus to the sub-bus. As a special PCI device, the PCI bridge mainly includes the following three types:

(1) host/PCI Bridge: used to connect the CPU and PCI root bus. The number of the 1st root bus is 0. In PC, the memory controller is usually integrated into the host/PCI bridge device chip. Therefore, the host/PCI bridge is also known as the "North Bridge chipset (North Bridge chipset )".

(2) PCI/ISA Bridge: used to connect to the old ISA bus. In general, devices like the i8359a Interrupt Controller in PCI are also integrated into PCI/ISA bridge devices. Therefore, the PCI/ISA bridge is also known as the "South Bridge chipset (South Bridge chipset )"

(3) PCI-to-PCI bridge (hereinafter referred to as the PCI-PCI Bridge): used to connect the PCI primary bus (primary Bus) and secondary bus (secondary bus ). The PCI bus of the PCI-PCI bridge is called the primary bus, that is, the secondary bus's parent bus; the PCI bus connected by the PCI-PCI bridge is called the secondary bus, that is, the subbus of the primary bus.

From PCI local bus specification revision 2.1, you can see that the class code of the PCI-PCI bridge (see figure 3) is 0x060400.

 

The CPU is connected to a PCI bus through the host/PCI bridge. The PCI bus at this location is called the root bus. A pc usually has only one host/PCI Bridge. On the basis of one PCI bus, you can connect to another layer of bus through the PCI bridge, for example, you can connect to another PCI bus through the PCI-PCI Bridge, A PCI-ISA bridge can be used to connect to an ISA bus. In fact, the ISA bus in modern PC is connected to the PCI bus through the PCI-ISA bridge. In this way, a hierarchical tree structure of PCI system is constructed by using the PCI-PCI bridge. For the upper-layer bus, the PCI bridge connected to the bus is also a device. However, this is a special device. It is not only a device on the upper-layer bus, but also an extension of the Upper-layer bus.
The so-calledEnumerationThat is, all the devices connected to the first PCI bus are listed and scanned from the host/PCI bridge. If one of these devices is a PCI-PCI bridge, further probe and scan the secondary PCI bus connected to the bridge. This recursion continues until all PCI devices in the system are exhausted. As a result, a PCI tree representing the PCI bus and devices is established in the memory. Each PCI device (including a PCI bridge device) is represented by a pci_dev structure, and each PCI bus is represented by a pci_bus structure. You have a Hardware Device Tree built through the PCI bridge, and I have a software tree built through the data structure in the memory. How harmonious is it?
Haha.

 

 

 

 

Figure 1 PCI system

Configuration: PCI devices generally have some RAM and Rom space. The common control/status registers and data registers are usually displayed in the form of Ram intervals, generally, the addresses in these intervals start from 0 in the device. When multiple devices are mounted on the bus, access to these spaces may conflict. Therefore, these addresses must be mapped to the system bus first, and then to the virtual address space of the kernel. The so-called configuration is to map the addresses by performing operations on the registers in the PCI configuration space (only the internal addressing is mapped to the bus addresses, the virtual address space mapped to the kernel is done by the driver of the device itself ).

(2) How does the Linux kernel work?

The first thing to note here is that for PCI device initialization (that is, the enumeration and configuration work mentioned above), both the bios and Linux kernel of the PC can be done. Generally, as long as a PC uses a PCI bus, its BIOS must provide support for PCI bus operations. Therefore, it is called pci bios. In addition, the earliest Linux kernel uses this BIOS call method to obtain information about PCI devices in the system, but not all platforms have BIOS (such as some embedded systems ), in addition, in practice, we also found that some PCI bios on the mother board had such a problem, so later we changed it to the Linux kernel. However, the Linux kernel is still considerate.
The menuconfig options provide us with the right to choose, that is, PCI access mode, which provides four options: bios, mmconfig, direct, and any. The direct mode means that the kernel completes the initialization without the BIOS.

2. Start our enumeration and Configuration

Note: For clarity, the initialization process of the PCI device is briefly described (because the device driver model has not been introduced in 2.4.18, so that we can focus on the PCI device driver itself ). Here is the analysis of the Linux-2.4.18 kernel, the main reason from the reference materials should also be able to understand, here a lot is to refer to the data in [1] to analyze. If you want to learn the PCI Driver, take a look at the section PCI bus in Chapter 8 of [1. Then you can find an example of the driver code to see, it can be said that it is a start to the PCI device driver, of course, the premise is to understand the haha.

Let's talk nonsense. Next we will go to the topic. As mentioned above, PCI has three address spaces. The PCI configuration space is used for the PCI initialization code in the Linux kernel, which is used for enumeration and configuration. So what is in the PCI configuration space? It should obviously be a register, called a configuration register group. When the PCI device is powered on, the hardware remains inactive. That is, the device only responds to the configuration transaction. When the device is powered on, no memory and I/O ports are mapped to the address space of the computer. Other device-related functions, such as interrupt reports, are also disabled.

The PCI Standard specifies that each device's configuration register group can have a maximum of 256 bytes of continuous space. The use and format of the first 64 bytes is standard, which is called the configuration register header. The system provides hardware-related mechanisms so that the PCI configuration code can detect all possible PCI configuration register headers on a given PCI bus, so as to know which PCI slot currently has a device and which slot has no device. This is done by reading a field in the PCI configuration register header (generally the "vendor ID" field ). If a slot is empty, the above operation will return some error return values, such as 0 xffffffff. There are three types of headers (64-byte headers), of which "Type 0" (Type
0) the header is used for general PCI devices, the "1" header is used for various PCI-PCI bridges, and the "2" header is used for PCI-carw.bridge, cartana is the bus used in the laptop and we don't care about it. The 16 bytes in the 64-byte header contain information about the header type, the type of the device, the nature of the device, and who made the device. Based on the information provided in the 16 bytes, determine how to further interpret and process the 48 bytes in the remaining header. For the 16-byte address, include/Linux/PCI. h defines such constants.

 

# Define pci_vendor_id 0x00/* 16 bits */
# Define pci_device_id 0x02/* 16 bits */
# Define pci_command 0x04/* 16 bits */

 

# Define pci_status 0x06/* 16 bits */

 

# Define pci_class_revision 0x08/* high 24 bits are class, low 8 Revision */
# Define pci_revision_id 0x08/* revision ID */
# Define pci_class_prog 0x09/* Reg. level programming interface */
# Define pci_class_device 0x0a/* Device class */

# Define pci_cache_line_size 0x0c/* 8 bits */
# Define pci_latency_timer 0x0d/* 8 bits */
# Define pci_header_type 0x0e/* 8 bits */

 

Corresponding to the first 16 bytes in Figure 3 (see below. In addition, we also see that the definitions next to pci_header_type (that is, the register storing the header type) are the three types of headers mentioned above:

# Define pci_header_type_normal 0
# Define pci_header_type_bridge 1
# Define pci_header_type_car1_2

 

 

In Linux, you can use commands such as CAT, Proc, and PCI to view the categories, models, vendors, and other information of all PCI devices in the system, that is, from these registers. The following is the information truncation using the lspci-x command in the Virtual Machine (the lspci command also uses the/proc file as its information source ):

 

00:00. 0 host bridge: Intel Corp. Sort BX/ZX/DX-Sort 43bx/ZX/dx host bridge (Rev 01)
00: 86 80 90 71 06 00 00 02 01 00 06 00 00 00 00 00
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 ad 15 76 19
30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

 

The first thing to note is that the PCI register is in the small-end byte sequence format. Then, according to the structure of the following PCI configuration register group (figure 3), it is clear that the vendor ID of the host bridge is 0x8086. You can also guess that the vendor is intel without saying it.

There is a problem here to be clear first, that is, the address of these registers, otherwise it will not be able to proceed in the future. The configuration register allows us to configure to access the storage space on the PCI device, but these configuration registers are also located in the address space of the PCI device, how to access this part of space becomes an entry point for our entire initialization work, just as every executable program has an entry point. The PCI method is to make the configuration register groups of all devices adopt the same address, which is distinguished by additional conditions when the PCI Bridge of the bus is accessed. The CPU sends a command to the host-PCI bridge through a unified entry address, and the corresponding PCI bridge indirectly completes the specific read/write. For i386 processors, the PCI bus designer reserves 8 bytes in the I/O address space for this purpose, that is, 0xcf8 ~ 0 xcff. These 8 bytes constitute two 32-bit registers, the first is "Address Register" 0xcf8, and the second is "data register" 0 xcfc. To access a configuration register of a device, the CPU first writes the target address to the address register, and then reads and writes data through the data register. However, the destination address written into the address register is a comprehensive address including the bus number, device number, function number, and device Register address. Format 2:

 

 

Figure 2 Integrated address of the write address register 0xcf8

The bus number, device number, and function number are used to expand the configuration register address, which is the additional conditions mentioned above. First, each PCI bus has a bus number, and the bus number of the main bus is 0, and the rest is specified by the CPU whenever a PCI bridge is detected in the enumeration phase, increments sequentially. The device number generally represents a PCI interface card (more specifically, the PCI bus interface chip), usually depends on the location of the slot. Each PCI interface card can have several functional modules that share a PCI bus interface chip, including the electronic lines used for address ing to reduce costs. Logically, every "function" is actually a device (people who have seen the USB driver are familiar with it ).
So the device number and function number can be combined as the "logical device number", and each card can accommodate up to eight devices. Obviously, these fields (the entire 32bit) are combined to uniquely identify a PCI logical device in the system. At the beginning, only Bus No. 0 can be accessed. When scanning Bus No. 0, if a device above is found to be a PCI bridge, a new bus No. is specified for it, for example, 1, in this way, Bus 1 can be accessed, which is one of the tasks in the enumeration phase.

The following section is taken from the PCI driver of ldd3:

Lspci output (a part of pciutils, which is available in most releases) and Information layout in/proc/PCI and/porc/bus/PCI. the sysfs of the PCI device also shows the addressing scheme and PCI domain information. when the hardware address is displayed, it can be displayed as two values (an 8-bit bus number and an 8-bit device and a function number), as three values (bus, device, and function), or as four values (domain, bus, device, and function ).
Hexadecimal display.

For example,/proc/bus/PCI/devices uses a single 16-bit field (for ease of analysis and sorting), And/proc/bus/busnumber divides the address into three fields. the following shows how these addresses are displayed and only the beginning of the output line is displayed:

$ lspci | cut -d: -f1-30000:00:00.0 Host bridge0000:00:00.1 RAM memory0000:00:00.2 RAM memory0000:00:02.0 USB Controller0000:00:04.0 Multimedia audio controller0000:00:06.0 Bridge0000:00:07.0 ISA bridge0000:00:09.0 USB Controller0000:00:09.1 USB Controller0000:00:09.2 USB Controller0000:00:0c.0 CardBus bridge0000:00:0f.0 IDE interface0000:00:10.0 Ethernet controller0000:00:12.0 Network controller0000:00:13.0 FireWire (IEEE 1394)0000:00:14.0 VGA compatible controller$ cat /proc/bus/pci/devices | cut -f1000000010002001000200030003800480049004a0060007800800090009800a0$ tree /sys/bus/pci/devices//sys/bus/pci/devices/|-- 0000:00:00.0 -> ../../../devices/pci0000:00/0000:00:00.0|-- 0000:00:00.1 -> ../../../devices/pci0000:00/0000:00:00.1|-- 0000:00:00.2 -> ../../../devices/pci0000:00/0000:00:00.2|-- 0000:00:02.0 -> ../../../devices/pci0000:00/0000:00:02.0|-- 0000:00:04.0 -> ../../../devices/pci0000:00/0000:00:04.0|-- 0000:00:06.0 -> ../../../devices/pci0000:00/0000:00:06.0|-- 0000:00:07.0 -> ../../../devices/pci0000:00/0000:00:07.0|-- 0000:00:09.0 -> ../../../devices/pci0000:00/0000:00:09.0|-- 0000:00:09.1 -> ../../../devices/pci0000:00/0000:00:09.1|-- 0000:00:09.2 -> ../../../devices/pci0000:00/0000:00:09.2|-- 0000:00:0c.0 -> ../../../devices/pci0000:00/0000:00:0c.0|-- 0000:00:0f.0 -> ../../../devices/pci0000:00/0000:00:0f.0|-- 0000:00:10.0 -> ../../../devices/pci0000:00/0000:00:10.0|-- 0000:00:12.0 -> ../../../devices/pci0000:00/0000:00:12.0|-- 0000:00:13.0 -> ../../../devices/pci0000:00/0000:00:13.0`-- 0000:00:14.0 -> ../../../devices/pci0000:00/0000:00:14.0

All three devices are listed in the same order, because lspci uses the/proc file as its source of information. take the VGA Video Controller as an example. 0x00a0 indicates. 0 when divided into domain (16 bits), bus (8 bits), device (5 bits) and function (3 bits ). why does 0x00a0 correspond. the value 0 indicates the content in Figure 2. 0x00a0 indicates the bus (8 bits), device (5 bits), and function (3 bits) according to the Register in Figure 2 ). 0x00a0 = 0000000010100000, it is easy to see that the high 8-bit is the bus number, that is, 0. The remaining 0xa0 = 10100000 indicates that if the 3-bit lower value represents the function number, the remaining 10100 indicates the device number, and the value of 8 bits is 00010100, that is, 0x14.

 

 

Figure 3 PCI configuration register group

 

References:

[1] Linux kernel source code scenario analysis (next)

[2] Linux device driver development

[3] Linux Device Driver (Third edition)

[4]pcipci.txt under Documentation

[5] proficient in Linux device driver development

[6] http://tldp.org/LDP/tlk/dd/pci.html

[7] http://linux.die.net/man/8/lspci