PCI Hot-swappable __ Embedded

A related technology and research

In 1997, PCI SIG developed the first PCI hot-Plug specification, which defines the platforms, boards, and software elements necessary to support hot-swappable. PCI SIG introduces the standard hot-swappable controller specification (SHPC spec), which clearly provides hot-swappable standard usage patterns and stringent register group requirements, and allows the operating system provider to provide hot-swappable support outside of platform-specific software, gradually completing the hot-pluggable standard-setting process, Enter the comprehensive promotion stage of technology. After 2002, Intel gave Hot plug as a natural attribute to the newly introduced PCI Express specification, PCI Express Hot plug summed up five years of experience in industry standards, with the following features: Based on SHPC mode and its function expansion; PCI Express from the design of the hot-swappable registers integrated into its standard performance Register group (and in SHPC 1.0, hot-swappable registers are additional); Provides a unified hot-swappable hardware register interface to the operating system, gives the operating system the ability to make native hot plug (bypassing the traditional BIOS based hot plug method); By defining the required hardware requirements for hot plug in the basic system hierarchy, a standard usage pattern is perfected, and specifications are provided to ensure the low cost and high platform reliability of OEM products.

The fully functional PCI Express hot-swappable system must also have the support of the operating system and device drivers in addition to the platform hardware and firmware support.

While major chip manufacturers have introduced products to support PCI Hot plug, Microsoft,novell,sco and other companies in their associated operating systems also include support for hot-swappable technology, Novell NetWare 4.11&5 and SCO UnixWare 7 and newer versions support full PCI hot swappable (Hot swap and hot add). Microsoft Windows NT 4.0 leverages the Compaq Server support Disk (SSD) for NT 4.0, which provides PCI hot swap support, and Microsoft Windows 2000 also builds comprehensive software support for PCI hot-swappable technology after 2000. For Gnu/linux, starting with the kernel 2.4 version of January 2001, PCI Hot plug began to become one of its standard features, to the 2.6 version, the hot-swappable feature has been integrated into the core device model. Today, almost all Linux distributions, including Redhat, Debian, and United Linux, provide good support for PCI hot Plug and extend them separately. The BSD branch started late in this area, and OpenBSD in the 3.6 version of March 2004 with no device-driven hot-swappable reference frame, and FreeBSD to 5.3 has not yet provided support for PCI hot-plug.

As a 2002 norm, the support for PCI Express by various general-purpose operating systems has only just begun. In Microsoft Windows XP, Windows xp-based, and Windows Server 2003, PCI Express devices are hot-swappable as PCI devices, and the advanced features of other PCI Express can be activated by the firmware. Native support for PCI Express advanced Features is available in Windows Longhorn. Gnu/linux in the kernel version of 2.6.9 to provide support for PCI Express Hot plug, in the middle of the software architecture changes, gradually developed into a kernel version 2.6.10 and the kernel version 2.6.12 two different PCI Express hot-swappable driver architecture. This paper mainly discusses the PCI Express hot-swappable drive architecture adopted by 2.6.10, and gives the new characteristics of 2.6.12 at the end.

Back to top three PCI Express hot-swappable software support

According to the specification, a complete Native PCI Express hot-swappable system requires several aspects of mutual coordination, respectively, hardware elements, firmware elements and software elements. Hardware elements refer to the electrical characteristics of the motherboard bus system, including the hot-swappable controller (Hot-plug Controller), the slot Power switch logic (card Slot-switching Logic), and the Board reset logic (card reset Logic), Power LED (indicator), hint button (Attention button), and board presence detect pins (card Present detect Pins), and so on; the firmware element refers to the support that the motherboard BIOS must provide for hot plug. To implement native PCI Express Hot plug, the firmware must provide either a oshp method or one of the ACPI _OSC methods; The software element refers to the functional components that must be provided by the operating system for the integrated use of PCI Express Hot plug. software Elements

In order to manipulate the hardware elements of the platform and provide PCI Express hot-swappable services, we must implement the main hot-swappable software elements shown in table 1. PCI Express Service Model

Standard hot-swappable system drivers in PCIe hot-swappable (PCIEHP) subsystems to obtain hot-swappable part information for hardware by checking the slot performance registers (Slot capabilities register) in the PCI Express performance data structure. The information that these registers can reflect is as follows: whether the socket supports hot-swappable devices supporting the sudden pull out of the software without notifying the hardware elements such as lights, controllers, etc.

Once the software completes the hot plug function of the PCI Express device, hot plug behavior (such as removal or insert request, power failure) can be used to submit system interrupt and power management events to the system hot-swappable processing mechanism. This hot-swappable processing mechanism is provided by the operating system independently, Figure 1 shows the PCI Express native hot-swappable service model, and demonstrates its difference from the traditional ACPI based model. Figure 1 PCI Express Hot swap service model comparison

Back to top four Linux 2.6.10 PCIE hot-swappable Subsystem Code Analysis Hot -swappable frame

In order to provide PCI Express Hot plug service, the Linux 2.6.10 PCIe hot-swappable (PCIEHP) subsystem must implement hot-swappable software elements such as user interface, hot-Plug service program and standard hot-swappable system driver, and the behavior of the software conforms to the hot-swappable standard usage mode [4]. In addition, some specific hot-swappable features must be combined with a device driver that supports hot-swappable.

The body of the PCIEHP kernel is a core thread, the control logic is loaded into the kernel in a modular way, which is responsible for monitoring the hot Plug events on the PCI Express bus and doing the related processing; the outer part is a user-space script that is invoked from the kernel and performs a post-processing process based on the information returned by the kernel.

1 Hot-swappable driver life cycle

Starting from the PCIEHP subsystem, the PCIEHP fully takes over the PCI Express slot Hot-plug event in the system while the subsystem is unloaded.

The PCIEHP subsystem starts with the user state daemon first. Then initialize the notification mechanism, open PCI Express Hot Plug Event processing core thread, for the operating system of the bus data structure of each of the initial initialization of a hot-swappable slot list. Next, the device resource management is initialized based on whether the ACPI-type resource management is specified at the time of kernel compilation. Finally, according to the device identification, register the PCI Express hot-swappable driver in the system and bind it to all PCIe bridging devices on the bus that may mount hot-swappable slots, then initialize the corresponding hot-swappable controller, allocate resources, and turn on the scheduled interrupt polling mechanism.

Uninstall the PCIEHP subsystem, first releasing the resource occupied by each controller in the hot-swappable controller global list, releasing the resource occupied by each slot in the hot-swappable slot global list, terminating the notification mechanism, and ending the PCI Express hot-Plug event processing core thread; and then, Release the system resources occupied by the associated device, depending on whether ACPI resource management is specified at compile time of the kernel; then unregister the PCI Express hot-swappable driver and terminate the nuclear daemon after the current hot swap event is processed.

After the PCIEHP load, in order to track its state changes during hot-plug execution, we abstract the status of the hot-swappable slots as follows: Static state,
Normal working or idle period, flashing prompt before opening (Blinkingon state)
Flashing prompts before the time period before the time before the confirmation of the open request (Blinkingoff State)
Turn off request to a period of time before confirmation (PowerOn State)
The time period shutdown state for the open operation (Poweroff)
Time period when the shutdown operation was performed

The transition relationship between each slot state is shown in Figure 2: Figure 2 Slot state transition diagram

2 User Control Scripts/sbin/hotplug and nuclear and internal communication mechanisms

After the kernel hot-plug process completes, it invokes a user state script hotplug in the kernel, which will follow up on the information provided by the kernel. It is the user space portion of the device hot-swappable notification, which receives the hot-pluggable operation types and environment variables from the kernel, handling device mount and uninstall operations. Typically, HotPlug invokes a policy script based on the device type to perform subsequent configuration work on the device and the current parameters of the system. For example, for a network card, the script may be invoked to specify an IP address, a gateway and a domain name server, and so on, and for storage devices, the Mount command may be invoked to load it into the file system.

The information that the kernel transmits to HotPlug contains hot-swappable operation types (for example, PCI) and provides information about this operation in an environment variable: Add or remove the PCI class, subclass, and programming interface object manufacturer flags and device flag object bus addresses that the object device belongs to, Slot Address and Function function number

The subsequent configuration of the nuclear process involves the following documents:

/etc/hotplug/pci.rc
/ect/hotplug/pci. Agent
/etc/rc.d/init.d/hotplug

In the kernel, the functions in the KERNEL/KMOD.C

int Call_usermodehelper (char *path, Char **argv, char **envp, int wait) to open the User state script/sbin/hotplug. In the parameter table, path represents the path of the startup's argv application, the parameter table for the application, ENVP is the list of environment variables, and wait indicates whether the synchronization waits for the application to complete and then returns the status of the execution result.

3 PCIE Hot-swappable module composition

In order to use the PCI Express Native Hotplug, we must open the corresponding function module at compile time.

In the kernel configuration, the PCIE hotplug corresponding switch is Hotplug_pci_pcie, which relies on HOTPLUG_PCI. If your motherboard supports PCI Express Native Hotplug, you can choose Y; if you just want to compile this driver as a module, then select M, which is called PCIEHP, in the source code \dirver\pci\hotplug\kconfig file, You can see:

Config Hotplug_pci_pcie
	tristate "PCI Express HOTPLUG Driver"
	depends on Hotplug_pci

The complete PCIEHP module function involves following several files pci_hotplug_core.c, pciehp_core.c,pciehp_ctrl.c,pciehp_pci.c pciehp_ HPC.C, in addition, depending on whether ACPI is enabled, including Pciehprm_acpi.c or PCIEHPRM_NONACPI.C, in the source code \dirver\pci\hotplug\makefile file, you can see:

Pci_hotplug-objs	: =	pci_hotplug_core.o
pciehp-objs		: =	pciehp_core.o	\
				pciehp_ CTRL.O	\
				pciehp_pci.o	\
				pciehp_hpc.o
ifdef config_acpi_bus pciehp-objs
	+ = Pciehprm_ ACPI.O
Else
	pciehp-objs + = PCIEHPRM_NONACPI.O
endif

The main task of the code is to load hot-plug drivers on all PCIe bridges that support hot-swappable, monitor hot-swappable events, and treat them according to type, such as hot-insert events, hot-unplug events, or power failures. Hot-swappable drive loading

The main task of hot-swappable driver loading is to open and initialize the PCIe hot-swappable kernel thread, which is in/driver/pci/hotplug/pciehp_core.c. The entry function is:

static int __init pcied_init (void)

#ifdef config_hotplug_pci_pcie_poll_event_mode
	pciehp_poll_mode = 1;
#endif
	retval = Pcie_start_thread ();
	if (retval)
		goto error_hpc_init;
	retval = Pciehprm_init (PCI);
	if (!retval) {
		retval = Pci_register_driver (&pcie_driver);
		DBG ("Pci_register_driver =%d\n", retval);
		Info (driver_desc "version:" Driver_version "\ n");
	}

The key functions involved in Pcied_init are analyzed as follows:

retval = Pcie_start_thread ();

Initialize and open notification mechanism:

Pciehp_event_start_thread () Start Event monitoring processing thread
The slot list is then initialized, and each bus in the system gives a slot list.
struct Pci_func *pciehp_slot_list[256]; are set to NULL

retval = Pciehprm_init (PCI);

Prototype: int pciehprm_init (enum Php_ctlr_type ctlr_type)

Initializing resources (distinguishing between both ACPI and non-ACPI) calls NULL function Legacy_pciehprm_init_pci () in the non-ACPI initialization mode, and in Pcihprm_nonacpi.h, defines the Irq_info,irq_ Routing_table two structures. In ACPI initialization, the search for PCI devices is traversed under the ACPI Tree by PCIEHPRM_ACPI_SCAN_PCI (). Both ACPI and non-ACPI require Php_ctlr_type to be PCI.

retval = Pci_register_driver (&pcie_driver);

Registers and initializes the PCI bridge hot-swappable driver module.

The device driver is added to the registered device driver list, and the driver remains in effect even if no corresponding device appears during the period.

	int count = 0;
	/* Initialize Common driver fields * *

Used to generalize, you can see Drv->driver as the base class information of DRV

	Drv->driver.name = drv->name;
	Drv->driver.bus = &pci_bus_type;
	Drv->driver.probe = Pci_device_probe;
	Drv->driver.remove = Pci_device_remove;
	Drv->driver.kobj.ktype = &pci_driver_kobj_type;
	Pci_init_dynids (&drv->dynids);
	Count = Driver_register (&drv->driver);

Note: Here is how to drv->driver the driver of a generic device from a base class to DRV the drive of this PCI device-using container macros

Pcie_driver is the PCIe driver object, defined in PCIEHP_CORE.C

	static struct Pci_driver Pcie_driver = {
	//driver name is defined as "PCIEHP"
. Name=	Pcie_module_name, 
//Id_ Table specifies the scope to be applied by the probe function probe, which is specified in the table as all PCI bridge devices
	. id_table	=	pcied_pci_tbl, 
//probe for the specified device probe function
	. Probe	=	pcie_probe, 
};
static struct pci_device_id pcied_pci_tbl[] = {
	//Here Select All PCI Bridges
. class =        (Pci_class_bridge_pci << 8) | 0x00),
 
	. Class_mask =	~0,
	. Vendor =       pci_any_id,.
	device =       pci_any_id,
	. Subvendor =    pci_any_id,
	. Subdevice =    pci_any_id,
	},

Device Detection Pcie_probe:

static int pcie_probe
(struct pci_dev *pdev, const struct pci_device_id *ent) in
pcie-core.c

It has been determined that Pdev is the PCI bridge device matched by Pcie_driver, and its corresponding device feature number in Pcie_driver is *ent, which can be further initialized and probed.

The specific behavior is as follows: Binding hot-swappable slots set the controller and its state registration interrupt processing function read and write configuration header, and then allocate resources, and finally to slot detection, Hook hot Plug Event monitoring processing thread

Hot-swappable drivers use asynchronous mechanisms for polling and notification of hot-swappable events, and the manipulation of hot-swappable functional components is done by reading and writing to the relevant register group. Main function relations See Figure 3: Figure 3 Socket Hot Swap event handler function relationship

The functions of the main functions in Figure 3 are described below:

A slot Event monitor thread

As the most direct information of hot plug activity, the slot event is operated by hardware and set up the relevant register group, the system software can get the event information through the timing polling or interrupt mode, and perform the corresponding event preprocessing function. Slot events are as follows: Hot-swappable command arrives in slot lock state change adapter card status change power supply failure

Php_ctlr->int_poll_timer.function = &int_poll_timeout where PHP_CTLR is the hot-swappable controller state php_ctlr_state_s type, which is defined in PCIEHP_ In Hpc.h, records the current hot-swappable controller's critical status and is used as a controller handle for HPC (Controller); hot-swappable