PCI Interrupt Asynchronous problem

The recent discovery that due to PCI data transmission error caused IO request to get the wrong information, this phenomenon let us think, why the system has been found PCI errors, IO Request can also be the right end? According to the usual thinking, PCI transfer error, IO request should be fail off, but the fact is not the case. Does this problem have anything to do with PCI interrupts Asynchrony? So here's a detailed discussion of PCI's interrupt asynchronous problem and PCI solution.

PCI has a variety of data transmission modes, post, Non-post, delayed and split mode, in order to improve transmission efficiency, the full use of bus bandwidth resources, PCI in the memory write usually using the Post method. That is, the PCI device through the DMA controller to write data to the memory, the first data submitted to bridge FIFO, and then immediately release the level of the bus resources, complete memory write operations. But in fact at this point the data has not yet reached memory, which is a typical asynchronous write operation behavior.

As shown in the figure above, device 2 writes data to memory through a DMA controller, and when DMA posts data to bridge 2, it releases BUS1, completes the DMA operation, and submits a DMA completion interrupt request to the processor. The driver on the processor side calls the Interrupt service program after receiving the interrupt request, but the data written by device 2 may not have reached memory yet, which is the "interrupt async" problem with the PCI bus.

How does the computer system solve this interrupting asynchronous problem? The key is the "execute order" defined in the PCI protocol specification. For traditional PCI buses, the order of execution of PCI requests needs to be strictly adhered to. In the above problem, after entering the driver, you need to get the interrupt status register content of the corresponding device, this is a read operation, this read operation will post in Bridge 2, the main bridge of data finally write to memory. This is the data consistency principle that the PCI bus specification adheres to.

As shown in the figure above, the device driver sends the Interrupt status register read request to device 2 after entering the driver at the processor. In PCI, the read operation can not be done by post, only non-post, delayed, or split mode can be used. Read request to reach the main bridge or Bridge 2 o'clock, if there is a post to the device 2 data, then the main bridge or Bridge 2 will eventually refresh the post data to device 2, so as to ensure the subsequent read requests data consistency. In the case of data return (Blue Line path), Bridge 2 and the main bridge also need to first write the data post there to memory, and then return to request data, so as to ensure data consistency. However, to analyze here, we encountered a problem of bus interlock: A typical bus interlock problem is that the red path will not be able to get the bus control and write the data to memory because the read request may be non-post, and the blue path waits for the red path to complete before it can continue. How does the PCI bus handle the above problems?

In the PCI protocol specification, post write bus transactions can traverse non-post bus transactions. For example, the main bridge waits for the Non-post bus transaction, at which point the Bridge 2 can still write the data post to the main bridge. This kind of processing method can avoid the above bus interlock problem.

Through the analysis, we can know through the Interrupt Service program read operation to the data in the post to bridge to the memory direction, when the driver Access DMA memory, the data must have arrived in memory. The execution order of PCI bus ensures the consistency of data.

Here we assume that the PCI transport problem occurs when the bridge 2 or the main bridge writes the post data to memory, and the resulting error is sent to the processor in the form of a break. If the system does not process this interrupt, the data obtained from DMA memory will have a corruption problem and the upper software is not known.

This article is from the "Save the Way" blog, please be sure to keep this source http://alanwu.blog.51cto.com/3652632/1073404

See more highlights of this column: http://www.bianceng.cn/Network/wlyy/