Life with AdeosPhilippe gerumrevision bcopyright? 2005Copyright? 2005 Philippe Gerum
License reproduction, distribution and/or modification of this document is granted subject to the release of the GNU Free Documentation License 1.2 or later released by the FSF. There are no fixed paragraphs, cover text and back text. A copy of the license is published on gnu.org: "GNU Free Documentation License" [http://www.gnu.org/licenses/fdl.html]. 10/9/2005
SummaryThis document is an overview of ADEOS technology and the support it provides for Xenomai. After addressing the event pipeline, the implementation of interrupt distribution, and the propagation of system events, we will learn about the interaction between Xenomai and ADEOS. We will explain the concepts of primary and secondary domains and we are also interested in how interrupts are spread. We will conclude by summarizing some tips and tricks for enabling/disabling interrupt sources and sharing interrupts between domains.Catalogue
1. What is ADEOS 1.1. Event Pipeline 1.2. Optimistic interrupt protection 1.3. System Event Propagation 2. What ADEOS provides for the Xenomai 2.1. Xenomai Primary and secondary domain 2.2. System call theft 2.3. Interrupt Propagation 3. Tips and Tricks 3.1. Enable/disable interrupt source 3.2. Sharing interrupts between domains 3.3. Interrupt sharing and latency 3.4. Conclusion 3.5. Link1. What is AdeosADEOS is a virtual layer of resources as a Linux kernel patch, and its universal design was proposed by Karim Yaghmour in a technical document as early as 2001. The current incarnation of this proposal is that it is a catalyst for a simple and efficient real-time system. It provides a way to run a regular gnu/linux environment and an RTOS to execute concurrently on the same hardware. To do this, ADEOS enables multiple entities-called domains-to exist on the same machine at the same time. These fields do not need to see each other, but they can all see ADEOS. A domain is likely to be a complete operating system, but there is no assumption to consider the complexity of the domain. However, all domains can contend with external events (such as interrupts) or internal (such as traps, exceptions), based on the system-wide priorities they have acquired. In addition, its simple virtualization capability, another key ADEOS advantage is its ability to export a common API to the client domain, a CPU-independent architecture. Therefore, many if not most of the migration work that occurs in the customer domain occurs at the ADEOS layer.
1.1. Event PipelineIt is important to keep in mind that the ADEOS basic structure is the customer domain solicit event control chain. A domain is a kernel software component that informs the ADEOS layer what is notified:
- Each incoming external interrupt or auto-generated virtual interrupt;
- A system call generated by each Linux application;
- Other system events triggered by kernel code (e.g., linux task switch, signal notification, Linux task exit, etc.)
ADEOS ensures that events are distributed to each client domain in a methodical manner based on their respective priorities in the system, so it is possible to provide timely and predictable distribution of these events. This prioritization strictly defines the order in which events are distributed to the domain. All active domains are queued according to their respective priority, forming the pipeline abstraction used by Adeos to form an event stream from the highest priority to the lowest priority domain. Incoming events (including interrupts) are pushed to the head of the pipeline (that is, the highest priority domain) to its tail (that is, the lowest priority domain). A generic attempt is given for a ADEOS-based system where multiple domains abstract shared events through pipelines:
Event pipeline in, the Linux kernel on the pipeline location can be arbitrary, that is, the Linux kernel has a special role because it represents the root domain, because all other domains need Linux to install them, usually means to load kernel modules.
1.2. Optimistic interruption protectionIn order to distribute interrupts in a prioritized manner while still allowing the domain to create a non-disruptive part, ADEOS implements the so-called optimistic interrupt protection model described by Stodolsky, Chen, and Bershad. In this essay. [Http://citeseer.nj.nec.com/stodolsky93fast.html] The stage in which any given domain occupies a pipeline can be postponed, This means that the next arriving interrupt will not be distributed to the domain's processor and will be blocked from streaming to the lowest priority domain in the same way. When a phase is postponed, the paused interrupt accumulates in the domain interrupt log and is eventually processed by an internal operation called synchronization when this phase is no longer deferred. Domains use this feature to protect their own critical areas from being handled by their own disruptive preemption. However, thanks to the disruption control virtualization that ADEOS brings, a higher priority domain can still receive interrupts and eventually preempt any low-priority domains. In fact, this means that even if a ADEOS-based Linux kernel often postpones its own phase to handle critical-area operations, an early-running real-time system can still receive interrupts at any time in the pipeline, without any delay. When a domain finishes processing the pending interrupt it receives, it calls a special ADEOS service to make the CPU to the next domain of the pipeline, so the latter can sequentially process the pending event that wakes it up and the cycle continues to the lowest priority of the pipeline. Illustrates how multiple domains running on multiple CPUs can share an incoming interrupt through the ADEOS pipeline abstraction. Of course, when a stage is postponed, the interrupt must be correctly marked to be resolved: this is achieved through a per-domain, per-CPU interrupt log, as explained below:
Optimistic interrupt protection model
1.3. System Time PropagationInterrupts are not the only events that can flow through the pipeline abstraction, internal events triggered by the Linux kernel itself, or so-called system events generated by its applications. Fundamentally, a system event is a synchronous notification of traps, exceptions, or actions of some Linux cores, and notifies any part of interest through a pipeline. Since the nature of these events is synchronous, there is no way to postpone their awakening by delaying the operation simply because you cannot delay their processing. The rationale for this design decision stems from the fact that code triggering a system event may simply not be able to proceed with an intervention that is not responding to the handler: for example, the fault of the pages handler should be executed immediately after the memory addressing exception, delaying it without any meaning. In other words, deferred/non-deferred operations on a given domain only care about interrupts, whether real or virtual.2. What does Adeos provide for Xenomai? On the contrary, the question can also be: what is the basic guarantee of the real-time service that Xenomai needs to provide? The answer is straightforward: Before the Linux kernel has a chance to notify them, it must be allowed to handle all incoming interrupts first, and must have the ability to handle them immediately regardless of the current process attempting to lock them with a CPU interrupt mask from the Linux kernel. It must also ensure that the appropriate threads are always executed with priority management regardless of their current execution domain. These guarantees give xenomai predictable interrupt latency at the lowest microsecond level, no matter what activity Linux is undergoing, and once the Xenomai fast co-scheduling technique (i.e., the shadow thread) is added to the Llinux task, the scheduling delay determined by the real-time thread is provided. Illustrates the position of the adoes layer in the Xenomai architecture: Adeos position in the Xenomai architecture you will notice that the ADEOS interface is directly exposed to the hardware abstraction layer which forms the basis of the Xenomai kernel. Therefore, most requests for ADEOS services are resolved from the HAL layer, and its implementation can be found in the relevant Arch/<archname>/hal directory, and the generic bits can be obtained under Arch/generic/hal. A look at the latter is the best way to understand how Xenomai uses Adeos for his own purposes.
2.1. Primary and secondary domains of Xenomai xenomai allows real-time threads to run in strict kernel space or in the Linux thread address space. In the remainder of the article, we refer to the latter as Xenomai threads, which are not confused with regular Linux tasks (even if they belong to the Sched_fifo Class). All threads managed by Xenomai can be learned from real-time nucleus. support for real-time thread exclusive running in kernel space is just a memory of a common nuclear age, when real-time applications can only run in embedded kernel modules until the real-time support of the user space is reached, and these features Xenomai still retain the primary purpose of supporting deferred applications , there is no discussion here. What's more interesting is that Xenomai has a symbiotic approach to Linux; For example, this makes it different from the implementation of RTAI/LXRT. For this purpose, the Xenomai thread can not only run in the context of the highest priority domain (i.e., the primary domain) of the pipeline as the kernel-based Xenomai thread, but can also run in regular Linux space (i.e., the secondary domain), even though a higher scheduling delay is still xenomai as real-time. In Xenomai terminology, the former is called running in the primary execution mode, while the latter is in secondary execution mode. In order to provide full real-time support for threads running in a secondary domain, Xenomai needs to implement the following points:
- The public priority model. Only dispatch, we need a way to make real-time kernel and Linux kernels share the same priority model relative to thread-sharing control. In other words, a xenomai thread needs to be its own priority at any time in strict force, regardless of its current domain, in all xenomai threads. The Xenomai uses a technique called a variable priority of the root thread that automatically inherits the priority of the Xenomai thread controlled by the real-time kernel, which happens when entering the secondary domain. In fact, this means that the xenomai thread that is currently running in the primary domain is not required to preempt the threads running in the secondary domain unless they have a higher effective priority. For example, the above behavior is in contrast to RTAI/LXRT, where migrating to Linux space in RTAI/LXRT will in fact lose their real-time finite levels, implemented by inheriting the lowest-priority defined by the Rtai scheduler. That is, the regular Linux task Force Xenomai knows nothing about it, it only happens in classes that belong to Sched_fifo, and is always preempted when competing with Xenomai threads from the primary domain, even if they are still competing with Xenomai threads running in the secondary domain.
- The predictability of the execution time of the program. When a Xenomai thread runs in a Linux (i.e., secondary) domain, its timing should not be stirred by non-real-time Linux interrupt activity, regardless of the execution kernel or application code, usually by any low-priority asynchronous asynchronous activity that occurs at the kernel level. A simple way to prevent the latter from occurring is to keep the Linux kernel from breaking hungry when the Xenomai thread runs in the Linux domain, so there is no deferred processing that triggers the upper half of the interrupt processing during this time period. The way to make the Linux kernel suffer from starvation is to block them when they need an intermediate ADEOS domain internally, sitting between interrupts that are occupied by the real-time kernel and Linux cores, and called interrupt masking in the Xenomai terminology. Whenever the Xenomai thread is called in the Linux kernel space, the mask will be occupied and different in other cases. It is important to note that masking support can be enabled/disabled on a per-thread basis, or on a system-wide basis when xenomai is built; By default, Xenomai is disabled and not built-in.
- Fine-grained Linux kernel. To get the best out of the secondary execution pattern, we need the Linux kernel to exhibit the shortest possible non-preemption, so the chance that the Xenomai thread will be re-dispatched after the secondary domain becomes ready to run is caught as soon as possible. In addition, this ensures that the Xenomai thread can migrate from the primary domain to the secondary domain in a short and time bound period, because this operation involves reaching a kernel re-dispatch point. Because of this,
2.2. System Call Theft
Since the Xenomai real-time API (i.e., skins) can accumulate on xenomai nucleus and can export their own set of services to the user space Xenomai thread, there must be a way to separate the appropriate handlers for the system calls that correspond to the The regular system calls are all together. Xenomai intercepts every xenomai thread that needs to handle the system call traps/exceptions for Xenomai or Linux domains. This is implemented using the appropriate ADEOS subscription event handler. Xenomai uses this ability to:
(1) Distributing the real-time service from the application request to the appropriate system call handler;
(2) Ensure that each system call executes in its respective domain, whether Xenomai or Linux.
2.3. Interrupt propagation
Because real-time nucleus is at the top of the pipeline, when there is an interest in the interrupt to be temporarily in the Xenomai domain the real-time nucleus will be notified first, after the interrupt is processed, nucleus will mark the interrupt and pass it to the pipeline, if necessary, and eventually pass it to the Linux kernel domain. When the resulting interrupt wakes up, the real-time nucleus is re-dispatched after the external interrupt handler returns (in case of a broken heap) and translates to the highest priority of the running thread it controls.
When no real-time activity is blocked, the Xenomai domain gives control of the CPU to the interrupt shield domain. This means that when the Xenomai domain is idle, the CPU is freed, and when Xenomai has event handling, the CPU is preempted. For interrupts to pass through the pipeline, the ADEOS has two propagation modes: implicit mode and display mode. Implicit mode is interrupted by automatic propagation of the system, and the display mode requires manual propagation of interrupts.
3. Tips and Tricks
3.1 Enable/disable interrupt sourceIn addition to being able to completely delay a domain so that no more interrupts can pass through it until it is explicitly revoked by delay, ADEOS allows the hardware layer to selectively disable/re-enable the actual source of interrupts. After taking over the box, ADEOS handles all the domain's interrupt-forbidden requests, including the Linux kernel and the real-time kernel. This means that the interrupt request source is disabled at the hardware PIC layer and the distribution of any interrupts from this interrupt source to the current domain at the pipeline layer is locked. Conversely, enabling interrupts means reactivating the source of the PIC layer's interrupts and allowing further distribution from this source to the current domain. Therefore, a domain that wants to enable a single interrupt source must be the same as the domain that prohibits the source of the interrupt, because such operations are domain independent. In fact, this means that, when paired, the Rthal_irq_disable () and rthal_irq_enable () services integrate the calls that make up the xenomai underlying real-time HAL-related ADEOS, which must be resolved by the same ADEOS domain. For example, if a real-time interrupt handler is connected with a rthal_irq_request () service to an interrupt source, disabling the interrupt source using rthal_irq_disable (), the source will be blocked by the Xenomai domain until the same interrupt source Rthal_irq _enable () is called by the same domain. Failure to handle this request will result in permanent loss of the affected interrupt channel.
3.2. Sharing interrupts between domainsA typical example of misuse of a ADEOS pipeline when sharing hardware interrupts between domains is as follows:
void Realtime_eth_handler (unsigned irq, void *cookie) {/* * This interrupt handler have been install Ed using * Rthal_irq_request (), so it'll always have invoked on behalf of * the Xenomai (primary) domain . */rthal_irq_disable (IRQ); /* The Xenomai domain won ' t receive this IRQ anymore */rthal_irq_host_pend (IRQ); /* This IRQ have been marked as pending for Linux */}void linux_eth_handler (int irq, void *dev_id, struct pt_regs *re GS) {/* * This interrupt handler have been installed using * rthal_irq_host_request (), so it'll always be invo Ked on * behalf of the Linux (secondary) domain, as a shared interrupt * handler (linux-wise). */rthal_irq_enable (IRQ); /* * Bug:this won ' t work as expected:we-unlocking the * interrupt source for the Linux domain which is Current here, * isn't for the Xenomai domain! */
In this non-working example above, because Xenomai always uses the display propagation mode for all intercepted interrupts, the next Ethernet will be marked as blocked in the Xenomai log, waiting for the Xenomai handler to manually propagate it to Linux. But because the interrupts are still locked in the pipeline layer (don't forget that no one actually xenomai the desired rthal_irq_enable () from the Xenomai domain), this is not going to happen because the Xenomai handler will not run until the lock has been removed. So, let's celebrate. Fortunately, there is a way to keep the interrupt source disabled until the final processing ends (for example, interrupt processing level is one of the problems) for proper sharing interrupts: in fact, you don't need to do anything because Adeos has blocked any interrupts coming in from the PIC layer before passing it to the pipeline. Therefore, you just need to deal with the interrupts you see that are appropriate for the relevant domain handler, and determine when the last one uses rthal_irq_enable () to re-enable the interrupt source. Whenever the Linux kernel is one of those receivers, the regular kernel handler is automatically re-enabled, so basically you just have to worry about calling Rthal_irq_enable () in the handler, and this function will not propagate incoming interrupts to the Linux kernel. Especially on x86 architectures, the ADEOS received clock interrupts will not be masked due to performance reasons. This means that the interrupt source will not be disabled in any way you want, so this is not a problem.
3.3. Interrupt Sharing and latencyHowever, when propagating through the entire pipeline remains shielded an interrupt source may increase latency. There is no need to disable the interrupt source in the Xenomai handler because the ADEOS guarantees that there is no stack overflow occurring due to interrupts accumulating on any domain, and because it delays the current phase before departing the interrupt handler. Instead, you might even want to re-enable it so that an impending interrupt can be immediately recorded and processed immediately after the current handler call returns. So, the workaround is to rewrite the previous example in this way, as follows:
void Realtime_eth_handler (unsigned irq, void *cookie) { rthal_irq_enable (IRQ); Rthal_irq_host_pend (IRQ); /* This IRQ have been marked as pending for Linux */}void linux_eth_handler (int irq, void *dev_id, struct pt_regs *regs) {
4. ConclusionADEOS is a fairly simple piece of code that contains a lot of interesting properties if used properly. The backbone of the ADEOS model is the time pipeline, which is why it provides all the important features we need in Xenomai:
- Predictable interrupt latency;
- Precise interrupt Virtualization control (per domain and per interrupt handler registration, interrupt masking per domain and per CPU);
- Unified, priority and domain-oriented event propagation model;
- A common and simple API to simplify porting of customer code.
Xenomai uses these features to seek the maximum possible integration of real-time services brought by the Linux kernel. The primary mode of Xenomai provides true real-time performance in the lowest microsecond latency. In addition, Xenomai in the future evolution of Linux to improve the overall granularity of the kernel, such as preempt_rt, so the secondary mode is still in the sense of real-time, even the worst-case delay is measurable. That's why Xenomai worked hard from day one to achieve a level of integration with the Linux kernel. Consider symbiosis and seek common benefit.5. Links
- Karim Yaghmour ' s Adeos proposal: Http://www.opersys.com/ftp/pub/Adeos/adeos.pdf
- Optimistic interrupt protection: http://citeseer.nj.nec.com/stodolsky93fast.html
- ADEOS working directory on GNA: https://gna.org/projects/adeos/
- Download the latest Adeos Patch: http://download.gna.org/adeos/patches/
- Adeos API Reference Manual: Http://home.gna.org/adeos/doc/api/globals.html
Life with Adeos
