One of the coolest features of the AT91RM9200 we have been designing with for a couple of years now is the FIQ, or Fast Interrupt Request. This is basically the NMI of the ARM world. It is a bit difficult to get working (Milosch Meriac helped me with the initial version some years ago) because its job is to interrupt WHATEVER is happening and start running your FIQ handler within 1us or so NO EXCUSES. This is the very very hard end of hard realtime, it does what it claims but it does not respect any privacy that Linux may need as an OS and we will see that needs care. Here is a patch against 2.6.20 with the AT91RM9200 patches: it should probably apply okay to later kernels. The patch adds many comments so you probably want to read that and this at the same time. First the things you can't do, which will save you much pain from finding them out yourself.

Things you can't do with FIQ

One of the "private times" Linux needs to itself is the virtual memory pagetable management action. It shuts off all interrupts and rewrites the pagetable at intervals, and then goes on as before. That stops driver interrupts coming in and trying to do stuff while the pagetable is empty, or incomplete or just full of garbage. However FIQ ignores any claim to privacy performed by shutting off normal interrupt response. That means your FIQ ISR code can come in at a "bad time", if it tries to access memory areas mapped through the pagetable it will instead find nothing or the wrong thing or... the end result is that the FIQ ISR cannot touch any memory mapped by vmalloc. Unfortunately, when a kernel module is loaded, its various memory footprints including the module code are allocated by... yep, vmalloc. That means your FIQ ISR code cannot live in a module, it has to be part of the monolithic kernel. Finally all sorts of Linux code also wants "private time" or to guard against multiple access to objects by spinlocks or whatever. FIQ ISRs cannot play those games, it comes it in the middle of anything and has to get out quickly again too. So unless it is a simple macro, you can't use any Linux APIs in the FIQ ISR.

Things you can do with FIQ

Well reading those constraints, you're probably wondering if it is still useful. It sure is! You can touch the memory-mapped IO in the chip using the AT91RM9200 APIs. FIQ has the super power it will run your ISR within ~1us NO EXCUSES. That means you can rely on the ISR code to act like hardware, you trigger it and 1us later your programmed sequence occurs without fail. In turn that means FIQ is perfect for many hardware interfacing tasks, in particular management of low latency (small buffer) PDC DMA setup. Low latency for audio traffic for example is highly desirable, but of course if there are ANY delays setting up the next PDC DMA, you get dropouts and clicks. If you allow the FIQ to handle generation of samples and management of PDC DMA, there won't be any delays for sure, you will have perfect audio. We found that the AT91RM9200 at 180MHz can easily handle 8kHz FIQs (a common rate for telephony) with an ISR duration of ~8us, without affecting the Ethernet or USB performance.

IPC between the FIQ and kernel worlds

The general communication for FIQ ISRs with the "real world" in the patch is to define a struct in include/asm-arm/arch-at91rm9200/at91rm9200_fiq_ipc_type.h that contains all of the data that is shared between FIQ and normal kernel code. The example one looks like this: 
struct at91rm9200_fiq_ipc {
	int nCountFiqEvents;
};
 One of these structs is defined in the main part of the patch code in arch/arm/mach-at91rm9200/at91_fiq.c like this struct at91rm9200_fiq_ipc at91rm9200_fiq_ipc; EXPORT_SYMBOL(at91rm9200_fiq_ipc); That means in your other kernel code -- which can be in a module, only the FIQ ISR must be in the kernel -- you can have 
#include <arch/at91rm9200_fiq_ipc_type.h>
extern struct at91rm9200_fiq_ipc at91rm9200_fiq_ipc;
 and use the same struct to communicate with the FIQ ISR.

How to customize summary

1) Change struct at91rm9200_fiq_ipc in include/asm-arm/arch-at91rm9200/at91rm9200_fiq_ipc_type.h to have the data types you need 2) Add your FIQ ISR code to arch/arm/mach-at91rm9200/at91_fiq.c where it says "your C code goes here" 3) Import extern struct at91rm9200_fiq_ipc at91rm9200_fiq_ipc; in your own kernel module and communicate with the FIQ ISR using that.

FIQ shadowing with IRQ

Ultimately the FIQ actions are going to need to interface to Linux kernel objects sooner or later, perhaps there has to be some locking or blocking action for usermode access. But we are banned from using Linux APIs in the FIQ ISR. A powerful solution is to physically tie the FIQ signal to an IRQ input additionally. Code that is REALLY hard realtime, like the PDC DMA management, goes in FIQ, and a count of FIQs is kept. Data out of FIQ can go into a software FIFO. The less reliable IRQ watches the count of FIQs and compares it to its own count of IRQs, if it sees it has blacked out and has fallen behind, it will loop up to a certain number of times "catching up", using the data placed by FIQ in the FIFO. In this way, by splitting the code into "no excuses" realtime in the FIQ ISR, and "reliable only on average" realtime in the IRQ ISR, it is possible to bind actions in the FIQ to code in the ISR which can execute Linux APIs, and have the best of both worlds.