From 9a3ee641b610f52619f4aa718ef40bc179337356 Mon Sep 17 00:00:00 2001 From: Alexey Brodkin Date: Thu, 29 Aug 2019 23:18:37 +0300 Subject: [PATCH] arc: interrupts: Explain return from interrupt to cooperative thread The code in question is very non-trivial so without good explanation it takes a lot of time to realize what's done there and why it still works in the end. Here I'm trying to save a couple of man-days for the next developers who's going to touch that piece of code. Signed-off-by: Alexey Brodkin --- arch/arc/core/regular_irq.S | 168 ++++++++++++++++++++++++++++++++++++ 1 file changed, 168 insertions(+) diff --git a/arch/arc/core/regular_irq.S b/arch/arc/core/regular_irq.S index 8d9a6f83469..8cd6b4dede1 100644 --- a/arch/arc/core/regular_irq.S +++ b/arch/arc/core/regular_irq.S @@ -35,6 +35,168 @@ GTEXT(_rirq_common_interrupt_swap) * TODO: Revist this if FIRQ becomes configurable. */ +/* + +=========================================================== + RETURN FROM INTERRUPT TO COOPERATIVE THREAD +=========================================================== + +That's a special case because: + 1. We return from IRQ handler to a cooperative thread + 2. During IRQ handling context switch did happen + 3. Returning to a thread which previously gave control + to another thread because of: + - Calling k_sleep() + - Explicitly yielding + - Bumping into locked sync primitive etc + +What (3) means is before passing control to another thread our thread +in question: + a. Stashed all precious caller-saved registers on its stack + b. Pushed return address to the top of the stack as well + +That's how thread's stack looks like right before jumping to another thread: +----------------------------->8--------------------------------- +PRE-CONTEXT-SWITCH STACK + + lower_addr, let's say: 0x1000 + + -------------------------------------- + SP -> | Return address; PC (Program Counter), in fact value taken from + | BLINK register in z_arch_switch() + -------------------------------------- + | STATUS32 value, we explicitly save it here for later usage, read-on + -------------------------------------- + | Caller-saved registers: some of R0-R12 + -------------------------------------- + |... + |... + + higher_addr, let's say: 0x2000 +----------------------------->8--------------------------------- + +When context gets switched the kernel saves callee-saved registers in the +thread's stack right on top of pre-switch contents so that's what we have: +----------------------------->8--------------------------------- +POST-CONTEXT-SWITCH STACK + + lower_addr, let's say: 0x1000 + + -------------------------------------- +SP -> | Callee-saved registers: see struct _callee_saved_stack{} + | |- R13 + | |- R14 + | | ... + | \- FP + | ... + -------------------------------------- + | Return address; PC (Program Counter) + -------------------------------------- + | STATUS32 value + -------------------------------------- + | Caller-saved registers: some of R0-R12 + -------------------------------------- + |... + |... + + higher_addr, let's say: 0x2000 +----------------------------->8--------------------------------- + +So how do we return in such a complex scenario. + +First we restore callee-saved regs with help of _load_callee_saved_regs(). +Now we're back to PRE-CONTEXT-SWITCH STACK (see above). + +Logically our next step is to load return address from the top of the stack +and jump to that address to continue execution of the desired thread, but +we're still in interrupt handling mode and the only way to return to normal +execution mode is to execute "rtie" instruction. And here we need to deal +with peculiarities of return from IRQ on ARCv2 cores. + +Instead of simple jump to a return address stored in the tip of thread's stack +(with subsequent interrupt enable) ARCv2 core additionally automatically +restores some registers from stack. Most important ones are +PC ("Program Counter") which holds address of the next instruction to execute +and STATUS32 which holds imortant flags including global interrupt enable, +zero, carry etc. + +To make things worse depending on ARC core configuration and run-time setup +of certain features different set of registers will be restored. + +Typically those same registers are automatically saved on stack on entry to +an interrupt, but remember we're returning to the thread which was +not interrupted by interrupt and so on its stack there're no automatically +saved registers, still inevitably on RTIE execution register restoration +will happen. So if we do nothing special we'll end-up with that: +----------------------------->8--------------------------------- + lower_addr, let's say: 0x1000 + + -------------------------------------- + # | Return address; PC (Program Counter) + | -------------------------------------- + | | STATUS32 value + | -------------------------------------- + | + sizeof(_irq_stack_frame) + | + | | Caller-saved registers: R0-R12 + V -------------------------------------- + |... + SP -> | < Some data on thread's stack> + |... + + higher_addr, let's say: 0x2000 +----------------------------->8--------------------------------- + +I.e. we'll go much deeper down the stack over needed return address, read +some value from unexpected location in stack and will try to jump there. +Nobody knows were we end-up then. + +To work-around that problem we need to mimic existance of IRQ stack frame +of which we really only need return address obviously to return where we +need to. For that we just shift SP so that it points sizeof(_irq_stack_frame) +above like that: +----------------------------->8--------------------------------- + lower_addr, let's say: 0x1000 + + SP -> | + A | < Some unrelated data > + | | + | + sizeof(_irq_stack_frame) + | + | -------------------------------------- + | | Return address; PC (Program Counter) + | -------------------------------------- + # | STATUS32 value + -------------------------------------- + | Caller-saved registers: R0-R12 + -------------------------------------- + |... + | < Some data on thread's stack> + |... + + higher_addr, let's say: 0x2000 +----------------------------->8--------------------------------- + +Indeed R0-R13 "restored" from IRQ stack frame will contain garbage but +it makes no difference because we're returning to execution of code as if +we're returning from yet another function call and so we will restore +all needed registers from the stack. + +One other important remark here is R13. + +CPU hardware automatically save/restore registers in pairs and since we +wanted to save/restore R12 in IRQ stack frame as a caller-saved register we +just happen to do that for R13 as well. But given compiler treats it as +a callee-saved register we save/restore it separately in _callee_saved_stack +structure. And when we restore callee-saved registers from stack we among +other registers recover R13. But later on return from IRQ with RTIE +instruction, R13 will be "restored" again from fake IRQ stack frame and +if we don't copy correct R13 value to fake IRQ stack frame R13 value +will be corrupted. + +*/ /** * @@ -207,6 +369,12 @@ _rirq_return_from_coop: bset r0, r0, _ARC_V2_SEC_STAT_IRM_BIT sflag r0 #endif + + /* + * See verbose explanation of + * RETURN FROM INTERRUPT TO COOPERATIVE THREAD above + */ + /* carve fake stack */ sub sp, sp, ___isf_t_pc_OFFSET