The xtensa-asm2 work included a patch that added nano_internal.h
includes in lots of places that needed to have _Swap defined, because
it had to break a cycle and this no longer got pulled in from the arch
headers.
Unfortunately those new includes created new and more amusing cycles
elsewhere which led to breakage on other platforms.
Break out the _Swap definition (only) into a separate header and use
that instead. Cleaner. Seems not to have any more hidden gotchas.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Simple test of spinlock semantics. Bounce between two CPUs locking
and releasing, validating that nothing changes at unexpected times.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Minimal spinlock API based on the existing atomic.h layer. Usage
works just like irq_lock(), but takes an argument to a specific struct
k_spinlock_t to un/lock. No attempt at implementing fairness or
backoff semantics. No attempt made at architecture-specific assembly.
When CONFIG_SMP is not enabled, this code falls back to a zero-size
struct and becomes functionally identical to irq_lock/unlock().
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Simply define the Kconfig variables in this patch so they can be used
in later patches. Define MP_NUM_CPUS correctly on esp32. No code
changes.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
This is a mostly-internal API to start a secondary system CPU, with an
implementation for the ESP-32 "APP" cpu. Exposed in kernel.h because
it's plausibly useful for asymmetric MP code managed by an app.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Xtensa register windows have a special exception that happens when the
stack pointer needs to be moved, but the caller function has already
spilled its registers below it.
I thought these were unexercised in Zephyr code, but they turn out to
be thrown by the existing mem_pool tests when run in the 32-register
qemu environment (but not on 64-register hardwre). Because the effect
of the exception is to unspill the caller, there is no good way to
handle this in a traditional handler. Instead put a 5-instruction
stub in front of the user exception handler (i.e. incurring that cost
on every trap and every L1 interrupt) to test before doing the normal
entry.
Works, but would be nicer to optimize this in the future so that only
true alloca exceptions take that cost.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
This macro was already available add an external symbol so C code can
access it (via CALL0 -- it's not and can't be an actual function).
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
The API allows any byte count for stack size, and tests in fact check
that a stack with a 499 byte stack works correctly. No choice, have
to do this at runtime.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
You'd this feature would be portable, but it's arch-specific.
Initialize the CONFIG_THREAD_MONITOR stuff, placing the __thread_entry
struct (which AFAICT is dead: nothing in the tree actually reads it)
at the top of the stack.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
The stack initilaization was calling the user-provided entry function
directly, which works fine until that function returns, at which point
it will try to unspill A0-A3 from the 16 bytes above the allocated
stack and then "return" to a NULL pointer.
The kernel provides a _thread_entry() function that does cleanup
properly, so use that.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
When using _arch_switch() context switching, the thread return value
is a generic hook and not provided by the architecture.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
This adds vectors for all interrupt levels defined by core-isa.h.
Modify the entry code a little bit to select correct linker sections
(levels 1, 6 and 7 get special names for... no particularly good
reason) and to constructed the interrupted PS value correctly (no EPS1
register for exceptions since they had to have interrupted level 0
code and thus differ only in the EXCM bit).
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
This python script reads the core-isa.h interrupt definitions (via
running a template file through the toolchain preprocessor to generate
an input file) and emits a fully populated, optimized C handling code
that binary searches only the declared interrupts at a given level and
correctly detects spurious interrupts (and/or incorrect core-isa.h
definitions).
The generated code, alas, turns out not to be any faster than simply
searching the interrupt mask with CLZ (er, NSAU in xtensese), though
it could be faster in theory if the compiler made different choices,
see comments. But I like this for the robustness of the fully
populated search trees and the checking of level vs. mask.
This simply commits the script output into the source tree, including
some checking code to force a build error if the toolchain changes the
headers incompatibly. It would be better long term to have these
headers be generated at build time, but that requires more cmake fu
than I have.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
The earlier xtensa layer put the timer initialization and update
directly into the interrupt handler, which is... weird. Under asm2,
it's just a regular ISR and needs to do the work in the driver.
Really, this driver needs a bunch of cleanup. The xtensa CPU timer is
two registers and one ISR: a global cycle count register, and a
compare register that will fire the IRQ when they match. There is
*way* too much code here.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
The asm2 layer will build alongside the traditional assembly, but the
reverse is not true. Add a CONFIG_XTENSA_ASM2 to force its use at
runtime and disable the older code.
Note that the older assembly had an initialization function that is
properly part of the timer driver. Move a C equivalent into the timer
driver itself for now to prevent a build breakage. Long term we need
to clean that driver up in a bunch of other ways.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Legacy xtensa had a rather complicated implementation of en/disabling
interrupts, owing to the "software priority" feature (which plays
games with INTENABLE and INTLEVEL to allow for interrupts to interrupt
each other outside their normal priorities). But that's not a Zephyr
feature, it's enabled by a XT_USE_SWPRI value that comes from platform
headers and isn't enabled on any of our boards. Dead code, basically.
Replace with the obvious implementation when asm2 is in use.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
This was a dead API. Nothing ever used it, it wasn't exposed in any
API headers. It never appeared in documentation. It's not
particularly clear why a Zephy app would want to hook
architecture-specific exceptions instead of simply using the portable
error framework anyway. And it's not supported by asm2. Delete.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
The xtensa arch code had this empty offsets.h header sitting around.
Its name collides with the autogenerated offsets.h, making it
dangerously dependent on include file path order. Seems to be benign,
but it's freaking me out. Remove.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
The existing __swap() mechanism is too high level for some
applications because of its scheduler-awareness. This introduces a
new _arch_switch() mechanism, which is a simpler primitive that looks
like:
void _arch_switch(void *handle, void **old_handle_out);
The new thread handle (typically just a stack pointer) is specified
explicitly instead of being picked up from the scheduler by
per-architecture code, and on return the "old" thread handle that got
switched out is returned through the pointer.
The new primitive (currently available only on xtensa) is selected
when CONFIG_USE_SWITCH is "y". A new C _Swap() implementation based
on this primitive is then added which operates compatibly.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
K_NUM_PRIORITIES and K_NUM_PRIO_BITMAPS were defined in
nano_internal.h, but used in only a handful of places. Move to
kernel_structs.h (somewhat higher up in the hierarchy) to help with
include file cycle-breaking. Arguably they are a better fit there
anyway.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
_Swap() is defined in nano_internal.h. Everything calls _Swap().
Pretty much nothing that called _Swap() included nano_internal.h,
expecting it to be picked up automatically through other headers (as
it happened, from the kernel arch-specific include file). A new
_Swap() is going to need some other symbols in the inline definition,
so I needed to break that cycle. Now nothing sees _Swap() defined
anymore. Put nano_internal.h everywhere it's needed.
Our kernel includes remain a big awful yucky mess. This makes things
more correct but no less ugly. Needs cleanup.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
This sample (which should eventually become a proper test) suite
builds from simple applications of the new primitives to a full
context switch test and interrupt handling suite (based on the
CPU-internal CCOMPARE2 timer).
It's been extraordinarily useful finding regressing as the asm2 code
gets modified and should probably stick around as long as possible.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
SMP needs a new context switch primitive (to disentangle _swap() from
the scheduler) and new interrupt entry behavior (to be able to take a
global spinlock on behalf of legacy drivers). The existing code is
very obtuse, and working with it led me down a long path of "this
would be so much better if..." So this is a new context and entry
framework, intended to replace the code that exists now, at least on
SMP platforms.
New features:
* The new context switch primitive is xtensa_switch(), which takes a
"new" context handle as an argument instead of getting it from the
scheduler, returns an "old" context handle through a pointer
(e.g. to save it to the old thread context), and restores the lock
state(PS register) exactly as it is at entry instead of taking it as
an argument.
* The register spill code understands wrap-around register windows and
can avoid spilling A4-A15 registers when they are unused by the
interrupted function, saving as much as 48 bytes of stack space on
the interrupted stacks.
* The "spill register windows" routine is entirely different, using a
different mechanism, and is MUCH FASTER (to the tune of almost 200
cycles). See notes in comments.
* Even better, interrupt entry can be done via a clever "cross stack
call" I worked up, meaning that the interrupted thread's registers
do not need to be spilled at all until they are naturally pushed out
by the interrupt handler or until we return from the interrupt into
a different thread. This is a big efficiency win for tiny
interrupts (e.g. timers), and a big latency win for all interrupts.
* Interrupt entry is 100% symmetric with respect to medium/high
interrupts, avoiding the problems seen with hooking high priority
interrupts with the current code (e.g. ESP-32's watchdog driver).
* Much smaller code size. No cut and paste assembly. No use of HAL
calls.
* Assumes "XEA2" interrupt architecture, the register window extension
(i.e. no CALL0 ABI), and the "high priority interrupts" extension.
Does not support the legacy processor variants for which we have no
targets. The old code has some stuff in there to support this, but
it seems bitrotten, untestable, and I'm all but certain it doesn't
work.
Note that this simply adds the primitives to the existing tree in a
form where they can be unit tested. It does not replace the existing
interrupt/exception handling or _Swap() implementation.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Xtensa has a "high priority" class of interrupt levels which ignore
the EXCM bit and can thus interrupt running exception handlers. These
can't be used for C handlers in the general case[1] because C code
needs to be able to throw window over/underflow exceptions, which are
not reentrant.
But the high priority interrupts might be useful to a carefully
designed application, or to unit tests of low level architecture code.
So make their generation optional with this kconfig option.
[1] ESP-32 has a high priority interrupt for its watchdog, apparently.
Which is sort of OK given that it never needs to return to the
interrupted code.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
The existing hand-written interrupt code is manually calling the timer
ISR, which is sort of silly and about to be replaced. Correctly
declare the ISR with IRQ_CONNECT() so that a conventional interrupt
handling implementation can find it. With current code this is a
noop.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
The new thread stack layout is as follow:
|---------------------|
| user stack |
|---------------------|
| stack guard (opt.) |
|---------------------|
| privilege stack |
-----------------------
For MPUv2
* user stack is aligned to the power of 2 of user stack size
* the stack guard is 2048 bytes
* the default size of privileg stack is 256 bytes.
For user thread, the following MPU regions are needded
* one region for user stack, no need of stack guard for user stack
* one region for stack guard when stack guard is enbaled
* regions for memory domain.
For kernel thread, the stack guard region will be at the top, adn
The user stack and privilege stack will be merged.
MPUv3 is the same as V2's layout, except no need of power of 2
alignment.
* reimplement the user mode enter function. Now it's possible for
kernel thread to drop privileg to user thread.
* add a separate entry for user thread
* bug fixes in the cleanup of regs when go to user mode
Signed-off-by: Wayne Ren <wei.ren@synopsys.com>
when USERSPACE is enabled, exception is handled in the privilege
stack of thread. This make thread context switch is possible in the
exception handler. For some case,e.g. tests, this is useful.
Signed-off-by: Wayne Ren <wei.ren@synopsys.com>
disable the U bit of irq.ctrl, so the user thread's context will
be saved into privilege stack when interrupts/exception come.
Signed-off-by: Wayne Ren <wei.ren@synopsys.com>
The application memory area has a requirement of address alignment,
especially when MPU requires power of 2.
Modify the linker tmemplate to apply application memory address
alignment generation
Signed-off-by: Wayne Ren <wei.ren@synopsys.com>
scrub all the regs of kernel context before returnning to userspace.
For sys call, ro is not cleared as it's a return value of sys call.
Signed-off-by: Wayne Ren <wei.ren@synopsys.com>
Enable us bit to check user mode more efficienly.
US is read as zero in user mode. This will allow use mode sleep
instructions, and it enables a form of denial-of-service attack
by putting the processor in sleep mode, but since interrupt
level/mask can't be set from user space that's not worse than
executing a loop without yielding.
Signed-off-by: Wayne Ren <wei.ren@synopsys.com>
* user space support requires THREAD_INFO
* for MPU version 2, the stack align is at least 2048 bytes
* the smallest mpu region is 2048 bytes
* the region size must bt power of 2
* the start address of region must be aligned to the region size
Signed-off-by: Wayne Ren <wei.ren@synopsys.com>
* add the implementation of syscall
* based on 'trap_s' intruction, id = 3
* add the privilege stack
* the privilege stack is allocted with thread stack
* for the kernel thread, the privilege stack is also a
part of thread stack, the start of stack can be configured
as stack guard
* for the user thread, no stack guard, when the user stack is
overflow, it will fall into kernel memory area which requires
kernel privilege, privilege violation will be raised
* modify the linker template and add MPU_ADDR_ALIGN
* add user space corresponding codes in mpu
* the user sp aux reg will be part of thread context
* When user thread is interruptted for the 1st time, the context is
saved in user stack (U bit of IRQ_CTLR is set to 1). When nest
interrupt comes, the context is saved in thread's privilege stack
* the arc_mpu_regions.c is moved to board folder, as it's board
specific
* the above codes have been tested through tests/kernel/mem_protect/
userspace for MPU version 2
Signed-off-by: Wayne Ren <wei.ren@synopsys.com>
Fixed incorrect comparison of FLASH_SLOT in microseconds
with ticks elapsed.
This caused Bluetooth controller to try scheduling events
beyond the acceptable soft real-time design limits, due to
the CPU halted for flash erase operation.
Fixes#6229.
Signed-off-by: Vinayak Kariappa Chettimada <vich@nordicsemi.no>
Added missing HCI Supported Commands bit fields for PHY
Update feature. Also, refactored with missing conditional
compilation for other bit fields.
Signed-off-by: Vinayak Kariappa Chettimada <vich@nordicsemi.no>
Revert incorrect calculation introduced in
commit ec5a787da2 ("Bluetooth: controller: Fix multiple
master role event scheduling") and revert a related
incorrect fix in commit a02606cbf9 ("Bluetooth:
controller: Fix missing ticks to us conversion").
Fixes the controller assert in ctrl.c line number 1477. A
64-bit arithmetic took ~35 us in Radio ISR for nRF51 causing
the ISR to take too much time before packet buffer could be
set.
Also, fixed master scheduling by correctly accounting for
the jitter between each master event.
Relates to: #5486
Signed-off-by: Vinayak Kariappa Chettimada <vich@nordicsemi.no>
Fix incorrect calculation of unreserved timespace which
should take into account the ticks_slot of current event,
and compensation for any reduced prepare in the current
event as well as in the next event.
This regression relates to the commit ad7c9d3d76
("Bluetooth: controller: Improved continuous scanning")
Signed-off-by: Vinayak Kariappa Chettimada <vich@nordicsemi.no>
Define a macro HAL_TIMER_SIGN_BIT to correspond to the most
significant bit support by counter hardware used by ticker.
Signed-off-by: Vinayak Kariappa Chettimada <vich@nordicsemi.no>
