Add semihosting support for Xtensa architecture.
Existing semihosting instructions are based on ARM, so they are
converted to Xtensa codes before the semihosting call is invoked.
Return codes of read, write and seek calls had to be converted to match
semihosting API definitions.
Signed-off-by: Tahsin Mutlugun <Tahsin.Mutlugun@analog.com>
When calling device_get_binding(NULL) from userspace, this eventually
funnels down to a call to arch_user_string_nlen() where it tried to
verify that the kernel has access to this address (0x0). But since
this originates from userspace, we really want to know if this is
accessible from userspace, so using arch_buffer_validate() instead
of xtensa_mem_kernel_has_access() is preferable.
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
This fixes GDB backtracing by forcibly spilling all registers,
and faking values for WINDOWSTART and WINDOWBASE. This is
effectively telling GDB that only A0-A3 and AR0-AR3 contain
active data and other physical registers do not. GDB then must
rely on spilled values on stack. Otherwise, GDB will try to
look at all AR* registers for previous frame(s). Since we
do not save all AR* register values, there is nothing for GDB
to look at, and thus failing to unwind stack.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Inside copy_to_ctx(), we are typecasting the stack pointer into
a 32-bit array pointer, and there was unbalanced const between
two sides. Since we should not be modifying anything inside
that array, add const to the 32-bit array pointer too. So now
the compiler will not complain about discarding the const
qualifier.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
The calculation of number of high registers is not entirely
correct. We need to get past the pointer to BSA in the stack
frame before reaching the high registers. The location address
difference between the BSA and start of high registers then
can be used to calculate how many high registers in the stack
frame. So correct the start location of high registers in
the calculation as it was incorrect before. Though the result
would be the same as further divisions would mask this error.
However, it is better to correct this for readability.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
With experience it becomes clear, that failing to resolve symbols
during the linking process is likely fatal for the module loading and
a simple warning isn't enough. Fail loading instead.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
This adds a new kconfig and corresponding code to allow flushing
auto-refill data TLBs when page tables are swapped (e.g. during
context switching). This is mainly used to avoid multi-hit TLB
exception raised by certain memory access pattern. If memory is
only marked for user mode access but not inside a memory domain,
accessing that page in kernel mode would result in a TLB being
filled with kernel ASID. When going back into user mode, access
to the memory would result in another TLB being filled with
the user mode ASID. Now there are two entries on the same memory
page, and the multi-hit TLB exception will be raised if that
memory page is accessed. This type of access is better served
using memory partition and memory domain to share data. However,
this type of access is not prohibited but highly discouraged.
Wrapping the code in kconfig is simply because of the execution
penalty as there will be unnecessary TLB refilling being done.
So only enable this if necessary.
Fixes#88772
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Add missing curly braces in if/while/for statements.
This is a style guideline we have that was not enforced in CI. All
issues fixed here were detected by sonarqube SCA.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Building gdbstub for xtensa is failing currently with multiple
failures like
arch/xtensa/core/gdbstub.c:432:24: error: invalid operands to \
binary - (have 'int *' and 'const struct arch_esf *')
432 | if ((int *)bsa - stack > 4) {
Fix them by using appropriate pointer types.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
Xtensa arch layer has some custom compilation commands to
generate the interrupt dispatchers and the core-isa* files.
However, the include path to find core-isa.h does not work
for Espressif ESP32. So update the mechanism to use correct
path pointing to Espressif HAL when targeting ESP32 family
SoCs.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
When looking for jump address in the syscall table, we need to
multiply the syscall ID by 4 before adding the address offset
of the beginning of the table. This is due to the jump address
being 32-bit (4 bytes). Instead of using two instructions to
shift the ID by 4 first and then the addition, we can use one
ADDX4 instruction to achieve the same result.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Since the necessary register values are now pre-computed and
stored in the memory domain struct, we can use them directly
in various assembly locations, thus replacing the function
call to xtensa_swap_update_page_tables().
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
When context switching and dealing with non-nested interrupts,
the context to be restored are saved in the thread stack.
When userspace is enabled, this means saving context into
the user stacks for user threads. This allows PS values to be
manipulated externally by setting PS.RING in the saved PS
value to 0, resulting in granting kernel access privilege when
the thread is restored. To prevent this, we store the PS value
into the thread struct instead, where user threads cannot
manipulate that. Note that nested interrupts and syscalls are
not using the user stack but the interrupt stack and thread
privileged stack respectively, where they are not accessible
under user mode.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
When syscall assembly is executed, the EPC points to the syscall
instruction, and we have to manually advance it so we will
return to the instruction after syscall to continue execution.
However, with zero-overhead loops and the syscall instruction is
the last instruction, this simple addition does not work as it
would point past the loop and would have skipped the loop.
Because of this, syscall entrance would need to look at the loop
registers and set the PC back to the beginning of loop if we are
still looping. Assuming most of the syscalls are not inside
loops, the extra handling code consumes quite a few cycles.
To workaround this, simply adds a nop after syscall so we no
longer have to deal with loops at syscall entrance, and that
a nop is faster than all the code to manipulate loop registers.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This extends arch_cohere_stacks() to handle privileged stacks of
user threads when userspace is enabled.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Towards the end of interrupt handling, and before restoring
context, we would spill all register windows. This requires
A0 and A1 to be restored from the saved context so spilling
would work correct. However, when coherence is enabled,
window spilling has already been done earlier so there is
no need to spill the register windows again. So there is
no need to restore A0 and A1. They will be restored again
before returning from interrupt anyway.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Instead of computing all the needed register values when
swapping page tables, we can pre-compute those values when
the memory domain is first initialized. Should save some
time during context switching.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
There is no need for ZSR_FLUSH when threads are pin only
(CONFIG_SCHED_CPU_MASK_PIN_ONLY=y), so there is no need to
reserve it.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
In xtensa_userspace_enter, we were hard-coding to use level 2
interrupt return mechanism to pivot to user mode and start
running the user thread. However, EPC2 and EPS2 may be used
for other purposes, and they could be used for interrupt
return if there are only two level interrupts. So change
the userspace enter to use ZSR_RFI_LEVEL, ZSR_EPC and ZSR_EPS
instead to be more explicit.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
When returning from syscall, we cannot use RFE by using EPC1.
As there is no EPS1, we would need to write to PS before
returning. However, this creates a situation where interrupts
are being enabled (since PS is set), and any interrupts will
overwrite EPC1 before we return (which ensures chaos as we
would be returning to the wrong address). So utilize the same
mechanism as interrupt returning by use ZSR_EPS, ZSR_EPC and
ZSR_RFI_LEVEL.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
There is no need to do a jump to _syscall_returned as it is
the next to be run anyway. Keep the label there so we can
set breakpoint if needed.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
We stuff the 7th argument into stack by moving the stack pointer
before calling syscall handler. The Xtensa ABI says stack must
be 16-byte aligned. So instead of moving stack pointer 4 bytes,
we move 16 bytes (assuming stack has been aligned so far).
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Xtensa ISA mentions that stack always needs to be aligned on
16 bytes. So we need to pad the stack frame to be also 16 bytes
aligned when dealing with interrupts. Or else the stack would
not be 16 bytes aligned when we add the stack frame to stack
during interrupt handling.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
When crossing stack during interrupt handling, we do two call4
to pivot to the interrupt stack, with arguments to these two
call4 stashed in A6, A10 and A11. However, A4-A11 may be marked
as invalid in the register file, and accessing them would
result in window overflowing. At that point, A0 and A1 are not
setup to handle window overflows, and will result in registers
being stashed in incorrect location, resulting in incorrect
value being restored during window underflowing. So move around
the code a bit to restore A0 and A1 properly before accessing
A4-A11.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Do not force CONFIG_XTENSA_SMALL_VECTOR_TABLE_ENTRY when MMU
is enabled. It is up to the SoC to decide whether they need
this.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
The logic to swap page tables or MPU entries is moved to the end
of cross stack call, since it is still running in the interrupt
stack instead of the thread stack. The old logic was calling
the swap functions in the outgoing thread stack, which is not
desirable.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
When switching page table and MPU enties, we need to call
corresponding functions via call4 which would move the register
window. So we need to do that earlier before we start saving
context into stack by manipulating stack pointer manually which
definitely would interfere with window spilling.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
During arch_cohere_stacks(), the used portion of the outgoing
thread is cache flushed, and then the unused portion of cache
invalidated. However, this results in the cache line at
the stack pointer being flushed and then invalidated due to
how sys_cache_data_*() operates. If we are swapping back to
the same thread (e.g. after handling interrupt), this cache
line will need to be retrieved again from main memory since
it has already been invalidated. This creates unnecessary
data move between cache and main memory. So create our own
version of cache flushing and invalidation routines just for
arch_cohere_stacks(). Bouns is that these work directly with
bounding addresses and skips the size calculation which should
save a little bit amount of execution time.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This removes _xstack_call1_* trampoline as we can simply use
callx4 to jump to the interrupt handler.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Before cross stack call is setup correctly, we cannot allow
interrupts to be triggered or it may interfere with register
window spilling since we are clobbering registers needed for
that to work. However, there was a brief period where higher
level interrupts could fire due to code writing to PS with
lowered interrupt mask before raising it again. So rework
that part to avoid writing PS with intermediate value, and
now we mask interrupt until everything is setup correctly
before interrupt is enabled again.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
When initializing the stack at thread creation, we should not
set the pointer to privileged stack pointer yet as the thread
can be a kernel thread. Only when a thread is transitioning to
user mode, then we need to set the pointer to point to
the privileged stack. This is a purely semantic change and
should not affect any functionality.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This moves the calculation of the return PC earlier in syscall
entry. There is no need to stash it into BSA, load it, do
the calculation and save it back. We can do the calculation
first and save it in BSA at the same time.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
We should not perform a straight OR operation on INTLEVEL as
we have no idea what existing PS.INTLEVEL is. Also, to avoid
any interferences, we disable interrupts very early during
syscall entrance. So we can remove the OR operation as
PS.INTLEVEL will still have all interrupts masked. Note that
we do not really need to OR PS_WOE into PS as we currently
only support windowed ABI which must have PS_WOE set in PS
anyway.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Syscall entrance makes use of ODD_REG_SAVE but it does not
really need to save FPU registers as it is technically
the same thread and same context. So extract call to
FPU_REG_SAVE to interrupt handling code.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This adds ODD_REG_RESTORE as a counterpart to ODD_REG_SAVE.
Both the code in interrupt handling and syscall exit have
been refactored to use this new macro.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This adds two parameters for ODD_REG_SAVE for scratch register
and BSA pointer, thus allowing a bit more flexibility on how
it can be called.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This removes the call to _xtensa_hifi_save() to save the HiFi
registers during syscalls. During syscall, we are not doing
context switching, and technically it is still the same thread.
There is no need to save HiFi registers.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
The Xtensa ISA reference manual says to do rsync after wsr to
make sure register is updated before rsr. So do that.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Remove CONFIG_XTENSA_INVALIDATE_MEM_DOMAIN_TLB_ON_SWAP as it is
remnant from early MMU enabling work which is not needed as
the page table code is different from early version where
the PTEVADDR would be the same for all memory domains.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
thread_page_tables_get() is only used when userspace is
enabled. So move it with userspace #ifdef, or else
compiler would complain about it being unused.
Fixes#88421
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Since commit 0aa6b1c9de, the 'ldr_parm' pointer is guaranteed to be
valid inside all the functions of the llext_load() call tree.
This commit fixes the only exception of llext_copy_strings(), which was
not passed the 'ldr_parm' pointer, and remove the redundant checks.
No functional change is intended by this commit.
Signed-off-by: Luca Burelli <l.burelli@arduino.cc>
Inside map_memory() with double mapping enabled, we should not
be mapping the memory with the incoming attributes as-is since
the incoming address may be on un-cached region but with
caching attribute. So we need to sanitize the attributes
according to the incoming address.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
When building pre-located LLEXTs of ET_DYN type (shared ELF object)
on Xtensa, all the R_XTENSA_RELATIVE relocations are already correct,
the current code actually breaks them by mobing the value from the
target address-space to a storage range address. Simply removing the
recalculation solves the issue.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
Mostly a revert of commit b1def7145f ("arch: deprecate `_current`").
This commit was part of PR #80716 whose initial purpose was about providing
an architecture specific optimization for _current. The actual deprecation
was sneaked in later on without proper discussion.
The Zephyr core always used _current before and that was fine. It is quite
prevalent as well and the alternative is proving rather verbose.
Furthermore, as a concept, the "current thread" is not something that is
necessarily architecture specific. Therefore the primary abstraction
should not carry the arch_ prefix.
Hence this revert.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Xtensa cache line sizes aren't an obtuse area of pedantry like they
are in x86. Different cores already in Zephyr are already using
variant cache line sizes (64 and 128 bytes are both common).
And I tripped over this by using the wrong value because the kconfig
was being inherited (incorrectly) from a default somewhere.
Xtensa exposes the correct value in core-isa.h (well, unless the
toolchain/hal gets messed up). Add a check to make sure that our
platform kconfig gets it right.
Note that qemu/dc233c was already getting this wrong, leaving the
value at the kconfig default of zero. That was benign (qemu doesn't
provide any cache emulation for incoherent DMA), but needs to be
fixed.
Signed-off-by: Andy Ross <andyross@google.com>
