Newer Xtensa toolchain needs to include xtensa-types.h so that
macros in tie.h can be used without compilation errors.
The exact verison needing this is unknown but first encountered
in RJ-2023.2. Tested with older toolchain and that did not
cause any compilation errors so just include xtensa-types.h
if xt-clang is used. Haven't seen newer toolchains being
generated with xcc, so skip that for now.
Note that Zephyr SDK and the public HAL in Zephyr do not provide
this file.
Signed-off-by: Anthony Giardina <anthony.giardina@intel.com>
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
xtensa_mmu_init() is called really early in the boot process
where the _kernel struct has not yet been initialized, and
thus we cannot use it to determine if the current CPU is
the boot CPU. In some cases, this may skip the call to
initialize the page tables which leaves us with incorrect
page table entries. Fix it by using a static variable to
determine whether the page tables have been initialized so
we only do it once per boot.
Fixes#76909
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
is_dblexc is constant for targets without MMU. There is no need
to check it when building without MMU.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
custom arch_cpu_idle and arch_cpu_atomic_idle implementation was done
differently on different architectures. riscv implemented those as weak
symbols, xtensa used a kconfig and all other architectures did not
really care, but this was a global kconfig that should apply to all
architectures.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
arch_kernel_init() was misused for all architecture initialization code
that is done in prep_c and prior to cstart on other architectures.
arch_kernel_init() is late in the init process and comes after EARLY
init level, making xtensa have a very special boot path.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
xtensa is the only architecutre doing thing differently and introduces
inconsistency in the init process and dependencies as we attemp to
cleanup init levels and remove misused of SYS_INIT.
Introduce prep_c for this architecture and align with other
architectures.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
ZSR_DEPC_SAVE is being used to determine whether we are faulting
inside double exception if this is not zero. It is possible that
the boot ROM or custom startup code leaves this non-zero, which
would result in a fake triple fault. So clear it at boot. Note
that the zeroing is done in MMU init code as these triple
faults are not actual hardware ones but only semantics, and will
occur once MMU is enabled.
Fixes#75194
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
The generic stack pointer checker in the architecture code is
enough so we can remove the platform specific one. Besides,
xtensa_dc233c_stack_ptr_is_sane() does not do much checking
either.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This checks for stack boundaries during backtrace to make sure
we are not stepping into invalid memory.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Check that the stack frame pointer is valid before dumping
any registers while handling exceptions. If the pointer is
invalid, anything it points to will probably be also be
invalid. Accessing them may result in another access
violation.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
It is observed that during logging in fatal exception,
it prints "FATAL EXCEPTION(null)". Exact reason is unknown
as debugging through GDB would make it all work again. So
separating it into two log statements to avoid this situation.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
There actually is no triple faults on Xtensa. Once PS.EXCM is
set, it keeps going through double exception vector for any
new exceptions. However, our exception code needs to unmask
PS.EXCM to enable register window operations. So after that,
any new exceptions will go through the kernel or user vectors
depending on PS.UM. If there is continuous faults, it may
keep ping-ponging between double and kernel/user exception
vectors that may never get resolved. Since we stash DEPC
during double exception, and the stashed one is only cleared
once the double exception has been processed, we can use
the stashed DEPC value to detect if the next exception could
be considered a triple fault. If such a case exists, simply
jump to an infinite loop, or quit the simulator, or invoke
debugger.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
arch_user_string_nlen() did not exactly work correctly as any
invalid pointers being passed are de-referenced naively, which
results in DTLB misses (MMU) or access errors (MPU). However,
arch_user_string_nlen() should always return to the caller
with appropriate error code set, and should never result in
thread termination. Since we are usually going through syscalls
when arch_user_string_nlen() is called, for MMU, the DTLB miss
goes through double exception. Since the pointer is invalid,
there is a high chance there is not even a L2 page table
associated with that bad address. So the DTLB miss cannot be
handled and it just keeps looping in double exception until
there is another exception type where we get to the C handler.
However, the stack frame is no longer the frame associated
with the call to arch_user_string_nlen(), and the call return
address would be incorrect. Forcing this incorrect address as
the next PC would result in some other exceptions, e.g.
illegal instruction, which would go to the C handler again.
This time it will go to the end of handler and would result
in thread termination. For MPU systems, access errors would
simply result in thread terminal in the C handler. Because of
these reasons, change the arch_user_string_nlen() to check if
the memory region can be accessed under kernel mode first
before feeding it to strnlen().
Also remove the exception fixup arrays as there is nothing
there anymore.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This adds a new function xtensa_mem_kernel_has_access() to
determine if a memory region can be accessed by kernel threads.
This allows checking for valid mapped memory before accessing
them to avoid relying on page faults to detect invalid access.
Also fixed an issue with arch_buffer_validate() on MPU where
it may return okay even if the incoming memory region has no
corresponding entry in the MPU table.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
arch_buffer_validate() is only to verify that user threads have
access to the memory region. It should not be used to verify
if kernel thread has access (which they should anyway). So
change the logic.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Introduce CONFIG_XTENSA_BREAK_ON_UNRECOVERABLE_EXCEPTIONS to
use BREAK instruction for unrecoverable exceptions. This
definitely requires debugger to be attached to the hardware
or simulator to catch that.
Also move the infinite loop to NOT result in an infinite
interrupt storm as the debug interrupt will be triggered over
and over again. Same for the simcall exit as it does not
need to be called repetitively.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
If there are any TLB misses during interrupt handling,
the user, kernel and double exception vector will be triggered
for the miss and the DEPC and EXCCAUSE overwritten as the TLB
missse are be handled in the assembly code and execution
returned to the original vector code. Because of this, both
DEPC and EXCCAUSE being read in the C handler are not the ones
that triggered the original exception (for example, level-1
interrupt). So stash both DEPC and EXCCAUSE such that
the original cause of exception is visible in the C handler.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Xtensa MPU code does not handle double exception in C. So there
is no need to clear DEPC on C handler exit.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
We have frame pointer struct and BSA struct to extract
the exception cause (exccause). There is no need to
resort to custom assembly to do that. Besides, given
that the BSA is different between different Xtensa cores,
there is no guarantee it is at the same place as what
the assembly assumes. So just do that without assembly.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Also any demand paging and page frame related bits are
renamed.
This is part of a series to move memory management related
stuff out of the Z_ namespace into its own namespace.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Move this to a call in the init process. arch_* calls are no services
and should be called consistently during initialization.
Place it between PRE_KERNEL_1 and PRE_KERNEL_2 as some drivers
initialized in PRE_KERNEL_2 might depend on SMP being setup.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
With power managment is enabled, depending on the SoC power state
used when idle, the MMU may lose context and may need to be re-initialized.
When re-initializing the MMU, we must not re-create the page table
because it may overwrite changes done during the execution, but we still
need to set the asid and page table for the current context.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Make `struct arch_esf` compulsory for all architectures by
declaring it in the `arch_interface.h` header.
After this commit, the named struct `z_arch_esf_t` is only used
internally to generate offsets, and is slated to be removed
from the `arch_interface.h` header in the future.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Namespaced the generated headers with `zephyr` to prevent
potential conflict with other headers.
Introduce a temporary Kconfig `LEGACY_GENERATED_INCLUDE_PATH`
that is enabled by default. This allows the developers to
continue the use of the old include paths for the time being
until it is deprecated and eventually removed. The Kconfig will
generate a build-time warning message, similar to the
`CONFIG_TIMER_RANDOM_GENERATOR`.
Updated the includes path of in-tree sources accordingly.
Most of the changes here are scripted, check the PR for more
info.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Existing solutions for linking the Xtensa vector table are a
cut-and-paste mess of inherited code, with more than a dozen special
sections that need to be linked into many special MEMORY{} regions.
Accept the existing convention used by C/asm code, but automatically
detect the needed offsets for the platform from core-isa.h (it can
share the preprocessing with gen_zsr.py) and emit a file that can be
included in lieu of all the existing boilerplate.
Signed-off-by: Andy Ross <andyross@google.com>
The original code would (unsurprisingly) only emit handler functions
for interrupt levels with interrupts associated with them. But it
turns out that it's possible to configure an xtensa device with an
empty-but-otherwise-real interrupt level (specifically mt8195 has a
"Level 3" interrupt not associated with any input IRQS, it's one level
above EXCM_LEVEL and one level below the DEBUG exception).
This script is old, and not set up to parse the full core-isa.h
directly, so modifying it to detect this condition is difficult.
Instead, just emit all 15 possible interrupt handlers, even empty
ones. The extra stubs are harmless as they'll be dropped if uncalled.
Signed-off-by: Andy Ross <andyross@google.com>
Some oddballs cores can be generated without an "NMI" interrupt, in
which case core-isa.h will not define XCHAL_NMILEVEL. This code is
trying to unconditionally mask interrupts, so XCHAL_EXCM_LEVEL is the
pedantically correct choice anyway (NMI's by definition, cannot be
masked).
Signed-off-by: Andy Ross <andyross@google.com>
The script to generate the _soc_inthandlers.h header has been run
manually for years, only because I was a cmake novice at the time and
unsure how to integrate it into the build. So every new platform has
to find the script and template file and figure out how to generate
the file. And in a few cases it looks like we've tried to EDIT the
resulting files in the tree.
Let's finally do this right. The file is now dropped (for every
xtensa platform) as a "xtensa_handlers.h" file, and there is a Kconfig
to control whether the original/manual file or the new one is used by
the platform code. We can migrate the other platforms slowly as
people have time to validate.
Signed-off-by: Andy Ross <andyross@google.com>
The only page table duplicated is the kernel page table. This function
does not need a parameter.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
We can use some extra bits available for SW implementation to
save original permissions and avoid duplicating the kernel page tables
for the default memory domain.
Whe duplicating the page table to a new domain we just ensure
to restore the original map.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Simplify the logic around the shared attribute. Checks if a memory
region should be shared only in the function that actually maps the
memory.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Introduce z_page_frame_set() and z_page_frame_clear() to manipulate
flags. Obtain the virtual address using the existing
z_page_frame_to_virt(). This will make changes to the page frame
structure easier.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
When duplicating a page table, we don't need to copy
the mapping to the kernel l1 page table virtual address.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
A workaround to avoid icache corruption was added in commit be881d4cf2
("arch: xtensa: add isync to interrupt vector").
This patch implements a different workaround by adding custom logic to
idle entry on affected Intel ADSP platforms. To safely enter "waiti"
when clock gating is enabled, we need to ensure icache is both unlocked
and invalidated upon entry.
Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Currently LLEXT on Xtensa supports relocatable extensions, linked for
a specific address range, while relocation itself takes place in a
temporary buffer. For this section addresses have to be set correctly
by the linker for their target locations.
This commit adds support for relocatable extensions, built without
using specific memory addresses and run at the same addresses, where
they are loaded.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
Some toolchains cannot create shared objects for Xtensa, with them we
have to use relocatable objects. Add support for them to llext.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
Simple rename to align the kernel naming scheme. This is being
used throughout the tree, especially in the architecture code.
As this is not a private API internal to kernel, prefix it
appropriately with K_.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
