Adds Kconfig option
CONFIG_XTENSA_EMULATE_UNSUPPORTED_UNSIGNED_LOAD_STORE
to enable exception handler for unsupported
narrow and / or unaligned unsigned load / stores that
reads the triggering instruction and performs the
operation manually with supported word sized and
aligned accesses.
Signed-off-by: Lauren Murphy <lauren.murphy@intel.com>
Co-authored-by: Anthony Giardina <88748592+agiardin@users.noreply.github.com>
If the (addr + size) overflows the memory address space,
the inner loop may not run to check for permission. Since
the default return value was 0 (meaning permitted), it
would incorrectly say memory access was okay. Fix this by
changing the default return value to -EINVAL. Only after
the loop of validating the whole input address range then
we set the return value to 0 to say memory access is
permitted. Also check for addition overflow.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This adds support for using L32EX/S32EX for atomic CAS operation
in the architecture layer. This is an alternative to S32C1I if
the SoC supports it.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Include the terminating NULL character in the path length passed to the
semihosting open call, as required by some debuggers.
Signed-off-by: Tahsin Mutlugun <Tahsin.Mutlugun@analog.com>
Add semihosting support for Xtensa QEMU targets. Although QEMU uses the
same simcall instruction, it relies on different flag values for file
open operations compared to Xtensa Instruction Set Simulator.
Prioritize QEMU-specific compatibility if both SIMULATOR_XTENSA and
QEMU_TARGET are enabled.
Signed-off-by: Tahsin Mutlugun <Tahsin.Mutlugun@analog.com>
When a memory domain is freed on Xtensa, it also has to be removed
from the global domain list. Leaving it on the list can cause
use-after-free exceptions.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
The custom memory range checks should be implemented in SoC or
board level as these checsk are SoC/board specific. So remove
it from the architecture level.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
On a cache incoherent system, we need to make sure the caching
of stack space is properly flushed to memory when creating new
threads. This is especially important if the thread starts
running on the CPU other than the one initializing the thread.
Without flushing, the other CPU would not have the up-to-date
data to correctly start the thread.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Add XTENSA_BACKTRACE_EXCEPTION_DUMP_HOOK Kconfig option for sending
backtrace through exception dump hook.
This commit also disables the printk backtrace dumping if Kconfig
option EXCEPTION_DUMP_HOOK_ONLY is set.
Signed-off-by: Jyri Sarha <jyri.sarha@linux.intel.com>
The new exception dump hooks provides helper function for draining or
flushing the accumulated dump data. These helpers are for the backend
to deal intelligently with often excessive amount of data for limited
bandwidth interfaces.
These calls are placed specifically for SOF application, but AFAIK SOF
is the most widely used Zephyr application running on Xtensa.
The helpers do not have any effect if CONFIG_EXCEPTION_DUMP_HOOK is
not set.
Signed-off-by: Jyri Sarha <jyri.sarha@linux.intel.com>
Add hooks for delivering exception dump prints over a specialized
interface. If CONFING_EXCEPTION_DUMP_HOOK=y then a client program can
set function pointers for printing, flushing, and draining exception
generated prints.
These hooks were implemented for SOF usage, but should be generic
enough to implement alternative exception reporting on any platform.
Signed-off-by: Jyri Sarha <jyri.sarha@linux.intel.com>
is_fatal_error is used to determine whether an exception is
a fatal one. In the default switch case for exception handling,
is_fatal_error needs to be set true. However, setting this
variable was done after stack bound check. So if stack bound
check fails, is_fatal_error is never set. So set the variable
earlier before the stack bound check.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
For !TRACING, most arch_cpu_idle and arch_cpu_atomic_idle implementation
relies on the fact that there's weak stub implementations in
subsys/tracing/tracing_none.c, this works, but the arch_cpu_idle sits in
hot code path, so we'd better to make it as efficient as possible.
Take the riscv implementation for example,
Before the patch:
80000a66 <arch_cpu_idle>:
80000a66: 1141 addi sp,sp,-16
80000a68: c606 sw ra,12(sp)
80000a6a: 37c5 jal 80000a4a <sys_trace_idle>
80000a6c: 10500073 wfi
80000a70: 3ff1 jal 80000a4c <sys_trace_idle_exit>
80000a72: 47a1 li a5,8
80000a74: 3007a073 csrs mstatus,a5
80000a78: 40b2 lw ra,12(sp)
80000a7a: 0141 addi sp,sp,16
80000a7c: 8082 ret
NOTE: the sys_trace_idle and sys_trace_idle_exit are just stubs when
!TRACING
after the patch:
80000a62 <arch_cpu_idle>:
80000a62: 10500073 wfi
80000a66: 47a1 li a5,8
80000a68: 3007a073 csrs mstatus,a5
80000a6c: 8082 ret
Signed-off-by: Jisheng Zhang <jszhang@kernel.org>
Instead of using __ASSERT() with an empty string as message,
simply convert it to use __ASSERT_NO_MSG().
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This supports de-initialization of memory domains to release
allocated page tables back to the pool.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
When a L2 table is no longer being used, we should set all PTEs
in the table to be illegal PTEs. This is simply a precautious
so that any stray references to the L2 table would not result
in incorrect permissions being applied.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
When handling an ISR (which does not have a context from which to
restore its own value of LCOUNT), we must clear LCOUNT to prevents
incorrect zero-overhead execution if calling a function such as
memmove() which could be implemented using zero-overhead loop.
A function such as memmove() implemented using zero-overhead loop
assumes LCOUNT to have properly been setup before being called; but
an ISR calling memmove() in assembly, will likely not know that.
Signed-off-by: William Tambe <williamt@cadence.com>
This reverts commit 8c02dde437.
For some unknown reasons, xt-clang emits two copies of
z_xt_init_pc if xtensa_mmu_init_paging() is in the same file
as xtensa_mmu_init() and xtensa_mmu_reinit(). So had to
revert the change.
Fixes#103055
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
When using LLEXT, instruction TLB multi-hit becomes a reality
as the same memory space can be occupied by different modules
with different permissions. The ITLB cache may still contain
entries of the unloaded module. So we need to manually
invalidate any cached ITLB corresponding to the exception
address so the TLB associated with the newly loaded module
can be used.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
There is no need to invalidate the whole auto-refilled data TLB
cache when DTLB multi-hit exception is raised. Now it only
invalidates the TLB entries corresponding to the one causing
the DTLB multi-hit. This allows other non-related TLB entries
to remain in the cache so they don't need to be reloaded.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
For both data TLB multi-hit and load/store ring error, we should
return to the interrupted thread immediately so that it can get
past the exception generated code. It is because both of these
exceptions are the result of having cached TLB entries not
aligning to the correct access pattern. So once we have handled
the exception, go back to the interrupted thread to continue
to minimize the chance of having another incompatible TLB being
cached.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
There is no need for an extra switch block to manipulate
the value of is_fatal_error, which defaults to false, and is set
according to the actual exception above. So remove that.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Since we have split source files for page table related stuff
and MMU related stuff, move the MMU initialization functions
from the page table source file into MMU source file.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This adds doxygen doc to the page table source file as
we are missing quite a bit of documentation there.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This changes the existing comments for functions into doxygen
style documentation for functions. Also adds missing doxygen
doc for functions.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
If there are not enough free L2 tables to map all predefined
memory regions at boot, halt the system in case assertion is
not enabled. Without all the needed memory regions mapped,
it is very unlikely that anything will run properly.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Adds some debug logs when we are allocating page tables.
This provides a more visible way of seeing whether
we need to have more free tables.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Add an assertions to halt the system if L2 table allocation
fails when we need to duplicate an existing L2 table, as it is
a must-have and must-success operation.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Preserve EXCCAUSE and EXCVADDR values on entry to _Level1Vector.
Restore EXCCAUSE when exiting TLB miss exception handling in the double
exception handler.
During first-level exception handling, a LoadStoreTLBMissException may
occur during the initial register dump to BSA. It modifies EXCCAUSE and
EXCVADDR registers before they are saved in BSA. Therefore, these values
must be captured as early as possible.
Signed-off-by: Adrian Warecki <adrian.warecki@intel.com>
Remove saving EXCCAUSE register in BSA through the _Level1Vector handler.
These value are later overwritten by the ODD_REG_SAVE macro called by
EXCINT_HANDLER, so saving it here is pointless.
Signed-off-by: Adrian Warecki <adrian.warecki@intel.com>
soc_per_core_init_hook() is usually called from arch_kernel_init() and
arch_secondary_cpu_init() which are C functions. As such, there is no need
to check for CONFIG_SOC_PER_CORE_INIT_HOOK since platform/hooks.h provides
a no-op function-like macro implementation if the Kconfig option is not
enabled.
Remove the Kconfig option check from all files.
Signed-off-by: Mathieu Choplain <mathieu.choplain-ext@st.com>
Use void * cleanly: avoid needless type-casts and use void * for
generic pointers instead of uint8_t *.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
Remove duplicated #include directives within the same
preprocessor scope across the Zephyr tree.
Duplicates inside different #ifdef branches are preserved
as they may be intentional.
Signed-off-by: Sylvio Alves <sylvio.alves@espressif.com>
During L2 page table duplication, there are cases where we
should copy the table over instead of restoring the PTEs.
Add the ability to copy the table over.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This adds a spinlock to guard page table counter updates.
For some reasons, without this, xt-clang generated code would
result in incorrect counter update possibly due to memory
access ordering (though GCC generated code works fine without
the spin lock).
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This removes the following kconfigs:
() CONFIG_XTENSA_RPO_CACHE
() CONFIG_XTENSA_CACHED_REGION
() CONFIG_XTENSA_UNCACHED_REGION
The associated feature is actually SoC specific and not general
architecture feature. Since the only user of these have its own
implementation, we can now safely remove these.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This re-implements the CONFIG_XTENSA_RPO_CACHE and its cached
and uncached region support in the SoC layer. This is in
preparation for removing RPO cache in the architecture layer
as this is a SoC feature and not an architecture feature.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This removes CONFIG_XTENSA_MMU_DOUBLE_MAP and its associated
code. The double mapping of a page into both cached and
uncached memory regions should be done in SoC and application,
as it gives finer control on what actually is being mapped.
For example, if a page only needs to mapped in cached region,
there is no need to map it in the uncached region. This may
save the need to allocate a L2 page table for it, and saves
some memory.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Adds kconfig CONFIG_XTENSA_MMU_USE_DEFAULT_MAPPINGS for arch
code to initialize the page tables with a few default mappings
covering text, data, BSS and RODATA. Disabling this allows
the SoC to customize and fine-tune the mappings if needed.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Set PPN (Physical Page Number) in L2 entry in the region_map_update
function to ensure correct mapping for memory that was not previously
mapped based on xtensa_soc_mmu_ranges.
When a new memory domain is created, the arch_mem_domain_init function
allocates new ptables using the dup_table function. This function only
copies original L2 entries created from xtensa_soc_mmu_ranges with
OPTION_SAVE_ATTRS. Entries added later by arch_mem_map are replaced with
XTENSA_MMU_PTE_L2_ILLEGAL and have PPN (Physical Page Number) set to zero.
When adding a new partition to a memory domain, the update_region function
is used. It sets new access attributes in the L2 table without setting the
PPN value. As a result, it does not work correctly for memory that was not
previously mapped based on xtensa_soc_mmu_ranges.
Signed-off-by: Adrian Warecki <adrian.warecki@intel.com>
Add macro PTE_PPN_SET to simplify setting the physical page
number assigned in the page table entry.
Signed-off-by: Adrian Warecki <adrian.warecki@intel.com>
Add macro PTE_PPN_GET to simplify retrieval of the physical page
number assigned to a page table entry.
Signed-off-by: Adrian Warecki <adrian.warecki@intel.com>
xt-clang complains about l2_page_tables_counter_inc() being
unused but not GCC. So fix that by using it somewhere else.
Fixes#99753
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Due to slight differences in the way that LLVM and GNU linkers work,
the call to `z_stack_space_get()` is not dead-stripped when linking
with `lld` but it is dead-stripped when linking with GNU `ld`.
The `z_stack_space_get()` function is only available when
`CONFIG_INIT_STACKS` and `CONFIG_THREAD_STACK_INFO` are defined.
The issue is reproducible (although requires building LLVM and
setting up some environment variables) and goes away with the proposed
workaround.
Signed-off-by: Robin Kastberg <robin.kastberg@iar.com>
Signed-off-by: Chris Friedt <cfriedt@tenstorrent.com>
Adds function for app to obtain page table usage statistics,
allowing fine tuning of numbers of L1 and L2 page table
array.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
