When adding new memory domains, we need to start from kernel
page tables by duplicating them. However, there is no need to
duplicate all in-use page tables as some memory regions remain
unchanged. Fror example, hardware register region where only
kernel has access does not need to be duplicated across all
memory domains. The same L2 table can be used among them.
This changes the L2 page table allocation to be copy-on-write
where we only need to duplicate a L2 table if changes need to
be made on it.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
If the page tables are not cached, there is no need to do any
cache ops to flush or invalidate the data in cache. So skip them
if the page tables are not cached. Saves a few CPU cycles.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
In l2_page_table_unmap(), invalidating TLBs should be limited to
the address and not the whole auto-refill TLBs. Also fix a bug
where the EXEC bit should be determined from the L2 PTE and not
the L1 PTE.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Some macros for the page tables array are local to the file so
there is no need to prefix them with XTENSA_. Simplify by
removing the XTENSA_ prefix, and clairfy if needed.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Almost all page table entry (PTE) related macros are only used
in one source file. These macros are considered the internal
working of MMU. There is no need to expose them in header file
for other to reference. So move them into the source file where
it is only place they are used. Bonus is that we can shorten
the macro names as they are now local to the file. Makes it
easier to read, and few keystrokes to input.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This skips the redirection of backup attributes and ring via
some SW field macros, and use the PTE bits directly. Should
make it easier to decode what's going on.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
The fields in macro should be done with both shift and mask
values to make it clear. So amend some macros so that
the presentation are unified for them.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This change suppress the warning:
variable 'ie' is used uninitialized whenever switch default is
taken [-Wsometimes-uninitialized]
Signed-off-by: William Tambe <williamt@cadence.com>
soc_prep_hook() is always called from z_prep_c() which is implemented
as a C function. As such, there is no need to check for the associated
CONFIG_SOC_PREP_HOOK since the platform/hooks.h header will define hooks
as no-op function-like macros if their associated Kconfig isn't enabled.
Remove the Kconfig check from all arch implementations of z_prep_c() and
call soc_prep_hook() directly instead, to avoid duplicating the Kconfig
check already performed in platform/hooks.h
Signed-off-by: Mathieu Choplain <mathieu.choplain-ext@st.com>
arch_mem_map() takes in some flags to describe the to-be mapped
memory regions' permissions and cache status. When the flags are
translated, they become attributes in PTEs. So for functions
being called by arch_mem_map() and beyond, rename flags to
attrs to better describe its purpose.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
In functions which manipulate both L1 and L2 tables, make
the variable names obvious by prefixing them with l1_ or l2_.
This is mainly done to avoid confusion when reading through
those functions.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
The assert error messages when l2_page_table_map() fails are not
correct. It returns false when it cannot allocate new L2 table,
and it is not able the address having already mapped. So correct
the error message.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This is simply done to conserve code space in the vector text
areas. These vector text areas are very small and we should
only put code that is absolutely necessary for interrupt and
exception entrance. The saving of PS into the thread struct
can be deferred a bit. So do that.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
xtensa_asm2_s.h is not exactly a C header. Rename it to
xtensa_asm2.inc.S to clearly state that it is has assembly
code in it as its main purpose is to declare assembly macros.
This is being done to keep checkpatch.pl from treating it as
C file and complaining about non-C syntax.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This moves the block of FPU and HIFI registers in the Base Save
Area (BSA), used during interrupts and exceptions, to the end of
the block. This is done to minimize code usage in the vector
text section. During interrupt entrance, the code stores a small
set of registers first before jumping to the long handler. When
the offset to these registers from the beginning of BSA is small
enough, the compiler will emit S32I.N instead of S32I. This
saves us one byte per store (2 vs 3 bytes). This is mainly done
to avoid overflowing the vector text area.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This adds a function to handle DTLB multihit exception when
userspace is enabled, as this exception may be raised due to
the same page being able to accessed by both kernel and user
threads. The auto-refill DTLBs may contain entries for same
page, one for kernel and one for user under some situations.
We need to invalidate those existing DTLB entries so that
hardware can reload from the page table.
This is an alternative to the kconfig invalidating DTLBs on
every swap: CONFIG_XTENSA_MMU_FLUSH_AUTOREFILL_DTLBS_ON_SWAP.
Although this does not have extra processing per context
switching, exception handling itself has a high cost. So
care needs to be taken on whether to enable that kconfig.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
When a page can be accessed by both kernel and user threads,
the autofill DTLB may contain an entry for kernel thread.
This will result in load/store ring exception when it is
accessed by user thread later. In this case, we need to
invalidate all associated TLBs related to kernel access so
hardware can reload the page table the correct permission
for user thread.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This saves the EXCCAUSE and EXCVADDR into BSA during exception.
These will then be used during exception handling. The reason
for doing this instead of reading from both registers during
exception handing is that another exception will override
the value in these register (e.g. DTLB miss). When returning to
the previous exception handler, these register no longer
contains the original exception cause and address. We need to
save it so that we are actually handling the orignal exception.
Coredump also now uses the EXCCAUSE and EXCVADDR from BSA
instead of reading the registers.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
The software bits inside PTE are used to store original PTE
attributes and ring value, and those bits are used to
restore PTE to previous state.
This modifies reset_region() to properly restore attributes
and ring value when resetting memory regions to the same as
when the system boots.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Several static functions in ptables.c always return 0, make them void
to improve readability.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
Select ARCH_SUPPORTS_COREDUMP_STACK_PTR on xtensa, and provide an
implementation for the arch_coredump_stack_ptr_get function.
Signed-off-by: Mark Holden <mholden@meta.com>
Do not directly include and use APIs from ksched.h outside of the
kernel. For now do this using more suitable (ipi.h and
kernel_internal.h) internal APIs until more cleanup is done.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
z_prep_c does not return, mark it as such consistently across
architectures. We had some arches do that, others not. This resolves a
few coding guideline violations in arch code.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Cleanup init.c code and move early boot code into arch/ and make it
accessible outside of the boot process/kernel.
All of this code is not related to the 'kernel' and is mostly used
within the architecture boot / setup process.
The way it was done, some soc code was including kernel_internal.h
directly, which shouldn't be done.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
If the GDB stub is enabled the exception handler will jump to the GDB
stub to allow remote GDB debugging.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
GCC 14.3 will happily delete any code that appears before
__builtin_unreachable that isn't separated with an obvious branch. That
includes __asm__ statements, even those which generate traps.
The failure case that I debugged was on x86 in
z_check_stack_sentinel. There is a store to restore the sentinel to the
correct value just before the ARCH_EXCEPT, and that macro emits 'int $32'
followed by CODE_UNREACHABLE. Because the compiler didn't understand that
ARCH_EXCEPT was changing execution flow, it decided that the sentinel
restoring store 'couldn't' be reached and elided it.
I added the "memory" clobber to the asm statement in ARCH_EXCEPT before
CODE_UNREACHABLE to enforce that all pending store operations be performed
before the asm statement occurs. This ensures that they are not deleted by
the compiler.
I think this might be a GCC bug. The GCC documentation explicitly documents
that asm statements which change the flow of control should be followed by
__builtin_unreachable.
Signed-off-by: Keith Packard <keithp@keithp.com>
Fixes a flaw in the the xtensa implementation of arch_is_in_isr()
that could manifest on SMP systems. If the reading of the current
CPU's nested interrupt count is not fully atomic on an SMP system,
then an ill-timed context switch could occur leaving the caller
reading the nested interrupt of a different CPU.
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
When building with user space and LLEXT support, syscall helpers need
to be exported.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
Add needed backtrace helpders routines and enable
backtrace for the Xtensa Fusion F1 DSP in the
IMXRT595S.
Signed-off-by: Mike J. Chen <mjchen@google.com>
When lazy HiFi context switching is enabled, the system starts with
the HiFi coprocessor disabled. Should the thread use that coprocessor,
it will generate an exception which in turn will enable the coprocessor
and save/restore the HiFi registers as appropriate. When switching
to a new thread, the HiFi coprocessor is again disabled.
For simplicity, there are no restrictions as to which thread is allowed
to use the coprocessor.
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
The intent of arch_coprocessors_disable() is to replace
arch_float_disable() in halt_thread() for the FPU will not
always be the only coprocessor that will need to be disabled.
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
As the BSA can not be used when lazy HiFi context switching is
used, a more permanent and predictable location in which to store
the registers is required. To this end ...
1. reserve some space in the arch-specific portion of the k_thread
structure for those registers.
2. clear that region when the thread is created.
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
Splits HiFi sharing into two different models.
1. XTENSA_EAGER_HIFI_SHARING - unconditional save/restore
of the HiFi registers when context switching
2. XTENSA_LAZY_HIFI_SHARING - on demand save/restore of
of the HiFi registers. If a thread does not use the
HiFi registers, they are neither saved nor restored.
To maintain backwards compatibility, the eager model is the
default model when XTENSA_HIFI_SHARING is enabled.
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
instead of depending on logging for exceptions and fatal messages, use
macro and dump exceptions if printk is enable with no logging.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Add missing call to thread_switched_in for the purpose of tracing.
Fixeszephyrproject-rtos/zephyr#76057
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Utilize a code spell-checking tool to scan for and correct spelling errors
in `Kconfig` files within the `arch`, `boards`, `kernel`, `modules`,
`samples`, and `share` directory.
Additionally, incorporates a fix recommended by the reviewer.
Signed-off-by: Pisit Sawangvonganan <pisit@ndrsolution.com>
Update a comment about GDB use of register index and "regno"
parameters. Only some GDB versions use "idx" instead of "regno."
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
To report unavailable registers to the GDB client, the GDB stub has
to return a string like "xx" in the value string.
Signed-off-by: Noah Klayman <noah.klayman@intel.com>
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
Move an assignment performed on both "if" branches as the last
operation to after the "if."
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
