ISR_TABLES_LOCAL_DECLARATION depends on GEN_IRQ_VECTOR_TABLE but
this is not enforced in Kconfig.
Building without the GEN_IRQ_VECTOR_TABLE and with LOCAL_DECLARATION
will produce the following misleading static assertion error:
"CONFIG_IRQ_VECTOR_TABLE_JUMP_BY_{ADDRESS,CODE} not set"
As the LOCAL_DECLARATION macros expect GEN_IRQ_VECTOR_TABLE to be
enabled. LOCAL_DECLARATION also depends on GEN_ISR_TABLES but that
is a dependency of GEN_IRQ_VECTOR_TABLE already.
Signed-off-by: Bjarki Arge Andreasen <bjarki.andreasen@nordicsemi.no>
The native_sim uses the pthread stack instead of the Zephyr allocated ones.
This adds CONFIG_ARCH_POSIX_UPDATE_STACK_INFO to the posix arch to make the
real stack bounds available in thread info. (CircuitPython uses this to do
it's own stack overflow checking and recovery.)
The original stack values are restored on abort for backwards
compatibility with CMSIS v1.
Signed-off-by: Scott Shawcroft <scott@adafruit.com>
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
Using only the fence instruction to gate the management of data in cache
is insufficient to prevent unordered access after flushing in some cases.
Gate dcache instructions like icache instructions.
Signed-off-by: Camille BAUD <mail@massdriver.space>
Static MMU region entries populated via
MMU_REGION_DT_COMPAT_FOREACH_FLAT_ENTRY() pass raw DTS reg address and
size values to __add_map(), which asserts page-alignment. DTS nodes may
legitimately have non-page-aligned reg sizes reflecting actual hardware
register footprints, causing an assert crash during early boot when
CONFIG_ASSERT=y.
Align the base address down and size up to CONFIG_MMU_PAGE_SIZE in
add_arm_mmu_region(), mirroring the k_mem_region_align() logic already
used by the dynamic DEVICE_MMIO_MAP path in kernel/mmu.c. This ensures
all static platform MMU region entries are mapped with page-granular
parameters regardless of DTS reg values.
Signed-off-by: Appana Durga Kedareswara rao <appana.durga.kedareswara.rao@amd.com>
Setting SCR_ST_BIT actually traps CNTPS access to EL3, opposite
to what the comment says. Remove to allow secure EL1 access.
Also initialize CNTPS_CVAL_EL1 to prevent spurious interrupts.
Signed-off-by: Joakim Tjernlund <joakim.tjernlund@infinera.com>
Co-authored-by: Sudan Landge <sudan.landge@arm.com>
In addition to pool literal, we want to avoid jump tables generally
associated to Table Branch Byte (TBB) and Table Branch Halfword (TBH)
instructions.
Signed-off-by: Jérôme Pouiller <jerome.pouiller@silabs.com>
In addition to -mslow-flash-data, we must also ensure that the assembler
does not generate literal pools. They are automatically generated by the
LDR pseudo-instruction[1]:
- If the constant can be constructed with a MOV or MVN instruction, the
assembler emits the corresponding instruction.
- Otherwise (when the value does not fit on 16bits), the assembler places
the value in the next literal pool.
No options was found in GNU assembler to disable literal pool generation.
Therefore, this patch explicitly uses MOVT and MOVW when the assembler
would otherwise generate literal pool. Note, that LDR must be kept under
ifdef since Cortex-M0 does not support MOVT/ MOVW.
This patch only change four occurrences of LDR. The other occurrences do
not appear to generate literal pool (likely because the literal values are
< 0xFFFF). If a literal pool is generated in the future, it will introduce
a performance penalty. No other limitations are expected.
[1]: https://developer.arm.com/documentation/dui0204/f/ \
writing-arm-assembly-language/loading-constants-into-registers/ \
loading-with-ldr-rd---const?lang=en
Signed-off-by: Jérôme Pouiller <jerome.pouiller@silabs.com>
On some SoC, no data cache is associated with the main flash. Therefore,
all accesses to data stored in flash, especially literal pools[1] penalizes
performance. Fortunately, GCC and IAR provide options (-mslow-flash-data
and --no_literal_pool) to prevent the generation of literal pools.
Unfortunately, current GCC versions (14.x) do not support -mslow-flash-data
when Thread Local Storage (TLS) variables are used. A patch is currently
under review[2][3] to address this limitation. Without this gcc patch,
using -mslow-flash-data is not very user friendly. The user must rebuild
the libc (CONFIG_PICOLIBC_USE_MODULE=y) without TLS support
(CONFIG_THREAD_LOCAL_STORAGE=n), and must ensure that the application does
not rely on thread-safe "errno".
Because of these interactions with the compiler, this option can't be
automatically selected by the SoC. Thus, this patch leaves the option
hidden. The SoC may expose it if relevant.
[1]: https://en.wikipedia.org/wiki/Literal_pool
[2]: https://gcc.gnu.org/pipermail/gcc-patches/2026-February/707887.html
[3]: https://github.com/zephyrproject-rtos/gcc/pull/65
Signed-off-by: Jérôme Pouiller <jerome.pouiller@silabs.com>
Implement arch_mem_domain_deinit() for ARM64 to release page tables
back to the pool when a memory domain is de-initialized. This reuses
the existing discard_table() mechanism to recursively free all
sub-tables in the hierarchy.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
gdb cannot unwind the stack from exceptions. This adds
CFI annotations to help gdb unwind.
Signed-off-by: Joakim Tjernlund <joakim.tjernlund@infinera.com>
Instead of using __ASSERT() with an empty string as message,
simply convert it to use __ASSERT_NO_MSG().
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
The MAX32 RV32 core does not implement the fence instruction used by the
RISC-V synchronization intrinsic, so don't enable the builtin barriers for
that target.
Signed-off-by: Pete Johanson <pete.johanson@analog.com>
This supports de-initialization of memory domains to release
allocated page tables back to the pool.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
When a L2 table is no longer being used, we should set all PTEs
in the table to be illegal PTEs. This is simply a precautious
so that any stray references to the L2 table would not result
in incorrect permissions being applied.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
The ifdef guard in isr.S was written without the CONFIG_ prefix,
making the mtval fallback path dead code on all platforms including
QEMU (which previously worked via CONFIG_QEMU_TARGET).
Signed-off-by: William Markezana <william.markezana@gmail.com>
zeroing CNTHCTL_EL2 traps physical timer/counter access from EL1 to EL2,
but Zephyr has no hypervisor to handle those traps.
Enabling access is the standard EL2→EL1 drop behavior.
Signed-off-by: Joakim Tjernlund <joakim.tjernlund@infinera.com>
Select ARCH_SUPPORTS_COREDUMP_THREADS (if !SMP) and
ARCH_SUPPORTS_COREDUMP_STACK_PTR for RISC-V, and implement
arch_coredump_stack_ptr_get().
This enables CONFIG_DEBUG_COREDUMP_MEMORY_DUMP_THREADS and
CONFIG_DEBUG_COREDUMP_THREAD_STACK_TOP.
For non-current threads, return thread->callee_saved.sp.
For the faulting current thread in stack-top mode, return the
exception-time SP from z_riscv_get_sp_before_exc() (cached during
arch_coredump_info_dump()) instead of thread->callee_saved.sp,
which reflects switch-time state.
Signed-off-by: Mirai SHINJO <oss@mshinjo.com>
Expand the RISC-V coredump register block to all 33 GDB registers
(x0-x31, pc) in register-number order.
Previously only 18 registers were serialized. Populate zero, sp, gp,
tp, s0, and s1-s11 (when available).
Bump ARCH_HDR_VER from 1 to 3 (RISC-V 32-bit layout) and from 2 to 4
(RISC-V 64-bit layout) for the new wire format.
Keep the RISC-V 32-bit block fixed at 33 fields on the RISC-V RV32E
profile; registers not implemented by RV32E remain zero-filled so
version 3 always has a stable size.
Signed-off-by: Mirai SHINJO <oss@mshinjo.com>
When handling an ISR (which does not have a context from which to
restore its own value of LCOUNT), we must clear LCOUNT to prevents
incorrect zero-overhead execution if calling a function such as
memmove() which could be implemented using zero-overhead loop.
A function such as memmove() implemented using zero-overhead loop
assumes LCOUNT to have properly been setup before being called; but
an ISR calling memmove() in assembly, will likely not know that.
Signed-off-by: William Tambe <williamt@cadence.com>
Add CONFIG_PMP_UNLOCK_ROM_FOR_DEBUG option to conditionally disable
the lock bit (L=0) for the ROM region PMP entry. This allows debuggers
running in machine mode to access ROM for setting breakpoints and
reading instructions while preserving userspace protection.
When PMP lock bits are set, they restrict access even in machine mode,
causing "unable to halt hart" errors with hardware debuggers like
OpenOCD. This option provides a surgical fix that only affects the ROM
region - NULL pointer guards and stack guards remain locked to catch
critical bugs during development.
The option integrates with existing PMP_NO_LOCK_GLOBAL configuration
using nested COND_CODE_1 macros and defaults to disabled for production
builds.
Fixes: zephyrproject-rtos/zephyr#82729
Signed-off-by: Alex Lyrakis <alex_gfd@hotmail.com>
This reverts commit 8c02dde437.
For some unknown reasons, xt-clang emits two copies of
z_xt_init_pc if xtensa_mmu_init_paging() is in the same file
as xtensa_mmu_init() and xtensa_mmu_reinit(). So had to
revert the change.
Fixes#103055
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
When using LLEXT, instruction TLB multi-hit becomes a reality
as the same memory space can be occupied by different modules
with different permissions. The ITLB cache may still contain
entries of the unloaded module. So we need to manually
invalidate any cached ITLB corresponding to the exception
address so the TLB associated with the newly loaded module
can be used.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
There is no need to invalidate the whole auto-refilled data TLB
cache when DTLB multi-hit exception is raised. Now it only
invalidates the TLB entries corresponding to the one causing
the DTLB multi-hit. This allows other non-related TLB entries
to remain in the cache so they don't need to be reloaded.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
For both data TLB multi-hit and load/store ring error, we should
return to the interrupted thread immediately so that it can get
past the exception generated code. It is because both of these
exceptions are the result of having cached TLB entries not
aligning to the correct access pattern. So once we have handled
the exception, go back to the interrupted thread to continue
to minimize the chance of having another incompatible TLB being
cached.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
There is no need for an extra switch block to manipulate
the value of is_fatal_error, which defaults to false, and is set
according to the actual exception above. So remove that.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
BTI requires that the C library be compiled with -mbranch-protection to
include BTI landing pads. Newlib from toolchains lacks this support, so
only minimal libc or picolibc built from source (PICOLIBC_USE_MODULE) can
be used with BTI.
Without this, the basic hello_world/ sample fails to execute.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Change the random number generator requirement for per-thread PAC keys
from a hard dependency to a more flexible approach:
- Use 'select CSPRNG_NEEDED' to automatically request cryptographic
RNG support rather than requiring specific RNG options to be
pre-enabled
- Use 'imply TEST_RANDOM_GENERATOR' as a fallback when no real CSPRNG
is available, enabling testing without hardware entropy
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Enable the ARM_PACBTI Kconfig choice for ARM64 architectures (ARMV8_A
and ARMV9_A) in addition to the existing ARM32 ARMV8_1_M_MAINLINE
support. Add the corresponding -mbranch-protection compiler flags to
both GCC and Clang target files for ARM64.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Add support for ARMv8.5+ Branch Target Identification to protect against
Jump-Oriented Programming (JOP) attacks. This complements PAC to offer
complete protection against both ROP and JOP attacks, ensuring
comprehensive control flow integrity.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Add support for ARMv8.3+ Pointer Authentication to protect against
Return-Oriented Programming (ROP) attacks. This implementation provides
PAC functionality with per-thread key isolation, secure key management,
and integration with Zephyr's thread model.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The Xen events channel driver consume 72K of RAM, but may not be
required in all use cases.
Added a XEN_EVENTS Kconfig option so that Xen events can be gracefully
disabled if not required. Updated the relevant CMakeLists.txt and
Kconfig files to guard the inclusion of the Xen events driver and its
source files by this option.
Signed-off-by: Grygorii Strashko <grygorii_strashko@epam.com>
Signed-off-by: Svitlana Drozd <svitlana_drozd@epam.com>
Functions in assembler file pm_s2ram.S are declared with the usual:
SECTION_FUNC(TEXT, <function name>)
Note the first argument (section name) is `TEXT` in capital letters which
a define in `include/zephyr/linker/sections.h` should replace with `text`,
such that the functions are placed in section `.text.<function name>` which
matches the ".text.*" pattern in linker script. However, this file is not
included by pm_s2ram.S: as such, the substitution never happens and the
functions go in `.TEXT.<function name>` instead! This has not caused issues
thanks to a workaround in the Cortex-M linker script, which also has
".TEXT.*" as input section name pattern (unlike all other archs!), but is a
bug nonetheless.
Fix this issue by adding the missing include which ensures the functions
are placed in sections with the proper name.
Signed-off-by: Mathieu Choplain <mathieu.choplain-ext@st.com>
The eponymous function in __aeabi_read_tp.S is declared using:
SECTION_FUNC(TEXT, __aeabi_read_tp)
Note the first argument (section name) is `TEXT` in capital letters which
a define in `include/zephyr/linker/sections.h` should replace with `text`,
such that the function is placed in section `.text.__aeabi_read_tp` which
matches the ".text.*" pattern in linker script. However, this file is not
included by __aeabi_read_tp.S: as such, the substitution never happens and
the function goes in `.TEXT.__aeabi_read_tp` instead! This has not caused
issues thanks to a workaround in the Cortex-M linker script, which also
has ".TEXT.*" as input section name pattern (unlike all other archs!), but
is a bug nonetheless.
Fix this issue by adding the missing include which ensures the function
is placed in a section with the proper name.
Signed-off-by: Mathieu Choplain <mathieu.choplain-ext@st.com>
The Zephyr kernel's generic `z_fatal_error()` function, which is
invoked by architecture-specific fatal error handlers, is not
guaranteed to be non-returning. For instance, it can return if an
essential thread aborts itself.
The RISC-V port's `z_riscv_fatal_error` function was previously
inconsistently marked as `FUNC_NORETURN`. This commit removes this
attribute to align with the core kernel behavior, allowing the
function to return if `z_fatal_error()` returns.
Specific changes include:
- Removed `FUNC_NORETURN` from `z_riscv_fatal_error` declarations
in `fatal.c` and `kernel_arch_func.h`.
- Removed `CODE_UNREACHABLE` after the call to `z_fatal_error`
within `z_riscv_fatal_error` as it can now return.
- In `isr.S`, changed `tail z_riscv_fatal_error` to
`call z_riscv_fatal_error` in the exception entry, followed by
a jump to `check_reschedule` to handle the return path.
- Added `CODE_UNREACHABLE` at call sites of `z_riscv_fatal_error`
(e.g., in `z_riscv_fault`, `z_check_user_fault`,
`arch_irq_spurious`) where the context ensures the call is
effectively terminal.
Signed-off-by: Firas Sammoura <fsammoura@google.com>
Cortex-R5F Technical Reference Manual by Arm says DMINLINE is the Log2 of
the minimum number of words (one word = four bytes) in a cache line.
For instance, say DMINLINE is 3, which means the cache line size is
2^3=8 words or 32 bytes, however with the current calculation, it comes
out to be 16 bytes. Therefore, we fix this calculation by correctly
calculating the number of bytes for the cache line size.
Signed-off-by: Amneesh Singh <amneesh@ti.com>
The arch_float_disable() function was not clearing the K_FP_REGS flag
from thread->base.user_options after disabling FPU access. This caused
the float_disable test to fail as it verifies the flag is properly
cleared after FPU disable.
Signed-off-by: Appana Durga Kedareswara rao <appana.durga.kedareswara.rao@amd.com>
Since we have split source files for page table related stuff
and MMU related stuff, move the MMU initialization functions
from the page table source file into MMU source file.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This adds doxygen doc to the page table source file as
we are missing quite a bit of documentation there.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This changes the existing comments for functions into doxygen
style documentation for functions. Also adds missing doxygen
doc for functions.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
1. it reguires that there are floating point registers,
so the extention f is required. (zfinx uses the int regs instead)
2. RV32E doesn't supports hardware floating-point calling convention.
Signed-off-by: Fin Maaß <f.maass@vogl-electronic.com>
Introduce the missing flag to compile code with Zbkb extension,
which has already been supported by the GCC 12 in current SDK.
Signed-off-by: Andy Lin <andylinpersonal@gmail.com>
Upgrades the thread user_options to 16 bits from an 8-bit value to
provide more space for future values.
Also, as the size of this field has changed, the values for the
existing architecture specific thread options have also shifted
from the upper end of the old 8-bit field, to the upper end of
the new 16-bit field.
Fixes#101034
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
If there are not enough free L2 tables to map all predefined
memory regions at boot, halt the system in case assertion is
not enabled. Without all the needed memory regions mapped,
it is very unlikely that anything will run properly.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Adds some debug logs when we are allocating page tables.
This provides a more visible way of seeing whether
we need to have more free tables.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
