It seems that, at least on tests, it's common to call k_thread_create()
on a thread multiple times. This trips a check for the CET shadow stack
- namely, set a shadow stack on a thread which already has a shadow
stack.
This patch adds a Kconfig option to allow that, iff the base address and
size of the new shadow stack are the same as before. This will trigger a
reset of the shadow stack, so it can be reused.
It may be the case that this behaviour (reusing threads) is more common
than only for tests, in which case it could make sense to change the
default - in this patch, is only true if ZTEST.
Even if being enabled by default becomes the reality, it would still
make sense to keep this option - more conscious apps could avoid the
need for the shadow stack reset code altogether.
Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
So that kernel created threads can use shadow stacks. Note that
CONFIG_X86_CET_SHADOW_STACK is abandoned in favour of
CONFIG_HW_SHADOW_STACK.
This means change some types, functions and macro throughout shadow
stack code.
Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
In order to allow kernel created threads (such as main and idle threads)
to make use of hardware shadow stack implementation, add an interface
for them.
This patch basically provides an infra that architectures need to
implement to provide hardware shadow stack.
Also, main and idle threads are updated to make use of this interface
(if hardware shadow stacks are enabled).
Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
Some SoCs may need to do some preparatory work before changing the
current shadow stack pointer (and thus, currently used shadow stack).
This patch adds a way for that, shielded by a Kconfig
(CONFIG_X86_CET_SOC_PREPARE_SHADOW_STACK_SWITCH).
As currently only 32 bit SoC may use this, support is only added to the
32 bit code.
Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
Most notable difference on base support is the need to keep the shadow
stack tokens, which are 8 bytes, 8 bytes aligned. Some helper macros are
used for that.
Also, an `ssp` entry is added to the task state segment (TSS).
Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
Currently, it's permitted to have threads that don't have a shadow
stack. When those are run, shadow stack is disabled on the CPU. To
identify those, the thread `shstk_addr` member is checked.
This patch adds an optional check, behind
CONFIG_X86_CET_VERIFY_KERNEL_SHADOW_STACK, that checks if an outgoing
thread has this pointer NULL with shadow stack currently enabled on
the CPU, meaning a 1) bug or 2) some attempt to tamper with the pointer.
If the check fails, k_panic() is called. Note that this verification is
not enough to guarantee `shstk_addr` can't be tampered with. For
instance, it only works on a running thread. Ideally, all threads should
be shadow stack capable, so a missing `shstk_addr` would simply be a
hard fault, but that is still to come.
Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
Main peculiarity is that if an exception results in current thread being
aborted, we need to clear the busy bit on the shadow stack on the swap
to the new thread, otherwise future exceptions will fail when trying to
use a busy shadow stack.
Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
Nested interrupts are supported, on the normal stack, by creating a
stack whose size is a multiple of CONFIG_ISR_DEPTH, and updating the
pointer used by Interrupt Stack Table (IST) to point to a new base,
inside the "oversized" stack.
The same approach is used for the shadow stack: shadow stack size is
multiplied by CONFIG_ISR_DEPTH, and the pointer to the stack on the
shadow stack pointer table is update to point to the next base.
Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
For IRQs, shadow stack support a mechanism similar to the Interrupt
Stack Table (IST) for x86_64: a table, indexed by the IST index, pointing
to a 64 byte table in memory containing the address of seven shadow stacks
to be used by the interrupt service routines.
This patch adds support to this mechanism. It is worth noting that, as
Zephyr may exit from an interrupt by going to a different thread than
the one that was interrupted, some housekeeping is done to ensure that
the necessary shadow stack tokens are on the right shadow stack before
return from the interrupt.
Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
Shadow Stack is one of the capabilities provided by Intel Control-flow
Enforcement Technology (CET), aimed at defending against Return Oriented
Programming.
This patch enables it for x86_64 (32-bit support coming in future
patches):
- Add relevant Kconfigs;
- Shadow stacks should live in specially defined memory pages, so
gen_mmu.py was updated to allow that;
- A new macro, Z_X86_SHADOW_STACK_DEFINE, added to define the area
for a shadow stack;
- A new function, z_x86_thread_attach_shadow_stack(), added to
attach a shadow stack to a never started thread;
- locore.S changed to enable/disable shadow stack when a thread
using it comes in/out of execution.
As not all threads are currently shadow stack capable, threads that do
not use it will still run with shadow stack disabled. Ideally, at some
point in the future, all threads would use the shadow stack, so no need
to disable it at all.
Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
Add code to enable it and sprinkle `endbr64` on asm code, where needed.
Namely, IRQs and excepts entrypoints.
Finally, tests added to ensure IBT behaves sanely.
Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
Indirect Branch Tracking (IBT) is one of the capabilities provided by
Intel Control-flow Enforcement Technology (CET), aimed at defending
against Jump/Call Oriented Programming.
This patch enables it for x86 (32-bit, 64-bit support coming in future
patches):
- Add relevant Kconfigs (everything is behind X86_CET);
- Code to enable it;
- Enable compiler flags to enable it;
- Add `endbr32` instructions to asm code, where needed.
Points in the code where an indirect branch is expected to land need
special instructions that tell the CPU they are valid indirect branch
targets. Those are added by the compiler, so toolchain support is
necessary. Note that any code added to the final ELF also need those
markers, such as libc or libgcc.
Finally, tests added to ensure IBT behaves sanely.
Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
This adds exception handling of control protection exception
in fatal code.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
Commit f9168ae464 made all non-cached memory
loadable by default.
However as nocache memory is typically used for reserving larger buffers to
be shared between peripherals, this comes at fairly large cost towards ROM
usage.
This commit creates two distinct sections for both loadable and
non-loadable nocache memory sections.
Signed-off-by: Pieter De Gendt <pieter.degendt@basalte.be>
The RXv2, RXv3 feature with a configuration to be able to relocate
the exception vector table by setting the extb register in the
CPU, this commit support to enable the config and code handling
for it
Signed-off-by: Duy Nguyen <duy.nguyen.xa@renesas.com>
Fix an issue where 1 vector is being requested when MSI-X is
enabled. The previous logic always assumed the PCIE device has
only fixed or single MSI when we are requesting 1 vector, which
is not entirely correct. So if there is no vector allocated
already, try to allocate one.
Fixes#93319
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
arch_pcie_msi_vectors_allocate() has a return type of uint8_t.
One of the error path returns -1 which would result in 255
being returned. So fix that by returning 0 instead, as there is
no vector being allocated anyway.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
If the GDB stub is enabled the exception handler will jump to the GDB
stub to allow remote GDB debugging.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
GCC 14.3 will happily delete any code that appears before
__builtin_unreachable that isn't separated with an obvious branch. That
includes __asm__ statements, even those which generate traps.
The failure case that I debugged was on x86 in
z_check_stack_sentinel. There is a store to restore the sentinel to the
correct value just before the ARCH_EXCEPT, and that macro emits 'int $32'
followed by CODE_UNREACHABLE. Because the compiler didn't understand that
ARCH_EXCEPT was changing execution flow, it decided that the sentinel
restoring store 'couldn't' be reached and elided it.
I added the "memory" clobber to the asm statement in ARCH_EXCEPT before
CODE_UNREACHABLE to enforce that all pending store operations be performed
before the asm statement occurs. This ensures that they are not deleted by
the compiler.
I think this might be a GCC bug. The GCC documentation explicitly documents
that asm statements which change the flow of control should be followed by
__builtin_unreachable.
Signed-off-by: Keith Packard <keithp@keithp.com>
Added helper Kcoinfig option USE_ISR_WRAPPER which can be used to
include isr_wrapper even if GEN_SW_ISR_TABLE is not enabled. This
is needed to enable configurations where only IRQ vector table is
used with multithreading (only direct isr used). This change is
backward compatibible with previous config.
Signed-off-by: Łukasz Stępnicki <lukasz.stepnicki@nordicsemi.no>
Add new API to save and restore SCB context. This is typically useful when
entering and exiting suspend-to-RAM low-power modes.
The scb_context_t and the backup/restore functions are designed to only
handle SCB registers that are:
- Mutable: Their values can be changed by software.
- Configurable: They control system behavior or features.
- Stateful: Their values represent a specific configuration that an
application might want to preserve and restore.
Registers excluded from backup/restore are:
1. CPU/feature identification registers
Motivation: These registers are fixed in hardware and read-only.
2. ICSR (Interrupt Control and State Register)
Motivation: Most bits of ICSR bits are read-only or write-only
and represent volatile system state. STTNS is the only read-write
field and could be considered part of the system state, but it is
only present on certain ARMv8-M CPUs, and Zephyr does not use it.
3. CFSR (Configurable Fault Status Register)
HFSR (HardFault Status Register)
DFSR (Debug Fault Status Register)
AFSR (Auxiliary Fault Status Register)
MMFAR (MemManage Fault Address Register)
BFAR (BusFault Address Register)
Motivation: These registers are read/write-one-to-clear and
contain only fault-related information (which is volatile).
Co-authored-by: Mathieu Choplain <mathieu.choplain@st.com>
Signed-off-by: Michele Sardo <msmttchr@gmail.com>
Signed-off-by: Mathieu Choplain <mathieu.choplain@st.com>
- There are linker file directives that must come at the
start of the noinit region. For example, the directive
that allow that section to not exist in RAM before a
certain address (. = MAX(ABSOLUTE(.), 0x34002000);).
- Before this update, those could only be added to the end
of that region. They will now have the option to be at the
beginning or the end.
Signed-off-by: Bill Waters <bill.waters@infineon.com>
More complex suspend and resume scheme might require exactly defined
location of this variable due to platform peculiar SW and HW requirement.
DTS zephyr,memory-region node with nodelabel `pm_s2ram` shall be used to
automatic definition of linker section for such objective.
Signed-off-by: Andrzej Puzdrowski <andrzej.puzdrowski@nordicsemi.no>
Currently this directive is not supported in EWARM 9.70.1,
it will be in future versions, but we want Zephyr 4.2
to work with IAR EWARM 9.70.1.
Signed-off-by: Robin Kastberg <robin.kastberg@iar.com>
Add initial support for the Cortex-M52 Core which is an implementation
of the Armv8.1-M mainline architecture.
The support is based on the Cortex-M55 support that already exists in
Zephyr.
Signed-off-by: Jisheng Zhang <jszhang@kernel.org>
Always use the cmsis_6 version for DWT_LSR_Present_Msk and
DWT_LSR_Access_Msk, the old ones are not going to be available anymore
when Cortex-M is selected..
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
Add struct z_mpu_context_retained and parameterized functions
z_arm_save_mpu_context() and z_arm_restore_mpu_context() to allow
saving and restoring MPU configuration to/from retained RAM.
This enables preserving MPU state across deep sleep or power-down
modes on Cortex-M devices.
The API is exposed under include/zephyr/arch/arm/mpu/arm_mpu.h
and implemented in arch/arm/core/mpu/arm_mpu.c.
Signed-off-by: Michele Sardo <msmttchr@gmail.com>
Fixes a flaw in implementations of arch_is_in_isr() that could manifest
on SMP systems. If the reading of the current CPU's nested interrupt
count is not fully atomic on an SMP system, then an ill-timed context
switch could occur leaving the caller reading the nested interrupt
count of a different CPU.
This also applies a little defensive programming to cortex_a_r's
arch_is_in_nested_exception(). Although this routine is presently
only called with interrupts locked (which will prevent the thread
from migrating), switching to use _current_cpu instead of
arch_curr_cpu() is safer as should the routine ever be called
without meeting the locking criteria, it can be detected and fixed.
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
Fixes a flaw in the the xtensa implementation of arch_is_in_isr()
that could manifest on SMP systems. If the reading of the current
CPU's nested interrupt count is not fully atomic on an SMP system,
then an ill-timed context switch could occur leaving the caller
reading the nested interrupt of a different CPU.
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
Since #75677, it has not been possible to build with
CONFIG_NULL_POINTER_EXCEPTION_DETECTION_DWT enabled since there was no
declaration of z_arm_debug_enable_null_pointer_detection before its use
in arch/arm/core/cortex_m/prep_c.c.
This change creates an arch/arm/include/cortex_m/debug.h header that
contains declarations for the functions in
arch/arm/core/cortex_m/debug.c.
Signed-off-by: Tom Hughes <tomhughes@chromium.org>
When building with user space and LLEXT support, syscall helpers need
to be exported.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
Add needed backtrace helpders routines and enable
backtrace for the Xtensa Fusion F1 DSP in the
IMXRT595S.
Signed-off-by: Mike J. Chen <mjchen@google.com>
This commit introduces the SRAM_SW_ISR_TABLE option which is selected by
DYNAMIC_INTERRUPT. It allows splitting the DYNAMIC_INTERRUPT option into
two parts:
- One for the relocation of the ISR vector table in RAM
- One for the inclusion of functions needed to install ISRs dynamically
The goal is to later only select the relocation of the ISR vector table in
RAM and not all the associated functions from the dynamic interrupt
mechanism.
Signed-off-by: Martin Hoff <martin.hoff@silabs.com>
GICC_IAR (GICv2) includes the source processor for SGIs in bits 12-10.
Mask them away otherwise IPIs sent from any CPU other than CPU0 will be
considered out of bounds.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Introduce CONFIG_RISCV_NO_MTVAL_ON_FP_TRAP to handle implementations
where the mtval CSR does not provide useful information during
floating-point illegal instruction exceptions.
The RISC-V specification states that mtval is "either set to zero or
written with exception-specific information" on traps. Some
implementations, including QEMU, do not populate mtval with the
faulting instruction value during FP-related illegal instruction
exceptions, making it unusable for FP exception handling.
Previously, this behavior was hardcoded for QEMU targets only, but
other CPU implementations may also lack useful mtval content for FP
traps. Decoupling this from CONFIG_QEMU_TARGET and allows other
platforms to properly declare this limitation.
The new Kconfig option defaults to enabled for QEMU targets to
maintain backward compatibility.
Signed-off-by: Afonso Oliveira <afonsoo@synopsys.com>
This reverts commit a90a47b1c9.
This commit was written with CMSIS 5 in mind, where some Cortex-M cores
have "SHP" in the SCB_Type, and some have "SHPR". This is not correct as
Zephyr is *supposed* to be using CMSIS 6 for Cortex-M... but CI actually
picks up CMSIS 5 instead (it includes both with CMSIS 5 taking priority).
The end result is that Zephyr's CI builds this happily but it causes build
failures on downstream users (e.g., example-application).
Revert the commit now, as it is not used yet by anyone. The revised version
using only "SHPR" shall be reintroduced once the CI issue has been fixed.
Signed-off-by: Mathieu Choplain <mathieu.choplain@st.com>
Add two API to save SCB context and restore it, typically
used in suspend to RAM use case.
The scb_context_t and the backup/restore functions are designed to only
handle SCB registers that are:
- Mutable: Their values can be changed by software.
- Configurable: They control system behavior or features.
- Stateful: Their values represent a specific configuration that an
application might want to preserve and restore.
Register excluded from backup/restore are:
1. CPUID (CPUID Base Register)
Motivation for Exclusion: This is a read-only identification register.
2. ICSR (Interrupt Control and State Register)
Motivation for Exclusion (from restoration): While its current value
can be read, directly restoring a saved ICSR value is highly
dangerous and generally unsafe in an RTOS context.
Contains Read-Only Status Bits: A significant portion of ICSR
consists of read-only bits (VECTACTIVE, VECTPENDING, ISRPREEMPT,
TSRUNPEND). These bits reflect the current state of the exception
system (e.g., which exception is active, which are pending) and are
managed dynamically by the CPU and the RTOS.
Forcing a previous state onto these bits would corrupt the live
system's interrupt handling.
Contains Write-Only Set/Clear Bits: Some bits are write-only to set
or clear a pending interrupt (PENDSVSET, PENDSVCLR, SYSTICKSET,
SYSTICKCLR). If these bits were set in the saved context, restoring
them might immediately trigger an interrupt or change its pending state
unexpectedly, outside the RTOS's control.
RTOS Management: In Zephyr (and other RTOSes), the kernel tightly
manages the interrupt and exception state.
Direct manipulation of ICSR's volatile bits could conflict with the
RTOS's internal state machine, leading to crashes or unpredictable
behavior.
3. CFSR (Configurable Fault Status Register)
Motivation for Exclusion: This is a read-only status register that
reports the current state of Memory Management, Bus Fault, and Usage
Faults. It's used by fault handlers to determine the cause of a fault.
4. HFSR (HardFault Status Register)
Motivation for Exclusion: Similar to CFSR, this is a read-only status
register that reports the current state of HardFaults. It's for
reporting, not for configuration or restoration.
5. DFSR (Debug Fault Status Register)
Motivation for Exclusion: This is a read-only status register that
reports debug-related faults. It's primarily used by debuggers and
is not part of the application's runtime context to be saved/restored.
6. MMFAR (MemManage Fault Address Register)
Motivation for Exclusion: This is a read-only register that stores the
address that caused a Memory Management fault. It's a diagnostic
register, not a configurable parameter.
7. BFAR (BusFault Address Register)
Motivation for Exclusion: Similar to MMFAR, this is a read-only
register that stores the address that caused a BusFault. It's a
diagnostic register.
8. AFSR (Auxiliary Fault Status Register)
Motivation for Exclusion: This register is implementation-defined and
read-only.
Signed-off-by: Michele Sardo <msmttchr@gmail.com>
When lazy HiFi context switching is enabled, the system starts with
the HiFi coprocessor disabled. Should the thread use that coprocessor,
it will generate an exception which in turn will enable the coprocessor
and save/restore the HiFi registers as appropriate. When switching
to a new thread, the HiFi coprocessor is again disabled.
For simplicity, there are no restrictions as to which thread is allowed
to use the coprocessor.
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
The intent of arch_coprocessors_disable() is to replace
arch_float_disable() in halt_thread() for the FPU will not
always be the only coprocessor that will need to be disabled.
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
As the BSA can not be used when lazy HiFi context switching is
used, a more permanent and predictable location in which to store
the registers is required. To this end ...
1. reserve some space in the arch-specific portion of the k_thread
structure for those registers.
2. clear that region when the thread is created.
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
