Do not directly include and use APIs from ksched.h outside of the
kernel. For now do this using more suitable (ipi.h and
kernel_internal.h) internal APIs until more cleanup is done.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
To support unprivileged mode (CONFIG_USERSPACE):
- Set unprivileged PAC key registers when system is in unprivileged
mode.
- Add `bti` after each svc call, to make sure that the indirect jumps on
`lr` while returning from an `svc` don't result in a usage fault.
Signed-off-by: Sudan Landge <sudan.landge@arm.com>
Add a config option to set unique PAC keys per thread and
make sure to retain them during context switch.
Signed-off-by: Sudan Landge <sudan.landge@arm.com>
As part of enabling PACBTI support:
- Add config options to enforce PAC and BTI features
- Enable these config options based on the branch protection choice
selected for `ARM_PACBTI`
- Enforce PACBTI, based on the new config options, by enabling
corresponding PACBTI bits in CONTROL register and in FVP.
Signed-off-by: Sudan Landge <sudan.landge@arm.com>
Rename and move PACBTI config options to common Kconfig so that
they could be re-used for arm64 in the future.
Signed-off-by: Sudan Landge <sudan.landge@arm.com>
z_prep_c does not return, mark it as such consistently across
architectures. We had some arches do that, others not. This resolves a
few coding guideline violations in arch code.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Do not use private API prefix and move to architecture interface as
those functions are primarily used across arches and can be defined by
the architecture.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Do not use private API prefix and move to architecture interface as
those functions are primarily used across arches and can be defined by
the architecture.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Do not use private API prefix and move to architecture interface as
those functions are primarily used across arches and can be defined by
the architecture.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Do not use private API prefix and move to architecture interface as
those functions are primarily used across arches and can be defined by
the architecture.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Cleanup init.c code and move early boot code into arch/ and make it
accessible outside of the boot process/kernel.
All of this code is not related to the 'kernel' and is mostly used
within the architecture boot / setup process.
The way it was done, some soc code was including kernel_internal.h
directly, which shouldn't be done.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Not really a kernel feature, more for architecture, which is reflected
in how XIP is enabled and tested. Move it to architecture code to keep
which much of the 'implementation' and usage is.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
GCC 14.3 will happily delete any code that appears before
__builtin_unreachable that isn't separated with an obvious branch. That
includes __asm__ statements, even those which generate traps.
The failure case that I debugged was on x86 in
z_check_stack_sentinel. There is a store to restore the sentinel to the
correct value just before the ARCH_EXCEPT, and that macro emits 'int $32'
followed by CODE_UNREACHABLE. Because the compiler didn't understand that
ARCH_EXCEPT was changing execution flow, it decided that the sentinel
restoring store 'couldn't' be reached and elided it.
I added the "memory" clobber to the asm statement in ARCH_EXCEPT before
CODE_UNREACHABLE to enforce that all pending store operations be performed
before the asm statement occurs. This ensures that they are not deleted by
the compiler.
I think this might be a GCC bug. The GCC documentation explicitly documents
that asm statements which change the flow of control should be followed by
__builtin_unreachable.
Signed-off-by: Keith Packard <keithp@keithp.com>
Add new API to save and restore SCB context. This is typically useful when
entering and exiting suspend-to-RAM low-power modes.
The scb_context_t and the backup/restore functions are designed to only
handle SCB registers that are:
- Mutable: Their values can be changed by software.
- Configurable: They control system behavior or features.
- Stateful: Their values represent a specific configuration that an
application might want to preserve and restore.
Registers excluded from backup/restore are:
1. CPU/feature identification registers
Motivation: These registers are fixed in hardware and read-only.
2. ICSR (Interrupt Control and State Register)
Motivation: Most bits of ICSR bits are read-only or write-only
and represent volatile system state. STTNS is the only read-write
field and could be considered part of the system state, but it is
only present on certain ARMv8-M CPUs, and Zephyr does not use it.
3. CFSR (Configurable Fault Status Register)
HFSR (HardFault Status Register)
DFSR (Debug Fault Status Register)
AFSR (Auxiliary Fault Status Register)
MMFAR (MemManage Fault Address Register)
BFAR (BusFault Address Register)
Motivation: These registers are read/write-one-to-clear and
contain only fault-related information (which is volatile).
Co-authored-by: Mathieu Choplain <mathieu.choplain@st.com>
Signed-off-by: Michele Sardo <msmttchr@gmail.com>
Signed-off-by: Mathieu Choplain <mathieu.choplain@st.com>
More complex suspend and resume scheme might require exactly defined
location of this variable due to platform peculiar SW and HW requirement.
DTS zephyr,memory-region node with nodelabel `pm_s2ram` shall be used to
automatic definition of linker section for such objective.
Signed-off-by: Andrzej Puzdrowski <andrzej.puzdrowski@nordicsemi.no>
Currently this directive is not supported in EWARM 9.70.1,
it will be in future versions, but we want Zephyr 4.2
to work with IAR EWARM 9.70.1.
Signed-off-by: Robin Kastberg <robin.kastberg@iar.com>
Add initial support for the Cortex-M52 Core which is an implementation
of the Armv8.1-M mainline architecture.
The support is based on the Cortex-M55 support that already exists in
Zephyr.
Signed-off-by: Jisheng Zhang <jszhang@kernel.org>
Always use the cmsis_6 version for DWT_LSR_Present_Msk and
DWT_LSR_Access_Msk, the old ones are not going to be available anymore
when Cortex-M is selected..
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
Add struct z_mpu_context_retained and parameterized functions
z_arm_save_mpu_context() and z_arm_restore_mpu_context() to allow
saving and restoring MPU configuration to/from retained RAM.
This enables preserving MPU state across deep sleep or power-down
modes on Cortex-M devices.
The API is exposed under include/zephyr/arch/arm/mpu/arm_mpu.h
and implemented in arch/arm/core/mpu/arm_mpu.c.
Signed-off-by: Michele Sardo <msmttchr@gmail.com>
Fixes a flaw in implementations of arch_is_in_isr() that could manifest
on SMP systems. If the reading of the current CPU's nested interrupt
count is not fully atomic on an SMP system, then an ill-timed context
switch could occur leaving the caller reading the nested interrupt
count of a different CPU.
This also applies a little defensive programming to cortex_a_r's
arch_is_in_nested_exception(). Although this routine is presently
only called with interrupts locked (which will prevent the thread
from migrating), switching to use _current_cpu instead of
arch_curr_cpu() is safer as should the routine ever be called
without meeting the locking criteria, it can be detected and fixed.
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
Since #75677, it has not been possible to build with
CONFIG_NULL_POINTER_EXCEPTION_DETECTION_DWT enabled since there was no
declaration of z_arm_debug_enable_null_pointer_detection before its use
in arch/arm/core/cortex_m/prep_c.c.
This change creates an arch/arm/include/cortex_m/debug.h header that
contains declarations for the functions in
arch/arm/core/cortex_m/debug.c.
Signed-off-by: Tom Hughes <tomhughes@chromium.org>
This reverts commit a90a47b1c9.
This commit was written with CMSIS 5 in mind, where some Cortex-M cores
have "SHP" in the SCB_Type, and some have "SHPR". This is not correct as
Zephyr is *supposed* to be using CMSIS 6 for Cortex-M... but CI actually
picks up CMSIS 5 instead (it includes both with CMSIS 5 taking priority).
The end result is that Zephyr's CI builds this happily but it causes build
failures on downstream users (e.g., example-application).
Revert the commit now, as it is not used yet by anyone. The revised version
using only "SHPR" shall be reintroduced once the CI issue has been fixed.
Signed-off-by: Mathieu Choplain <mathieu.choplain@st.com>
Add two API to save SCB context and restore it, typically
used in suspend to RAM use case.
The scb_context_t and the backup/restore functions are designed to only
handle SCB registers that are:
- Mutable: Their values can be changed by software.
- Configurable: They control system behavior or features.
- Stateful: Their values represent a specific configuration that an
application might want to preserve and restore.
Register excluded from backup/restore are:
1. CPUID (CPUID Base Register)
Motivation for Exclusion: This is a read-only identification register.
2. ICSR (Interrupt Control and State Register)
Motivation for Exclusion (from restoration): While its current value
can be read, directly restoring a saved ICSR value is highly
dangerous and generally unsafe in an RTOS context.
Contains Read-Only Status Bits: A significant portion of ICSR
consists of read-only bits (VECTACTIVE, VECTPENDING, ISRPREEMPT,
TSRUNPEND). These bits reflect the current state of the exception
system (e.g., which exception is active, which are pending) and are
managed dynamically by the CPU and the RTOS.
Forcing a previous state onto these bits would corrupt the live
system's interrupt handling.
Contains Write-Only Set/Clear Bits: Some bits are write-only to set
or clear a pending interrupt (PENDSVSET, PENDSVCLR, SYSTICKSET,
SYSTICKCLR). If these bits were set in the saved context, restoring
them might immediately trigger an interrupt or change its pending state
unexpectedly, outside the RTOS's control.
RTOS Management: In Zephyr (and other RTOSes), the kernel tightly
manages the interrupt and exception state.
Direct manipulation of ICSR's volatile bits could conflict with the
RTOS's internal state machine, leading to crashes or unpredictable
behavior.
3. CFSR (Configurable Fault Status Register)
Motivation for Exclusion: This is a read-only status register that
reports the current state of Memory Management, Bus Fault, and Usage
Faults. It's used by fault handlers to determine the cause of a fault.
4. HFSR (HardFault Status Register)
Motivation for Exclusion: Similar to CFSR, this is a read-only status
register that reports the current state of HardFaults. It's for
reporting, not for configuration or restoration.
5. DFSR (Debug Fault Status Register)
Motivation for Exclusion: This is a read-only status register that
reports debug-related faults. It's primarily used by debuggers and
is not part of the application's runtime context to be saved/restored.
6. MMFAR (MemManage Fault Address Register)
Motivation for Exclusion: This is a read-only register that stores the
address that caused a Memory Management fault. It's a diagnostic
register, not a configurable parameter.
7. BFAR (BusFault Address Register)
Motivation for Exclusion: Similar to MMFAR, this is a read-only
register that stores the address that caused a BusFault. It's a
diagnostic register.
8. AFSR (Auxiliary Fault Status Register)
Motivation for Exclusion: This register is implementation-defined and
read-only.
Signed-off-by: Michele Sardo <msmttchr@gmail.com>
The intent of arch_coprocessors_disable() is to replace
arch_float_disable() in halt_thread() for the FPU will not
always be the only coprocessor that will need to be disabled.
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
instead of depending on logging for exceptions and fatal messages, use
macro and dump exceptions if printk is enable with no logging.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
z_arm_save_fp_context and z_arm_restore_fp_context
save and restore fpu context regardless of the
CONFIG_FPU_SHARING setting.
This modification is required to support suspend to ram use cases
where save and restore of FPU state is needed to ensure proper bahaviour
after wakeup.
Signed-off-by: Michele Sardo <michele.sardo@st.com>
When LTO is enabled, linker may relocate literal pools out of Thumb LDR
instruction's reach causing "offset out of range" errors while linking.
This commit adds `.ltorg` directive in the inline asm blocks where absolute
addresses are loaded using the `ldr` instructions, in order to ensure that
the literal pool containing the absolute addresses are placed near the
`ldr` instructions.
Note that the `.ltorg` directive is recognised by all toolchains supported
by Zephyr and no toolchain abstraction is provided for now.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
Utilize a code spell-checking tool to scan for and correct spelling errors
in `Kconfig` files within the `arch`, `boards`, `kernel`, `modules`,
`samples`, and `share` directory.
Additionally, incorporates a fix recommended by the reviewer.
Signed-off-by: Pisit Sawangvonganan <pisit@ndrsolution.com>
Floating-Point System ID register (FPSID) = 0x41033094
AArch32 Media and VFP Feature Register 0 (MVFR0) = 0x10110222
AArch32 Media and VFP Feature Register 0 (MVFR1) = 0x1111111
MVFR1 SIMDFMAC, bits [31:28] = 0; FMAC is not supported
Signed-off-by: Simon Maurer <mail@maurer.systems>
When not using dynamic interrupt mapping, various interrupt tables are
configured to be stored in read-only memory in the linker script.. Mark
them const so that the linker doesn't complain.
This affects _sw_isr_table, _irq_vector_table, and z_shared_sw_isr_table in
arch/common along with _VectorTable in arch/arc.
Signed-off-by: Keith Packard <keithp@keithp.com>
move mov r7, #0 to use it on both branches as aarch64
implementation uses wzr which returns 0 on read
but r7 might have an undefined value after a reset.
Signed-off-by: Julien Racki <julien.racki@st.com>
The difference between __irq_disable() and irq_lock() is that the former
essentially translates to `cpsid i`, whereas `irq_lock()` translates to
setting BASEPRI (on cores with BASEPRI). This means that using
irq_lock() does not disable zero-latency interrupts (ZLIs), which
reduces the potential execution latency of ZLIs.
In both isr_wrapper and _arch_isr_direct_pm() (which is just
really an implementation of ISR_DIRECT_PM()), we were using
__irq_disable() to disable all interrups, including ZLIs. But the code
executed with interrupts disabled handles waking up from idle, and so
must only be protected against regular interrupts being executed, not
ZLIs, which should have no effect on the correct execution of the code.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Commit 87719828ac introduced FSR definitions
for ARMv7-A/R; however, the value 16 which is documented in the ARM[1] as
a "TLB Conflict Abort", was introduced as "TLB Conflict Fault" instead and
described as a "Table Conflict Fault".
Update all affected files to use the ARM's naming for this error instead.
[1] Architecture Reference Manual (Document ID: ARM DDI 0406C.d)
"ARM Architecture Reference Manual for ARMv7-A and ARMv7-R Edition"
Signed-off-by: Mathieu Choplain <mathieu.choplain@st.com>
Introduce a Kconfig choice to select Pointer Authentication and Branch
Target Identification (PACBTI) features on Armv8.1-M Mainline CPUs.
The available options map to supported `-mbranch-protection` levels
(`standard`, `pac-ret`, `bti`, and combinations) documented at
https://gcc.gnu.org/onlinedocs/gcc/ARM-Options.html
These options enable hardware-assisted control-flow integrity mechanisms on
targets like Cortex-M85, and require a toolchain with PACBTI support, such
as GCC 14.2 or newer.
Signed-off-by: Sudan Landge <sudan.landge@arm.com>
Due to erratum 1013783-B, speculative accesses might be performed
to normal memory unmapped in the MPU. This can be avoided by using
MPU region 0 to cover all unmapped memory and make this region
execute-never and inaccessible.
Note that this applies when CPU Cortex-M7 implements MPU and D-cache.
Signed-off-by: Manuel Argüelles <manuel.arguelles@nxp.com>
Zephyr switched to using CMSIS_6 module in f726cb51
which breaks certain boards like `nucleo_h745zi_q/stm32h745xx/m7` when
CONFIG_CORTEX_M_DWT, CONFIG_TIMING_FUNCTIONS are enabled and cmsis from
`module/hal/cmsis` is not available (deleted explicitly after west
update).
This commit adds a provision to be able to use CMSIS_6 macros when the
module cmsis is not available.
Signed-off-by: Sudan Landge <sudan.landge@arm.com>
The original 'arch_pm_s2ram_resume' implementation saved lr on the stack
using 'push {lr}' and restored it using 'pop {lr}'. However, the Thumb-1
'pop' does not support lr as a target register, so this code would not
compile for ARMv6-M or ARMv8-M Baseline. r0 was added to these push/pop
later in 2590c48d40.
In 474d4c3249, arch_pm_s2ram* functions were
modified to no longer use the stack, which incidentally "fixed" this issue.
b4fb5d38eb reverted this commit and brought
back 'pop {r0, lr}' as-is, without taking compatibility into account.
Modify the sequence to use "pop {r0, pc}" which is supported on all
ARM M-profile implementations (v6/v7/v8 Baseline/v8 Mainline), and
add comments to (hopefully) prevent this issue from re-appearing.
Signed-off-by: Mathieu Choplain <mathieu.choplain@st.com>
On the Xilinx MPSoC (Cortex-R5) platform, erratic operation was often
seen when an operation which disabled the dcache, such as sys_reboot,
was performed. Usually this manifested as an undefined instruction trap
due to the CPU jumping to an invalid memory address.
It appears the problem was due to dirty cache lines being present at the
time the cache is disabled. Once the cache is disabled, the CPU will
ignore the cache contents and read the possibly out-of-date data in main
memory. Likewise, since the cache was being cleaned after it was already
disabled, if the CPU had already written through changes to some memory
locations, cleaning the cache at that point would potentially overwrite
those changes with older data.
The fact that the arch_dcache_flush_and_invd_all function was being
called to do the cleaning and invalidation also contributed to this
problem, because it is a non-inline function which means the compiler
will generate memory writes to the stack when the function is called and
returns. Corruption of the stack can result in the CPU ending up jumping
to garbage addresses when trying to return from functions.
To avoid this problem, the cache is now cleaned and invalidated prior to
the dcache being disabled. This is done by directly calling the
L1C_CleanInvalidateDCacheAll function, which, as it is declared as force
inline, should help ensure there are no memory accesses, which would
populate new cache lines, between the cache cleaning and disabling the
cache.
Ideally, for maximum safety, the cache cleaning and cache disabling
should be done in assembler code, to guarantee that there are no memory
accesses generated by the compiler during these operations. However, the
present change does appear to solve this issue.
Signed-off-by: Robert Hancock <robert.hancock@calian.com>
This file previously defined an MPU access permission mode of 0x7 which
corresponded to privileged read-only, unprivileged read-only, similar to
mode 0x6. However, it appears that at least Cortex-R5 does not support
this mode, defining 0x7 as UNP (Unpredictable) or a value which should
not be used.
This value was in turn referenced by the REGION_FLASH_ATTR macro, which
caused the offending value to be used when a memory region was declared
as DT_MEM_ARM(ATTR_MPU_FLASH) in the device tree, causing such regions
to not work properly on Cortex-R5.
Since 0x6 is supported by both Cortex-M and Cortex-R and does the same
thing, there is no reason to use 0x7. Remove the RO_Msk definition which
referenced it, and change REGION_FLASH_ATTR to use P_RO_U_RO_Msk instead.
Signed-off-by: Robert Hancock <robert.hancock@calian.com>
Add Kconfig option to dump only a portion of stack from the
current stack pointer to the stack end. This is enough to
let gdb reconstruct the stack trace and can significantly
reduce the dump size. This is crucial if the core dump needs
to be sent over radio.
Additionally, add another option to set the limit for the
dumped stack portion.
Signed-off-by: Damian Krolik <damian.krolik@nordicsemi.no>
There is no need for kernel/internal/smp.h as SOF does not call
z_sched_ipi(). Actually... git log over there has no mention of
z_sched_ipi() anywhere, just arch_sched_ipi().
And include <ksched.h> for source using z_sched_ipi() since
they are using scheduling functions, and would be the correct
file to include.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
