When a device using a second or third level interrupt makes a request to
set priority, the request should be routed via the multilevel interrupt
API. Otherwise, we will configure the interrupt priority for one of the
core interrupts on the ARC controller (which is usually not what the API
caller wants)
This occurs because the ARC interrupt priority register encodes
interrupts from 0-255, so a multilevel interrupt number would simply be
masked to the lowest 8 bits (which matches the interrupt number of that
device's parent controller)
Signed-off-by: Daniel DeGrasse <ddegrasse@tenstorrent.com>
Signed-off-by: Alex Apostolu <aapostolu@tenstorrent.com>
Update the arch_irq_enable/disable functions to support multilevel
interrupts on the ARC architecture.
Signed-off-by: Aaron Fong <afong@tenstorrent.com>
Signed-off-by: Alex Apostolu <apostolu240@gmail.com>
arch_buffer_validate() on ARC computes the end address of the
validated range using unprotected unsigned addition. When addr+size
wraps around the 32-bit address space the check can succeed for a
buffer that spans non-user-accessible memory, violating the
arch_buffer_validate() contract.
Add overflow detection using u32_add_overflow(), mirroring the fix
already applied to ARM32 (PR #23239) and Xtensa (PR #109000).
Handle size == 0 explicitly so that size - 1 does not underflow.
This affects ARC MPUv2/v3/v4/v6/v8 implementations.
Fixes#113832
Signed-off-by: Hongquan Li <hongquan.li@processmission.com>
We already determine whether the target is PMSAv8 or (PMSAv6/PMSAv7)
using the selected architecture profile (ARMv6-M/ARMv7-M or ARMv8-M
Baseline/Mainline) and define a symbol accordingly. Make use of that
symbol everywhere instead of repeating CPU checks all over arch code.
While at it, drop the allowlist which generated a build error when the
MPU was enabled on an "unknown" CPU. The header is only included from
SoC-specific code which knows whether an MPU exists or not, or generic
code gated behind CONFIG_ARM_MPU, which guarantees that the MPU exists
as CONFIG_ARM_MPU depends on CONFIG_CPU_HAS_ARM_MPU.
Note: this tangentially fixes a bug that misclassified ARMv8-R targets
as using the PMSAv6/PMSAv7 MPU. This bug did not seem to have any
adverse effect though...
Signed-off-by: Mathieu Choplain <mathieu.choplain-ext@st.com>
Xtensa architecture does not have native inter-processor
interrupts and must rely on mechanism on SoC or board to do
that. Because of this, the architecture should not select
CONFIG_ARCH_HAS_DIRECTED_IPIS and should let SoC or board
layer to do that, where CONFIG_SCHED_IPI_SUPPORTED is also
being selected.
Note that CONFIG_XTENSA_LAZY_HIFI_SHARING is now dependent
on CONFIG_ARCH_HAS_DIRECTED_IPIS as it uses directed IPI.
Due to that it keeps sending IPI under the other has saved
HIFI state, it is not really appopriate to broadcast IPIs.
That's why it can only be enabled when directed IPI is
supported.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
The Xtensa implementation validated the whole maxsize range with
arch_buffer_validate() before calling strnlen(), which breaks the
k_usermode_string_nlen() contract in two ways:
- With maxsize == 0 the string must not be touched at all, but the
validation probed the MPU/MMU region containing the pointer anyway
(alignment rounds a zero size up to a full region), wrongly
failing with -1 for inaccessible addresses.
- The validation required user mode accessibility, while the API
contract explicitly does not guarantee it: user access is checked
by the syscall marshalling layer via K_SYSCALL_MEMORY_READ() once
the length is known. Kernel mode callers measuring kernel strings
(e.g. a buffer on the kernel stack) were wrongly rejected.
Return early for maxsize == 0, and check kernel mode readability
with the new xtensa_buffer_is_kernel_readable() (MPU and MMU
variants) instead of arch_buffer_validate().
On the MMU side, extract a shared mem_buffer_validate() that takes a
ring parameter, so arch_buffer_validate() and the new
xtensa_buffer_is_kernel_readable() share the same logic without
duplication.
Fixeszephyrproject-rtos/zephyr#113722
Signed-off-by: Hongquan Li <hongquan.li@processmission.com>
The loop read s[counter] before checking whether maxsize bytes had
been examined (lp_count was set to maxsize + 1 and tested only after
the load), so it read s[maxsize] when the first maxsize bytes were
all non-NUL and dereferenced s[0] when maxsize == 0, violating the
strnlen() semantics required by arch_user_string_nlen(). The
over-read byte cannot change the returned length, but if it is not
accessible the exception fixup reports a spurious error and callers
reject valid input with EFAULT.
Set lp_count to maxsize and check it before the load instead.
Verified by build and disassembly for qemu_arc/qemu_arc_em; no ARC
simulator is available locally, runtime coverage is expected from
the syscalls suite on nsim platforms in CI.
Fixeszephyrproject-rtos/zephyr#113722
Signed-off-by: Hongquan Li <hongquan.li@processmission.com>
Both the ia32 and intel64 implementations loaded s[counter] before
checking counter == maxsize, so they read s[maxsize] when the first
maxsize bytes were all non-NUL and dereferenced s[0] when
maxsize == 0, violating the strnlen() semantics required by
arch_user_string_nlen(). The over-read byte cannot change the
returned length, but if it is not accessible the exception fixup
reports a spurious error and callers reject valid input with EFAULT.
Check the limit before the load instead.
Fixeszephyrproject-rtos/zephyr#113722
Signed-off-by: Hongquan Li <hongquan.li@processmission.com>
The loop loaded s[counter] before checking counter == maxsize, so it
read s[maxsize] when the first maxsize bytes were all non-NUL and
dereferenced s[0] when maxsize == 0, violating the strnlen()
semantics required by arch_user_string_nlen(). The over-read byte
cannot change the returned length, but if it is not accessible the
exception fixup reports a spurious error and callers reject valid
input with EFAULT.
Check the limit before the load instead.
Fixeszephyrproject-rtos/zephyr#113722
Signed-off-by: Hongquan Li <hongquan.li@processmission.com>
The loop loaded s[counter] before checking counter == maxsize, so it
read s[maxsize] when the first maxsize bytes were all non-NUL and
dereferenced s[0] when maxsize == 0, violating the strnlen()
semantics required by arch_user_string_nlen(). The over-read byte
cannot change the returned length, but if it is not accessible the
exception fixup reports a spurious error and callers reject valid
input with EFAULT.
Check the limit before the load instead.
Fixeszephyrproject-rtos/zephyr#113722
Signed-off-by: Hongquan Li <hongquan.li@processmission.com>
This reverts commit c18885e95f:
the issue that this commit worked around was fixed in the
previous commit by preserving the original stack alignment.
Signed-off-by: Laurie Fay <Laurie.Fay@arm.com>
Preserve the alignment indication and rebuild the hardware frame with its
padding word to ensure the stack is always 8-byte aligned when returning
from exception: this is required on Armv8-M.
When an exception is taken with a stack pointer that is not 8-byte
aligned, Armv8-M, and Armv7-M when configured, adds a 4-byte padding word
to align the exception frame. Bit 9 of xPSR records the presence of this
padding.
However, arm_m_switch_to_cpu() does not restore the padding word when
rebuilding the hardware frame. The existing workaround clears the xPSR
alignment bit, which is sufficient on Armv7-M but is invalid on Armv8-M.
Signed-off-by: Laurie Fay <Laurie.Fay@arm.com>
Declare dummy as a read-write output operand, to reflect how it
is used in the assembly block.
The inline assembly overwrites dummy before reading it, but declares the
operand as input-only. The compiler may therefore assume that the backing
register is unchanged and reuse it for another live value, even though the
assembly clobbers it.
Signed-off-by: Laurie Fay <Laurie.Fay@arm.com>
arch_timing_cycles_to_ns() multiplied before it divided, so the
intermediate cycles * NSEC_PER_SEC left the 64-bit range at 1.8e10
cycles. On a 3.2 GHz TSC that is under six seconds of uptime, after
which the returned time wraps back towards zero.
Anything sampling this clock for longer than that sees time jump
backwards. It shows up plainly in CTF tracing, where a capture longer
than a few seconds is rejected outright by babeltrace2: a stream whose
timestamps are not monotonic cannot be read, and per-CPU streams
cannot be merged.
Divide first and fold the remainder back in. The result is exact and
cannot overflow for any counter frequency below roughly 18 GHz, since
the remainder is by construction smaller than the frequency.
Assisted-by: Claude:claude-opus-5
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Add a Cortex-M perf backend that samples interrupted context from the
SysTick exception frame.
Install a SysTick wrapper when stack-sampling perf is enabled and
capture the interrupted stack pointer, EXC_RETURN, and r7 on exception
entry. Validate the complete hardware-stacked frame, including optional
alignment padding and extended FP state, against the Thread-mode or
Handler-mode stack before unwinding with arch_stack_walk().
The Arm stack walker uses EHABI unwind tables, so the backend does not
require CONFIG_FRAME_POINTER. Preserve r7 because EHABI unwind recipes
may use it to recover VSP even when frame pointers are not enabled
globally.
Return -EAGAIN when the interrupted context is invalid or unavailable
so the perf core discards that sample without reporting buffer
exhaustion.
Limit the backend to uniprocessor configurations because its captured
sample state is global and remote SMP sampling does not enter the
SysTick wrapper.
Disable the backend for Non-secure Trusted Execution images. A
Non-secure SysTick can be taken while the core executes Secure code,
but Non-secure firmware cannot access the Secure exception frame.
The stack pointer read by the wrapper instead refers to the suspended
Non-secure context, so it cannot be unwound as the interrupted frame.
Sampling across Security states is unsupported.
Signed-off-by: Sudan Landge <sudan.landge@arm.com>
Cortex-M exception frames can be stacked on MSP when execution is
interrupted in handler mode. The Arm EHABI stack walker currently seeds
VSP from the saved PSP unconditionally, which only works for frames
stacked on PSP.
Select the exception frame stack pointer from EXC_RETURN.SPSEL so
MSP-backed frames seed the unwinder from extra_info.msp, while preserving
the existing PSP path. This also allows perf sampling backends to unwind
handler-mode call chains captured on MSP.
Signed-off-by: Sudan Landge <sudan.landge@arm.com>
Otherwise, "warning: implicit declaration of function 'dump_fault'" is
generated, which may be promoted to an error when also using
CONFIG_COMPILER_WARNINGS_AS_ERRORS=y.
Some variables may be unused when the dump_fault isn't called, so mark them
as unused to avoid warnings.
Signed-off-by: Phil Hindman <phindman@xes-inc.com>
Commit 2222fa1426 removed all callers of
the function arm_core_mpu_mem_partition_config_update()... but not the
function itself, which became dead code lingering in tree since then...
Get rid of this function - it has served no purpose for long enough.
Signed-off-by: Mathieu Choplain <mathieu.choplain-ext@st.com>
Clear the fpu_recently_used flag when disabling FPU access for a
thread. This flag is used by the context switch path to decide whether
to re-enable FPU access; leaving it set caused k_float_disable() to be
silently reverted on the next context switch back to the thread.
Fixes#113645
Signed-off-by: Hongquan Li <hongquan.li@processmission.com>
Set K_FP_REGS in z_riscv_fpu_load() when a thread becomes the FPU
owner, and clear it in arch_float_disable(). This keeps user_options
consistent across lazy-FPU architectures rather than only clearing
the flag.
Fixes#113645
Signed-off-by: Hongquan Li <hongquan.li@processmission.com>
Introduce z_riscv_fpu_flush_thread() to perform the
architecture-specific FPU context flush without touching thread
options or fpu_recently_used, and make arch_float_disable() use it.
Reject NULL thread pointers with -EINVAL in arch_float_disable().
Unlike Cortex-M, RISC-V does not restrict the call to the current
thread or exclude ISR context.
Switch arch_spin_relax() to the new helper: it only needs to flush
the current CPU FPU context in response to a pending IPI, so it must
not go through arch_float_disable(), which now rejects NULL pointers
while _current_cpu->arch.fpu_owner may legitimately be NULL.
Fixes#113645
Signed-off-by: Hongquan Li <hongquan.li@processmission.com>
Rename pm_s2ram.S to pm_s2ram_asm.S and update the
Cortex-M source list to reference the new filename.
This avoids generating colliding object paths for the
C and assembly suspend-to-RAM sources when both are
built together.
Assisted-by: GitHub Copilot:GPT-5.4
Signed-off-by: Evan Chen <bugena123@gmail.com>
Code using this header is only built when CONFIG_ARM_MPU=y so it isn't
necessary to gate the header's declarations behind that option.
Signed-off-by: Mathieu Choplain <mathieu.choplain-ext@st.com>
`arm_core_mpu_dev.h` is the header describing the API that must be
implemented by drivers compatible with the core. Some functions were
were however accessed using an extern declaration inside the ARM MPU
Core module instead of a declaration in the API header.
Move the offending functions to the API header, making the extern
declaration unnecessary. While at it, also make `z_arm_mpu_init()`
part of the header because it is required, even if not consumed by
the ARM MPU Core module itself.
Signed-off-by: Mathieu Choplain <mathieu.choplain-ext@st.com>
XN on device-type regions only protects mapped regions; unmapped holes
in the MPU map remain reachable through the PRIVDEFENA background map,
both by Cortex-M7 speculative instruction fetches (Cortex-M7 TRM) and
by PLD linefills to faulting addresses (erratum 1013783, SDEN-1068427,
all M7 revisions).
Add opt-in CONFIG_ARM_MPU_CM7_UNMAPPED_REGION to program region 0 as a
4GB Strongly-ordered, no-access, Execute-Never catch-all, as
recommended by the erratum workaround; static regions start from
region 1 and take precedence. Opt-in because the catch-all defeats the
PRIVDEFENA background map: the static MPU region table must explicitly
cover all memory the firmware uses.
Signed-off-by: Lucien Zhao <lucien.zhao@nxp.com>
Header file include/zephyr/sys_clock.h is deprecated and will be removed
someday. Update the whole file tree to include zephyr/sys/clock.h
straight instead of zephyr/sys_clock.h.
This change was made running the sed shell command below:
$ sed -i 's/zephyr\/sys_clock\.h/zephyr\/sys\/clock\.h/' \
`grep -rsl "zephyr/sys_clock\.h" arch/`
Signed-off-by: Etienne Carriere <etienne.carriere@st.com>
Add support for stacktrace of threads. The shell command
"kernel unwind <thread_ptr>" now works.
Signed-off-by: Mike J. Chen <mjchen@google.com>
Co-authored-by: Filip Kokosinski <fkokosinski@antmicro.com>
Signed-off-by: Filip Kokosinski <fkokosinski@antmicro.com>
Mapping large memory regions at the soc level will consume too
many l2 entries, adding l1 section mapping functionality to
reduce excessive use of l2 entries.
Signed-off-by: CHEN Xing <xing.chen@microchip.com>
Select ARCH_SUPPORTS_COREDUMP_THREADS and ARCH_SUPPORTS_COREDUMP_STACK_PTR
for CPU_CORTEX_A in arch/arm64/core/Kconfig, enabling MEMORY_DUMP_THREADS
coredump mode on ARM64. Mirrors the existing Cortex-M declarations in
arch/Kconfig.
Implement arch_coredump_stack_ptr_get() using thread->callee_saved.sp_elx,
the EL1 stack pointer saved by the context-switch path for sleeping
threads. For the faulting thread the saved context is stale, so the
coredump falls back to dumping the full stack allocation,
which is always correct.
Signed-off-by: Appana Durga Kedareswara rao <appana.durga.kedareswara.rao@amd.com>
arm_cpu_boot_params stack pointer fields used pointer arithmetic on
two-dimensional K_KERNEL_STACK_ARRAY_DECLARE arrays before casting to
char *, producing invalid initialized pointer values.
Use K_KERNEL_STACK_BUFFER() on the CPU 0 stack elements instead,
matching the runtime initialization in arch_cpu_start().
Fixes#113125
Signed-off-by: Hongquan Li <hongquan.li@processmission.com>
In `movw_reloc_handler()`, the MOVZ/MOVN opcode selection for signed
AArch64 MOVW relocations was inverted: positive relocation values were
encoded with MOVN (negate) and negative values with MOVZ (zero), causing
all signed MOVW relocations to produce wrong immediate values.
Swap the two branches so that `x >= 0` selects MOVZ and `x < 0` selects
MOVN with immediate inversion, matching the AArch64 instruction semantics
where MOVZ zero-extends and MOVN negates the immediate.
Fixeszephyrproject-rtos/zephyr#113383.
Signed-off-by: Hongquan Li <hongquan.li@processmission.com>
When __arch_mem_map() fails to map a page, arch_mem_map() should
panic instead of continuing mapping the remaining pages.
Otherwise, there would be hole in the memory map which results
in confusing errors.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Instead of every new xtensa core having to modify xtensa_backtrace.c
and provide a header file with custom functions needed by the common
xtensa backtrace functions to validate stack and code addresses, now
xtensa_backtrace.c uses default weak functions that the soc can replace
with their own implementations.
Modify the current in-tree instances that used to have custom helper
functions to provide the standardize weak replacements.
Signed-off-by: Mike J. Chen <mjchen@google.com>
Co-authored-by: Filip Kokosinski <fkokosinski@antmicro.com>
Signed-off-by: Filip Kokosinski <fkokosinski@antmicro.com>
This PR migrates the MIT-licensed Xen public headers into the
`include/zephyr/xen/public` directory to the zephyr-xenlib module.
Since the xen public headers require GCC extensions,
enable GNU_C_EXTENSIONS along with enabling CONFIG_XEN.
Signed-off-by: TOKITA Hiroshi <tokita.hiroshi@gmail.com>
Instead of defaulting to 0 in the absence of a devicetree value, set
cache line size to the arch bit width which is a good proxy for cache
line size, and it's safer to use in code.
This default will also eliminate the need for many redundant checks
across the tree of CACHE_LINE_SIZE's value.
Signed-off-by: Abderrahmane JARMOUNI <git@jarmouni.me>
z_main_thread is only needed if multithreading is enabled, so move its
definition inside a multithreading check.
Signed-off-by: Josh DeWitt <josh.dewitt@garmin.com>
Commit f5a1585426 removed the extra
pointers for the IAR toolchain tls pointer setup. Update the size
returned from arch_tls_stack_setup to keep in sync with the tls pointer.
Signed-off-by: Josh DeWitt <josh.dewitt@garmin.com>
Store SBI_ERR_NOT_SUPPORTED in the saved a0 slot for unsupported
SRST functions. The previous store used the saved t2 offset.
S-mode callers kept the old a0 value, and t2 was restored as -1.
Fixes#113149.
Signed-off-by: Hongquan Li <hongquan.li@processmission.com>
The RISC-V coredump hook arch_coredump_priv_stack_dump() derives the
privilege stack region from thread->arch.priv_stack_start. That field
is only initialized when a thread enters user mode via
arch_user_mode_enter(). For a K_USER thread that has been created but
has never run, priv_stack_start is still 0, causing the hook to dump a
low-address region near 0 and potentially trigger a nested memory
fault.
Add an early return when priv_stack_start is 0 so that unstarted user
threads are skipped.
Fixes#113875
Signed-off-by: Hongquan Li <hongquan.li@processmission.com>
arc_vpx_sem_init() initialized vpx_sem[0] once per CPU instead of
initializing each CPU's semaphore: the loop passed the array (decaying
to &vpx_sem[0]) to k_sem_init() rather than the i-th element, leaving
the semaphores of CPUs 1..N-1 uninitialized on multi-core
configurations.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
arch_buffer_validate() computes end = addr + size with a plain
64-bit add and never checks for unsigned wrap. When addr + size
wraps past 2^64, x1 < x0, so the blo abv_loop condition is false
after the first iteration — only the page containing addr is
validated, and the function returns 0 (success) for a range that
spans the entire address space including kernel memory.
Reject wrapping ranges immediately after computing the end
address, before entering the page loop. This mirrors how the
Xtensa MPU implementation (PR #109000) and ARM32 is_in_region()
(PR #23239) reject wrapping ranges.
Fixes#113800
Signed-off-by: Hongquan Li <hongquan.li@processmission.com>
arch_mem_domain_max_partitions_get() returns an optimistic slot
estimate and its comment says resync_pmp_domain() should deny
availability when the estimate is too high, but that path asserted
instead. On a SoC whose global regions leave few slots, a domain can
hold more partitions than fit a thread's remaining PMP entries, so the
assert fired during a context switch and reset the whole system. Stop
programming and log instead: the thread runs with the partitions that
fit and faults, only itself, on an access to an unmapped one, which is
recoverable.
Signed-off-by: Sylvio Alves <sylvio.alves@espressif.com>
On SoCs whose flash-mapped read-only data sits in an address window
separate from the executable text, the __rom_region symbol only spans
the text. A user-mode thread passing a const string or other rodata
pointer to a syscall then failed the buffer validation, because the
rodata fell outside the only read-only range that was checked.
Accept the rodata region as well so reads of flash-mapped constants
from user mode validate correctly. SoCs with contiguous text and
rodata are unaffected since __rom_region already covers both.
Signed-off-by: Sylvio Alves <sylvio.alves@espressif.com>
On RISC-V with CONFIG_USERSPACE, arch_is_user_context() read the
thread-local is_user_mode symbol directly inline. Every syscall stub
inlines this check, so any extension (llext) that called a syscall
pulled in a thread-local relocation the loader cannot resolve, failing
to link with an undefined is_user_mode symbol.
Move the thread-local read into an exported, non-inline accessor
z_riscv_thread_is_user_mode(), declared in arch/riscv/thread.h and
called from arch_is_user_context(), mirroring the Arm
z_arm_thread_is_in_user_mode() approach. Extensions now resolve the
exported symbol instead of a thread-local one, so a loaded module can
call syscalls. The tp-not-initialized fast path stays inline.
Signed-off-by: Sylvio Alves <sylvio.alves@espressif.com>
The CONFIG_X2APIC help text frames the option as safe to enable on any
APIC that supports x2APIC mode. This is not accurate: with
CONFIG_X2APIC=y, an AP reads its local APIC ID from the x2APIC ID MSR
unconditionally, even when the AP has not been switched into x2APIC
mode earlier in the bootflow. The MSR access then faults, and the
fault cannot be handled this early in SMP bring-up, so the AP hangs.
Signed-off-by: Patryk Koscik <pkoscik@antmicro.com>
