Commit graph zephyr/arch
Author SHA1 Message Date
Silesh C V
9d8ace20d0 arch: arm: cortex_a_r: smp: power on secondary cores via pm_cpu_ops
arch_cpu_start() relied solely on the reset.S voting lock to bring up
secondary cores. That is sufficient only on platforms that release all
cores at reset. On platforms that hold secondaries powered off until
requested (e.g. the Arm FVP Base model), they must be explicitly
powered on with their reset vector pointing at the Zephyr entry point.

Use the generic pm_cpu_on() API to start each secondary at __start.
This dispatches to whichever CPU power driver is enabled. The call is
guarded by CONFIG_PM_CPU_OPS, leaving platforms that release
all cores at reset unaffected.

Signed-off-by: Silesh C V <silesh@alifsemi.com>
2026-09-16 09:22:52 +02:00
Silesh C V
f7fb4047c9 arch: arm: cortex_a_r: smp: flush boot params before starting secondary
arm_cpu_boot_params is populated by the primary core and consumed by
the secondary being brought up. On platforms that start secondaries
cold, the secondary begins executing with its caches and MMU disabled
and reads the parameters directly from main memory.

The primary used sys_cache_data_invd_range() on the structure, which
invalidates the cache lines without writing them back. The freshly
written parameters can therefore be discarded before reaching memory,
leaving the secondary to read stale values (e.g. a wrong MPID).

Use sys_cache_data_flush_range() to write the lines back instead,
followed by a DSB to guarantee the flush has completed before the
secondary is brought up.

Signed-off-by: Silesh C V <silesh@alifsemi.com>
2026-09-16 09:22:52 +02:00
Silesh C V
5680176ea9 arch: arm: aarch32: mmu: only build shared page tables on primary core
z_arm_mmu_init() runs on every CPU and rebuilds the single, globally
shared L1/L2 page tables on each call. On SMP this is not only
unnecessary duplicate work on the secondary cores (the primary has
already built the tables), but also unsafe. While a secondary core
rebuilds the tables, descriptors that are live and in use by other
cores (the primary or any secondaries with their MMUs enabled) can
temporarily hold invalid values, which can lead to aborts on the
cores using the tables. In practice this manifested as a prefetch
abort on the primary core as soon as a secondary core was brought up
and began (re)building the shared tables.

Fix this by mirroring the arm64 z_arm64_mm_init(is_primary_core)
design: build the tables only on the primary core, in a new
arm_mmu_setup_ptables() helper, and have every core program its MMU
registers and enable translation using the shared tables. As the
page table region's memory attributes are needed while programming
TTBR0, introduce another helper arm_mmu_get_pt_attrs() that is run
by all the cores to look this up.

Also, remove the unused z_arm_mmu_init declaration from
cortex_m/kernel_arch_func.h as that header is included only under
CPU_CORTEX_M.

Single-core behaviour is unchanged.

Signed-off-by: Silesh C V <silesh@alifsemi.com>
2026-09-16 09:22:52 +02:00
Adrian Śliwa
78087d0662 arch: riscv: enable U-mode access to the Zicntr counters
Reads of cycle, time and instret below M-mode are gated by mcounteren
and scounteren, both zero out of reset. Zephyr sets neither in a build
with user threads, so user mode cannot read these counters.

Set them during per-CPU init, since the registers are per-hart. An
M-mode kernel writes mcounteren, and scounteren when misa reports
S-mode, as it does not exist otherwise. An S-mode kernel writes
scounteren alone, because mcounteren is out of reach and is expected
to be set by the SBI, as reset.S does for the built-in one.

Signed-off-by: Adrian Śliwa <asliwa@internships.antmicro.com>
2026-09-16 05:58:39 +02:00
Adrian Śliwa
56164d22a4 arch: riscv: share the counter-enable bit definitions
The counter enable bits are identical in mcounteren, scounteren and
hcounteren, so move them out of reset.S into csr.h. This lets the arch
code that opens the counters to user mode reuse them instead of adding
a second copy.

Signed-off-by: Adrian Śliwa <asliwa@internships.antmicro.com>
2026-09-16 05:58:39 +02:00
Dhruv Menon
908a2e5f0d arch: riscv: switch to main stack before unmasking interrupts
If an interrupt was already pending when z_cstart() completed (such as an
early timer compare event or peripheral interrupt), the ISR fired
immediately while sp still pointed to the interrupt stack. Because
_current_cpu->nested is 0, _isr_wrapper assumed the interruption came
from thread mode and reset sp to the top of _kernel.cpus[0].irq_stack,
causing subsequent ISR C call frames to write directly over the hardware-
saved ESF. This corrupted the saved mepc, leading to an Illegal Instruction
exception (mcause: 2) upon mret.

this commit fixes this by moving interrupt enablement
(csrs RV_STATUS_CSR, %2)  inside the inline assembly block after the stack
pointer has been  switched to main_stack (mv sp, %0)

Signed-off-by: Dhruv Menon <dhruvmenon1104@gmail.com>
2026-09-16 05:55:37 +02:00
Sylvio Alves
886d9899db arch: riscv: add soc hook to close the syscall ecall
Some RISC-V SoCs cannot deliver a fault raised by the syscall
body itself while the ECALL exception of a user-mode syscall is
still being handled. The stack guard below the privileged stack
catches a syscall whose call chain runs too deep, but on such a
SoC that access fault is reported asynchronously inside the open
ECALL and locks the core up, resetting the chip with nothing
delivered to software. A user-mode application can therefore
reset the board through the depth of a granted service call.

Masking interrupts does not help here. The existing
RISCV_SOC_HAS_SYSCALL_INTMASK holds off the interrupt
controller, while this fault comes from the body itself.

Add the hidden RISCV_SOC_SYSCALL_CLOSE_ECALL option. When a SoC
selects it, the IRQ wrapper leaves the exception before running
the syscall body: it returns to the following instruction with
the previous privilege set to machine mode, so the body runs on
the same privileged stack with the same privileges but outside
the exception. A fault taken there is an ordinary top-level trap
the SoC delivers normally, and a syscall that overflows the
privileged stack is reported as a stack overflow that kills only
the offending thread.

The exception exit path reloads the exception program counter
and status register from the saved frame, so borrowing both here
does not disturb the return to user mode.

Assisted-by: Claude:opus-5
Signed-off-by: Sylvio Alves <sylvio.alves@espressif.com>
2026-09-14 15:41:06 -04:00
Sylvio Alves
32381dcbfb arch: riscv: add soc hook to mask syscall interrupts
Some RISC-V SoCs cannot take an interrupt in the syscall body: an
interrupt, or another asynchronously reported event such as an
imprecise access fault, taken while the ECALL exception is still
being handled locks the CPU up, with no fault delivered to
software. Precise synchronous traps, such as the ECALL used to
switch out of the body, nest normally. The syscall path sets
mstatus.MIE before running the body, so such a SoC must hold
interrupts off another way.

This is not the Smdbltrp double trap. On the affected cores
reading mstatush traps, so there is no mstatus.MDT to clear, and
the vendor status CSR exposes the in-exception state as a
read-only bit that stays set until mret. Software cannot end the
exception window early, so the only option is to keep interrupts
from being taken while it is open.

Add the hidden RISCV_SOC_HAS_SYSCALL_INTMASK option following
the existing RISCV_SOC_HAS_* hook pattern. When a SoC selects it,
the IRQ wrapper invokes a SoC-provided SOC_SYSCALL_INTMASK macro
on syscall entry, typically raising a hardware interrupt level
threshold, while the generic code carries no SoC register
knowledge. This is a SoC hook rather than part of the generic
CLIC interrupt level support because the SoCs that need it drive
their interrupt controller from SoC code and do not select
RISCV_HAS_CLIC.

The mask is per exception frame. The SoC context hooks save the
state on entry and restore it on the exception exit path, and
with RISCV_ALWAYS_SWITCH_THROUGH_ECALL every context switch goes
through that path: a thread that blocks in the body hands the
CPU to a thread that restores its own saved state, and a new
thread starts from SOC_ESF_INIT. The option therefore depends on
both. While the mask is raised the body is not preemptible and
pending interrupts, the tick included, wait for the syscall to
return or block; the help text says so.

Assisted-by: Claude:opus-5
Signed-off-by: Sylvio Alves <sylvio.alves@espressif.com>
2026-09-14 15:41:06 -04:00
Sylvio Alves
37d3022714 arch: riscv: pre-validate user strings on imprecise-fault socs
arch_user_string_nlen() dereferences a user pointer and recovers
from a bad address through a fault fixup that matches the
faulting mepc against the load instruction's range. On some
RISC-V SoCs the load access fault is imprecise: the reported mepc
lands past the faulting load, so the fixup misses it and the
fault escalates to a fatal reset.

Rename the asm routine to z_riscv_user_string_nlen() and alias
arch_user_string_nlen() to it by default. Under the new
RISCV_USER_STRING_NLEN_VALIDATE option a C implementation takes
over instead: it checks the whole range with
arch_buffer_validate() first and, when that fails, walks the
string in PMP-granularity chunks, validating each chunk before
reading it, so no load that could fault is ever issued. The
faulting load may sit at any offset (a string that starts in an
accessible region and runs unterminated into an inaccessible
one), which is why one check of the first byte is not enough.
The generic syscall handler is untouched.

Add a syscalls test scenario that enables the option on RISC-V.

Assisted-by: Claude:opus-5
Signed-off-by: Sylvio Alves <sylvio.alves@espressif.com>
2026-09-14 15:41:06 -04:00
Benjamin Cabé
a811b0f1cd arch: arm: cortex_m: drop redundant ISBs from IRQ lock and unlock
arch_irq_lock() and arch_irq_unlock() executed an ISB after every
BASEPRI write on ARMv7-M/ARMv8-M Mainline, and after CPSIE on ARMv6-M/
ARMv8-M Baseline unlock: two pipeline flushes for every kernel critical
section. Neither is architecturally required. Priority-raising MSR
writes are self-synchronizing (Arm DAI 0321A, section 4.8), and on
unlock a pended interrupt is taken within a couple of instructions
anyway; only arch_swap() relies on the pended PendSV being recognized
before the next instruction executes, so it gains an explicit ISB.

Cortex-M7 r0p0/r0p1 keeps the ISB on the raising side, where erratum
440977 (formerly 837070) can delay a priority-raising BASEPRI write. The
CONFIG_CORTEX_M_ERRATUM_440977_WORKAROUND option defaults to y on M7;
the same ISB in the PendSV handler prologue is handled identically.

The ISB occupies the instruction that can still be preempted before the
critical region begins. Retain this existing sequence to avoid masking
higher-priority interrupts through PRIMASK, as Arm's documented
CPSID i/MSR/CPSIE i workaround would do.

Measured standalone against main (latency_measure in cycles, lower is
better; thread_metric scores, higher is better):

* az3166_iotdevkit (STM32F412, Cortex-M4 @ 96 MHz):
  - latency_measure: mean -3.3% over 47 ops, min/max/median
    -8.1/+0.6/-2.9%; semaphore give no-waiter 74 -> 68 cycles.
  - thread_metric: synchronization +15.4%, cooperative +3.3%,
    preemptive +4.6%, interrupt +2.9%.

* mps2/an385 (Cortex-M3, QEMU icount):
  - latency_measure: mean -3.8%, min/max/median -6.8/+0.0/-3.8%.
  - thread_metric: synchronization +8.3%.

* Flash, az3166 latency_measure image: -400 B at -Os, -424 B at -O2.

Assisted-by: Codex:gpt-6-astra
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
2026-09-14 08:42:05 -04:00
Hongquan Li
dc577c7239 arch: arm64: fix DAIF restore on arch_buffer_validate() wrap path
Preserve the DAIF state across failed buffer validation.

Fixes #115141

Signed-off-by: Hongquan Li <hongquan.li@processmission.com>
2026-09-13 21:38:39 -04:00
Andrei-Edward Popa
ef5b9dc5c1 arch: arm: mpu: fix region init for Cortex-R
ARM_MPU_REGION_INIT() currently includes the MPU region size in the region
attributes for both Cortex-M and Cortex-R.

On Cortex-M, the region attributes and size are both programmed through
RASR, so this is correct. Cortex-R uses separate registers for region
access attributes and region size. Including the size in the attributes
therefore corrupts the memory type attributes programmed into DRACR.

Store the encoded region size in the dedicated size field for Cortex-R
and exclude it from the region attributes. This fixes devicetree-defined
MPU regions on Cortex-R, where the region size could alter the TEX, S, C,
and B attribute bits.

Signed-off-by: Andrei-Edward Popa <andrei.popa105@yahoo.com>
2026-09-13 21:38:15 -04:00
Hongquan Li
a0824279b9 arch: arm64: require stack protection for safe exception stack
ARM64_SAFE_EXCEPTION_STACK is user-selectable without
ARM64_STACK_PROTECTION, but thread->arch.stack_limit is only assigned
when the latter is on (arch_new_thread). With the safe exception stack
enabled standalone, current_stack_limit stays 0, so the overflow check
in z_arm64_quick_stack_check() can never trip: every EL1 exception pays
the entry overhead and the kernel stack overflow detection silently
does not exist.

Make ARM64_STACK_PROTECTION the only way in. No in-tree configuration
is affected: no AArch64 target selects ARM_MPU today, so
ARM64_SAFE_EXCEPTION_STACK was never enabled in any built config.

Fixes #118385

Signed-off-by: Hongquan Li <hongquan.li@processmission.com>
Assisted-by: Codex:GPT-5 gh
2026-09-13 21:38:07 -04:00
Daniel Leung
2756f2c863 xtensa: mmu: simplify TLB shootdown
Since we are caching all the necessary register values to be
used when switching page tables, we can now remove the extra
bits inside xtensa_mmu_tlb_shootdown() originally used to
prevent re-computing all those register values. We can now
simply call xtensa_mmu_set_paging().

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2026-09-13 17:59:42 -04:00
Daniel Leung
60c9284620 xtensa: mmu: no TLB shootdown if not needed during thread add
Inside arch_mem_domain_thread_add(), if the thread is not
running on other CPUs and it is not migrating between domains,
there is no need to send TLB shootdown to other CPUs. Next time
the thread is scheduled, the new page tables will be used.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2026-09-13 17:59:42 -04:00
Brian Cain
8b64665e96 hexagon: implement arch_irq_offload()
Run a routine in interrupt context by posting a software interrupt
through the VM and letting it fire when irq_unlock() re-enables IE.

IRQ 0 is used as the trigger line: the board's DTS assigns the lowest
hardware IRQ at 0x0c, so it does not collide with a real device here.

Give the test suite a trigger_irq() for Hexagon as well, going through
the same hypercall, so the shared interrupt tests can drive a line from
software.

Signed-off-by: Brian Cain <brian.cain@oss.qualcomm.com>
2026-09-12 07:52:39 -04:00
Brian Cain
5761d7d707 hexagon: add architecture port
Add Hexagon DSP architecture support to Zephyr RTOS, targeting the
Qualcomm Hexagon V67+ ISA running as a guest under the H2 hypervisor.

This includes:
- Architecture scaffolding (Kconfig, CMakeLists, arch selection)
- Public headers (arch.h, thread.h, exception.h, error.h)
- Core runtime: context switch, interrupt/exception handling, idle, TLS
- Memory management stubs for flat-memory H2 guest model
- SoC support for QEMU hexagon virt machine
- CMake/LLVM toolchain integration for hexagon cross-compilation

Hexagon uses H2's GEVB (Guest Event Vector Base) for interrupt dispatch
rather than a traditional function-pointer vector table, so
GEN_IRQ_VECTOR_TABLE is disabled by default.

Signed-off-by: Brian Cain <brian.cain@oss.qualcomm.com>
2026-09-12 07:52:39 -04:00
Liu Qian
18b49e8379 arch: riscv: pmp: add NAPOT multi-slot mode for non-aligned regions
When TOR is unsupported (PMP_NO_TOR), the Kconfig forced
PMP_POWER_OF_TWO_ALIGNMENT=y, wasting memory: 33 KB of code+rodata
aligns to 64 KB, 257 KB aligns to 512 KB.

Add PMP_NAPOT_USE_MULTI_SLOTS: splits a non-naturally-aligned region
into multiple NAPOT entries, each covering the largest naturally-
aligned block at the current address. [0x1800, 0x3800) becomes
0x800 + 0x1000 + 0x800 (3 slots instead of padding to 8192).

Depends on !USERSPACE: arch_mem_domain_max_partitions_get() assumes
1-2 slots per partition and resync_pmp_domain() can only log on
failure, so a partition needing more slots would run unmapped (silent
loss of protection). Help text documents that disabling
PMP_POWER_OF_TWO_ALIGNMENT also splits stack guards and u-mode stacks,
and that PMP_GRANULARITY=64 cuts both ways for that case.

try_multi_entries_set() uses clz/ctz builtins, clears already-written
config bytes on failure before restoring *index_p (otherwise the
trailing write_pmp_entries(0, PMP_SLOTS) in z_riscv_pmp_init() would
program locked partial entries), and folds start/size into the inner
failure logs.

Signed-off-by: Liu Qian <liuqian.andy@picoheart.com>
2026-09-11 17:34:41 +02:00
Liu Qian
96c7e335e4 arch/riscv: add release barrier before publishing switch_handle
Under RVWMO the context stores (callee-saved registers, stack
pointer) may be reordered before the switch_handle store, so a hart
spinning in z_sched_switch_spin() could observe the handle before
the saved context is visible and restore stale registers.

Add fence rw, w before publishing switch_handle, pairing with the
acquire barrier in z_sched_switch_spin(), same as ARM64.

Signed-off-by: Liu Qian <liuqian.andy@picoheart.com>
2026-09-11 17:32:12 +02:00
Daniel Leung
8acbdd4b5b x86: mmu: do not compile out memory mapping error handling
Now that arch_mem_map() and arch_mem_unmap() return error codes,
the x86 implementation must be able to report failures in every
build configuration.

All error detection in the mapping path used CHECKIF(), which
expands to if (0) under CONFIG_NO_RUNTIME_CHECKS and also under
CONFIG_ASSERT_ON_ERRORS. With either option the error handling
blocks were compiled out and arch_mem_map() returned 0
unconditionally even though the mapping had not been performed.
The ARG_UNUSED(ret2) statements preceding the checks existed only
to silence the resulting unused variable warnings.

CHECKIF() is meant for conditions the caller can be blamed for, so
it is kept for argument validation. The "large page encountered"
and "missing page table level" cases in page_map_set(), and the
propagation of those failures in range_map_ptables() and
range_map(), are genuine runtime conditions and now use a plain
if so that -EFAULT always reaches the caller.

flags_to_entry() only asserted on an unsupported caching mode and
otherwise mapped the region write-back, silently ignoring the
request. Since K_MEM_CACHE_MASK is three bits wide while only
three modes are defined, the remaining values are reachable. It
now returns an error so that arch_mem_map() reports -ENOTSUP, as
the other architectures already do.

arch_mem_map() intentionally keeps passing no OPTION_FLUSH, as
that option is documented as being needed only when altering
existing mappings.

Assisted-by: Claude:claude-opus-5
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2026-09-11 05:23:20 -04:00
Daniel Leung
a778adcb2b arm64: mmu: memory mappings to return error value not panic
This changes the error handling of the call chain of both
arch_mem_map() and arch_mem_unmap() in ARM64 to return errors
if possible when actions fail, instead of causing kernel panic
or silently ignoring them. This allows the error code to be
propagated to the caller so it can be handled properly.

new_table() used to call k_panic() when running out of
translation tables, as its comment stated that many code paths
were not ready for failure. This is no longer the case since
expand_to_table(), dup_table() and set_mapping() all handle
the allocation failure already, so it simply returns NULL now.
Allocating the base translation table at boot keeps halting the
system as there is no way to recover from that.

del_mapping() needs to split a block mapping when only part of
it is being unmapped. It ignored the failure to allocate the
table for the split, and went on to clear the entry, which
unmapped the whole block instead of the requested range. It now
reports the failure and leaves the block alone, and this is
propagated through remove_map() and globalize_table().

sync_domains() only logged the failure to propagate mappings to
memory domains, so arch_mem_map() and arch_mem_unmap() could
report success while some domains were left inconsistent with
the kernel page tables. The error is now returned, though the
remaining domains are still synchronized first.

Domain synchronization and TLB invalidation are now performed
even if the operation has failed, since page tables may have
been modified before bailing out.

Assisted-by: Claude:claude-opus-5
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2026-09-11 05:23:20 -04:00
Daniel Leung
2d088dd1c1 arm: mmu: memory mappings to return error value not panic
This changes the error handling of the call chain of both
arch_mem_map() and arch_mem_unmap() in ARM AArch32 to return
errors if possible when actions fail, instead of causing kernel
panic or silently ignoring them. This allows the error code to
be propagated to the caller so it can be handled properly.

arm_mmu_assign_l2_table() previously relied solely on assertions
to catch running out of L2 page tables. With assertions disabled,
it would underflow the free table counter and hand out a table
which is still in use, silently corrupting existing mappings.
It now returns NULL instead, and the failure is propagated up
through arm_mmu_l2_map_page() and its callers as -ENOMEM.

Unsupported cache modes are now rejected with -ENOTSUP instead
of being silently mapped as device memory, and unmapping a page
table entry of an unexpected type is reported as -EFAULT instead
of only being logged. Boot time mappings keep using assertions
as there is no way to recover from those failures.

Both wrappers now invalidate the TLB even if the operation has
failed, since page tables may have been modified before bailing
out.

Assisted-by: Claude:claude-opus-5
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2026-09-11 05:23:20 -04:00
Daniel Leung
c546cb235f xtensa: mmu: memory mappings to return error value not panic
This changes the error handling of the call chain of both
arch_mem_map() and arch_mem_unmap() in Xtensa to return errors
if possible when actions fail, instead of simply causing
kernel panic. This allows the error code to be propagated to
the caller so it can be handled properly.

Assisted-by: Claude:claude-opus-5
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2026-09-11 05:23:20 -04:00
Daniel Leung
c21850fdd6 arch: change arch_mem_unmap() to return an error code
Mirror the arch_mem_map() change and give arch_mem_unmap() the same
treatment: change the prototype to return int instead of void, so
architecture code has a consistent way to report unmapping failures.

Assisted-by: Claude:claude-sonnet-5
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2026-09-11 05:23:20 -04:00
Daniel Leung
aec25bce22 arch: change arch_mem_map() to return an error code
Previously arch_mem_map() was a void function, so architecture
code had no consistent way to report mapping failures to callers
other than panicking or asserting. Change arch_mem_map() to
return int.

Note that this mostly maintains the k_panic() calls unless
the error is easy to handle (like invalid arguments). Each
architecture will need to be amended in the future to properly
handle the error conditions. Also some of the users are
converted to do k_panic() after failure. This is to keep
the signature change commit easier to review.

Assisted-by: Claude:claude-sonnet-5
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2026-09-11 05:23:20 -04:00
Kai Vehmanen
c770d5c5d1 xtensa: initialize ptables to NULL for new threads
Initialize ptables for newly created threads if CONFIG_XTENSA_MMU is
enabled. This is required as as arch_mem_domain_thread_add() uses a
non-NULL arch.ptables to detect that the thread is migrating from a
previous memory domain and reset its stack there. For a brand new thread
- in particular a dynamically allocated one (k_object_alloc()), whose
backing memory is not zeroed - this field would otherwise hold garbage
and the "migration" would walk a bogus page table.

Issue found when debugging DSP panics on Intel ADSPs (EXCAUSE 13) with
SOF user threads running on secondary cores.

Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
2026-09-09 18:14:29 -04:00
Bjarki Arge Andreasen
fa61529665 arch: arm: cortex-m: exception: add system control regs to extra_info
During fault handling on the cortex-m, namely ARMV7_M_ARMV8_M_MAINLINE,
some system control registers are modified before reaching
z_fatal_error(). Applications may want to inspect the state of these
registers at time of fault from within k_sys_fatal_error_handler().

- Extend struct __extra_esf_info with the following relevant system
  control registers:

    - CFSR
    - HFSR
    - DFSR
    - MMFAR
    - BFAR

  and extend it with the following SAU registers if
  CONFIG_ARM_SECURE_FIRMWARE=y

    - SFSR
    - SFAR

  and the following field indicating whether the ESF is from the
  non-secure stack

    - bool non_secure

- Update z_arm_fault() to snapshot these registers into the esf copy
  extra_info before fault handling touches any of them.

- Extend esf_dump() to include these registers in case they are
  present. If CONFIG_ARM_SECURE_FIRMWARE=y, the value of the
  non_secure member is dumped, preceeding the system registers, to
  indicate whether the SCB registers are from the SCB or SCB_NS
  instance.

Signed-off-by: Bjarki Arge Andreasen <bjarki.andreasen@nordicsemi.no>
2026-09-09 10:36:08 -04:00
Hongquan Li
cb2958045a arch: arm64: fix arch_early_memset() for unaligned dst and short lengths
arch_early_memset() only initialized the fill register x8 in the branch
taken when the destination is 8-byte aligned and the length is at least
8 bytes. Any call with a non-8-byte-aligned destination or a length < 8
reached the byte loop, which stores uninitialized register content
instead of the requested fill value. The C version this assembly
replaced in commit 696edbe841 ("arm64: move simple memcpy/memset
alternatives to assembly") handled all inputs correctly, so the
assembly port introduced the defect.

Rather than widening the fill value unconditionally, have the byte loop
store the low byte of the intact argument directly: x1 is preserved on
every path reaching the loop and strb already stores only the low byte,
so short and unaligned calls no longer pay for the 64-bit pattern
setup.

Fixes #118341

Suggested-by: Nicolas Pitre <npitre@baylibre.com>
Signed-off-by: Hongquan Li <hongquan.li@processmission.com>
2026-09-08 19:18:10 -04:00
Jamie McCrae
1581682a27 arch: kconfig: Add missing deprecation text
Adds a missing [DEPRECATED] tag to the Kconfig prompt

Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
2026-09-08 16:04:16 +02:00
Aaron Wisner
f1b8bdae9a arch: arm64: skip redundant FPU access-control updates
Only modify CPACR_EL1 when it actually needs to be changed.

Read the current CPACR_EL1 value and compute the required FP/SVE access
permissions before writing it. Avoid the write and instruction barrier
when those permissions already match, both on exception entry and when
updating access before leaving an exception.

Preserve the existing access policy and execute the write and barrier
whenever permissions change. Keep the IRQ-disabled assertions and the
original lazy FPU ownership, context saving and migration behavior. This
reduces exception-path overhead even for integer-only execution.

Internal ARMv8 SMP microbenchmarks show 1.4-2.1% lower latency for this
change. Together with cached FPU-flush requests, the lazy-manager changes
measured 2.3% higher aggregate throughput and 3.8% lower aggregate latency
with integer hot paths held fixed. This patch's standalone throughput
effect was not isolated, and these are not fresh upstream-main results.

Assisted-by: Codex:GPT-5
Signed-off-by: Aaron Wisner <aaronwisner@gmail.com>
2026-09-08 13:40:18 +02:00
Yiren Guo
c720659e6b arch: arm: timing: fix hang in DWT frequency calibration
if the first call to arch_timing_freq_get() happens after
arch_timing_stop(), CYCCNT stays frozen, ddwt is always 0 and the
calibration loop never terminates.

Enable the cycle counter for the measurement and restore its previous
state afterwards.

Signed-off-by: Yiren Guo <guoyr_2013@hotmail.com>
2026-09-07 10:30:00 +02:00
Hongquan Li
b8fd82d7c8 arch: arm64: mmu: fix arch_virt_region_align() for large regions
The alignment loop computes level_size with a 32-bit "1 <<" shift,
which is undefined behavior once the shift count reaches 32 (level 0
with 4KB granule, level 1 with 16KB/64KB granule). It also compares
int level against the unsigned BASE_XLAT_LEVEL, so with base level 0
the loop never terminates at level 0 and runs into negative levels,
leaving garbage in the returned alignment. Use 1ULL and cast the
bound to int.

The garbage alignment feeds virt_region_alloc(), which can
under-reserve the VA bitmap for large mappings.

Fixes #115139

Signed-off-by: Hongquan Li <hongquan.li@processmission.com>
2026-09-07 10:26:09 +02:00
Appana Durga Kedareswara rao
3b3bd23a9d debug: coredump: arm64: freeze other CPUs and capture live registers
A thread actively running on another CPU at panic time was previously
invisible to real debugging: coredump only captured the panicking CPU's
exception frame, so every other thread's registers came from its
k_thread.callee_saved struct -- correct for genuinely sleeping threads,
but stale (reflecting whenever it last voluntarily context switched) for
one that's actually running elsewhere right now.

Add CONFIG_DEBUG_COREDUMP_SMP_FREEZE_CPUS (default y where the arch sets
ARCH_SUPPORTS_COREDUMP_SMP_FREEZE, i.e. arm64 Cortex-A SMP). On panic,
coredump() freezes every other online CPU before dumping and thaws them
after the memory-region walk, and emits each frozen CPU's live registers
as a new coredump section (COREDUMP_CPU_SNAPSHOT_HDR_ID, tagged with CPU
index and thread pointer, reusing the arch-info block register layout).
The panicking CPU (never frozen, since freeze only targets other CPUs)
emits a zero-payload marker recording only which thread was panicking
there, which the host side uses to identify it unambiguously.

arm64 implementation (arch/arm64/core/smp.c, coredump.c): send a new SGI
to every other online CPU. Each handler captures its exact live state --
x0-x18/lr/spsr/elr via the arch_esf that _isr_wrapper() stashes at a
fixed, nesting-independent slot on its IRQ stack, plus x19-x29 via inline
asm (valid per AAPCS64) -- then spins holding whatever it held before
being frozen, untouched, until the dump finishes, and resumes via the
standard exception-return path exactly where it was interrupted. This
matters because a panic here can be recoverable (this tree's own
coredump_threads test doesn't halt/reboot after dumping).

The SGI uses a deliberately low priority (SGI_COREDUMP_FREEZE_PRIO), not
IRQ_DEFAULT_PRIORITY: a frozen CPU never returns from its handler so
_isr_wrapper() never EOIs, and per GIC priority rules only a strictly
higher-priority interrupt can preempt a still-active one. At the default
priority a frozen CPU could not service same-priority peripheral
interrupts (confirmed on hardware as Ethernet TX-done starvation during
the UDP backend); the lowest usable priority lets everything else keep
making progress while a CPU waits.

The freeze/thaw handshake uses two separate per-CPU signals: freeze_state
(IDLE/REQUESTED/CAPTURED) and a dedicated release_requested flag that
only thaw writes. Kept separate so a late-arriving CPU (whose SGI was
delayed past the freeze-side timeout) still sees a release signal instead
of stomping it with its own CAPTURED update and spinning forever; thaw
sets release_requested unconditionally for every other CPU. The handler
wait is also bounded by a 60-second wall-clock backstop (k_cycle_get_32(),
safe from any context) so no CPU is ever stuck indefinitely. A CPU that
never responds within the freeze-side timeout (never booted, or stuck
with IRQs masked) is skipped and never blocks the dump.

The host coredump log parser learns to parse the new snapshot section.

Signed-off-by: Appana Durga Kedareswara rao <appana.durga.kedareswara.rao@amd.com>
2026-09-04 21:45:11 -04:00
Anas Nashif
c86d7cea4b arch: add ARCH_HAS_DIRECT_INTERRUPTS capability symbol
Direct interrupts are an optional architecture feature: only ARC, ARM
(aarch32), RISC-V, x86 ia32 and the POSIX/native boards implement the
ARCH_IRQ_DIRECT_CONNECT() and ARCH_ISR_DIRECT_DECLARE() hooks. There was
no way to express that dependency in Kconfig, so a symbol enabling code
that uses IRQ_DIRECT_CONNECT() had to enumerate the supporting
architectures itself, and the only capability test available to C code
was probing whether the arch hook macro happened to be defined.

Add ARCH_HAS_DIRECT_INTERRUPTS, selected by the architectures that
implement the hooks, and switch the gen_isr_table test over to it. Note
that the capability is not uniform across an architecture: x86 provides
it only on the 32-bit (ia32) variant, and arm64, xtensa, MIPS, SPARC and
RX do not provide it at all.

No functional change: the symbol is selected exactly where the arch hooks
are defined today.

Assisted-by: Claude:claude-opus-5
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2026-09-04 13:41:41 -04:00
Hongquan Li
595a3aaf11 llext: arm64: range-check movw relocation value before truncation
movw_reloc_handler() stored the relocation value in a 32-bit
variable, so the range check only saw the truncated value: 2^32 + 5
and the like aliased into range, and the instruction was silently
patched with a truncated immediate.

Widen the immediate to 64 bits so the single imm > UINT16_MAX
comparison covers both directions: the MOVN path stores the bitwise
inverse of the immediate, and ~ brings a negated value back under
UINT16_MAX exactly when the value is no smaller than
-(UINT16_MAX + 1).

Fixes #115707

Signed-off-by: Hongquan Li <hongquan.li@processmission.com>
2026-09-03 14:08:17 +02:00
Hongquan Li
cc2d80de43 llext: arm64: fix immediate relocation overflow check
imm_reloc_handler() tested its sign-extension check value (exactly 0
or -1 for values that fit the signed field) with (int64_t)x++ > 1,
which only rejects check values of 2 or more: a positive overflow by
one bit (check value 1) and every negative overflow pass, and the
instruction is then silently patched with a truncated immediate. The
post-increment is also dead: it compares the value before
incrementing and nothing reads x afterwards.

Reject everything but 0 and -1.

Fixes #115707

Signed-off-by: Hongquan Li <hongquan.li@processmission.com>
2026-09-03 14:08:17 +02:00
Aditya Ganesh
6b1f8dbb5f arch: x86: core: Fix unused parameter warning
Apply ARG_UNUSED() macro to resolve unused parameter warning

Signed-off-by: Aditya Ganesh <aditya.ganesh@intel.com>
2026-09-03 09:41:26 +02:00
Benjamin Cabé
833321182a dts: bindings: remove deprecated zephyr,memory-region-mpu
Remove the "zephyr,memory-region-mpu" property, deprecated in Zephyr 3.5
in favor of the "zephyr,memory-attr" bitmask.

The three BUILD_ASSERT tripwires in the ARM/ARM64 MPU drivers already
turned any remaining use of the property into a build failure, so they go
away with it, together with the now unused NODE_HAS_PROP_AND_OR helper.

The migration section in doc/services/mem_mgmt/index.rst documented a
property that no longer exists, so it is dropped and its value mapping
table moved to the 4.5 migration guide.

Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
Assisted-by: Claude:opus-5
2026-09-01 16:25:35 -04:00
Benjamin Cabé
6ff5f3b89e arch: arm: mpu: drop redundant RNR write in ARMv7-M region_init
Writing RBAR with the VALID bit and the region number set also updates
RNR (Arm v7-M ARM, B3.5.7), so explicitly selecting the region with a
separate RNR write first is only needed on Cortex-R, whose RBAR variant
has no VALID/REGION fields. Move the RNR write into the Cortex-R
branch, saving one MPU register write per programmed region on every
context switch when MPU_STACK_GUARD or USERSPACE is enabled.

Measured standalone against main (latency_measure in cycles, lower is
better; thread_metric scores, higher is better):

* mps2/an385 (Cortex-M3, QEMU icount, instruction-exact):
  - latency_measure, stack guard: mean -1.3%, min/max/median
    -3.4/+0.0/-0.4%.
  - latency_measure, userspace: mean -1.1%, min/max/median
    -2.9/+0.0/-0.6%.
  - thread_metric, stack guard: cooperative and preemptive +0.5%.

* az3166_iotdevkit (STM32F412, Cortex-M4 @ 96 MHz):
  - thread_metric: +0.2–0.3%.
  - latency_measure: within the platform's ±2–3% code-placement
    noise (stack guard min/max/median -3.2/+2.4/+0.0%).

* Flash, az3166 stack-guard image: -8 B at -Os, +8 B at -O2.

Assisted-by: Claude:fable-5
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
2026-09-01 16:25:29 -04:00
Anas Nashif
101330945c cpp: name the parameters in the __cxa_atexit declarations
MISRA C:2012 Rule 8.2 requires every parameter in a function type to be
named, including the parameters of function pointer parameters.
The C++ ABI __cxa_atexit() stub, its local prototype in the ARM
__aeabi_atexit() wrapper and that wrapper itself left the destructor's
parameter unnamed.

Name them after the arguments the documentation or the
definitions already use. No functional change.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2026-09-01 16:25:13 -04:00
Laurie Fay
790c09cdcc arch: arm: cortex_m: Auto-enable FP workaround where needed
Automatically select ARM_GCC_FP_WORKAROUND (renamed to
ARM_FP_CLOBBER_WORKAROUND as the workaround is needed for
clang as well as GCC) whenever CONFIG_FRAME_POINTER is set.
CONFIG_FRAME_POINTER passes -fno-omit-frame-pointer as a
compile flag, forcing frame pointers to be enabled on all
files. Thus the FP workaround is needed to manually
spill/restore r7 around the context switch.

Similarly the workaround is needed if the scheduler is using
a red-black tree, as this uses dynamic stack allocation.

Signed-off-by: Laurie Fay <Laurie.Fay@arm.com>
2026-09-01 15:19:11 +01:00
Pei Cheng Sung
0c080b9cc4 arch: arm: add ARM_A_PROFILE_AARCH32 capability flag
The AArch32 A-profile support was shared between ARMv7-A and ARMv8-A
AArch32 by threading "defined(CONFIG_ARMV7_A) ||
defined(CONFIG_AARCH32_ARMV8_A)" through every affected site. Each new
A-profile AArch32 variant would have to be added to all of those guards.

Introduce a hidden capability flag, ARM_A_PROFILE_AARCH32, selected by
both ARMV7_A and AARCH32_ARMV8_A, and migrate the shared guards to the
single CONFIG_ARM_A_PROFILE_AARCH32 symbol. Since the flag is selected
by exactly those two symbols, the guards are logically unchanged; a
future A-profile AArch32 variant now only needs to select the flag to
pick up the shared code paths.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Pei Cheng Sung <pc_sung@realtek.com>
2026-09-01 11:04:18 +01:00
Abderrahmane JARMOUNI
e6140770c5 arch: Kconfig: add range for cache line size
Limit I/DCache line size options' value to
[2, UINT32_MAX] at Kconfig level instead of having
to check the value in code needlessly.
Signed-off-by: Abderrahmane JARMOUNI <git@jarmouni.me>
2026-09-01 11:03:32 +01:00
Abderrahmane JARMOUNI
f04b68e2fd arch: Kconfig: remove I/DCACHE_LINE_SIZE depend on !LINE_SIZE_DETECT
Across the tree, I/DCACHE_LINE_SIZE is required at compile-time for
macros, compiler attributes, & linker scripts to function correctly
even if run-time cache line detection is available.
So remove the dependency of ICACHE_LINE_SIZE on
ICACHE_LINE_SIZE_DETECT=n, and of DCACHE_LINE_SIZE on
DCACHE_LINE_SIZE_DETECT=n.

DCACHE_LINE_SIZE_DETECT & ICACHE_LINE_SIZE_DETECT Kconfig options
enable querying some SoC register for finding the i/d-cache line size
at the expense of taking more memory and code and a slightly
increased boot time.

Add new hidden options D/ICACHE_LINE_SIZE_DETECT_SUPPORT only selected
by the software component (arch/SoC layer, driver) implementing the system
cache API.

Signed-off-by: Abderrahmane JARMOUNI <git@jarmouni.me>
2026-09-01 11:03:32 +01:00
Alberto Escolar Piedras
95b605d7d4 arch: posix: Provide hooks for new irq pending and irq get active API
Provide a standard API towards boards, and hooks at the arch level
so boards can provide the ARCH_HAS_IRQ_PENDING_OPS and
ARCH_HAS_IRQ_GET_ACTIVE kernel APIs.

Note: Boards do not necessarily need to implement this, so this does
not break out of tree boards forcing anybody to immediately provide
those. But it is recommended for all POSIX arch boards to provide them.

Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
2026-08-31 15:11:39 -04:00
Anas Nashif
12ceddf6f2 drivers: intc: irqmp: rename the arch-prefixed source query
The IRQMP driver implements the interrupt source query used by the
SPARC interrupt trap path under an arch-namespace name,
z_sparc_int_get_source(), declared in the SPARC arch header. The
function is driver code, not arch code: rename it to
intc_irqmp_get_source(), declare it in the driver header next to
the rest of the API, and have the trap path call the driver API
directly, matching how the arch_irq_* control functions are already
mapped to this driver in the same file.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2026-08-31 06:59:53 -04:00
Anas Nashif
36865a1767 drivers: intc: irqmp: namespace the driver API
The IRQMP driver defines arch_irq_enable(), arch_irq_disable() and
arch_irq_is_enabled() directly. Interrupt controller drivers should
only expose their own namespaced API; providing the architecture
interrupt control functions is platform glue and does not belong in
a driver.

Rename the functions to intc_irqmp_irq_* and declare them in a
driver header. The arch_irq_* mapping goes into the SPARC arch
layer, not the SoCs: the port is already coupled to an IRQMP-style
controller -- the interrupt trap path queries it for the interrupt
source -- and every SPARC SoC uses it, so mapping it once in
arch/sparc/core/irq_manage.c under CONFIG_LEON_IRQMP serves all of
them. An SoC with a different controller can leave the option
disabled and provide its own implementation.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2026-08-31 06:59:53 -04:00
Hongquan Li
999f33ee60 arch: riscv: smp: panic on secondary hartid lookup failure
If the loop searching for a matching hartid in
arch_secondary_cpu_init() exited without a match, cpu_num would
stay 0 and the secondary hart would initialize itself using
CPU 0's per-CPU state, TLS, PMP and startup callback.

This state is currently unreachable: reset.S gates each secondary
hart until riscv_cpu_wake_flag equals its own mhartid, and the
flag is only ever written from _kernel.cpus[].arch.hartid in
arch_cpu_start(), so a hart whose hartid is absent from the CPU
table never reaches this function. Add the check as defensive
hardening: break on the first match and call k_panic() if the
loop completes without one.

The check sits after the mscratch write because arch_curr_cpu()
reads mscratch, so the fatal path needs per-CPU state set first.

Signed-off-by: Hongquan Li <hongquan.li@processmission.com>
2026-08-31 06:58:37 -04:00
Hongquan Li
181907202d arch: riscv: pass NULL thread to custom stack guard without multithreading
In !CONFIG_MULTITHREADING mode the inline assembly in
z_riscv_switch_to_main_no_multithreading() calls
z_riscv_custom_stack_guard_enable() without setting a0 to the
k_thread * the callee's contract requires. The Andes implementation
only survives because it ignores the argument; any implementation
that dereferences thread faults.

Change the contract to accept a NULL thread in no-multithreading mode
(no thread object exists there) and pass NULL from the caller; the
Andes HSP implementation now checks for NULL and guards the main
stack. Also pin main_entry to callee-saved s1 so the jalr target
survives the call.

Fixes #113190

Signed-off-by: Hongquan Li <hongquan.li@processmission.com>
2026-08-28 15:28:18 -04:00
Lior David
58a459a991 arch/riscv: align TLS storage to ARCH_STACK_PTR_ALIGN
The TLS storage is stored in a reserved stack area, so it
needs to be aligned with ARCH_STACK_PTR_ALIGN (16 bytes),
otherwise the stack pointer gets out of alignment which
violates the RISCV psABI specification.
In addition, the Zcmp instructions such as cm.push and cm.pop
will have undefined behavior if called with unaligned stack
pointer. This can cause unexpected runtime failures.

Signed-off-by: Lior David <liord@qti.qualcomm.com>
2026-08-27 22:11:25 -04:00