Remove the "zephyr,memory-region-mpu" property, deprecated in Zephyr 3.5
in favor of the "zephyr,memory-attr" bitmask.
The three BUILD_ASSERT tripwires in the ARM/ARM64 MPU drivers already
turned any remaining use of the property into a build failure, so they go
away with it, together with the now unused NODE_HAS_PROP_AND_OR helper.
The migration section in doc/services/mem_mgmt/index.rst documented a
property that no longer exists, so it is dropped and its value mapping
table moved to the 4.5 migration guide.
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
Assisted-by: Claude:opus-5
Writing RBAR with the VALID bit and the region number set also updates
RNR (Arm v7-M ARM, B3.5.7), so explicitly selecting the region with a
separate RNR write first is only needed on Cortex-R, whose RBAR variant
has no VALID/REGION fields. Move the RNR write into the Cortex-R
branch, saving one MPU register write per programmed region on every
context switch when MPU_STACK_GUARD or USERSPACE is enabled.
Measured standalone against main (latency_measure in cycles, lower is
better; thread_metric scores, higher is better):
* mps2/an385 (Cortex-M3, QEMU icount, instruction-exact):
- latency_measure, stack guard: mean -1.3%, min/max/median
-3.4/+0.0/-0.4%.
- latency_measure, userspace: mean -1.1%, min/max/median
-2.9/+0.0/-0.6%.
- thread_metric, stack guard: cooperative and preemptive +0.5%.
* az3166_iotdevkit (STM32F412, Cortex-M4 @ 96 MHz):
- thread_metric: +0.2–0.3%.
- latency_measure: within the platform's ±2–3% code-placement
noise (stack guard min/max/median -3.2/+2.4/+0.0%).
* Flash, az3166 stack-guard image: -8 B at -Os, +8 B at -O2.
Assisted-by: Claude:fable-5
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
MISRA C:2012 Rule 8.2 requires every parameter in a function type to be
named, including the parameters of function pointer parameters.
The C++ ABI __cxa_atexit() stub, its local prototype in the ARM
__aeabi_atexit() wrapper and that wrapper itself left the destructor's
parameter unnamed.
Name them after the arguments the documentation or the
definitions already use. No functional change.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Automatically select ARM_GCC_FP_WORKAROUND (renamed to
ARM_FP_CLOBBER_WORKAROUND as the workaround is needed for
clang as well as GCC) whenever CONFIG_FRAME_POINTER is set.
CONFIG_FRAME_POINTER passes -fno-omit-frame-pointer as a
compile flag, forcing frame pointers to be enabled on all
files. Thus the FP workaround is needed to manually
spill/restore r7 around the context switch.
Similarly the workaround is needed if the scheduler is using
a red-black tree, as this uses dynamic stack allocation.
Signed-off-by: Laurie Fay <Laurie.Fay@arm.com>
The AArch32 A-profile support was shared between ARMv7-A and ARMv8-A
AArch32 by threading "defined(CONFIG_ARMV7_A) ||
defined(CONFIG_AARCH32_ARMV8_A)" through every affected site. Each new
A-profile AArch32 variant would have to be added to all of those guards.
Introduce a hidden capability flag, ARM_A_PROFILE_AARCH32, selected by
both ARMV7_A and AARCH32_ARMV8_A, and migrate the shared guards to the
single CONFIG_ARM_A_PROFILE_AARCH32 symbol. Since the flag is selected
by exactly those two symbols, the guards are logically unchanged; a
future A-profile AArch32 variant now only needs to select the flag to
pick up the shared code paths.
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Pei Cheng Sung <pc_sung@realtek.com>
Limit I/DCache line size options' value to
[2, UINT32_MAX] at Kconfig level instead of having
to check the value in code needlessly.
Signed-off-by: Abderrahmane JARMOUNI <git@jarmouni.me>
Across the tree, I/DCACHE_LINE_SIZE is required at compile-time for
macros, compiler attributes, & linker scripts to function correctly
even if run-time cache line detection is available.
So remove the dependency of ICACHE_LINE_SIZE on
ICACHE_LINE_SIZE_DETECT=n, and of DCACHE_LINE_SIZE on
DCACHE_LINE_SIZE_DETECT=n.
DCACHE_LINE_SIZE_DETECT & ICACHE_LINE_SIZE_DETECT Kconfig options
enable querying some SoC register for finding the i/d-cache line size
at the expense of taking more memory and code and a slightly
increased boot time.
Add new hidden options D/ICACHE_LINE_SIZE_DETECT_SUPPORT only selected
by the software component (arch/SoC layer, driver) implementing the system
cache API.
Signed-off-by: Abderrahmane JARMOUNI <git@jarmouni.me>
Provide a standard API towards boards, and hooks at the arch level
so boards can provide the ARCH_HAS_IRQ_PENDING_OPS and
ARCH_HAS_IRQ_GET_ACTIVE kernel APIs.
Note: Boards do not necessarily need to implement this, so this does
not break out of tree boards forcing anybody to immediately provide
those. But it is recommended for all POSIX arch boards to provide them.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
The IRQMP driver implements the interrupt source query used by the
SPARC interrupt trap path under an arch-namespace name,
z_sparc_int_get_source(), declared in the SPARC arch header. The
function is driver code, not arch code: rename it to
intc_irqmp_get_source(), declare it in the driver header next to
the rest of the API, and have the trap path call the driver API
directly, matching how the arch_irq_* control functions are already
mapped to this driver in the same file.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
The IRQMP driver defines arch_irq_enable(), arch_irq_disable() and
arch_irq_is_enabled() directly. Interrupt controller drivers should
only expose their own namespaced API; providing the architecture
interrupt control functions is platform glue and does not belong in
a driver.
Rename the functions to intc_irqmp_irq_* and declare them in a
driver header. The arch_irq_* mapping goes into the SPARC arch
layer, not the SoCs: the port is already coupled to an IRQMP-style
controller -- the interrupt trap path queries it for the interrupt
source -- and every SPARC SoC uses it, so mapping it once in
arch/sparc/core/irq_manage.c under CONFIG_LEON_IRQMP serves all of
them. An SoC with a different controller can leave the option
disabled and provide its own implementation.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
If the loop searching for a matching hartid in
arch_secondary_cpu_init() exited without a match, cpu_num would
stay 0 and the secondary hart would initialize itself using
CPU 0's per-CPU state, TLS, PMP and startup callback.
This state is currently unreachable: reset.S gates each secondary
hart until riscv_cpu_wake_flag equals its own mhartid, and the
flag is only ever written from _kernel.cpus[].arch.hartid in
arch_cpu_start(), so a hart whose hartid is absent from the CPU
table never reaches this function. Add the check as defensive
hardening: break on the first match and call k_panic() if the
loop completes without one.
The check sits after the mscratch write because arch_curr_cpu()
reads mscratch, so the fatal path needs per-CPU state set first.
Signed-off-by: Hongquan Li <hongquan.li@processmission.com>
In !CONFIG_MULTITHREADING mode the inline assembly in
z_riscv_switch_to_main_no_multithreading() calls
z_riscv_custom_stack_guard_enable() without setting a0 to the
k_thread * the callee's contract requires. The Andes implementation
only survives because it ignores the argument; any implementation
that dereferences thread faults.
Change the contract to accept a NULL thread in no-multithreading mode
(no thread object exists there) and pass NULL from the caller; the
Andes HSP implementation now checks for NULL and guards the main
stack. Also pin main_entry to callee-saved s1 so the jalr target
survives the call.
Fixes#113190
Signed-off-by: Hongquan Li <hongquan.li@processmission.com>
The TLS storage is stored in a reserved stack area, so it
needs to be aligned with ARCH_STACK_PTR_ALIGN (16 bytes),
otherwise the stack pointer gets out of alignment which
violates the RISCV psABI specification.
In addition, the Zcmp instructions such as cm.push and cm.pop
will have undefined behavior if called with unaligned stack
pointer. This can cause unexpected runtime failures.
Signed-off-by: Lior David <liord@qti.qualcomm.com>
On every non-nested interrupt exit, the ISR wrapper called
z_get_next_switch_handle() to ask the scheduler whether a context
switch is needed, including the stack juggling needed around the call
and a second stack sentinel check inside the callee (isr.S already
performs one on this path). For the vast majority of interrupts no
reschedule is needed and all of that work amounts to nothing.
On uniprocessor builds the scheduler's decision is fully captured by
_kernel.ready_q.cache: when it still designates the current thread,
z_get_next_switch_handle() reduces to no-op bookkeeping. Add a short
inline comparison (four instructions) to the interrupt exit path and
skip the call entirely in that case, the same way the Arm Cortex-M
exception exit decides whether to pend PendSV. The fast path is
disabled on SMP (where the decision requires the scheduler lock) and
when thread usage accounting needs to run on every switch decision.
Measured on ESP32-C6 @ 160 MHz (m5stack_nanoc6/esp32c6/hpcore):
* latency_measure: return from ISR to interrupted thread
464 -> 401 ns (-14%); return from ISR to a different thread
1099 -> 1124 ns (+2.3%, the added instructions, taken only when
an actual reschedule follows); all other metrics unchanged. The
regression is a constant cost paid only when a context switch
follows anyway, and is amortized: the change nets out positive
whenever more than ~28% of interrupt exits (~10% on QEMU) return
to the interrupted thread, which tick and device interrupts
overwhelmingly do.
* thread_metric: interrupt processing +3.3% (10303330 -> 10645986);
interrupt preemption -0.5% (5027551 -> 5001353), every interrupt
there reschedules so it only pays the added check; preemptive and
cooperative scheduling unchanged (<0.01%).
Measured on qemu_riscv32 (QEMU icount, deterministic):
* latency_measure: return from ISR to interrupted thread
50 -> 31 cycles (-38%); return from ISR to a different thread
63 -> 65 cycles (+3.2%).
* thread_metric: interrupt processing +9.6% (1420879 -> 1557774);
interrupt preemption -0.45% (713682 -> 710438).
Code size: +12 B text (+0.03% at -Os, +0.02% at -O2) on
qemu_riscv32; +16 B (+0.02%, -Os) on ESP32-C6.
Assisted-by: Claude:fable-5
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
Remove remaining uses of the internal __ASSERT_ON macro. Let __ASSERT()
handle disabled assertions, mark assert-only values as unused where needed,
and use CONFIG_ASSERT for assertion-only state.
Signed-off-by: Måns Ansgariusson <mansgariusson@gmail.com>
USE_SWITCH isn't yet supported for a non-secure Zephyr image on
Cortex-M, as CONFIG_ARM_STORE_EXC_RETURN is not implemented.
Prevent such a non-functional build by only selecting
USE_SWITCH_SUPPORTED if Zephyr is not running as non-secure.
(USE_SWITCH is fine if Zephyr is running as secure or if there
is no security extension.)
Signed-off-by: Laurie Fay <Laurie.Fay@arm.com>
Updates the fast-path assembly in isr.S to explicitly filter out
vector instructions that share opcodes with FP loads/stores,
preventing them from corrupting the FPU context state.
Additionally, introduces CONFIG_RISCV_FPU_INSN_VALIDATE (default
'y' with CONFIG_ASSERT) to provide a C-level runtime validator in
z_riscv_fpu_trap(). This intercepts misrouted non-FPU instructions,
prints the faulting opcode, and throws a kernel oops.
Fixes#96551
Signed-off-by: Kaveesha Yalegama <kaveesha.yalegama@gmail.com>
intc_system_apic.c is not an interrupt controller driver: it is the
x86 platform glue that implements arch_irq_enable(),
arch_irq_disable() and the vector programming hook by dispatching
between the IOAPIC and LOAPIC drivers, which expose their own APIs.
Interrupt controller drivers should only expose their own namespaced
API, so move the file to arch/x86/core where the glue belongs,
unchanged apart from a comment stating its role.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
ARMv9-A CPUs such as Cortex-A720 select ARMV9_A instead of ARMV8_A.
Allow Xen support to be enabled on these platforms.
Signed-off-by: Nam Nguyen <nam.nguyen.xn@renesas.com>
Vendor HAL interrupt handlers are often entered without their vector
number and have to discover it themselves. On Cortex-M the IPSR
register answers that, but on Cortex-A/R with a GIC the INTID exists
only at acknowledge time inside the ISR wrapper. The Renesas RZ SoCs
work around this today by selecting ARM_CUSTOM_INTERRUPT_CONTROLLER
with pass-through z_soc_irq_* wrappers around the standard GIC driver,
purely to intercept the ack/eoi moments and log the INTID into a
fixed-depth side stack for the FSP HAL, duplicated across six SoCs.
Add k_irq_get_active(), backed by arch_irq_get_active(): the
interrupt line whose handler is executing on the current CPU, or
K_IRQ_ACTIVE_NONE outside interrupt context. The capability symbol
ARCH_HAS_IRQ_GET_ACTIVE only promises the query; whether an
architecture reads it from a register or records it in the dispatch
path is its own business.
On Cortex-A/R recording is the only option, so the implementation is
gated behind an arch-level opt-in, ARM_TRACK_ACTIVE_IRQ, keeping the
cost decision where the cost lives: the shared ISR wrapper publishes
the acknowledged INTID in a per-CPU field of _cpu_arch and keeps the
previous value on the exception stack across the dispatch, so nested
interrupts unwind to the preempted INTID and the outermost exit back
to "none". The stored value is biased by one so the zero-initialized
boot state reads as "none" on every CPU without explicit
initialization, including secondary SMP cores. The wrapper records
whatever get_active returned, so it works with both the GIC and a
custom interrupt controller. Cost is zero when the option is off and
a few instructions per interrupt entry and exit when on.
Assisted-by: Claude:claude-fable-5
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Drivers that need to touch an interrupt's latched pending state
currently reach into the interrupt controller themselves.
NVIC_ClearPendingIRQ() alone appears at 88 call sites across 50 files
under drivers/, NVIC_SetPendingIRQ() at 34 (26 of them in
drivers/counter/, which pends its own IRQ to fire an alarm whose
deadline has already passed) and NVIC_GetPendingIRQ() at 11. Each
site pulls cmsis_core.h and a CONFIG_CPU_CORTEX_M guard into
otherwise portable code, and several counter drivers had already
written private GIC-or-NVIC dispatch helpers for exactly these
operations.
Add k_irq_set_pending(), k_irq_clear_pending() and k_irq_is_pending()
as ALWAYS_INLINE wrappers over matching arch_irq_* functions, and
implement them for the ARM NVIC (AArch32 Cortex-M) and the GIC
(AArch32 Cortex-A/R and AArch64), where the GIC driver already
exported all three as arm_gic_irq_*.
The k_ prefix is deliberate: vendor HAL headers already declare
plain irq_set_pending()/irq_clear_pending() as their own functions
(the Realtek Ameba ROM ABI, pico-sdk's hardware/irq.h), and several
drivers carry private static helpers or API struct members with the
bare names. A namespaced API collides with none of them, needs no
macro tricks, and no renaming of existing code.
The operations are gated on a single capability symbol,
CONFIG_ARCH_HAS_IRQ_PENDING_OPS, so an unported target fails to
build instead of silently doing nothing. Both implemented backends
support all three operations; should an architecture with partial
support materialize (a RISC-V PLIC can report pending state but only
clears it by claiming, and cannot latch from software), the symbol
can be split then.
Assisted-by: Claude:claude-fable-5
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
The RISC-V specific shadow definition of EXTRA_EXCEPTION_INFO was
deprecated in Zephyr 4.3 and is now removed as part of the 4.5
deprecation removal cycle.
The RISC-V exception handling code (isr.S, fatal.c, coredump.c and the
offsets definitions) has been keyed off CONFIG_EXCEPTION_DEBUG since the
deprecation, so removing the symbol requires no code conversion. RISC-V
applications that still set CONFIG_EXTRA_EXCEPTION_INFO must use
CONFIG_EXCEPTION_DEBUG instead.
Note this only drops the deprecated RISC-V local definition. The generic
EXTRA_EXCEPTION_INFO option in arch/Kconfig, guarded by
ARCH_HAS_EXTRA_EXCEPTION_INFO and used by Arm and SPARC, is unaffected.
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
Assisted-by: Claude:opus-5
The ra register is caller-saved. For a non-leaf function interrupted
after a call, esf->ra points back into that function instead of to its
caller. Emitting it unconditionally adds a bogus frame.
A leaf frame may have no saved return address, so its direct caller must
come from esf->ra. Detect the compact leaf layout using a validated,
monotonically increasing caller frame pointer, then continue through the
normal walk loop. This preserves callback and maximum-depth semantics.
Request leaf frame pointers when CONFIG_FRAME_POINTER is enabled so
supporting compilers cannot omit the frame entirely. Older compact leaf
frames remain supported.
Add exact RV32E, RV32, and RV64 traces for leaf and non-leaf exception
frames. Trigger the fault with an illegal instruction so M-mode and
S-mode exercise the same exception path without relying on ebreak
behavior.
Signed-off-by: Jinming Zhao <jinmzhao@qti.qualcomm.com>
- When launching Zephyr from a bootloader (eg: u-boot), the image binary
launches in S-mode. Thus we cannot call M-mode only functions, or
perform M to S mode transition.
- SMP is currently broken in this setup. MPU, FPU not tested yet either
since PolarFire soc does not enable it by default.
- Tested on beaglev_fire with PolarFire soc.
Signed-off-by: Ayush Singh <ayush@beagleboard.org>
arch_mem_page_out()/arch_mem_page_in() and the AF/dirty sync paths call
sync_domains(), which globalizes the affected range into every memory
domain. globalize_table() replaced each destination leaf entry with the
global one, stripping the EL0 accessibility and nG attributes that
private_map() had set, so user threads lost access to their partitions
and stacks as soon as demand paging touched them.
Add a preserve_private mode to the globalization walk that keeps the
private permission attributes and only propagates the structural state
(descriptor type, physical address or location token, access flag). The
read-only bit is dirty-tracking state only where the mapping is
writable, so it is propagated only when the private entry marks the page
writable; a genuinely read-only private mapping keeps its RO bit even
after the kernel dirties the global entry. The guarded-page bit stays
on the private side as well, so BTI-guarded executable partitions
survive the sync.
The same paging updates also invalidate the TLB, but
invalidate_tlb_page() issues TLBI VAE1{IS} with the ASID field zero,
while domain page tables are allocated ASIDs from 1 up and their
entries are non-global: a stale translation survives under each
domain's ASID. For a page out that leaves a thread with a valid TLB
entry pointing at a frame the kernel may have handed to someone else;
for the dirty-tracking write the retried access faults again on a page
the kernel already made writable; and for the AF clear in
arch_page_info_get() the eviction algorithm stops seeing accesses and
NRU evicts hot pages. All callers are such paging paths, so switch the
implementation to TLBI VAAE1{IS}, which matches every ASID.
Signed-off-by: Hongquan Li <hongquan.li@processmission.com>
z_arm64_do_demand_paging() only accepted instruction/data aborts taken
from the current EL (EC 0x21/0x25). Aborts taken from EL0 (EC 0x20/0x24)
fell through to the fatal error path, so any user-mode access to a
paged-out page killed the thread instead of paging it back in.
Accept lower-EL aborts as well, and validate the access against the
faulting thread's own page tables: an abort from EL0 was raised against
the current domain's tables, so unless the leaf entry there grants EL0
the attempted access the fault is genuine and must not reach
k_mem_page_fault() or the AF/dirty handling on behalf of an
unprivileged thread. The validation walks the domain tables under
xlat_lock, as sync_domains() may free domain subtables on another CPU,
and requires more than AP_ELx: UXN must be clear for instruction
aborts, and PTE_SW_WRITABLE must be set for data writes, since the
dirty path judges writability from the kernel entry — without that
check a user write to a read-only partition page would still flip
ARCH_DATA_PAGE_DIRTY before the retry kills the thread.
Signed-off-by: Hongquan Li <hongquan.li@processmission.com>
The inline asm that hands control to main with CONFIG_MULTITHREADING=n
left the l.jalr delay slot empty, so the first instruction of the
following irq_lock() ran in it and l.jalr pointed r9 at the second
one. main returned mid-sequence and the leftover scratch register was
written to SR, clearing SM and enabling the IMMU; qemu_or1k faulted
with an I-TLB miss right after the test reported success.
Emit the l.nop explicitly so the link address lands past it on both
delay-slot and no-delay cores, and add the memory clobber the other
ports use.
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
Assisted-by: Claude:opus-5
EXCVADDR describes exceptions, not interrupts, and only the level 1
vector serves exceptions. The frames built by interrupt-only vectors
never have their excvaddr slot read (the C interrupt handlers do not
look at it, and nothing restores it on exit), so stop saving it
there. Fatal errors raised from ISRs still get a proper EXCVADDR:
they trap through the level 1 exception vector, which keeps the save.
Benchmarks on qemu_xtensa/dc233c (QEMU icount shift=6, 1 insn = 1
cycle) and ESP32-S3 (m5stack_stamps3 at 240 MHz), exercising a
level 3 software interrupt:
* thread_metric interrupt (higher is better):
- qemu_xtensa/dc233c: 759968 -> 762439 (+0.3%)
- ESP32-S3: 9281007 -> 9304992 (+0.3%)
* latency_measure: unchanged within +/-1 cycle alignment noise (its
ISR metrics start timing inside the ISR, after vector entry)
* code size: unchanged (vector slots are fixed-size)
Assisted-by: Claude:fable-5
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
With KPTI enabled, when a page fault happened, code would never load the
page when it should due an `ifdef` confusion.
Fixes: #116502
Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
set_pmp_mem_attr() ignored the result of set_pmp_entry(), so when the
zephyr,memory-attr regions needed more PMP slots than available, the
trailing regions were silently dropped and their permissions never
enforced (default-allowed per the RISC-V privileged spec).
Log the region that could not be installed and call k_panic() so a
configuration that cannot be honored fails the boot loudly instead of
silently degrading security.
Fixes#113777
Signed-off-by: Hongquan Li <hongquan.li@processmission.com>
Apply the ARG_UNUSED() macro to resolve unused parameter warnings
when building with "-Wextra".
Assisted-by: GitHub Copilot:GPT-5.3-Codex
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
Apply the ARG_UNUSED() macro to resolve unused parameter warnings
when building with "-Wextra".
Assisted-by: GitHub Copilot:GPT-5.3-Codex
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
Remove CONFIG_PLATFORM_SPECIFIC_INIT and its z_arm_platform_init() hook,
deprecated in favour of CONFIG_SOC_RESET_HOOK / soc_reset_hook() well
before Zephyr 4.2 and therefore due for removal in 4.5.
The option had no reader left: neither cortex_m/reset.S nor
cortex_a_r/reset.S tests it, both only branch to soc_reset_hook(). Its
single in-tree selector, soc/renode/cortex_r8_virtual, is migrated to
select SOC_RESET_HOOK and its hook renamed to soc_reset_hook(); the body
is unchanged. Note that this re-activates the SCTLR.V clear on that SoC,
which had silently been dead code since the reset paths stopped calling
z_arm_platform_init() - the SoC now again selects the low exception
vector base as originally intended. It runs slightly later in the reset
sequence, after the per-mode stack pointers are set up. This should be
smoke-tested under Renode.
The stale z_arm_platform_init mention in the IAR linker script helper
comment is updated to soc_reset_hook.
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
Assisted-by: Claude:opus-5
Remove the deprecated CONFIG_SSE and CONFIG_SSE_FP_MATH aliases, which
were only selecting CONFIG_X86_SSE and CONFIG_X86_SSE_FP_MATH. They have
carried a "This option is deprecated" help text since well before Zephyr
4.2, and are due for removal in 4.5.
include/zephyr/arch/x86/ia32/arch.h was the last in-tree reader, and it
tested CONFIG_SSE while every other consumer (crt0.S, float.c, swap.S,
ia32/thread.h) already keys off CONFIG_X86_SSE. It now tests
CONFIG_X86_SSE too, so a configuration that enables X86_SSE directly
gets the required 16-byte dynamic thread object alignment for
fxsave/fxrstor instead of pointer alignment.
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
Assisted-by: Claude:opus-5
Commit d78bf6c165 ("kconfig: Deprecate LEGACY_GENERATED_INCLUDE_PATH")
made it mandatory that generated includes need to include the
"zephyr" namespace, which was missed for a shadow stack related
include on `locore.S`.
Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
Align with native_simulator's upstream main
3f81511a813ca38e9afac430f15a19ee53c9f407
Which includes:
3f81511 native: timer_model: Improvements
bc98cfe nsi_errno: Fix typo in comment
8961d7a nsi_scheduler: Terminate gracefully at end of time if no end of
time set
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
printk_unlocked() serves callers that know they cannot lock, not the ones
that find out too late. A fault taken inside printk() leaves the spinlock
held by a context about to die, so every later printk() blocks and the
crash report is lost along with everything after it. CONFIG_SPIN_VALIDATE
makes it louder rather than better: validation fails, the assertion is
reported through printk, and the system spins there repeating itself.
That happens on a uniprocessor too, where the lock is uncontended but
still recorded as held.
Converting call sites cannot fix it, because code reached after the
failure does not know it is in a crash. So make printk() itself aware:
printk_panic() switches it to the unlocked path and the reporting paths
call it before emitting anything.
The switch is one way. Clearing it would also require releasing the
orphaned lock, and telling "orphaned" from "held by a CPU that is still
running" needs the owner, which only CONFIG_SPIN_VALIDATE records. So a
system that has reported a fatal error keeps unserialized output for the
rest of its life, which costs interleaving and no content.
Linux does the same: bust_spinlocks(), called from its oops and panic
paths, raises oops_in_progress so printk skips console locking. This
mechanism is narrower, printk simply stops locking, but the reasoning is
identical.
Because it cannot be undone, it must only be thrown once the fault is
terminal, which rules out the architecture entry points: several return.
z_arm64_fatal_error() resumes once demand paging has serviced the fault,
so switching there would cost a paging system its printk locking on the
first page fault. EXCEPTION_DUMP() runs only after the recoverable cases
are ruled out and also covers arch-specific dumps such as the Cortex-M
one behind PR_FAULT_INFO, and z_fatal_error() covers reports with no
dump, like k_panic() from a failed assertion. OpenRISC switches at entry
because it reports with LOG_ERR(), which is safe there since its handler
never returns.
k_str_out() shares the lock and honours the switch, so crashes reported
through printf() or puts() are covered too.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The page table helpers run before the MMU is enabled, while the tables
are being built, and again at run time from arch_mem_map(). In the first
phase neither the logging subsystem nor printk()'s spinlock is usable,
because the atomics under them need the shareable memory attributes the
MMU provides, so a diagnostic there faults instead of printing. That is
where this started: a page table failure during early bring-up produced
silence.
Test the MMU and pick the path accordingly. mmu_is_enabled() reads
SCTLR_EL1 and tests M, which compiles to an mrs and a tbz. With the MMU
on the messages take the ordinary LOG_*() path, keeping level filtering,
timestamps, the backends and serialized output. With it off they degrade
to an unlocked printk() to the character hook.
Gating on the log core's own Z_LOG_CONST_LEVEL_CHECK() keeps the
build-time level filtering: a filtered-out level drops both branches and
its arguments. Verified on qemu_cortex_a53, where the INF format string
is in the binary with CONFIG_KERNEL_LOG_LEVEL_INF and gone with
CONFIG_KERNEL_LOG_LEVEL_ERR.
Errors here precede a k_panic() and must appear with logging compiled out
too, so they keep this file's existing printk() fallback.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
z_arm64_sync_exc() demuxes SVC exceptions by the low 8 bits of the
SVC immediate (svc #1: IRQ offload, svc #2: runtime except, svc #3:
system call) without checking which exception level the SVC was
taken from. On ARM64, SVC from EL0 and EL1 both report ESR EC 0x15,
so hardware does not distinguish them.
With CONFIG_USERSPACE=y and CONFIG_IRQ_OFFLOAD=y, any user thread can
execute a raw svc #1 with x0/x1 pointing at an arbitrary kernel
function and argument; the handler then runs 'blr x1' at EL1, i.e.
privilege escalation / arbitrary code execution in kernel mode. The
RISC-V port avoids this class of bug by distinguishing ECALLU/ECALLM
via mcause.
Note that CONFIG_IRQ_OFFLOAD depends on TEST (arch/Kconfig), so this
is not reachable in a production build; only test builds that enable
irq_offload() are exposed. This distinction matters for backport
decisions.
Read the saved SPSR from the ESF and route any SVC taken from EL0 to
a user-only path that accepts just the system call and runtime-except
immediates. The runtime-except entry must stay reachable from EL0
because user threads raise stack-check failures and oopses through
it; z_arm64_do_kernel_oops() sanitizes the user-supplied reason, so
this is not a privilege boundary. All other EL0 SVCs raise a fatal
CPU exception. Kernel-mode SVCs keep their previous behavior, except
that svc #3 from EL1 is no longer dispatched to the syscall handler,
which kernel code never uses.
Fixes#116254
Signed-off-by: Hongquan Li <hongquan.li@processmission.com>
Add 'config CPU_CORTEX_A5' to arch/arm/core/cortex_a_r/Kconfig file.
Set 'GCC_M_CPU' to 'cortex-a5' in cmake/gcc-m-cpu.cmake for the
compiler.
Signed-off-by: Tony Han <tony.han@microchip.com>
Expose the Xen xsm_op hypercall through the arm64 Xen hypercall
interface.
The arm64 Xen backend already keeps raw hypercall entry points in
hypercall.S and declares them in <zephyr/arch/arm64/hypercall.h>. Add
the missing xsm_op entry point there so callers that need Xen security
operations can issue the hypercall directly.
Signed-off-by: Vladyslav Goncharuk <vladyslav_goncharuk@epam.com>
Assisted-by: Codex:gpt-5.5
Expose the Xen vcpu_op hypercall through the arm64 Xen hypercall
interface.
The arm64 Xen backend already keeps raw hypercall entry points in
hypercall.S and declares them in <zephyr/arch/arm64/hypercall.h>. Add
the missing vcpu_op entry point there so callers that need Xen vCPU
operations can issue the hypercall directly.
Signed-off-by: Vladyslav Goncharuk <vladyslav_goncharuk@epam.com>
Assisted-by: Codex:gpt-5.5
MWDT does not honor memory orders on __atomic_* or emit fences for
__sync_synchronize() / __atomic_thread_fence(). Route ARC HS MWDT builds
through ARC_MWDT_MEM_ORDERING_WORKAROUND, which selects arch atomics and
barriers implemented in atomic_arc.h and barrier.h.
GCC ARC builds keep ATOMIC_OPERATIONS_BUILTIN and
BARRIER_OPERATIONS_BUILTIN. Revert when MWDT fixes ordering in the
toolchain/runtime.
Signed-off-by: Mohamed Moawad <melhosar@gfoundries.com>
Add memory ordering around the arc_cpu_wake_flag / arc_cpu_sp protocol
used when arch_cpu_start() brings up secondary cores.
The primary must publish _curr_cpu[], arc_cpu_init[], and arc_cpu_sp
before the wake flag is visible, and must not reuse arc_cpu_sp until
the secondary acknowledges. The secondary must not load sp until it
has observed its wake flag, and must complete that load before
clearing the flag.
Use barrier_dmem_fence_full() in arch_cpu_start() and dmb 1 / dmb 3
in the secondary wait path in reset.S.
Signed-off-by: Mohamed Moawad <melhosar@gfoundries.com>
Ensure callee-saved context stores are globally visible before
switch_handle is published on SMP. Other cores poll switch_handle in
z_sched_switch_spin() and may restore context immediately; ARC
store buffering can otherwise expose the publish too early.
Add "dmb 2" in _store_old_thread_callee_regs and fault_s.S.
Signed-off-by: Mohamed Moawad <melhosar@gfoundries.com>
The CFSR register uses write-1-to-clear semantics for all fault
status bits per the ARMv7-M/ARMv8-M Architecture Reference Manual.
Two locations in the MemManage and BusFault handlers incorrectly
used `&= ~Msk` (write-0) instead of `= Msk` (write-1), which has
no effect on these sticky bits.
This caused stale VALID bits to persist, leading subsequent fault
handlers to read expired MMFAR/BFAR addresses and potentially
misdiagnose stack overflows.
Fixeszephyrproject-rtos/zephyr#113515
Signed-off-by: Hongquan Li <hongquan.li@processmission.com>
MISRA C:2012 Rule 8.2 requires every parameter in a function type to be
named, including the parameters of function pointer parameters.
z_isr_install(), z_isr_uninstall() and the generic
arch_irq_connect_dynamic() fallback left the ISR they take unnamed, in
the header and in the definitions.
Name them after the arguments the documentation or the
definitions already use. No functional change.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Previously, every asid_free() immediately flushed TLB autorefill
entries on all cores via IPI. This is expensive for workloads that
rapidly create and destroy memory domains (e.g. SOF DP tasks), since
every domain teardown triggers a system-wide TLB invalidation even
when freed ASIDs may never be reused before fresh ones are available.
Add a second bitarray (asid_dirty) where freed ASIDs are placed
without touching the TLB. asid_alloc() uses sys_bitarray_alloc() on
asid_used to find a free ASID; when none remain, performs a single
TLB flush and XORs dirty bits back into asid_used (clearing the
in-use bits for reclaimed ASIDs), amortizing the flush cost across
many alloc/free cycles.
Signed-off-by: Jyri Sarha <jyri.sarha@linux.intel.com>
