Everything the primary core writes before the MMU is enabled bypasses
the data cache: the saved firmware arguments, the boot stack, .data,
.bss and then the translation tables built by z_arm64_mm_init(). Lines
a previous boot stage left over that RAM are untouched by those writes.
A clean one is read back in place of what was written once
enable_mmu_el1() turns on the MMU and data cache, and a dirty one can be
written back over it at any point before then. Either way the first
table walk or data access after the MMU enable may see garbage, which
shows up as a random crash right at that point.
Commit 6c6f1a5e99 ("arch: arm64: mmu: revert useless cache handling")
removed the invalidation setup_page_tables() used to do over these
ranges along with the per-mapping one, leaving nothing to cover this.
Invalidate the image RAM by VA in reset.S as soon as the primary core is
known, before the firmware arguments and the boot stack are written.
DC IVAC reaches caches beyond the cluster, which set/way operations do
not, and invalidating rather than cleaning avoids writing the stale
lines back. The boot vote before that point only writes
arm64_cpu_boot_params in .data, which the loader must already have left
clean, so lines over it can only be clean and are dropped as well.
Secondary cores must not do this: they start while the primary core
runs with its data cache on.
This does not depend on CONFIG_DCACHE: the data cache is enabled along
with the MMU either way.
Reproduced on fvp_corstone1000/a320 with cache state modelling enabled
and code standing in for a previous boot stage that leaves dirty lines
over .bss before jumping to Zephyr: the boot hangs without this change
and completes with it.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
With the MMU off, a core's stack writes go straight to memory. Once
SCTLR_EL1.C is set, a stale cached line for the same address can be
read back instead, for example as a saved x19 or link register, and
the core faults during bring-up.
Invalidate the calling core's z_interrupt_stacks slot, found from sp,
right before the MMU is enabled. Other slots are left alone: they may
be live stacks of running cores, holding dirty lines. The arch cache
op is used, as sys_cache_data_invd_range() does nothing without
CONFIG_CACHE_MANAGEMENT.
Seen on RK3588 as a rare fault on a secondary core right after
z_arm64_mm_init().
Fixes#119630
Signed-off-by: Piter Zhang <zhangtdvip@163.com>
So far, forever busy loop with interrupts disabled was executed when
CONFIG_MULTITHREADING=n and main() returned. Change it to forever
k_cpu_idle() which is better from power consumption perspective.
Signed-off-by: Krzysztof Chruściński <krzysztof.chruscinski@nordicsemi.no>
Utilize a code spell-checking tool to scan for and correct spelling errors
in `arch/Kconfig`.
Assisted-by: Claude:claude-sonnet-5-5 codespell
Signed-off-by: Pisit Sawangvonganan <pisit@ndrsolution.com>
NULL_POINTER_EXCEPTION_REGION_SIZE defaults to 16 bytes, less than the
128-byte PMP granularity of the ESP32-C5, so the null pointer guard
always fails the alignment assertion in set_pmp_entry(), which GCC
turns into an -Warray-bounds error in z_riscv_pmp_kernelmode_disable().
Round the size up to the PMP granularity.
Assisted-by: Claude:opus-5.5
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
Pair the ARC range operations' arch_irq_lock() with arch_irq_unlock()
to avoid decrementing an outer SMP global lock. Use local IRQ locking
for the all-cache operations too; shared SLC locking is unchanged.
Pair irq_lock() with irq_unlock() in the ESP32 USB serial callback to
balance its SMP global lock state.
Signed-off-by: Jinming Zhao <jinmzhao@qti.qualcomm.com>
arch_mem_domain_partition_remove() ignored partition_id and rescanned
the whole domain, but the kernel zeroes partitions[p_idx].size only
after the arch hook returns, so the removed partition was programmed
right back into every started thread's MPU context: revocation of
dynamic memory domain partitions was silently ineffective.
Additionally, the partition loop in configure_dynamic_mpu_regions()
decremented the remaining-partition counter in the loop increment
expression, so it also ran for zero-sized holes in partitions[].
Each hole made the loop terminate one valid partition early, leaving
legitimately permitted memory unmapped and accesses to it faulting.
Exclude the removed partition from the rescan, and decrement the
remaining-partition counter only for non-empty partitions, matching
the arm32 implementation in arm_core_mpu.c.
Fixes: #115177
Signed-off-by: Hongquan Li <hongquan.li@processmission.com>
Replace scripts/kconfig/hardened.csv with a distributed YAML database.
The CSV format could not carry any explanation for its recommendations,
had no validation (a malformed line was silently dropped), no notion of
profiles, no ownership story, and no way to extend it out of tree.
The database now lives where the code lives: the central
scripts/kconfig/hardening.yaml defines the hardening profiles (plus
rules for top-level Kconfig symbols), and per-subsystem hardening.yaml
fragments next to the relevant Kconfig files carry the rules, so each
rule is reviewed by the maintainers of the code it hardens. Every rule
has a required rationale and optional references, supports exact-value
and integer min/max recommendations (STACK_POINTER_RANDOM is now
'>=100' instead of '==100'), and belongs to a profile: 'base' (memory
protection and inherently insecure features) or 'strict' (extends base
with debug/observability feature removal). The default profile is
'strict', matching the rule set the CSV checked.
hardenconfig is restructured around a serializable HardeningReport
model so results can be consumed programmatically, and gains:
- a Rationale column in the report
- profile selection:
west build -t hardenconfig -- -DHARDENCONFIG_PROFILE=base
- HARDENCONFIG_SHOW_ALL to also list PASS/NA results
- HARDENCONFIG_STRICT to exit non-zero on failures, for CI use
- HARDENCONFIG_JSON to write machine-readable results, including which
database file each rule came from
- HARDENCONFIG_EXTRA_SOURCES to layer out-of-tree database files that
may add profiles/rules and override in-tree ones
The EXPERIMENTAL/DEPRECATED/NOT_SECURE marker symbol detection is
preserved unchanged.
Three CSV entries are intentionally not carried over because they were
redundant: BT_CONN_DISABLE_SECURITY and BT_OOB_DATA_FIXED can only be
enabled together with BT_TESTING, and TIMER_RANDOM_GENERATOR together
with TEST_RANDOM_GENERATOR, all of which the database already
recommends disabling for the same profiles.
Behavioral fix: the CSV's DEBUG_COREDUMP line used '=' instead of ','
and was silently ignored, so CONFIG_DEBUG_COREDUMP was never actually
checked; it now is.
Also add tabulate to requirements-base.txt: hardenconfig has always
imported it, but it was only pulled in via other requirements files.
Assisted-by: Claude:fable-5
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
Configure MNSTATUS.NMIE for secondary cores to enable NMI handling.
The current configuration to enable it in reset.S only applies to
the primary core.
Signed-off-by: Omar Naffaa <onaffaa@qti.qualcomm.com>
The new RX74x MCU from Renesas is using RXv3e core which have
new ISA, this commit is only aim to add the definition support
for this target.
Adding bindings for rx cpu core version
Signed-off-by: Duy Nguyen <duy.nguyen.xa@renesas.com>
irq_offload() uses a global semaphore when
ARCH_HAS_IRQ_OFFLOAD_NESTED is not selected. This serializes offloads
across all CPUs and causes kernel.smp_abort to deadlock before reaching
k_thread_abort(), since both CPUs must enter their offloaded handlers
simultaneously.
ARC keeps IRQ offload parameters per CPU and supports concurrent
offloads across SMP CPUs. Select ARCH_HAS_IRQ_OFFLOAD_NESTED for SMP
builds to bypass the global semaphore.
This fixes kernel.smp_abort on ARC SMP targets.
Signed-off-by: Mohamed Moawad <melhosar@gfoundries.com>
When a double exception escalates to a triple fault, the Xtensa CPU
escapes into an unrecoverable spin loop.
Add a weak xtensa_triple_fault_breadcrumb symbol, called via CALL0 from
the shared _TripleFault handler in xtensa_asm2_util.S before entering the
spin loop. The default is a no-op ret; an SoC family that wants to record
triple fault state links in its own strong override.
Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
When a fatal exception is taken on Xtensa, crash diagnostics are lost
if no console or mtrace output is available.
Add a generic architecture-level fatal exception breadcrumb API:
- xtensa_fatal_breadcrumb() provides a weak no-op default.
- XTENSA_RECORD_FATAL_BREADCRUMB() macro hooks into the default
exception switch in vector_handlers.c.
- An SoC family that supports breadcrumbs provides a strong override
to record the faulting PC, exception cause, and virtual address into
its own diagnostic storage.
Signed-off-by: Liam Girdwood <liam.r.girdwood@linux.intel.com>
The kernel creates one idle thread per CPU, so the number of thread
objects a userspace build needs grows with CONFIG_MP_MAX_NUM_CPUS, yet
the default stayed at 2 and nineteen boards hand-tuned it instead. The
values disagree: six four-CPU boards pick 3 or 4, and qemu_cortex_a72
sets 4 in its defconfig and 3 in its Kconfig.defconfig. Derive the
default from the CPU count and drop the board settings it covers.
Boards that need more than the CPU count implies keep their own value.
No board's effective value goes down.
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
Assisted-by: Claude:opus-5
HCR_EL2 may be left configured by firmware/loaders before handing off
to Zephyr at EL2. z_arm64_el2_init() already clears TGE/AMO/IMO but not
FMO. With HCR_EL2.FMO=1, an EL1 write to ICC_SGI1R_EL1 is trapped to
EL2 before any SGI is generated. Zephyr runs at EL1 and installs no EL2
vectors, so the first IPI takes this trap with no handler in place and
crashes in firmware (observed on Phytium D3000M, where firmware leaves
FMO set on the primary CPU only).
Clear FMO alongside IMO/AMO so the ICC_SGI1R_EL1 access is not trapped
to EL2.
Signed-off-by: Chang Jianpeng <changjianpeng@kylinos.cn>
Guard references to _sw_isr_table in irq_manage.c so that applications can
build without generated ISR tables when interrupts are unused.
Assisted-by: Gemini:gemini-3.8-flash
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Make GEN_SW_ISR_TABLE depend on GEN_ISR_TABLES, and guard the import of
_sw_isr_table in isr.S so that applications can build without generated
ISR tables when interrupts are unused.
Assisted-by: Gemini:gemini-3.8-flash
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Make GEN_SW_ISR_TABLE depend on GEN_ISR_TABLES, and guard references to
_sw_isr_table in vects.c so that applications can build without generated
ISR tables when interrupts are unused.
Assisted-by: Gemini:gemini-3.8-flash
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Make GEN_SW_ISR_TABLE depend on GEN_ISR_TABLES, and guard references to
_sw_isr_table in irq_manage.c so that applications can build without
generated ISR tables when interrupts are unused.
Assisted-by: Gemini:gemini-3.8-flash
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Make GEN_SW_ISR_TABLE depend on GEN_ISR_TABLES, and guard references to
_sw_isr_table in irq_manage.c so that applications can build without
generated ISR tables when interrupts are unused.
Assisted-by: Gemini:gemini-3.8-flash
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
Make GEN_SW_ISR_TABLE depend on GEN_ISR_TABLES, and guard references to
_sw_isr_table in irq_manage.c so that applications can build without
generated ISR tables when interrupts are unused.
Assisted-by: Gemini:gemini-3.8-flash
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
On AndeStar V5 cores such as the N25/N25F, the NMI vector base address
register (mnvec, CSR 0x7C3) is read-only and reads back the reset_vector
input signal. An NMI therefore enters the system at __reset, and the
boot path re-initialises the core, destroying the machine state the NMI
was raised to let software examine.
Add CONFIG_RISCV_CUSTOM_CSR_ANDES_NMI, gated on a CPU_HAS_ANDES_NMI
capability that the SoC selects, so the option is only offered on cores
implementing the Andes NMI extension rather than the RISC-V RNMI
extension. It tells the two cases apart at the top of __reset and
branches to _andes_nmi_entry instead of booting. Coming out of reset
mcause and mepc both read 0, whereas an NMI sets mcause to 1 and leaves
the interrupted PC in mepc. The option defaults to n, so behaviour is
unchanged unless a SoC enables it.
Note that Andes reports the NMI with the mcause interrupt bit clear,
rather than set as the privileged specification recommends, so the check
compares mcause against 1 exactly. That value is also the instruction
access fault code. Such exceptions are taken through mtvec, which is set
up early in boot, so in practice they do not reach the reset vector;
mtvec.BASE does reset to 0 however, so a fault raised before that setup
cannot be told apart from an NMI.
_andes_nmi_entry is weak and only parks the core, so that the option
links on its own. A SoC enabling NMIs is expected to provide a real
handler overriding it, and to set up whatever that handler needs, such
as an NMI stack in mscratch, before enabling any NMI source.
Signed-off-by: Sefa Celik <sefa.celik@analog.com>
Use the existing custom-current architecture hooks to keep the running
thread pointer in TPIDR_EL1. A register read avoids the call and IRQ
masking needed to prevent migration between a CPU lookup and the
current-thread load.
Keep the per-CPU current pointer for cross-CPU readers. The scheduler
updates the register with preemption disabled, and reset initializes it
before the kernel installs the dummy thread on each CPU. TPIDR_EL0
remains available for TLS and TPIDRRO_EL0 for CPU and exception state.
Enable this for Cortex-A SMP without system power management.
Uniprocessor and PM builds keep the existing lookup; power-state
restoration is not extended to save the register.
A53 quad core microbenchmarks show ~1.2% higher throughput and up
to 4.7% lower latency.
Assisted-by: Codex:GPT-5
Signed-off-by: Aaron Wisner <aaronwisner@gmail.com>
k_thread_abort() pends PendSV with SCB->ICSR |= PENDSVSET, the third
such site after arch_swap() and z_arm_exc_exit(). ICSR's writable bits
are write-one-to-set or write-one-to-clear and writing zero is a no-op,
so the read-modify-write is equivalent to a plain store while also
writing back a stale snapshot of the other write-one bits.
This path is not hot, so the motivation is consistency and dropping
that stale write-back rather than the saved access. The SHCSR update
immediately below stays a read-modify-write: its bits are ordinary
read/write ones that have to be preserved.
Assisted-by: Claude:claude-opus-5
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
arch_swap() and z_arm_exc_exit() pended PendSV with a read-modify-write
of SCB->ICSR. The register's writable bits are all write-one-to-set or
write-one-to-clear and writing zero to them has no effect, so the read
and the OR are pointless: a plain store of PENDSVSET is equivalent and
one memory access shorter on the two hottest exception paths. It also
avoids writing back a stale snapshot of the other write-one bits (for
instance re-pending a SysTick whose pend bit was cleared between the
read and the write, were the sequence ever preempted).
Measured standalone against main (latency_measure in cycles, lower is
better; thread_metric scores, higher is better):
* mps2/an385 (Cortex-M3, QEMU icount):
- latency_measure: mean -1.8% over 47 ops, min/max/median
-5.0/+0.0/-1.1%; k_yield context switch 182 -> 180 cycles.
- thread_metric: cooperative and preemptive +1.1%.
* az3166_iotdevkit (STM32F412, Cortex-M4 @ 96 MHz):
- thread_metric: preemptive +2.0%, cooperative +1.1%.
- latency_measure: ops stay within ±2 cycles of code-placement
noise, min/max/median -2.0/+2.2/+0.0%.
* Flash, az3166 latency_measure image: -8 B at both -Os and -O2.
Assisted-by: Claude:fable-5
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
Clang's default ARM TLS codegen accesses the TLS block through
TPIDRURO (the per-CPU pointer) instead of TPIDRURW, which Zephyr
uses as the TLS base pointer on Cortex-A/R. This produced corrupted
z_tls_current and other thread-local accesses since the wrong base
was used.
Pass -mtp=soft to force calls through Zephyr's own __aeabi_read_tp,
matching the GCC toolchain's target_arm.cmake and Zephyr's TPIDRURW
based TLS mechanism.
Signed-off-by: Chidvilas Yerramsetti <cyerrams@qti.qualcomm.com>
An interrupt taken between setting to_reclaim and z_thread_abort()
marking the thread dead made isr_wrapper reclaim the CSAs of a thread
that was still runnable. Switching back to it trapped with a call stack
underflow (seen in benchmark.posix.threads on qemu_tc3x).
Lock interrupts before setting to_reclaim.
Assisted-by: Claude:opus-5.5
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
The seed CSR of the Zkr extension is only accessible from M-mode after
reset. When the kernel runs in S-mode on top of the in-tree M-mode
runtime, set mseccfg.SSEED before dropping to S-mode, so that the
kernel can use the entropy source. U-mode access stays disabled.
With an external SBI implementation this is up to the firmware, OpenSBI
sets SSEED on harts that implement Zkr.
Assisted-by: Claude:claude-fable-5-1
Signed-off-by: Fin Maaß <f.maass@vogl-electronic.com>
Add RISCV_ISA_EXT_ZKR for the Zkr entropy source extension. As for the
other ISA extensions, it is enabled from the riscv,isa-extensions
devicetree property. It is also enabled by Zk, which includes Zkr.
Add the addresses of the seed and mseccfg CSRs and the mseccfg.SSEED
bit, which grants S-mode access to the seed CSR.
Zkr is not added to -march: it adds no instructions and the CSRs are
addressed by number, while the LLVM toolchain of the Zephyr SDK selects
its multilib from the exact -march string and has none matching zkr.
Assisted-by: Claude:claude-fable-5-1
Signed-off-by: Fin Maaß <f.maass@vogl-electronic.com>
The window-increment mask and the exception cause were kept in
file-scope statics written only by xtensa_backtrace_print(). Two
cores panicking at once clobbered each other's values, and a
direct caller of the public xtensa_backtrace_get_next_frame() ran
with a stale mask and got wrong PCs.
Move both into struct xtensa_backtrace_frame_t and pass them to
the PC fixup, making it a pure function. Output is unchanged.
Assisted-by: Claude:opus-5
Signed-off-by: Sylvio Alves <sylvio.alves@espressif.com>
Save the target architecture as a cmake cache variable, so other
components can adapt their build if they depend on it.
Note this target architecture is not necessarily the host processor
as posix arch based targets can also be crosscompiled.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
Clarify the CONFIG_XIP description to explain its effect on LMA/VMA
alignment for .text/.rodata and RAM usage, and note that it should be
disabled for images copied into RAM before execution.
Signed-off-by: Abderrahmane JARMOUNI <git@jarmouni.me>
Fixes various issues with Kconfigs, including:
- Using the wrong indentation
- Not having [EXPERIMENTAL] in the prompt, or not having it at
the end
- Wrongly stating the CONFIG_EXPERIMENTAL is needed to enable
an experimental Kconfig
Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
arc_core_mpu_remove_mem_partition() disabled the region at
base_region + partition_id, counting upward from the domain-partition
base region, while arc_core_mpu_configure_mem_domain() assigns array
slots counting downward (partitions[i] -> region base - i) and, since
the hole-skip fix, skips zero-sized entries. The two disagree even
without holes, so the wrong region would be disabled if the function
were ever wired up.
Derive the region slot from the partition's array position: count the
valid partitions before partition_id and subtract from the base region,
matching how the configure loop programs them. No-op on holes and
out-of-range ids.
The function has no in-tree callers today; this aligns it with the
configure-side mapping so it is correct if it is ever used.
Fixes: #116321
Signed-off-by: Hongquan Li <hongquan.li@processmission.com>
k_mem_domain_remove_partition() zeroes a partition slot in place and
decrements num_partitions, leaving a hole in partitions[]. The ARC MPU
domain region programming loops bounded their iteration with
num_partitions, so each hole consumed one iteration and any valid
partition behind it was never programmed into the MPU, faulting
user-mode accesses with EV_ProtV.
Scan the whole partitions array (bounded by
CONFIG_MAX_DOMAIN_PARTITIONS, since num_partitions no longer reflects
the number of slots to inspect) and program only the non-zero-sized
entries, decrementing the remaining-partition counter only for valid
ones. This matches the arm32 implementation in arm_core_mpu.c and is
applied in:
- arc_core_mpu_configure_mem_domain() (MPU v2/v3/v6, common header)
- arc_core_mpu_configure_thread() gap-filling path (MPU v4/v8)
- arc_core_mpu_configure_mem_domain() non-gap-filling path (MPU v4/v8)
- arc_core_mpu_remove_mem_domain() (MPU v4/v8)
Fixes: #116321
Signed-off-by: Hongquan Li <hongquan.li@processmission.com>
Add a common hook declaration and Kconfig switch for firmware
arguments provided by architecture startup code.
The hook receives an architecture-provided argument array and count.
This keeps the common interface independent from register naming used
by a specific architecture.
On ARM64, preserve x0-x3 from the primary CPU reset path before
startup code reuses those registers. The saved values are stored only
after the primary CPU path is selected and are passed to the hook after
RAM has been initialized for C code. Secondary CPUs do not update the
saved storage.
Signed-off-by: Vladyslav Goncharuk <vladyslav_goncharuk@epam.com>
Assisted-by: Codex:gpt-5
Add TriCore architecture port supporting TC1.6P and TC1.8P core
variants used in Infineon AURIX TC3x and TC4x SoC families.
Includes context switching using TriCore upper/lower context save
areas (CSA), interrupt management, trap and exception handling,
syscall interface, reset vector and linker script.
Supports cooperative and preemptive threading, nested interrupt
handling with priority-based preemption.
Signed-off-by: Christoph Seitz <christoph.seitz@infineon.com>
Signed-off-by: Parthiban Nallathambi <parthiban@linumiz.com>
A component should not rely on the API it is implementing.
In this case, the arch layer is implementing the arch cache API
(include/zephyr/arch/cache.h), that is used by the public sys cache
API (include/zephyr/cache.h), so it can't call the latter.
Also rework init_dcache_line_size logic since now
CONFIG_DCACHE_LINE_SIZE is always defined even if DCache line size
runtime detection is available.
Signed-off-by: Abderrahmane JARMOUNI <git@jarmouni.me>
Before z_arm64_mm_init() enables the MMU, all accesses are treated as
Device-nGnRnE, which the architecture requires to be naturally aligned.
This is independent of SCTLR_ELx.A: clearing the A bit relaxes alignment
checking for Normal memory only, it does not make unaligned Device
accesses legal.
Clang may lower struct assignments and small memory copies in this
library into 128-bit Advanced SIMD accesses (ldp/stp q, ldur/stur q).
When such an access is not 16-byte aligned and executes before the MMU
is enabled, it raises an alignment fault (data abort, ESR_ELx.DFSC
0b100001). As this happens before console initialisation, the failure
presents as a silent hang.
GCC already avoids this instruction class here via
-moverride=tune=no_ldp_stp_qregs, but there is no clang equivalent, and
clang rejects that argument. That option would also be insufficient on
its own, as it only suppresses ldp/stp pairs and not a single unaligned
128-bit access.
Apply -mstrict-align to this library when building with clang. The flag
is scoped with zephyr_library_compile_options() rather than
zephyr_cc_option() so that code running with the MMU enabled is not
restricted. Assembly is unaffected, so the FPU context save/restore in
fpu.S continues to use the Qn registers.
Signed-off-by: Shubhankar Kulkarni <shukul@qti.qualcomm.com>
Replace hand-written assembly with CMSIS APIs in the implementation
of USE_SWITCH, as in the review comments for #85248. This should
allow better maintainability and compiler compatibility, and ensure
correctness with respect to instruction barriers.
A small change to include order is needed to avoid errors when the
cmsis_core.h header includes zephyr/irq.h before the
arch/arm/irq.h implementation is included.
Assisted-by: Codex:gpt-5.6-sol
Signed-off-by: Laurie Fay <Laurie.Fay@arm.com>
The shared Cortex-A/R SMP support (GIC SGI-based scheduler IPIs,
secondary core bring-up via reset-time voting locks) is sufficient to
run SMP on ARMv8-A AArch32. Enabling it only requires selecting the
capability flags SCHED_IPI_SUPPORTED (under SMP) and
ARCH_HAS_DIRECTED_IPIS. Also drop the stale "UP only" note.
Signed-off-by: Silesh C V <silesh@alifsemi.com>
send_ipi() means to skip CPUs whose cpu_map[] entry is not yet
populated, but it compares the sender's MPIDR against INV_MPID
instead of the target's. GET_MPIDR() never returns INV_MPID, so the
guard never fires and an unmapped entry reaches gic_raise_sgi().
Fix this.
Signed-off-by: Silesh C V <silesh@alifsemi.com>
Remove unused extern declarations of offload_routine from cortex_m
and cortex_a_r exception.h as currently there are no users for these.
The global offload_routine (and the irq_offload code that set it to
the offloaded routine and cleared it back to NULL) was introduced by
commit 75caa2b084 ("arm: exception-assisted kernel panic/oops support")
so that _IsInIsr() could tell whether the active SVC was an irq_offload()
call.
On cortex-m, commit 4f11b6f8cf ("arch: arm: re-implement
z_arch_is_in_isr") switched arch_is_in_isr() to read the IPSR and
dropped the offload_routine != NULL check.
On cortex_a_r, the declaration was introduced by commit c30a71df95
("arch: arm: Add Cortex-R support") but it was never referenced.
arch_is_in_isr() derives interrupt context from arch_curr_cpu()->nested.
Signed-off-by: Silesh C V <silesh@alifsemi.com>
The Cortex-A/R cores previously shared the Cortex-M irq_offload
implementation (arch/arm/core/irq_offload.c), which stores the offloaded
routine and parameter in global state and brackets the triggering svc
with k_sched_lock(). This is unsuitable for Cortex-A/R because:
On SMP, CPUs might need to run their offloaded functions at the same
time (for eg. the smp_abort test) so a single global routine/parameter
pair cannot be used. Also, k_sched_lock() is illegal in interrupt context,
so the shared backend cannot honour CONFIG_IRQ_OFFLOAD_NESTED
(irq_offload() called from an ISR, for eg. test_nested_irq_offload test).
Add a dedicated cortex_a_r/irq_offload.c that fixes the above by:
1. keeping the offloaded routine/parameter per-CPU, indexed by
_current_cpu->id, so concurrent CPUs no longer interfere with each
other.
2. using arch_irq_lock()/arch_irq_unlock() instead of k_sched_lock() to
pin the caller to its CPU while the per-CPU slot is written and the
svc is taken, while remaining legal from interrupt context. On
Cortex-A/R the SVC is not masked by CPSR.I. So it still traps with
interrupts locked.
The offloaded routine/parameter are kept in a per-CPU slot rather than
passed in registers (as arm64 does) to keep this change self-contained
in C and to avoid modifying the shared SVC exception entry used for
context switch, oops and syscalls.
Build the new file for Cortex-A/R, and select ARCH_HAS_IRQ_OFFLOAD_NESTED
for CPU_AARCH32_CORTEX_A and CPU_AARCH32_CORTEX_R.
Signed-off-by: Silesh C V <silesh@alifsemi.com>
arch_cpu_start() relied solely on the reset.S voting lock to bring up
secondary cores. That is sufficient only on platforms that release all
cores at reset. On platforms that hold secondaries powered off until
requested (e.g. the Arm FVP Base model), they must be explicitly
powered on with their reset vector pointing at the Zephyr entry point.
Use the generic pm_cpu_on() API to start each secondary at __start.
This dispatches to whichever CPU power driver is enabled. The call is
guarded by CONFIG_PM_CPU_OPS, leaving platforms that release
all cores at reset unaffected.
Signed-off-by: Silesh C V <silesh@alifsemi.com>
arm_cpu_boot_params is populated by the primary core and consumed by
the secondary being brought up. On platforms that start secondaries
cold, the secondary begins executing with its caches and MMU disabled
and reads the parameters directly from main memory.
The primary used sys_cache_data_invd_range() on the structure, which
invalidates the cache lines without writing them back. The freshly
written parameters can therefore be discarded before reaching memory,
leaving the secondary to read stale values (e.g. a wrong MPID).
Use sys_cache_data_flush_range() to write the lines back instead,
followed by a DSB to guarantee the flush has completed before the
secondary is brought up.
Signed-off-by: Silesh C V <silesh@alifsemi.com>
