For !TRACING, most arch_cpu_idle and arch_cpu_atomic_idle implementation
relies on the fact that there's weak stub implementations in
subsys/tracing/tracing_none.c, this works, but the arch_cpu_idle sits in
hot code path, so we'd better to make it as efficient as possible.
Take the riscv implementation for example,
Before the patch:
80000a66 <arch_cpu_idle>:
80000a66: 1141 addi sp,sp,-16
80000a68: c606 sw ra,12(sp)
80000a6a: 37c5 jal 80000a4a <sys_trace_idle>
80000a6c: 10500073 wfi
80000a70: 3ff1 jal 80000a4c <sys_trace_idle_exit>
80000a72: 47a1 li a5,8
80000a74: 3007a073 csrs mstatus,a5
80000a78: 40b2 lw ra,12(sp)
80000a7a: 0141 addi sp,sp,16
80000a7c: 8082 ret
NOTE: the sys_trace_idle and sys_trace_idle_exit are just stubs when
!TRACING
after the patch:
80000a62 <arch_cpu_idle>:
80000a62: 10500073 wfi
80000a66: 47a1 li a5,8
80000a68: 3007a073 csrs mstatus,a5
80000a6c: 8082 ret
Signed-off-by: Jisheng Zhang <jszhang@kernel.org>
Using only the fence instruction to gate the management of data in cache
is insufficient to prevent unordered access after flushing in some cases.
Gate dcache instructions like icache instructions.
Signed-off-by: Camille BAUD <mail@massdriver.space>
The MAX32 RV32 core does not implement the fence instruction used by the
RISC-V synchronization intrinsic, so don't enable the builtin barriers for
that target.
Signed-off-by: Pete Johanson <pete.johanson@analog.com>
The ifdef guard in isr.S was written without the CONFIG_ prefix,
making the mtval fallback path dead code on all platforms including
QEMU (which previously worked via CONFIG_QEMU_TARGET).
Signed-off-by: William Markezana <william.markezana@gmail.com>
Select ARCH_SUPPORTS_COREDUMP_THREADS (if !SMP) and
ARCH_SUPPORTS_COREDUMP_STACK_PTR for RISC-V, and implement
arch_coredump_stack_ptr_get().
This enables CONFIG_DEBUG_COREDUMP_MEMORY_DUMP_THREADS and
CONFIG_DEBUG_COREDUMP_THREAD_STACK_TOP.
For non-current threads, return thread->callee_saved.sp.
For the faulting current thread in stack-top mode, return the
exception-time SP from z_riscv_get_sp_before_exc() (cached during
arch_coredump_info_dump()) instead of thread->callee_saved.sp,
which reflects switch-time state.
Signed-off-by: Mirai SHINJO <oss@mshinjo.com>
Expand the RISC-V coredump register block to all 33 GDB registers
(x0-x31, pc) in register-number order.
Previously only 18 registers were serialized. Populate zero, sp, gp,
tp, s0, and s1-s11 (when available).
Bump ARCH_HDR_VER from 1 to 3 (RISC-V 32-bit layout) and from 2 to 4
(RISC-V 64-bit layout) for the new wire format.
Keep the RISC-V 32-bit block fixed at 33 fields on the RISC-V RV32E
profile; registers not implemented by RV32E remain zero-filled so
version 3 always has a stable size.
Signed-off-by: Mirai SHINJO <oss@mshinjo.com>
Add CONFIG_PMP_UNLOCK_ROM_FOR_DEBUG option to conditionally disable
the lock bit (L=0) for the ROM region PMP entry. This allows debuggers
running in machine mode to access ROM for setting breakpoints and
reading instructions while preserving userspace protection.
When PMP lock bits are set, they restrict access even in machine mode,
causing "unable to halt hart" errors with hardware debuggers like
OpenOCD. This option provides a surgical fix that only affects the ROM
region - NULL pointer guards and stack guards remain locked to catch
critical bugs during development.
The option integrates with existing PMP_NO_LOCK_GLOBAL configuration
using nested COND_CODE_1 macros and defaults to disabled for production
builds.
Fixes: zephyrproject-rtos/zephyr#82729
Signed-off-by: Alex Lyrakis <alex_gfd@hotmail.com>
The Zephyr kernel's generic `z_fatal_error()` function, which is
invoked by architecture-specific fatal error handlers, is not
guaranteed to be non-returning. For instance, it can return if an
essential thread aborts itself.
The RISC-V port's `z_riscv_fatal_error` function was previously
inconsistently marked as `FUNC_NORETURN`. This commit removes this
attribute to align with the core kernel behavior, allowing the
function to return if `z_fatal_error()` returns.
Specific changes include:
- Removed `FUNC_NORETURN` from `z_riscv_fatal_error` declarations
in `fatal.c` and `kernel_arch_func.h`.
- Removed `CODE_UNREACHABLE` after the call to `z_fatal_error`
within `z_riscv_fatal_error` as it can now return.
- In `isr.S`, changed `tail z_riscv_fatal_error` to
`call z_riscv_fatal_error` in the exception entry, followed by
a jump to `check_reschedule` to handle the return path.
- Added `CODE_UNREACHABLE` at call sites of `z_riscv_fatal_error`
(e.g., in `z_riscv_fault`, `z_check_user_fault`,
`arch_irq_spurious`) where the context ensures the call is
effectively terminal.
Signed-off-by: Firas Sammoura <fsammoura@google.com>
1. it reguires that there are floating point registers,
so the extention f is required. (zfinx uses the int regs instead)
2. RV32E doesn't supports hardware floating-point calling convention.
Signed-off-by: Fin Maaß <f.maass@vogl-electronic.com>
Introduce the missing flag to compile code with Zbkb extension,
which has already been supported by the GCC 12 in current SDK.
Signed-off-by: Andy Lin <andylinpersonal@gmail.com>
Upgrades the thread user_options to 16 bits from an 8-bit value to
provide more space for future values.
Also, as the size of this field has changed, the values for the
existing architecture specific thread options have also shifted
from the upper end of the old 8-bit field, to the upper end of
the new 16-bit field.
Fixes#101034
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
Add infrastructure for SoCs to define additional PMP regions
that need protection beyond the standard ROM region. This uses
iterable sections to collect region definitions at link time.
The PMP_SOC_REGION_DEFINE macro allows SoCs to register memory
regions with specific permissions. These regions become global
PMP entries shared between M-mode and U-mode.
Signed-off-by: Sylvio Alves <sylvio.alves@espressif.com>
soc_per_core_init_hook() is usually called from arch_kernel_init() and
arch_secondary_cpu_init() which are C functions. As such, there is no need
to check for CONFIG_SOC_PER_CORE_INIT_HOOK since platform/hooks.h provides
a no-op function-like macro implementation if the Kconfig option is not
enabled.
Remove the Kconfig option check from all files.
Signed-off-by: Mathieu Choplain <mathieu.choplain-ext@st.com>
The new function 'z_riscv_pmp_change_permissions' provides a mechanism
to modify the Read, Write, and Execute (R/W/X) permissions of an
existing PMP region based on its memory attribute index.
Signed-off-by: Firas Sammoura <fsammoura@google.com>
Adds the Kconfig option PMP_NO_LOCK_GLOBAL to remove the PMP Lock bit
usage. The global entry is an internal detail of the driver
implementation and should not be reflected in the user interface. This
allows the application to dynamically reconfigure the PMP entries
without requiring hard reset. This is essential for firmware that
performs an RO-to-RW jump. By keeping these system entries unlocked,
higher-privileged M-mode code can dynamically reconfigure memory
permissions during the secure handover process, which is not possible if
the entries are permanently locked during early boot.
Signed-off-by: Firas Sammoura <fsammoura@google.com>
Due to slight differences in the way that LLVM and GNU linkers work,
the call to `z_stack_space_get()` is not dead-stripped when linking
with `lld` but it is dead-stripped when linking with GNU `ld`.
The `z_stack_space_get()` function is only available when
`CONFIG_INIT_STACKS` and `CONFIG_THREAD_STACK_INFO` are defined.
The issue is reproducible (although requires building LLVM and
setting up some environment variables) and goes away with the proposed
workaround.
Signed-off-by: Robin Kastberg <robin.kastberg@iar.com>
Signed-off-by: Chris Friedt <cfriedt@tenstorrent.com>
Add support for stacktrace in dummy thread which is used to run
the early system initialization code before the kernel switches
to the main thread.
On RISC-V, the dummy thread will be running temporarily on the
interrupt stack, but currently we do not initialize the stack
info for the dummy thread, hence check the address against the
interrupt stack.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
This commit implements a new unit test suite to validate the
integration of Device Tree memory attributes (`zephyr,memory-attr`)
with the RISC-V Physical Memory Protection (PMP) hardware.
The test suite includes:
1. **`test_pmp_devicetree_memattr_config`**: Verifies that the PMP
Control and Status Registers (CSRs) are programmed correctly based
on the memory regions defined with `zephyr,memory-attr` in the
Device Tree. It iterates through the active PMP entries and
asserts a match against the expected DT-defined regions.
2. **`test_riscv_mprv_mpp_config`**: Checks the initial state of the
Machine Privilege Register Virtualization (MPRV) bit and Machine
Previous Privilege (MPP) field in the `mstatus` CSR to ensure PMP
is configured for correct privilege level switching during boot.
3. **`test_dt_pmp_perm_conversion`**: Validates the
`DT_MEM_RISCV_TO_PMP_PERM` macro to ensure the conversion from
Device Tree memory attribute flags to RISC-V PMP permission bits
(R/W/X) is correct.
Signed-off-by: Firas Sammoura <fsammoura@google.com>
The logic to decode PMP addressing modes (**TOR**, **NA4**, **NAPOT**) into
physical start and end addresses was previously embedded in
`print_pmp_entries()`.
Extract this calculation into a new static helper function,
`pmp_decode_region()`, to significantly improve the readability and
modularity of the PMP debug printing code.
The new helper function is fully self-contained and exposes a defined API
for the PMP address decoding logic. This enables **direct reuse** in
**unit tests** (e.g., using **Ztest**) to verify the core address
calculation accuracy for all PMP modes and boundary conditions, independent
of the main PMP initialization or logging path.
Signed-off-by: Firas Sammoura <fsammoura@google.com>
The Physical Memory Protection (PMP) initialization is updated to support
custom entries defined in the Device Tree (DT) using the `zephyr,memattr`
property, contingent on `CONFIG_MEM_ATTR` being enabled. A new function,
`set_pmp_mem_attr()`, iterates over DT-defined regions and programs PMP
entries in `z_riscv_pmp_init()`, allowing for early, flexible, and
hardware-specific R/W/X protection for critical memory areas. DT-based
entries are also installed in `z_riscv_pmp_kernelmode_prepare()` for
thread-specific configuration. The logic for the temporary PMP "catch-all"
entry is adjusted to account for new DT entries. Furthermore, the PMP
domain resync logic now masks user partition permissions against DT-defined
region permissions, preventing privilege escalation. `CONFIG_RISCV_PMP` is
updated to select `PMP_KERNEL_MODE_DYNAMIC` if `MEM_ATTR`. Finally, the
`pmp_cfg` array in `z_riscv_pmp_init()` is initialized to zero to prevent
writing uninitialized stack data to unused PMP entries.
Signed-off-by: Firas Sammoura <fsammoura@google.com>
When CONFIG_SMP is enabled, per-CPU IRQ stack guards are added. To prevent
unintended TOR (Top of Range) entry sharing, the PMP address entry
preceding each guard region in `pmp_addr` is marked with -1L.
The previously used index to access `pmp_addr` could become stale, as
additional PMP entries may be allocated after its initial calculation
but before the SMP loop for IRQ guards.
Signed-off-by: Firas Sammoura <fsammoura@google.com>
Split global PMP state variables (index and last address) into
mode-specific counterparts to correctly track the end of global PMP
ranges for both M-mode (kernel) and U-mode (userspace).
This ensures correct per-thread PMP initialization when configuring
mode-specific dynamic PMP entries.
Signed-off-by: Firas Sammoura <fsammoura@google.com>
Introduce `CONFIG_PMP_KERNEL_MODE_DYNAMIC` to enable dynamic
configuration and activation of Machine mode PMP entries. This allows
PMP settings to be managed efficiently during transitions between
kernel and thread contexts.
Signed-off-by: Firas Sammoura <fsammoura@google.com>
Rename the `z_riscv_pmp_stackguard_*` functions to
`z_riscv_pmp_kernelmode_*`. This change better reflects that
these functions are used for general kernel mode PMP configuration,
not strictly limited to stack guard purposes.
Call sites in fatal.c, isr.S, and switch.S have been updated accordingly.
Signed-off-by: Firas Sammoura <fsammoura@google.com>
Add option to enable NMI delivery on boot for SMRNMI hardware.
Changes:
- Add CONFIG_RISCV_SMRNMI_ENABLE_NMI_DELIVERY Kconfig option
- Define SMRNMI CSRs in arch/riscv/include/csr.h
- Set NMIE bit during boot to enable NMI delivery
SMRNMI hardware generates but doesn't deliver NMIs when NMIE=0 (default).
This causes twister test failures and prevents handling of critical
hardware events like watchdog NMIs and ECC errors.
Setting NMIE=1 enables NMI delivery, but note that this implementation
only sets the enable bit - it does not provide full SMRNMI support
(no mnret instruction handling, no RNMI handlers). Users must implement
proper RNMI handlers in SoC-specific code to avoid undefined behavior.
Signed-off-by: Afonso Oliveira <afonsoo@synopsys.com>
Add support for the RISC-V Smcsrind extension, which provides
indirect access to CSRs through the MISELECT and MIREG registers.
Changes:
- Added CONFIG_RISCV_ISA_EXT_SMCSRIND Kconfig option
- Implemented 4 helper functions for indirect CSR access:
* icsr_read/write - basic access
* icsr_read_set/clear - bit manipulation
- Defined 7 CSR registers (MISELECT, MIREG, MIREG2-6)
This is a CSR-only extension that does not require any compiler
support or march flags. The helper functions compile to standard
CSR instructions and work with any toolchain that supports Zicsr.
Primary use case: RISC-V AIA (Advanced Interrupt Architecture)
uses indirect CSRs to access IMSIC (Incoming MSI Controller)
registers.
Signed-off-by: Afonso Oliveira <afonsoo@synopsys.com>
Adds a new test suite to verify the behavior of `riscv_pmp_clear_all()`.
These tests ensure that the function correctly clears all unlocked PMP
entries while preserving any entries that are locked.
Signed-off-by: Firas Sammoura <fsammoura@google.com>
Introduce the new function `riscv_pmp_clear_all()` to reset the Physical
Memory Protection (PMP) configuration.
This function iterates through all configured PMP slots. For each entry,
it writes 0x0 to the entry's 8-bit configuration register. This action
attempts to clear all fields, including the Address Matching Mode (A) bits
(setting the region type to OFF), the permission bits (R, W, X), and
the Lock (L) bit.
According to the RISC-V specification, any writes to the configuration
or address registers of a locked PMP entry are ignored. Thus, locked
entries will remain unchanged, while all non-locked entries will be
effectively disabled and their permissions cleared.
The function ensures it operates in Machine mode with MSTATUS.MPRV = 0
and MSTATUS.MPP = M-mode before modifying any PMP Control and Status
Registers (CSRs).
This provides a mechanism to clear all non-locked PMP regions,
returning them to a default disabled state. The function declaration is
exposed in the `include/zephyr/arch/riscv/pmp.h` header file, making it
available for inclusion and use by external modules.
It is recommended for firmware to call this function before transitioning
from a Read-Only (RO) stage to a Read-Write (RW) stage. This ensures
that any PMP settings established during the RO phase, which might no
longer be appropriate, are cleared, providing a clean and secure base
PMP configuration for the RW firmware.
Signed-off-by: Firas Sammoura <fsammoura@google.com>
For RISCV vector table needs to be aligned depending on
CONFIG_ARCH_IRQ_VECTOR_TABLE_ALIGN. This was missing
when using LTO making issues when direct ISR were in use.
Signed-off-by: Łukasz Stępnicki <lukasz.stepnicki@nordicsemi.no>
use CONFIG_RISCV_ISA_EXT_F to set CONFIG_CPU_HAS_FPU.
Same for CONFIG_RISCV_ISA_EXT_D and
CONFIG_CPU_HAS_FPU_DOUBLE_PRECISION.
Signed-off-by: Fin Maaß <f.maass@vogl-electronic.com>
`CONFIG_EXTRA_EXCEPTION_INFO` that was added in #78065 doesn't
seem necessary, as we were already storing and printing the
callee-saved-registers before that. All `CONFIG_EXTRA_EXCEPTION_INFO`
does in RISCV is to add an additional `_callee_saved_t *csf` in the
`struct arch_esf`, which overhead is negligible to what's being enabled
by `CONFIG_EXCEPTION_DEBUG`.
Let's remove `CONFIG_EXTRA_EXCEPTION_INFO`, and have that extra
`_callee_saved_t *csf` in the `struct arch_esf` as long as
`CONFIG_EXCEPTION_DEBUG` is enabled.
TL;DR: it doesn't make sense to not enable `CONFIG_EXTRA_EXCEPTION_INFO`
when `CONFIG_EXCEPTION_DEBUG` is enabled, so let's merge them.
Then, since `*csf` is always available in the `struct arch_esf` when
`CONFIG_EXCEPTION_DEBUG=y`, we can simply rely on that pointer in
`z_riscv_fatal_error()` instead of an additional argument in
`z_riscv_fatal_error_csf()`, rendering the latter redundant and thus
can be removed.
Additionally, save the callee-saved registers before jumping to
to `z_riscv_fault()`, so that callee-saved-registers are printed on
generic CPU exception as well.
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Fixes local ISR generation so that devices actually boot, also
allows enabling LTO for these builds (tested working on nrf54l15
flpr device)
Signed-off-by: Jamie McCrae <spam@helper3000.net>
Let's move the null pointer trap first so to give set_pmp_entry() the
opportunity to use a single-slot TOR entry.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Refactor the code to read all PMP address CSRs (pmpaddr0 through
pmpaddrN) into a new helper function, `z_riscv_pmp_read_addr`.
This change encapsulates the register reading loop, improving code
organization and potential reusability. The new function includes size
assertions
Signed-off-by: Firas Sammoura <fsammoura@google.com>
Replace `sizeof(unsigned long)` with `PMPCFG_STRIDE` (defined as
`__riscv_xlen / 8`) for dimensioning PMP configuration register arrays
(`pmpcfg_regs`).
The size of PMP configuration registers should be derived directly from
the target architecture's XLEN. Using `sizeof(unsigned long)` can cause
size mismatches, particularly with static analysis tools like SonarQube.
These tools might assume a host-specific size for `unsigned long`,
leading to spurious out-of-bounds access warnings when analyzing code
for different RISCV base architectures (e.g., RV32 vs. RV64).
This change ensures the array sizing is correctly and consistently tied
to the target's register width (XLEN).
Signed-off-by: Firas Sammoura <fsammoura@google.com>
soc_prep_hook() is always called from z_prep_c() which is implemented
as a C function. As such, there is no need to check for the associated
CONFIG_SOC_PREP_HOOK since the platform/hooks.h header will define hooks
as no-op function-like macros if their associated Kconfig isn't enabled.
Remove the Kconfig check from all arch implementations of z_prep_c() and
call soc_prep_hook() directly instead, to avoid duplicating the Kconfig
check already performed in platform/hooks.h
Signed-off-by: Mathieu Choplain <mathieu.choplain-ext@st.com>
Introduce `z_riscv_pmp_write_config` to abstract writing to pmpcfg
registers. This function handles the differing register layouts and slot
counts between RV32 and RV64 architectures, writing to the appropriate
pmpcfg0, pmpcfg1, pmpcfg2, or pmpcfg3 CSRs as needed based on
CONFIG_PMP_SLOTS.
Signed-off-by: Firas Sammoura <fsammoura@google.com>
Move Xuantie supprot from arch/riscv/core/xuantie to the custom common
layer arch/riscv/custom/thead, with the following changes:
1. Rename Kconfig name
CACHE_XTHEADCMO -> RISCV_CUSTOM_CSR_THEAD_CMO
2. Split the original arch/riscv/core/xuantie/Kconfig to
a. arch/riscv/custom/thead/Kconfig: for T-Head extension
b. arch/riscv/custom/thead/Kconfig.core: for T-Head CPU series
(e.g. Xuantie E907)
3. Move cache line size defaults to SoC devicetree
Signed-off-by: Jimmy Zheng <jimmyzhe@andestech.com>
Update OpenISA RI5CY and Zero-RISCY CSR handling to use RISC-V custom
CSR common code. Move these stuff to 'arch/riscv/custom/openisa':
1. Rename 'soc_ri5cy.h' to 'ri5cy_csr.h' for CSR definitions.
2. Rename 'soc_zero_riscy.h' to 'zero_riscy_csr.h' for CSR definitions.
3. Move CSR context to common macro '__custom_csr_save/restore_context'.
4. Move compiler option '-march=rv32imcxpulpv2' to common code.
Signed-off-by: Jimmy Zheng <jimmyzhe@andestech.com>
CVA6 supports custom CSR. Move 'cva6.h' to 'arch/riscv/custom/cva6_csr.h',
allowing other SoCs using the CVA6 core to reuse the same CSR definitions.
Signed-off-by: Jimmy Zheng <jimmyzhe@andestech.com>
GD32VF103 uses Nuclei-specific CSR. Move 'nuclei_csr.h' to
'arch/riscv/custom' to allow reuse across SoCs with the same Nuclei core.
Signed-off-by: Jimmy Zheng <jimmyzhe@andestech.com>
