Initializing the C++ stack unwinding data structures takes quite a bit
of stack space. Increase the TEST_EXTRA_STACK_SIZE when using these.
Signed-off-by: Keith Packard <keithp@keithp.com>
CLIC supports mintstatus.MIL (RO) and mcause.MPIL (RW) for the current
interrupt level and the previous interrut level before a trap. Each ISR
must execute MRET to set mcause.MPIL back to mintstatus.MIL.
This commit introduces CONFIG_CLIC_SUPPORT_INTERRUPT_LEVEL to handle
mcause.MPIL for interrupt preemption in nested ISR, and uses
CONFIG_RISCV_ALWAYS_SWITCH_THROUGH_ECALL to ensure ISR always switch out
with MRET.
e.g.
With CONFIG_RISCV_ALWAYS_SWITCH_THROUGH_ECALL=n, a context-switch in
ISR may skip MRET in this flow:
IRQ -> _isr_wrapper -> z_riscv_switch() -> retrun to arch_switch()
Signed-off-by: Jimmy Zheng <jimmyzhe@andestech.com>
When RISCV_SOC_HAS_ISR_STACKING is used, it may
be needed to initialize custom hw stacked esf members.
Some initial values may need to be aligned with
hw stacking mechanism to avoid any side effects.
Signed-off-by: Łukasz Stępnicki <lukasz.stepnicki@nordicsemi.no>
Mostly a revert of commit b1def7145f ("arch: deprecate `_current`").
This commit was part of PR #80716 whose initial purpose was about providing
an architecture specific optimization for _current. The actual deprecation
was sneaked in later on without proper discussion.
The Zephyr core always used _current before and that was fine. It is quite
prevalent as well and the alternative is proving rather verbose.
Furthermore, as a concept, the "current thread" is not something that is
necessarily architecture specific. Therefore the primary abstraction
should not carry the arch_ prefix.
Hence this revert.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Introduce `CONFIG_RISCV_GP_PURPOSE` choice to make sure that only
one of `CONFIG_RISCV_GP` or `CONFIG_RISCV_CURRENT_VIA_GP` can be
enabled, instead of relying of dependencies.
To do that, introduce a new
`CONFIG_RISCV_SOC_HAS_GP_RELATIVE_ADDRESSING` that can be selected
by SoC when it implemented global pointer (GP) initialization for
relative addressing in its linker.
`CONFIG_RISCV_GP` will be the default choice when
`CONFIG_RISCV_SOC_HAS_GP_RELATIVE_ADDRESSING=y`
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
Reset the the `gp` register to `_kernel->cpus[i].current` when
`CONFIG_USERSPACE` is enabled on exception to keep it sane.
Updated the testcase to test both `CONFIG_RISCV_GP` and
`CONFIG_RISCV_CURRENT_VIA_GP`.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
`_current` is now functionally equals to `arch_curr_thread()`, remove
its usage in-tree and deprecate it instead of removing it outright,
as it has been with us since forever.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
Implement `arch_curr_thread()` & `arch_set_curr_thread()`
with the global pointer (GP) register.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
Without multithreading only two stacks present: ISR and main.
As any stack they also could overflow, so it make sense to add stack
guard for them also.
Remove stack guard dependency on multithreading and mark
`Z_RISCV_STACK_GUARD_SIZE` bytes at the beginning of stack as read-only
region with PMP entry.
Signed-off-by: Volodymyr Fialko <vfialko@marvell.com>
According to the RISC-V Instruction Set Manual: Volume II, Version 20240411
(Section 3.1.6.6), some implementations may choose to track the dirtiness
of the floating-point register state imprecisely by reporting the state to
be dirty even when it has not been modified. This option reflects that.
Also add a filter in `tests/arch/riscv/fpu_sharing/` based on imprecise
FPU state tracking
Signed-off-by: Jakub Wasilewski <jwasilewski@internships.antmicro.com>
Signed-off-by: Filip Kokosinski <fkokosinski@antmicro.com>
The currently IPI implementation assumes that CLINT exists in the
system, however, that might not be the case as IPI can be implemented
with PLIC that supports software-triggering as well, such as the Andes
NCEPLIC100.
Refactor the CLINT-based IPI implementations into `ipi_clint.c`, and
create Kconfig that selects the CLINT implementation when
`sifive-clint0` exists and enabled, otherwise default to
`RISCV_SMP_IPI_CUSTOM` which allows OOT implementation. This also
makes way for the upstreaming of non-clint IPI implementation later.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
The callee-saved-registers can be helpful to debug the state of
a core upon an exception, however, currently there's no way to
access that information in user-implemented
`k_sys_fatal_error_handler()`, even though the csf is already stored
in the stack.
This patch conditionally add a `csf` member in the `arch_esf` when
`CONFIG_EXTRA_EXCEPTION_INFO=y`*, which the `_isr_wrapper` would update
when a fatal error occurs before invoking `z_riscv_fatal_error_csf()`.
Functions such as `k_sys_fatal_error_handler()` would then be able
to access the callee-saved-registers at the time of exception via
`esf->csf`.
* For SoCs that select `RISCV_SOC_HAS_ISR_STACKING`, the
`SOC_ISR_STACKING_ESF_DECLARE` has to include the `csf` member,
otherwise the build would fail.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
This commit introduces a new ARCH_STACKWALK Kconfig which
determines if the `arch_stack_walk()` is available should the
arch supports it.
Starting from RISCV, this will be able to converge the exception
stack trace implementation & stack walking features. Existing
exception stack trace implementation will be updated later.
Eventually we will end up with the following:
1. If an arch implements `arch_stack_walk()`
`ARCH_HAS_STACKWALK` should be selected.
2. If the above is enabled, `ARCH_SUPPORTS_STACKWALK` indicates
if the dependencies are met for arch to enable stack walking.
This Kconfig replaces `<arch>_EXCEPTION_STACK_TRACE`
2. If the above is enabled, then, `ARCH_STACKWALK` determines
if `arch_stack_walk()` should be compiled.
3. `EXCEPTION_STACK_TRACE` should build on top of the
`ARCH_STACKWALK`, stack traces will be printed when it
is enabled.
4. `ARCH_STACKWALK_MAX_FRAMES` will be removed as it is
replaced by `ARCH_STACKWALK_MAX_FRAMES`
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
Before this, stack protection would be effective only after switching to
the first thread.
Even before the first thread is created, the kernel init code uses the
IRQ stack to set things up. Let's make sure this is safeguarded as well.
This also fixes the incompatibility between CONFIG_RISCV_PMP and
CONFIG_RISCV_ALWAYS_SWITCH_THROUGH_ECALL, the later needing an exception
call to switch to the first thread and exception code assuming stack
guard is already set up in the PMP.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Update the description of the `INCLUDE_RESET_VECTOR` Kconfig so
that it is more clear to the user what it does.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
The CLIC requires that mtvec.base is aligned to 64 bytes.
_isr_wrapper is used as mtvec.base, so align it to 64 bytes.
Signed-off-by: Piotr Wojnarowski <pwojnarowski@antmicro.com>
Fix the dependencies of `CONFIG_EXCEPTION_STACK_TRACE`:
- Architecture-specific Kconfig, i.e.
`X86_EXCEPTION_STACK_TRACE`, will be enabled automatically
when all the dependencies are met.
- `EXCEPTION_STACK_TRACE` depends on architecture-specific
Kconfig to be enabled.
- The stack trace implementations should be compiled only if
user enables `CONFIG_EXCEPTION_STACK_TRACE`.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
RISC-V PMP implementation supports isolating thread stacks
within the same memory domain, and also is the only
supported operating mode. So select the corresponding
kconfig by default.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Influenced heavily by the RISCV64 stack unwinding
implementation in the Linux kernel.
`CONFIG_RISCV_EXCEPTION_STACK_TRACE` can be enabled by
configuring the following Kconfigs:
```prj.conf
CONFIG_DEBUG_INFO=y
CONFIG_EXCEPTION_STACK_TRACE=y
CONFIG_OVERRIDE_FRAME_POINTER_DEFAULT=y
CONFIG_OMIT_FRAME_POINTER=n
```
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
This commit removes the `Kconfig.core` file. It's been largely unused, and
the only symbol it provides (`RISCV_CORE_E31`) overlaps with the SoC-layer
provided `SOC_SERIES_SIFIVE_FREEDOM_FE300`.
As of date, the only SoC that uses the E31 core in Zephyr is the FE310 SoC.
Signed-off-by: Filip Kokosinski <fkokosinski@antmicro.com>
Configure a new Kconfig (`CONFIG_RISCV_HART_MASK`) so that it
is possible to mask the `mhartid` of a processor. This is
helpful in the cases when the hart id starts from non-zero
value.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Because the riscv32/riscv64 is redundant, one can get the same
information combining CONFIG_ARCH + CONFIG_64BIT.
Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
It looks like all SoCs in tree check if an exception comes from an IRQ
the same way, so let's provide a common logic by default, still
customizable if the SoC selects RISCV_SOC_ISR_CHECK.
Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
According to the clic specification
(https://github.com/riscv/riscv-fast-interrupt), the mnxti register has
be written, in order to clear the pending bit for non-vectored
interrupts. For vectored interrupts, this is automatically done.
From the spec:
"If the pending interrupt is edge-triggered, hardware will automatically
clear the corresponding pending bit when the CSR instruction that
accesses xnxti includes a write."
I added a kconfig `RISCV_SOC_HAS_CUSTOM_IRQ_HANDLING` to allow custom
irq handling. If enabled, `__soc_handle_all_irqs` has to be implemented.
For clic, non-vectored mode, I added a `__soc_handle_all_irqs`, that
handles the pending interrupts according to the pseudo code in the spec.
Signed-off-by: Greter Raffael <rgreter@baumer.com>
Because it was exclusively used by the "common" RISC-V privileged code
to build CPU idle routines that are now handled by arch level code.
Also, all platforms defaulted to "y", making it pointless in practice.
Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
Introduce a new arch level Kconfig option to signal the implementation
of the RISCV Privileged ISA spec. This replaces
SOC_FAMILY_RISCV_PRIVILEGED, because this is not a SoC specific
property, nor a SoC family.
Note that the SoC family naming scheme will be fixed in upcoming
commits.
Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
RISC-V Spec requires minimum alignment of trap handling code to be
dependent from MTVEC.BASE field size. Minimum alignment for RISC-V
platforms is 4 bytes, but maximum is platform or application-specific.
Currently there is no common approach to align the trap handling
code for RISC-V and some platforms use custom wrappers to align
_isr_wrapper properly.
This change introduces a generic solution,
CONFIG_RISCV_TRAP_HANDLER_ALIGNMENT configuration option which sets
the alignment of a RISC-V trap handling code.
The existing custom solutions for some platforms remain operational,
since the default alignment is set to minimal possible (4 bytes)
and will be overloaded by potentially larger alignment of custom solutions.
Signed-off-by: Alexander Razinkov <alexander.razinkov@syntacore.com>
Add Kconfig RISCV_SOC_HAS_CUSTOM_SYS_IO symbol so that a riscv
SoC can set to specify that it has a custom implementation for
sys_io functions.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Some RISCV platforms shipping a CLIC have a peculiar interrupt ID
ordering / mapping.
According to the "Core-Local Interrupt Controller (CLIC) RISC-V
Privileged Architecture Extensions" Version 0.9-draft at paragraph 16.1
one of these ordering recommendations is "CLIC-mode interrupt-map for
systems retaining interrupt ID compatible with CLINT mode" that is
described how:
The CLINT-mode interrupts retain their interrupt ID in CLIC mode.
[...]
The existing CLINT software interrupt bits are primarily intended for
inter-hart interrupt signaling, and so are retained for that purpose.
[...]
CLIC interrupt inputs are allocated IDs beginning at interrupt ID
17. Any fast local interrupts that would have been connected at
interrupt ID 16 and above should now be mapped into corresponding
inputs of the CLIC.
That is a very convoluted way to say that interrupts 0 to 15 are
reserved for internal use and CLIC only controls interrupts reserved for
platform use (16 up to n + 16, where n is the maximum number of
interrupts supported).
Let's now take now into consideration this situation in the DT:
clic: interrupt-controller {
...
};
device0: some-device {
interrupt-parent = <&clic>;
interrupts = <0x1>;
...
};
and in the driver for device0:
IRQ_CONNECT(DT_IRQN(node), ...);
From the hardware prospective:
(1a) device0 is using the first IRQ line of the CLIC
(2a) the interrupt ID / exception code of the `MSTATUS` register
associated to this IRQ is 17, because the IDs 0 to 15 are reserved
From the software / Zephyr prospective:
(1b) Zephyr is installing the IRQ vector into the SW ISR table (and into
the IRQ vector table for DIRECT ISRs in case of CLIC vectored mode)
at index 0x1.
(2b) Zephyr is using the interrupt ID of the `MSTATUS` register to index
into the SW ISR table (or IRQ vector table)
It's now clear how (2a) and (2b) are in contrast with each other.
To fix this problem we have to take into account the offset introduced
by the reserved interrupts. To do so we introduce
CONFIG_RISCV_RESERVED_IRQ_ISR_TABLES_OFFSET as hidden option for the
platforms to set.
This Kconfig option is used to shift the interrupt numbers when
installing the IRQ vector into the SW ISR table and/or IRQ vector table.
So for example in the previous case and using
CONFIG_RISCV_RESERVED_IRQ_ISR_TABLES_OFFSET == 16, the IRQ vector
associated to the device0 would be correctly installed at index 17 (16 +
1), matching what is reported by the `MSTATUS` register.
CONFIG_NUM_IRQS must be increased accordingly.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Before adding support for the CLIC vectored mode, rename
CONFIG_RISCV_MTVEC_VECTORED_MODE to CONFIG_RISCV_VECTORED_MODE to be
more generic and eventually include also the CLIC vectored mode.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
This will avoid unconditionally pulling z_riscv_switch() into the build
as it is not used, reducing the resulting binary some more.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Enable single-threading support for the riscv architecture.
Add z_riscv_switch_to_main_no_multithreading function for
supporting single-threading.
The single-threading does not work with enabling PMP_STACK_GUARD.
It is because single-threading does not use context-switching.
But the privileged mode transition that PMP depends on implicitly
presupposes using context-switching. It is a contradiction.
Thus, disable PMP_STACK_GUARD when MULTITHREADING is not enabled.
Signed-off-by: TOKITA Hiroshi <tokita.hiroshi@fujitsu.com>
Looks like some implementors decided not to implement the full set of
PMP range matching modes. Let's rearrange the code so that any of those
modes can be disabled.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Let's honor CONFIG_MPU_REQUIRES_POWER_OF_TWO_ALIGNMENT even for kernel
stacks. This saves one global PMP slot when creating the guard area for
the IRQ stack, and some hw implementations might require that anyway.
With this changes, arch_mem_domain_max_partitions_get() becomes much
more reliable and tests/kernel/mem_protect is more likely to pass even
with the stack guard enabled.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
RISC-V multi-hart systems that employ a heterogeneous core complex are
not guaranteed to have the smp capable harts starting with a unique id
of zero, matching Zephyr's sequential zero indexed cpu numbering scheme.
Add option, RV_BOOT_HART to choose the hart to boot from.
On reset, check the current hart equals RV_BOOT_HART: if so, boot first
core. else, loop in the boot secondary core and wait to be brought up.
For multi-hart systems that are not running a Zephyr mp or smp
application, park the non zephyr related harts in a wfi loop.
Signed-off-by: Conor Paxton <conor.paxton@microchip.com>
RISC-V has a modular design. Some hardware with a custom interrupt
controller needs a bit more work to lock / unlock IRQs.
Account for this hardware by introducing a set of new
z_soc_irq_* functions that can override the default behaviour.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Some RISC-V SoCs implement a mechanism for hardware supported stacking /
unstacking of registers during ISR / exceptions. What happens is that on
ISR / exception entry part of the context is automatically saved by the
hardware on the stack without software intervention, and the same part
of the context is restored by the hardware usually on mret.
This is currently not yet supported by Zephyr, where the full context
must be saved by software in the full fledged ESF. This patcheset is
trying to address exactly this case.
At least three things are needed to support in a general fashion this
problem: (1) a way to store in software only the part of the ESF not
already stacked by hardware, (2) a way to restore in software only the
part of the context that is not going to be restored by hardware and (3)
a way to define a custom ESF.
Point (3) is important because the full ESF frame is now composed by a
custom part depending on the hardware (that can choose which register to
stack / unstack and the order they are saved onto the stack) and a part
defined in software for the remaining part of the context.
In this patch a new CONFIG_RISCV_SOC_HAS_ISR_STACKING is introduced that
enables the code path supporting the three points by the mean of three
macros that must be implemented by the user in a soc_stacking.h file:
SOC_ISR_SW_STACKING, SOC_ISR_SW_UNSTACKING and SOC_ISR_STACKING_ESF
(refer to the symbol help for more details).
This is an example of soc_isr_stacking.h for an hardware that doesn't do
any hardware stacking / unstacking but everything is managed in
software:
#ifndef __SOC_ISR_STACKING
#define __SOC_ISR_STACKING
#if !defined(_ASMLANGUAGE)
#define SOC_ISR_STACKING_ESF_DECLARE \
struct __esf { \
unsigned long ra; \
unsigned long t0; \
unsigned long t1; \
unsigned long t2; \
unsigned long t3; \
unsigned long t4; \
unsigned long t5; \
unsigned long t6; \
unsigned long a0; \
unsigned long a1; \
unsigned long a2; \
unsigned long a3; \
unsigned long a4; \
unsigned long a5; \
unsigned long a6; \
unsigned long a7; \
unsigned long mepc; \
unsigned long mstatus; \
unsigned long s0; \
} __aligned(16)
#else
#define SOC_ISR_SW_STACKING \
addi sp, sp, -__z_arch_esf_t_SIZEOF; \
DO_CALLER_SAVED(sr);
#define SOC_ISR_SW_UNSTACKING \
DO_CALLER_SAVED(lr);
#endif /* _ASMLANGUAGE */
#endif /* __SOC_ISR_STACKING */
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
This reverts commit a7b5d606c7.
The assumption behind that commit was wrong. The software-based stack
sentinel writes to the very bottom of the _writable_ stack area i.e.
right next to the actual PMP based guard area. So they are compatible.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The software-based stack sentinel writes to the very bottom of the
stack area triggering the PMP stack protection. Obviously they can't
be used together.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
For vectored interrupts use the generated IRQ vector table instead of
relying on a custom-generated table.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Some early RISC-V SoCs have a problem when an `mret` instruction is used
outside a trap handler.
After the latest Zephyr RISC-V huge rework, the arch_switch code is
indeed calling `mret` when not in handler mode, breaking some early
RISC-V platforms.
Optionally restore the old behavior by adding a new
CONFIG_RISCV_ALWAYS_SWITCH_THROUGH_ECALL symbol.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
This reverts the bulk of commit c8bfc2afda ("riscv: make
arch_is_user_context() SMP compatible") and replaces it with a flag
stored in the thread local storage (TLS) area, therefore making TLS
mandatory for userspace support on RISC-V.
This has many advantages:
- The tp (x4) register is already dedicated by the standard for this
purpose, making TLS support almost free.
- This is very efficient, requiring only a single instruction to clear
and 2 instructions to set.
- This makes the SMP case much more efficient. No need for funky
exception code any longer.
- SMP and non-SMP now use the same implementation making maintenance
easier.
- The is_user_mode variable no longer requires a dedicated PMP mapping
and therefore freeing one PMP slot for other purposes.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
5f65dbcc9dab3d39473b05397e05.
