riscv_cpu_wake_flag and riscv_cpu_sp are variables, not functions, so
they should use the GDATA macro instead of the GTEXT macro. Otherwise,
the linker will warn when LTO is enabled:
ld.bfd: warning: type of symbol `riscv_cpu_wake_flag' changed from 2 to
1 in /tmp/cc39w5oK.ltrans0.ltrans.o
ld.bfd: warning: type of symbol `riscv_cpu_sp' changed from 2 to 1 in
/tmp/cc39w5oK.ltrans0.ltrans.o
Signed-off-by: Tom Hughes <tomhughes@chromium.org>
Initializing the C++ stack unwinding data structures takes quite a bit
of stack space. Increase the TEST_EXTRA_STACK_SIZE when using these.
Signed-off-by: Keith Packard <keithp@keithp.com>
The QingKe V2C has an integer multiplier but no divide. Add support
for the corresponding Zmmul extension and, as the extension was added
in GCC 13.0, add a test for the compiler version.
Signed-off-by: Michael Hope <michaelh@juju.nz>
The declaration for the text region '__text_region_start' and
'__text_region_end' should be consistent with the declaration in the
include file 'zephyr/linker/linker-defs.h'.
The local declarations are:
extern uintptr_t __text_region_start, __text_region_end;
whereas 'linker_defs.h' declares them as:
extern char __text_region_start[];
extern char __text_region_end[];
This may result in conflicting types when 'linker_defs.h' is indirectly
included. Hence, remove the local declarations.
Signed-off-by: Christoph Winklhofer <cj.winklhofer@gmail.com>
Add sys_trace_idle_exit to RISCV cpu_idle functions and allow
enabling CPU_LOAD module for RISCV and disable it for SMP.
Signed-off-by: Krzysztof Chruściński <krzysztof.chruscinski@nordicsemi.no>
Since 3466dab804 the generic llext_symbol_name() function abstracts
the use of llext_string() for (non-section) symbols. Define a similar
llext_section_name() function and replace current occurrences of
llext_string() with the proper abstraction.
By extending llext_symbol_name(), this commit also allows to print the
correct name of sections that are referred to by a symbol.
Signed-off-by: Luca Burelli <l.burelli@arduino.cc>
Currently, RISC-V's architecture-specific relocations assume that
all relocations of type R_RISCV_PCREL_LO12_I and _S are processed
immediately after the R_RISCV_PCREL_HI20 relocation that they
share a relocation target with. While this is the case most of
the time, the RISC-V PSABI specification does not guarantee that.
This commit corrects this by determining the R_RISCV_PCREL_HI20
relocation based on the symbol value of the R_RISCV_PCREL_LO12
relocation, as specified in the PSABI.
Signed-off-by: Eric Ackermann <eric.ackermann@cispa.de>
The RISC-V port of llext requires additional parameters for
handling non-adjacent HI20/LO12 relocations in arch_elf_relocate():
the current extension (struct llext), the current extension loader
(struct llext_loader), the current section header (elf_shdr_t) and
the current symbol (elf_sym_t).
This changes the signature of arch_elf_relocate accordingly.
Signed-off-by: Eric Ackermann <eric.ackermann@cispa.de>
The Zaamo and Zalrsc Extension (Version v1.0.0, 2024-04-25; Ratified) split
the standard A extension into two subextensions. As of date, the `_zaamo`
and `_zlrsc` extension specifications are accepted by the upstream
in-development GCC through the `march` argument. This means that those
subextensions are not yet supported by GCC shipped with Zephyr SDK.
Signed-off-by: Filip Kokosinski <fkokosinski@antmicro.com>
CLIC supports mintstatus.MIL (RO) and mcause.MPIL (RW) for the current
interrupt level and the previous interrut level before a trap. Each ISR
must execute MRET to set mcause.MPIL back to mintstatus.MIL.
This commit introduces CONFIG_CLIC_SUPPORT_INTERRUPT_LEVEL to handle
mcause.MPIL for interrupt preemption in nested ISR, and uses
CONFIG_RISCV_ALWAYS_SWITCH_THROUGH_ECALL to ensure ISR always switch out
with MRET.
e.g.
With CONFIG_RISCV_ALWAYS_SWITCH_THROUGH_ECALL=n, a context-switch in
ISR may skip MRET in this flow:
IRQ -> _isr_wrapper -> z_riscv_switch() -> retrun to arch_switch()
Signed-off-by: Jimmy Zheng <jimmyzhe@andestech.com>
The exception fault handler function is named _Fault which is inconsistent
with all other arch-specific functions present in Zephyr. This patch
replaces the name with z_riscv_fault.
Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
When RISCV_SOC_HAS_ISR_STACKING is used, it may
be needed to initialize custom hw stacked esf members.
Some initial values may need to be aligned with
hw stacking mechanism to avoid any side effects.
Signed-off-by: Łukasz Stępnicki <lukasz.stepnicki@nordicsemi.no>
The RISC-V architecture-specific relocations need to check whether
each required relocation can fit into the modified instruction's
immediate. All immediates in RISC-V are encoded as two's complement.
The current truncation check has an off-by-one error for checking
the maximum negative distance, as two's complement encoding can
represent a negative value that is the maximum positive value plus
one, causing LLEXT to refuse loading valid code.
This commit adds an additional condition to the check that fixes
the aforementioned issue.
Signed-off-by: Eric Ackermann <eric.ackermann@cispa.de>
Mostly a revert of commit b1def7145f ("arch: deprecate `_current`").
This commit was part of PR #80716 whose initial purpose was about providing
an architecture specific optimization for _current. The actual deprecation
was sneaked in later on without proper discussion.
The Zephyr core always used _current before and that was fine. It is quite
prevalent as well and the alternative is proving rather verbose.
Furthermore, as a concept, the "current thread" is not something that is
necessarily architecture specific. Therefore the primary abstraction
should not carry the arch_ prefix.
Hence this revert.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Introduce `CONFIG_RISCV_GP_PURPOSE` choice to make sure that only
one of `CONFIG_RISCV_GP` or `CONFIG_RISCV_CURRENT_VIA_GP` can be
enabled, instead of relying of dependencies.
To do that, introduce a new
`CONFIG_RISCV_SOC_HAS_GP_RELATIVE_ADDRESSING` that can be selected
by SoC when it implemented global pointer (GP) initialization for
relative addressing in its linker.
`CONFIG_RISCV_GP` will be the default choice when
`CONFIG_RISCV_SOC_HAS_GP_RELATIVE_ADDRESSING=y`
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
Reset the the `gp` register to `_kernel->cpus[i].current` when
`CONFIG_USERSPACE` is enabled on exception to keep it sane.
Updated the testcase to test both `CONFIG_RISCV_GP` and
`CONFIG_RISCV_CURRENT_VIA_GP`.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
In most implements, the msip base address is 0x2000000. But the address
is not fixed in all boards.
Signed-off-by: Yang Jialong <yangjialong@vcore.com>
`_current` is now functionally equals to `arch_curr_thread()`, remove
its usage in-tree and deprecate it instead of removing it outright,
as it has been with us since forever.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
Implement `arch_curr_thread()` & `arch_set_curr_thread()`
with the global pointer (GP) register.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
Previously PMP was avaible only with Multithreading, since it's now
available without MT - add extra checks to prevent user/kernel mode
reconfiguration.
Signed-off-by: Volodymyr Fialko <vfialko@marvell.com>
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Without multithreading only two stacks present: ISR and main.
As any stack they also could overflow, so it make sense to add stack
guard for them also.
Remove stack guard dependency on multithreading and mark
`Z_RISCV_STACK_GUARD_SIZE` bytes at the beginning of stack as read-only
region with PMP entry.
Signed-off-by: Volodymyr Fialko <vfialko@marvell.com>
According to the RISC-V Instruction Set Manual: Volume II, Version 20240411
(Section 3.1.6.6), some implementations may choose to track the dirtiness
of the floating-point register state imprecisely by reporting the state to
be dirty even when it has not been modified. This option reflects that.
Also add a filter in `tests/arch/riscv/fpu_sharing/` based on imprecise
FPU state tracking
Signed-off-by: Jakub Wasilewski <jwasilewski@internships.antmicro.com>
Signed-off-by: Filip Kokosinski <fkokosinski@antmicro.com>
Allow SoC to implement their custom per-core initialization function by
selecting `CONFIG_SOC_PER_CORE_INIT_HOOK` and implement
`soc_per_core_init_hook()`.
Signed-off-by: Maxim Adelman <imax@meta.com>
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
The currently IPI implementation assumes that CLINT exists in the
system, however, that might not be the case as IPI can be implemented
with PLIC that supports software-triggering as well, such as the Andes
NCEPLIC100.
Refactor the CLINT-based IPI implementations into `ipi_clint.c`, and
create Kconfig that selects the CLINT implementation when
`sifive-clint0` exists and enabled, otherwise default to
`RISCV_SMP_IPI_CUSTOM` which allows OOT implementation. This also
makes way for the upstreaming of non-clint IPI implementation later.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Reset the gp on exception entry from u-mode to protect the kernel
against a possible rogue user thread.
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
The `CONFIG_PLIC_IRQ_AFFINITY` depends on `CONFIG_SMP` in the
Kconfig layer, nested #ifdef can be avoided for readability.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
This commit introduces architecture-specific ELF relocations for RISC-V,
in accordance with the RISC-V PSABI specification:
https://github.com/riscv-non-isa/riscv-elf-psabi-doc/blob/master/riscv-elf.adoc
Also, the necessary compiler configurations for compiling LLEXT
extensions on RISC-V are added, and the llext tests are executed on
RISC-V targets.
Calling llext extensions from user threads in RISC-V is still
unsupported as of this commit.
Signed-off-by: Eric Ackermann <eric.ackermann@cispa.de>
- Implement irq-set-affinity in RISCV PLIC.
- Added new affinity shell command to get/set the irq(s)
affinity in runtime, when `0` is sent as the `local_irq`, it
means set/get all IRQs affinity.
- Some minor optimizations
Updated the build_all test to build this new configuration.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
Restore the s0 we saved early in ISR entry so it shows up
properly in the CSF.
Signed-off-by: David Reiss <dreiss@meta.com>
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Up until now, the `__thread` keyword has been used for declaring
variables as Thread local storage. However, `__thread` is a GNU
specific keyword which thus limits compatibility with other
toolchains (for instance IAR).
This PR intoduces a new macro `Z_THREAD_LOCAL` which expands to the
corresponding C11, C23 or C++11 standard keyword based on the standard
that is specified during compilation, else it uses the old `__thread`
keyword.
Signed-off-by: Daniel Flodin <daniel.flodin@iar.com>
Do not use SYS_INIT for initializing irq_offload when enabled, instead
using a new interface that is called during the boot process for all
architectures.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Add a new call for initializing cache on architectures that need that.
Avoid using SYS_INIT for this and instead call the hook in a fixed place
and run if implemented.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Fill the memory of all CPU's IRQ stack with 0xAA on init, so
that `z_stack_space_get` can calculate the remaining space
correctly.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
The callee-saved-registers can be helpful to debug the state of
a core upon an exception, however, currently there's no way to
access that information in user-implemented
`k_sys_fatal_error_handler()`, even though the csf is already stored
in the stack.
This patch conditionally add a `csf` member in the `arch_esf` when
`CONFIG_EXTRA_EXCEPTION_INFO=y`*, which the `_isr_wrapper` would update
when a fatal error occurs before invoking `z_riscv_fatal_error_csf()`.
Functions such as `k_sys_fatal_error_handler()` would then be able
to access the callee-saved-registers at the time of exception via
`esf->csf`.
* For SoCs that select `RISCV_SOC_HAS_ISR_STACKING`, the
`SOC_ISR_STACKING_ESF_DECLARE` has to include the `csf` member,
otherwise the build would fail.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
Get rid of the switch statement and use an string array
for the cause instead. This saves about ~600 bytes.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
Rename `cause_str` to `z_riscv_mcause_str` and make it non-static,
so that it can be used in user-implemented `k_sys_fatal_error_handler`.
Also, this function should return a constant string, so add `const`
to it.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
Relocate the logging of mcause & mtval from `_Fault` to
`z_riscv_fatal_error_csf` so that they are always printed
upon exception.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
Introduce soc and board hooks to replace arch specific code
and replace usages of SYS_INIT for platform initialization.
include/zephyr/platform/hooks.h introduces the hooks to be implemented
by boards and SoCs.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Add the possibility to disable fault handling in spurious
interrupt handler on RISCs and replacce it with an infinite loop.
Signed-off-by: Magdalena Pastula <magdalena.pastula@nordicsemi.no>
The change of alignment check in #76045 could be wrong and
isn't unnecessary to fix the stack traces output, undo it for
now.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
This implements arch_thread_priv_stack_space_get() so this can
be used to figure out how much privileged stack space is used.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This adds the bits to initialize the privileged stack when
a thread is transitioning to user mode. This prevents
information leaking if the stack is reused, and also aids
in calculating stack space usage during system calls.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Having `CONFIG_EXCEPTION_STACK_TRACE_SYMTAB` to select the
`CONFIG_SYMTAB` or to explicitly not print the symbol name
during exception stack unwind seems unnecessary, as the extra
code to print the symbol name is negligible when compared with
the symbol table, so just remove it.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
This commit introduces a new ARCH_STACKWALK Kconfig which
determines if the `arch_stack_walk()` is available should the
arch supports it.
Starting from RISCV, this will be able to converge the exception
stack trace implementation & stack walking features. Existing
exception stack trace implementation will be updated later.
Eventually we will end up with the following:
1. If an arch implements `arch_stack_walk()`
`ARCH_HAS_STACKWALK` should be selected.
2. If the above is enabled, `ARCH_SUPPORTS_STACKWALK` indicates
if the dependencies are met for arch to enable stack walking.
This Kconfig replaces `<arch>_EXCEPTION_STACK_TRACE`
2. If the above is enabled, then, `ARCH_STACKWALK` determines
if `arch_stack_walk()` should be compiled.
3. `EXCEPTION_STACK_TRACE` should build on top of the
`ARCH_STACKWALK`, stack traces will be printed when it
is enabled.
4. `ARCH_STACKWALK_MAX_FRAMES` will be removed as it is
replaced by `ARCH_STACKWALK_MAX_FRAMES`
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
Relocate stack unwind backends from `arch/` to perf's
`backends/` folder, just like logging/shell/..
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
Implement stack trace function for riscv arch, that get required
thread register values and unwind stack with it.
Signed-off-by: Mikhail Kushnerov <m.kushnerov@yadro.com>
