michaelh/zephyr
1
0
Fork 0
Code Releases Activity
zephyr/arch/riscv/core/isr.S

538 lines
14 KiB
ArmAsm
Raw Normal View History

arch: added support for the riscv32 architecture RISC-V is an open-source instruction set architecture. Added support for the 32bit version of RISC-V to Zephyr. 1) exceptions/interrupts/faults are handled at the architecture level via the __irq_wrapper handler. Context saving/restoring of registers can be handled at both architecture and SOC levels. If SOC-specific registers need to be saved, SOC level needs to provide __soc_save_context and __soc_restore_context functions that shall be accounted by the architecture level, when corresponding config variable RISCV_SOC_CONTEXT_SAVE is set. 2) As RISC-V architecture does not provide a clear ISA specification about interrupt handling, each RISC-V SOC handles it in its own way. Hence, at the architecture level, the __irq_wrapper handler expects the following functions to be provided by the SOC level: __soc_is_irq: to check if the exception is the result of an interrupt or not. __soc_handle_irq: handle pending IRQ at SOC level (ex: clear pending IRQ in SOC-specific IRQ register) 3) Thread/task scheduling, as well as IRQ offloading are handled via the RISC-V system call ("ecall"), which is also handled via the __irq_wrapper handler. The _Swap asm function just calls "ecall" to generate an exception. 4) As there is no conventional way of handling CPU power save in RISC-V, the default nano_cpu_idle and nano_cpu_atomic_idle functions just unlock interrupts and return to the caller, without issuing any CPU power saving instruction. Nonetheless, to allow SOC-level to implement proper CPU power save, nano_cpu_idle and nano_cpu_atomic_idle functions are defined as __weak at the architecture level. Change-Id: I980a161d0009f3f404ad22b226a6229fbb492389 Signed-off-by: Jean-Paul Etienne <fractalclone@gmail.com>
2017-01-11 00:24:30 +01:00
/*
* Copyright (c) 2016 Jean-Paul Etienne <fractalclone@gmail.com>
arch: riscv32: provide a general mechanism for saving SoC context RISC-V permits myriad extensions to the ISA, any of which may imply additional context that must be saved and restored on ISR entry and exit. The current in-tree example is the Pulpino core, which has extra registers used by ISA extensions for running loops that shouldn't get clobbered by an ISR. This is currently supported by including pulpino-specific definitions in the generic architecture code. This works, but it's a bit inelegant and is something of a layering violation. A more generic mechanism is required to support other RISC-V SoCs with similar requirements without cluttering the arch code too much. Provide that by extending the semantics of the existing CONFIG_RISCV_SOC_CONTEXT_SAVE option to allow other SoCs to allocate space for saving and restoring their own state, promoting the currently pulpino-specific __soc_save_context / __soc_restore_context routines to a RISC-V arch API. The cost of making this generic is two more instructions in each ISR to pass the SoC specific context to these routines in a0 rather than just assuming the stack points to the right place. This is minimal, and should have been done anyway to keep with the ABI. As a first (and currently only in-tree) customer, convert the Pulpino SoC code to this new mechanism. Signed-off-by: Marti Bolivar <marti@foundries.io>
2018-11-01 22:25:13 -06:00
* Copyright (c) 2018 Foundries.io Ltd
arch: riscv: add memory protection support The IRQ handler has had a major changes to manage syscall, reschedule and interrupt from user thread and stack guard. Add userspace support: - Use a global variable to know if the current execution is user or machine. The location of this variable is read only for all user thread and read/write for kernel thread. - Memory shared is supported. - Use dynamic allocation to optimize PMP slot usage. If the area size is a power of 2, only one PMP slot is used, else 2 are used. Add stack guard support: - Use MPRV bit to force PMP rules to machine mode execution. - IRQ stack have a locked stack guard to avoid re-write PMP configuration registers for each interruption and then win some cycle. - The IRQ stack is used as "temporary" stack at the beginning of IRQ handler to save current ESF. That avoid to trigger write fault on thread stack during store ESF which that call IRQ handler to infinity. - A stack guard is also setup for privileged stack of a user thread. Thread: - A PMP setup is specific to each thread. PMP setup are saved in each thread structure to improve reschedule performance. Signed-off-by: Alexandre Mergnat <amergnat@baylibre.com> Reviewed-by: Nicolas Royer <nroyer@baylibre.com>
2020-07-21 16:00:39 +02:00
* Copyright (c) 2020 BayLibre, SAS
arch: added support for the riscv32 architecture RISC-V is an open-source instruction set architecture. Added support for the 32bit version of RISC-V to Zephyr. 1) exceptions/interrupts/faults are handled at the architecture level via the __irq_wrapper handler. Context saving/restoring of registers can be handled at both architecture and SOC levels. If SOC-specific registers need to be saved, SOC level needs to provide __soc_save_context and __soc_restore_context functions that shall be accounted by the architecture level, when corresponding config variable RISCV_SOC_CONTEXT_SAVE is set. 2) As RISC-V architecture does not provide a clear ISA specification about interrupt handling, each RISC-V SOC handles it in its own way. Hence, at the architecture level, the __irq_wrapper handler expects the following functions to be provided by the SOC level: __soc_is_irq: to check if the exception is the result of an interrupt or not. __soc_handle_irq: handle pending IRQ at SOC level (ex: clear pending IRQ in SOC-specific IRQ register) 3) Thread/task scheduling, as well as IRQ offloading are handled via the RISC-V system call ("ecall"), which is also handled via the __irq_wrapper handler. The _Swap asm function just calls "ecall" to generate an exception. 4) As there is no conventional way of handling CPU power save in RISC-V, the default nano_cpu_idle and nano_cpu_atomic_idle functions just unlock interrupts and return to the caller, without issuing any CPU power saving instruction. Nonetheless, to allow SOC-level to implement proper CPU power save, nano_cpu_idle and nano_cpu_atomic_idle functions are defined as __weak at the architecture level. Change-Id: I980a161d0009f3f404ad22b226a6229fbb492389 Signed-off-by: Jean-Paul Etienne <fractalclone@gmail.com>
2017-01-11 00:24:30 +01:00
*
license: Replace Apache boilerplate with SPDX tag Replace the existing Apache 2.0 boilerplate header with an SPDX tag throughout the zephyr code tree. This patch was generated via a script run over the master branch. Also updated doc/porting/application.rst that had a dependency on line numbers in a literal include. Manually updated subsys/logging/sys_log.c that had a malformed header in the original file. Also cleanup several cases that already had a SPDX tag and we either got a duplicate or missed updating. Jira: ZEP-1457 Change-Id: I6131a1d4ee0e58f5b938300c2d2fc77d2e69572c Signed-off-by: David B. Kinder <david.b.kinder@intel.com> Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2017-01-18 17:01:01 -08:00
* SPDX-License-Identifier: Apache-2.0
arch: added support for the riscv32 architecture RISC-V is an open-source instruction set architecture. Added support for the 32bit version of RISC-V to Zephyr. 1) exceptions/interrupts/faults are handled at the architecture level via the __irq_wrapper handler. Context saving/restoring of registers can be handled at both architecture and SOC levels. If SOC-specific registers need to be saved, SOC level needs to provide __soc_save_context and __soc_restore_context functions that shall be accounted by the architecture level, when corresponding config variable RISCV_SOC_CONTEXT_SAVE is set. 2) As RISC-V architecture does not provide a clear ISA specification about interrupt handling, each RISC-V SOC handles it in its own way. Hence, at the architecture level, the __irq_wrapper handler expects the following functions to be provided by the SOC level: __soc_is_irq: to check if the exception is the result of an interrupt or not. __soc_handle_irq: handle pending IRQ at SOC level (ex: clear pending IRQ in SOC-specific IRQ register) 3) Thread/task scheduling, as well as IRQ offloading are handled via the RISC-V system call ("ecall"), which is also handled via the __irq_wrapper handler. The _Swap asm function just calls "ecall" to generate an exception. 4) As there is no conventional way of handling CPU power save in RISC-V, the default nano_cpu_idle and nano_cpu_atomic_idle functions just unlock interrupts and return to the caller, without issuing any CPU power saving instruction. Nonetheless, to allow SOC-level to implement proper CPU power save, nano_cpu_idle and nano_cpu_atomic_idle functions are defined as __weak at the architecture level. Change-Id: I980a161d0009f3f404ad22b226a6229fbb492389 Signed-off-by: Jean-Paul Etienne <fractalclone@gmail.com>
2017-01-11 00:24:30 +01:00
*/
#include <toolchain.h>
linker: move all linker headers to include/linker Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2017-06-17 11:30:47 -04:00
#include <linker/sections.h>
arch: added support for the riscv32 architecture RISC-V is an open-source instruction set architecture. Added support for the 32bit version of RISC-V to Zephyr. 1) exceptions/interrupts/faults are handled at the architecture level via the __irq_wrapper handler. Context saving/restoring of registers can be handled at both architecture and SOC levels. If SOC-specific registers need to be saved, SOC level needs to provide __soc_save_context and __soc_restore_context functions that shall be accounted by the architecture level, when corresponding config variable RISCV_SOC_CONTEXT_SAVE is set. 2) As RISC-V architecture does not provide a clear ISA specification about interrupt handling, each RISC-V SOC handles it in its own way. Hence, at the architecture level, the __irq_wrapper handler expects the following functions to be provided by the SOC level: __soc_is_irq: to check if the exception is the result of an interrupt or not. __soc_handle_irq: handle pending IRQ at SOC level (ex: clear pending IRQ in SOC-specific IRQ register) 3) Thread/task scheduling, as well as IRQ offloading are handled via the RISC-V system call ("ecall"), which is also handled via the __irq_wrapper handler. The _Swap asm function just calls "ecall" to generate an exception. 4) As there is no conventional way of handling CPU power save in RISC-V, the default nano_cpu_idle and nano_cpu_atomic_idle functions just unlock interrupts and return to the caller, without issuing any CPU power saving instruction. Nonetheless, to allow SOC-level to implement proper CPU power save, nano_cpu_idle and nano_cpu_atomic_idle functions are defined as __weak at the architecture level. Change-Id: I980a161d0009f3f404ad22b226a6229fbb492389 Signed-off-by: Jean-Paul Etienne <fractalclone@gmail.com>
2017-01-11 00:24:30 +01:00
#include <offsets_short.h>
headers: Refactor kernel and arch headers. This commit refactors kernel and arch headers to establish a boundary between private and public interface headers. The refactoring strategy used in this commit is detailed in the issue This commit introduces the following major changes: 1. Establish a clear boundary between private and public headers by removing "kernel/include" and "arch/*/include" from the global include paths. Ideally, only kernel/ and arch/*/ source files should reference the headers in these directories. If these headers must be used by a component, these include paths shall be manually added to the CMakeLists.txt file of the component. This is intended to discourage applications from including private kernel and arch headers either knowingly and unknowingly. - kernel/include/ (PRIVATE) This directory contains the private headers that provide private kernel definitions which should not be visible outside the kernel and arch source code. All public kernel definitions must be added to an appropriate header located under include/. - arch/*/include/ (PRIVATE) This directory contains the private headers that provide private architecture-specific definitions which should not be visible outside the arch and kernel source code. All public architecture- specific definitions must be added to an appropriate header located under include/arch/*/. - include/ AND include/sys/ (PUBLIC) This directory contains the public headers that provide public kernel definitions which can be referenced by both kernel and application code. - include/arch/*/ (PUBLIC) This directory contains the public headers that provide public architecture-specific definitions which can be referenced by both kernel and application code. 2. Split arch_interface.h into "kernel-to-arch interface" and "public arch interface" divisions. - kernel/include/kernel_arch_interface.h * provides private "kernel-to-arch interface" definition. * includes arch/*/include/kernel_arch_func.h to ensure that the interface function implementations are always available. * includes sys/arch_interface.h so that public arch interface definitions are automatically included when including this file. - arch/*/include/kernel_arch_func.h * provides architecture-specific "kernel-to-arch interface" implementation. * only the functions that will be used in kernel and arch source files are defined here. - include/sys/arch_interface.h * provides "public arch interface" definition. * includes include/arch/arch_inlines.h to ensure that the architecture-specific public inline interface function implementations are always available. - include/arch/arch_inlines.h * includes architecture-specific arch_inlines.h in include/arch/*/arch_inline.h. - include/arch/*/arch_inline.h * provides architecture-specific "public arch interface" inline function implementation. * supersedes include/sys/arch_inline.h. 3. Refactor kernel and the existing architecture implementations. - Remove circular dependency of kernel and arch headers. The following general rules should be observed: * Never include any private headers from public headers * Never include kernel_internal.h in kernel_arch_data.h * Always include kernel_arch_data.h from kernel_arch_func.h * Never include kernel.h from kernel_struct.h either directly or indirectly. Only add the kernel structures that must be referenced from public arch headers in this file. - Relocate syscall_handler.h to include/ so it can be used in the public code. This is necessary because many user-mode public codes reference the functions defined in this header. - Relocate kernel_arch_thread.h to include/arch/*/thread.h. This is necessary to provide architecture-specific thread definition for 'struct k_thread' in kernel.h. - Remove any private header dependencies from public headers using the following methods: * If dependency is not required, simply omit * If dependency is required, - Relocate a portion of the required dependencies from the private header to an appropriate public header OR - Relocate the required private header to make it public. This commit supersedes #20047, addresses #19666, and fixes #3056. Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
2019-10-25 00:08:21 +09:00
#include <arch/cpu.h>
riscv: Add support for floating point This change adds full shared floating point support for the RISCV architecture with minimal impact on threads with floating point support not enabled. Signed-off-by: Corey Wharton <coreyw7@fb.com>
2020-03-11 18:15:29 -07:00
#include <sys/util.h>
#include <kernel.h>
arch: riscv: add memory protection support The IRQ handler has had a major changes to manage syscall, reschedule and interrupt from user thread and stack guard. Add userspace support: - Use a global variable to know if the current execution is user or machine. The location of this variable is read only for all user thread and read/write for kernel thread. - Memory shared is supported. - Use dynamic allocation to optimize PMP slot usage. If the area size is a power of 2, only one PMP slot is used, else 2 are used. Add stack guard support: - Use MPRV bit to force PMP rules to machine mode execution. - IRQ stack have a locked stack guard to avoid re-write PMP configuration registers for each interruption and then win some cycle. - The IRQ stack is used as "temporary" stack at the beginning of IRQ handler to save current ESF. That avoid to trigger write fault on thread stack during store ESF which that call IRQ handler to infinity. - A stack guard is also setup for privileged stack of a user thread. Thread: - A PMP setup is specific to each thread. PMP setup are saved in each thread structure to improve reschedule performance. Signed-off-by: Alexandre Mergnat <amergnat@baylibre.com> Reviewed-by: Nicolas Royer <nroyer@baylibre.com>
2020-07-21 16:00:39 +02:00
#include <syscall.h>
#include <arch/riscv/csr.h>
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
#include <arch/riscv/syscall.h>
riscv: better abstraction for register-wide load/store opcodes Those are prominent enough that having RV_OP_LOADREG and RV_OP_STOREREG shouting at you all over the place is rather unpleasant and bad taste. Let's create pseudo-instructions of our own with assembler macros rather than preprocessor defines and only in assembly scope. This makes the asm code way more uniform and readable. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-22 14:09:55 -05:00
#include "asm_macros.inc"
riscv: Add support for floating point This change adds full shared floating point support for the RISCV architecture with minimal impact on threads with floating point support not enabled. Signed-off-by: Corey Wharton <coreyw7@fb.com>
2020-03-11 18:15:29 -07:00
/* Convenience macros for loading/storing register states. */
#define DO_FP_CALLER_SAVED(op, reg) \
op ft0, __z_arch_esf_t_ft0_OFFSET(reg) ;\
op ft1, __z_arch_esf_t_ft1_OFFSET(reg) ;\
op ft2, __z_arch_esf_t_ft2_OFFSET(reg) ;\
op ft3, __z_arch_esf_t_ft3_OFFSET(reg) ;\
op ft4, __z_arch_esf_t_ft4_OFFSET(reg) ;\
op ft5, __z_arch_esf_t_ft5_OFFSET(reg) ;\
op ft6, __z_arch_esf_t_ft6_OFFSET(reg) ;\
op ft7, __z_arch_esf_t_ft7_OFFSET(reg) ;\
op ft8, __z_arch_esf_t_ft8_OFFSET(reg) ;\
op ft9, __z_arch_esf_t_ft9_OFFSET(reg) ;\
op ft10, __z_arch_esf_t_ft10_OFFSET(reg) ;\
op ft11, __z_arch_esf_t_ft11_OFFSET(reg) ;\
op fa0, __z_arch_esf_t_fa0_OFFSET(reg) ;\
op fa1, __z_arch_esf_t_fa1_OFFSET(reg) ;\
op fa2, __z_arch_esf_t_fa2_OFFSET(reg) ;\
op fa3, __z_arch_esf_t_fa3_OFFSET(reg) ;\
op fa4, __z_arch_esf_t_fa4_OFFSET(reg) ;\
op fa5, __z_arch_esf_t_fa5_OFFSET(reg) ;\
op fa6, __z_arch_esf_t_fa6_OFFSET(reg) ;\
op fa7, __z_arch_esf_t_fa7_OFFSET(reg) ;
arch: riscv: add memory protection support The IRQ handler has had a major changes to manage syscall, reschedule and interrupt from user thread and stack guard. Add userspace support: - Use a global variable to know if the current execution is user or machine. The location of this variable is read only for all user thread and read/write for kernel thread. - Memory shared is supported. - Use dynamic allocation to optimize PMP slot usage. If the area size is a power of 2, only one PMP slot is used, else 2 are used. Add stack guard support: - Use MPRV bit to force PMP rules to machine mode execution. - IRQ stack have a locked stack guard to avoid re-write PMP configuration registers for each interruption and then win some cycle. - The IRQ stack is used as "temporary" stack at the beginning of IRQ handler to save current ESF. That avoid to trigger write fault on thread stack during store ESF which that call IRQ handler to infinity. - A stack guard is also setup for privileged stack of a user thread. Thread: - A PMP setup is specific to each thread. PMP setup are saved in each thread structure to improve reschedule performance. Signed-off-by: Alexandre Mergnat <amergnat@baylibre.com> Reviewed-by: Nicolas Royer <nroyer@baylibre.com>
2020-07-21 16:00:39 +02:00
#define DO_CALLER_SAVED(op) \
op ra, __z_arch_esf_t_ra_OFFSET(sp) ;\
op t0, __z_arch_esf_t_t0_OFFSET(sp) ;\
op t1, __z_arch_esf_t_t1_OFFSET(sp) ;\
op t2, __z_arch_esf_t_t2_OFFSET(sp) ;\
op t3, __z_arch_esf_t_t3_OFFSET(sp) ;\
op t4, __z_arch_esf_t_t4_OFFSET(sp) ;\
op t5, __z_arch_esf_t_t5_OFFSET(sp) ;\
op t6, __z_arch_esf_t_t6_OFFSET(sp) ;\
op a0, __z_arch_esf_t_a0_OFFSET(sp) ;\
op a1, __z_arch_esf_t_a1_OFFSET(sp) ;\
op a2, __z_arch_esf_t_a2_OFFSET(sp) ;\
op a3, __z_arch_esf_t_a3_OFFSET(sp) ;\
op a4, __z_arch_esf_t_a4_OFFSET(sp) ;\
op a5, __z_arch_esf_t_a5_OFFSET(sp) ;\
op a6, __z_arch_esf_t_a6_OFFSET(sp) ;\
op a7, __z_arch_esf_t_a7_OFFSET(sp) ;
riscv: isr.S: compute _current_cpu using CPU number on SMP To do so efficiently on systems without the mul instruction, we use shifts and adds which is faster and sometimes smaller than a plain loop. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-09 23:47:38 -05:00
#ifdef CONFIG_SMP
#define GET_CURRENT_CPU(dst, tmp) \
csrr tmp, mhartid ;\
la dst, _kernel + ___kernel_t_cpus_OFFSET ;\
shiftmul_add dst, tmp, ___cpu_t_SIZEOF
#else
#define GET_CURRENT_CPU(dst, tmp) \
la dst, _kernel + ___kernel_t_cpus_OFFSET
#endif
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
#define ASSUME_EQUAL(x, y) .if x != y; .err; .endif
arch: riscv: add memory protection support The IRQ handler has had a major changes to manage syscall, reschedule and interrupt from user thread and stack guard. Add userspace support: - Use a global variable to know if the current execution is user or machine. The location of this variable is read only for all user thread and read/write for kernel thread. - Memory shared is supported. - Use dynamic allocation to optimize PMP slot usage. If the area size is a power of 2, only one PMP slot is used, else 2 are used. Add stack guard support: - Use MPRV bit to force PMP rules to machine mode execution. - IRQ stack have a locked stack guard to avoid re-write PMP configuration registers for each interruption and then win some cycle. - The IRQ stack is used as "temporary" stack at the beginning of IRQ handler to save current ESF. That avoid to trigger write fault on thread stack during store ESF which that call IRQ handler to infinity. - A stack guard is also setup for privileged stack of a user thread. Thread: - A PMP setup is specific to each thread. PMP setup are saved in each thread structure to improve reschedule performance. Signed-off-by: Alexandre Mergnat <amergnat@baylibre.com> Reviewed-by: Nicolas Royer <nroyer@baylibre.com>
2020-07-21 16:00:39 +02:00
arch: added support for the riscv32 architecture RISC-V is an open-source instruction set architecture. Added support for the 32bit version of RISC-V to Zephyr. 1) exceptions/interrupts/faults are handled at the architecture level via the __irq_wrapper handler. Context saving/restoring of registers can be handled at both architecture and SOC levels. If SOC-specific registers need to be saved, SOC level needs to provide __soc_save_context and __soc_restore_context functions that shall be accounted by the architecture level, when corresponding config variable RISCV_SOC_CONTEXT_SAVE is set. 2) As RISC-V architecture does not provide a clear ISA specification about interrupt handling, each RISC-V SOC handles it in its own way. Hence, at the architecture level, the __irq_wrapper handler expects the following functions to be provided by the SOC level: __soc_is_irq: to check if the exception is the result of an interrupt or not. __soc_handle_irq: handle pending IRQ at SOC level (ex: clear pending IRQ in SOC-specific IRQ register) 3) Thread/task scheduling, as well as IRQ offloading are handled via the RISC-V system call ("ecall"), which is also handled via the __irq_wrapper handler. The _Swap asm function just calls "ecall" to generate an exception. 4) As there is no conventional way of handling CPU power save in RISC-V, the default nano_cpu_idle and nano_cpu_atomic_idle functions just unlock interrupts and return to the caller, without issuing any CPU power saving instruction. Nonetheless, to allow SOC-level to implement proper CPU power save, nano_cpu_idle and nano_cpu_atomic_idle functions are defined as __weak at the architecture level. Change-Id: I980a161d0009f3f404ad22b226a6229fbb492389 Signed-off-by: Jean-Paul Etienne <fractalclone@gmail.com>
2017-01-11 00:24:30 +01:00
/* imports */
GDATA(_sw_isr_table)
GTEXT(__soc_is_irq)
GTEXT(__soc_handle_irq)
GTEXT(_Fault)
arch: riscv32: provide a general mechanism for saving SoC context RISC-V permits myriad extensions to the ISA, any of which may imply additional context that must be saved and restored on ISR entry and exit. The current in-tree example is the Pulpino core, which has extra registers used by ISA extensions for running loops that shouldn't get clobbered by an ISR. This is currently supported by including pulpino-specific definitions in the generic architecture code. This works, but it's a bit inelegant and is something of a layering violation. A more generic mechanism is required to support other RISC-V SoCs with similar requirements without cluttering the arch code too much. Provide that by extending the semantics of the existing CONFIG_RISCV_SOC_CONTEXT_SAVE option to allow other SoCs to allocate space for saving and restoring their own state, promoting the currently pulpino-specific __soc_save_context / __soc_restore_context routines to a RISC-V arch API. The cost of making this generic is two more instructions in each ISR to pass the SoC specific context to these routines in a0 rather than just assuming the stack points to the right place. This is minimal, and should have been done anyway to keep with the ABI. As a first (and currently only in-tree) customer, convert the Pulpino SoC code to this new mechanism. Signed-off-by: Marti Bolivar <marti@foundries.io>
2018-11-01 22:25:13 -06:00
#ifdef CONFIG_RISCV_SOC_CONTEXT_SAVE
GTEXT(__soc_save_context)
GTEXT(__soc_restore_context)
#endif /* CONFIG_RISCV_SOC_CONTEXT_SAVE */
arch: added support for the riscv32 architecture RISC-V is an open-source instruction set architecture. Added support for the 32bit version of RISC-V to Zephyr. 1) exceptions/interrupts/faults are handled at the architecture level via the __irq_wrapper handler. Context saving/restoring of registers can be handled at both architecture and SOC levels. If SOC-specific registers need to be saved, SOC level needs to provide __soc_save_context and __soc_restore_context functions that shall be accounted by the architecture level, when corresponding config variable RISCV_SOC_CONTEXT_SAVE is set. 2) As RISC-V architecture does not provide a clear ISA specification about interrupt handling, each RISC-V SOC handles it in its own way. Hence, at the architecture level, the __irq_wrapper handler expects the following functions to be provided by the SOC level: __soc_is_irq: to check if the exception is the result of an interrupt or not. __soc_handle_irq: handle pending IRQ at SOC level (ex: clear pending IRQ in SOC-specific IRQ register) 3) Thread/task scheduling, as well as IRQ offloading are handled via the RISC-V system call ("ecall"), which is also handled via the __irq_wrapper handler. The _Swap asm function just calls "ecall" to generate an exception. 4) As there is no conventional way of handling CPU power save in RISC-V, the default nano_cpu_idle and nano_cpu_atomic_idle functions just unlock interrupts and return to the caller, without issuing any CPU power saving instruction. Nonetheless, to allow SOC-level to implement proper CPU power save, nano_cpu_idle and nano_cpu_atomic_idle functions are defined as __weak at the architecture level. Change-Id: I980a161d0009f3f404ad22b226a6229fbb492389 Signed-off-by: Jean-Paul Etienne <fractalclone@gmail.com>
2017-01-11 00:24:30 +01:00
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
GTEXT(z_riscv_fatal_error)
riscv: implement arch_switch() The move to arch_switch() is a prerequisite for SMP support. Make it optimal without the need for an ECALL roundtrip on every context switch. Performance numbers from tests/benchmarks/sched: Before: unpend 107 ready 102 switch 188 pend 218 tot 615 (avg 615) After: unpend 107 ready 102 switch 170 pend 217 tot 596 (avg 595) Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-07 17:01:36 -05:00
GTEXT(z_get_next_switch_handle)
GTEXT(z_riscv_switch)
GTEXT(z_riscv_thread_start)
arch: added support for the riscv32 architecture RISC-V is an open-source instruction set architecture. Added support for the 32bit version of RISC-V to Zephyr. 1) exceptions/interrupts/faults are handled at the architecture level via the __irq_wrapper handler. Context saving/restoring of registers can be handled at both architecture and SOC levels. If SOC-specific registers need to be saved, SOC level needs to provide __soc_save_context and __soc_restore_context functions that shall be accounted by the architecture level, when corresponding config variable RISCV_SOC_CONTEXT_SAVE is set. 2) As RISC-V architecture does not provide a clear ISA specification about interrupt handling, each RISC-V SOC handles it in its own way. Hence, at the architecture level, the __irq_wrapper handler expects the following functions to be provided by the SOC level: __soc_is_irq: to check if the exception is the result of an interrupt or not. __soc_handle_irq: handle pending IRQ at SOC level (ex: clear pending IRQ in SOC-specific IRQ register) 3) Thread/task scheduling, as well as IRQ offloading are handled via the RISC-V system call ("ecall"), which is also handled via the __irq_wrapper handler. The _Swap asm function just calls "ecall" to generate an exception. 4) As there is no conventional way of handling CPU power save in RISC-V, the default nano_cpu_idle and nano_cpu_atomic_idle functions just unlock interrupts and return to the caller, without issuing any CPU power saving instruction. Nonetheless, to allow SOC-level to implement proper CPU power save, nano_cpu_idle and nano_cpu_atomic_idle functions are defined as __weak at the architecture level. Change-Id: I980a161d0009f3f404ad22b226a6229fbb492389 Signed-off-by: Jean-Paul Etienne <fractalclone@gmail.com>
2017-01-11 00:24:30 +01:00
kernel: event_logger: remove kernel_event_logger Move to more generic tracing hooks that can be implemented in different ways and do not interfere with the kernel. Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-04-06 07:48:53 -04:00
#ifdef CONFIG_TRACING
debug: tracing: Remove unneeded abstraction Various C and Assembly modules make function calls to z_sys_trace_*. These merely call corresponding functions sys_trace_*. This commit is to simplify these by making direct function calls to the sys_trace_* functions from these modules. Subsequently, the z_sys_trace_* functions are removed. Signed-off-by: Mrinal Sen <msen@oticon.com>
2019-09-19 09:25:19 +02:00
GTEXT(sys_trace_isr_enter)
arch: added support for the riscv32 architecture RISC-V is an open-source instruction set architecture. Added support for the 32bit version of RISC-V to Zephyr. 1) exceptions/interrupts/faults are handled at the architecture level via the __irq_wrapper handler. Context saving/restoring of registers can be handled at both architecture and SOC levels. If SOC-specific registers need to be saved, SOC level needs to provide __soc_save_context and __soc_restore_context functions that shall be accounted by the architecture level, when corresponding config variable RISCV_SOC_CONTEXT_SAVE is set. 2) As RISC-V architecture does not provide a clear ISA specification about interrupt handling, each RISC-V SOC handles it in its own way. Hence, at the architecture level, the __irq_wrapper handler expects the following functions to be provided by the SOC level: __soc_is_irq: to check if the exception is the result of an interrupt or not. __soc_handle_irq: handle pending IRQ at SOC level (ex: clear pending IRQ in SOC-specific IRQ register) 3) Thread/task scheduling, as well as IRQ offloading are handled via the RISC-V system call ("ecall"), which is also handled via the __irq_wrapper handler. The _Swap asm function just calls "ecall" to generate an exception. 4) As there is no conventional way of handling CPU power save in RISC-V, the default nano_cpu_idle and nano_cpu_atomic_idle functions just unlock interrupts and return to the caller, without issuing any CPU power saving instruction. Nonetheless, to allow SOC-level to implement proper CPU power save, nano_cpu_idle and nano_cpu_atomic_idle functions are defined as __weak at the architecture level. Change-Id: I980a161d0009f3f404ad22b226a6229fbb492389 Signed-off-by: Jean-Paul Etienne <fractalclone@gmail.com>
2017-01-11 00:24:30 +01:00
#endif
arch: riscv: add memory protection support The IRQ handler has had a major changes to manage syscall, reschedule and interrupt from user thread and stack guard. Add userspace support: - Use a global variable to know if the current execution is user or machine. The location of this variable is read only for all user thread and read/write for kernel thread. - Memory shared is supported. - Use dynamic allocation to optimize PMP slot usage. If the area size is a power of 2, only one PMP slot is used, else 2 are used. Add stack guard support: - Use MPRV bit to force PMP rules to machine mode execution. - IRQ stack have a locked stack guard to avoid re-write PMP configuration registers for each interruption and then win some cycle. - The IRQ stack is used as "temporary" stack at the beginning of IRQ handler to save current ESF. That avoid to trigger write fault on thread stack during store ESF which that call IRQ handler to infinity. - A stack guard is also setup for privileged stack of a user thread. Thread: - A PMP setup is specific to each thread. PMP setup are saved in each thread structure to improve reschedule performance. Signed-off-by: Alexandre Mergnat <amergnat@baylibre.com> Reviewed-by: Nicolas Royer <nroyer@baylibre.com>
2020-07-21 16:00:39 +02:00
#ifdef CONFIG_USERSPACE
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
GDATA(_k_syscall_table)
arch: riscv: add memory protection support The IRQ handler has had a major changes to manage syscall, reschedule and interrupt from user thread and stack guard. Add userspace support: - Use a global variable to know if the current execution is user or machine. The location of this variable is read only for all user thread and read/write for kernel thread. - Memory shared is supported. - Use dynamic allocation to optimize PMP slot usage. If the area size is a power of 2, only one PMP slot is used, else 2 are used. Add stack guard support: - Use MPRV bit to force PMP rules to machine mode execution. - IRQ stack have a locked stack guard to avoid re-write PMP configuration registers for each interruption and then win some cycle. - The IRQ stack is used as "temporary" stack at the beginning of IRQ handler to save current ESF. That avoid to trigger write fault on thread stack during store ESF which that call IRQ handler to infinity. - A stack guard is also setup for privileged stack of a user thread. Thread: - A PMP setup is specific to each thread. PMP setup are saved in each thread structure to improve reschedule performance. Signed-off-by: Alexandre Mergnat <amergnat@baylibre.com> Reviewed-by: Nicolas Royer <nroyer@baylibre.com>
2020-07-21 16:00:39 +02:00
#endif
arch: added support for the riscv32 architecture RISC-V is an open-source instruction set architecture. Added support for the 32bit version of RISC-V to Zephyr. 1) exceptions/interrupts/faults are handled at the architecture level via the __irq_wrapper handler. Context saving/restoring of registers can be handled at both architecture and SOC levels. If SOC-specific registers need to be saved, SOC level needs to provide __soc_save_context and __soc_restore_context functions that shall be accounted by the architecture level, when corresponding config variable RISCV_SOC_CONTEXT_SAVE is set. 2) As RISC-V architecture does not provide a clear ISA specification about interrupt handling, each RISC-V SOC handles it in its own way. Hence, at the architecture level, the __irq_wrapper handler expects the following functions to be provided by the SOC level: __soc_is_irq: to check if the exception is the result of an interrupt or not. __soc_handle_irq: handle pending IRQ at SOC level (ex: clear pending IRQ in SOC-specific IRQ register) 3) Thread/task scheduling, as well as IRQ offloading are handled via the RISC-V system call ("ecall"), which is also handled via the __irq_wrapper handler. The _Swap asm function just calls "ecall" to generate an exception. 4) As there is no conventional way of handling CPU power save in RISC-V, the default nano_cpu_idle and nano_cpu_atomic_idle functions just unlock interrupts and return to the caller, without issuing any CPU power saving instruction. Nonetheless, to allow SOC-level to implement proper CPU power save, nano_cpu_idle and nano_cpu_atomic_idle functions are defined as __weak at the architecture level. Change-Id: I980a161d0009f3f404ad22b226a6229fbb492389 Signed-off-by: Jean-Paul Etienne <fractalclone@gmail.com>
2017-01-11 00:24:30 +01:00
/* exports */
GTEXT(__irq_wrapper)
/* use ABI name of registers for the sake of simplicity */
/*
arch: riscv32: provide a general mechanism for saving SoC context RISC-V permits myriad extensions to the ISA, any of which may imply additional context that must be saved and restored on ISR entry and exit. The current in-tree example is the Pulpino core, which has extra registers used by ISA extensions for running loops that shouldn't get clobbered by an ISR. This is currently supported by including pulpino-specific definitions in the generic architecture code. This works, but it's a bit inelegant and is something of a layering violation. A more generic mechanism is required to support other RISC-V SoCs with similar requirements without cluttering the arch code too much. Provide that by extending the semantics of the existing CONFIG_RISCV_SOC_CONTEXT_SAVE option to allow other SoCs to allocate space for saving and restoring their own state, promoting the currently pulpino-specific __soc_save_context / __soc_restore_context routines to a RISC-V arch API. The cost of making this generic is two more instructions in each ISR to pass the SoC specific context to these routines in a0 rather than just assuming the stack points to the right place. This is minimal, and should have been done anyway to keep with the ABI. As a first (and currently only in-tree) customer, convert the Pulpino SoC code to this new mechanism. Signed-off-by: Marti Bolivar <marti@foundries.io>
2018-11-01 22:25:13 -06:00
* Generic architecture-level IRQ handling, along with callouts to
* SoC-specific routines.
*
* Architecture level IRQ handling includes basic context save/restore
* of standard registers and calling ISRs registered at Zephyr's driver
* level.
*
* Since RISC-V does not completely prescribe IRQ handling behavior,
* implementations vary (some implementations also deviate from
* what standard behavior is defined). Hence, the arch level code expects
* the following functions to be provided at the SOC level:
*
* - __soc_is_irq: decide if we're handling an interrupt or an exception
* - __soc_handle_irq: handle SoC-specific details for a pending IRQ
* (e.g. clear a pending bit in a SoC-specific register)
arch: added support for the riscv32 architecture RISC-V is an open-source instruction set architecture. Added support for the 32bit version of RISC-V to Zephyr. 1) exceptions/interrupts/faults are handled at the architecture level via the __irq_wrapper handler. Context saving/restoring of registers can be handled at both architecture and SOC levels. If SOC-specific registers need to be saved, SOC level needs to provide __soc_save_context and __soc_restore_context functions that shall be accounted by the architecture level, when corresponding config variable RISCV_SOC_CONTEXT_SAVE is set. 2) As RISC-V architecture does not provide a clear ISA specification about interrupt handling, each RISC-V SOC handles it in its own way. Hence, at the architecture level, the __irq_wrapper handler expects the following functions to be provided by the SOC level: __soc_is_irq: to check if the exception is the result of an interrupt or not. __soc_handle_irq: handle pending IRQ at SOC level (ex: clear pending IRQ in SOC-specific IRQ register) 3) Thread/task scheduling, as well as IRQ offloading are handled via the RISC-V system call ("ecall"), which is also handled via the __irq_wrapper handler. The _Swap asm function just calls "ecall" to generate an exception. 4) As there is no conventional way of handling CPU power save in RISC-V, the default nano_cpu_idle and nano_cpu_atomic_idle functions just unlock interrupts and return to the caller, without issuing any CPU power saving instruction. Nonetheless, to allow SOC-level to implement proper CPU power save, nano_cpu_idle and nano_cpu_atomic_idle functions are defined as __weak at the architecture level. Change-Id: I980a161d0009f3f404ad22b226a6229fbb492389 Signed-off-by: Jean-Paul Etienne <fractalclone@gmail.com>
2017-01-11 00:24:30 +01:00
*
arch: riscv32: Fix trivial comment The Kconfig sybmol referenced in the comment is called CONFIG_RISCV_SOC_CONTEXT_SAVE not CONFIG_RISCV_SOC_CONTEXT. Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-01-31 04:40:07 -06:00
* If CONFIG_RISCV_SOC_CONTEXT_SAVE=y, calls to SoC-level context save/restore
arch: riscv32: provide a general mechanism for saving SoC context RISC-V permits myriad extensions to the ISA, any of which may imply additional context that must be saved and restored on ISR entry and exit. The current in-tree example is the Pulpino core, which has extra registers used by ISA extensions for running loops that shouldn't get clobbered by an ISR. This is currently supported by including pulpino-specific definitions in the generic architecture code. This works, but it's a bit inelegant and is something of a layering violation. A more generic mechanism is required to support other RISC-V SoCs with similar requirements without cluttering the arch code too much. Provide that by extending the semantics of the existing CONFIG_RISCV_SOC_CONTEXT_SAVE option to allow other SoCs to allocate space for saving and restoring their own state, promoting the currently pulpino-specific __soc_save_context / __soc_restore_context routines to a RISC-V arch API. The cost of making this generic is two more instructions in each ISR to pass the SoC specific context to these routines in a0 rather than just assuming the stack points to the right place. This is minimal, and should have been done anyway to keep with the ABI. As a first (and currently only in-tree) customer, convert the Pulpino SoC code to this new mechanism. Signed-off-by: Marti Bolivar <marti@foundries.io>
2018-11-01 22:25:13 -06:00
* routines are also made here. For details, see the Kconfig help text.
arch: added support for the riscv32 architecture RISC-V is an open-source instruction set architecture. Added support for the 32bit version of RISC-V to Zephyr. 1) exceptions/interrupts/faults are handled at the architecture level via the __irq_wrapper handler. Context saving/restoring of registers can be handled at both architecture and SOC levels. If SOC-specific registers need to be saved, SOC level needs to provide __soc_save_context and __soc_restore_context functions that shall be accounted by the architecture level, when corresponding config variable RISCV_SOC_CONTEXT_SAVE is set. 2) As RISC-V architecture does not provide a clear ISA specification about interrupt handling, each RISC-V SOC handles it in its own way. Hence, at the architecture level, the __irq_wrapper handler expects the following functions to be provided by the SOC level: __soc_is_irq: to check if the exception is the result of an interrupt or not. __soc_handle_irq: handle pending IRQ at SOC level (ex: clear pending IRQ in SOC-specific IRQ register) 3) Thread/task scheduling, as well as IRQ offloading are handled via the RISC-V system call ("ecall"), which is also handled via the __irq_wrapper handler. The _Swap asm function just calls "ecall" to generate an exception. 4) As there is no conventional way of handling CPU power save in RISC-V, the default nano_cpu_idle and nano_cpu_atomic_idle functions just unlock interrupts and return to the caller, without issuing any CPU power saving instruction. Nonetheless, to allow SOC-level to implement proper CPU power save, nano_cpu_idle and nano_cpu_atomic_idle functions are defined as __weak at the architecture level. Change-Id: I980a161d0009f3f404ad22b226a6229fbb492389 Signed-off-by: Jean-Paul Etienne <fractalclone@gmail.com>
2017-01-11 00:24:30 +01:00
*/
/*
* Handler called upon each exception/interrupt/fault
* In this architecture, system call (ECALL) is used to perform context
* switching or IRQ offloading (when enabled).
*/
SECTION_FUNC(exception.entry, __irq_wrapper)
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
#ifdef CONFIG_USERSPACE
arch: riscv: add memory protection support The IRQ handler has had a major changes to manage syscall, reschedule and interrupt from user thread and stack guard. Add userspace support: - Use a global variable to know if the current execution is user or machine. The location of this variable is read only for all user thread and read/write for kernel thread. - Memory shared is supported. - Use dynamic allocation to optimize PMP slot usage. If the area size is a power of 2, only one PMP slot is used, else 2 are used. Add stack guard support: - Use MPRV bit to force PMP rules to machine mode execution. - IRQ stack have a locked stack guard to avoid re-write PMP configuration registers for each interruption and then win some cycle. - The IRQ stack is used as "temporary" stack at the beginning of IRQ handler to save current ESF. That avoid to trigger write fault on thread stack during store ESF which that call IRQ handler to infinity. - A stack guard is also setup for privileged stack of a user thread. Thread: - A PMP setup is specific to each thread. PMP setup are saved in each thread structure to improve reschedule performance. Signed-off-by: Alexandre Mergnat <amergnat@baylibre.com> Reviewed-by: Nicolas Royer <nroyer@baylibre.com>
2020-07-21 16:00:39 +02:00
/*
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
* The scratch register contains either the privileged stack pointer
* to use when interrupting a user mode thread, or 0 when interrupting
* kernel mode in which case the current stack should be used.
arch: riscv: add memory protection support The IRQ handler has had a major changes to manage syscall, reschedule and interrupt from user thread and stack guard. Add userspace support: - Use a global variable to know if the current execution is user or machine. The location of this variable is read only for all user thread and read/write for kernel thread. - Memory shared is supported. - Use dynamic allocation to optimize PMP slot usage. If the area size is a power of 2, only one PMP slot is used, else 2 are used. Add stack guard support: - Use MPRV bit to force PMP rules to machine mode execution. - IRQ stack have a locked stack guard to avoid re-write PMP configuration registers for each interruption and then win some cycle. - The IRQ stack is used as "temporary" stack at the beginning of IRQ handler to save current ESF. That avoid to trigger write fault on thread stack during store ESF which that call IRQ handler to infinity. - A stack guard is also setup for privileged stack of a user thread. Thread: - A PMP setup is specific to each thread. PMP setup are saved in each thread structure to improve reschedule performance. Signed-off-by: Alexandre Mergnat <amergnat@baylibre.com> Reviewed-by: Nicolas Royer <nroyer@baylibre.com>
2020-07-21 16:00:39 +02:00
*/
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
csrrw sp, mscratch, sp
bnez sp, 1f
arch: added support for the riscv32 architecture RISC-V is an open-source instruction set architecture. Added support for the 32bit version of RISC-V to Zephyr. 1) exceptions/interrupts/faults are handled at the architecture level via the __irq_wrapper handler. Context saving/restoring of registers can be handled at both architecture and SOC levels. If SOC-specific registers need to be saved, SOC level needs to provide __soc_save_context and __soc_restore_context functions that shall be accounted by the architecture level, when corresponding config variable RISCV_SOC_CONTEXT_SAVE is set. 2) As RISC-V architecture does not provide a clear ISA specification about interrupt handling, each RISC-V SOC handles it in its own way. Hence, at the architecture level, the __irq_wrapper handler expects the following functions to be provided by the SOC level: __soc_is_irq: to check if the exception is the result of an interrupt or not. __soc_handle_irq: handle pending IRQ at SOC level (ex: clear pending IRQ in SOC-specific IRQ register) 3) Thread/task scheduling, as well as IRQ offloading are handled via the RISC-V system call ("ecall"), which is also handled via the __irq_wrapper handler. The _Swap asm function just calls "ecall" to generate an exception. 4) As there is no conventional way of handling CPU power save in RISC-V, the default nano_cpu_idle and nano_cpu_atomic_idle functions just unlock interrupts and return to the caller, without issuing any CPU power saving instruction. Nonetheless, to allow SOC-level to implement proper CPU power save, nano_cpu_idle and nano_cpu_atomic_idle functions are defined as __weak at the architecture level. Change-Id: I980a161d0009f3f404ad22b226a6229fbb492389 Signed-off-by: Jean-Paul Etienne <fractalclone@gmail.com>
2017-01-11 00:24:30 +01:00
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
/* restore privileged stack pointer and zero the scratch reg */
csrrw sp, mscratch, sp
1:
#endif
Revert "arch/riscv: Use arch_switch() for context swap" This reverts commit be28de692c5e2c5546841c49d6a85d5f046474a4. The purpose of this commit will be reintroduced later on top of a cleaner codebase. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-09 17:46:30 -05:00
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
/* Save caller-saved registers on current thread stack. */
addi sp, sp, -__z_arch_esf_t_SIZEOF
DO_CALLER_SAVED(sr) ;
riscv: Add support for floating point This change adds full shared floating point support for the RISCV architecture with minimal impact on threads with floating point support not enabled. Signed-off-by: Corey Wharton <coreyw7@fb.com>
2020-03-11 18:15:29 -07:00
riscv: isr.S: dedicate a register to &current_cpu Stop using &_kernel as this is not SMP friendly. Let's use s0 (after preserving its content) to hold &current_cpu instead so it won't have to be reloaded each time it is needed. This will be even more relevant when SMP support is added. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-09 17:21:27 -05:00
/* Save s0 in the esf and load it with &_current_cpu. */
sr s0, __z_arch_esf_t_s0_OFFSET(sp)
riscv: isr.S: compute _current_cpu using CPU number on SMP To do so efficiently on systems without the mul instruction, we use shifts and adds which is faster and sometimes smaller than a plain loop. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-09 23:47:38 -05:00
GET_CURRENT_CPU(s0, t0)
riscv: isr.S: dedicate a register to &current_cpu Stop using &_kernel as this is not SMP friendly. Let's use s0 (after preserving its content) to hold &current_cpu instead so it won't have to be reloaded each time it is needed. This will be even more relevant when SMP support is added. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-09 17:21:27 -05:00
arch: riscv: add memory protection support The IRQ handler has had a major changes to manage syscall, reschedule and interrupt from user thread and stack guard. Add userspace support: - Use a global variable to know if the current execution is user or machine. The location of this variable is read only for all user thread and read/write for kernel thread. - Memory shared is supported. - Use dynamic allocation to optimize PMP slot usage. If the area size is a power of 2, only one PMP slot is used, else 2 are used. Add stack guard support: - Use MPRV bit to force PMP rules to machine mode execution. - IRQ stack have a locked stack guard to avoid re-write PMP configuration registers for each interruption and then win some cycle. - The IRQ stack is used as "temporary" stack at the beginning of IRQ handler to save current ESF. That avoid to trigger write fault on thread stack during store ESF which that call IRQ handler to infinity. - A stack guard is also setup for privileged stack of a user thread. Thread: - A PMP setup is specific to each thread. PMP setup are saved in each thread structure to improve reschedule performance. Signed-off-by: Alexandre Mergnat <amergnat@baylibre.com> Reviewed-by: Nicolas Royer <nroyer@baylibre.com>
2020-07-21 16:00:39 +02:00
#ifdef CONFIG_USERSPACE
/*
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
* The scratch register now contains either the user mode stack
* pointer, or 0 if entered from kernel mode. Retrieve that value
* and zero the scratch register as we are in kernel mode now.
arch: riscv: add memory protection support The IRQ handler has had a major changes to manage syscall, reschedule and interrupt from user thread and stack guard. Add userspace support: - Use a global variable to know if the current execution is user or machine. The location of this variable is read only for all user thread and read/write for kernel thread. - Memory shared is supported. - Use dynamic allocation to optimize PMP slot usage. If the area size is a power of 2, only one PMP slot is used, else 2 are used. Add stack guard support: - Use MPRV bit to force PMP rules to machine mode execution. - IRQ stack have a locked stack guard to avoid re-write PMP configuration registers for each interruption and then win some cycle. - The IRQ stack is used as "temporary" stack at the beginning of IRQ handler to save current ESF. That avoid to trigger write fault on thread stack during store ESF which that call IRQ handler to infinity. - A stack guard is also setup for privileged stack of a user thread. Thread: - A PMP setup is specific to each thread. PMP setup are saved in each thread structure to improve reschedule performance. Signed-off-by: Alexandre Mergnat <amergnat@baylibre.com> Reviewed-by: Nicolas Royer <nroyer@baylibre.com>
2020-07-21 16:00:39 +02:00
*/
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
csrrw t0, mscratch, zero
bnez t0, 1f
arch: riscv: add memory protection support The IRQ handler has had a major changes to manage syscall, reschedule and interrupt from user thread and stack guard. Add userspace support: - Use a global variable to know if the current execution is user or machine. The location of this variable is read only for all user thread and read/write for kernel thread. - Memory shared is supported. - Use dynamic allocation to optimize PMP slot usage. If the area size is a power of 2, only one PMP slot is used, else 2 are used. Add stack guard support: - Use MPRV bit to force PMP rules to machine mode execution. - IRQ stack have a locked stack guard to avoid re-write PMP configuration registers for each interruption and then win some cycle. - The IRQ stack is used as "temporary" stack at the beginning of IRQ handler to save current ESF. That avoid to trigger write fault on thread stack during store ESF which that call IRQ handler to infinity. - A stack guard is also setup for privileged stack of a user thread. Thread: - A PMP setup is specific to each thread. PMP setup are saved in each thread structure to improve reschedule performance. Signed-off-by: Alexandre Mergnat <amergnat@baylibre.com> Reviewed-by: Nicolas Royer <nroyer@baylibre.com>
2020-07-21 16:00:39 +02:00
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
/* came from kernel mode: adjust stack value */
add t0, sp, __z_arch_esf_t_SIZEOF
1:
/* save stack value to be restored later */
sr t0, __z_arch_esf_t_sp_OFFSET(sp)
arch: riscv: add memory protection support The IRQ handler has had a major changes to manage syscall, reschedule and interrupt from user thread and stack guard. Add userspace support: - Use a global variable to know if the current execution is user or machine. The location of this variable is read only for all user thread and read/write for kernel thread. - Memory shared is supported. - Use dynamic allocation to optimize PMP slot usage. If the area size is a power of 2, only one PMP slot is used, else 2 are used. Add stack guard support: - Use MPRV bit to force PMP rules to machine mode execution. - IRQ stack have a locked stack guard to avoid re-write PMP configuration registers for each interruption and then win some cycle. - The IRQ stack is used as "temporary" stack at the beginning of IRQ handler to save current ESF. That avoid to trigger write fault on thread stack during store ESF which that call IRQ handler to infinity. - A stack guard is also setup for privileged stack of a user thread. Thread: - A PMP setup is specific to each thread. PMP setup are saved in each thread structure to improve reschedule performance. Signed-off-by: Alexandre Mergnat <amergnat@baylibre.com> Reviewed-by: Nicolas Royer <nroyer@baylibre.com>
2020-07-21 16:00:39 +02:00
/* Clear user mode variable */
la t0, is_user_mode
riscv: fix wrong access width in assembly code The thread->base.user_options field is an uint8_t. Access it using lb. A "copy" of it is made into __esf.fp_state. Make that field an uint8_t too and access it with lb/sb. _callee_saved.fcsr is an uint32_t. Access it with lw/sw. Ditto for is_user_mode. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-21 23:18:20 -05:00
sw zero, 0(t0)
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
#endif
riscv: isr.S: optimize FP regs save/restore decision Rely on mstatus rather than thread->base.user_options since it is always up to date (updated by z_riscv_switch) to simplify the code and be SMP proof. Also carry over SF_INIT to the mstatus being restored in case it was changed in the mean time. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-07 17:31:13 -05:00
/* Save MEPC register */
csrr t0, mepc
sr t0, __z_arch_esf_t_mepc_OFFSET(sp)
/* Save MSTATUS register */
csrr t4, mstatus
sr t4, __z_arch_esf_t_mstatus_OFFSET(sp)
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
#if defined(CONFIG_FPU) && defined(CONFIG_FPU_SHARING)
/* Assess whether floating-point registers need to be saved. */
riscv: isr.S: optimize FP regs save/restore decision Rely on mstatus rather than thread->base.user_options since it is always up to date (updated by z_riscv_switch) to simplify the code and be SMP proof. Also carry over SF_INIT to the mstatus being restored in case it was changed in the mean time. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-07 17:31:13 -05:00
li t1, MSTATUS_FS_INIT
and t0, t4, t1
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
beqz t0, skip_store_fp_caller_saved
DO_FP_CALLER_SAVED(fsr, sp)
skip_store_fp_caller_saved:
#endif /* CONFIG_FPU && CONFIG_FPU_SHARING */
#ifdef CONFIG_RISCV_SOC_CONTEXT_SAVE
/* Handle context saving at SOC level. */
addi a0, sp, __z_arch_esf_t_soc_context_OFFSET
jal ra, __soc_save_context
#endif /* CONFIG_RISCV_SOC_CONTEXT_SAVE */
arch: riscv: add memory protection support The IRQ handler has had a major changes to manage syscall, reschedule and interrupt from user thread and stack guard. Add userspace support: - Use a global variable to know if the current execution is user or machine. The location of this variable is read only for all user thread and read/write for kernel thread. - Memory shared is supported. - Use dynamic allocation to optimize PMP slot usage. If the area size is a power of 2, only one PMP slot is used, else 2 are used. Add stack guard support: - Use MPRV bit to force PMP rules to machine mode execution. - IRQ stack have a locked stack guard to avoid re-write PMP configuration registers for each interruption and then win some cycle. - The IRQ stack is used as "temporary" stack at the beginning of IRQ handler to save current ESF. That avoid to trigger write fault on thread stack during store ESF which that call IRQ handler to infinity. - A stack guard is also setup for privileged stack of a user thread. Thread: - A PMP setup is specific to each thread. PMP setup are saved in each thread structure to improve reschedule performance. Signed-off-by: Alexandre Mergnat <amergnat@baylibre.com> Reviewed-by: Nicolas Royer <nroyer@baylibre.com>
2020-07-21 16:00:39 +02:00
Revert "arch/riscv: Use arch_switch() for context swap" This reverts commit be28de692c5e2c5546841c49d6a85d5f046474a4. The purpose of this commit will be reintroduced later on top of a cleaner codebase. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-09 17:46:30 -05:00
/*
* Check if exception is the result of an interrupt or not.
* (SOC dependent). Following the RISC-V architecture spec, the MSB
* of the mcause register is used to indicate whether an exception
* is the result of an interrupt or an exception/fault. But for some
* SOCs (like pulpino or riscv-qemu), the MSB is never set to indicate
* interrupt. Hence, check for interrupt/exception via the __soc_is_irq
* function (that needs to be implemented by each SOC). The result is
* returned via register a0 (1: interrupt, 0 exception)
*/
jal ra, __soc_is_irq
/* If a0 != 0, jump to is_interrupt */
bnez a0, is_interrupt
arch: added support for the riscv32 architecture RISC-V is an open-source instruction set architecture. Added support for the 32bit version of RISC-V to Zephyr. 1) exceptions/interrupts/faults are handled at the architecture level via the __irq_wrapper handler. Context saving/restoring of registers can be handled at both architecture and SOC levels. If SOC-specific registers need to be saved, SOC level needs to provide __soc_save_context and __soc_restore_context functions that shall be accounted by the architecture level, when corresponding config variable RISCV_SOC_CONTEXT_SAVE is set. 2) As RISC-V architecture does not provide a clear ISA specification about interrupt handling, each RISC-V SOC handles it in its own way. Hence, at the architecture level, the __irq_wrapper handler expects the following functions to be provided by the SOC level: __soc_is_irq: to check if the exception is the result of an interrupt or not. __soc_handle_irq: handle pending IRQ at SOC level (ex: clear pending IRQ in SOC-specific IRQ register) 3) Thread/task scheduling, as well as IRQ offloading are handled via the RISC-V system call ("ecall"), which is also handled via the __irq_wrapper handler. The _Swap asm function just calls "ecall" to generate an exception. 4) As there is no conventional way of handling CPU power save in RISC-V, the default nano_cpu_idle and nano_cpu_atomic_idle functions just unlock interrupts and return to the caller, without issuing any CPU power saving instruction. Nonetheless, to allow SOC-level to implement proper CPU power save, nano_cpu_idle and nano_cpu_atomic_idle functions are defined as __weak at the architecture level. Change-Id: I980a161d0009f3f404ad22b226a6229fbb492389 Signed-off-by: Jean-Paul Etienne <fractalclone@gmail.com>
2017-01-11 00:24:30 +01:00
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
#ifdef CONFIG_PMP_STACK_GUARD
li t0, MSTATUS_MPRV
csrs mstatus, t0
#endif
arch: riscv: add memory protection support The IRQ handler has had a major changes to manage syscall, reschedule and interrupt from user thread and stack guard. Add userspace support: - Use a global variable to know if the current execution is user or machine. The location of this variable is read only for all user thread and read/write for kernel thread. - Memory shared is supported. - Use dynamic allocation to optimize PMP slot usage. If the area size is a power of 2, only one PMP slot is used, else 2 are used. Add stack guard support: - Use MPRV bit to force PMP rules to machine mode execution. - IRQ stack have a locked stack guard to avoid re-write PMP configuration registers for each interruption and then win some cycle. - The IRQ stack is used as "temporary" stack at the beginning of IRQ handler to save current ESF. That avoid to trigger write fault on thread stack during store ESF which that call IRQ handler to infinity. - A stack guard is also setup for privileged stack of a user thread. Thread: - A PMP setup is specific to each thread. PMP setup are saved in each thread structure to improve reschedule performance. Signed-off-by: Alexandre Mergnat <amergnat@baylibre.com> Reviewed-by: Nicolas Royer <nroyer@baylibre.com>
2020-07-21 16:00:39 +02:00
Revert "arch/riscv: Use arch_switch() for context swap" This reverts commit be28de692c5e2c5546841c49d6a85d5f046474a4. The purpose of this commit will be reintroduced later on top of a cleaner codebase. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-09 17:46:30 -05:00
/*
* If the exception is the result of an ECALL, check whether to
* perform a context-switch or an IRQ offload. Otherwise call _Fault
* to report the exception.
*/
csrr t0, mcause
li t2, SOC_MCAUSE_EXP_MASK
and t0, t0, t2
arch/riscv: Use arch_switch() for context swap Enable `arch_switch()` as preparation for SMP support. This patch doesn't try to keep support for old style context swap - only switch based swap is supported, to keep things simple. A fair amount of refactoring was done in this patch, specially regarding the code that decides what to do about the ISR. In RISC-V, ECALL instructions are used to signalize several events, such as user space system calls, forced syscall, IRQ offload, return from syscall and context switch. All those handled by the ISR - which also handles interrupts. After refactor, this "dispatching" step is done at the beginning of ISR (just after saving generic registers). As with other platforms, the thread object itself is used as the thread "switch handle" for the context swap. Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
2021-12-21 16:44:57 -08:00
Revert "arch/riscv: Use arch_switch() for context swap" This reverts commit be28de692c5e2c5546841c49d6a85d5f046474a4. The purpose of this commit will be reintroduced later on top of a cleaner codebase. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-09 17:46:30 -05:00
/*
* If mcause == SOC_MCAUSE_ECALL_EXP, handle system call from
* kernel thread.
arch: riscv: add memory protection support The IRQ handler has had a major changes to manage syscall, reschedule and interrupt from user thread and stack guard. Add userspace support: - Use a global variable to know if the current execution is user or machine. The location of this variable is read only for all user thread and read/write for kernel thread. - Memory shared is supported. - Use dynamic allocation to optimize PMP slot usage. If the area size is a power of 2, only one PMP slot is used, else 2 are used. Add stack guard support: - Use MPRV bit to force PMP rules to machine mode execution. - IRQ stack have a locked stack guard to avoid re-write PMP configuration registers for each interruption and then win some cycle. - The IRQ stack is used as "temporary" stack at the beginning of IRQ handler to save current ESF. That avoid to trigger write fault on thread stack during store ESF which that call IRQ handler to infinity. - A stack guard is also setup for privileged stack of a user thread. Thread: - A PMP setup is specific to each thread. PMP setup are saved in each thread structure to improve reschedule performance. Signed-off-by: Alexandre Mergnat <amergnat@baylibre.com> Reviewed-by: Nicolas Royer <nroyer@baylibre.com>
2020-07-21 16:00:39 +02:00
*/
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
li t1, SOC_MCAUSE_ECALL_EXP
Revert "arch/riscv: Use arch_switch() for context swap" This reverts commit be28de692c5e2c5546841c49d6a85d5f046474a4. The purpose of this commit will be reintroduced later on top of a cleaner codebase. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-09 17:46:30 -05:00
beq t0, t1, is_kernel_syscall
arch: riscv: add memory protection support The IRQ handler has had a major changes to manage syscall, reschedule and interrupt from user thread and stack guard. Add userspace support: - Use a global variable to know if the current execution is user or machine. The location of this variable is read only for all user thread and read/write for kernel thread. - Memory shared is supported. - Use dynamic allocation to optimize PMP slot usage. If the area size is a power of 2, only one PMP slot is used, else 2 are used. Add stack guard support: - Use MPRV bit to force PMP rules to machine mode execution. - IRQ stack have a locked stack guard to avoid re-write PMP configuration registers for each interruption and then win some cycle. - The IRQ stack is used as "temporary" stack at the beginning of IRQ handler to save current ESF. That avoid to trigger write fault on thread stack during store ESF which that call IRQ handler to infinity. - A stack guard is also setup for privileged stack of a user thread. Thread: - A PMP setup is specific to each thread. PMP setup are saved in each thread structure to improve reschedule performance. Signed-off-by: Alexandre Mergnat <amergnat@baylibre.com> Reviewed-by: Nicolas Royer <nroyer@baylibre.com>
2020-07-21 16:00:39 +02:00
#ifdef CONFIG_USERSPACE
Revert "arch/riscv: Use arch_switch() for context swap" This reverts commit be28de692c5e2c5546841c49d6a85d5f046474a4. The purpose of this commit will be reintroduced later on top of a cleaner codebase. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-09 17:46:30 -05:00
/*
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
* If mcause == SOC_MCAUSE_USER_ECALL_EXP, handle system call
* for user mode thread.
arch: added support for the riscv32 architecture RISC-V is an open-source instruction set architecture. Added support for the 32bit version of RISC-V to Zephyr. 1) exceptions/interrupts/faults are handled at the architecture level via the __irq_wrapper handler. Context saving/restoring of registers can be handled at both architecture and SOC levels. If SOC-specific registers need to be saved, SOC level needs to provide __soc_save_context and __soc_restore_context functions that shall be accounted by the architecture level, when corresponding config variable RISCV_SOC_CONTEXT_SAVE is set. 2) As RISC-V architecture does not provide a clear ISA specification about interrupt handling, each RISC-V SOC handles it in its own way. Hence, at the architecture level, the __irq_wrapper handler expects the following functions to be provided by the SOC level: __soc_is_irq: to check if the exception is the result of an interrupt or not. __soc_handle_irq: handle pending IRQ at SOC level (ex: clear pending IRQ in SOC-specific IRQ register) 3) Thread/task scheduling, as well as IRQ offloading are handled via the RISC-V system call ("ecall"), which is also handled via the __irq_wrapper handler. The _Swap asm function just calls "ecall" to generate an exception. 4) As there is no conventional way of handling CPU power save in RISC-V, the default nano_cpu_idle and nano_cpu_atomic_idle functions just unlock interrupts and return to the caller, without issuing any CPU power saving instruction. Nonetheless, to allow SOC-level to implement proper CPU power save, nano_cpu_idle and nano_cpu_atomic_idle functions are defined as __weak at the architecture level. Change-Id: I980a161d0009f3f404ad22b226a6229fbb492389 Signed-off-by: Jean-Paul Etienne <fractalclone@gmail.com>
2017-01-11 00:24:30 +01:00
*/
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
li t1, SOC_MCAUSE_USER_ECALL_EXP
Revert "arch/riscv: Use arch_switch() for context swap" This reverts commit be28de692c5e2c5546841c49d6a85d5f046474a4. The purpose of this commit will be reintroduced later on top of a cleaner codebase. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-09 17:46:30 -05:00
beq t0, t1, is_user_syscall
#endif /* CONFIG_USERSPACE */
arch: riscv32: Move MEPC increment in irq_wrapper Move the MEPC increment logic in __irq_wrapper such that the saved MEPC is only incremented on a syscall (ecall instruction). Signed-off-by: Nathaniel Graff <nathaniel.graff@sifive.com>
2018-08-02 10:38:57 -07:00
/*
* Call _Fault to handle exception.
kernel: rename NANO_ESF This is now called z_arch_esf_t, conforming to our naming convention. This needs to remain a typedef due to how our offset generation header mechanism works. Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-07-16 15:21:19 -07:00
* Stack pointer is pointing to a z_arch_esf_t structure, pass it
arch: riscv32: Move MEPC increment in irq_wrapper Move the MEPC increment logic in __irq_wrapper such that the saved MEPC is only incremented on a syscall (ecall instruction). Signed-off-by: Nathaniel Graff <nathaniel.graff@sifive.com>
2018-08-02 10:38:57 -07:00
* to _Fault (via register a0).
arch: riscv: add memory protection support The IRQ handler has had a major changes to manage syscall, reschedule and interrupt from user thread and stack guard. Add userspace support: - Use a global variable to know if the current execution is user or machine. The location of this variable is read only for all user thread and read/write for kernel thread. - Memory shared is supported. - Use dynamic allocation to optimize PMP slot usage. If the area size is a power of 2, only one PMP slot is used, else 2 are used. Add stack guard support: - Use MPRV bit to force PMP rules to machine mode execution. - IRQ stack have a locked stack guard to avoid re-write PMP configuration registers for each interruption and then win some cycle. - The IRQ stack is used as "temporary" stack at the beginning of IRQ handler to save current ESF. That avoid to trigger write fault on thread stack during store ESF which that call IRQ handler to infinity. - A stack guard is also setup for privileged stack of a user thread. Thread: - A PMP setup is specific to each thread. PMP setup are saved in each thread structure to improve reschedule performance. Signed-off-by: Alexandre Mergnat <amergnat@baylibre.com> Reviewed-by: Nicolas Royer <nroyer@baylibre.com>
2020-07-21 16:00:39 +02:00
* If _Fault shall return, set return address to
* no_reschedule to restore stack.
arch: riscv32: Move MEPC increment in irq_wrapper Move the MEPC increment logic in __irq_wrapper such that the saved MEPC is only incremented on a syscall (ecall instruction). Signed-off-by: Nathaniel Graff <nathaniel.graff@sifive.com>
2018-08-02 10:38:57 -07:00
*/
riscv: use simplest asm expression when possible Let's take advantage of assembler pseudoinstructions: - convert `addi rd, rs, 0` to `mv rd, rs` - convert `jal x0, somewhere` to `j somewhere` - convert `csrrs x0, csrreg, rs` to `csrs csrreg, rs` - convert `fscsr x0, rs` to `fscsr rs` And simplify zero offsets to simply 0. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-21 21:36:55 -05:00
mv a0, sp
arch: riscv: fix sp of supervisor thread in _Fault function. Although CONFIG_USERSPACE is enabled, there are supervisor threads who don't have privileged stack using exception handler. Only let user threads to switch to privileged stack in exception handler. Signed-off-by: Jim Shu <cwshu@andestech.com>
2021-07-18 02:07:48 +08:00
la ra, no_reschedule
arch: riscv32: Move MEPC increment in irq_wrapper Move the MEPC increment logic in __irq_wrapper such that the saved MEPC is only incremented on a syscall (ecall instruction). Signed-off-by: Nathaniel Graff <nathaniel.graff@sifive.com>
2018-08-02 10:38:57 -07:00
tail _Fault
arch: added support for the riscv32 architecture RISC-V is an open-source instruction set architecture. Added support for the 32bit version of RISC-V to Zephyr. 1) exceptions/interrupts/faults are handled at the architecture level via the __irq_wrapper handler. Context saving/restoring of registers can be handled at both architecture and SOC levels. If SOC-specific registers need to be saved, SOC level needs to provide __soc_save_context and __soc_restore_context functions that shall be accounted by the architecture level, when corresponding config variable RISCV_SOC_CONTEXT_SAVE is set. 2) As RISC-V architecture does not provide a clear ISA specification about interrupt handling, each RISC-V SOC handles it in its own way. Hence, at the architecture level, the __irq_wrapper handler expects the following functions to be provided by the SOC level: __soc_is_irq: to check if the exception is the result of an interrupt or not. __soc_handle_irq: handle pending IRQ at SOC level (ex: clear pending IRQ in SOC-specific IRQ register) 3) Thread/task scheduling, as well as IRQ offloading are handled via the RISC-V system call ("ecall"), which is also handled via the __irq_wrapper handler. The _Swap asm function just calls "ecall" to generate an exception. 4) As there is no conventional way of handling CPU power save in RISC-V, the default nano_cpu_idle and nano_cpu_atomic_idle functions just unlock interrupts and return to the caller, without issuing any CPU power saving instruction. Nonetheless, to allow SOC-level to implement proper CPU power save, nano_cpu_idle and nano_cpu_atomic_idle functions are defined as __weak at the architecture level. Change-Id: I980a161d0009f3f404ad22b226a6229fbb492389 Signed-off-by: Jean-Paul Etienne <fractalclone@gmail.com>
2017-01-11 00:24:30 +01:00
Revert "arch/riscv: Use arch_switch() for context swap" This reverts commit be28de692c5e2c5546841c49d6a85d5f046474a4. The purpose of this commit will be reintroduced later on top of a cleaner codebase. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-09 17:46:30 -05:00
is_kernel_syscall:
/*
* A syscall is the result of an ecall instruction, in which case the
* MEPC will contain the address of the ecall instruction.
* Increment saved MEPC by 4 to prevent triggering the same ecall
* again upon exiting the ISR.
*
* It's safe to always increment by 4, even with compressed
* instructions, because the ecall instruction is always 4 bytes.
*/
riscv: better abstraction for register-wide load/store opcodes Those are prominent enough that having RV_OP_LOADREG and RV_OP_STOREREG shouting at you all over the place is rather unpleasant and bad taste. Let's create pseudo-instructions of our own with assembler macros rather than preprocessor defines and only in assembly scope. This makes the asm code way more uniform and readable. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-22 14:09:55 -05:00
lr t0, __z_arch_esf_t_mepc_OFFSET(sp)
Revert "arch/riscv: Use arch_switch() for context swap" This reverts commit be28de692c5e2c5546841c49d6a85d5f046474a4. The purpose of this commit will be reintroduced later on top of a cleaner codebase. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-09 17:46:30 -05:00
addi t0, t0, 4
riscv: better abstraction for register-wide load/store opcodes Those are prominent enough that having RV_OP_LOADREG and RV_OP_STOREREG shouting at you all over the place is rather unpleasant and bad taste. Let's create pseudo-instructions of our own with assembler macros rather than preprocessor defines and only in assembly scope. This makes the asm code way more uniform and readable. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-22 14:09:55 -05:00
sr t0, __z_arch_esf_t_mepc_OFFSET(sp)
Revert "arch/riscv: Use arch_switch() for context swap" This reverts commit be28de692c5e2c5546841c49d6a85d5f046474a4. The purpose of this commit will be reintroduced later on top of a cleaner codebase. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-09 17:46:30 -05:00
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
/* Determine what to do. Operation code is in a7. */
lr a7, __z_arch_esf_t_a7_OFFSET(sp)
Revert "arch/riscv: Use arch_switch() for context swap" This reverts commit be28de692c5e2c5546841c49d6a85d5f046474a4. The purpose of this commit will be reintroduced later on top of a cleaner codebase. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-09 17:46:30 -05:00
riscv: implement arch_switch() The move to arch_switch() is a prerequisite for SMP support. Make it optimal without the need for an ECALL roundtrip on every context switch. Performance numbers from tests/benchmarks/sched: Before: unpend 107 ready 102 switch 188 pend 218 tot 615 (avg 615) After: unpend 107 ready 102 switch 170 pend 217 tot 596 (avg 595) Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-07 17:01:36 -05:00
ASSUME_EQUAL(RV_ECALL_RUNTIME_EXCEPT, 0)
beqz a7, do_fault
arch: riscv: add memory protection support The IRQ handler has had a major changes to manage syscall, reschedule and interrupt from user thread and stack guard. Add userspace support: - Use a global variable to know if the current execution is user or machine. The location of this variable is read only for all user thread and read/write for kernel thread. - Memory shared is supported. - Use dynamic allocation to optimize PMP slot usage. If the area size is a power of 2, only one PMP slot is used, else 2 are used. Add stack guard support: - Use MPRV bit to force PMP rules to machine mode execution. - IRQ stack have a locked stack guard to avoid re-write PMP configuration registers for each interruption and then win some cycle. - The IRQ stack is used as "temporary" stack at the beginning of IRQ handler to save current ESF. That avoid to trigger write fault on thread stack during store ESF which that call IRQ handler to infinity. - A stack guard is also setup for privileged stack of a user thread. Thread: - A PMP setup is specific to each thread. PMP setup are saved in each thread structure to improve reschedule performance. Signed-off-by: Alexandre Mergnat <amergnat@baylibre.com> Reviewed-by: Nicolas Royer <nroyer@baylibre.com>
2020-07-21 16:00:39 +02:00
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
#if defined(CONFIG_IRQ_OFFLOAD)
addi a7, a7, -1
ASSUME_EQUAL(RV_ECALL_IRQ_OFFLOAD, 1)
beqz a7, do_irq_offload
#endif
arch: riscv: add memory protection support The IRQ handler has had a major changes to manage syscall, reschedule and interrupt from user thread and stack guard. Add userspace support: - Use a global variable to know if the current execution is user or machine. The location of this variable is read only for all user thread and read/write for kernel thread. - Memory shared is supported. - Use dynamic allocation to optimize PMP slot usage. If the area size is a power of 2, only one PMP slot is used, else 2 are used. Add stack guard support: - Use MPRV bit to force PMP rules to machine mode execution. - IRQ stack have a locked stack guard to avoid re-write PMP configuration registers for each interruption and then win some cycle. - The IRQ stack is used as "temporary" stack at the beginning of IRQ handler to save current ESF. That avoid to trigger write fault on thread stack during store ESF which that call IRQ handler to infinity. - A stack guard is also setup for privileged stack of a user thread. Thread: - A PMP setup is specific to each thread. PMP setup are saved in each thread structure to improve reschedule performance. Signed-off-by: Alexandre Mergnat <amergnat@baylibre.com> Reviewed-by: Nicolas Royer <nroyer@baylibre.com>
2020-07-21 16:00:39 +02:00
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
/* default fault code is K_ERR_KERNEL_OOPS */
li a0, 3
j 1f
arch: riscv: add memory protection support The IRQ handler has had a major changes to manage syscall, reschedule and interrupt from user thread and stack guard. Add userspace support: - Use a global variable to know if the current execution is user or machine. The location of this variable is read only for all user thread and read/write for kernel thread. - Memory shared is supported. - Use dynamic allocation to optimize PMP slot usage. If the area size is a power of 2, only one PMP slot is used, else 2 are used. Add stack guard support: - Use MPRV bit to force PMP rules to machine mode execution. - IRQ stack have a locked stack guard to avoid re-write PMP configuration registers for each interruption and then win some cycle. - The IRQ stack is used as "temporary" stack at the beginning of IRQ handler to save current ESF. That avoid to trigger write fault on thread stack during store ESF which that call IRQ handler to infinity. - A stack guard is also setup for privileged stack of a user thread. Thread: - A PMP setup is specific to each thread. PMP setup are saved in each thread structure to improve reschedule performance. Signed-off-by: Alexandre Mergnat <amergnat@baylibre.com> Reviewed-by: Nicolas Royer <nroyer@baylibre.com>
2020-07-21 16:00:39 +02:00
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
do_fault:
/* Handle RV_ECALL_RUNTIME_EXCEPT. Retrieve reason in a0, esf in A1. */
riscv: better abstraction for register-wide load/store opcodes Those are prominent enough that having RV_OP_LOADREG and RV_OP_STOREREG shouting at you all over the place is rather unpleasant and bad taste. Let's create pseudo-instructions of our own with assembler macros rather than preprocessor defines and only in assembly scope. This makes the asm code way more uniform and readable. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-22 14:09:55 -05:00
lr a0, __z_arch_esf_t_a0_OFFSET(sp)
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
1: mv a1, sp
tail z_riscv_fatal_error
arch: riscv: add memory protection support The IRQ handler has had a major changes to manage syscall, reschedule and interrupt from user thread and stack guard. Add userspace support: - Use a global variable to know if the current execution is user or machine. The location of this variable is read only for all user thread and read/write for kernel thread. - Memory shared is supported. - Use dynamic allocation to optimize PMP slot usage. If the area size is a power of 2, only one PMP slot is used, else 2 are used. Add stack guard support: - Use MPRV bit to force PMP rules to machine mode execution. - IRQ stack have a locked stack guard to avoid re-write PMP configuration registers for each interruption and then win some cycle. - The IRQ stack is used as "temporary" stack at the beginning of IRQ handler to save current ESF. That avoid to trigger write fault on thread stack during store ESF which that call IRQ handler to infinity. - A stack guard is also setup for privileged stack of a user thread. Thread: - A PMP setup is specific to each thread. PMP setup are saved in each thread structure to improve reschedule performance. Signed-off-by: Alexandre Mergnat <amergnat@baylibre.com> Reviewed-by: Nicolas Royer <nroyer@baylibre.com>
2020-07-21 16:00:39 +02:00
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
#if defined(CONFIG_IRQ_OFFLOAD)
do_irq_offload:
riscv: irq_offload: simpler implementation Get rid of all those global variables and IRQ locking. Use the regular IRQ exit path to let tests validate preemption properly. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-07 13:24:38 -05:00
/*
* Retrieve provided routine and argument from the stack.
* Routine pointer is in saved a0, argument in saved a1
* so we load them with a1/a0 (reversed).
*/
lr a1, __z_arch_esf_t_a0_OFFSET(sp)
lr a0, __z_arch_esf_t_a1_OFFSET(sp)
riscv: isr.S: dedicate a register to &current_cpu Stop using &_kernel as this is not SMP friendly. Let's use s0 (after preserving its content) to hold &current_cpu instead so it won't have to be reloaded each time it is needed. This will be even more relevant when SMP support is added. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-09 17:21:27 -05:00
/* Increment _current_cpu->nested */
lw t3, ___cpu_t_nested_OFFSET(s0)
riscv: implement CONFIG_IRQ_OFFLOAD_NESTED It can easily be done now, so why not. Suffice to increment the nested count like with actual IRQs. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-14 14:51:04 -04:00
addi t4, t3, 1
riscv: isr.S: dedicate a register to &current_cpu Stop using &_kernel as this is not SMP friendly. Let's use s0 (after preserving its content) to hold &current_cpu instead so it won't have to be reloaded each time it is needed. This will be even more relevant when SMP support is added. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-09 17:21:27 -05:00
sw t4, ___cpu_t_nested_OFFSET(s0)
riscv: implement CONFIG_IRQ_OFFLOAD_NESTED It can easily be done now, so why not. Suffice to increment the nested count like with actual IRQs. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-14 14:51:04 -04:00
bnez t3, 1f
arch: added support for the riscv32 architecture RISC-V is an open-source instruction set architecture. Added support for the 32bit version of RISC-V to Zephyr. 1) exceptions/interrupts/faults are handled at the architecture level via the __irq_wrapper handler. Context saving/restoring of registers can be handled at both architecture and SOC levels. If SOC-specific registers need to be saved, SOC level needs to provide __soc_save_context and __soc_restore_context functions that shall be accounted by the architecture level, when corresponding config variable RISCV_SOC_CONTEXT_SAVE is set. 2) As RISC-V architecture does not provide a clear ISA specification about interrupt handling, each RISC-V SOC handles it in its own way. Hence, at the architecture level, the __irq_wrapper handler expects the following functions to be provided by the SOC level: __soc_is_irq: to check if the exception is the result of an interrupt or not. __soc_handle_irq: handle pending IRQ at SOC level (ex: clear pending IRQ in SOC-specific IRQ register) 3) Thread/task scheduling, as well as IRQ offloading are handled via the RISC-V system call ("ecall"), which is also handled via the __irq_wrapper handler. The _Swap asm function just calls "ecall" to generate an exception. 4) As there is no conventional way of handling CPU power save in RISC-V, the default nano_cpu_idle and nano_cpu_atomic_idle functions just unlock interrupts and return to the caller, without issuing any CPU power saving instruction. Nonetheless, to allow SOC-level to implement proper CPU power save, nano_cpu_idle and nano_cpu_atomic_idle functions are defined as __weak at the architecture level. Change-Id: I980a161d0009f3f404ad22b226a6229fbb492389 Signed-off-by: Jean-Paul Etienne <fractalclone@gmail.com>
2017-01-11 00:24:30 +01:00
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
/* Switch to interrupt stack */
mv t0, sp
riscv: isr.S: dedicate a register to &current_cpu Stop using &_kernel as this is not SMP friendly. Let's use s0 (after preserving its content) to hold &current_cpu instead so it won't have to be reloaded each time it is needed. This will be even more relevant when SMP support is added. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-09 17:21:27 -05:00
lr sp, ___cpu_t_irq_stack_OFFSET(s0)
arch: riscv: add memory protection support The IRQ handler has had a major changes to manage syscall, reschedule and interrupt from user thread and stack guard. Add userspace support: - Use a global variable to know if the current execution is user or machine. The location of this variable is read only for all user thread and read/write for kernel thread. - Memory shared is supported. - Use dynamic allocation to optimize PMP slot usage. If the area size is a power of 2, only one PMP slot is used, else 2 are used. Add stack guard support: - Use MPRV bit to force PMP rules to machine mode execution. - IRQ stack have a locked stack guard to avoid re-write PMP configuration registers for each interruption and then win some cycle. - The IRQ stack is used as "temporary" stack at the beginning of IRQ handler to save current ESF. That avoid to trigger write fault on thread stack during store ESF which that call IRQ handler to infinity. - A stack guard is also setup for privileged stack of a user thread. Thread: - A PMP setup is specific to each thread. PMP setup are saved in each thread structure to improve reschedule performance. Signed-off-by: Alexandre Mergnat <amergnat@baylibre.com> Reviewed-by: Nicolas Royer <nroyer@baylibre.com>
2020-07-21 16:00:39 +02:00
riscv: implement CONFIG_IRQ_OFFLOAD_NESTED It can easily be done now, so why not. Suffice to increment the nested count like with actual IRQs. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-14 14:51:04 -04:00
/* Save thread stack pointer on interrupt stack */
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
addi sp, sp, -16
sr t0, 0(sp)
riscv: implement CONFIG_IRQ_OFFLOAD_NESTED It can easily be done now, so why not. Suffice to increment the nested count like with actual IRQs. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-14 14:51:04 -04:00
1:
riscv: irq_offload: simpler implementation Get rid of all those global variables and IRQ locking. Use the regular IRQ exit path to let tests validate preemption properly. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-07 13:24:38 -05:00
/* Execute provided routine (argument is in a0 already). */
jalr ra, a1, 0
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
riscv: irq_offload: simpler implementation Get rid of all those global variables and IRQ locking. Use the regular IRQ exit path to let tests validate preemption properly. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-07 13:24:38 -05:00
/* Leave through the regular IRQ exit path */
j irq_done
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
#endif /* CONFIG_IRQ_OFFLOAD */
arch: added support for the riscv32 architecture RISC-V is an open-source instruction set architecture. Added support for the 32bit version of RISC-V to Zephyr. 1) exceptions/interrupts/faults are handled at the architecture level via the __irq_wrapper handler. Context saving/restoring of registers can be handled at both architecture and SOC levels. If SOC-specific registers need to be saved, SOC level needs to provide __soc_save_context and __soc_restore_context functions that shall be accounted by the architecture level, when corresponding config variable RISCV_SOC_CONTEXT_SAVE is set. 2) As RISC-V architecture does not provide a clear ISA specification about interrupt handling, each RISC-V SOC handles it in its own way. Hence, at the architecture level, the __irq_wrapper handler expects the following functions to be provided by the SOC level: __soc_is_irq: to check if the exception is the result of an interrupt or not. __soc_handle_irq: handle pending IRQ at SOC level (ex: clear pending IRQ in SOC-specific IRQ register) 3) Thread/task scheduling, as well as IRQ offloading are handled via the RISC-V system call ("ecall"), which is also handled via the __irq_wrapper handler. The _Swap asm function just calls "ecall" to generate an exception. 4) As there is no conventional way of handling CPU power save in RISC-V, the default nano_cpu_idle and nano_cpu_atomic_idle functions just unlock interrupts and return to the caller, without issuing any CPU power saving instruction. Nonetheless, to allow SOC-level to implement proper CPU power save, nano_cpu_idle and nano_cpu_atomic_idle functions are defined as __weak at the architecture level. Change-Id: I980a161d0009f3f404ad22b226a6229fbb492389 Signed-off-by: Jean-Paul Etienne <fractalclone@gmail.com>
2017-01-11 00:24:30 +01:00
arch: riscv: add memory protection support The IRQ handler has had a major changes to manage syscall, reschedule and interrupt from user thread and stack guard. Add userspace support: - Use a global variable to know if the current execution is user or machine. The location of this variable is read only for all user thread and read/write for kernel thread. - Memory shared is supported. - Use dynamic allocation to optimize PMP slot usage. If the area size is a power of 2, only one PMP slot is used, else 2 are used. Add stack guard support: - Use MPRV bit to force PMP rules to machine mode execution. - IRQ stack have a locked stack guard to avoid re-write PMP configuration registers for each interruption and then win some cycle. - The IRQ stack is used as "temporary" stack at the beginning of IRQ handler to save current ESF. That avoid to trigger write fault on thread stack during store ESF which that call IRQ handler to infinity. - A stack guard is also setup for privileged stack of a user thread. Thread: - A PMP setup is specific to each thread. PMP setup are saved in each thread structure to improve reschedule performance. Signed-off-by: Alexandre Mergnat <amergnat@baylibre.com> Reviewed-by: Nicolas Royer <nroyer@baylibre.com>
2020-07-21 16:00:39 +02:00
#ifdef CONFIG_USERSPACE
is_user_syscall:
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
/* It is safe to re-enable IRQs now */
csrs mstatus, MSTATUS_IEN
arch: riscv: add memory protection support The IRQ handler has had a major changes to manage syscall, reschedule and interrupt from user thread and stack guard. Add userspace support: - Use a global variable to know if the current execution is user or machine. The location of this variable is read only for all user thread and read/write for kernel thread. - Memory shared is supported. - Use dynamic allocation to optimize PMP slot usage. If the area size is a power of 2, only one PMP slot is used, else 2 are used. Add stack guard support: - Use MPRV bit to force PMP rules to machine mode execution. - IRQ stack have a locked stack guard to avoid re-write PMP configuration registers for each interruption and then win some cycle. - The IRQ stack is used as "temporary" stack at the beginning of IRQ handler to save current ESF. That avoid to trigger write fault on thread stack during store ESF which that call IRQ handler to infinity. - A stack guard is also setup for privileged stack of a user thread. Thread: - A PMP setup is specific to each thread. PMP setup are saved in each thread structure to improve reschedule performance. Signed-off-by: Alexandre Mergnat <amergnat@baylibre.com> Reviewed-by: Nicolas Royer <nroyer@baylibre.com>
2020-07-21 16:00:39 +02:00
Revert "arch/riscv: Use arch_switch() for context swap" This reverts commit be28de692c5e2c5546841c49d6a85d5f046474a4. The purpose of this commit will be reintroduced later on top of a cleaner codebase. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-09 17:46:30 -05:00
/*
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
* Same as for is_kernel_syscall: increment saved MEPC by 4 to
* prevent triggering the same ecall again upon exiting the ISR.
Revert "arch/riscv: Use arch_switch() for context swap" This reverts commit be28de692c5e2c5546841c49d6a85d5f046474a4. The purpose of this commit will be reintroduced later on top of a cleaner codebase. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-09 17:46:30 -05:00
*/
riscv: better abstraction for register-wide load/store opcodes Those are prominent enough that having RV_OP_LOADREG and RV_OP_STOREREG shouting at you all over the place is rather unpleasant and bad taste. Let's create pseudo-instructions of our own with assembler macros rather than preprocessor defines and only in assembly scope. This makes the asm code way more uniform and readable. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-22 14:09:55 -05:00
lr t1, __z_arch_esf_t_mepc_OFFSET(sp)
Revert "arch/riscv: Use arch_switch() for context swap" This reverts commit be28de692c5e2c5546841c49d6a85d5f046474a4. The purpose of this commit will be reintroduced later on top of a cleaner codebase. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-09 17:46:30 -05:00
addi t1, t1, 4
riscv: better abstraction for register-wide load/store opcodes Those are prominent enough that having RV_OP_LOADREG and RV_OP_STOREREG shouting at you all over the place is rather unpleasant and bad taste. Let's create pseudo-instructions of our own with assembler macros rather than preprocessor defines and only in assembly scope. This makes the asm code way more uniform and readable. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-22 14:09:55 -05:00
sr t1, __z_arch_esf_t_mepc_OFFSET(sp)
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
arch: riscv: add memory protection support The IRQ handler has had a major changes to manage syscall, reschedule and interrupt from user thread and stack guard. Add userspace support: - Use a global variable to know if the current execution is user or machine. The location of this variable is read only for all user thread and read/write for kernel thread. - Memory shared is supported. - Use dynamic allocation to optimize PMP slot usage. If the area size is a power of 2, only one PMP slot is used, else 2 are used. Add stack guard support: - Use MPRV bit to force PMP rules to machine mode execution. - IRQ stack have a locked stack guard to avoid re-write PMP configuration registers for each interruption and then win some cycle. - The IRQ stack is used as "temporary" stack at the beginning of IRQ handler to save current ESF. That avoid to trigger write fault on thread stack during store ESF which that call IRQ handler to infinity. - A stack guard is also setup for privileged stack of a user thread. Thread: - A PMP setup is specific to each thread. PMP setup are saved in each thread structure to improve reschedule performance. Signed-off-by: Alexandre Mergnat <amergnat@baylibre.com> Reviewed-by: Nicolas Royer <nroyer@baylibre.com>
2020-07-21 16:00:39 +02:00
/* Restore argument registers from user stack */
riscv: better abstraction for register-wide load/store opcodes Those are prominent enough that having RV_OP_LOADREG and RV_OP_STOREREG shouting at you all over the place is rather unpleasant and bad taste. Let's create pseudo-instructions of our own with assembler macros rather than preprocessor defines and only in assembly scope. This makes the asm code way more uniform and readable. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-22 14:09:55 -05:00
lr a0, __z_arch_esf_t_a0_OFFSET(sp)
lr a1, __z_arch_esf_t_a1_OFFSET(sp)
lr a2, __z_arch_esf_t_a2_OFFSET(sp)
lr a3, __z_arch_esf_t_a3_OFFSET(sp)
lr a4, __z_arch_esf_t_a4_OFFSET(sp)
lr a5, __z_arch_esf_t_a5_OFFSET(sp)
arch: riscv: add memory protection support The IRQ handler has had a major changes to manage syscall, reschedule and interrupt from user thread and stack guard. Add userspace support: - Use a global variable to know if the current execution is user or machine. The location of this variable is read only for all user thread and read/write for kernel thread. - Memory shared is supported. - Use dynamic allocation to optimize PMP slot usage. If the area size is a power of 2, only one PMP slot is used, else 2 are used. Add stack guard support: - Use MPRV bit to force PMP rules to machine mode execution. - IRQ stack have a locked stack guard to avoid re-write PMP configuration registers for each interruption and then win some cycle. - The IRQ stack is used as "temporary" stack at the beginning of IRQ handler to save current ESF. That avoid to trigger write fault on thread stack during store ESF which that call IRQ handler to infinity. - A stack guard is also setup for privileged stack of a user thread. Thread: - A PMP setup is specific to each thread. PMP setup are saved in each thread structure to improve reschedule performance. Signed-off-by: Alexandre Mergnat <amergnat@baylibre.com> Reviewed-by: Nicolas Royer <nroyer@baylibre.com>
2020-07-21 16:00:39 +02:00
mv a6, sp
riscv: better abstraction for register-wide load/store opcodes Those are prominent enough that having RV_OP_LOADREG and RV_OP_STOREREG shouting at you all over the place is rather unpleasant and bad taste. Let's create pseudo-instructions of our own with assembler macros rather than preprocessor defines and only in assembly scope. This makes the asm code way more uniform and readable. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-22 14:09:55 -05:00
lr a7, __z_arch_esf_t_a7_OFFSET(sp)
arch: riscv: add memory protection support The IRQ handler has had a major changes to manage syscall, reschedule and interrupt from user thread and stack guard. Add userspace support: - Use a global variable to know if the current execution is user or machine. The location of this variable is read only for all user thread and read/write for kernel thread. - Memory shared is supported. - Use dynamic allocation to optimize PMP slot usage. If the area size is a power of 2, only one PMP slot is used, else 2 are used. Add stack guard support: - Use MPRV bit to force PMP rules to machine mode execution. - IRQ stack have a locked stack guard to avoid re-write PMP configuration registers for each interruption and then win some cycle. - The IRQ stack is used as "temporary" stack at the beginning of IRQ handler to save current ESF. That avoid to trigger write fault on thread stack during store ESF which that call IRQ handler to infinity. - A stack guard is also setup for privileged stack of a user thread. Thread: - A PMP setup is specific to each thread. PMP setup are saved in each thread structure to improve reschedule performance. Signed-off-by: Alexandre Mergnat <amergnat@baylibre.com> Reviewed-by: Nicolas Royer <nroyer@baylibre.com>
2020-07-21 16:00:39 +02:00
/* validate syscall limit */
li t0, K_SYSCALL_LIMIT
bltu a7, t0, valid_syscall_id
/* bad syscall id. Set arg1 to bad id and set call_id to SYSCALL_BAD */
mv a0, a7
li a7, K_SYSCALL_BAD
valid_syscall_id:
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
la t0, _k_syscall_table
arch: riscv: add memory protection support The IRQ handler has had a major changes to manage syscall, reschedule and interrupt from user thread and stack guard. Add userspace support: - Use a global variable to know if the current execution is user or machine. The location of this variable is read only for all user thread and read/write for kernel thread. - Memory shared is supported. - Use dynamic allocation to optimize PMP slot usage. If the area size is a power of 2, only one PMP slot is used, else 2 are used. Add stack guard support: - Use MPRV bit to force PMP rules to machine mode execution. - IRQ stack have a locked stack guard to avoid re-write PMP configuration registers for each interruption and then win some cycle. - The IRQ stack is used as "temporary" stack at the beginning of IRQ handler to save current ESF. That avoid to trigger write fault on thread stack during store ESF which that call IRQ handler to infinity. - A stack guard is also setup for privileged stack of a user thread. Thread: - A PMP setup is specific to each thread. PMP setup are saved in each thread structure to improve reschedule performance. Signed-off-by: Alexandre Mergnat <amergnat@baylibre.com> Reviewed-by: Nicolas Royer <nroyer@baylibre.com>
2020-07-21 16:00:39 +02:00
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
slli t1, a7, RV_REGSHIFT # Determine offset from indice value
add t0, t0, t1 # Table addr + offset = function addr
lr t3, 0(t0) # Load function address
arch: riscv: add memory protection support The IRQ handler has had a major changes to manage syscall, reschedule and interrupt from user thread and stack guard. Add userspace support: - Use a global variable to know if the current execution is user or machine. The location of this variable is read only for all user thread and read/write for kernel thread. - Memory shared is supported. - Use dynamic allocation to optimize PMP slot usage. If the area size is a power of 2, only one PMP slot is used, else 2 are used. Add stack guard support: - Use MPRV bit to force PMP rules to machine mode execution. - IRQ stack have a locked stack guard to avoid re-write PMP configuration registers for each interruption and then win some cycle. - The IRQ stack is used as "temporary" stack at the beginning of IRQ handler to save current ESF. That avoid to trigger write fault on thread stack during store ESF which that call IRQ handler to infinity. - A stack guard is also setup for privileged stack of a user thread. Thread: - A PMP setup is specific to each thread. PMP setup are saved in each thread structure to improve reschedule performance. Signed-off-by: Alexandre Mergnat <amergnat@baylibre.com> Reviewed-by: Nicolas Royer <nroyer@baylibre.com>
2020-07-21 16:00:39 +02:00
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
/* Execute syscall function */
jalr ra, t3, 0
arch: riscv: add memory protection support The IRQ handler has had a major changes to manage syscall, reschedule and interrupt from user thread and stack guard. Add userspace support: - Use a global variable to know if the current execution is user or machine. The location of this variable is read only for all user thread and read/write for kernel thread. - Memory shared is supported. - Use dynamic allocation to optimize PMP slot usage. If the area size is a power of 2, only one PMP slot is used, else 2 are used. Add stack guard support: - Use MPRV bit to force PMP rules to machine mode execution. - IRQ stack have a locked stack guard to avoid re-write PMP configuration registers for each interruption and then win some cycle. - The IRQ stack is used as "temporary" stack at the beginning of IRQ handler to save current ESF. That avoid to trigger write fault on thread stack during store ESF which that call IRQ handler to infinity. - A stack guard is also setup for privileged stack of a user thread. Thread: - A PMP setup is specific to each thread. PMP setup are saved in each thread structure to improve reschedule performance. Signed-off-by: Alexandre Mergnat <amergnat@baylibre.com> Reviewed-by: Nicolas Royer <nroyer@baylibre.com>
2020-07-21 16:00:39 +02:00
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
/* Update a0 (return value) on the stack */
riscv: better abstraction for register-wide load/store opcodes Those are prominent enough that having RV_OP_LOADREG and RV_OP_STOREREG shouting at you all over the place is rather unpleasant and bad taste. Let's create pseudo-instructions of our own with assembler macros rather than preprocessor defines and only in assembly scope. This makes the asm code way more uniform and readable. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-22 14:09:55 -05:00
sr a0, __z_arch_esf_t_a0_OFFSET(sp)
arch: riscv: add memory protection support The IRQ handler has had a major changes to manage syscall, reschedule and interrupt from user thread and stack guard. Add userspace support: - Use a global variable to know if the current execution is user or machine. The location of this variable is read only for all user thread and read/write for kernel thread. - Memory shared is supported. - Use dynamic allocation to optimize PMP slot usage. If the area size is a power of 2, only one PMP slot is used, else 2 are used. Add stack guard support: - Use MPRV bit to force PMP rules to machine mode execution. - IRQ stack have a locked stack guard to avoid re-write PMP configuration registers for each interruption and then win some cycle. - The IRQ stack is used as "temporary" stack at the beginning of IRQ handler to save current ESF. That avoid to trigger write fault on thread stack during store ESF which that call IRQ handler to infinity. - A stack guard is also setup for privileged stack of a user thread. Thread: - A PMP setup is specific to each thread. PMP setup are saved in each thread structure to improve reschedule performance. Signed-off-by: Alexandre Mergnat <amergnat@baylibre.com> Reviewed-by: Nicolas Royer <nroyer@baylibre.com>
2020-07-21 16:00:39 +02:00
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
/* Disable IRQs again before leaving */
csrc mstatus, MSTATUS_IEN
riscv: isr.S: dedicate a register to &current_cpu Stop using &_kernel as this is not SMP friendly. Let's use s0 (after preserving its content) to hold &current_cpu instead so it won't have to be reloaded each time it is needed. This will be even more relevant when SMP support is added. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-09 17:21:27 -05:00
j might_have_rescheduled
arch: riscv: add memory protection support The IRQ handler has had a major changes to manage syscall, reschedule and interrupt from user thread and stack guard. Add userspace support: - Use a global variable to know if the current execution is user or machine. The location of this variable is read only for all user thread and read/write for kernel thread. - Memory shared is supported. - Use dynamic allocation to optimize PMP slot usage. If the area size is a power of 2, only one PMP slot is used, else 2 are used. Add stack guard support: - Use MPRV bit to force PMP rules to machine mode execution. - IRQ stack have a locked stack guard to avoid re-write PMP configuration registers for each interruption and then win some cycle. - The IRQ stack is used as "temporary" stack at the beginning of IRQ handler to save current ESF. That avoid to trigger write fault on thread stack during store ESF which that call IRQ handler to infinity. - A stack guard is also setup for privileged stack of a user thread. Thread: - A PMP setup is specific to each thread. PMP setup are saved in each thread structure to improve reschedule performance. Signed-off-by: Alexandre Mergnat <amergnat@baylibre.com> Reviewed-by: Nicolas Royer <nroyer@baylibre.com>
2020-07-21 16:00:39 +02:00
#endif /* CONFIG_USERSPACE */
arch: added support for the riscv32 architecture RISC-V is an open-source instruction set architecture. Added support for the 32bit version of RISC-V to Zephyr. 1) exceptions/interrupts/faults are handled at the architecture level via the __irq_wrapper handler. Context saving/restoring of registers can be handled at both architecture and SOC levels. If SOC-specific registers need to be saved, SOC level needs to provide __soc_save_context and __soc_restore_context functions that shall be accounted by the architecture level, when corresponding config variable RISCV_SOC_CONTEXT_SAVE is set. 2) As RISC-V architecture does not provide a clear ISA specification about interrupt handling, each RISC-V SOC handles it in its own way. Hence, at the architecture level, the __irq_wrapper handler expects the following functions to be provided by the SOC level: __soc_is_irq: to check if the exception is the result of an interrupt or not. __soc_handle_irq: handle pending IRQ at SOC level (ex: clear pending IRQ in SOC-specific IRQ register) 3) Thread/task scheduling, as well as IRQ offloading are handled via the RISC-V system call ("ecall"), which is also handled via the __irq_wrapper handler. The _Swap asm function just calls "ecall" to generate an exception. 4) As there is no conventional way of handling CPU power save in RISC-V, the default nano_cpu_idle and nano_cpu_atomic_idle functions just unlock interrupts and return to the caller, without issuing any CPU power saving instruction. Nonetheless, to allow SOC-level to implement proper CPU power save, nano_cpu_idle and nano_cpu_atomic_idle functions are defined as __weak at the architecture level. Change-Id: I980a161d0009f3f404ad22b226a6229fbb492389 Signed-off-by: Jean-Paul Etienne <fractalclone@gmail.com>
2017-01-11 00:24:30 +01:00
is_interrupt:
riscv: isr.S: dedicate a register to &current_cpu Stop using &_kernel as this is not SMP friendly. Let's use s0 (after preserving its content) to hold &current_cpu instead so it won't have to be reloaded each time it is needed. This will be even more relevant when SMP support is added. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-09 17:21:27 -05:00
/* Increment _current_cpu->nested */
lw t3, ___cpu_t_nested_OFFSET(s0)
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
addi t4, t3, 1
riscv: isr.S: dedicate a register to &current_cpu Stop using &_kernel as this is not SMP friendly. Let's use s0 (after preserving its content) to hold &current_cpu instead so it won't have to be reloaded each time it is needed. This will be even more relevant when SMP support is added. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-09 17:21:27 -05:00
sw t4, ___cpu_t_nested_OFFSET(s0)
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
bnez t3, on_irq_stack
arch: added support for the riscv32 architecture RISC-V is an open-source instruction set architecture. Added support for the 32bit version of RISC-V to Zephyr. 1) exceptions/interrupts/faults are handled at the architecture level via the __irq_wrapper handler. Context saving/restoring of registers can be handled at both architecture and SOC levels. If SOC-specific registers need to be saved, SOC level needs to provide __soc_save_context and __soc_restore_context functions that shall be accounted by the architecture level, when corresponding config variable RISCV_SOC_CONTEXT_SAVE is set. 2) As RISC-V architecture does not provide a clear ISA specification about interrupt handling, each RISC-V SOC handles it in its own way. Hence, at the architecture level, the __irq_wrapper handler expects the following functions to be provided by the SOC level: __soc_is_irq: to check if the exception is the result of an interrupt or not. __soc_handle_irq: handle pending IRQ at SOC level (ex: clear pending IRQ in SOC-specific IRQ register) 3) Thread/task scheduling, as well as IRQ offloading are handled via the RISC-V system call ("ecall"), which is also handled via the __irq_wrapper handler. The _Swap asm function just calls "ecall" to generate an exception. 4) As there is no conventional way of handling CPU power save in RISC-V, the default nano_cpu_idle and nano_cpu_atomic_idle functions just unlock interrupts and return to the caller, without issuing any CPU power saving instruction. Nonetheless, to allow SOC-level to implement proper CPU power save, nano_cpu_idle and nano_cpu_atomic_idle functions are defined as __weak at the architecture level. Change-Id: I980a161d0009f3f404ad22b226a6229fbb492389 Signed-off-by: Jean-Paul Etienne <fractalclone@gmail.com>
2017-01-11 00:24:30 +01:00
/* Switch to interrupt stack */
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
mv t0, sp
riscv: isr.S: dedicate a register to &current_cpu Stop using &_kernel as this is not SMP friendly. Let's use s0 (after preserving its content) to hold &current_cpu instead so it won't have to be reloaded each time it is needed. This will be even more relevant when SMP support is added. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-09 17:21:27 -05:00
lr sp, ___cpu_t_irq_stack_OFFSET(s0)
arch: added support for the riscv32 architecture RISC-V is an open-source instruction set architecture. Added support for the 32bit version of RISC-V to Zephyr. 1) exceptions/interrupts/faults are handled at the architecture level via the __irq_wrapper handler. Context saving/restoring of registers can be handled at both architecture and SOC levels. If SOC-specific registers need to be saved, SOC level needs to provide __soc_save_context and __soc_restore_context functions that shall be accounted by the architecture level, when corresponding config variable RISCV_SOC_CONTEXT_SAVE is set. 2) As RISC-V architecture does not provide a clear ISA specification about interrupt handling, each RISC-V SOC handles it in its own way. Hence, at the architecture level, the __irq_wrapper handler expects the following functions to be provided by the SOC level: __soc_is_irq: to check if the exception is the result of an interrupt or not. __soc_handle_irq: handle pending IRQ at SOC level (ex: clear pending IRQ in SOC-specific IRQ register) 3) Thread/task scheduling, as well as IRQ offloading are handled via the RISC-V system call ("ecall"), which is also handled via the __irq_wrapper handler. The _Swap asm function just calls "ecall" to generate an exception. 4) As there is no conventional way of handling CPU power save in RISC-V, the default nano_cpu_idle and nano_cpu_atomic_idle functions just unlock interrupts and return to the caller, without issuing any CPU power saving instruction. Nonetheless, to allow SOC-level to implement proper CPU power save, nano_cpu_idle and nano_cpu_atomic_idle functions are defined as __weak at the architecture level. Change-Id: I980a161d0009f3f404ad22b226a6229fbb492389 Signed-off-by: Jean-Paul Etienne <fractalclone@gmail.com>
2017-01-11 00:24:30 +01:00
/*
* Save thread stack pointer on interrupt stack
* In RISC-V, stack pointer needs to be 16-byte aligned
*/
addi sp, sp, -16
riscv: better abstraction for register-wide load/store opcodes Those are prominent enough that having RV_OP_LOADREG and RV_OP_STOREREG shouting at you all over the place is rather unpleasant and bad taste. Let's create pseudo-instructions of our own with assembler macros rather than preprocessor defines and only in assembly scope. This makes the asm code way more uniform and readable. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-22 14:09:55 -05:00
sr t0, 0(sp)
arch: added support for the riscv32 architecture RISC-V is an open-source instruction set architecture. Added support for the 32bit version of RISC-V to Zephyr. 1) exceptions/interrupts/faults are handled at the architecture level via the __irq_wrapper handler. Context saving/restoring of registers can be handled at both architecture and SOC levels. If SOC-specific registers need to be saved, SOC level needs to provide __soc_save_context and __soc_restore_context functions that shall be accounted by the architecture level, when corresponding config variable RISCV_SOC_CONTEXT_SAVE is set. 2) As RISC-V architecture does not provide a clear ISA specification about interrupt handling, each RISC-V SOC handles it in its own way. Hence, at the architecture level, the __irq_wrapper handler expects the following functions to be provided by the SOC level: __soc_is_irq: to check if the exception is the result of an interrupt or not. __soc_handle_irq: handle pending IRQ at SOC level (ex: clear pending IRQ in SOC-specific IRQ register) 3) Thread/task scheduling, as well as IRQ offloading are handled via the RISC-V system call ("ecall"), which is also handled via the __irq_wrapper handler. The _Swap asm function just calls "ecall" to generate an exception. 4) As there is no conventional way of handling CPU power save in RISC-V, the default nano_cpu_idle and nano_cpu_atomic_idle functions just unlock interrupts and return to the caller, without issuing any CPU power saving instruction. Nonetheless, to allow SOC-level to implement proper CPU power save, nano_cpu_idle and nano_cpu_atomic_idle functions are defined as __weak at the architecture level. Change-Id: I980a161d0009f3f404ad22b226a6229fbb492389 Signed-off-by: Jean-Paul Etienne <fractalclone@gmail.com>
2017-01-11 00:24:30 +01:00
on_irq_stack:
tracing: add TRACING_ISR Kconfig Add TRACING_ISR Kconfig to help high latency backend working well. Currently the ISR tracing hook function is put at the begining and ending of ISR wrapper, when there is ISR needed in the tracing path (especially tracing backend), it will cause tracing buffer easily be exhausted if async tracing method enabled. Also it will increase system latency if all the ISRs are traced. So add TRACING_ISR to enable/disable ISR tracing here. Later a filter out mechanism based on irq number will be added. Signed-off-by: Wentong Wu <wentong.wu@intel.com>
2020-01-28 16:44:59 +08:00
#ifdef CONFIG_TRACING_ISR
debug: tracing: Remove unneeded abstraction Various C and Assembly modules make function calls to z_sys_trace_*. These merely call corresponding functions sys_trace_*. This commit is to simplify these by making direct function calls to the sys_trace_* functions from these modules. Subsequently, the z_sys_trace_* functions are removed. Signed-off-by: Mrinal Sen <msen@oticon.com>
2019-09-19 09:25:19 +02:00
call sys_trace_isr_enter
arch: added support for the riscv32 architecture RISC-V is an open-source instruction set architecture. Added support for the 32bit version of RISC-V to Zephyr. 1) exceptions/interrupts/faults are handled at the architecture level via the __irq_wrapper handler. Context saving/restoring of registers can be handled at both architecture and SOC levels. If SOC-specific registers need to be saved, SOC level needs to provide __soc_save_context and __soc_restore_context functions that shall be accounted by the architecture level, when corresponding config variable RISCV_SOC_CONTEXT_SAVE is set. 2) As RISC-V architecture does not provide a clear ISA specification about interrupt handling, each RISC-V SOC handles it in its own way. Hence, at the architecture level, the __irq_wrapper handler expects the following functions to be provided by the SOC level: __soc_is_irq: to check if the exception is the result of an interrupt or not. __soc_handle_irq: handle pending IRQ at SOC level (ex: clear pending IRQ in SOC-specific IRQ register) 3) Thread/task scheduling, as well as IRQ offloading are handled via the RISC-V system call ("ecall"), which is also handled via the __irq_wrapper handler. The _Swap asm function just calls "ecall" to generate an exception. 4) As there is no conventional way of handling CPU power save in RISC-V, the default nano_cpu_idle and nano_cpu_atomic_idle functions just unlock interrupts and return to the caller, without issuing any CPU power saving instruction. Nonetheless, to allow SOC-level to implement proper CPU power save, nano_cpu_idle and nano_cpu_atomic_idle functions are defined as __weak at the architecture level. Change-Id: I980a161d0009f3f404ad22b226a6229fbb492389 Signed-off-by: Jean-Paul Etienne <fractalclone@gmail.com>
2017-01-11 00:24:30 +01:00
#endif
/* Get IRQ causing interrupt */
csrr a0, mcause
riscv32: move riscv privileged architecture specifics within a common header file Added a riscv-privilege.h header file that contains common definitions for all riscv SOCs supporting the riscv privileged architecture specification. This shall ease addition of future riscv SOCs supporting the riscv privileged architecture spec. Change-Id: I5714bf70eeda738a25967ed26d3d0d2aaa0c9989 Signed-off-by: Jean-Paul Etienne <fractalclone@gmail.com>
2017-02-13 01:17:01 +01:00
li t0, SOC_MCAUSE_EXP_MASK
arch: added support for the riscv32 architecture RISC-V is an open-source instruction set architecture. Added support for the 32bit version of RISC-V to Zephyr. 1) exceptions/interrupts/faults are handled at the architecture level via the __irq_wrapper handler. Context saving/restoring of registers can be handled at both architecture and SOC levels. If SOC-specific registers need to be saved, SOC level needs to provide __soc_save_context and __soc_restore_context functions that shall be accounted by the architecture level, when corresponding config variable RISCV_SOC_CONTEXT_SAVE is set. 2) As RISC-V architecture does not provide a clear ISA specification about interrupt handling, each RISC-V SOC handles it in its own way. Hence, at the architecture level, the __irq_wrapper handler expects the following functions to be provided by the SOC level: __soc_is_irq: to check if the exception is the result of an interrupt or not. __soc_handle_irq: handle pending IRQ at SOC level (ex: clear pending IRQ in SOC-specific IRQ register) 3) Thread/task scheduling, as well as IRQ offloading are handled via the RISC-V system call ("ecall"), which is also handled via the __irq_wrapper handler. The _Swap asm function just calls "ecall" to generate an exception. 4) As there is no conventional way of handling CPU power save in RISC-V, the default nano_cpu_idle and nano_cpu_atomic_idle functions just unlock interrupts and return to the caller, without issuing any CPU power saving instruction. Nonetheless, to allow SOC-level to implement proper CPU power save, nano_cpu_idle and nano_cpu_atomic_idle functions are defined as __weak at the architecture level. Change-Id: I980a161d0009f3f404ad22b226a6229fbb492389 Signed-off-by: Jean-Paul Etienne <fractalclone@gmail.com>
2017-01-11 00:24:30 +01:00
and a0, a0, t0
/*
* Clear pending IRQ generating the interrupt at SOC level
* Pass IRQ number to __soc_handle_irq via register a0
*/
jal ra, __soc_handle_irq
/*
* Call corresponding registered function in _sw_isr_table.
riscv: make core code 64-bit compatible There are two aspects to this: CPU registers are twice as big, and the load and store instructions must use the 'd' suffix instead of the 'w' one. To abstract register differences, we simply use a ulong_t instead of u32_t given that RISC-V is either ILP32 or LP64. And the relevant lw/sw instructions are replaced by LR/SR (load/store register) that get defined as either lw/sw or ld/sd. Finally a few constants to deal with register offsets are also provided. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2019-07-24 16:21:58 -04:00
* (table is 2-word wide, we should shift index accordingly)
arch: added support for the riscv32 architecture RISC-V is an open-source instruction set architecture. Added support for the 32bit version of RISC-V to Zephyr. 1) exceptions/interrupts/faults are handled at the architecture level via the __irq_wrapper handler. Context saving/restoring of registers can be handled at both architecture and SOC levels. If SOC-specific registers need to be saved, SOC level needs to provide __soc_save_context and __soc_restore_context functions that shall be accounted by the architecture level, when corresponding config variable RISCV_SOC_CONTEXT_SAVE is set. 2) As RISC-V architecture does not provide a clear ISA specification about interrupt handling, each RISC-V SOC handles it in its own way. Hence, at the architecture level, the __irq_wrapper handler expects the following functions to be provided by the SOC level: __soc_is_irq: to check if the exception is the result of an interrupt or not. __soc_handle_irq: handle pending IRQ at SOC level (ex: clear pending IRQ in SOC-specific IRQ register) 3) Thread/task scheduling, as well as IRQ offloading are handled via the RISC-V system call ("ecall"), which is also handled via the __irq_wrapper handler. The _Swap asm function just calls "ecall" to generate an exception. 4) As there is no conventional way of handling CPU power save in RISC-V, the default nano_cpu_idle and nano_cpu_atomic_idle functions just unlock interrupts and return to the caller, without issuing any CPU power saving instruction. Nonetheless, to allow SOC-level to implement proper CPU power save, nano_cpu_idle and nano_cpu_atomic_idle functions are defined as __weak at the architecture level. Change-Id: I980a161d0009f3f404ad22b226a6229fbb492389 Signed-off-by: Jean-Paul Etienne <fractalclone@gmail.com>
2017-01-11 00:24:30 +01:00
*/
la t0, _sw_isr_table
riscv: make core code 64-bit compatible There are two aspects to this: CPU registers are twice as big, and the load and store instructions must use the 'd' suffix instead of the 'w' one. To abstract register differences, we simply use a ulong_t instead of u32_t given that RISC-V is either ILP32 or LP64. And the relevant lw/sw instructions are replaced by LR/SR (load/store register) that get defined as either lw/sw or ld/sd. Finally a few constants to deal with register offsets are also provided. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2019-07-24 16:21:58 -04:00
slli a0, a0, (RV_REGSHIFT + 1)
arch: added support for the riscv32 architecture RISC-V is an open-source instruction set architecture. Added support for the 32bit version of RISC-V to Zephyr. 1) exceptions/interrupts/faults are handled at the architecture level via the __irq_wrapper handler. Context saving/restoring of registers can be handled at both architecture and SOC levels. If SOC-specific registers need to be saved, SOC level needs to provide __soc_save_context and __soc_restore_context functions that shall be accounted by the architecture level, when corresponding config variable RISCV_SOC_CONTEXT_SAVE is set. 2) As RISC-V architecture does not provide a clear ISA specification about interrupt handling, each RISC-V SOC handles it in its own way. Hence, at the architecture level, the __irq_wrapper handler expects the following functions to be provided by the SOC level: __soc_is_irq: to check if the exception is the result of an interrupt or not. __soc_handle_irq: handle pending IRQ at SOC level (ex: clear pending IRQ in SOC-specific IRQ register) 3) Thread/task scheduling, as well as IRQ offloading are handled via the RISC-V system call ("ecall"), which is also handled via the __irq_wrapper handler. The _Swap asm function just calls "ecall" to generate an exception. 4) As there is no conventional way of handling CPU power save in RISC-V, the default nano_cpu_idle and nano_cpu_atomic_idle functions just unlock interrupts and return to the caller, without issuing any CPU power saving instruction. Nonetheless, to allow SOC-level to implement proper CPU power save, nano_cpu_idle and nano_cpu_atomic_idle functions are defined as __weak at the architecture level. Change-Id: I980a161d0009f3f404ad22b226a6229fbb492389 Signed-off-by: Jean-Paul Etienne <fractalclone@gmail.com>
2017-01-11 00:24:30 +01:00
add t0, t0, a0
/* Load argument in a0 register */
riscv: better abstraction for register-wide load/store opcodes Those are prominent enough that having RV_OP_LOADREG and RV_OP_STOREREG shouting at you all over the place is rather unpleasant and bad taste. Let's create pseudo-instructions of our own with assembler macros rather than preprocessor defines and only in assembly scope. This makes the asm code way more uniform and readable. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-22 14:09:55 -05:00
lr a0, 0(t0)
arch: added support for the riscv32 architecture RISC-V is an open-source instruction set architecture. Added support for the 32bit version of RISC-V to Zephyr. 1) exceptions/interrupts/faults are handled at the architecture level via the __irq_wrapper handler. Context saving/restoring of registers can be handled at both architecture and SOC levels. If SOC-specific registers need to be saved, SOC level needs to provide __soc_save_context and __soc_restore_context functions that shall be accounted by the architecture level, when corresponding config variable RISCV_SOC_CONTEXT_SAVE is set. 2) As RISC-V architecture does not provide a clear ISA specification about interrupt handling, each RISC-V SOC handles it in its own way. Hence, at the architecture level, the __irq_wrapper handler expects the following functions to be provided by the SOC level: __soc_is_irq: to check if the exception is the result of an interrupt or not. __soc_handle_irq: handle pending IRQ at SOC level (ex: clear pending IRQ in SOC-specific IRQ register) 3) Thread/task scheduling, as well as IRQ offloading are handled via the RISC-V system call ("ecall"), which is also handled via the __irq_wrapper handler. The _Swap asm function just calls "ecall" to generate an exception. 4) As there is no conventional way of handling CPU power save in RISC-V, the default nano_cpu_idle and nano_cpu_atomic_idle functions just unlock interrupts and return to the caller, without issuing any CPU power saving instruction. Nonetheless, to allow SOC-level to implement proper CPU power save, nano_cpu_idle and nano_cpu_atomic_idle functions are defined as __weak at the architecture level. Change-Id: I980a161d0009f3f404ad22b226a6229fbb492389 Signed-off-by: Jean-Paul Etienne <fractalclone@gmail.com>
2017-01-11 00:24:30 +01:00
/* Load ISR function address in register t1 */
riscv: better abstraction for register-wide load/store opcodes Those are prominent enough that having RV_OP_LOADREG and RV_OP_STOREREG shouting at you all over the place is rather unpleasant and bad taste. Let's create pseudo-instructions of our own with assembler macros rather than preprocessor defines and only in assembly scope. This makes the asm code way more uniform and readable. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-22 14:09:55 -05:00
lr t1, RV_REGSIZE(t0)
arch: added support for the riscv32 architecture RISC-V is an open-source instruction set architecture. Added support for the 32bit version of RISC-V to Zephyr. 1) exceptions/interrupts/faults are handled at the architecture level via the __irq_wrapper handler. Context saving/restoring of registers can be handled at both architecture and SOC levels. If SOC-specific registers need to be saved, SOC level needs to provide __soc_save_context and __soc_restore_context functions that shall be accounted by the architecture level, when corresponding config variable RISCV_SOC_CONTEXT_SAVE is set. 2) As RISC-V architecture does not provide a clear ISA specification about interrupt handling, each RISC-V SOC handles it in its own way. Hence, at the architecture level, the __irq_wrapper handler expects the following functions to be provided by the SOC level: __soc_is_irq: to check if the exception is the result of an interrupt or not. __soc_handle_irq: handle pending IRQ at SOC level (ex: clear pending IRQ in SOC-specific IRQ register) 3) Thread/task scheduling, as well as IRQ offloading are handled via the RISC-V system call ("ecall"), which is also handled via the __irq_wrapper handler. The _Swap asm function just calls "ecall" to generate an exception. 4) As there is no conventional way of handling CPU power save in RISC-V, the default nano_cpu_idle and nano_cpu_atomic_idle functions just unlock interrupts and return to the caller, without issuing any CPU power saving instruction. Nonetheless, to allow SOC-level to implement proper CPU power save, nano_cpu_idle and nano_cpu_atomic_idle functions are defined as __weak at the architecture level. Change-Id: I980a161d0009f3f404ad22b226a6229fbb492389 Signed-off-by: Jean-Paul Etienne <fractalclone@gmail.com>
2017-01-11 00:24:30 +01:00
/* Call ISR function */
riscv: fix non-standard assembly of RISC-V Non-standard `jalr rd, rs` pseudo-instructions are used. This commit changes them to `ret` for standard return pseudo-instruction or `jalr rd, rs, 0` for no offset jump register and link. Fixes #41100. Signed-off-by: Henry Hsieh <r901042004@yahoo.com.tw>
2021-12-14 22:46:42 +08:00
jalr ra, t1, 0
arch: added support for the riscv32 architecture RISC-V is an open-source instruction set architecture. Added support for the 32bit version of RISC-V to Zephyr. 1) exceptions/interrupts/faults are handled at the architecture level via the __irq_wrapper handler. Context saving/restoring of registers can be handled at both architecture and SOC levels. If SOC-specific registers need to be saved, SOC level needs to provide __soc_save_context and __soc_restore_context functions that shall be accounted by the architecture level, when corresponding config variable RISCV_SOC_CONTEXT_SAVE is set. 2) As RISC-V architecture does not provide a clear ISA specification about interrupt handling, each RISC-V SOC handles it in its own way. Hence, at the architecture level, the __irq_wrapper handler expects the following functions to be provided by the SOC level: __soc_is_irq: to check if the exception is the result of an interrupt or not. __soc_handle_irq: handle pending IRQ at SOC level (ex: clear pending IRQ in SOC-specific IRQ register) 3) Thread/task scheduling, as well as IRQ offloading are handled via the RISC-V system call ("ecall"), which is also handled via the __irq_wrapper handler. The _Swap asm function just calls "ecall" to generate an exception. 4) As there is no conventional way of handling CPU power save in RISC-V, the default nano_cpu_idle and nano_cpu_atomic_idle functions just unlock interrupts and return to the caller, without issuing any CPU power saving instruction. Nonetheless, to allow SOC-level to implement proper CPU power save, nano_cpu_idle and nano_cpu_atomic_idle functions are defined as __weak at the architecture level. Change-Id: I980a161d0009f3f404ad22b226a6229fbb492389 Signed-off-by: Jean-Paul Etienne <fractalclone@gmail.com>
2017-01-11 00:24:30 +01:00
riscv: irq_offload: simpler implementation Get rid of all those global variables and IRQ locking. Use the regular IRQ exit path to let tests validate preemption properly. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-07 13:24:38 -05:00
irq_done:
riscv: isr.S: dedicate a register to &current_cpu Stop using &_kernel as this is not SMP friendly. Let's use s0 (after preserving its content) to hold &current_cpu instead so it won't have to be reloaded each time it is needed. This will be even more relevant when SMP support is added. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-09 17:21:27 -05:00
/* Decrement _current_cpu->nested */
lw t2, ___cpu_t_nested_OFFSET(s0)
arch: added support for the riscv32 architecture RISC-V is an open-source instruction set architecture. Added support for the 32bit version of RISC-V to Zephyr. 1) exceptions/interrupts/faults are handled at the architecture level via the __irq_wrapper handler. Context saving/restoring of registers can be handled at both architecture and SOC levels. If SOC-specific registers need to be saved, SOC level needs to provide __soc_save_context and __soc_restore_context functions that shall be accounted by the architecture level, when corresponding config variable RISCV_SOC_CONTEXT_SAVE is set. 2) As RISC-V architecture does not provide a clear ISA specification about interrupt handling, each RISC-V SOC handles it in its own way. Hence, at the architecture level, the __irq_wrapper handler expects the following functions to be provided by the SOC level: __soc_is_irq: to check if the exception is the result of an interrupt or not. __soc_handle_irq: handle pending IRQ at SOC level (ex: clear pending IRQ in SOC-specific IRQ register) 3) Thread/task scheduling, as well as IRQ offloading are handled via the RISC-V system call ("ecall"), which is also handled via the __irq_wrapper handler. The _Swap asm function just calls "ecall" to generate an exception. 4) As there is no conventional way of handling CPU power save in RISC-V, the default nano_cpu_idle and nano_cpu_atomic_idle functions just unlock interrupts and return to the caller, without issuing any CPU power saving instruction. Nonetheless, to allow SOC-level to implement proper CPU power save, nano_cpu_idle and nano_cpu_atomic_idle functions are defined as __weak at the architecture level. Change-Id: I980a161d0009f3f404ad22b226a6229fbb492389 Signed-off-by: Jean-Paul Etienne <fractalclone@gmail.com>
2017-01-11 00:24:30 +01:00
addi t2, t2, -1
riscv: isr.S: dedicate a register to &current_cpu Stop using &_kernel as this is not SMP friendly. Let's use s0 (after preserving its content) to hold &current_cpu instead so it won't have to be reloaded each time it is needed. This will be even more relevant when SMP support is added. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-09 17:21:27 -05:00
sw t2, ___cpu_t_nested_OFFSET(s0)
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
bnez t2, no_reschedule
arch: added support for the riscv32 architecture RISC-V is an open-source instruction set architecture. Added support for the 32bit version of RISC-V to Zephyr. 1) exceptions/interrupts/faults are handled at the architecture level via the __irq_wrapper handler. Context saving/restoring of registers can be handled at both architecture and SOC levels. If SOC-specific registers need to be saved, SOC level needs to provide __soc_save_context and __soc_restore_context functions that shall be accounted by the architecture level, when corresponding config variable RISCV_SOC_CONTEXT_SAVE is set. 2) As RISC-V architecture does not provide a clear ISA specification about interrupt handling, each RISC-V SOC handles it in its own way. Hence, at the architecture level, the __irq_wrapper handler expects the following functions to be provided by the SOC level: __soc_is_irq: to check if the exception is the result of an interrupt or not. __soc_handle_irq: handle pending IRQ at SOC level (ex: clear pending IRQ in SOC-specific IRQ register) 3) Thread/task scheduling, as well as IRQ offloading are handled via the RISC-V system call ("ecall"), which is also handled via the __irq_wrapper handler. The _Swap asm function just calls "ecall" to generate an exception. 4) As there is no conventional way of handling CPU power save in RISC-V, the default nano_cpu_idle and nano_cpu_atomic_idle functions just unlock interrupts and return to the caller, without issuing any CPU power saving instruction. Nonetheless, to allow SOC-level to implement proper CPU power save, nano_cpu_idle and nano_cpu_atomic_idle functions are defined as __weak at the architecture level. Change-Id: I980a161d0009f3f404ad22b226a6229fbb492389 Signed-off-by: Jean-Paul Etienne <fractalclone@gmail.com>
2017-01-11 00:24:30 +01:00
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
/* nested count is back to 0: Return to thread stack */
lr sp, 0(sp)
stack_sentinel: change cooperative check One of the stack sentinel policies was to check the sentinel any time a cooperative context switch is done (i.e, _Swap is called). This was done by adding a hook to _check_stack_sentinel in every arch's __swap function. This way is cleaner as we just have the hook in one inline function rather than implemented in several different assembly dialects. The check upon interrupt is now made unconditionally rather than checking if we are calling __swap, since the check now is only called on cooperative _Swap(). The interrupt is always serviced first. Issue: ZEP-2244 Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2017-06-07 09:33:16 -07:00
debug: add stack sentinel feature This places a sentinel value at the lowest 4 bytes of a stack memory region and checks it at various intervals, including when servicing interrupts or context switching. This is implemented on all arches except ARC, which supports stack bounds checking directly in hardware. Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2017-05-11 13:29:15 -07:00
#ifdef CONFIG_STACK_SENTINEL
all: Update reserved function names Update reserved function names starting with one underscore, replacing them as follows: '_k_' with 'z_' '_K_' with 'Z_' '_handler_' with 'z_handl_' '_Cstart' with 'z_cstart' '_Swap' with 'z_swap' This renaming is done on both global and those static function names in kernel/include and include/. Other static function names in kernel/ are renamed by removing the leading underscore. Other function names not starting with any prefix listed above are renamed starting with a 'z_' or 'Z_' prefix. Function names starting with two or three leading underscores are not automatcally renamed since these names will collide with the variants with two or three leading underscores. Various generator scripts have also been updated as well as perf, linker and usb files. These are drivers/serial/uart_handlers.c include/linker/kobject-text.ld kernel/include/syscall_handler.h scripts/gen_kobject_list.py scripts/gen_syscall_header.py Signed-off-by: Patrik Flykt <patrik.flykt@intel.com>
2019-03-08 14:19:05 -07:00
call z_check_stack_sentinel
debug: add stack sentinel feature This places a sentinel value at the lowest 4 bytes of a stack memory region and checks it at various intervals, including when servicing interrupts or context switching. This is implemented on all arches except ARC, which supports stack bounds checking directly in hardware. Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2017-05-11 13:29:15 -07:00
#endif
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
reschedule:
Revert "arch/riscv: Use arch_switch() for context swap" This reverts commit be28de692c5e2c5546841c49d6a85d5f046474a4. The purpose of this commit will be reintroduced later on top of a cleaner codebase. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-09 17:46:30 -05:00
riscv: isr.S: dedicate a register to &current_cpu Stop using &_kernel as this is not SMP friendly. Let's use s0 (after preserving its content) to hold &current_cpu instead so it won't have to be reloaded each time it is needed. This will be even more relevant when SMP support is added. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-09 17:21:27 -05:00
/* Get pointer to current thread on this CPU */
lr a1, ___cpu_t_current_OFFSET(s0)
arch: riscv: add memory protection support The IRQ handler has had a major changes to manage syscall, reschedule and interrupt from user thread and stack guard. Add userspace support: - Use a global variable to know if the current execution is user or machine. The location of this variable is read only for all user thread and read/write for kernel thread. - Memory shared is supported. - Use dynamic allocation to optimize PMP slot usage. If the area size is a power of 2, only one PMP slot is used, else 2 are used. Add stack guard support: - Use MPRV bit to force PMP rules to machine mode execution. - IRQ stack have a locked stack guard to avoid re-write PMP configuration registers for each interruption and then win some cycle. - The IRQ stack is used as "temporary" stack at the beginning of IRQ handler to save current ESF. That avoid to trigger write fault on thread stack during store ESF which that call IRQ handler to infinity. - A stack guard is also setup for privileged stack of a user thread. Thread: - A PMP setup is specific to each thread. PMP setup are saved in each thread structure to improve reschedule performance. Signed-off-by: Alexandre Mergnat <amergnat@baylibre.com> Reviewed-by: Nicolas Royer <nroyer@baylibre.com>
2020-07-21 16:00:39 +02:00
arch: added support for the riscv32 architecture RISC-V is an open-source instruction set architecture. Added support for the 32bit version of RISC-V to Zephyr. 1) exceptions/interrupts/faults are handled at the architecture level via the __irq_wrapper handler. Context saving/restoring of registers can be handled at both architecture and SOC levels. If SOC-specific registers need to be saved, SOC level needs to provide __soc_save_context and __soc_restore_context functions that shall be accounted by the architecture level, when corresponding config variable RISCV_SOC_CONTEXT_SAVE is set. 2) As RISC-V architecture does not provide a clear ISA specification about interrupt handling, each RISC-V SOC handles it in its own way. Hence, at the architecture level, the __irq_wrapper handler expects the following functions to be provided by the SOC level: __soc_is_irq: to check if the exception is the result of an interrupt or not. __soc_handle_irq: handle pending IRQ at SOC level (ex: clear pending IRQ in SOC-specific IRQ register) 3) Thread/task scheduling, as well as IRQ offloading are handled via the RISC-V system call ("ecall"), which is also handled via the __irq_wrapper handler. The _Swap asm function just calls "ecall" to generate an exception. 4) As there is no conventional way of handling CPU power save in RISC-V, the default nano_cpu_idle and nano_cpu_atomic_idle functions just unlock interrupts and return to the caller, without issuing any CPU power saving instruction. Nonetheless, to allow SOC-level to implement proper CPU power save, nano_cpu_idle and nano_cpu_atomic_idle functions are defined as __weak at the architecture level. Change-Id: I980a161d0009f3f404ad22b226a6229fbb492389 Signed-off-by: Jean-Paul Etienne <fractalclone@gmail.com>
2017-01-11 00:24:30 +01:00
/*
riscv: implement arch_switch() The move to arch_switch() is a prerequisite for SMP support. Make it optimal without the need for an ECALL roundtrip on every context switch. Performance numbers from tests/benchmarks/sched: Before: unpend 107 ready 102 switch 188 pend 218 tot 615 (avg 615) After: unpend 107 ready 102 switch 170 pend 217 tot 596 (avg 595) Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-07 17:01:36 -05:00
* Get next thread to schedule with z_get_next_switch_handle().
* We pass it a NULL as we didn't save the whole thread context yet.
* If no scheduling is necessary then NULL will be returned.
arch: added support for the riscv32 architecture RISC-V is an open-source instruction set architecture. Added support for the 32bit version of RISC-V to Zephyr. 1) exceptions/interrupts/faults are handled at the architecture level via the __irq_wrapper handler. Context saving/restoring of registers can be handled at both architecture and SOC levels. If SOC-specific registers need to be saved, SOC level needs to provide __soc_save_context and __soc_restore_context functions that shall be accounted by the architecture level, when corresponding config variable RISCV_SOC_CONTEXT_SAVE is set. 2) As RISC-V architecture does not provide a clear ISA specification about interrupt handling, each RISC-V SOC handles it in its own way. Hence, at the architecture level, the __irq_wrapper handler expects the following functions to be provided by the SOC level: __soc_is_irq: to check if the exception is the result of an interrupt or not. __soc_handle_irq: handle pending IRQ at SOC level (ex: clear pending IRQ in SOC-specific IRQ register) 3) Thread/task scheduling, as well as IRQ offloading are handled via the RISC-V system call ("ecall"), which is also handled via the __irq_wrapper handler. The _Swap asm function just calls "ecall" to generate an exception. 4) As there is no conventional way of handling CPU power save in RISC-V, the default nano_cpu_idle and nano_cpu_atomic_idle functions just unlock interrupts and return to the caller, without issuing any CPU power saving instruction. Nonetheless, to allow SOC-level to implement proper CPU power save, nano_cpu_idle and nano_cpu_atomic_idle functions are defined as __weak at the architecture level. Change-Id: I980a161d0009f3f404ad22b226a6229fbb492389 Signed-off-by: Jean-Paul Etienne <fractalclone@gmail.com>
2017-01-11 00:24:30 +01:00
*/
riscv: implement arch_switch() The move to arch_switch() is a prerequisite for SMP support. Make it optimal without the need for an ECALL roundtrip on every context switch. Performance numbers from tests/benchmarks/sched: Before: unpend 107 ready 102 switch 188 pend 218 tot 615 (avg 615) After: unpend 107 ready 102 switch 170 pend 217 tot 596 (avg 595) Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-07 17:01:36 -05:00
addi sp, sp, -16
sr a1, 0(sp)
mv a0, zero
call z_get_next_switch_handle
lr a1, 0(sp)
addi sp, sp, 16
beqz a0, no_reschedule
riscv: Add support for floating point This change adds full shared floating point support for the RISCV architecture with minimal impact on threads with floating point support not enabled. Signed-off-by: Corey Wharton <coreyw7@fb.com>
2020-03-11 18:15:29 -07:00
/*
riscv: implement arch_switch() The move to arch_switch() is a prerequisite for SMP support. Make it optimal without the need for an ECALL roundtrip on every context switch. Performance numbers from tests/benchmarks/sched: Before: unpend 107 ready 102 switch 188 pend 218 tot 615 (avg 615) After: unpend 107 ready 102 switch 170 pend 217 tot 596 (avg 595) Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-07 17:01:36 -05:00
* Perform context switch:
* a0 = new thread
* a1 = old thread
riscv: Add support for floating point This change adds full shared floating point support for the RISCV architecture with minimal impact on threads with floating point support not enabled. Signed-off-by: Corey Wharton <coreyw7@fb.com>
2020-03-11 18:15:29 -07:00
*/
riscv: implement arch_switch() The move to arch_switch() is a prerequisite for SMP support. Make it optimal without the need for an ECALL roundtrip on every context switch. Performance numbers from tests/benchmarks/sched: Before: unpend 107 ready 102 switch 188 pend 218 tot 615 (avg 615) After: unpend 107 ready 102 switch 170 pend 217 tot 596 (avg 595) Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-07 17:01:36 -05:00
call z_riscv_switch
tests: benchmarks: timing_info: Enable benchmarks for riscv32. This patch provides support needed to get timing related information from riscv32 based SOC. Signed-off-by: Adithya Baglody <adithya.nagaraj.baglody@intel.com>
2018-07-23 15:54:22 +05:30
riscv: implement arch_switch() The move to arch_switch() is a prerequisite for SMP support. Make it optimal without the need for an ECALL roundtrip on every context switch. Performance numbers from tests/benchmarks/sched: Before: unpend 107 ready 102 switch 188 pend 218 tot 615 (avg 615) After: unpend 107 ready 102 switch 170 pend 217 tot 596 (avg 595) Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-07 17:01:36 -05:00
z_riscv_thread_start:
riscv: isr.S: dedicate a register to &current_cpu Stop using &_kernel as this is not SMP friendly. Let's use s0 (after preserving its content) to hold &current_cpu instead so it won't have to be reloaded each time it is needed. This will be even more relevant when SMP support is added. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-09 17:21:27 -05:00
might_have_rescheduled:
riscv: isr.S: compute _current_cpu using CPU number on SMP To do so efficiently on systems without the mul instruction, we use shifts and adds which is faster and sometimes smaller than a plain loop. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-09 23:47:38 -05:00
#ifdef CONFIG_SMP
/* reload s0 with &_current_cpu as it might have changed */
GET_CURRENT_CPU(s0, t0)
#endif
riscv: isr.S: dedicate a register to &current_cpu Stop using &_kernel as this is not SMP friendly. Let's use s0 (after preserving its content) to hold &current_cpu instead so it won't have to be reloaded each time it is needed. This will be even more relevant when SMP support is added. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-09 17:21:27 -05:00
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
no_reschedule:
arch: riscv: add memory protection support The IRQ handler has had a major changes to manage syscall, reschedule and interrupt from user thread and stack guard. Add userspace support: - Use a global variable to know if the current execution is user or machine. The location of this variable is read only for all user thread and read/write for kernel thread. - Memory shared is supported. - Use dynamic allocation to optimize PMP slot usage. If the area size is a power of 2, only one PMP slot is used, else 2 are used. Add stack guard support: - Use MPRV bit to force PMP rules to machine mode execution. - IRQ stack have a locked stack guard to avoid re-write PMP configuration registers for each interruption and then win some cycle. - The IRQ stack is used as "temporary" stack at the beginning of IRQ handler to save current ESF. That avoid to trigger write fault on thread stack during store ESF which that call IRQ handler to infinity. - A stack guard is also setup for privileged stack of a user thread. Thread: - A PMP setup is specific to each thread. PMP setup are saved in each thread structure to improve reschedule performance. Signed-off-by: Alexandre Mergnat <amergnat@baylibre.com> Reviewed-by: Nicolas Royer <nroyer@baylibre.com>
2020-07-21 16:00:39 +02:00
#ifdef CONFIG_RISCV_SOC_CONTEXT_SAVE
/* Restore context at SOC level */
addi a0, sp, __z_arch_esf_t_soc_context_OFFSET
jal ra, __soc_restore_context
#endif /* CONFIG_RISCV_SOC_CONTEXT_SAVE */
riscv32: fixed context restore upon exiting ISR By now, t0 register restored value is overwritten by mepc and mstatus values prior to returning from ISR. Fixed by restoring mstatus and mepc registers before restoring the caller-saved registers. As t0 is a temporary register within the riscv ABI, this issue was unnoticed for most applications, except for computation intensive apps, like crypto tests. Signed-off-by: Jean-Paul Etienne <fractalclone@gmail.com>
2017-06-29 22:45:55 +02:00
/* Restore MEPC register */
riscv: better abstraction for register-wide load/store opcodes Those are prominent enough that having RV_OP_LOADREG and RV_OP_STOREREG shouting at you all over the place is rather unpleasant and bad taste. Let's create pseudo-instructions of our own with assembler macros rather than preprocessor defines and only in assembly scope. This makes the asm code way more uniform and readable. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-22 14:09:55 -05:00
lr t0, __z_arch_esf_t_mepc_OFFSET(sp)
riscv32: fixed context restore upon exiting ISR By now, t0 register restored value is overwritten by mepc and mstatus values prior to returning from ISR. Fixed by restoring mstatus and mepc registers before restoring the caller-saved registers. As t0 is a temporary register within the riscv ABI, this issue was unnoticed for most applications, except for computation intensive apps, like crypto tests. Signed-off-by: Jean-Paul Etienne <fractalclone@gmail.com>
2017-06-29 22:45:55 +02:00
csrw mepc, t0
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
/* Restore MSTATUS register */
lr t4, __z_arch_esf_t_mstatus_OFFSET(sp)
riscv: isr.S: optimize FP regs save/restore decision Rely on mstatus rather than thread->base.user_options since it is always up to date (updated by z_riscv_switch) to simplify the code and be SMP proof. Also carry over SF_INIT to the mstatus being restored in case it was changed in the mean time. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-07 17:31:13 -05:00
csrrw t5, mstatus, t4
#if defined(CONFIG_FPU) && defined(CONFIG_FPU_SHARING)
/*
* Determine if we need to restore FP regs based on the previous
* (before the csr above) mstatus value available in t5.
*/
li t1, MSTATUS_FS_INIT
and t0, t5, t1
beqz t0, no_fp
/* make sure FP is enabled in the restored mstatus */
csrs mstatus, t1
DO_FP_CALLER_SAVED(flr, sp)
j 1f
no_fp: /* make sure this is reflected in the restored mstatus */
csrc mstatus, t1
1:
#endif /* CONFIG_FPU && CONFIG_FPU_SHARING */
riscv32: fixed context restore upon exiting ISR By now, t0 register restored value is overwritten by mepc and mstatus values prior to returning from ISR. Fixed by restoring mstatus and mepc registers before restoring the caller-saved registers. As t0 is a temporary register within the riscv ABI, this issue was unnoticed for most applications, except for computation intensive apps, like crypto tests. Signed-off-by: Jean-Paul Etienne <fractalclone@gmail.com>
2017-06-29 22:45:55 +02:00
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
#ifdef CONFIG_USERSPACE
riscv: Restore floating-point caller saved registers before integer This fixes an issue where the t0 register is overwritten after it has been restored. Signed-off-by: Corey Wharton <coreyw7@fb.com>
2020-04-14 14:56:12 -07:00
/*
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
* Check if we are returning to user mode. If so then we must
* set is_user_mode to true and load the scratch register with
* the stack pointer to be used with the next exception to come.
riscv: Restore floating-point caller saved registers before integer This fixes an issue where the t0 register is overwritten after it has been restored. Signed-off-by: Corey Wharton <coreyw7@fb.com>
2020-04-14 14:56:12 -07:00
*/
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
li t1, MSTATUS_MPP
and t0, t4, t1
bnez t0, 1f
riscv: Restore floating-point caller saved registers before integer This fixes an issue where the t0 register is overwritten after it has been restored. Signed-off-by: Corey Wharton <coreyw7@fb.com>
2020-04-14 14:56:12 -07:00
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
/* Set user mode variable */
li t0, 1
la t1, is_user_mode
sw t0, 0(t1)
/* load scratch reg with stack pointer for next exception entry */
add t0, sp, __z_arch_esf_t_SIZEOF
csrw mscratch, t0
1:
#endif
riscv: Restore floating-point caller saved registers before integer This fixes an issue where the t0 register is overwritten after it has been restored. Signed-off-by: Corey Wharton <coreyw7@fb.com>
2020-04-14 14:56:12 -07:00
riscv: isr.S: dedicate a register to &current_cpu Stop using &_kernel as this is not SMP friendly. Let's use s0 (after preserving its content) to hold &current_cpu instead so it won't have to be reloaded each time it is needed. This will be even more relevant when SMP support is added. Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-09 17:21:27 -05:00
/* Restore s0 (it is no longer ours) */
lr s0, __z_arch_esf_t_s0_OFFSET(sp)
arch: added support for the riscv32 architecture RISC-V is an open-source instruction set architecture. Added support for the 32bit version of RISC-V to Zephyr. 1) exceptions/interrupts/faults are handled at the architecture level via the __irq_wrapper handler. Context saving/restoring of registers can be handled at both architecture and SOC levels. If SOC-specific registers need to be saved, SOC level needs to provide __soc_save_context and __soc_restore_context functions that shall be accounted by the architecture level, when corresponding config variable RISCV_SOC_CONTEXT_SAVE is set. 2) As RISC-V architecture does not provide a clear ISA specification about interrupt handling, each RISC-V SOC handles it in its own way. Hence, at the architecture level, the __irq_wrapper handler expects the following functions to be provided by the SOC level: __soc_is_irq: to check if the exception is the result of an interrupt or not. __soc_handle_irq: handle pending IRQ at SOC level (ex: clear pending IRQ in SOC-specific IRQ register) 3) Thread/task scheduling, as well as IRQ offloading are handled via the RISC-V system call ("ecall"), which is also handled via the __irq_wrapper handler. The _Swap asm function just calls "ecall" to generate an exception. 4) As there is no conventional way of handling CPU power save in RISC-V, the default nano_cpu_idle and nano_cpu_atomic_idle functions just unlock interrupts and return to the caller, without issuing any CPU power saving instruction. Nonetheless, to allow SOC-level to implement proper CPU power save, nano_cpu_idle and nano_cpu_atomic_idle functions are defined as __weak at the architecture level. Change-Id: I980a161d0009f3f404ad22b226a6229fbb492389 Signed-off-by: Jean-Paul Etienne <fractalclone@gmail.com>
2017-01-11 00:24:30 +01:00
/* Restore caller-saved registers from thread stack */
riscv: exception code mega simplification and optimization Complete revamp of the exception entry code, including syscall handling. Proper syscall frame exception trigger. Many correctness fixes, hacks removal, etc. etc. I tried to make this into several commits, but this stuff is all inter-related and a pain to split. The diffstat summary: 14 files changed, 250 insertions(+), 802 deletions(-) Binary size (before): text data bss dec hex filename 1104 0 0 1104 450 isr.S.obj 64 0 0 64 40 userspace.S.obj Binary size (after): text data bss dec hex filename 600 0 0 600 258 isr.S.obj 36 0 0 36 24 userspace.S.obj Run of samples/userspace/syscall_perf (before): *** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010140): 77312 cycles 77437 instructions Run of samples/userspace/syscall_perf (after): *** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 *** Main Thread started; qemu_riscv32 Supervisor thread started User thread started Supervisor thread(0x80010048): 384 cycles 509 instructions User thread(0x80010138): 7040 cycles 7165 instructions Yes, that's more than a 10x speed-up! Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-24 22:30:03 -05:00
DO_CALLER_SAVED(lr)
#ifdef CONFIG_USERSPACE
/* retrieve saved stack pointer */
lr sp, __z_arch_esf_t_sp_OFFSET(sp)
#else
/* remove esf from the stack */
addi sp, sp, __z_arch_esf_t_SIZEOF
#endif
arch: added support for the riscv32 architecture RISC-V is an open-source instruction set architecture. Added support for the 32bit version of RISC-V to Zephyr. 1) exceptions/interrupts/faults are handled at the architecture level via the __irq_wrapper handler. Context saving/restoring of registers can be handled at both architecture and SOC levels. If SOC-specific registers need to be saved, SOC level needs to provide __soc_save_context and __soc_restore_context functions that shall be accounted by the architecture level, when corresponding config variable RISCV_SOC_CONTEXT_SAVE is set. 2) As RISC-V architecture does not provide a clear ISA specification about interrupt handling, each RISC-V SOC handles it in its own way. Hence, at the architecture level, the __irq_wrapper handler expects the following functions to be provided by the SOC level: __soc_is_irq: to check if the exception is the result of an interrupt or not. __soc_handle_irq: handle pending IRQ at SOC level (ex: clear pending IRQ in SOC-specific IRQ register) 3) Thread/task scheduling, as well as IRQ offloading are handled via the RISC-V system call ("ecall"), which is also handled via the __irq_wrapper handler. The _Swap asm function just calls "ecall" to generate an exception. 4) As there is no conventional way of handling CPU power save in RISC-V, the default nano_cpu_idle and nano_cpu_atomic_idle functions just unlock interrupts and return to the caller, without issuing any CPU power saving instruction. Nonetheless, to allow SOC-level to implement proper CPU power save, nano_cpu_idle and nano_cpu_atomic_idle functions are defined as __weak at the architecture level. Change-Id: I980a161d0009f3f404ad22b226a6229fbb492389 Signed-off-by: Jean-Paul Etienne <fractalclone@gmail.com>
2017-01-11 00:24:30 +01:00
/* Call SOC_ERET to exit ISR */
SOC_ERET
Copy permalink