Commit graph

349 commits

Author SHA1 Message Date
Carlo Caione
10061efdc4 riscv: Rework and cleanup Kconfig
This patch is doing several things:

- Core ISA and extension Kconfig symbols have now a formalized name
  (CONFIG_RISCV_ISA_* and CONFIG_RISCV_ISA_EXT_*)

- a new Kconfig.isa file was introduced with the full set of extensions
  currently supported by the v2.2 spec

- a new Kconfig.core file was introduced to host all the RISCV cores
  (currently only E31)

- ISA and extensions settings are moved to SoC configuration files

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2022-06-05 14:28:42 +02:00
Carles Cufi
56512dae8b arch: riscv: switch: Add a comment on the return of z_riscv_switch
When returning from z_riscv_switch, depending on whether the thread that
has just been swapped in was earlier swapped out synchronously (i.e. via
regular function call) or asynchronously (i.e. via exception/irq) we
will return to arch_switch() or __irq_wrapper respectively. Comment this
fact for clarity.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-05-26 17:15:21 +02:00
Carles Cufi
11da0b6f28 arch: riscv: Remove outdated comment
After the introduction of arch_switch() in #43085, ECALL is no longer
used for context switching by default, so remove the comment stating so.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-05-26 17:15:21 +02:00
Nicolas Pitre
1cb557dccf riscv: rationalize PMP related Kconfig options
ARCH_HAS_USERSPACE and ARCH_HAS_STACK_PROTECTION are direct functions
of RISCV_PMP regardless of the SoC.

PMP_STACK_GUARD is a function of HW_STACK_PROTECTION (from
ARCH_HAS_STACK_PROTECTION) and not the other way around.

This allows for tests/kernel/fatal/exception to test protection against
various stack overflows based on the PMP stack guard functionality.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-05-18 10:54:53 +02:00
Nicolas Pitre
e76fb204db riscv: report stack overflow errors correctly
Add the necessary checks to determine when the stack pointer is
 out of bounds.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-05-18 10:54:53 +02:00
Nicolas Pitre
a4b82ab4fe riscv: fix IRQ stack guard location
_current_cpu->irq_stack is not yet initialized when this is executed on
CPU 0. Also the guard area is outside of CONFIG_ISR_STACK_SIZE now
e.g. it is within the K_KERNEL_STACK_RESERVED area at the start of
the buffer. So simply use z_interrupt_stacks[] directly instead.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-05-18 10:54:53 +02:00
Nicolas Pitre
92409f36de riscv: drop user stack guard area when using separate privileged stacks
A separate privileged stack is used when CONFIG_GEN_PRIV_STACKS=y. The
main stack guard area is no longer needed and can be made available to
the application upon transitioning to user mode. And that's actually
required if we want a naturally aligned power-of-two buffer to let the
PMP map a NAPOT entry on it which is the whole point of having this
CONFIG_PMP_POWER_OF_TWO_ALIGNMENT option in the first place.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-05-18 10:54:53 +02:00
Nicolas Pitre
6051ea7d3c riscv: clarify stack size and alignment parameters
The StackGuard area is used to save the esf and run the exception code
resulting from a StackGuard trap. Size it appropriately.

Remove redundancy, clarify documentation, etc.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-05-18 10:54:53 +02:00
Nicolas Pitre
3997f7bed2 riscv: pmp: make PMP debug display more comprehensive
Decoding those values by hand gets tedious.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-05-18 10:54:53 +02:00
Robert Szczepanski
8647e2f63c tracing: riscv: Add missing invoke of sys_trace_isr_exit()
Change suggested by @WealianLiao in #41995.

Signed-off-by: Robert Szczepanski <rszczepanski@antmicro.com>
2022-05-11 12:03:41 -04:00
Gerard Marull-Paretas
4b91c2d79f asm: update files with <zephyr/...> include prefix
Assembler files were not migrated with the new <zephyr/...> prefix.
Note that the conversion has been scripted, refer to #45388 for more
details.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2022-05-09 12:45:29 -04:00
Gerard Marull-Paretas
16811660ee arch: migrate includes to <zephyr/...>
In order to bring consistency in-tree, migrate all arch code to the new
prefix <zephyr/...>. Note that the conversion has been scripted, refer
to zephyrproject-rtos#45388 for more details.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2022-05-06 19:57:22 +02:00
Nicolas Pitre
f51d89df30 riscv: pmp: work around a QEMU bug
The NAPOT mode isn't computed properly in qemu when the full address
range is covered. Let's hardcode the value that the qemu code checks
explicitly until the appropriate fix is applied to qemu itself.

For reference, here's the qemu patch:
https://lists.gnu.org/archive/html/qemu-devel/2022-04/msg00961.html

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-04-29 15:30:00 +02:00
Nicolas Pitre
ec9c2ec2d8 riscv: pmp: rename CONFIG_PMP_SLOT
The plural form is clearer.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-04-29 15:30:00 +02:00
Nicolas Pitre
554f24661f riscv: pmp: remove previous implementation
Overall diffstat with the new PMP code in place:

 18 files changed, 866 insertions(+), 1372 deletions(-)

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-04-29 15:30:00 +02:00
Nicolas Pitre
2fece49a14 riscv: pmp: switch over to the new implementation
Add the appropriate hooks effectively replacing the old implementation
with the new one.

Also the stackguard wasn't properly enforced especially with the
usermode combination. This is now fixed.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-04-29 15:30:00 +02:00
Nicolas Pitre
7a55bda7e1 riscv: pmp: add new usermode support
The idea here is to compute the PMP register set on demand i.e. upon
scheduling in the affected threads, and only if changes occurred.
A simple sequence number is used to stay in sync with the latest update.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-04-29 15:30:00 +02:00
Nicolas Pitre
68b8f0e5ce riscv: pmp: new stackguard implementation
Stackguard uses the PMP to prevents many types of stack overflow by
making any access to the bottom stack area raise a CPU exception. Each
thread has its set of precomputed PMP entries and those are written to
PMP registers at context switch time.

This is the code to set it up. It will be connected later.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-04-29 15:30:00 +02:00
Nicolas Pitre
2e66da3bc3 riscv: pmp: new implementation
This is the core code to manage PMP entries with only the global entries
initialisation for now. It is not yet linked into the build.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-04-29 15:30:00 +02:00
Keith Packard
f623571a73 riscv: Initialize TP register when starting threads
Set TP in exception context so that it gets loaded into the CPU when
first running the thread. Set TP for secondary cores to related idle TLS
area.

Signed-off-by: Keith Packard <keithp@keithp.com>
2022-04-28 11:09:01 +09:00
Nicolas Pitre
f61b8b8c16 semihosting: fix inline assembly output dependency
Commit d8f186aa4a ("arch: common: semihost: add semihosting
operations") encapsulated semihosting invocation in a per-arch
semihost_exec() function. There is a fixed register variable declaration
for the return value but this variable is not listed as an output
operand to respective inline assembly segments which is an error.
This is not reported as such by gcc and the generated code is still OK
in those particular instances but this is not guaranteed, and clang
does complain about such cases.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-04-24 19:46:15 +02:00
Jordan Yates
d8f186aa4a arch: common: semihost: add semihosting operations
Add an API that utilizes the ARM semihosting mechanism to interact with
the host system when a device is being emulated or run under a debugger.

RISCV is implemented in terms of the ARM implementation, and therefore
the ARM definitions cross enough architectures to be defined 'common'.

Functionality is exposed as a separate API instead of syscall
implementations (`_lseek`, `_open`, etc) due to various quirks with
the ARM mechanisms that means function arguments are not standard.

For more information see:
https://developer.arm.com/documentation/dui0471/m/what-is-semihosting-

Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>

impl
2022-04-21 13:04:52 +02:00
Nathan Krueger
6a5520c626 arch/riscv: Adding KConfig options for 'A' and 'M' RISC-V extensions
New KConfig options for 'A' and 'M' RISC-V extensions have been
added.  These are used to configure the '-march' string used by GCC
to produce a compatible binary for the requested RISC-V variant.
In order to maintain compatibility with all currently defined SoC,
default the options for HW mul / Atomics support to 'y', but allow
them to be overridden for any SoC which does not support these.

I tested this change locally via twister agaisnt a few RISC-V platforms
including some 32bit and 64bit. To verify the 4 possibilities of Atomics
& HW Mul: (No, No), (No, Yes), (Yes, No), (Yes, Yes -- current behavior),
I used an out-of-tree GCC (xPack RISC-V GCC) which has multilib support
for rv32i, rv32ia, rv32ima to test against our out-of-tree Intel Nios V/m
processor in HW.  The Zephyr SDK RISCV GCC currently does not contain
multilib support for all variants exposed by these new KConfig options.

Signed-off-by: Nathan Krueger <nathan.krueger@intel.com>
2022-03-22 18:00:32 -04:00
Nicolas Pitre
c8bfc2afda riscv: make arch_is_user_context() SMP compatible
This is painful. There is no way for u-mode code to know if we're
currently executing in u-mode without generating a fault, besides
stealing a general purpose register away from the standard ABI
that is. And a global variable doesn't work on SMP as this must be
per-CPU and we could be migrated to another CPU just at the right
moment to peek at the wrong CPU variable (and u-mode can't disable
preemption either).

So, given that we'll have to pay the price of an exception entry
anyway, let's at least make it free to privileged threads by using
the mscratch register as the non-user context indicator (it must
be zero in m-mode for exception entry to work properly). In the
case of u-mode we'll simulate a proper return value in the
exception trap code. Let's settle on the return value in t0
and omit the volatile to give the compiler a chance to cache
the result.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-21 07:28:05 -04:00
Nicolas Pitre
af2d875c5d riscv: isr.S: compute _current_cpu using CPU number on SMP
To do so efficiently on systems without the mul instruction, we use
shifts and adds which is faster and sometimes smaller than a plain loop.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-21 07:28:05 -04:00
Nicolas Pitre
4f5374854e riscv: isr.S: dedicate a register to &current_cpu
Stop using &_kernel as this is not SMP friendly. Let's use s0 (after
preserving its content) to hold &current_cpu instead so it won't have
to be reloaded 	each time it is needed. This will be even more relevant
when SMP support is added.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-21 07:28:05 -04:00
Nicolas Pitre
69d06a901c riscv: isr.S: optimize FP regs save/restore decision
Rely on mstatus rather than thread->base.user_options since it is always
up to date (updated by z_riscv_switch) to simplify the code and be SMP
proof. Also carry over SF_INIT to the mstatus being restored in case
it was changed in the mean time.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-21 07:28:05 -04:00
Nicolas Pitre
ce8dabfe9e riscv: implement arch_switch()
The move to arch_switch() is a prerequisite for SMP support.

Make it optimal without the need for an ECALL roundtrip on every
context switch. Performance numbers from tests/benchmarks/sched:

Before:
unpend  107 ready  102 switch  188 pend  218 tot  615 (avg  615)

After:
unpend  107 ready  102 switch  170 pend  217 tot  596 (avg  595)

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-21 07:28:05 -04:00
Nicolas Pitre
247d2c8e3b riscv: move the tp register from caller-saved to callee-saved
This is a per-thread register that gets updated only when context
switching. No need to load and save it on every exception entry.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-21 07:28:05 -04:00
Nicolas Pitre
50c0df1bd2 riscv: align struct __esf properly
The minimum stack alignment is 16. Therefore, the stack space to store
a struct __esf object must be rounded up to the next 16-byte boundary.

It is not sufficient to do the rounding on the __z_arch_esf_t_SIZEOF
definition. When the stack is constructed in arch_new_thread() it is
also necessary to do the rounding there too.

Let's make the structure itself carry the alignment attribute instead to
make it work in all cases.

While at it, remove the unused _K_THREAD_NO_FLOAT_SIZEOF definition.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-21 07:28:05 -04:00
Nicolas Pitre
df852a0b77 riscv: implement CONFIG_IRQ_OFFLOAD_NESTED
It can easily be done now, so why not. Suffice to increment the nested
count like with actual IRQs.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-21 07:28:05 -04:00
Nicolas Pitre
cb5221c087 riscv: irq_offload: simpler implementation
Get rid of all those global variables and IRQ locking.
Use the regular IRQ exit path to let tests validate preemption properly.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-21 07:28:05 -04:00
Nicolas Pitre
a50c433012 riscv: exception code mega simplification and optimization
Complete revamp of the exception entry code, including syscall handling.
Proper syscall frame exception trigger. Many correctness fixes, hacks
removal, etc. etc.

I tried to make this into several commits, but this stuff is all
inter-related and a pain to split.

The diffstat summary:

 14 files changed, 250 insertions(+), 802 deletions(-)

Binary size (before):

   text	   data	    bss	    dec	    hex	filename
   1104	      0	      0	   1104	    450	isr.S.obj
     64	      0	      0	     64	     40	userspace.S.obj

Binary size (after):

   text	   data	    bss	    dec	    hex	filename
    600	      0	      0	    600	    258	isr.S.obj
     36	      0	      0	     36	     24	userspace.S.obj

Run of samples/userspace/syscall_perf (before):

*** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018  ***
Main Thread started; qemu_riscv32
Supervisor thread started
User thread started
Supervisor thread(0x80010048):       384 cycles	     509 instructions
User thread(0x80010140):           77312 cycles	   77437 instructions

Run of samples/userspace/syscall_perf (after):

*** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3  ***
Main Thread started; qemu_riscv32
Supervisor thread started
User thread started
Supervisor thread(0x80010048):       384 cycles	     509 instructions
User thread(0x80010138):            7040 cycles     7165 instructions

Yes, that's more than a 10x speed-up!

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-21 07:28:05 -04:00
Nicolas Pitre
bfb7919ed0 riscv: better abstraction for register-wide FP load/store opcodes
Same rationale as preceding commit. Let's create pseudo-instructions in
assembly scope to make the code more uniform and readable.

Furthermore the definition of COPY_ESF_FP() was wrong as the width of
floating point registers vary not according to CONFIG_64BIT but
CONFIG_CPU_HAS_FPU_DOUBLE_PRECISION. It is therefore wrong to use
lr/sr (previously RV_OP_LOADREG/RV_OP_STOREREG) and a regular temporary
register to transfer such content.

Note: There are far more efficient ways to copy FP context around but
      such optimisations will come separately.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-21 07:28:05 -04:00
Nicolas Pitre
1fd79b3ef4 riscv: better abstraction for register-wide load/store opcodes
Those are prominent enough that having RV_OP_LOADREG and RV_OP_STOREREG
shouting at you all over the place is rather unpleasant and bad taste.

Let's create pseudo-instructions of our own with assembler macros
rather than preprocessor defines and only in assembly scope.
This makes the asm code way more uniform and readable.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-21 07:28:05 -04:00
Nicolas Pitre
94f39e5a80 riscv: fix wrong access width in assembly code
The thread->base.user_options field is an uint8_t. Access it using lb.
A "copy" of it is made into __esf.fp_state. Make that field an uint8_t
too and access it with lb/sb.

_callee_saved.fcsr is an uint32_t. Access it with lw/sw.
Ditto for is_user_mode.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-21 07:28:05 -04:00
Nicolas Pitre
9ed17943b9 riscv: use simplest asm expression when possible
Let's take advantage of assembler pseudoinstructions:

- convert `addi rd, rs, 0` to `mv rd, rs`
- convert `jal x0, somewhere` to `j somewhere`
- convert `csrrs x0, csrreg, rs` to `csrs csrreg, rs`
- convert `fscsr x0, rs` to `fscsr rs`

And simplify zero offsets to simply 0.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-21 07:28:05 -04:00
Nicolas Pitre
f2bb937547 Revert "arch/riscv: Get current CPU properly instead of assuming single CPU"
This reverts commit 8686ab5472.

The purpose of this commit will be reintroduced later on top of
a cleaner codebase.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-21 07:28:05 -04:00
Nicolas Pitre
442ab22bdc Revert "arch/riscv: Use arch_switch() for context swap"
This reverts commit be28de692c.

The purpose of this commit will be reintroduced later on top of
a cleaner codebase.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-21 07:28:05 -04:00
Nicolas Pitre
13a7047ea9 Revert "arch/riscv: Do not use irq_lock() on arch_irq_offload"
This reverts commit b0458201cc.

The purpose of this commit will be reintroduced later on top of
a cleaner codebase.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-03-21 07:28:05 -04:00
Nazar Kazakov
f483b1bc4c everywhere: fix typos
Fix a lot of typos

Signed-off-by: Nazar Kazakov <nazar.kazakov.work@gmail.com>
2022-03-18 13:24:08 -04:00
Corey Wharton
72afd96c9b arch: riscv: ensure fcsr is cleared on thread start or FPU enable
Ensure fcsr is always initially cleared for FPU enabled threads.

Signed-off-by: Corey Wharton <xodus7@cwharton.com>
2022-03-16 10:25:50 +01:00
Gerard Marull-Paretas
95fb0ded6b kconfig: remove Enable from boolean prompts
According to Kconfig guidelines, boolean prompts must not start with
"Enable...". The following command has been used to automate the changes
in this patch:

sed -i "s/bool \"[Ee]nables\? \(\w\)/bool \"\U\1/g" **/Kconfig*

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2022-03-09 15:35:54 +01:00
Ederson de Souza
2aab236c12 arch/riscv: Add IPI support
Use CLINT to send interrupts to another CPU. SMP support is kinda
incomplete without it.

This patch only enables it for riscv-privilege platforms - specifically,
"virt" one.

Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
2022-02-25 19:13:50 -05:00
Ederson de Souza
b0458201cc arch/riscv: Do not use irq_lock() on arch_irq_offload
With SMP, it's the wrong with to do, according to
3b145c0d4b.

Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
2022-02-25 19:13:50 -05:00
Ederson de Souza
d9ab35577b arch/riscv: Boot secondary CPUs for SMP support
Secondary CPUs are now initialised and made available to the system. If
the system has more CPUs than configured via CONFIG_MP_NUM_CPUS, those
are still left looping as before.

Some implementations of `soc_interrupt_init` also changed to use
`arch_irq_lock` instead of `irq_lock`.

Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
2022-02-25 19:13:50 -05:00
Ederson de Souza
be28de692c arch/riscv: Use arch_switch() for context swap
Enable `arch_switch()` as preparation for SMP support. This patch
doesn't try to keep support for old style context swap - only switch
based swap is supported, to keep things simple.

A fair amount of refactoring was done in this patch, specially regarding
the code that decides what to do about the ISR. In RISC-V, ECALL
instructions are used to signalize several events, such as user space
system calls, forced syscall, IRQ offload, return from syscall and
context switch. All those handled by the ISR - which also handles
interrupts. After refactor, this "dispatching" step is done at the
beginning of ISR (just after saving generic registers).

As with other platforms, the thread object itself is used as the thread
"switch handle" for the context swap.

Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
2022-02-25 19:13:50 -05:00
Ederson de Souza
8686ab5472 arch/riscv: Get current CPU properly instead of assuming single CPU
isr.S code currently gets CPU information from global `_kernel` assuming
there's only one CPU. In order to prepare for upcoming SMP support,
change code to actually get current CPU information.

Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
2022-02-25 19:13:50 -05:00
Ederson de Souza
fdf7c96994 arch/riscv: Implement arch_curr_cpu()
Implement function that will be necessary for upcoming SMP support.

Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
2022-02-25 19:13:50 -05:00
Carles Cufi
e83a13aabf kconfig: Rename the TEST_EXTRA stack size option to align with the rest
All stack sizes should end with STACK_SIZE.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-02-22 08:23:05 -05:00