Rely on mstatus rather than thread->base.user_options since it is always
up to date (updated by z_riscv_switch) to simplify the code and be SMP
proof. Also carry over SF_INIT to the mstatus being restored in case
it was changed in the mean time.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The move to arch_switch() is a prerequisite for SMP support.
Make it optimal without the need for an ECALL roundtrip on every
context switch. Performance numbers from tests/benchmarks/sched:
Before:
unpend 107 ready 102 switch 188 pend 218 tot 615 (avg 615)
After:
unpend 107 ready 102 switch 170 pend 217 tot 596 (avg 595)
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
This is a per-thread register that gets updated only when context
switching. No need to load and save it on every exception entry.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The minimum stack alignment is 16. Therefore, the stack space to store
a struct __esf object must be rounded up to the next 16-byte boundary.
It is not sufficient to do the rounding on the __z_arch_esf_t_SIZEOF
definition. When the stack is constructed in arch_new_thread() it is
also necessary to do the rounding there too.
Let's make the structure itself carry the alignment attribute instead to
make it work in all cases.
While at it, remove the unused _K_THREAD_NO_FLOAT_SIZEOF definition.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Get rid of all those global variables and IRQ locking.
Use the regular IRQ exit path to let tests validate preemption properly.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Complete revamp of the exception entry code, including syscall handling.
Proper syscall frame exception trigger. Many correctness fixes, hacks
removal, etc. etc.
I tried to make this into several commits, but this stuff is all
inter-related and a pain to split.
The diffstat summary:
14 files changed, 250 insertions(+), 802 deletions(-)
Binary size (before):
text data bss dec hex filename
1104 0 0 1104 450 isr.S.obj
64 0 0 64 40 userspace.S.obj
Binary size (after):
text data bss dec hex filename
600 0 0 600 258 isr.S.obj
36 0 0 36 24 userspace.S.obj
Run of samples/userspace/syscall_perf (before):
*** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 ***
Main Thread started; qemu_riscv32
Supervisor thread started
User thread started
Supervisor thread(0x80010048): 384 cycles 509 instructions
User thread(0x80010140): 77312 cycles 77437 instructions
Run of samples/userspace/syscall_perf (after):
*** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 ***
Main Thread started; qemu_riscv32
Supervisor thread started
User thread started
Supervisor thread(0x80010048): 384 cycles 509 instructions
User thread(0x80010138): 7040 cycles 7165 instructions
Yes, that's more than a 10x speed-up!
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Same rationale as preceding commit. Let's create pseudo-instructions in
assembly scope to make the code more uniform and readable.
Furthermore the definition of COPY_ESF_FP() was wrong as the width of
floating point registers vary not according to CONFIG_64BIT but
CONFIG_CPU_HAS_FPU_DOUBLE_PRECISION. It is therefore wrong to use
lr/sr (previously RV_OP_LOADREG/RV_OP_STOREREG) and a regular temporary
register to transfer such content.
Note: There are far more efficient ways to copy FP context around but
such optimisations will come separately.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Those are prominent enough that having RV_OP_LOADREG and RV_OP_STOREREG
shouting at you all over the place is rather unpleasant and bad taste.
Let's create pseudo-instructions of our own with assembler macros
rather than preprocessor defines and only in assembly scope.
This makes the asm code way more uniform and readable.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The thread->base.user_options field is an uint8_t. Access it using lb.
A "copy" of it is made into __esf.fp_state. Make that field an uint8_t
too and access it with lb/sb.
_callee_saved.fcsr is an uint32_t. Access it with lw/sw.
Ditto for is_user_mode.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
This reverts commit 8686ab5472.
The purpose of this commit will be reintroduced later on top of
a cleaner codebase.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
This reverts commit be28de692c.
The purpose of this commit will be reintroduced later on top of
a cleaner codebase.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
This reverts commit b0458201cc.
The purpose of this commit will be reintroduced later on top of
a cleaner codebase.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
According to Kconfig guidelines, boolean prompts must not start with
"Enable...". The following command has been used to automate the changes
in this patch:
sed -i "s/bool \"[Ee]nables\? \(\w\)/bool \"\U\1/g" **/Kconfig*
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
Use CLINT to send interrupts to another CPU. SMP support is kinda
incomplete without it.
This patch only enables it for riscv-privilege platforms - specifically,
"virt" one.
Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
Secondary CPUs are now initialised and made available to the system. If
the system has more CPUs than configured via CONFIG_MP_NUM_CPUS, those
are still left looping as before.
Some implementations of `soc_interrupt_init` also changed to use
`arch_irq_lock` instead of `irq_lock`.
Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
Enable `arch_switch()` as preparation for SMP support. This patch
doesn't try to keep support for old style context swap - only switch
based swap is supported, to keep things simple.
A fair amount of refactoring was done in this patch, specially regarding
the code that decides what to do about the ISR. In RISC-V, ECALL
instructions are used to signalize several events, such as user space
system calls, forced syscall, IRQ offload, return from syscall and
context switch. All those handled by the ISR - which also handles
interrupts. After refactor, this "dispatching" step is done at the
beginning of ISR (just after saving generic registers).
As with other platforms, the thread object itself is used as the thread
"switch handle" for the context swap.
Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
isr.S code currently gets CPU information from global `_kernel` assuming
there's only one CPU. In order to prepare for upcoming SMP support,
change code to actually get current CPU information.
Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
When XIP is not enabled, z_data_copy() already falls back to an empty
function. No need to ifdef it.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Non-standard `jalr rd, rs` pseudo-instructions are used.
This commit changes them to `ret` for standard return pseudo-instruction
or `jalr rd, rs, 0` for no offset jump register and link.
Fixes#41100.
Signed-off-by: Henry Hsieh <r901042004@yahoo.com.tw>
Currently, is_user_mode is 8-byte in riscv64 and it breaks a 4-byte PMP
region protecting it. Because is_user_mode is a single flag, we could
just fix it's size to 4-byte in both riscv32 and riscv64.
Signed-off-by: Jim Shu <cwshu09@gmail.com>
In RV64, all general-purpose registers and pmpcfg CSR are 64-bit
instead of 32-bit. Fix these registers and related C variables/literals
to be 32/64-bit compatible.
Signed-off-by: Jim Shu <cwshu09@gmail.com>
For functions returning nothing, there is no need to document
with @return, as Doxgen complains about "documented empty
return type of ...".
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This commit enable PMP-based memory protection of code and rodata
instead of relying on non-writable real HW (e.g. flash). Use static
PMP region with PMP Lock bit to protect them in both user/supervisor
mode.
Signed-off-by: Jim Shu <cwshu@andestech.com>
Implement new mechanism of arch_buffer_validate() to support checking
static PMP regions. This is preparation patch for code/rodate protection
via RISC-V PMP.
Signed-off-by: Jim Shu <cwshu@andestech.com>
Thread init related to PMP & userspace contains 5 parts:
1. User/supervisor thread clear PMP context
2. User thread clear it's context
3. User/supervisor thread assign to different entry
4. Supervisor thread assign mstatus.MPRV for M-mode PMP protection
5. User/supervisor thread setup PMP regions of stack guard if enabled
Signed-off-by: Jim Shu <cwshu@andestech.com>
Reorder the memory domain async functions to:
arch_mem_domain_partition_add()
arch_mem_domain_partition_remove()
arch_mem_domain_thread_add()
arch_mem_domain_thread_remove()
Signed-off-by: Jim Shu <cwshu@andestech.com>
Simplify multiple ifdef case in computing region number. Also move these
macros to core_pmp.c because they are only used in one file.
Signed-off-by: Jim Shu <cwshu@andestech.com>
Using struct riscv_pmp_region to modulize PMP CSR handling, including
PMP NAPOT/TOR mode handling. This patch can make us more easily to
add/remove RISC-V PMP regions without considering register handling.
Signed-off-by: Jim Shu <cwshu@andestech.com>
Cleanup logging API in core_pmp.c. Remove old printf-based debugging API
and change the log module of PMP to individual MPU log module.
Signed-off-by: Jim Shu <cwshu@andestech.com>
This commit add 2 minor fixes of IRQ handling:
1. Save caller registers before calling z_riscv_configure_stack_guard()
in RISC-V assembly.
2. reschedule and no_reschdule code paths use different interrupt
return path after supporting of CONFIG_PMP_STACK_GUARD. back-to-back
interrupt checking is in the reschedule code path so that it should
jump to interrupt return path of reschedule.
Signed-off-by: Jim Shu <cwshu09@gmail.com>
If no thread use this memory domain, there isn't any user PMP region
translated from memory partitions in domain. In this case, memory
partition removal doesn't need to remove user PMP region and
arch_mem_domain_partition_remove() could just successfully return.
Signed-off-by: Jim Shu <cwshu@andestech.com>
Although CONFIG_USERSPACE is enabled, there are supervisor threads who
don't have privileged stack using exception handler. Only let user
threads to switch to privileged stack in exception handler.
Signed-off-by: Jim Shu <cwshu@andestech.com>
Enable ARCH_EXCEPT macro for non-usermode scenario for RISC-V
Macro will now raise an illegal instruction exception so that mepc will
hold expected value in exception handler, and generated coredump can
reconstruct the failing stack
Coredump tests running on renode (for RISC-V) can now utilize fatal error
path through k_panic
Signed-off-by: Mark Holden <mholden@fb.com>
GD32V processor core is used non-standard bitmask
for mcause register. Add option to configure the bitmask
to support GD32V.
Signed-off-by: TOKITA Hiroshi <tokita.hiroshi@gmail.com>
This changes the arch_mem_domain_*() functions to return errors.
This allows the callers a chance to recover if needed.
Note that:
() For assertions where it can bail out early without side
effects, these are converted to CHECKIF(). (Usually means
that updating of page tables or translation tables has not
been started yet.)
() Other assertions are retained to signal fatal errors during
development.
() The additional CHECKIF() are structured so that it will bail
early if possible. If errors are encountered inside a loop,
it will still continue with the loop so it works as before
this changes with assertions disabled.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
In some cases the 'reschedule' code path is executed when the current
thread is the same as the next thread in the ready Q. If this happens,
the swap_return_value of the thread is ifalsely being reset to -EAGAIN.
This commit prevents the rescheduling code to run if the current thread
is the same as the thread in the ready Q.
Signed-off-by: Wolfgang Reißnegger <gnagflow@fb.com>
Cleanup and preparation commit for linker script generator.
Zephyr linker scripts provides start and end symbols for each larger
areas in the linker script.
The symbols _image_rom_start and _image_rom_end corresponds to the group
ROMABLE_REGION defined in the ld linker scripts.
The symbols _image_rodata_start and _image_rodata_end is not placed as
independent group but covers common-rom.ld, thread-local-storage.ld,
kobject-rom.ld and snippets-rodata.ld.
This commit align those names and prepares for generation of groups in
linker scripts.
The symbols describing the ROMABLE_REGION will be renamed to:
_image_rom_start -> __rom_region_start
_image_rom_end -> __rom_region_end
The rodata will also use the group symbol notation as:
_image_rodata_start -> __rodata_region_start
_image_rodata_end -> __rodata_region_end
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Enable RISC-V GP relative addressing by linker relaxation to reduce
the code size. It optimizes addressing of globals in small data section
(.sdata).
The gp initialization at program start needs each SoC support. Also,
if RISC-V SoC has custom linker script, SoC should provide
__global_pointer$ symbol in it's linker script.
Signed-off-by: Jim Shu <cwshu@andestech.com>
Increases the default CONFIG_TEST_EXTRA_STACKSIZE for the 32-bit RISC-V
architecture. This fixes the portability.posix.fs test on the
qemu_riscv32 platform.
Signed-off-by: Maureen Helm <maureen.helm@nxp.com>
This reverts commit 7b09d031fa. Because
context save of GP register is removed, we don't need to initialize GP
at thread init. GP will be a constant value so that it could only be
initialized at program start.
Signed-off-by: Jim Shu <cwshu@andestech.com>
RISC-V global pointer (GP) register is neither caller nor callee
register, and it's a constant value in the single ELF file. Thus, we
don't need to save/restore GP at ISR enter/exit. Remove it to optimize
context switch performance.
Signed-off-by: Jim Shu <cwshu@andestech.com>
