This reverts commit 7c90f1bca1.
Xen initialization maps enlighten page to Zephyr memory and also
initializes Xen event channels. It is used for communication between
Xen domains and based on interrupt connected to domain virtual GIC.
Moving event channel initialization to arch_kernel_init() make it call
irq_enable() when GIC is not initialized. Since GIC is initialized
on PRE_KERNEL_1 stage, this lead to fatal error during boot.
Revert these changes to make xenvm operable again.
Signed-off-by: Dmytro Firsov <dmytro_firsov@epam.com>
The original idea of the syscall helpers is to workaround
an issue where the compiler could not correctly model register
usage with inline functions. It was supposed to be only used
when there are more then 3 arguments to syscalls. However,
during the original MMU code development, the helper got
expanded to cover "less then 2 arguments syscalls" as a way
for debugging syscall handling code and was never removed.
So fix that now by limiting the helper for syscalls with
more than 3 arguments.
Moreover, instead of using one helper with 6 arguments, now
we have separate implementations for both 4 and 5 arguments
syscall helpers. Now the compiler does not have to generate
code to always handle 6 arguments which saves a few code bytes
as there is no need to pass extra zeroes as arguments.
This has been verified to work with xt-clang RI-2022.10.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This is currently selected by the arch, but some devices (in paticular
xtensa ones which have configurable interrupt setups) might not
actually be able to effect an irq_offload() from within an ISR even
though we have code for it.
Make this a default and not a select so that lower layers can override
the setting.
Signed-off-by: Andy Ross <andyross@google.com>
Introduce `CONFIG_RISCV_GP_PURPOSE` choice to make sure that only
one of `CONFIG_RISCV_GP` or `CONFIG_RISCV_CURRENT_VIA_GP` can be
enabled, instead of relying of dependencies.
To do that, introduce a new
`CONFIG_RISCV_SOC_HAS_GP_RELATIVE_ADDRESSING` that can be selected
by SoC when it implemented global pointer (GP) initialization for
relative addressing in its linker.
`CONFIG_RISCV_GP` will be the default choice when
`CONFIG_RISCV_SOC_HAS_GP_RELATIVE_ADDRESSING=y`
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
Reset the the `gp` register to `_kernel->cpus[i].current` when
`CONFIG_USERSPACE` is enabled on exception to keep it sane.
Updated the testcase to test both `CONFIG_RISCV_GP` and
`CONFIG_RISCV_CURRENT_VIA_GP`.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
In most implements, the msip base address is 0x2000000. But the address
is not fixed in all boards.
Signed-off-by: Yang Jialong <yangjialong@vcore.com>
On Arm Cortex R52, cache segregation policy controls the
number of L1 I/D cache ways that are allocated to Flash
and AXIM interface. Adding Kconfig options for configuring
it.
Writing to IMP_CSCTRL is only permitted before the caches
have been enabled, following a system reset.
Signed-off-by: Dat Nguyen Duy <dat.nguyenduy@nxp.com>
This adds support for the single-vector trapping (SVT) model, defined by
SPARC-V8 Embedded (V8E) Architecture Specification. SVT is available in
most LEON processors.
With single-vector trapping enabled, all traps are vectored through a
single trap vector (TBR.A) rather than one 16-byte entry for each trap
type. This improves memory utilization because the full 4 KiB trap table
is not needed.
This implementation uses a two-level of lookup table to find the handler
for the trap type (0..255).
- Execution time is constant.
- Condition flags are preserved.
- The implementation footprint is 60 bytes .text and 284 bytes .rodata.
For comparison, a non-SVT table is always 4096 .text.
Signed-off-by: Martin Åberg <martin.aberg@gaisler.com>
Enabling CONFIG_FPU and CONFIG_FPU_SHARING requires the
definition of `arch_float_disable` and `arch_float_enable`.
Signed-off-by: Matthias Alleman <matthias.alleman@basalte.be>
`_current` is now functionally equals to `arch_curr_thread()`, remove
its usage in-tree and deprecate it instead of removing it outright,
as it has been with us since forever.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
Implement `arch_curr_thread()` & `arch_set_curr_thread()`
with the global pointer (GP) register.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
Add the following arch-specific APIs:
- arch_curr_thread()
- arch_set_curr_thread()
which allow SMP architectures to implement a faster "get current
thread pointer" than the default provided by the kernel. The 'set'
function is required for the 'get' to work, more on that later.
When `CONFIG_ARCH_HAS_CUSTOM_CURRENT_IMPL` is selected, calls to
`_current` & `k_sched_current_thread_query()` will be redirected to
`arch_curr_thread()`, which ideally should translate into a single
instruction read, avoiding the current
"lock > read CPU > read current thread > unlock" path in SMP
architectures and thus greatly improves the read performance.
However, since the kernel relies on a copy of the "current thread"s on
every CPU for certain operations (i.e. to compare the priority of the
currently scheduled thread on another CPU to determine if IPI should be
sent), we can't eliminate the copy of "current thread" (`current`) from
the `struct _cpu` and therefore the kernel now has to invoke
`arch_set_curr_thread()` in addition to what it has been doing. This
means that it will take slightly longer (most likely one instruction
write) to change the current thread pointer on the current
CPU.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
We hit some unaligned access faults running our internal tests.
Not every 32-bit instruction is 32-bit aligned; some are 16-bit aligned.
Make all reads and writes potentially unaligned to be on the safe side.
Signed-off-by: Ilya Tagunov <Ilya.Tagunov@synopsys.com>
Previously PMP was avaible only with Multithreading, since it's now
available without MT - add extra checks to prevent user/kernel mode
reconfiguration.
Signed-off-by: Volodymyr Fialko <vfialko@marvell.com>
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Without multithreading only two stacks present: ISR and main.
As any stack they also could overflow, so it make sense to add stack
guard for them also.
Remove stack guard dependency on multithreading and mark
`Z_RISCV_STACK_GUARD_SIZE` bytes at the beginning of stack as read-only
region with PMP entry.
Signed-off-by: Volodymyr Fialko <vfialko@marvell.com>
According to the RISC-V Instruction Set Manual: Volume II, Version 20240411
(Section 3.1.6.6), some implementations may choose to track the dirtiness
of the floating-point register state imprecisely by reporting the state to
be dirty even when it has not been modified. This option reflects that.
Also add a filter in `tests/arch/riscv/fpu_sharing/` based on imprecise
FPU state tracking
Signed-off-by: Jakub Wasilewski <jwasilewski@internships.antmicro.com>
Signed-off-by: Filip Kokosinski <fkokosinski@antmicro.com>
This adds a new kconfig for eviction algorithm which needs page
tracking. When enabled, k_mem_paging_eviction_add()/_remove()
and k_mem_paging_eviction_accessed() must be implemented.
If an algorithm does not do page tracking, there is no need to
implement these functions, and no need for the kernel MMU code
to call into empty functions. This should save a few function
calls and some CPU cycles.
Note that arm64 unconditionally calls those functions so
forces CONFIG_EVICTION_TRACKING to be enabled there.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
What is changed?
Use CMSIS SystemCoreClock via a dedicated flag instead of using
soc flags.
Why do we need this change?
This change is part of cleaning soc specific code out of arch folder.
Signed-off-by: Sudan Landge <sudan.landge@arm.com>
When detached sections are used, STB_GLOBAL relocations also have to
be processed depending on the relocation type. This commit unified
STB_GLOBAL and STB_LOCAL flows by making them use the same relocation
type parser, adds support for R_XTENSA_GLOB_DAT and R_XTENSA_JMP_SLOT
type relocations on Xtensa and fixes STT_SECTION address calculation
for such sections.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
Adds compiler flag(s) and some architecture-specific relocations
for ARC. No userspace support, doesn't support all relocations.
Signed-off-by: Lauren Murphy <lauren.murphy@intel.com>
Allow SoC to implement their custom per-core initialization function by
selecting `CONFIG_SOC_PER_CORE_INIT_HOOK` and implement
`soc_per_core_init_hook()`.
Signed-off-by: Maxim Adelman <imax@meta.com>
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
Extend the ARM M-profile suspend-to-RAM implementation to be compatible
with all versions of the M-profile supported by Zephyr: ARMv6-M, ARMv7-M,
and ARMv8-M Baseline.
Signed-off-by: Mathieu Choplain <mathieu.choplain@st.com>
Wrap the CPU register save/restore operations (GPR and special registers)
in macros to make core logic simpler to follow. This is also a preparatory
step to introduce ARMv6-M and ARMv7-M support.
Signed-off-by: Mathieu Choplain <mathieu.choplain@st.com>
Use macros to wrap the interaction between the assembly code and the
struct __cpu_context. This helps making the assembly more readable.
Signed-off-by: Mathieu Choplain <mathieu.choplain@st.com>
Remove all xPSR-related registers from struct __cpu_context, and the
associated save/restore code in S2RAM code, as they are not needed:
* EPSR and IPSR are read-only - they cannot be "restored"
* Bits N, V, Z, C, V, Q, and GE (if DSP Extension is implemented) of APSR
could be restored, but this is not needed as the AAPCS indicates these
bits to be "undefined on entry to or return from a public interface"
Signed-off-by: Mathieu Choplain <mathieu.choplain@st.com>
The currently IPI implementation assumes that CLINT exists in the
system, however, that might not be the case as IPI can be implemented
with PLIC that supports software-triggering as well, such as the Andes
NCEPLIC100.
Refactor the CLINT-based IPI implementations into `ipi_clint.c`, and
create Kconfig that selects the CLINT implementation when
`sifive-clint0` exists and enabled, otherwise default to
`RISCV_SMP_IPI_CUSTOM` which allows OOT implementation. This also
makes way for the upstreaming of non-clint IPI implementation later.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
The STKALIGN mask is not present for CONFIG_ARMV8_M_BASELINE as
well as CONFIG_ARMV8_M_MAINLINE. So filter out that check when
setting the sp for ARM core dumps.
Signed-off-by: Mark Holden <mholden@meta.com>
Reset the gp on exception entry from u-mode to protect the kernel
against a possible rogue user thread.
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
ramfunc region is copied into RAM from FLASH region during XIP init. We
copy from the loadaddr of the region, and were previously loading to the
symbol __ramfunc_start. This is incorrect when using an MPU with
alignment requirements, as the __ramfunc_start symbol may have padding
placed before it in the region. The __ramfunc_start symbol still needs
to be aligned in order to be used by the MPU though, so define a new
symbol __ramfunc_region_start, and use that symbol when copying the
__ramfunc region from FLASH to RAM.
Fixes#75296
Signed-off-by: Daniel DeGrasse <daniel.degrasse@nxp.com>
The FPU is already disabled by the z_arm_svc function when the first
thread starts. Therefore, disabling the FPU at boot is unnecessary for
lazy FPU; instead, it must be enabled to handle floating-point instructions
before the lazy FPU works.
Signed-off-by: Ha Duong Quang <ha.duongquang@nxp.com>
Gdb is typically able to reconstruct the first two frames of the
failing stack using the "pc" and "lr" registers. After that, (if
the frame pointer is omitted) it appears to need the stack pointer
(sp register) to point to the top of the stack before a fatal
error occurred.
The ARM Cortex-M processors push registers r0-r3, r12, LR,
{possibly FPU registers}, PC, SPSR onto the stack before entering the
exception handler. We adjust the stack pointer back to the point
before these registers were pushed for preservation in the dump.
During k_oops/k_panic, the sp wasn't stored in the core dump at all.
Apply similar logic to store it when failures occur in that path.
Signed-off-by: Mark Holden <mholden@meta.com>
This change was in the same commit previously reverted and seem to be
unrelated and should not be reverted.
Fixes the problem:
..... /swap_helper.S:432:(.text.z_arm_svc+0x26):
relocation truncated to fit: R_ARM_THM_JUMP11 against symbol
`z_arm_int_exit' defined in .text._HandlerModeExit section in
....core/cortex_m/libarch__arm__core__cortex_m.a(exc_exit.c.obj)
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Add "memory" to the clobber list"
From GCC 14 the compiler optimizes away memory accesses that do not
impact the asm block. Adding the memory to the clobber list lets the
compiler know that the memory state is to be preserved.
Signed-off-by: Wilfried Chauveau <wilfried.chauveau@arm.com>
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
Move the _main argument to the input list rather than the output one on
the asm block and change the spec to "r". The ASM block does not return,
so it does not make sense for it to expect any output.
Signed-off-by: Wilfried Chauveau <wilfried.chauveau@arm.com>
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
To use Xtensa toolchain, various environment variables must be
set so the executables can find necessary files and what core
to compile for. This becomes an annoyance when you have to
test multiple boards with different cores. You have to use
one set of environment variables per core. Twister cannot test
them all in one setting, and it is especially annoying doing
west builds. So enhance the environment variables handling so
that TOOLCHAIN_VER and XTENSA_CORE can be replaced by
TOOLCHAIN_VAR_<board> and XTENSA_CORE_<board> where <board>
is the normalized board target (think <board>.yaml). CMake
will then figure out the core ID for the toolchain to use.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
The coprocessor number in ARM `mrc` and `mcr` instructions must be prefixed
with `p`.
GNU assembler allows specifying coprocessor number without the `p` prefix;
but, LLVM assembler is more picky about this and prints out "invalid
instruction" error otherwise.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
Added casts to uint32_t in arch_timing_cycles_get() to handle the
wrap-around of the 32-bit cycle counter correctly.
Signed-off-by: Simon Tomschik <simon.tomschik@askgroup.global>
_xtos_pso_savearea and _xtos_core_restore_nw have been renamed to
xthal_pso_savearea and xthal_core_restore_nw in recent Xtensa
toolchains. Rename them in reset_vector.S to prevent linker errors
when building Zephyr with Xtensa toolchain.
Signed-off-by: Tahsin Mutlugun <Tahsin.Mutlugun@analog.com>
What is changed?
Updated the condition thats prevents mpu config for null dereference.
Added a new check so that mpu is configured for null dereference if
devicetree contains a memory-region node with:
- node address starting at 0
- size covered by the node is more than the null dereference page
size (0x400) and
- contains a memory-attr
Why is the change needed?
The check relied on flash base address to align with 0 for
configuring the mpu for null dereference but, a device tree
could have a flash starting at an address other than 0 and
still need the mpu config for null dereference.
The new extra check provides a way to connfigure mpu for
null dereference even if flash base address is not 0.
Note, though this change helps with mpu config for new boards having
flash address other than 0, this change does not change existing
behaviour for existing boards.
Signed-off-by: Sudan Landge <sudan.landge@arm.com>
The S2RAM procedure requires marker checking after reset.
Such checking is performed on very early stage of the system initialization
and must ensure that the stack is not used due to the TLS pointer which is
not initialized yet.
Signed-off-by: Adam Kondraciuk <adam.kondraciuk@nordicsemi.no>
The commit introduced regression for hsdk4xd platform.
The hsdk4xd SoC setup from soc_early_asm_init_percpu need to be done
in early code before any C code execution.
The current approach has multiple issues
- we call function (which can be easily implemented in C for
this or other SoC) from the place where we haven't setup stack
pointer (so we can't use stack) - which is very error-prone
- we never return back from soc_reset_hook on hsdk4xd platform
So let's just revert it for now. If any other ARC SoC need to use
soc_reset_hook - than it can be implemented properly.
This reverts commit 8c32a82e47.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
Adds support for cooperative locking/unlocking the VPX vector registers.
Provided that all VPX enabled threads use these routines to control
access to the VPX vector registers, it will allow multiple threads to
safely use them without the need for saving/restoring them upon each
context switch.
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
Support for R_XTENSA_SLOT0_OP was implemented with a relocation table
test case, containing only one entry at offset 0. For multiple
entries .r_addend has to be taken into account.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
What is changed?
1. Updated the data sync barrier to make sure the other parameters of
`arm_cpu_boot_params` are updated before updating its member `mpidr`
2. Updated the MPIDR affinity level mask to account for affinity level
1 and 2 along with level 0.
Why do we need this change?
1. As reported in issue #76182, on Cortex_A_R, the current code
execution fails to consider the correct sequence of data sync
barrier and cache maintenece for the code to work on non cache
coherent cores in SMP enabled mode.
The secondary cores are waiting in a loop for primary core to set
`arm_cpu_boot_params.mpidr`. As soon as primary core set this,
the secondary cores start reading other parameters from the
`arm_cpu_boot_params` however, the existing position of DSB
instruction doesn't guarantee that `arg`, `cpu_num` and other
parameters of `arm_cpu_boot_params` would be updated before `mpidr`
is udpated and this could lead to a unpredicatble behaviour so,
we need to move the DSB instruction.
2. The affinity level mask is updated because it didn't account for
level 1 to identify individual cores within a cluster and
level 2 to identify different clusters within the system which can
lead to an incorrect conversion between mpidr to core-id.
Signed-off-by: Sudan Landge <sudan.landge@arm.com>
