What is the change?
- Fixes#83660 allowing device to now enter suspend mode even if
CONFIG_DEBUG_THREAD_INFO is enabled.
Why is this needed?
- z_sys_post_kernel was cleared as part of #d778d5c
to "allow debuggers to display the correct thread state after
the first 3 instructions have run".
This is not required while resuming from suspend and
it prevents the device from entering suspend so,
move it out of resume path.
Signed-off-by: Sudan Landge <sudan.landge@arm.com>
The RISC-V architecture-specific relocations need to check whether
each required relocation can fit into the modified instruction's
immediate. All immediates in RISC-V are encoded as two's complement.
The current truncation check has an off-by-one error for checking
the maximum negative distance, as two's complement encoding can
represent a negative value that is the maximum positive value plus
one, causing LLEXT to refuse loading valid code.
This commit adds an additional condition to the check that fixes
the aforementioned issue.
Signed-off-by: Eric Ackermann <eric.ackermann@cispa.de>
Define the generic _current directly and get rid of the generic
arch_current_get().
The SMP default implementation is now known as z_smp_current_get().
It is no longer inlined which saves significant binary size (about 10%
for some random test case I checked).
Introduce z_current_thread_set() and use it in place of
arch_current_thread_set() for updating the current thread pointer
given this is not necessarily an architecture specific operation.
The architecture specific optimization, when enabled, should only care
about its own things and not have to also update the generic
_current_cpu->current copy.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Mostly a revert of commit b1def7145f ("arch: deprecate `_current`").
This commit was part of PR #80716 whose initial purpose was about providing
an architecture specific optimization for _current. The actual deprecation
was sneaked in later on without proper discussion.
The Zephyr core always used _current before and that was fine. It is quite
prevalent as well and the alternative is proving rather verbose.
Furthermore, as a concept, the "current thread" is not something that is
necessarily architecture specific. Therefore the primary abstraction
should not carry the arch_ prefix.
Hence this revert.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Previously, there were two separate Kconfig definitions
of ARM_MPU located at:
- arch/arm/core/mpu/Kconfig
- arch/arm64/core/cortex_r/Kconfig
This lead to incomplete default settings and unexpected
missing configurations.
This commit combines the two into a single unified definition
now located at arch/common/Kconfig to prevent this.
Signed-off-by: Samuel Chee <samche01@arm.com>
Xtensa cache line sizes aren't an obtuse area of pedantry like they
are in x86. Different cores already in Zephyr are already using
variant cache line sizes (64 and 128 bytes are both common).
And I tripped over this by using the wrong value because the kconfig
was being inherited (incorrectly) from a default somewhere.
Xtensa exposes the correct value in core-isa.h (well, unless the
toolchain/hal gets messed up). Add a check to make sure that our
platform kconfig gets it right.
Note that qemu/dc233c was already getting this wrong, leaving the
value at the kconfig default of zero. That was benign (qemu doesn't
provide any cache emulation for incoherent DMA), but needs to be
fixed.
Signed-off-by: Andy Ross <andyross@google.com>
Fix arch_timing_cycles_get() to prevent overflow on the clock rollover.
The issue is observable on tests/benchmarks/wait_queues and
tests/benchmarks/sched_queues with BENCHMARK_NUM_ITERATIONS is large
enough, e.g. default 1000.
Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
The return value of the system_off call is overwritten by the the call to
pm_s2ram_mark_check_and_clear. As arch_pm_s2ram_suspend needs to specify
why system_off failed, we need to make sure the rv of system_off is moved
to a safe register before calling pm_s2ram_mark_check_and_clear and moved
to r0 als rv of arch_pm_s2ram_suspend when the suspend call exits.
Signed-off-by: Hessel van der Molen <hvandermolen@dexels.com>
With dynamically linked / shared ELF objects the displacement
between file offsets and memory addresses can differ between
sections. Therefore we cannot use .text for all such relocations and
have to locate the respective section instead.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
To obtain a section header reference we don't need to calculate its
location in the ELF image, we already have a pointer to all section
headers, just use them.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
Currently llext_link_plt() calculates offsets to relocation addresses
relative to the .text section and passes them to
arch_elf_relocate_global() and arch_elf_relocate_local(), where then
.text memory address is added to them. Instead it's more logical to
concentrate the entire calculation in the caller, which then also
removes the assumption, that all sections have the same VMA - LMA
offset from those functions.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
With writable LLEXT but without pre-assigned section addresses
.bss section offset in its ELF header will not match .bss eventual
location as it's allocated on the heap. Use llext_loaded_sect_ptr()
to get a correct address in both cases.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
The ARM Thumb-1 instruction set, used by ARMv6-M and ARMv8-M Baseline
cores, does not have a valid encoding for "immediate-to-register move
without affecting flags" instruction (i.e. `mov reg, imm`), and the only
valid variant of immediate-to-register move instruction for it is `movs`,
which affects the flags.
Since none of the register initialisation instructions used here are
flag-sensitive in their context, this commit changes `mov` to `movs`.
This fixes the compilation errors with Clang/LLVM, which is more picky
about the `mov` mnemonic usage and prints out an "invalid instruction"
error when `mov reg, imm` is specified in Thumb-1 mode.
Note that GNU assembler implicitly converts `mov reg, imm` to `movs reg,
imm` when assembling in Thumb-1 mode.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
Moves the arch_swap() declaration out of kernel_arch_interface.h
and into the various architectures' kernel_arch_func.h. This
permits the arch_swap() to be inlined on ARM, but extern'd on
the other architectures that still implement arch_swap().
Inlining this function on ARM has shown at least a +5% performance
boost according to the thread_metric benchmark on the disco_l475_iot1
board.
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
In some Cortex-M3 implementations SCB_VTOR bit[29] is called
the TBLBASE bit.
This enables setting VTOR to an SRAM address for qemu_cortex_m3
Signed-off-by: Grant Ramsay <gramsay@enphaseenergy.com>
Following the binding rename to "nxp,sysmpu", update the Kconfig
option to align with the binding name and to better reflect the
option's purpose.
Signed-off-by: Manuel Argüelles <manuel.arguelles@nxp.com>
When not using CONFIG_XIP (CONFIG_XIP=n)
the FLASH_0 mpu region needs to be removed,
otherwise it will have the default base
address = 0, which means that MPU will try
to configure the region with address 0.
We don't want this as in some situations
address 0 can be a restricted memory region
such as ROM code.
Signed-off-by: Alexandru Lastur <alexandru.lastur@nxp.com>
This reverts commit 7c90f1bca1.
Xen initialization maps enlighten page to Zephyr memory and also
initializes Xen event channels. It is used for communication between
Xen domains and based on interrupt connected to domain virtual GIC.
Moving event channel initialization to arch_kernel_init() make it call
irq_enable() when GIC is not initialized. Since GIC is initialized
on PRE_KERNEL_1 stage, this lead to fatal error during boot.
Revert these changes to make xenvm operable again.
Signed-off-by: Dmytro Firsov <dmytro_firsov@epam.com>
The original idea of the syscall helpers is to workaround
an issue where the compiler could not correctly model register
usage with inline functions. It was supposed to be only used
when there are more then 3 arguments to syscalls. However,
during the original MMU code development, the helper got
expanded to cover "less then 2 arguments syscalls" as a way
for debugging syscall handling code and was never removed.
So fix that now by limiting the helper for syscalls with
more than 3 arguments.
Moreover, instead of using one helper with 6 arguments, now
we have separate implementations for both 4 and 5 arguments
syscall helpers. Now the compiler does not have to generate
code to always handle 6 arguments which saves a few code bytes
as there is no need to pass extra zeroes as arguments.
This has been verified to work with xt-clang RI-2022.10.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This is currently selected by the arch, but some devices (in paticular
xtensa ones which have configurable interrupt setups) might not
actually be able to effect an irq_offload() from within an ISR even
though we have code for it.
Make this a default and not a select so that lower layers can override
the setting.
Signed-off-by: Andy Ross <andyross@google.com>
Introduce `CONFIG_RISCV_GP_PURPOSE` choice to make sure that only
one of `CONFIG_RISCV_GP` or `CONFIG_RISCV_CURRENT_VIA_GP` can be
enabled, instead of relying of dependencies.
To do that, introduce a new
`CONFIG_RISCV_SOC_HAS_GP_RELATIVE_ADDRESSING` that can be selected
by SoC when it implemented global pointer (GP) initialization for
relative addressing in its linker.
`CONFIG_RISCV_GP` will be the default choice when
`CONFIG_RISCV_SOC_HAS_GP_RELATIVE_ADDRESSING=y`
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
Reset the the `gp` register to `_kernel->cpus[i].current` when
`CONFIG_USERSPACE` is enabled on exception to keep it sane.
Updated the testcase to test both `CONFIG_RISCV_GP` and
`CONFIG_RISCV_CURRENT_VIA_GP`.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
In most implements, the msip base address is 0x2000000. But the address
is not fixed in all boards.
Signed-off-by: Yang Jialong <yangjialong@vcore.com>
On Arm Cortex R52, cache segregation policy controls the
number of L1 I/D cache ways that are allocated to Flash
and AXIM interface. Adding Kconfig options for configuring
it.
Writing to IMP_CSCTRL is only permitted before the caches
have been enabled, following a system reset.
Signed-off-by: Dat Nguyen Duy <dat.nguyenduy@nxp.com>
This adds support for the single-vector trapping (SVT) model, defined by
SPARC-V8 Embedded (V8E) Architecture Specification. SVT is available in
most LEON processors.
With single-vector trapping enabled, all traps are vectored through a
single trap vector (TBR.A) rather than one 16-byte entry for each trap
type. This improves memory utilization because the full 4 KiB trap table
is not needed.
This implementation uses a two-level of lookup table to find the handler
for the trap type (0..255).
- Execution time is constant.
- Condition flags are preserved.
- The implementation footprint is 60 bytes .text and 284 bytes .rodata.
For comparison, a non-SVT table is always 4096 .text.
Signed-off-by: Martin Åberg <martin.aberg@gaisler.com>
Enabling CONFIG_FPU and CONFIG_FPU_SHARING requires the
definition of `arch_float_disable` and `arch_float_enable`.
Signed-off-by: Matthias Alleman <matthias.alleman@basalte.be>
`_current` is now functionally equals to `arch_curr_thread()`, remove
its usage in-tree and deprecate it instead of removing it outright,
as it has been with us since forever.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
Implement `arch_curr_thread()` & `arch_set_curr_thread()`
with the global pointer (GP) register.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
Add the following arch-specific APIs:
- arch_curr_thread()
- arch_set_curr_thread()
which allow SMP architectures to implement a faster "get current
thread pointer" than the default provided by the kernel. The 'set'
function is required for the 'get' to work, more on that later.
When `CONFIG_ARCH_HAS_CUSTOM_CURRENT_IMPL` is selected, calls to
`_current` & `k_sched_current_thread_query()` will be redirected to
`arch_curr_thread()`, which ideally should translate into a single
instruction read, avoiding the current
"lock > read CPU > read current thread > unlock" path in SMP
architectures and thus greatly improves the read performance.
However, since the kernel relies on a copy of the "current thread"s on
every CPU for certain operations (i.e. to compare the priority of the
currently scheduled thread on another CPU to determine if IPI should be
sent), we can't eliminate the copy of "current thread" (`current`) from
the `struct _cpu` and therefore the kernel now has to invoke
`arch_set_curr_thread()` in addition to what it has been doing. This
means that it will take slightly longer (most likely one instruction
write) to change the current thread pointer on the current
CPU.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
We hit some unaligned access faults running our internal tests.
Not every 32-bit instruction is 32-bit aligned; some are 16-bit aligned.
Make all reads and writes potentially unaligned to be on the safe side.
Signed-off-by: Ilya Tagunov <Ilya.Tagunov@synopsys.com>
Previously PMP was avaible only with Multithreading, since it's now
available without MT - add extra checks to prevent user/kernel mode
reconfiguration.
Signed-off-by: Volodymyr Fialko <vfialko@marvell.com>
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Without multithreading only two stacks present: ISR and main.
As any stack they also could overflow, so it make sense to add stack
guard for them also.
Remove stack guard dependency on multithreading and mark
`Z_RISCV_STACK_GUARD_SIZE` bytes at the beginning of stack as read-only
region with PMP entry.
Signed-off-by: Volodymyr Fialko <vfialko@marvell.com>
According to the RISC-V Instruction Set Manual: Volume II, Version 20240411
(Section 3.1.6.6), some implementations may choose to track the dirtiness
of the floating-point register state imprecisely by reporting the state to
be dirty even when it has not been modified. This option reflects that.
Also add a filter in `tests/arch/riscv/fpu_sharing/` based on imprecise
FPU state tracking
Signed-off-by: Jakub Wasilewski <jwasilewski@internships.antmicro.com>
Signed-off-by: Filip Kokosinski <fkokosinski@antmicro.com>
This adds a new kconfig for eviction algorithm which needs page
tracking. When enabled, k_mem_paging_eviction_add()/_remove()
and k_mem_paging_eviction_accessed() must be implemented.
If an algorithm does not do page tracking, there is no need to
implement these functions, and no need for the kernel MMU code
to call into empty functions. This should save a few function
calls and some CPU cycles.
Note that arm64 unconditionally calls those functions so
forces CONFIG_EVICTION_TRACKING to be enabled there.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
What is changed?
Use CMSIS SystemCoreClock via a dedicated flag instead of using
soc flags.
Why do we need this change?
This change is part of cleaning soc specific code out of arch folder.
Signed-off-by: Sudan Landge <sudan.landge@arm.com>
When detached sections are used, STB_GLOBAL relocations also have to
be processed depending on the relocation type. This commit unified
STB_GLOBAL and STB_LOCAL flows by making them use the same relocation
type parser, adds support for R_XTENSA_GLOB_DAT and R_XTENSA_JMP_SLOT
type relocations on Xtensa and fixes STT_SECTION address calculation
for such sections.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
Adds compiler flag(s) and some architecture-specific relocations
for ARC. No userspace support, doesn't support all relocations.
Signed-off-by: Lauren Murphy <lauren.murphy@intel.com>
Allow SoC to implement their custom per-core initialization function by
selecting `CONFIG_SOC_PER_CORE_INIT_HOOK` and implement
`soc_per_core_init_hook()`.
Signed-off-by: Maxim Adelman <imax@meta.com>
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
Extend the ARM M-profile suspend-to-RAM implementation to be compatible
with all versions of the M-profile supported by Zephyr: ARMv6-M, ARMv7-M,
and ARMv8-M Baseline.
Signed-off-by: Mathieu Choplain <mathieu.choplain@st.com>
Wrap the CPU register save/restore operations (GPR and special registers)
in macros to make core logic simpler to follow. This is also a preparatory
step to introduce ARMv6-M and ARMv7-M support.
Signed-off-by: Mathieu Choplain <mathieu.choplain@st.com>
Use macros to wrap the interaction between the assembly code and the
struct __cpu_context. This helps making the assembly more readable.
Signed-off-by: Mathieu Choplain <mathieu.choplain@st.com>
Remove all xPSR-related registers from struct __cpu_context, and the
associated save/restore code in S2RAM code, as they are not needed:
* EPSR and IPSR are read-only - they cannot be "restored"
* Bits N, V, Z, C, V, Q, and GE (if DSP Extension is implemented) of APSR
could be restored, but this is not needed as the AAPCS indicates these
bits to be "undefined on entry to or return from a public interface"
Signed-off-by: Mathieu Choplain <mathieu.choplain@st.com>
The currently IPI implementation assumes that CLINT exists in the
system, however, that might not be the case as IPI can be implemented
with PLIC that supports software-triggering as well, such as the Andes
NCEPLIC100.
Refactor the CLINT-based IPI implementations into `ipi_clint.c`, and
create Kconfig that selects the CLINT implementation when
`sifive-clint0` exists and enabled, otherwise default to
`RISCV_SMP_IPI_CUSTOM` which allows OOT implementation. This also
makes way for the upstreaming of non-clint IPI implementation later.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
The STKALIGN mask is not present for CONFIG_ARMV8_M_BASELINE as
well as CONFIG_ARMV8_M_MAINLINE. So filter out that check when
setting the sp for ARM core dumps.
Signed-off-by: Mark Holden <mholden@meta.com>
Reset the gp on exception entry from u-mode to protect the kernel
against a possible rogue user thread.
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
ramfunc region is copied into RAM from FLASH region during XIP init. We
copy from the loadaddr of the region, and were previously loading to the
symbol __ramfunc_start. This is incorrect when using an MPU with
alignment requirements, as the __ramfunc_start symbol may have padding
placed before it in the region. The __ramfunc_start symbol still needs
to be aligned in order to be used by the MPU though, so define a new
symbol __ramfunc_region_start, and use that symbol when copying the
__ramfunc region from FLASH to RAM.
Fixes#75296
Signed-off-by: Daniel DeGrasse <daniel.degrasse@nxp.com>
