Support booting from any usable core in systems with partially
fused-off CPUs. Update get_cpu_logic_id to iterate over the actual
number of enabled CPUs using DT_CHILD_NUM_STATUS_OKAY(DT_PATH(cpus))
instead of CONFIG_MP_MAX_NUM_CPUS.
Resize the voting[] array based on DT_CHILD_NUM_STATUS_OKAY to ensure
each CPU can vote correctly.
Signed-off-by: urvashi sharma <urvashis@qti.qualcomm.com>
In the commit 573a712bed patch "arm64:
reset: disable cache and MMU for safety", it disables D-Cache and MMU
for safety, but in some cases, for example the code is loaded into memory
by hardware debugger, we need to flush D-Cache before disable it in
order to make sure the data is coherent in the system, otherwise it
will report "Synchronous Abort" when D-Cache is disabled.
Signed-off-by: Jiafei Pan <Jiafei.Pan@nxp.com>
This reverts commit 573a712bed.
The original patch disable cache and MMU, but in some cases, for example
the code is loaded into memory by hardware debugger, we need to flush
D-Cache before disable it in order to make sure the data is coherent
in the system, otherwise it will report "Synchronous Abort" when D-Cache
is disabled.
This patch revert the old one and the following patch will flush
the cache before disable it.
Signed-off-by: Jiafei Pan <Jiafei.Pan@nxp.com>
Added new configuration item to optionally enable APIs for operation
all data cache, by default these APIs are disabled.
Signed-off-by: Jiafei Pan <Jiafei.Pan@nxp.com>
This patch fixes memory corruption that can happen when running in
multi-thread and multi-core environment with heavy scheduling stress.
In SMP configuration, we must ensure that all thread's context is
stored before writing the switch_handle flag. Otherwise some of the
thread context writes could be delayed after another CPU begins to
schedule this thread which could lead to memory corruptions.
Signed-off-by: Sylvain Chouleur <schouleur@snapchat.com>
Add missing curly braces in if/while/for statements.
This is a style guideline we have that was not enforced in CI. All
issues fixed here were detected by sonarqube SCA.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
GCC and Clang support the undefined behavior sanitizer in any
configuration, the only restriction is that if you want to get nice
messages printed, then you need the ubsan library routines which are only
present for posix architecture or when using picolibc.
This patch adds three new compiler properties:
* sanitizer_undefined. Enables the undefined behavior sanitizer.
* sanitizer_undefined_library. Calls ubsan library routines on fault.
* sanitizer_undefined_trap. Invokes __builtin_trap() on fault.
Overhead for using the trapping sanitizer is fairly low and should be
considered for use in CI once all of the undefined behavior faults in
Zephyr are fixed.
Signed-off-by: Keith Packard <keithp@keithp.com>
When Zephyr boot from EL2, D-Cache, I-Cache and MMU should be disabled
by default, but in some cases, for example use U-Boot "go" command
to boot Zephyr Image, Cache and MMU are already been enabled by U-Boot.
So disable them at reset code to make it safety when the code goes
into C code.
Signed-off-by: Jiafei Pan <Jiafei.Pan@nxp.com>
The RISC-V port of llext requires additional parameters for
handling non-adjacent HI20/LO12 relocations in arch_elf_relocate():
the current extension (struct llext), the current extension loader
(struct llext_loader), the current section header (elf_shdr_t) and
the current symbol (elf_sym_t).
This changes the signature of arch_elf_relocate accordingly.
Signed-off-by: Eric Ackermann <eric.ackermann@cispa.de>
Examine the FP and LR values of each frame to prevent nested
exceptions caused by data aborts. Sometimes, abnormal FP and LR
values are not mapped by the MMU. If we access memory addresses
based on FP or LR values, a CPU data abort will occur. This issue
often accompanies stack overflow occurrences.
Based on this, it is safe to check the validity of memory mapping
with FP and LR values before accessing them.
Signed-off-by: Jungo Lin <jungolin.tw@gmail.com>
This isn't a FUNC_NORETURN function. On a handled
k_sys_fatal_error_handler(), we'll end up back here. And the
CODE_UNREACHABLE was causing code generateion bugs that prevented
return up the stack.
Note that the same pattern exists in MANY other architectures (it
seems like the original was part of x86). But in most cases the arch
assembly isn't actually capable of handling the return to an aborted
thread frame correctly. On arm64, it does actually work. So fix it
here at least.
Signed-off-by: Andy Ross <andyross@google.com>
It's possible to have MEM_ATTR=n and ARM_MPU=y. This fixes the compile
issue with it by compiling out the calls to define the DT mpu regions.
Signed-off-by: Ryan McClelland <ryanmcclelland@meta.com>
`CONFIG_ARM64_ENABLE_FRAME_POINTER` had been deprecated since #72646
for 2 releases and served not functional effect, it's now time to
say goodbye.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
Mostly a revert of commit b1def7145f ("arch: deprecate `_current`").
This commit was part of PR #80716 whose initial purpose was about providing
an architecture specific optimization for _current. The actual deprecation
was sneaked in later on without proper discussion.
The Zephyr core always used _current before and that was fine. It is quite
prevalent as well and the alternative is proving rather verbose.
Furthermore, as a concept, the "current thread" is not something that is
necessarily architecture specific. Therefore the primary abstraction
should not carry the arch_ prefix.
Hence this revert.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Previously, there were two separate Kconfig definitions
of ARM_MPU located at:
- arch/arm/core/mpu/Kconfig
- arch/arm64/core/cortex_r/Kconfig
This lead to incomplete default settings and unexpected
missing configurations.
This commit combines the two into a single unified definition
now located at arch/common/Kconfig to prevent this.
Signed-off-by: Samuel Chee <samche01@arm.com>
This reverts commit 7c90f1bca1.
Xen initialization maps enlighten page to Zephyr memory and also
initializes Xen event channels. It is used for communication between
Xen domains and based on interrupt connected to domain virtual GIC.
Moving event channel initialization to arch_kernel_init() make it call
irq_enable() when GIC is not initialized. Since GIC is initialized
on PRE_KERNEL_1 stage, this lead to fatal error during boot.
Revert these changes to make xenvm operable again.
Signed-off-by: Dmytro Firsov <dmytro_firsov@epam.com>
`_current` is now functionally equals to `arch_curr_thread()`, remove
its usage in-tree and deprecate it instead of removing it outright,
as it has been with us since forever.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
Allow SoC to implement their custom per-core initialization function by
selecting `CONFIG_SOC_PER_CORE_INIT_HOOK` and implement
`soc_per_core_init_hook()`.
Signed-off-by: Maxim Adelman <imax@meta.com>
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
Do not use SYS_INIT for initializing irq_offload when enabled, instead
using a new interface that is called during the boot process for all
architectures.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Add a new call for initializing cache on architectures that need that.
Avoid using SYS_INIT for this and instead call the hook in a fixed place
and run if implemented.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Adds support for all relocation type produced by GCC
on AARCH64 platform using partial linking (-r flag) or
shared link (-fpic and -shared flag).
Signed-off-by: Adam Wojasinski <awojasinski@baylibre.com>
If PSCI is enabled, it will leverage psci reset API to achieve system
reboot, otherwise a weak dump reboot API is provided, and platform
can override these APIs if platform specified implementation provided.
Signed-off-by: Jiafei Pan <Jiafei.Pan@nxp.com>
Convention is to call k_mem_page_fault() with IRQs enabled if they were
enabled when the fault occurred.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
This has two purposes: maintain the accessed and dirty page states, or
call the generic demand paging fault handler otherwise.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
A page table entry used for demand paging is always from the last level
page table and never completely zeroed. Anything else is considered
ARCH_DATA_PAGE_NOT_MAPPED.
Loaded pages use a PTE_PAGE_DESC table entry type. Paged-out pages use a
PTE_INVALID_DESC table entry type and the physical address field is
reused to hold the backing store location token value.
ARCH_DATA_PAGE_ACCESSED corresponds to the AF flag. It is initially
unset and manually set to catch when pages are being accessed and
eventually do something about it.
ARCH_DATA_PAGE_DIRTY corresponds to the lack of the AP_RO flag.
Similarly to the AF flag, pages are initially made read-only to catch
writes.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Let's consider free entries as being completely zeroed. Future patches
for demand paging support will populate entries and still mark them as
"invalid" which should not be considered free. Current code always clear
entries to be freed so no issues there.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Add missing braces to comply with MISRA C:2012 Rule 15.6 and
also following Zephyr's style guideline.
Signed-off-by: Pisit Sawangvonganan <pisit@ndrsolution.com>
Introduce soc and board hooks to replace arch specific code
and replace usages of SYS_INIT for platform initialization.
include/zephyr/platform/hooks.h introduces the hooks to be implemented
by boards and SoCs.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Add Kconfig option to specify Xen interface version to use. This will
make it easier to switch between different versions of Xen.
Signed-off-by: Mykyta Poturai <mykyta_poturai@epam.com>
Before commit baa70d8d36 ("arch/arm64/mmu: fix page table reference
counting part 2") it was possible to perform a partial unmap of a block
mapping. Restore that ability and provide a test case to validate it.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Currently it supports `esf` based unwinding only.
Then, update the exception stack unwinding to use
`arch_stack_walk()`, and update the Kconfigs & testcase
accordingly.
Also, `EXCEPTION_STACK_TRACE_MAX_FRAMES` is unused and
made redundant after this change, so remove it.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
custom arch_cpu_idle and arch_cpu_atomic_idle implementation was done
differently on different architectures. riscv implemented those as weak
symbols, xtensa used a kconfig and all other architectures did not
really care, but this was a global kconfig that should apply to all
architectures.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Commit f7e11649fd ("arch/arm64/mmu: fix page table reference
counting") missed another case where the freeing of a whole table
"branch" didn't take into account the fact that some sub-tables might
be shared and therefore must be cleared only if the reference count is
down to 1.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
None of the moved definitions are meant to be used by any code outside
of arch/arm64/core/mmu.c. Move them away from global scope to the
private header where more such definitions already live.
This is especially relevant as the previous commit fixed some of those
definitions which then caused conflicts with some external SDK that
carries a copy of those original buggy definitions.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Inverting a mask whose type has only 32 bits doesn't produce the
expected result. Fix those to be 64-bit values.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Assembly implementation for z_early_memset() and z_early_memcpy().
Otherwise the compiler will happily replace our C code with a direct
call to memset/memcpy which kind of defeats the purpose.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
We need those simple alternatives to be used during early boot when the
MMU is not yet enabled. However they don't have to be the slowest they
can be. Those functions are mainly used to clear .bss sections and copy
.data to final destination when doing XIP, etc. Therefore it is very
likely for provided pointers to be 64-bit aligned. Let's optimize for
that case.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Commit f7e11649fd ("arch/arm64/mmu: fix page table reference
counting") missed a case where the freeing of a table wasn't propagated
properly to all domains. To fix this, the page freeing logic was removed
from set_mapping() and a del_mapping() was created instead, to be usedby
both by remove_map() and globalize_table().
A test covering this case should definitely be created but that'll come
later. Proper operation was verified through manual debug log
inspection for now.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
This code was never formally tested before... and without the preceding
commit it obviously didn't work either.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Existing code confused table usage and table reference counts together.
This obviously doesn't work. A table with one reference to it and one
populated PTE is not the same as a table with 2 references to it and
no PTe in use.
So split the two concepts and adjust the code accordingly. A page needs
to have its PTE usage count drop to zero before the last reference is
released. When both counts are 0 then the page is free.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Move this to a call in the init process. arch_* calls are no services
and should be called consistently during initialization.
Place it between PRE_KERNEL_1 and PRE_KERNEL_2 as some drivers
initialized in PRE_KERNEL_2 might depend on SMP being setup.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Platforms that support IPIs allow them to be broadcast via the
new arch_sched_broadcast_ipi() routine (replacing arch_sched_ipi()).
Those that also allow IPIs to be directed to specific CPUs may
use arch_sched_directed_ipi() to do so.
As the kernel has the capability to track which CPUs may need an IPI
(see CONFIG_IPI_OPTIMIZE), this commit updates the signalling of
tracked IPIs to use the directed version if supported; otherwise
they continue to use the broadcast version.
Platforms that allow directed IPIs may see a significant reduction
in the number of IPI related ISRs when CONFIG_IPI_OPTIMIZE is
enabled and the number of CPUs increases. These platforms can be
identified by the Kconfig option CONFIG_ARCH_HAS_DIRECTED_IPIS.
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
Make `struct arch_esf` compulsory for all architectures by
declaring it in the `arch_interface.h` header.
After this commit, the named struct `z_arch_esf_t` is only used
internally to generate offsets, and is slated to be removed
from the `arch_interface.h` header in the future.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
