During arch_cohere_stacks(), the used portion of the outgoing
thread is cache flushed, and then the unused portion of cache
invalidated. However, this results in the cache line at
the stack pointer being flushed and then invalidated due to
how sys_cache_data_*() operates. If we are swapping back to
the same thread (e.g. after handling interrupt), this cache
line will need to be retrieved again from main memory since
it has already been invalidated. This creates unnecessary
data move between cache and main memory. So create our own
version of cache flushing and invalidation routines just for
arch_cohere_stacks(). Bouns is that these work directly with
bounding addresses and skips the size calculation which should
save a little bit amount of execution time.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This removes _xstack_call1_* trampoline as we can simply use
callx4 to jump to the interrupt handler.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Before cross stack call is setup correctly, we cannot allow
interrupts to be triggered or it may interfere with register
window spilling since we are clobbering registers needed for
that to work. However, there was a brief period where higher
level interrupts could fire due to code writing to PS with
lowered interrupt mask before raising it again. So rework
that part to avoid writing PS with intermediate value, and
now we mask interrupt until everything is setup correctly
before interrupt is enabled again.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
When initializing the stack at thread creation, we should not
set the pointer to privileged stack pointer yet as the thread
can be a kernel thread. Only when a thread is transitioning to
user mode, then we need to set the pointer to point to
the privileged stack. This is a purely semantic change and
should not affect any functionality.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This moves the calculation of the return PC earlier in syscall
entry. There is no need to stash it into BSA, load it, do
the calculation and save it back. We can do the calculation
first and save it in BSA at the same time.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
We should not perform a straight OR operation on INTLEVEL as
we have no idea what existing PS.INTLEVEL is. Also, to avoid
any interferences, we disable interrupts very early during
syscall entrance. So we can remove the OR operation as
PS.INTLEVEL will still have all interrupts masked. Note that
we do not really need to OR PS_WOE into PS as we currently
only support windowed ABI which must have PS_WOE set in PS
anyway.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Syscall entrance makes use of ODD_REG_SAVE but it does not
really need to save FPU registers as it is technically
the same thread and same context. So extract call to
FPU_REG_SAVE to interrupt handling code.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This adds ODD_REG_RESTORE as a counterpart to ODD_REG_SAVE.
Both the code in interrupt handling and syscall exit have
been refactored to use this new macro.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This adds two parameters for ODD_REG_SAVE for scratch register
and BSA pointer, thus allowing a bit more flexibility on how
it can be called.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This removes the call to _xtensa_hifi_save() to save the HiFi
registers during syscalls. During syscall, we are not doing
context switching, and technically it is still the same thread.
There is no need to save HiFi registers.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
The Xtensa ISA reference manual says to do rsync after wsr to
make sure register is updated before rsr. So do that.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Remove CONFIG_XTENSA_INVALIDATE_MEM_DOMAIN_TLB_ON_SWAP as it is
remnant from early MMU enabling work which is not needed as
the page table code is different from early version where
the PTEVADDR would be the same for all memory domains.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
__cxa_atexit was originally defined by the IA64 C++ ABI, but it has been
adopted by other C++ implementations since then. This function is already
present in lib/cpp/api/cpp_dtors.c, which is built by default when C++
support is enabled, so we don't need another copy in the MWDT-specific
code.
Signed-off-by: Keith Packard <keithp@keithp.com>
thread_page_tables_get() is only used when userspace is
enabled. So move it with userspace #ifdef, or else
compiler would complain about it being unused.
Fixes#88421
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Allow relocate_vector_table() to be overwritten. This is necessary if
the IRQ vector table is handled proprietary, e.g. before the zephyr
kernel is initialized. In this case we want this function to be empty to
avoid overriding the previous configuration.
Co-authored-by: Krzysztof Chruściński <krzysztof.chruscinski@nordicsemi.no>
Signed-off-by: Eivind Jølsgard <eivind.jolsgard@nordicsemi.no>
Add support for processing the Fault Status Registers and recoverable
data abort for the cortex A7.
Based on Cortex-A7 MPCore Technical Reference Manual (ARM DDI 0406).
(see https://developer.arm.com/documentation/ddi0406)
Signed-off-by: Julien Racki <julien.racki@st.com>
When Zephyr boot from EL2, D-Cache, I-Cache and MMU should be disabled
by default, but in some cases, for example use U-Boot "go" command
to boot Zephyr Image, Cache and MMU are already been enabled by U-Boot.
So disable them at reset code to make it safety when the code goes
into C code.
Signed-off-by: Jiafei Pan <Jiafei.Pan@nxp.com>
The TOOLCHAIN_DISABLE_WARNING/TOOLCHAIN_ENABLE_WARNING macros are easier
to read and compiler agnostic.
Signed-off-by: Tom Hughes <tomhughes@chromium.org>
Add sys_trace_idle_exit to RISCV cpu_idle functions and allow
enabling CPU_LOAD module for RISCV and disable it for SMP.
Signed-off-by: Krzysztof Chruściński <krzysztof.chruscinski@nordicsemi.no>
What is the change?
- PXN attributes is set for static mpu regions such that
__ramfunc and __ram_text_reloc having userspace code
cannot be executed in privileged mode.
- Updated the notable change section to inform about the change in
behaviour of code executed from __ramfunc and __ram_text_reloc MPU
regions.
Why do we need this change?
- The current static MPU regions allows executing userspace/unprivileged
code from privileged mode which may not be expected and can lead to
secure privileged escalation attacks.
Signed-off-by: Sudan Landge <sudan.landge@arm.com>
What is the change?
This commit adds support for Armv8.1-M MPU architecture's PXN attribute.
This includes support for configuring MPU regions with PXN via
custom mpu_config, devicetree and static mpu_config.
The existing MPU region attribute macros are updated to retain existing
behaviour with this change with an addition of REGION_RAM_ATTR_PXN to be
used if one needs to configure a RAM region with PXN MPU attribute.
Why do we need this change?
Armv8.1-M architecture introduced a new MPU region attribute called
Privilege eXecute Never (PXN).
If an MPU region is configured with the PXN attribute set and if the,
processor attempts to execute code in this region while at
privileged level, a Memory Management Fault exception is triggered.
This ensures that specific tasks are executed only in unprivileged mode
and helps in preventing secure privilege escalation attacks.
Signed-off-by: Sudan Landge <sudan.landge@arm.com>
It has been discovered that direct IPI support does not work
correctly when CONFIG_X2APIC is enabled. Until that can be
fixed, restrict this feature on x86 to platforms that do not
enable CONFIG_X2APIC.
Fixes#87046
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
Since commit 0aa6b1c9de, the 'ldr_parm' pointer is guaranteed to be
valid inside all the functions of the llext_load() call tree.
This commit fixes the only exception of llext_copy_strings(), which was
not passed the 'ldr_parm' pointer, and remove the redundant checks.
No functional change is intended by this commit.
Signed-off-by: Luca Burelli <l.burelli@arduino.cc>
Since 3466dab804 the generic llext_symbol_name() function abstracts
the use of llext_string() for (non-section) symbols. Define a similar
llext_section_name() function and replace current occurrences of
llext_string() with the proper abstraction.
By extending llext_symbol_name(), this commit also allows to print the
correct name of sections that are referred to by a symbol.
Signed-off-by: Luca Burelli <l.burelli@arduino.cc>
IAR doesn't support the __volatile__ keyword, and since the
toolchain.h isn't included by these arm-specific files and
other arm-specific files already uses volatile, I've made
them all use volatile.
Signed-off-by: Lars-Ove Karlsson <lars-ove.karlsson@iar.com>
Previously, the arch_icache_invd_range function simply returned
-ENOTSUP for Cortex A/R. However, this causes a correctness issue when
the processor modifies memory and then executes it as instructions, like
the LLEXT subsystem does. The CPU might not see the up-to-date instructions
in memory.
This caused an LLEXT test failure when running on an emulated Cortex-R5
in QEMU with caches enabled. QEMU doesn't actually emulate the cache,
but it does seem to need the ISB barrier that is executed as part of the
cache invalidate operation in order to handle self-modifying code
properly.
These CPUs do in fact support a selective I-cache invalidate operation
(ICIMVAU) but the CMSIS library doesn't currently support it. For now,
just invalidate the entire I-cache when an icache_invd_range operation
is performed.
Signed-off-by: Robert Hancock <robert.hancock@calian.com>
The current_fp field in the z_kernel structure is only used
by 32-bit x86 (which does not support SMP). As such, it should
reside in the arch specific of section of _kernel.cpus[0].
This also changes the name of 'current_fp' to 'fpu_owner' to
be more consistent with other architectures.
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
Inside map_memory() with double mapping enabled, we should not
be mapping the memory with the incoming attributes as-is since
the incoming address may be on un-cached region but with
caching attribute. So we need to sanitize the attributes
according to the incoming address.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
CONFIG_DYNAMIC_BOOTARGS currently depends on CONFIG_MULTIBOOT_INFO
and CONFIG_BUILD_OUTPUT_EFI which are x86 specific.
This change removes those dependencies so that another architecture
can use CONFIG_DYNAMIC_BOOTARGS without the x86 specifc features.
Signed-off-by: William Tambe <williamt@cadence.com>
native_posix does not exist anymore. This header still applies for
targets setting CONFIG_NATIVE_APPLICATION until this option is removed.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
This comment "moved" over the years as more code was added.
Let's put it where it belongs, and simplify it a bit.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
Add calls to sys_trace_idle_exit when leaving idle state. It is
used to track CPU load.
Signed-off-by: Krzysztof Chruściński <krzysztof.chruscinski@nordicsemi.no>
Multi-level interrupts should be handled in irq init. Only level1
interrupts going directly to interrupt controller need to be
initialized.
Signed-off-by: Yangbo Lu <yangbo.lu@nxp.com>
Adds support for directed IPIs to x86/intel64. Use of direct IPIs
can further reduce the number of schedule IPIs sent and processed
in a system. Fewer IPI related ISRs mean that ...
1. Application code is interrupted less frequently
2. Lower likelihood of scheduler spinlock contention
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
Add a new kconfig option to enable building the new component provided
in the native simulator, nsi_errno, which allows converting the errno
value between libCs.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
What is the change?
- This reverts commit 474d4c3249
Why do we need this change?
- This commit was added because Cortex-M didn't have a valid
stack to make required functionality work however,
the previous commit fixes this and makes interrupt stack
available for use. This removes Arm specific limitation from
these generic APIs so revert the commit to reflect the same.
Signed-off-by: Sudan Landge <sudan.landge@arm.com>
What is the change?
- APIs `pm_s2ram_mark_set` and `pm_s2ram_mark_check_and_clear`,
used to set/check pm_s2ram magic marker,
no longer have Arm specific limitations.
Why is this needed?
- These APIs are generic and should not have arch specific dependency.
Signed-off-by: Sudan Landge <sudan.landge@arm.com>
Currently, RISC-V's architecture-specific relocations assume that
all relocations of type R_RISCV_PCREL_LO12_I and _S are processed
immediately after the R_RISCV_PCREL_HI20 relocation that they
share a relocation target with. While this is the case most of
the time, the RISC-V PSABI specification does not guarantee that.
This commit corrects this by determining the R_RISCV_PCREL_HI20
relocation based on the symbol value of the R_RISCV_PCREL_LO12
relocation, as specified in the PSABI.
Signed-off-by: Eric Ackermann <eric.ackermann@cispa.de>
The RISC-V port of llext requires additional parameters for
handling non-adjacent HI20/LO12 relocations in arch_elf_relocate():
the current extension (struct llext), the current extension loader
(struct llext_loader), the current section header (elf_shdr_t) and
the current symbol (elf_sym_t).
This changes the signature of arch_elf_relocate accordingly.
Signed-off-by: Eric Ackermann <eric.ackermann@cispa.de>
