Commit graph

5475 commits

Author SHA1 Message Date
Charlie Xiong 314b72f332 arch: arm64: Re-init HCR_EL2 in z_arm64_el2_init
HCR_EL2 is configured to certain value by some
loaders such as Uboot on some arm64 boards(such as roc_rk3568_pc),
When HCR_EL2.TGE, HCR_EL2.AMO and HCR_EL2.IMO bits are
set to 1, some unpredictable behaviors may occur during
zephyr boot. So we clear these bits to avoid it.

Signed-off-by: Charlie Xiong <1981639884@qq.com>
2023-11-06 10:14:20 +01:00
Anas Nashif 4e396174ce kernel: move syscall_handler.h to internal include directory
Move the syscall_handler.h header, used internally only to a dedicated
internal folder that should not be used outside of Zephyr.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-11-03 11:46:52 +01:00
Torsten Rasmussen 3e82eb976e soc: move arm cortex_m common mpu code to arch/arm/core/mpu
Locate common mpu code together with other arm / nxp mpu code in the
arch folder where it logically belongs.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2023-11-02 09:45:18 +01:00
Yong Cong Sin d7302f417e irq: relocate multi-level irq out of irq.h
Relocate multi-level interrupts APIs out of `irq.h` into
a new file named `irq_multilevel.h` to provide cleaner
separation between typical irq & multilevel ones.

Added preprocessor versions of `irq_to_level_x` as `IRQ_TO_Lx`.

Signed-off-by: Yong Cong Sin <ycsin@meta.com>
2023-10-30 11:43:39 -04:00
Tomasz Bursztyka f5ce4ddc79 arch/x86: Remove useless legacy ACPI code
ACPI is now being handled through ACPICA.

Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
2023-10-27 17:36:31 +01:00
Martin Åberg fd4e66499c SPARC: Update the Flush windows software trap
This commit re-implements the SPARC V8 ABI "Flush windows" software
trap. The trap is generated by C++ compilers for exceptions and also by
the C standard library function longjmp().

There were two issues with the previous implementation:
1. It did reads and writes via the stack pointer of the trap window,
   which is not defined.
2. It executed with traps enabled but without the processor run-time
   state set to safely handle traps. In particular there was no valid
   stack for trap processing. Even though interrupt priority was set to
   highest level, the behavior at other traps was not deterministic. For
   example non-maskable interrupt (15) trap or bus error trap for
   instruction fetch.

This new implementation does not store backup copies of CPU registers to
the stack, and it executes with traps disabled.

Fixes #63901

Signed-off-by: Martin Åberg <martin.aberg@gaisler.com>
2023-10-25 09:54:31 -05:00
Rick Tsao 7c25a7562b riscv: pmp: Fix assertion for PMP misaligned start address and size
Fix assertion to check whether the start address and size align with
PMP granularity.

Signed-off-by: Rick Tsao <rick592@andestech.com>
2023-10-25 10:05:24 +02:00
Lingutla Chandrasekhar 64aa25a8a4 RISCV: Support pm cpu ops for SMP
Add pm cpu ops to call the platform specific implementations for
bringing up secondary cores.

Signed-off-by: Lingutla Chandrasekhar <quic_lingutla@quicinc.com>
2023-10-23 11:36:01 +02:00
Gerard Marull-Paretas 915cb05bb6 dts: drop HAS_DTS
HAS_DTS has become a redundant option. All Zephyr architectures now
select this option, meaning devicetree has become a de-facto
requirement.  In fact, if any board does not provide a devicetree
source, the build system uses an empty stub, meaning the devicetree
machinery always runs.

Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
2023-10-20 12:18:17 -07:00
Daniel Leung d1813e547e arm: using -EAGAIN directly instead of _k_neg_eagain
_k_neg_eagain is there for used in assembly where including
errno.h is not possible. However, the usage in ARM was simply
to assign value to swap_return_value in a C file, which is
no need to use _k_neg_eagain as errno.h can be included.
So change that to use -EAGAIN directly. Saves 4 bytes in
built binaries in rodata.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-10-20 15:14:13 +02:00
Jaxson Han 7904c6f0f3 arch: arm64: Use voting lock for multi-core boot race condition
The exclusive load/store instructions don't work well when MMU and cache
are disabled on some cores e.g. Cortex-A72. Change it to voting lock[1]
to select the primary core when multi-cores boot simultaneously.

The voting lock has reasonable but minimal requirements on the memory
system.

[1] https://www.kernel.org/doc/html/next/arch/arm/vlocks.html

Signed-off-by: Jaxson Han <jaxson.han@arm.com>
2023-10-20 15:09:34 +02:00
Daniel Leung cdd4d84703 xtensa: add custom mem range check functions
This provides custom memory range check functions as
it gets a bit complicated with cached/uncached regions.
These functions are marked as __weak so SoC or board
can override these if needed.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-10-20 15:08:34 +02:00
Tom Burdick 1369a1d152 llext: Cleanups noted in initial PR
There were a few small nits that were pointed out in the initial PR.
Fixes the LOG_ macro in the arm elf implementation, replaces a few stray
mentions of modules in llext.h with extensions.

Signed-off-by: Tom Burdick <thomas.burdick@intel.com>
2023-10-01 09:22:18 +03:00
Andrzej Kuros 3d89d58cb8 arch: arm: aarch32: Introduce z_arm_on_enter_cpu_idle_prepare() hook
Introduce an optional hook to be called when the CPU is made idle.
If needed, this hook can be used to prepare data for upcoming
call to z_arm_on_enter_cpu_idle(). The main difference is that
z_arm_on_enter_cpu_idle_prepare() hook is called before interrupts
are disabled.

Signed-off-by: Andrzej Kuroś <andrzej.kuros@nordicsemi.no>
2023-09-30 18:48:19 +02:00
Tom Burdick 41e0a4a371 llext: Linkable loadable extensions
Adds the linkable loadable extensions (llext) subsystem which provides
functionality for reading, parsing, and linking ELF encoded executable
code into a managed extension to the running elf base image.

A loader interface, and default buffer loader implementation,
make available to the llext subsystem the elf data. A simple management
API provide the ability to load and unload extensions as needed. A shell
interface for extension loading and unloading makes it easy to try.

Adds initial support for armv7 thumb built elfs with very specific
compiler flags.

Signed-off-by: Tom Burdick <thomas.burdick@intel.com>
Co-authored-by: Chen Peng1 <peng1.chen@intel.com>
Co-authored-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
2023-09-29 20:50:38 -04:00
Wilfried Chauveau 8fe6150e0c arch: arm: cortex_m: restrict usage of atomic_operations_c for Cortex-m0/+
Armv8-m baseline support various instruction carrying exclusive-monitor and
acquire-release semantic. By adding this guard we let armv8-m.baseline
fall-back to arch defined or compiler built-in support for atomic
operations.

Signed-off-by: Wilfried Chauveau <wilfried.chauveau@arm.com>
2023-09-29 10:34:20 +02:00
Yonatan Schachter 6e188d9757 arch: Add ARCH_SUPPORTS_ROM_START kconfig symbol
Add an ARCH_SUPPORTS_ROM_START kconfig symbol to mark architectures
that support ROM_START as an argument to zephyr_linker_sources.
This was added so that features relying on this feature could
depend on this kconfig symbol.

Signed-off-by: Yonatan Schachter <yonatan.schachter@gmail.com>
2023-09-28 07:39:09 -04:00
Daniel Leung b4da11f929 gdbstub: xtensa: add support for dc233c core
This adds support for using coredump with Xtensa DC233C core,
which are being used by qemu_xtensa and qemu_xtensa_mmu.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-09-27 19:30:15 -05:00
Daniel Leung 9f9b4a8afa uart: ns16550: use io-mapped DT property for IO port access
The old CONFIG_UART_NS16550_ACCESS_IOPORT has been used to
indicate whether to access the NS16550 UART via IO port
before device tree is used to describe hardware. Now we have
device tree, and we can specify whether a particular UART
needs to be accessed via IO port using property io-mapped.
Therefore, CONFIG_UART_NS16550_ACCESS_IOPORT is no longer
needed (and thus also CONFIG_UART_NS16550_SIMULT_ACCESS).
Remove these two kconfigs and modify code to use device tree
to figure out how to access the UART hardware.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-09-26 12:03:04 +02:00
Daniel Leung ba6c9c2136 xtensa: dc233c: enable backtrace support
Adds the necessary bits to enable backtrace support
for Xtensa DC233C core.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-09-26 08:37:43 +02:00
Daniel Leung 1194a35aa2 xtensa: cast char* to void* during stack dump with %p
cbprintf_package() warns about using char* for %p. So cast
it to void* to avoid the warning.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-09-26 08:37:29 +02:00
Daniel Leung fcf22e59b8 xtensa: mark arch_switch ALWAYS_INLINE
arch_switch() is basically an alias to xtensa_switch() so
we can mark arch_switch() as ALWAYS_INLINE to avoid another
function call, especially when no optimization is used when
debugging.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-09-26 08:37:29 +02:00
Carlo Caione 85c4111002 memory-attr: Rationalize _MASK and _GET(x) macros
Let's make this official: we use the suffix `_MASK` for the define
carrying the GENMASK for the attributes, and the suffix `_GET(x)` for
the actual macro extracting the attributes.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2023-09-22 09:27:57 +02:00
Jaxson Han 0928830409 arch: arm64: Enable stack guard for v8R
Enable stack guard for v8R which is backed by MPU.

Signed-off-by: Jaxson Han <jaxson.han@arm.com>
2023-09-22 09:25:12 +02:00
Jaxson Han 5878898328 arch: arm64: Add stack guard support for MPU
To make the stack guard works well, clean and refine the MPU code. To
save the MPU regions (the number of MPU regions are limited), we choose
to remove the guard region. Comparing to add an individual region to
guard the stack, removing the guard region can at least save 2 regions
per core.

Similarly with userspace, the stack guard will leverage the dynamic
regions switching mechanism which means we need a region switch during
the context switch. Otherwise, the other option is using stack guard
region, but this is very limited since the number of MPU regions is
limited.

Signed-off-by: Jaxson Han <jaxson.han@arm.com>
2023-09-22 09:25:12 +02:00
Jaxson Han d3ec98806d arch: arm64: Refactor the stack relevant macros
Refactor the stack relevant macros to prepare to introduce the stack
guard. Also add comments about the changes related to stack layout.

Signed-off-by: Jaxson Han <jaxson.han@arm.com>
2023-09-22 09:25:12 +02:00
Jaxson Han d4dd47b9c5 arch: arm64: Add stack check at z_arm64_fatal_error
Add the stack check function z_arm64_stack_corruption_check at
z_arm64_fatal_error to handle the stack overflow triggered by the
hardware region.

Signed-off-by: Jaxson Han <jaxson.han@arm.com>
2023-09-22 09:25:12 +02:00
Jaxson Han 1fe9a06d8b arch: arm64: Kconfig: Introduce ARM64_STACK_PROTECTION
Introduce the ARM64_STACK_PROTECTION config. This option leverages the
MMU or MPU to cause a system fatal error if the bounds of the current
process stack are overflowed. This is done by preceding all stack areas
with a fixed guard region. The config depends on MPU for now since MMU
stack protection is not ready.

Signed-off-by: Jaxson Han <jaxson.han@arm.com>
2023-09-22 09:25:12 +02:00
Jaxson Han 1ac3d1cc5e arch: arm64: thread: Clean thread arch when creating thread
Clean the thread->arch during the arch_new_thread to avoid unexpected
behavior. If the thread struct is allocated from heap or in stack, the
data in thread->arch might be dirty.

Signed-off-by: Jaxson Han <jaxson.han@arm.com>
2023-09-22 09:25:12 +02:00
Jaxson Han c57fa2c231 arch: arm64: Fix the cache coherence issue
Accessing mem before mmu or mpu init will cause a cache coherence issue.
To avoid such a problem, move the safe exception stack init function
after the mmu or mpu is initiated.

Also change the data section attribute from INNER_SHAREABLE to
OUTER_SHAREABLE. Otherwise there will be a cache coherence issue during
the memory regions switch. Because we are using background region to do
the regions switch, and the default background region is
OUTER_SHAREABLE, if we use INNER_SHAREABLE as the foreground region,
then we have to flush all cache regions to make sure the cached values
are right. However, flushing all regions is too heavy, so we set
OUTER_SHAREABLE to fix this issue.

Signed-off-by: Jaxson Han <jaxson.han@arm.com>
2023-09-22 09:25:12 +02:00
Dmytro Firsov ac0f731b2e xen: add Kconfig option for Zephyr on Dom0less setups
Dom0less is Xen mode without privileged domain. All guests are created
according to hypervisor device tree configuration on boot. Thus, there
is no Dom0 with console daemon, that usually manages console output
from domains.

Zephyr OS contains 2 serial drivers related to Xen hypervisor: regular
with console shared page and consoleio-based. The first one is for
setups with console daemon and usually was used for Zephyr DomU guests.
The second one previously was used only for Zephyr Dom0 and had
corresponding Kconfig options. But consoleio is also used as interface
for DomU output on Dom0less setups and should be configurable without
XEN_DOM0 option.

Add corresponding XEN_DOM0LESS config to Xen Kconfig files and proper
dependencies in serial drivers.

Signed-off-by: Dmytro Firsov <dmytro_firsov@epam.com>
Co-authored-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
2023-09-18 10:43:45 +01:00
Carlo Caione e4a125b6a4 dt: Make zephyr,memory-attr a capabilities bitmask
This is the final step in making the `zephyr,memory-attr` property
actually useful.

The problem with the current implementation is that `zephyr,memory-attr`
is an enum type, this is making very difficult to use that to actually
describe the memory capabilities. The solution proposed in this PR is to
use the `zephyr,memory-attr` property as an OR-ed bitmask of memory
attributes.

With the change proposed in this PR it is possible in the DeviceTree to
mark the memory regions with a bitmask of attributes by using the
`zephyr,memory-attr` property. This property and the related memory
region can then be retrieved at run-time by leveraging a provided helper
library or the usual DT helpers.

The set of general attributes that can be specified in the property are
defined and explained in
`include/zephyr/dt-bindings/memory-attr/memory-attr.h` (the list can be
extended when needed).

For example, to mark a memory region in the DeviceTree as volatile,
non-cacheable, out-of-order:

   mem: memory@10000000 {
       compatible = "mmio-sram";
       reg = <0x10000000 0x1000>;
       zephyr,memory-attr = <( DT_MEM_VOLATILE |
			       DT_MEM_NON_CACHEABLE |
			       DT_MEM_OOO )>;
   };

The `zephyr,memory-attr` property can also be used to set
architecture-specific custom attributes that can be interpreted at run
time. This is leveraged, among other things, to create MPU regions out
of DeviceTree defined memory regions on ARM, for example:

   mem: memory@10000000 {
       compatible = "mmio-sram";
       reg = <0x10000000 0x1000>;
       zephyr,memory-region = "NOCACHE_REGION";
       zephyr,memory-attr = <( DT_ARM_MPU(ATTR_MPU_RAM_NOCACHE) )>;
   };

See `include/zephyr/dt-bindings/memory-attr/memory-attr-mpu.h` to see
how an architecture can define its own special memory attributes (in
this case ARM MPU).

The property can also be used to set custom software-specific
attributes. For example we can think of marking a memory region as
available to be used for memory allocation (not yet implemented):

   mem: memory@10000000 {
       compatible = "mmio-sram";
       reg = <0x10000000 0x1000>;
       zephyr,memory-attr = <( DT_MEM_NON_CACHEABLE |
			       DT_MEM_SW_ALLOCATABLE )>;
   };

Or maybe we can leverage the property to specify some alignment
requirements for the region:

   mem: memory@10000000 {
       compatible = "mmio-sram";
       reg = <0x10000000 0x1000>;
       zephyr,memory-attr = <( DT_MEM_CACHEABLE |
			       DT_MEM_SW_ALIGN(32) )>;
   };

The conventional and recommended way to deal and manage with memory
regions marked with attributes is by using the provided `mem-attr`
helper library by enabling `CONFIG_MEM_ATTR` (or by using the usual DT
helpers).

When this option is enabled the list of memory regions and their
attributes are compiled in a user-accessible array and a set of
functions is made available that can be used to query, probe and act on
regions and attributes, see `include/zephyr/mem_mgmt/mem_attr.h`

Note that the `zephyr,memory-attr` property is only a descriptive
property of the capabilities of the associated memory  region, but it
does not result in any actual setting for the memory to be set. The
user, code or subsystem willing to use this information to do some work
(for example creating an MPU region out of the property) must use either
the provided `mem-attr` library or the usual DeviceTree helpers to
perform the required work / setting.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2023-09-15 12:46:54 +02:00
Dmytro Firsov 3581527b53 xen: implement Xen domain control interface
Add Xen domctl API implementation for Zephyr as control domain.

Previously Zephyr OS was used as unprivileged Xen domain (Domain-U),
but it also can be used as lightweight Xen control domain (Domain-0).
To implement such fuctionality additional Xen interfaces are needed.
One of them is Xen domain controls (domctls) - it allows to create,
configure and manage Xen domains.

Also, used it as a possibility to update files copyright and licenses
identifiers in touched files.

Signed-off-by: Dmytro Firsov <dmytro_firsov@epam.com>
2023-09-15 11:15:00 +01:00
Dmytro Firsov c9d2fb7d40 xen: refactor Xen hypervisor Kconfig options
Xen-related Kconfig options were highly dependand on BOARD/SOC xenvm.
It is not correct because Xen support may be used on any board and SoC.

So, Kconfig structure was refactored, now CONFIG_XEN is located in
arch/ directory (same as in Linux kernel) and can be selected for
any Cortex-A arm64 setup (no other platforms are currently supported).

Also remove confusion in Domain 0 naming: Domain-0, initial domain,
Dom0, privileged domain etc. Now all options related to Xen Domain 0
will be controlled by CONFIG_XEN_DOM0.

Signed-off-by: Dmytro Firsov <dmytro_firsov@epam.com>
2023-09-15 11:15:00 +01:00
Carles Cufi ef041c6b98 modules: acpica: Fix header inclusion issue
After 79d0bf39b8 was merged, the inclusion
of <zephyr/acpi/acpi.h> with CONFIG_ACPI=n caused a build failure because
<acpica/source/include/acpi.h> could no longer be included due to the
inlcude path not being injected anymore.

Fix this by guarding the header inclusion when CONFIG_ACPI
is not set.

Fixes #62679.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2023-09-15 05:41:04 -04:00
Gerard Marull-Paretas 691facc20f include: always use <> for Zephyr includes
Double quotes "" should only be used for local headers.

Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
2023-09-14 13:49:58 +02:00
Laurentiu Mihalcea b8d487e54b arch: Add support for dynamically disconnecting shared interrupts
This commit provides the users a way to disconnect dynamic
interrupts. This is needed because we don't want to keep
piling up ISR/arg pairs until the number of registrable
clients is reached.

This feature is only relevant for shared and dynamic interrupts.

Signed-off-by: Laurentiu Mihalcea <laurentiu.mihalcea@nxp.com>
2023-09-14 08:35:12 +02:00
Laurentiu Mihalcea 02f52503bb arch: Add support for dynamic shared interrupts
This works by overwriting z_isr_install()'s definition
(possible since the symbol is now weak) with our own definiton.

Whenever trying to register a new ISR/arg pair, z_isr_install()
will check to see if the interrupt line is already in use. If it's
not then it will not share the interrupt and will work exactly
as the old z_isr_install(), meaning it will just write the new
ISR/arg pair to _sw_isr_table.

If the interrupt line is already being used by an ISR/arg pair
then that line will become shared, meaning we'll overwrite
_sw_isr_table with our own (z_shared_isr, z_shared_sw_isr_table[irq])
pair.

Signed-off-by: Laurentiu Mihalcea <laurentiu.mihalcea@nxp.com>
2023-09-14 08:35:12 +02:00
Laurentiu Mihalcea 30d362dbac arch: common: sw_isr_common: Move table index computing logic to function
Since the shared IRQ code will also use the same logic to compute
the _sw_isr_table index, move the computing logic to a separate
function: z_get_sw_isr_table_idx(), which can be used by other
modules.

Signed-off-by: Laurentiu Mihalcea <laurentiu.mihalcea@nxp.com>
2023-09-14 08:35:12 +02:00
Laurentiu Mihalcea 017cf89a83 arch: Add support for static shared interrupts
This commit introduces all the necessary changes for
enabling the usage of shared interrupts.

This works by using a second interrupt table: _shared_sw_isr_table
which keeps track of all of the ISR/arg pairs sharing the same
interrupt line. Whenever a second ISR/arg pair is registered
on the same interrupt line using IRQ_CONNECT(), the entry in
_sw_isr_table will be overwriten by a
(shared_isr, _shared_sw_isr_table[irq]) pair. In turn, shared_isr()
will invoke all of the ISR/arg pairs registered on the same
interrupt line.

This feature only works statically, meaning you can only make use
of shared interrupts using IRQ_CONNECT(). Attempting to dynamically
register a ISR/arg pair will overwrite the hijacked _sw_isr_table
entry.

Signed-off-by: Laurentiu Mihalcea <laurentiu.mihalcea@nxp.com>
2023-09-14 08:35:12 +02:00
Patryk Duda a17fe86de2 arch: common: Force linker to fill empty spaces in rom_start with 0x00
LLVM LLD fills empty spaces (created using ALIGN() or moving the
location counter) in executable segments with TrapInstr pattern,
e.g. for ARM the TrapInstr pattern is 0xd4d4d4d4. GNU LD fills
empty spaces with 0x00 pattern.

We may want to have some section (e.g. rom_start) filled with 0x00,
e.g. because MCU can interpret the pattern as a configuration data.

Signed-off-by: Patryk Duda <pdk@semihalf.com>
2023-09-13 11:46:56 +02:00
Huifeng Zhang 37a6118372 arch: arm: Separate common kernel_arch_func code
This commit separates kernel_arch_func.h into two header file,
'cortex_a_r/kernel_arch_func.h' and 'cortex_m/kernel_arch_func.h', it
also removes some functions which is empty.

Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
2023-09-13 10:08:05 +01:00
Huifeng Zhang 9ba3d31310 arch: arm: Separate common cpu_idle codes
This commit separates cpu_idle.S into two asm files,
'cortex_a_r/cpu_idle.S' and 'cortex_m/cpu_idle.S'.

Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
2023-09-13 10:08:05 +01:00
Huifeng Zhang 7e5e08b3ab arch: arm: Separate common thread code
This commit separates thread.c into two source files,
'cortex_a_r/thread.c' and 'cortex_m/thread.c, it also introduces some
changes.

1. Migrate 'thread.c' and 'cortex_m/thread.c'.
2. Migrate 'thread.c' and 'cortex_a_r/thread.c'
3. Remove the 'z_arm_mpu_stack_guard_and_fpu_adjust' function as this is
   obviously written for Cortex-M architecture.
4. Remove the 'z_arm_prepare_switch_to_main' function as this is only
   used by Cortex-M architecture.

Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
2023-09-13 10:08:05 +01:00
Huifeng Zhang 7dbbc49c07 arch: arm: Separate common prep_c code
This commit Separate 'prep_c.c' into two file based on the architecture,
one is 'cortex_m/prep_c.c', the other is 'cortex_a_r/prep_c.c'

Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
2023-09-13 10:08:05 +01:00
Huifeng Zhang 9b739bc0f2 arch: arm: Separate common irq_manage and isr_wrapper code
There are too many differences between Cortex-A/R and Cortex-M on irq
code, e.g. Cortex-A/R use GIC and Cortex-M uses NVIC. For reducing
the complexity and easier to maintain, this commit separates irq_manage.c
and isr_wrapper.S into two different parts based on the architecture.

This commit also Removes the part related to the option
'CONFIG_ARM_SECURE_FIRMWARE' in 'cortex_a_r/irq_manage.c' because
this code is written for the Cortex-M architecture.

Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
2023-09-13 10:08:05 +01:00
Huifeng Zhang c86785297c arch: arm: Separate common swap code
There are too many differences between Cortex-A/R and Cortex-M on swap
code. For reducing the complexity and easier to maintain, this commit
introduces the following major changes

  1. Separate swap.c and swap_helper.S into two different parts based on
    the architecture.

  2. Rename 'z_arm_pendsv' to 'z_arm_do_swap' for Cortex-A/R.

  3. Removes the part related to the option 'CONFIG_BUILTIN_STACK_GUARD'
    in 'cortex_a_r/swap_helper.S' because this code is written for
    the Cortex-M architecture.

Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
2023-09-13 10:08:05 +01:00
Huifeng Zhang 1e79cfcd2e arch: arm: cortex-m: Move irq_relay.S to the cortex-m directory
'irq_relay.S' is only used for Arm Cortex-M architecture, so it's better
to place it to the 'arch/arm/core/cortex_m' directory.

Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
2023-09-13 10:08:05 +01:00
Huifeng Zhang 2c22e83dfb include: arch: arm: Remove aarch32 directory
This commit follows the parent commit work.

This commit introduces the following major changes.

  1. Move all directories and files in 'include/zephyr/arch/arm/aarch32'
    to the 'include/zephyr/arch/arm' directory.

  2. Change the path string which is influenced by the changement 1.

Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
2023-09-13 10:08:05 +01:00
Huifeng Zhang df41deac1c arch: arm: Remove aarch32 directory
It doesn't make sense to keep the aarch32 directory in the
'arch/arm/core' directory as the aarch64 has been moved out.

This commit introduces the following major changes.

  1. Move all directories and files in 'arch/arm/core/aarch32' to
    'arch/arm/core' and remove the 'arch/arm/core/aarch32' directory.
  2. Move all directories and files in 'arch/include/aarch32' to
    'arch/include' and remove the 'arch/include/aarch32' directory.
  3. Remove the nested including in the 'arch/include/kernel_arch_func.h'
    and 'arch/include/offsets_short_arch.h' header files.
  4. Change the path string which is influenced by the changement 1
    and 2.

Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
2023-09-13 10:08:05 +01:00
Anas Nashif 1fd3171f7b arch: remove wait_q.h include
This include is not used directly in those files, so remove it.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-09-12 12:55:36 -04:00
Keith Short 7695a4d5e6 x86: Remove unused ACPI include
The prep_c.c source doesn't depend on the ACPI library.  As ACPI support
now requires the ACPICA module, the extra header breaks projects using
x86 without ACPI support.

Signed-off-by: Keith Short <keithshort@google.com>
2023-09-09 00:20:20 +02:00
Najumon B.A a68204d8b8 arch: x86: update with new acpica lib interface
The old acpi implimentation is replaced with acpica interface
and updated x86 arch porting with the new interface.

Signed-off-by: Najumon B.A <najumon.ba@intel.com>
2023-09-07 09:42:38 +02:00
Alexander Razinkov 176713abfe arch: riscv: Trap handler alignment configuration
RISC-V Spec requires minimum alignment of trap handling code to be
dependent from MTVEC.BASE field size. Minimum alignment for RISC-V
platforms is 4 bytes, but maximum is platform or application-specific.

Currently there is no common approach to align the trap handling
code for RISC-V and some platforms use custom wrappers to align
_isr_wrapper properly.

This change introduces a generic solution,
CONFIG_RISCV_TRAP_HANDLER_ALIGNMENT configuration option which sets
the alignment of a RISC-V trap handling code.

The existing custom solutions for some platforms remain operational,
since the default alignment is set to minimal possible (4 bytes)
and will be overloaded by potentially larger alignment of custom solutions.

Signed-off-by: Alexander Razinkov <alexander.razinkov@syntacore.com>
2023-09-05 16:16:46 +02:00
Alberto Escolar Piedras b021c7611d native simulator: Add property to collect libraries to link w runner
Add a property to the native_simulator target, to collect
the libraries we want to link with the runner, instead of
abusing the link options to pass them.

Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
2023-09-04 13:16:58 +02:00
Umar Nisar 31a6594212 drivers: loapic: add device tree support for loapic
As per #26393, Local APIC is using Kconfig based option for
the base address. This patch adds DTS binding support in the driver,
just like its conunter part I/O APIC.

Signed-off-by: Umar Nisar <umar.nisar@intel.com>
2023-09-01 16:36:18 +02:00
Huifeng Zhang b38e1cd62a boards: fvp_baser_aemv8r_aarch32: disable d-cache
Disable d-cache on the fvp_baser_aemv8r_aarch32 platform because there
are some issues when d-cache is enabled for now.

Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
2023-09-01 13:23:26 +02:00
Huifeng Zhang 1b9c824f0b arch: arm64: arm_mpu: enable d-cache
Enable d-cache at the stage of the EL1 initialization and fix some cache
coherence issues.

Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
2023-09-01 13:23:26 +02:00
Anas Nashif 6baa622958 arch: move exc_handle.h under zephyr/arch/common
This header is private and included only in architecture code, no need for
it to be in the top of the public include directory.

Note: This might move to a more private location later. For now just
cleaning up the obvious issues.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-08-31 09:19:19 -04:00
Chirag Kochar b05c2f4718 arch: arm64: core: Fix issue IPI for non-booted core
During Zephyr boot with SMP enabled,
while z_smp_init is not completed yet and only boot core is running,
incorrect code in broadcast_ipi API will cause following:
1. Generate IPI even if other cores are not booted.
2. Incorrect setting of sgi1r register.
3. All the affinity(1/2/3) value will be incorrect.

Signed-off-by: Chirag Kochar <chirag.kochar@intel.com>
2023-08-31 10:24:58 +02:00
Chad Karaginides f5ff62f35a arch: arm64: Reserved Cores
Enhanced arch_start_cpu so if a core is not available based on pm_cpu_on
return value, booting does not halt.  Instead the next core in
cpu_node_list will be tried.  If the number of CPU nodes described in the
device tree is greater than CONFIG_MP_MAX_NUM_CPUS then the extra cores
will be reserved and used if any previous cores in the cpu_node_list fail
to power on.  If the number of cores described in the device tree matches
CONFIG_MP_MAX_NUM_CPUS then no cores are in reserve and booting will
behave as previous, it will halt.

Signed-off-by: Chad Karaginides <quic_chadk@quicinc.com>
2023-08-28 15:57:19 +02:00
Hou Zhiqiang 3b98cfe1c4 arm64: mmu: add Non-cacheable normal memory mapping support
In some shared-memory use cases between Zephyr and other parallel
running OS, for data coherent, the non-cacheable normal memory
mapping is needed.

Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
2023-08-28 11:41:51 +02:00
Daniel Leung ca23a5f0cf xtensa: mmu: allow SoC to do additional MMU init steps
This adds a function arch_xtensa_mmu_post_init() which can
be implemented on the SoC layer to perform additional MMU
initialization steps if necessary.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-08-26 16:50:40 -04:00
Daniel Leung 088a31e2bf xtensa: mmu: preload ITLB for VECBASE before restoring...
...VECBASE during MMU initialization. This is to make sure
that we can use the TLB miss handling in the exception
vector after we have moved back the VECBASE during MMU
initialization. Or else we would be forever stuck in ITLB
miss because the exception vectors are not in TLB and we
cannot populate the TLB because those vectors are not in
TLB.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-08-26 16:50:40 -04:00
Daniel Leung 40f2486b68 xtensa: mmu: rename MMU_KERNEL_RING to Z_XTENSA_KERNEL_RING...
...and move it to xtensa_mmu_priv.h.

This would allow the SoC layer to use the RING number if needed.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-08-26 16:50:40 -04:00
Daniel Leung 18eb17f4cd xtensa: mmu: add arch_reserved_pages_update
This adds arch_reserved_pages_update() which is called in
k_mem_manage_init() to reserve some physical pages so they
are not re-mapped. This is due to Zephyr's linker scripts
for Xtensa which often puts something before z_mapped_start
(aka .text, for example, vecbase). That space needs to be
reserved or else k_mem_map() would be mapping those that
could result in faults.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-08-26 16:50:40 -04:00
Daniel Leung 4778c13bbe xtensa: mmu: handle all data TLB misses in double exception
Instead of only handling data TLB misses for VECBASE, change it
to handle all data TLB misses in the double exception handler.
It is because we may encounter data TLB misses when trying to
preload page table entries inside user exception handler.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2023-08-26 16:50:40 -04:00
Flavio Ceolin c723d8b8d3 xtensa: Add missing synchronization
rsync after writing MISC0..3 or EXCSAVE1..7 registers is
needed before reading them.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2023-08-26 16:50:40 -04:00
Daniel Leung 24148718fc xtensa: mmu: cache common data and heap if !XTENSA_RPO_CACHE
If CONFIG_XTENSA_RPO_CACHE is not enabled, it can be assumed
that memory is not double mapped in hardware for cached and
uncached access. So we can specify those regions to have
cache via TLB.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-08-26 16:50:40 -04:00
Daniel Leung b6ccbae58d xtensa: mmu: use _image_ram_start/end for data region
Simply using __data_start and __data_end is not enough as
it leaves out kobject regions which is supposed to be
near .data section. So use _image_ram_start and
_image_ram_end instead to enclose data, bss and various
kobject regions (among others).

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-08-26 16:50:40 -04:00
Daniel Leung 257404a143 xtensa: mmu: init: only clear enough entries in way 6
During MMU initialization, we clear TLB way 6 to remove all
identity mapping. Depending on CPU configuration, there are
certain number of entries per way. So use the number from
core-isa.h to clear enough entries instead of hard-coded
number 8. Specifying an entry number outside of permitted
range may result in CPU reacting in weird way so better to
avoid that.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-08-26 16:50:40 -04:00
Daniel Leung 614e64325d xtensa: mmu: no longer identity map the first 512MB
This removes the identity map of the first 512MB in TLB way 6.
Or else it would interfere with mapped entries resulting in
double mapped TLB exception.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-08-26 16:50:40 -04:00
Daniel Leung b5016714b0 xtensa: mmu: handle TLB misses during user exception
This adds code to deal with TLB misses as these comes as
level 1 interrupts.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-08-26 16:50:40 -04:00
Daniel Leung 98ffd1addd xtensa: crt1: call z_xtensa_mmu_init
MMU needs to be initialized before going in to C, so
z_xtensa_mmu_init() is called in crt1.S before call
to z_cstart(). Note that this is the default case
and crt1.S can be disabled if board and SoC desire
to do so.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-08-26 16:50:40 -04:00
Daniel Leung 38d4b78724 xtensa: mmu: remove printing vaddr registers during exception
Turns out not all MMU enabled Xtensa cores have vaddrstatus,
vaddr0 and vaddr1. And there does not seem to be a way to
determine whether they are available. So remove them from
the exception printout for now.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-08-26 16:50:40 -04:00
Alberto Escolar Piedras 232b4b0a7a native_sim: Define path to native_simulator on arch cmake files
To allow reusing the path between boards,
instead of defining it in the board, let's define it in
the arch cmake file.

Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
2023-08-24 15:40:00 +02:00
Alberto Escolar Piedras 9eeb78d86d COVERAGE: Fix COVERAGE_GCOV dependencies
CONFIG_COVERAGE has been incorrectly used to
change other kconfig options (stack sizes, etc)
code defaults, as well as some samples behaviour,
which should not have dependend on it.

Instead those should have depended on COVERAGE_GCOV,
which, being the one which adds special code and
temporary RAM storage for embedded targets,
require changes to many features.

When building for the native targets, all this was
unnecessary.

=> Fix the dependency.

Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
2023-08-24 15:36:31 +02:00
Gerard Marull-Paretas 9c961571a2 modules: cmsis: move glue code to modules/cmsis
The CMSIS module glue code was part of arch/ directory. Move it to
modules/cmsis, and provide a single entry point for it: cmsis_core.h.
This entry header will include the right CMSIS header (M or A/R).

To make this change possible, CMSIS module Kconfig/CMake are declared as
external, allowing us to add a new Zephyr include directory.

All files including CMSIS have been updated.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2023-08-24 13:20:21 +02:00
Florian La Roche 5727503489 style: move ALWAYS_INLINE to the beginning to resolve compiler warnings
With gcc from the zephyr sdk and -Wold-style-declaration is giving this
output:
zephyr/arch/arm/core/aarch32/cortex_a_r/fault.c:101:1: warning:
  'inline' is not at beginning of declaration [-Wold-style-declaration]
  101 | static void ALWAYS_INLINE
                    z_arm_fpu_caller_save(struct __fpu_sf *fpu)
      | ^~~~~~

I searched to all of the source code to find these further occurances
where inline is not at the beginning of a function declaration.

Signed-off-by: Florian La Roche <Florian.LaRoche@gmail.com>
2023-08-23 14:44:23 +02:00
Florian La Roche 533c913b86 arm64: core/prep_c.c func prototype before function itself
Move the function prototype before declaration of the function itself.
Maybe the prototype could be removed altogether?

Signed-off-by: Florian La Roche <Florian.LaRoche@gmail.com>
2023-08-23 10:02:54 +02:00
Andrei Emeltchenko 6e69d759a5 x86: acpi: Remove unneeded assignment
Is rsdp_phys is assigned in the loop below code jumps to found label,
otherwise we return and value rsdp_phys is never used after 0
assignment.

Signed-off-by: Andrei Emeltchenko <andrei.emeltchenko@intel.com>
2023-08-22 10:46:41 +01:00
Daniel Leung e38fc6de8a cmake: enable -Wshadow partially for in-tree code
This enables -Wshadow to warn about shadow variables on
in tree code under arch/, boards/, drivers/, kernel/,
lib/, soc/, and subsys/.

Note that this does not enable it globally because
out-of-tree modules will probably take some time to fix
(or not at all depending on the project), and it would be
great to avoid introduction of any new shadow variables
in the meantime.

Also note that this tries to be done in a minimally
invasive way so it is easy to revert when we enable
-Wshadow globally. Source files under modules/, samples/
and tests/ are currently excluded because there does not
seem to be a trivial way to add -Wshadow there without
going through all CMakeLists.txt to add the option
(as there are 1000+ files to change).

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-08-22 11:39:58 +02:00
Daniel Leung 16bd0c861f x86: rename shadow variables
Rename shadow variables found by -Wshadow.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-08-22 11:39:58 +02:00
Jaxson Han 0f7bbff050 arch: arm64: Refine v8R AArch64 MPU regions switch
The current mechanism of the MPU region switching configures and
reprograms the regions (including inserting, splitting the dynamic
region, and flushing the regions to the registers) every time during the
context switch. This, not only causes a large usage of the kernel stack
but also a lower performance.

To improve it, move the configuration operations ahead to make sure the
context swtich only flushes the current thread regions to the registers
and does not configure the regions anymore. To achieve this, configure
the regions during any operations related to partitions (partition
add/remove, and domain add/remove thread), flush the sys_dyn_regions if
the current thread is the privileged thread, and flush the thread's own
regions if it's a user thread.

Signed-off-by: Jaxson Han <jaxson.han@arm.com>
2023-08-21 13:27:07 +02:00
Jaxson Han c039e05724 arch: arm64: mpu: Use BR mode to flush the MPU regions
Using BR(background region) during the flushing regions instead of
enabling/disabling the MPU which is a heavy operation.

Signed-off-by: Jaxson Han <jaxson.han@arm.com>
2023-08-21 13:27:07 +02:00
YuLong Yao 823e6b70d2 arch: xtensa: Implement arch_float_enable&disable
Every arch must have arch_float_enable&disable functions.

Signed-off-by: YuLong Yao <feilongphone@gmail.com>
2023-08-21 10:10:06 +02:00
Manuel Argüelles 26d398548f arm: cortex_r: support recoverable data abort
Add support for processing the Fault Status Registers and recoverable
data abort for Armv8-R AArch32.

Based on Arm Architecture Reference Manual Supplement Armv8, for the
Armv8-R AArch32 architecture profile (ARM DDI 0568).

Signed-off-by: Manuel Argüelles <manuel.arguelles@nxp.com>
2023-08-21 10:07:29 +02:00
Grant Ramsay 20e297a295 arch: arm64: Make ARM64_VA_BITS/ARM64_PA_BITS named choices
Naming these choices allows setting a default value in defconfig.

Signed-off-by: Grant Ramsay <gramsay@enphaseenergy.com>
2023-08-18 10:13:12 +02:00
Mykola Kvach c9b534c4eb arch: arm64: mmu: avoid using of set/way cache instructions
Architecturally, Set/Way operations are not guaranteed to affect all
caches prior to the PoC, and may require other IMPLEMENTATION DEFINED
maintenance (e.g. MMIO control of system-level caches).

First of all this patch was designed for Xen domain Zephyr build, set/way
ops are not easily virtualized by Xen. S/W emulation is disabled, because
IP-MMU is active for Dom0. IP-MMU is a IO-MMU made by Renesas, as any good
IO-MMU, it shares page-tables with CPU. Trying to emulate S/W with IP-MMU
active will lead to IO-MMU faults. So if we build Zephyr as a Xen Initial
domain, it won't work with cache management support enabled.

Exposing set/way cache maintenance to a virtual machine is unsafe, not
least because the instructions are not permission-checked, but also
because they are not broadcast between CPUs.

In this commit, VA data invalidate invoked after every mapping instead of
using set/way instructions on init MMU. So, it was easy to delete
sys_cache_data_invd_all from enable MMU function, becase every adding of
a new memory region to xclat tabes will cause invalidating of this memory
and in this way we sure that there are not any stale data inside.

Signed-off-by: Mykola Kvach <mykola_kvach@epam.com>
2023-08-17 15:15:15 +02:00
Daniel Leung d59944465a arch: common: use zephyr_library for all source files
The zephyr_library() used to only be called if
CONFIG_GEN_ISR_TABLES is enabled. Which means that any
zephyr_library_*() calls are putting things into another
library if CONFIG_GEN_ISR_TABLES is disabled. So pull
the call to zephyr_library() to outside. Also moves
semihost.c into the library.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-08-16 15:00:49 +02:00
Alberto Escolar Piedras d4e48d5feb POSIX arch: Fix literal floating comparison in 32bit targets
When building the 32bit native board targets variants
for x86(-64) hosts, gcc will promote float literals to double
(See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=92875 )

This can result in unexpected comparison differences.

This is due to the compiler using the 8087 float mode by
default.
Instead let's tell the compiler to use the SSE float path,
which is the default for 64 bit x86-64 builds.

The assumption that any x86 host used for development
will have SSE support should be safe enough.

For more background see
https://github.com/zephyrproject-rtos/zephyr/issues/61345

Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
2023-08-14 13:37:27 +00:00
Daniel Leung 6b740b20de riscv: renames shadow variables
In print_pmp_entries(), start and end are function arguments
and yet another start and end are declared inside the for
loop. So rename the function arguments to fix shadow variables
warning.

The changes in csr_*() macros are needed to avoid shadowing
__v when nesting those functions together, for example,
csr_write(..., csr_read(...)).

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-08-10 08:14:12 +00:00
Flavio Ceolin d16c5b9048 kernel: canaries: Allow using TLS to store it
Add new option to use thread local storage for stack
canaries. This makes harder to find the canaries location
and value. This is made optional because there is
a performance and size penalty when using it.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2023-08-08 19:08:04 -04:00
Flavio Ceolin 20c9bb856f x86-64: Early TLS initialization
Allow early boot code to use thread local storage
when it is enabled.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2023-08-08 19:08:04 -04:00
Flavio Ceolin 596e77f562 x86: Early TLS initialization
Allow early boot code using thread local storage when
it is enabled.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2023-08-08 19:08:04 -04:00
Gerard Marull-Paretas 28c139f653 drivers: pm_cpu_ops: psci: provide sys_poweroff hook
Instead of implementing a custom power off API (pm_system_off),
implement the sys_poweroff hook, and indicate power off is supported by
selecting HAS_POWEROFF. Note that according to the PSCI specification
(DEN0022E), the SYSTEM_OFF operation does not return, however, an error
is printed and system is halted in case this occurs.

Note that the pm_system_off has also been deleted, from now on, systems
supporting PSCI should enable CONFIG_POWEROFF and call the standard
sys_poweroff() API.

Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
2023-08-04 16:59:36 +02:00
Jonathan Rico 05cc2f37a3 arch: posix: break debugger on fatal error
Add option to raise a SIGTRAP on fatal error, making the debugger break
instead of exiting.

Signed-off-by: Jonathan Rico <jonathan.rico@nordicsemi.no>
2023-08-04 14:15:19 +02:00
Patryk Duda f16c4324a7 clang: Don't specify FP16 format
Support for 16 bit floats is enabled by default in Clang [1].
The ARM alternative format is not supported, so __fp16 always uses
IEEE 754-2008 format [2].

[1] https://github.com/llvm/llvm-project/blob/main/llvm/lib/Target/ARM/ARMAsmPrinter.cpp#L750-L755
[2] https://clang.llvm.org/docs/LanguageExtensions.html#half-precision-floating-point

Fixes: #55562

Signed-off-by: Patryk Duda <pdk@semihalf.com>
2023-08-01 11:24:07 +02:00
Patryk Duda cf55f8329e core: aarch32: Fix standard for _Float16 in CONFIG_FP16 help message
Both Clang[1] and GCC[2] says that _Float16 type is defined by
ISO/IEC TS 18661-3:2015 not the IEEE 754-2008. Fix help message under
FP16 config option.

[1] https://clang.llvm.org/docs/LanguageExtensions.html#half-precision-floating-point
[2] https://gcc.gnu.org/onlinedocs/gcc/Half-Precision.html

Signed-off-by: Patryk Duda <pdk@semihalf.com>
2023-08-01 11:24:07 +02:00
Alberto Escolar Piedras c044b2e916 posix arch: Fix very rare segfault on program termination
In some very rare cases (< 1/1000 runs), in very loaded machines,
a race in the glibc pthread_cancel() seems to be triggered.

In this the cancelled thread cleanup overtakes the pthread_cancel()
code, and frees the pthread structure before pthread_cancel()
has finished, resulting in a dereference into already
free'd memory, and therefore a segfault.
Calling pthread_cancel() during cleanup is not required beyond
preventing a valgrind memory leak report (all threads will be
stopped immediately on exit).
Therefore we stop doing this, to avoid this very rare crashes.

This issue was reproduced in Ubuntu 22.04, with its default
gcc 11.3.0 and glibc 2.35.
The issue may also have been seen very rarely in Zephyr's CI.

Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
2023-07-27 17:02:36 +02:00
Yong Cong Sin 84b86d9b0c soc: riscv: Add ability to use custom sys_io functions
Add Kconfig RISCV_SOC_HAS_CUSTOM_SYS_IO symbol so that a riscv
SoC can set to specify that it has a custom implementation for
sys_io functions.

Signed-off-by: Yong Cong Sin <ycsin@meta.com>
2023-07-26 09:43:59 +02:00
Girisha Dengi 75547dd522 soc: arm64: Add agilex5 soc folder and its configurations
Add Agilex5 soc folder, MMU table and its configurations for
Intel SoC FPGA Agilex5 platform for initial bring up.
Add ARM Cortex-a76 and Cortex-a55 HMP cluster type.

Signed-off-by: Teik Heng Chong <teik.heng.chong@intel.com>
Signed-off-by: Girisha Dengi <girisha.dengi@intel.com>
2023-07-25 16:58:01 +00:00
Marek Matej 6b57b3b786 soc: xtensa,riscv: esp32xx: refactor folder structure
Refactor the ESP32 target SOCs together with
all related boards. Most braking changes includes:

- changing the CONFIG_SOC_ESP32* to refer to
  the actual soc line (esp32,esp32s2,esp32s3,esp32c3)
- replacing CONFIG_SOC with the CONFIG_SOC_SERIES
- creating CONFIG_SOC_FAMILY_ESP32 to embrace all
  the ESP32 across all used architectures
- introducing CONFIG_SOC_PART_NUMBER_* to
  provide a SOC model config
- introducing the 'common' folder to hide all
  commonly used configs and files.
- updating west.yml to reflect previous changes in hal

Signed-off-by: Marek Matej <marek.matej@espressif.com>
2023-07-25 18:12:33 +02:00
Daniel Leung e6d8926857 xtensa: set no optimization for arch_cpu_idle() with xt-clang
xt-clang likes to remove any consecutive NOPs more than 8. So
we need to force the function to have no optimization to avoid
this behavior and to retain all those NOPs.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-07-24 11:07:30 -04:00
Daniel Leung a458d0443a xtensa: allow arch-specific arch_spin_relax() with more NOPs
This adds a Kconfig to introduce the Xtensa specific
arch_spin_relax() which can do more NOPs. Some Xtensa SoCs
may need more NOPs after failure to lock a spinlock,
especially under SMP. This gives the bus extra time to
propagate the RCW transactions among CPUs.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-07-20 10:47:47 +00:00
Michael Zimmermann 6168f47ac3 arch/posix: put fuzzing kconfigs into submenu
Without fuzzing enabled they don't do anything and should not be
selectable.

Signed-off-by: Michael Zimmermann <sigmaepsilon92@gmail.com>
2023-07-18 19:19:53 +00:00
Carlo Caione 5b2d716e76 arm64: Keep the frame pointer in leaf functions
It helps with debugging and usually on ARM64 we don't care about a small
increase in code size.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2023-07-17 10:14:18 +00:00
Patryk Duda cab341d29c arch/common: Fix moving location counter backwards when using LLD
In GNU LD, the location counter (the 'dot' variable) always refers to
the byte offset from the start of current object as mentioned in
documentation[1]:

```
'.' actually refers to the byte offset from the start of the current
containing object. Normally this is the SECTIONS statement, whose start
address is 0, hence '.' can be used as an absolute address. If '.' is
used inside a section description however, it refers to the byte offset
from the start of that section, not an absolute address.
```

For example, if the section 'rom_start':

rom_start : {
	. = 0x400;
	_vector_start = ABSOLUTE(.);
} > FLASH

has a starting address of 0x8000000, then _vector_start will be
0x8000400

However, behavior of LLVM LLD is quite different, the value of the
location counter is always absolute (see discussion [2]), so in the
example above, the linker will return error, because it will interpret
'. = 0x400' as an attempt to move the location counter backwards.

It could be fixed by changing line to '. += 0x400' (#54796) which will
move the location counter by 0x400 for both linkers, but it would work
only when we are at the beginning of section. Consider the following
example:

rom_start : {
	. = 0x400;
	KEEP(*(.boot_hdr.conf))
	. = 0x1000;
	KEEP(*(.boot_hdr.ivt))
	KEEP(*(.boot_hdr.data))
	KEEP(*(.boot_hdr.dcd_data))
	. = 0x2000;
	_vector_start = .;
} > FLASH

In this case, _vector_start will be 0x2000, but if we change
'. = 0x2000' to '. += 0x2000', then the value of _vector_start depends
on size of data in input sections (but it's 0x3000 at least).

Actually, this example comes from final linker script when compiling
firmware for mimxrt1170_evk_cm7 board. This board failed to boot
(#55296) after #54796 was merged.

This patch introduces method compatible with both linkers. We calculate
relative offset from the beginning of the section and use that value to
calculate number of bytes by which we should move the location counter
to get CONFIG_ROM_START_OFFSET.

[1] https://sourceware.org/binutils/docs/ld/Location-Counter.html
[2] https://discourse.llvm.org/t/lld-location-counter-inside-objects

Signed-off-by: Patryk Duda <pdk@semihalf.com>
2023-07-14 12:08:02 +02:00
Carlo Caione c9a68afede arm64: Set FP to NULL before jumping to C code
When the frame-pointer based unwinding is enabled, the stop condition
for the stack backtrace is (FP == NULL).

Set FP to 0 before jumping to C code.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2023-07-14 09:33:00 +00:00
Carlo Caione 18636af6d6 arm64: Add frame-pointer based stack unwinding
Add the frame-pointer based stack unwinding on ARM64.

This is a typical output with the feature enabled:

*** Booting Zephyr OS build zephyr-v3.4.0-1029-gae22ff648c16 ***
E: ELR_ELn: 0x00000000400011c8
E: ESR_ELn: 0x0000000096000046
E:   EC:  0x25 (Data Abort taken without a change in Exception level)
E:   IL:  0x1
E:   ISS: 0x46
E: FAR_ELn: 0x0000000000000000
E: TPIDRRO: 0x0100000040011650
E: x0:  0x0000000000000000  x1:  0x0000000000000003
E: x2:  0x0000000000000003  x3:  0x000000004005c6a0
E: x4:  0x0000000000000000  x5:  0x000000004005c7f0
E: x6:  0x0000000048000000  x7:  0x0000000048000000
E: x8:  0x0000000000000005  x9:  0x0000000000000000
E: x10: 0x0000000000000000  x11: 0x0000000000000000
E: x12: 0x0000000000000000  x13: 0x0000000000000000
E: x14: 0x0000000000000000  x15: 0x0000000000000000
E: x16: 0x0000000040004290  x17: 0x0000000000000000
E: x18: 0x0000000000000000  lr:  0x0000000040001208
E:
E: backtrace  0: fp: 0x000000004005c690 lr: 0x0000000040001270
E: backtrace  1: fp: 0x000000004005c6b0 lr: 0x0000000040001290
E: backtrace  2: fp: 0x000000004005c7d0 lr: 0x0000000040004ac0
E: backtrace  3: fp: 0x000000004005c7f0 lr: 0x00000000400013a4
E: backtrace  4: fp: 0x000000004005c800 lr: 0x0000000000000000
E:
E: >>> ZEPHYR FATAL ERROR 0: CPU exception on CPU 0
E: Current thread: 0x40011310 (unknown)
E: Halting system

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2023-07-13 17:04:05 +02:00
Wilfried Chauveau 965d30537f arch: arm: cortex_m: fault: Improve handling of fault escalation on armv6-m
The current implementation causes a lockup of the core when the exception
originates from an invalid/unreachable pc. This fix first verifies on
armv6-m and armv8-m.base that pc was in an expected runnable region,
namely:

- .text
- .ramfunc
- .itcm

Signed-off-by: Wilfried Chauveau <wilfried.chauveau@arm.com>
2023-07-13 09:22:46 +02:00
Dong Wang 4774a02b3b arch/x86: add more arch dcache functions
Adapt to the reworked zephyr cache API.
Fix build errors when building tests/kernel/cache with CACHE_MANAGEMENT and
CPU_HAS_DCACHE enabled for x86 SoCs

Signed-off-by: Dong Wang <dong.d.wang@intel.com>
2023-07-11 11:33:16 -04:00
Jaxson Han 5d643ecd24 arch: arm64: Fix z_arm64_fatal_error declaration error
The z_arm64_fatal_error should be
extern void z_arm64_fatal_error(unsigned int reason, z_arch_esf_t *esf);

Signed-off-by: Jaxson Han <jaxson.han@arm.com>
2023-07-11 15:04:54 +02:00
Jaxson Han 6fc3903bbe arch: arm64: mpu: Fix some minor CHECKIF issues
There are two CHECKIFs whose check condition is wrong.

Signed-off-by: Jaxson Han <jaxson.han@arm.com>
2023-07-11 15:04:54 +02:00
Mykola Kvach d0472aae7a soc: arm64: add Renesas Rcar Gen3 SoC support
Add files for supporting arm64 Renesas r8a77951 SoC.
Add config option CPU_CORTEX_A57.

Enable build of clock_control_r8a7795_cpg_mssr.c for
a new ARM64 SoC R8A77951.

Signed-off-by: Mykola Kvach <mykola_kvach@epam.com>
2023-07-11 11:17:41 +02:00
Alberto Escolar Piedras ee268a6d4b posix arch cmake: Add clarification on use of target_link_options
Clarify why we use target_link_options() instead of
target_link_libraries()

Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
2023-07-10 09:51:25 +02:00
Florian Grandel e256b7d244 kernel: spinlock: LOCKED -> K_SPINLOCK
Let the kernel use the new K_SPINLOCK macro and remove the alias.

Signed-off-by: Florian Grandel <fgrandel@code-for-humans.de>
2023-07-10 09:27:21 +02:00
Alberto Escolar Piedras 265010a4b6 native boards: Make native simulator host trampolines avaliable to all
To ease writing common drivers, let's make the host trampolines
from the native simulator avaliable to all posix based boards.

Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
2023-07-07 14:32:41 +02:00
Andrzej Kuros abd90085ac soc: arm: nrf53: workaround pop lr after wfi crash
On nRF5340 net core it was observed that when `wfi` instruction was
followed by `pop {r0, lr}` in the `arch_cpu_idle` function,
the value of `lr` sometimes got read as 0 from memory despite
having correct value stored in the memory.

This commit inserts additional `nop` instruction after waking up
to delay access to the memory.

Signed-off-by: Andrzej Kuroś <andrzej.kuros@nordicsemi.no>
2023-07-07 14:32:27 +02:00
Alberto Escolar Piedras dadacfaac2 soc (native): Use nce directly from native_simulator
Instead of keeping a replica of the nce code,
now that the native_simulator is in tree, let's use
it directly.

Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
2023-07-07 12:54:19 +02:00
Alberto Escolar Piedras 2096006cd8 POSIX arch: Give a nicer error message if built in unsuported platform
Instead of failing badly later, let's give a clear error
message if the user tries to build in an unsupported platform.

Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
2023-07-06 14:46:48 +02:00
Alberto Escolar Piedras ece74b7b7f native_sim: Add new native_sim board
Add the new native_sim board, based on the native simulator.

Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
2023-07-05 07:01:19 -04:00
Alberto Escolar Piedras 945429e8c3 arch POSIX: provide a bottom which uses natsim's thread emul
This allows building with embedded libCs in the Zephyr side,
as the POSIX arch bottom is not anymore built in Zephyr context.

Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
2023-07-05 07:01:19 -04:00
Alberto Escolar Piedras 41a185a1b5 arch POSIX: Use posix cheats only for native apps
native libraries do not need it

Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
2023-07-05 07:01:19 -04:00
Alberto Escolar Piedras 56dc20eb1d Kconfig: Tidy up dependants of NATIVE_APPLICATION
So they depend or select on the right NATIVE_BUILD
instead of NATIVE_APPLICATION.

Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
2023-07-05 07:01:19 -04:00
Gerard Marull-Paretas 0d4e5bf67f arch: arm: nmi: simplify NMI initialization
Zephyr provides a default NMI handler (`z_SysNmiOnReset`), which will
basically call `wfi` endlessly. It is allowed to override such handler
when CONFIG_RUNTIME_NMI=y, via `z_arm_nmi_set_handler`. However,
enabling such option also provided `z_arm_nmi_init` (via `NMI_INIT()`),
which basically sets the handler to `DefaultHandler` (a new handler that
basically printks and reboots). This is strictly not needed, and
independent of the runtime NMI option. As a result, most SoCs were
blindly calling `NMI_INIT()`, probably because of a copy&paste effect.
In the majority of cases, this was a no-op, but most SoCs do IRQ
enable/disable, making this even more convoluted. To make things worse,
the init call is expected to run after console has been initialized (for
printk to work?), but most SoCs just called it in PRE_KERNEL_1+0.

This patch just drops this NMI initializer API, and leaves only the
handler set call when CONFIG_RUNTIME_NMI=y.

NMI_INIT() dummy definition is left in this patch to preserve
bisectability, will be dropped later.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2023-07-05 09:15:36 +02:00
Roberto Medina 6622735ea8 arch: arm64: add support for coredump
* Add support for coredump on ARM64 architectures.
* Add the script used for post-processing coredump output.

Signed-off-by: Marcelo Ruaro <marcelo.ruaro@huawei.com>
Signed-off-by: Rodrigo Cataldo <rodrigo.cataldo@huawei.com>
Signed-off-by: Roberto Medina <roberto.medina@huawei.com>
2023-07-03 09:32:26 +02:00
Najumon Ba b3d78de656 arch: x86: updated acpi config macro
renamed x86 CONFIG_ACPI config macro to CONFIG_x86_ACPI.

Signed-off-by: Najumon Ba <najumon.ba@intel.com>
2023-06-30 17:53:01 +03:00
Gerard Marull-Paretas 33b116407b arch: arm: cortex_m: scb: fix MPU code compilation guards
The implementation of `z_arm_clear_arm_mpu_config` was compiled for all
ARM cores that declare to have an MPU. However, we only want to compile
it if the MPU is actually enabled.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2023-06-29 23:46:39 +02:00
Alberto Escolar Piedras 7ee41b8776 soc inf (native): Refactor into a top and bottom
Where the bottom is the only one which interacts with
the host operating system.
And the top the only one that interacts or is aware
of the hosted operating system (Zephyr).

The bottom uses the native simulator CPU
start/stop emulation.
By now we replicate its code as a provisional measure,
until the native simulator becomes standard.

Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
2023-06-23 12:16:26 +02:00
Alberto Escolar Piedras bd66c1d953 arch POSIX: Refactor into a top and bottom
Where the bottom is the only one which interacts with
the host operating system, and does not necessarily
need to know about the Zephyr OS.
This is in preparation for the native simulator,
which which the bottom is also fully Zephy agnostic.

Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
2023-06-23 12:16:26 +02:00
Alberto Escolar Piedras 78e1d6fa5e arch posix: Set C standard version in arch instead of boards
The issue due to which we set the default compiler C std version
to C11 is not specific to any particular POSIX arch board,
but to all. Instead of setting this property for each board,
let's set it at the architecture level.

Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
2023-06-23 08:59:21 +02:00
Lucas Tamborrino eb028ccf55 debug: coredump: xtensa: Add esp32s3
Add coredump support for esp32s3.

Signed-off-by: Lucas Tamborrino <lucas.tamborrino@espressif.com>
2023-06-21 16:06:06 -04:00
Lucas Tamborrino ba3766a75f debug: coredump: xtensa: add esp32s2
Add coredump support for esp32s2.

Signed-off-by: Lucas Tamborrino <lucas.tamborrino@espressif.com>
2023-06-21 16:06:06 -04:00
Huifeng Zhang c1ecb8faaa arch: arm: enable FPU and FPU sharing for v8r aarch32
This commit is to enable FPU and FPU_SHARING for v8r aarch32.

Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
2023-06-21 16:06:08 +02:00
Huifeng Zhang 89ea503b29 arch: arm: Add support for 32 double-precision registers
This adds support for 32 double-precision registers in the context
switching of aarch32 architecture.

Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
2023-06-21 16:06:08 +02:00
Huifeng Zhang b0612e14e0 arch: arm: save 'fpscr' in the svc and isr handler
`fpscr` is assigned from `struct __fpu_sf.fpscr` in `vfp_restore`, but it
wasn't saved into `struct __fpu_sf.fpscr` in the svc and isr handler, So
it may be a dirty value.

    - Fix it by saving `fpscr` in the svc hand isr handler.
    - Jump out if FPU isn't enabled

Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
2023-06-21 16:06:08 +02:00
Björn Stenberg 234fec579a scripts: gen_relocate_app.py: Give sections unique names
The code_relocation feature creates generic section names that sometimes
conflict with already existing names.

This patch adds a '_reloc_' word to the created names to reduce the risk
of conflict.

This solves #54785.

Signed-off-by: Björn Stenberg <bjorn@haxx.se>
2023-06-20 16:33:49 +02:00
Daniel Leung 97dc67e666 riscv: syscalls: use zephyr_syscall_header
This adds a few line use zephyr_syscall_header() to include
headers containing syscall function prototypes.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-06-17 07:57:45 -04:00
Carlo Caione d395b7f1e8 riscv: Add CONFIG_RISCV_RESERVED_IRQ_ISR_TABLES_OFFSET
Some RISCV platforms shipping a CLIC have a peculiar interrupt ID
ordering / mapping.

According to the "Core-Local Interrupt Controller (CLIC) RISC-V
Privileged Architecture Extensions" Version 0.9-draft at paragraph 16.1
one of these ordering recommendations is "CLIC-mode interrupt-map for
systems retaining interrupt ID compatible with CLINT mode" that is
described how:

  The CLINT-mode interrupts retain their interrupt ID in CLIC mode.
  [...]
  The existing CLINT software interrupt bits are primarily intended for
  inter-hart interrupt signaling, and so are retained for that purpose.
  [...]
  CLIC interrupt inputs are allocated IDs beginning at interrupt ID
  17. Any fast local interrupts that would have been connected at
  interrupt ID 16 and above should now be mapped into corresponding
  inputs of the CLIC.

That is a very convoluted way to say that interrupts 0 to 15 are
reserved for internal use and CLIC only controls interrupts reserved for
platform use (16 up to n + 16, where n is the maximum number of
interrupts supported).

Let's now take now into consideration this situation in the DT:

  clic: interrupt-controller {
    ...
  };

  device0: some-device {
    interrupt-parent = <&clic>;
    interrupts = <0x1>;
    ...
  };

and in the driver for device0:

  IRQ_CONNECT(DT_IRQN(node), ...);

From the hardware prospective:

(1a) device0 is using the first IRQ line of the CLIC
(2a) the interrupt ID / exception code of the `MSTATUS` register
     associated to this IRQ is 17, because the IDs 0 to 15 are reserved

From the software / Zephyr prospective:

(1b) Zephyr is installing the IRQ vector into the SW ISR table (and into
     the IRQ vector table for DIRECT ISRs in case of CLIC vectored mode)
     at index 0x1.
(2b) Zephyr is using the interrupt ID of the `MSTATUS` register to index
     into the SW ISR table (or IRQ vector table)

It's now clear how (2a) and (2b) are in contrast with each other.

To fix this problem we have to take into account the offset introduced
by the reserved interrupts. To do so we introduce
CONFIG_RISCV_RESERVED_IRQ_ISR_TABLES_OFFSET as hidden option for the
platforms to set.

This Kconfig option is used to shift the interrupt numbers when
installing the IRQ vector into the SW ISR table and/or IRQ vector table.
So for example in the previous case and using
CONFIG_RISCV_RESERVED_IRQ_ISR_TABLES_OFFSET == 16, the IRQ vector
associated to the device0 would be correctly installed at index 17 (16 +
1), matching what is reported by the `MSTATUS` register.

CONFIG_NUM_IRQS must be increased accordingly.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2023-06-17 07:49:16 -04:00
Carlo Caione fc480c9382 riscv: privileged: Add support for CLIC vectored mode
Zephyr currently only supports CLINT direct mode and CLINT vectored
mode. Add support for CLIC vectored mode as well.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2023-06-17 07:48:52 -04:00
Carlo Caione 6160383ec7 riscv: Rename RISCV_MTVEC_VECTORED_MODE to RISCV_VECTORED_MODE
Before adding support for the CLIC vectored mode, rename
CONFIG_RISCV_MTVEC_VECTORED_MODE to CONFIG_RISCV_VECTORED_MODE to be
more generic and eventually include also the CLIC vectored mode.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2023-06-17 07:48:52 -04:00
Dino Li 252d68ff9f arch/riscv: add support for detecting null pointer exception using PMP
This change uses a PMP slot to implement null pointer detection.

Signed-off-by: Dino Li <Dino.Li@ite.com.tw>
2023-06-17 07:45:30 -04:00
Daniel Leung ad826f3d69 x86: x86_64 can only support max 4 CPUs
With all the stacks and TSS (etc), the x86_64 arch code can only
support maximum of 4 CPUs at the moment. So add a build assert
if more CPUs are specified via CONFIG_MP_MAX_NUM_CPUS, also
overwrite the range value for CONFIG_MP_MAX_NUM_CPUS.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-06-17 07:28:10 -04:00
Carlo Caione edd3437826 riscv: Rename Kconfig symbol to *_PRIVILEGED
Rename SOC_FAMILY_RISCV_PRIVILEGE to SOC_FAMILY_RISCV_PRIVILEGED because
the spec is "privileged".

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2023-06-09 11:46:29 -04:00
Evgeniy Paltsev 9d6ced3822 ARC: SMP: fix livelock in thread abort due to exception
We are missing setting of switch_handle for the thread which
is aborting due to exception (i.e. in case of k_panic or
__ASSERT triggered). This may cause livelock in SMP code
after a08e23f68e commit ("kernel/sched: Fix SMP
must-wait-for-switch conditions in abort/join").

Fix that.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
2023-06-08 20:27:06 -04:00
Luca Fancellu c8a9634a30 arch: arm64: Use atomic for fpu_owner pointer
Currently the lazy fpu saving algorithm in arm64 is using the fpu_owner
pointer from the cpu structure to understand the owner of the context
in the cpu and save it in case someone different from the owner is
accessing the fpu.

The semantics for memory consistency across smp systems is quite prone
to errors and reworks on the current code might miss some barriers that
could lead to inconsistent state across cores, so to overcome the issue,
use atomics to hide the complexity and be sure that the code will behave
as intended.

While there, add some isb barriers after writes to cpacr_el1, following
the guidance of ARM ARM specs about writes on system registers.

Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
2023-06-08 09:35:11 -04:00
Nicolas Pitre 5c7d2f1a90 riscv: exclude more code when multithreading is disabled
This will avoid unconditionally pulling z_riscv_switch() into the build
as it is not used, reducing the resulting binary some more.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2023-05-31 06:17:10 -04:00
Keith Packard 6ccdf47f60 lib/cpp: Remove CONFIG_CPP_MAIN
With Zephyr now always using `int main(void)`, there's no longer any need
for this definition. The last remaining use which gated the declaration of
_posix_zephyr_main isn't necessary as adding that declaration
unconditionally is harmless.

Signed-off-by: Keith Packard <keithp@keithp.com>
2023-05-31 06:16:35 -04:00
Jaxson Han 08791d5fae arch: arm64: Enable FPU and FPU_SHARING for v8r aarch64
This commit is to enable FPU and FPU_SHARING for v8r aarch64.

Signed-off-by: Jaxson Han <jaxson.han@arm.com>
2023-05-30 10:14:55 +02:00
Jaxson Han 0b3136c2e8 arch: arm64: Add cpu_map to map cpu id and mpid
cpu_node_list does not hold the corrent mapping of cpu id and mpid when
core booting sequence does not follow the DTS cpu node sequence. This
will cause an issue that sgi cannot deliver to the right target.

Add the cpu_map array to hold the corrent mapping between cpu id and
mpid.

Signed-off-by: Jaxson Han <jaxson.han@arm.com>
2023-05-30 10:14:55 +02:00
Jaxson Han f03a4cec57 arch: arm64: Fix the STACK_INIT logic during the reset
Each core should init their own stack during the reset when SMP enabled,
but do not touch others. The current init results in each core starting
init the stack from the same address which will break others.

Fix the issue by setting a correct start address.

Signed-off-by: Jaxson Han <jaxson.han@arm.com>
2023-05-30 10:14:55 +02:00
Jaxson Han 101ae5d240 arch: arm64: mpu: Remove LOG print before mpu enabled
LOG system has unalignment access instruction which will cause an
alignment exception before MPU is enabled. Remove the LOG print before
MPU is enabled to avoid this issue.

Signed-off-by: Jaxson Han <jaxson.han@arm.com>
2023-05-30 10:14:55 +02:00
Andy Ross b89e427bd6 kernel/sched: Rename/redocument wait_for_switch() -> z_sched_switch_spin()
This trick turns out also to be needed by the abort/join code.
Promote it to a more formal-looking internal API and clean up the
documentation to (hopefully) clarify the exact behavior and better
explain the need.

This is one of the more... enchanted bits of the scheduler, and while
the trick is IMHO pretty clean, it remains a big SMP footgun.

Signed-off-by: Andy Ross <andyross@google.com>
2023-05-26 17:09:35 -04:00
Chad Karaginides 9ab7354a46 arch: arm64: SCR_EL3 EEL2 Enablement
For secure EL2 to be entered the EEL2 bit in SCR_EL3 must be set.  This
should only be set if Zephyr has not been configured for NS mode only,
if the device is currently in secure EL3, and if secure EL2 is supported
via the SEL2 bit in AA64PFRO_EL1.  Added logic to enable EEL2 if all
conditions are met.

Signed-off-by: Chad Karaginides <quic_chadk@quicinc.com>
2023-05-26 13:51:50 -04:00
Marek Matej 92b7d37397 arch: riscv: Fix bogus condition
Remove double preprocessor condition.

Signed-off-by: Marek Matej <marek.matej@espressif.com>
2023-05-26 10:36:15 -04:00
Nicolas Pitre 6935ea54d5 Revert "arch: riscv: Enable builds without the multithreading"
This reverts commit f0b458a619.

This is a pointless change that simply increases footprint.
Existing code already supports compilation without multithreading.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2023-05-26 09:04:30 -04:00
Chad Karaginides f8b9faff24 arch: arm64: Added ISBs after SCTLR Modifications
Per the ARMv8 architecture document, modification of the system control
register is a context-changing operation. Context-changing operations are
only guaranteed to be seen after a context synchronization event.
An ISB is a context synchronization event.  One has been placed after
each SCTLR modification. Issue was found running full speed on target.

Signed-off-by: Chad Karaginides <quic_chadk@quicinc.com>
2023-05-25 16:33:03 -04:00
Marek Matej 1c130d0060 arch: xtensa: Enable builds without the multithreading
Allow builds which has CONFIG_MULTITHREADING disabled.
This is reduce code footprint which is handy for
constrained targets as bootloaders.

Signed-off-by: Marek Matej <marek.matej@espressif.com>
2023-05-25 16:15:54 +02:00
Marek Matej f0b458a619 arch: riscv: Enable builds without the multithreading
Allow builds which has CONFIG_MULTITHREADING disabled.
This is reduce code footprint which is handy for
constrained targets as bootloaders.

Signed-off-by: Marek Matej <marek.matej@espressif.com>
2023-05-25 16:15:54 +02:00
Marek Matej 4796746b5e soc: esp32: MCUboot support
This make MCUboot build as Zephyr application.
Providing optinal 2nd stage bootloader to the
IDF bootloader, which is used by default.
This provides more flexibility when building
and loading multiple images and aims to
brings better DX to users by using the sysbuild.
MCUboot and applications has now separate
linker scripts.

Signed-off-by: Marek Matej <marek.matej@espressif.com>
2023-05-25 16:15:54 +02:00
Carlo Caione f4cbb01074 barriers: Enable builtin barriers for RISCV
By setting CONFIG_BARRIER_OPERATIONS_BUILTIN.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2023-05-25 06:29:03 -04:00
Carlo Caione 637c3ed159 barriers: Enable builtin barriers for POSIX
By setting CONFIG_BARRIER_OPERATIONS_BUILTIN.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2023-05-25 11:58:31 +02:00
Rander Wang 445f4e8877 arch/xtensa: undefine NOP32
It should not be NOP16 since it is not defined by this file

Signed-off-by: Rander Wang <rander.wang@intel.com>
2023-05-25 04:49:14 -04:00
Nikolay Agishev 78be96b152 ARC: MPU: Add MPUv8 fixes
Add mpuv8-related fixes that setup alignment and region overlapping

Signed-off-by: Nikolay Agishev <agishev@synopsys.com>
2023-05-25 08:27:34 +00:00
Nicolas Pitre 8e9872a7c8 arm64: prevent possible deadlock on SMP with FPU sharing
Let's consider CPU1 waiting on a spinlock already taken by CPU2.

It is possible for CPU2 to invoke the FPU and trigger an FPU exception
when the FPU context for CPU2 is not live on that CPU. If the FPU context
for the thread on CPU2 is still held in CPU1's FPU then an IPI is sent
to CPU1 asking to flush its FPU to memory.

But if CPU1 is spinning on a lock already taken by CPU2, it won't see
the pending IPI as IRQs are disabled. CPU2 won't get its FPU state
restored and won't complete the required work to release the lock.

Let's prevent this deadlock scenario by looking for pending FPU IPI from
the spinlock loop using the arch_spin_relax() hook.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2023-05-25 08:25:11 +00:00
Nicolas Pitre 18ff7ae998 riscv: prevent possible deadlock on SMP with FPU sharing
Let's consider CPU1 waiting on a spinlock already taken by CPU2.

It is possible for CPU2 to invoke the FPU and trigger an FPU exception
when the FPU context for CPU2 is not live on that CPU. If the FPU context
for the thread on CPU2 is still held in CPU1's FPU then an IPI is sent
to CPU1 asking to flush its FPU to memory.

But if CPU1 is spinning on a lock already taken by CPU2, it won't see
the pending IPI as IRQs are disabled. CPU2 won't get its FPU state
restored and won't complete the required work to release the lock.

Let's prevent this deadlock scenario by looking for a pending FPU IPI
from the arch_spin_relax() hook and honor it.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2023-05-25 08:25:11 +00:00
Nicolas Pitre 03a73fa0db riscv: use atomic bit helpers with IPI values
This is cleaner and less error prone, especially when comes the time
to test and clear a bit.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2023-05-25 08:25:11 +00:00
Carlo Caione 6f3a13d974 barriers: Move __ISB() to the new API
Remove the arch-specific ARM-centric __ISB() macro and use the new
barrier API instead.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2023-05-24 13:13:57 -04:00
Carlo Caione cb11b2e84b barriers: Move __DSB() to the new API
Remove the arch-specific ARM-centric __DSB() macro and use the new
barrier API instead.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2023-05-24 13:13:57 -04:00
Carlo Caione 7cbb1f8b82 barriers: arm64: Move to arch-specific barriers
The compiler is not able to emit a proper DSB operation for ARM64. Move
to the arch-specific implementation and use assembly code instead.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2023-05-24 13:13:57 -04:00
Carlo Caione 2fa807bcd1 barriers: Move __DMB() to the new API
Remove the arch-specific ARM-centric __DMB() macro and use the new
barrier API instead.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2023-05-24 13:13:57 -04:00
Carlo Caione 4512712cd6 barriers: arm: Use CMSIS instead of built-in functions
For ARM (AArch32) we fall back on the CMSIS implementation that it is
supposedly more refined.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2023-05-24 13:13:57 -04:00
Qipeng Zha 1c095239b7 arch: x86: ia32: don't create FP context for NULL thread
Enhance for cases when call z_float_enable() with NULL thread.

Signed-off-by: Dong Wang <dong.d.wang@intel.com>
Signed-off-by: Qipeng Zha <qipeng.zha@intel.com>
2023-05-24 12:41:06 -04:00
Daniel Leung e444cc9fb9 xtensa: mmu: always map data TLB for VECBASE
This adds code to always map data TLB for VECBASE so that
we would be dealing with fewer data TLB misses during
exception handling. With VECBASE always mapped, there is
no need to pre-load anymore.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-05-23 08:54:29 +02:00
Daniel Leung c3d1fa2138 xtensa: mmu: handle TLB misses in C exception handler
This moves the TLB miss handling to the C exception handler.
This also allows us to handle page faults (for example,
unmapped pages) during this time as any more exceptions
handled in the C handler will not trigger the double
exception handler but the same C handler.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-05-23 08:54:29 +02:00
Daniel Leung ce1bf365b6 xtensa: make CONFIG_XTENSA_MMU_PTEVADDR based on choice
Instead of being able to arbitrarily set the PTEVADDR for page
table, this provides choices (currently just one). This is in
preparation to enable handling memory management exception in
C code. For that to work, we will need to pre-load the page
table address (PTEVADDR) for the memory page containing
exception code and data (containing jump addresses), and
various stacks. This is to prempt any TLB misses during handling
the level 1 interrupt code. If a TLB miss is encountered during
handling of level 1 interrupt, we will be thrown into double
exception handling code where we will get stuck in infinite
loop. However, in order to pre-load the page table entries,
PTEVADDR needs to be calculated. This requires the use of
PTEVADDR base which cannot be loaded via l32r, as we may cause
a data TLB miss. So we must be able to grab the PTEVADDR base
address strictly within code, and must be without any data
load. So changing CONFIG_XTENSA_MMU_PTEVADDR to be based on
choice so we can have pre-defined bit shift value for shift
operation. This shift value will be used in exception handling
code.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-05-23 08:54:29 +02:00
Daniel Leung dfc87e2754 xtensa: gen_zsr: add _STR for extra registers
This also generates the correspoing _STR entries for
extra registers.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-05-23 08:54:29 +02:00
Flavio Ceolin d091740b00 xtensa: mmu: Add option to map memory in cached/uncached
Add a build option to tell if memory should be mapped in cached
and uncachedr regions.

If the memory is neither in cached nor uncached region it is not double
mapped.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2023-05-23 08:54:29 +02:00
Flavio Ceolin 020df54ba4 xtensa: mmu: Initial implementation
Initial support for Xtensa MMU version 3. It is using a two level page
table based on fact that the page table is in the virtual space.  Only
the top level (page directory) is wired mapped in the TLB to avoid
second level page miss.

The mapped memory is completely fragmented in multiple sections, maybe
we find a better way in future.

The exception handler is where we effectively map the memory, the way it
works is:

1) SW try to access some memory address
2) The address is not mapped, so the MMU will try the auto-refill,
   looking the page table
3) The page table contents is not mapped (remember, just the top-level page
   is mapped)
4) An exception will be triggered, in the exception we try to read the
   portion of the page table that maps the original address
5) The address is not mapped, so the MMU will try again the auto-refill.
   This time though, the address is mapped by the top level page that is
   properly mapped. (The top-level page maps the page table itself).

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2023-05-23 08:54:29 +02:00
Qipeng Zha d963767369 arch: x86: add irq runtime statistics
Unlike tracing module mainly for debug usage, this is
to allow runtime profiling IRQ performance data, and
target to enable it in product release since platform
can choose to make it work with low weight protocol.

Enable this option and implement runtime_irq_stats()
in platform code, such as Intel ISH platform implement
with SHMI protocol to allow host profiling irq stats.

Signed-off-by: Qipeng Zha <qipeng.zha@intel.com>
2023-05-22 13:29:14 -04:00
Gerard Marull-Paretas dacb3dbfeb iterable_sections: move to specific header
Until now iterable sections APIs have been part of the toolchain
(common) headers. They are not strictly related to a toolchain, they
just rely on linker providing support for sections. Most files relied on
indirect includes to access the API, now, it is included as needed.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2023-05-22 10:42:30 +02:00
Stephanos Ioannidis a1d71bf05e arch: arm: aarch32: Remove CPU type dependency from CONFIG_FP16
The commit 434ca63e2f introduced the
Cortex-A and Cortex-R CPU type dependency to `CONFIG_FP16` based on
the reasoning that the hardware half-precision support is only
available on them.

While it is true that the _hardware_ half-precision support is limited
to these targets, the compiler will provide the _software_ emulation
for the targets that lack the hardware half-precision support, as
mentioned in 41fd6e003c (the original
commit that introduced `CONFIG_FP16`).

Signed-off-by: Stephanos Ioannidis <stephanos.ioannidis@nordicsemi.no>
2023-05-22 10:17:02 +02:00
Flavio Ceolin c4025f026f arch: xtensa: Remove unecessary logic in backtrace
In z_xtensa_backtrace_print the parameter depth is checked for <= 0.
There is no need to check it again later, also, since the variable is
not used after the while loop we can use directly the parameter without
an additional variable.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2023-05-12 18:31:13 -04:00
TOKITA Hiroshi b674bf6e71 arch: riscv: add ARCH_HAS_SINGLE_THREAD_SUPPORT
Enable single-threading support for the riscv architecture.

Add z_riscv_switch_to_main_no_multithreading function for
supporting single-threading.

The single-threading does not work with enabling PMP_STACK_GUARD.
It is because single-threading does not use context-switching.
But the privileged mode transition that PMP depends on implicitly
presupposes using context-switching. It is a contradiction.
Thus, disable PMP_STACK_GUARD when MULTITHREADING is not enabled.

Signed-off-by: TOKITA Hiroshi <tokita.hiroshi@fujitsu.com>
2023-05-12 09:56:40 +02:00
Flavio Ceolin f3bec2ffee xtensa: tls: Fix invalid reference
bsa is not defined. It should be access through frame pointer.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2023-05-11 17:38:16 -04:00
Qipeng Zha a638223df3 arch: x86: make sys_arch_reboot as weak function
Intel ISH SoC can't reboot via RST_CNT register,
so make sys_arch_reboot as weak function to allow
implement different arch reboot in SoC layer.

Signed-off-by: Dong Wang <dong.d.wang@intel.com>
Signed-off-by: Qipeng Zha <qipeng.zha@intel.com>
2023-05-10 09:21:15 -05:00
Qipeng Zha 44c3e10064 arch: x86: pin boot arg before paging is up
x86_cpu_boot_arg is accessed in boot code before
enable paging.

Signed-off-by: Qipeng Zha <qipeng.zha@intel.com>
2023-05-09 13:04:08 +02:00
Andy Ross e31ae60058 arch/xtensa: Fix nested interrupt entry
The "cross stack call" mechanism has intermediate states where the
stack frames are not valid for our own interrupt entry code, which
causes corruption if an interrupt races at exactly the right time.
Leave interrupts masked until just before the call.

The fix is midly complicated by the fact that we RELY on nested window
exception frames to spill registers from the interruptee, so have to
do the masking with PS.INTLEVEL, which requires a register to save its
contents, which we don't have since everything needs to happen in one
4-register window.  But thankfully our Zephyr-reserved EPS register is
guaranteed to be available through this process.

Fixes #57009

Signed-off-by: Andy Ross <andyross@google.com>
2023-05-08 16:56:17 -04:00
Qipeng Zha c626bac016 arch: x86: fix SSE init issue when enable paging
With paging config, need to use physical address as
paging is not enabled here.

From IA manual, LDMXCSR instruction description is,
Loads the source operand into the MXCSR control/status
register, the source operand is a 32-bit memory location.

Signed-off-by: Qipeng Zha <qipeng.zha@intel.com>
2023-05-08 16:55:27 -04:00
Kumar Gala f8ddd3d77f xtensa: limit speical exit() to XT_SIMULATOR
Use the common exit() provided by libc so we get standard behavior
across all architectures.  So only implement a special exit when
XT_SIMULATOR is defined.

Signed-off-by: Kumar Gala <kumar.gala@intel.com>
2023-05-08 09:59:54 +02:00
Evgeniy Paltsev f4f2fad800 ARC: tweak _st32_huge_offset macro to have same behaviour with GNU & MWDT
Local variables in ASM macro works differently for GNU and MWDT
toolchains. In case of GNU toolchain they are local per each macro
instance, but in case of MWDT they are local per file where macro
is used.

To avoid issues when macro is used multiple times in one file let's
align _st32_huge_offset to have same behaviour with GNU & MWDT
toolchains.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
2023-05-04 22:50:14 +09:00
Nikolay Agishev c729e12beb ARC: Add mpuv8 support
Turn on mpuv8 support for ARC SEM cpus

Signed-off-by: Nikolay Agishev <agishev@synopsys.com>
2023-05-04 10:48:06 +02:00
Nikolay Agishev 0d8292ab6b ARC: Add HS4x support
Minimal HSDK4xD support

Signed-off-by: Nikolay Agishev <agishev@synopsys.com>
2023-05-02 16:54:24 +02:00
Anas Nashif 6388f5f106 xtensa: use sys_cache API instead of custom interfaces
Use sys_cache instead of custom and internal APIs.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-04-26 07:31:22 -04:00
Daniel Leung 5aa1aaddc2 xtensa: fatal: no backtrace if no stack is passed in
The backtrace requires a valid stack pointer to start
printing backtraces. So if there is no stack pointer
being passed in, skip printing backtraces.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-04-21 16:27:50 +02:00
Manojkumar Subramaniam d57363bc97 riscv: refactor: Utilize the available helper macro
use helper macros from csr.h instead of inline assembly which results
in cleaner and more maintainable code

Signed-off-by: Manojkumar Subramaniam <manoj@electrolance.com>
2023-04-21 12:54:55 +02:00
Carlo Caione c2ec3d49cd arm64: cache: Enable full inlining of the cache ops
Move all the cache functions to a standalone header to allow for the
inlining of these functions.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2023-04-20 14:56:09 -04:00
Daniel Leung 5def4ab915 xtensa: fix inline assembly of rsil in exception code for XCC
Commit 408472673e added inline
assembly to lock interrupt. However, XCC doesn't like the syntax
using STRINGIFY, and also an empty clobber section. So parameterize
the second argument to rsil, and remove the last colon.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-04-20 08:14:35 -04:00
Daniel Leung 1e9d4602ab xtensa: add some structs for interrupt stack frames
This adds some structs for interrupt stack frames to make it
easier to access individual elements, and ultimately getting
rid of magic array element numbers in the code. Hopefully,
this would aid in debugging where you can view the whole
struct in debugger.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-04-20 04:45:52 -04:00
Kumar Gala 7469ad426a drivers: pcie: Use TYPE_SECTION macros for irq_alloc
Clean up irq_alloc to utilize TYPE_SECTION macros for handling sections.

Signed-off-by: Kumar Gala <kumar.gala@intel.com>
2023-04-19 17:14:46 +02:00
Peter Mitsis b832b325b9 arch: sparc: Remove unused absolute symbols
Removes SPARC specific unused absolute symbols that are defined
via the GEN_ABSOLUTE_SYM() macro.

Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
2023-04-18 10:51:28 -04:00
Peter Mitsis d84582081f arch: nios2: Remove unused absolute symbols
Removes NIOS2 specific unused absolute symbols that are defined
via the GEN_ABSOLUTE_SYM() macro.

Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
2023-04-18 10:51:28 -04:00
Peter Mitsis 6a9839f9c3 arch: arc: Remove unused absolute symbols
Removes ARC specific unused absolute symbols that are defined
via the GEN_ABSOLUTE_SYM() macro.

Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
2023-04-18 10:51:28 -04:00
Peter Mitsis 1e250b37df arch: arm64: Remove unused absolute symbols
Removes ARM64 specific unused absolute symbols that are defined
via the GEN_ABSOLUTE_SYM() macro.

Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
2023-04-18 10:51:28 -04:00
Peter Mitsis a6781b7f17 arch: arm: Remove unused absolute symbols
Removes ARM specific unused absolute symbols that are defined
via the GEN_ABSOLUTE_SYM() macro.

Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
2023-04-18 10:51:28 -04:00
Ryan McClelland 2808711e35 arch: arm: aarch32: include m55 for fp16 support
The M55 supports half-precision floating point.

Signed-off-by: Ryan McClelland <ryanmcclelland@meta.com>
2023-04-17 10:20:17 +02:00
Alberto Escolar Piedras 926def6510 arch posix: posix_cheats.h main() type changed in comment
Align the comment with the new int main(void) convention
used in Zephyr.

Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
2023-04-14 09:55:48 +02:00
Leifu Zhao 1cb259a43b arch/x86/ia32: Add option to reserve GDT entries
Some functions such as the ISH power management function needs to
reserve GDT entry place holders which are filled during runtime.
Add config option to define the number of GDT entry place holders
reserved and change the ia32 linker script accordingly.

Signed-off-by: Leifu Zhao <leifu.zhao@intel.com>
2023-04-13 05:43:20 -04:00
Gerard Marull-Paretas a5fd0d184a init: remove the need for a dummy device pointer in SYS_INIT functions
The init infrastructure, found in `init.h`, is currently used by:

- `SYS_INIT`: to call functions before `main`
- `DEVICE_*`: to initialize devices

They are all sorted according to an initialization level + a priority.
`SYS_INIT` calls are really orthogonal to devices, however, the required
function signature requires a `const struct device *dev` as a first
argument. The only reason for that is because the same init machinery is
used by devices, so we have something like:

```c
struct init_entry {
	int (*init)(const struct device *dev);
	/* only set by DEVICE_*, otherwise NULL */
	const struct device *dev;
}
```

As a result, we end up with such weird/ugly pattern:

```c
static int my_init(const struct device *dev)
{
	/* always NULL! add ARG_UNUSED to avoid compiler warning */
	ARG_UNUSED(dev);
	...
}
```

This is really a result of poor internals isolation. This patch proposes
a to make init entries more flexible so that they can accept sytem
initialization calls like this:

```c
static int my_init(void)
{
	...
}
```

This is achieved using a union:

```c
union init_function {
	/* for SYS_INIT, used when init_entry.dev == NULL */
	int (*sys)(void);
	/* for DEVICE*, used when init_entry.dev != NULL */
	int (*dev)(const struct device *dev);
};

struct init_entry {
	/* stores init function (either for SYS_INIT or DEVICE*)
	union init_function init_fn;
	/* stores device pointer for DEVICE*, NULL for SYS_INIT. Allows
	 * to know which union entry to call.
	 */
	const struct device *dev;
}
```

This solution **does not increase ROM usage**, and allows to offer clean
public APIs for both SYS_INIT and DEVICE*. Note that however, init
machinery keeps a coupling with devices.

**NOTE**: This is a breaking change! All `SYS_INIT` functions will need
to be converted to the new signature. See the script offered in the
following commit.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>

init: convert SYS_INIT functions to the new signature

Conversion scripted using scripts/utils/migrate_sys_init.py.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>

manifest: update projects for SYS_INIT changes

Update modules with updated SYS_INIT calls:

- hal_ti
- lvgl
- sof
- TraceRecorderSource

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>

tests: devicetree: devices: adjust test

Adjust test according to the recently introduced SYS_INIT
infrastructure.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>

tests: kernel: threads: adjust SYS_INIT call

Adjust to the new signature: int (*init_fn)(void);

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2023-04-12 14:28:07 +00:00
Thomas Stranger 77d2490164 arch: arm: core: aarch32: rename z_NmiHandlerSet
rename the function that sets the handler for the nmi.
It should be namespaced and not camel-case:
z_NmiHandlerSet to z_arm_nmi_set_handler

Signed-off-by: Thomas Stranger <thomas.stranger@outlook.com>
2023-04-12 08:59:36 +02:00
Aastha Grover 408472673e arch: xtensa: Fix xtensa error handler
In case of recoverable fatal errors the execution should
switch to another thread. This will ensure the current_cpu nested
count is reset  when there is a context switch.

Signed-off-by: Aastha Grover <aastha.grover@intel.com>
2023-04-11 14:48:51 -04:00
Rick Tsao d839194a5c riscv: pmp: Add kconfig option for granularity of PMP
Add Kconfig option CONIFG_PMP_GRANULARITY to specify the granularity
of PMP.

Signed-off-by: Rick Tsao <rick592@andestech.com>
2023-04-06 11:50:43 +02:00
Kumar Gala 14d59b3b56 cortex_m: tz_ns.h: Fix compiling with arm-clang
We get the following error when building with arm-clang:

error: non-ASM statement in naked function is not supported
        __TZ_WRAP_FUNC(preface, foo1, postface);
        ^
tests/arch/arm/arm_tz_wrap_func/src/main.c:69:25: note: attribute is here
uint32_t __attribute__((naked)) wrap_foo1(uint32_t arg1, uint32_t arg2,
                        ^
1 error generated.

Remove the do/while wrapper to make this a true naked function.

Signed-off-by: Kumar Gala <kumar.gala@intel.com>
2023-03-31 08:43:18 -05:00
Declan Snyder 62d4d2a2a3 revert: "linker: rom_start_offset: add to address"
Revert commit 44628735b8

This commit broke the ability for nxp rt series to
reset except with power cycle

Signed-off-by: Declan Snyder <declan.snyder@nxp.com>
2023-03-30 18:19:32 -04:00
Manuel Argüelles 98d4faf099 arch: arm: cortex_a_r: implement cache API
Implement cache management APIs for Cortex-A/R AArch32 L1 caches.

Signed-off-by: Manuel Argüelles <manuel.arguelles@nxp.com>
2023-03-29 09:03:37 +02:00
Manuel Argüelles c08fab16cd arch: arm: move cache management sources to cortex_m directory
Current implementation of cache management APIs for ARM only applies to
Cortex-M, so move it to its own directory.

Signed-off-by: Manuel Argüelles <manuel.arguelles@nxp.com>
2023-03-29 09:03:37 +02:00
Andrei Emeltchenko b3545b34cf arch: x86: zefi: Remove unneeded include
Fixes unneeded chain of includes. Since zefi is built separately
(using python script), any dependency creates include chain with
possibly missing configuration options.

Signed-off-by: Andrei Emeltchenko <andrei.emeltchenko@intel.com>
2023-03-22 20:32:35 +09:00
Jaxson Han e416c5f1bd arch: arm64: Update current stack limit on every context switch
Update current stack limit on every context switch, including switching
to irq stack and switching back to thread stack.

Signed-off-by: Jaxson Han <jaxson.han@arm.com>
2023-03-14 10:49:22 +01:00
Jaxson Han 00adc0b493 arch: arm64: Enable safe exception stack
This commit mainly enable the safe exception stack including the stack
switch. Init the safe exception stack by calling
z_arm64_safe_exception_stack during the boot stage on every core. Also,
tweaks several files to properly switch the mode with different cases.

1) The same as before, when executing in userspace, SP_EL0 holds the
user stack and SP_EL1 holds the privileged stack, using EL1h mode.

2) When entering exception from EL0 then SP_EL0 will be saved in the
_esf_t structure. SP_EL1 will be the current SP, then retrieves the safe
exception stack to SP_EL0, making sure the always pointing to safe
exception stack as long as the system running in kernel space.

3) When exiting exception from EL1 to EL0 then SP_EL0 will be restored
from the stack value previously saved in the _esf_t structure. Still at
EL1h mode.

4) Either entering or exiting exception from EL1 to EL1, SP_EL0 will
keep holding the safe exception stack unchanged as memtioned above.

5) Do a quick stack check every time entering the exception from EL1 to
EL1. If check fail, set SP_EL1 to safe exception stack, and then handle
the fatal error.

Overall, the exception from user mode will be handled with kernel stack
at the assumption that it is impossible the stackoverflow happens at the
entry of exception from EL0 to EL1. However the exception from kernel
mode will be firstly checked with the safe exception stack to see if the
kernel stack overflows, because the exception might be triggered by
stack invalid accessing.

Signed-off-by: Jaxson Han <jaxson.han@arm.com>
2023-03-14 10:49:22 +01:00
Jaxson Han 463b1c9396 arch: arm64: Add safe exception stack init function
Add safe exception stack init function which does several things:
1) setting current cpu safe exception stack pointer to its corresponding
stack top.
2) init sp_el0 with the above safe exception stack.
That makes sure the sp_el0 points to per-cpu safe_stack in the kernel
space.
3) init the current_stack_limit and corrupted_sp with 0

Signed-off-by: Jaxson Han <jaxson.han@arm.com>
2023-03-14 10:49:22 +01:00
Jaxson Han 3a5fa0498f arch: arm64: Add stack_limit to thread_arch_t
Add stack_limit to thread_arch_t to store the thread's stack limit.

Signed-off-by: Jaxson Han <jaxson.han@arm.com>
2023-03-14 10:49:22 +01:00
Jaxson Han 61b8b34b27 arch: arm64: Add the sp variable in _esf_t
As the preparation for enabling safe exception stack, add a variable in
_esf_t to save the user stack held by sp_el0 at the point of the
exception happening from EL0. The newly added variable in _esf_t is
named sp from which the user stack will be restored when exceptions eret
to EL0.

Signed-off-by: Jaxson Han <jaxson.han@arm.com>
2023-03-14 10:49:22 +01:00
Jaxson Han 7040c55438 arch: arm64: Add stack check relevant variables to _cpu_arch_t
Add three per-cpu variables for the convenience of quickly accessing.

The safe_exception_stack stores the top of safe exception stack pointer.
The current_stack_limit stores the current thread's priv stack limit.
The corrputed_sp stores the priv sp or irq sp for the stack overflow
case, or 0 for the normal case.

Signed-off-by: Jaxson Han <jaxson.han@arm.com>
2023-03-14 10:49:22 +01:00
Jaxson Han d8d74b1320 arch: arm64: Add el label for vector entry macro
Add a new label el for z_arm64_enter_exc to indicate which el the
exception comes from.

Signed-off-by: Jaxson Han <jaxson.han@arm.com>
2023-03-14 10:49:22 +01:00
Jaxson Han 6c40abb99f arch: arm64: Introduce safe exception stack
Introduce two configs to prepare to enable the safe exception stack for
the kernel space. This is the preparation for enabling hardware stack
guard. Also define the safe exception stack for kernel exception stack
check.

Signed-off-by: Jaxson Han <jaxson.han@arm.com>
2023-03-14 10:49:22 +01:00
Nicolas Pitre bcef633316 arch/arm64/mmu: arch_mem_map() should not overwrite existing mappings
If so this is most certainly a bug. arch_mem_unmap() should be
used before mapping the same area again.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2023-03-13 09:15:37 +01:00
Nicolas Pitre 364d7527c1 arch/arm64/mmu: minor addition to debugging code
Display table allocations and whether or not mappings
can be overwritten.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2023-03-13 09:15:37 +01:00
Nicolas Pitre abb50e1605 arch/arm64/mmu: fix table discarding code
First, we have commit 7d27bd0b85 ("arch: arm64: Disable infinite
recursion warning for `discard_table`") that blindly shut up a compiler
warning that did actually highlighted a real bug. Revert that and fix
the bug properly. And yes, mea culpa for having been the first to
approve that commit, or even creating the bug in the first place.

Then let's add proper table usage cound handling for discard_table() to
work properly and avoid leaking table pages.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2023-03-07 08:33:05 +01:00
Jonas Otto 60b8773491 arch: riscv enable flash config
For RISCV arch, enable FLASH_SIZE and FLASH_BASE_ADDRESS config.
To avoid duplicated work, remove flash config from RISCV soc.

Signed-off-by: Jonas Otto <jonas@jonasotto.com>
2023-02-28 10:29:03 +01:00
Ayan Kumar Halder 958dcf98e8 arch: arm: aarch32: Add ability to generate zImage header
The image header is compatible for zImage(32) protocol.

Offset  Value          Description
0x24    0x016F2818     Magic number to identify ARM Linux zImage
0x28    start address  The address the zImage starts at
0x2C    end address    The address the zImage ends at

As Zephyr can be built with a fixed load address, Xen/Uboot can read
the image header and decide where to copy the Zephyr image.

Also, it is to be noted that for AArch32 A/R, the vector table should
be aligned to 0x20 address. Refer ARM DDI 0487I.a ID081822, G8-9815,
G8.2.168, VBAR, Vector Base Address Register :-
Bits[4:0] = RES0.
For AArch32 M (Refer DDI0553B.v ID16122022, D1.2.269, VTOR, Vector Table
Offset Register), Bits [6:0] = RES0.
As zImage header occupies 0x30 bytes, thus it is necessary to align
the vector table base address to 0x80 (which satisfies both VBAR and
VTOR requirements).

Also, it is to be noted that not all the AArch32 M class have VTOR, thus
ARM_ZIMAGE_HEADER header depends on
CPU_AARCH32_CORTEX_R || CPU_AARCH32_CORTEX_A || CPU_CORTEX_M_HAS_VTOR.
The reason being the processors which does not have VBAR or VTOR, needs
to have exception vector table at a fixed address in the beginning of
ROM (Refer the comment in arch/arm/core/aarch32/cortex_m/CMakeLists.txt)
. They cannot support any headers.

Also, the first instruction in zImage header is to branch to the kernel
start address. This is to support booting in situations where the zImage
header need not be parsed.

In case of Arm v8M, the first two entries in the reset vector should be
"Initial value for the main stack pointer on reset" and "Start address
for the reset handler" (Refer Armv8M DDI0553B.vID16122022, B3.30,
Vector tables).
In case of Armv7M (ARM DDI 0403E. ID021621, B1.5.3 The vector table),
the first entry is "SP_main. This is the reset value of the Main stack
pointer.".
Thus when v7M or v8M starts from reset, it expects to see these values
at the default reset vector location.
See the following text from Armv7M (ARM DDI 0403E. ID021621, B1-526)
"On powerup or reset, the processor uses the entry at offset 0 as the
initial value for SP_main..."

Signed-off-by: Ayan Kumar Halder <ayan.kumar.halder@amd.com>
2023-02-27 17:34:12 +01:00
George Ruinelli b2512d2f53 arm: Add missing include
Add missing include to prevent `'EINVAL' undeclared` when
using `CONFIG_NULL_POINTER_EXCEPTION_DETECTION_DWT=y`

Signed-off-by: George Ruinelli <caco3@ruinelli.ch>
2023-02-25 07:59:56 -05:00
Peter Mitsis 2ab7286c71 arch: riscv: Remove unused offset symbols
Removes unused offset symbols under the RISCV architecture.

Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
2023-02-23 16:44:07 +01:00
Peter Mitsis a9e5038c2b arch: x86: Remove unused offset symbols
Removes unused offset symbols under the x86 architecture.

Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
2023-02-23 16:44:07 +01:00
Peter Mitsis 9b61418427 arch: sparc: Remove unused offset symbols
Removes unused offset symbols under the SPARC architecture.

Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
2023-02-23 16:44:07 +01:00
Peter Mitsis 66af4f443d arch: posix: Remove unused offset symbols
Removes unused offset symbols under the POSIX architecture.

Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
2023-02-23 16:44:07 +01:00
Peter Mitsis 9d83993db0 arch: arm: Remove unused generated offset symbols
Removes unused generated offset symbols under the ARM architecture.

Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
2023-02-23 16:44:07 +01:00
Kumar Gala 434ca63e2f arch: arm: limit FP16 support to Cortex-A or Cortex-R
FP16 isn't something that is supported on Cortex-M so limit the
Kconfig feature to Cortex-A or Cortex-R.

Signed-off-by: Kumar Gala <kumar.gala@intel.com>
2023-02-23 08:48:01 +01:00
Daniel Leung 44628735b8 linker: rom_start_offset: add to address instead of set
The CONFIG_ROM_START_OFFSET is supposed to be added to
the current when linking, instead of having the current
address set to it. So fix that.

Not sure why it worked up to this point, but llvm/clang/lld
complained that it could not move location counter backward.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-02-22 08:46:52 -05:00
Andrzej Głąbek 22b17e490b arch: arm: aarch32: Introduce z_arm_on_enter_cpu_idle() hook
Introduce an optional hook to be called when the CPU is made idle.
If needed, this hook can be used to prevent the CPU from actually
entering sleep by skipping the WFE/WFI instruction.

Signed-off-by: Andrzej Głąbek <andrzej.glabek@nordicsemi.no>
2023-02-21 15:03:30 +01:00
Nicolas Pitre 3c440af975 riscv: pmp: provision for implementations with partial PMP support
Looks like some implementors decided not to implement the full set of
PMP range matching modes. Let's rearrange the code so that any of those
modes can be disabled.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2023-02-20 10:57:11 +01:00
Nicolas Pitre ea34acb62c riscv: stack PMP: fix CONFIG_MPU_REQUIRES_POWER_OF_TWO_ALIGNMENT=y case
Let's honor CONFIG_MPU_REQUIRES_POWER_OF_TWO_ALIGNMENT even for kernel
stacks. This saves one global PMP slot when creating the guard area for
the IRQ stack, and some hw implementations might require that anyway.

With this changes, arch_mem_domain_max_partitions_get() becomes much
more reliable and tests/kernel/mem_protect is more likely to pass even
with the stack guard enabled.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2023-02-20 10:57:11 +01:00
Aaron Massey 1ee96f14af arch: Double privileged stack space with emulation
Additional privileged stack space is used by peripheral emulators when
userspace is enabled. This is largely due to additional function calls and
data structures allocated on the stack. This can potentially lead to stack
smashing if the privileged stack size isn't high enough, causing an
exception.

Increase the privileged stack space when userspace and peripheral emulation
are enabled.

Signed-off-by: Aaron Massey <aaronmassey@google.com>
2023-02-19 20:38:38 -05:00
Carlo Caione 61a204c831 riscv: Do not remove ESF when SOC_ISR_SW_UNSTACKING
When CONFIG_SOC_ISR_SW_UNSTACKING is defined, it's up to the custom soc
code to remove the ESF, because the software-managed part of the ESF is
depending on the hardware. Fix this in the ISR code.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2023-02-01 03:52:14 +09:00
Nicolas Pitre 10500f1b41 riscv: cope with MTVAL not updated on illegal instruction faults
Some implementations may not capture the faulting instruction in mtval
and set it to zero when an illegal instruction fault is raised This is
notably the case with QEMU version 7.0.0 when a CSR instruction is
involved.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2023-01-30 23:47:36 +00:00
Nicolas Pitre 83f849e00e riscv: FPU trap: catch CSR access to fcsr, frm and fflags
The FRCSR, FSCSR, FRRM, FSRM, FSRMI, FRFLAGS, FSFLAGS and FSFLAGSI
are in fact CSR instructions targeting the fcsr, frm and fflags
registers. They should be caught as FPU instructions as well.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2023-01-30 23:47:36 +00:00
Nicolas Pitre b2ffee7fe2 riscv: FPU switching fixes
- IRQ state for the interrupted context corresponds to the PIE bit not
  the IE bit.

- Restoring the saved FPU state should clear the entire field before
  or'ing wanted bits in.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2023-01-30 23:47:36 +00:00
Conor Paxton 804aa29f89 riscv: smp: use devicetree to map hartids to zephyr cpus
For RISC-V, the reg property of a cpu node in the devicetree describes
the low level unique ID of each hart. Using devicetree macro's, a list
of all cpus with status "okay" can be generated.

Using devicetree overlays, a hart or multiple harts can be marked as
"disabled", thus excluding them from the list. This allows platforms
that have non-zero indexed SMP capable harts to be functionally mapped
to Zephyr's sequential CPU numbering scheme.

On kernel init, if the application has MP_MAX_NUM_CPUS greater than 1,
generate the list of cpu nodes from the device tree with status "okay"
and  map the unique hartid's to zephyr cpu's

While we are at it, as the hartid is the value that gets passed to
z_riscv_secondary_cpu_init, use that as the variable name instead of
cpu_num

Signed-off-by: Conor Paxton <conor.paxton@microchip.com>
2023-01-30 23:45:35 +00:00
Conor Paxton 02391ed00d riscv: enable booting from non-zero indexed RISC-V hart
RISC-V multi-hart systems that employ a heterogeneous core complex are
not guaranteed to have the smp capable harts starting with a unique id
of zero, matching Zephyr's sequential zero indexed cpu numbering scheme.

Add option, RV_BOOT_HART to choose the hart to boot from.
On reset, check the current hart equals RV_BOOT_HART: if so, boot first
core. else, loop in the boot secondary core and wait to be brought up.

For multi-hart systems that are not running a Zephyr mp or smp
application, park the non zephyr related harts in a wfi loop.

Signed-off-by: Conor Paxton <conor.paxton@microchip.com>
2023-01-30 23:45:35 +00:00
David Reiss fb9cdee02d riscv: Fix whitespace in ISR handler
This one line was using spaces instead of tabs for indentation.

Signed-off-by: David Reiss <dreiss@meta.com>
2023-01-27 19:23:21 +09:00
Jordan Yates a3774fd51a arch: option to generate simplified error codes
Add an option to generate simplified error codes instead of more
specific architecture specific error codes. Enable this by default in
tests to make exception tests more generic across hardware.

Fixes #54053.

Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
2023-01-27 18:09:32 +09:00
Jamie McCrae ec7044437e treewide: Disable automatic argparse argument shortening
Disables allowing the python argparse library from automatically
shortening command line arguments, this prevents issues whereby
a new command is added and code that wrongly uses the shortened
command of an existing argument which is the same as the new
command being added will silently change script behaviour.

Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
2023-01-26 20:12:36 +09:00
Nicolas Pitre 6b9526c09b riscv: properly clear pending IPI flags
Commit 4f9b547ebd ("riscv: smp: prepare for more than one IPI type")
didn't clear pending IPI flags.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2023-01-25 19:51:03 -05:00
Nicolas Pitre a211970b42 riscv: improve contended FPU switching
We can leverage the FPU dirty state as an indicator for preemptively
reloading the FPU content when a thread that did use the FPU before
being scheduled out is scheduled back in. This avoids the FPU access
trap overhead when switching between multiple threads with heavy FPU
usage.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2023-01-24 15:26:18 +01:00
Nicolas Pitre ff07da6ff1 riscv: integrate the new FPU context switching support
FPU context switching is always performed on demand through the FPU
access exception handler. Actual task switching only grants or denies
FPU access depending on the current FPU owner.

Because RISC-V doesn't have a dedicated FPU access exception, we must
catch the Illegal Instruction exception and look for actual FP opcodes.

There is no longer a need to allocate FPU storage on the stack for every
exception making esf smaller and stack overflows less likely.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2023-01-24 15:26:18 +01:00
Nicolas Pitre cb4c0f6c94 riscv: smarter FPU context switching support
Instead of saving/restoring FPU content on every exception and task
switch, this replaces FPU sharing support with a "lazy" (on-demand)
context switching algorithm similar to the one used on ARM64.

Every thread starts with FPU access disabled. On the first access the
FPU trap is invoked to:

- flush the FPU content to the previous thread's memory storage;

- restore the current thread's FPU content from memory.

When a thread loads its data in the FPU, it becomes the FPU owner.

FPU content is preserved across task switching, however FPU access is
either allowed if the new thread is the FPU owner, or denied otherwise.
A thread may claim FPU ownership only through the FPU trap. This way,
threads that don't use the FPU won't force an FPU context switch.
If only one running thread uses the FPU, there will be no FPU context
switching to do at all.

It is possible to do FP accesses in ISRs and syscalls. This is not the
norm though, so the same principle is applied here, although exception
contexts may not own the FPU. When they access the FPU, the FPU content
is flushed and the exception context is granted FPU access for the
duration of the exception. Nested IRQs are disallowed in that case to
dispense with the need to save and restore exception's FPU context data.

This is the core implementation only to ease reviewing. It is not yet
hooked into the build.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2023-01-24 15:26:18 +01:00
Nicolas Pitre 4f9b547ebd riscv: smp: prepare for more than one IPI type
Right now this is hardcoded to z_sched_ipi(). Make it so that other IPI
services can be added in the future.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2023-01-24 15:26:18 +01:00
Daniel Leung db495a5ebe xtensa: stop execution under simulator for double exception
If running under Xtensa simulator, it is good to tell simulator
to stop execution once we reach double exception, as the current
double exception handler is simply an endless loop. If we turn
on tracing in the simulator, the output file would contain
an infinite iteration of this endless loop, and the simulator
needs to be stopped manually before the file size goes out of
control. So we need to tell the simulator to stop once
we reach this point instead of doing an endless loop.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-01-23 10:09:18 +00:00
Nicolas Pitre 883e9d367f riscv: translate CPU numbers to hartid values for IPI
Given the Zephyr CPU number is no longer tied to the hartid, we must
consider the actual hartid when sending an IPI to a given CPU. Since
those hartids can be anything, let's just save them in the cpu structure
as each CPU is brought online.

While at it, throw in some `get_hart_msip()` cleanups.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2023-01-19 13:48:42 +01:00
Nicolas Pitre 26d7bd47a0 riscv: decouple the Zephyr CPU number from the hart ID
Currently it is assumed that Zephyr CPU numbers match their hartid
value one for one. This assumption was relied upon to efficiently
retrieve the current CPU's `struct _cpu` pointer.

People are starting to have systems with a mix of different usage for
each CPU and such assumption may no longer be true.

Let's completely decouple the hartid from the Zephyr CPU number by
stuffing each CPU's `struct _cpu` pointer in their respective scratch
register instead. `arch_curr_cpu()` becomes more efficient as well.

Since the scratch register was previously used to store userspace's
exception stack pointer, that is now moved into `struct _cpu_arch`
which implied minor user space entry code cleanup and rationalization.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2023-01-19 13:48:42 +01:00
Nicolas Pitre 96a65e2fc0 riscv: don't include the secondary CPU boot code when not needed
Linker garbage collection couldn't work due to the explicit reference
in reset.S.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2023-01-19 13:48:42 +01:00
Dat Nguyen Duy 50e77c2f9a arch: arm: aarch32: cortex_a_r: disable interrupts before context switching
Ultil now Cortex A/R aarch32 implementation for context
switching expects that interrupts was disabled. This is
true if a context switching happens at thread context.

But if a context switching happens at last action during
interrupt context, this assumption is not true because the
interrupts are still enabled (to allow nesting interrupts).

Disable interrupts at the last interrupt action to ensure
the interrupts are always disabled before context switching
is processed

Signed-off-by: Dat Nguyen Duy <dat.nguyenduy@nxp.com>
2023-01-18 16:22:29 +01:00
Flavio Ceolin c896b1e911 userspace: Do not use --relax flag
In platforms where the linker is capable of doing global optimizations,
like relaxing address mode and synthesize new instructions, Zephyr has to
disable it when enabling USERSPACE since the build expects that address
don't change after the first stage build.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2023-01-16 11:20:32 +00:00
Stephanos Ioannidis 4a64bfe351 treewide: Use CONFIG_CPP instead of CONFIG_CPLUSPLUS
This commit updates all in-tree code to use `CONFIG_CPP` instead of
`CONFIG_CPLUSPLUS`, which is now deprecated.

Signed-off-by: Stephanos Ioannidis <stephanos.ioannidis@nordicsemi.no>
2023-01-13 17:42:55 -05:00
Jordan Yates 35e78c4502 arch: arm: return arm specific fatal error reasons
Return specific fault reasons instead of the generic
`K_ERR_CPU_EXCEPTION`, which provides minimal debugging aid.

Fixes #53093.

Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
2023-01-12 10:01:05 +01:00
Carlo Caione 34e0294652 arm: aarch32: Use proper sys functions for cache mainteinance
This patchset is fixing two things:

1. The proper sys_* functions are used for cache mainteinance
   operations.

2. To check the status of the L1 cache the SCB registers are probed so
   the code is assuming a core architecture cache is present, thus make
   the code conditionally compiled on CONFIG_ARCH_CACHE.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2023-01-10 18:22:32 -05:00
Tomasz Bursztyka c9ef674f0e arch/x86: Fix compilation error
arch_dcache_range() function does not exist anymore, nor K_CACHE_WB
macro. Removing it entirely.

arch_dcache_flush_range() signature changed, so relevantly applying
these.

Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
2023-01-10 14:06:33 +00:00
Carlo Caione 43a61329cc riscv: Allow SOC to override arch_irq_{lock,unlock,unlocked}
RISC-V has a modular design. Some hardware with a custom interrupt
controller needs a bit more work to lock / unlock IRQs.

Account for this hardware by introducing a set of new
z_soc_irq_* functions that can override the default behaviour.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2023-01-09 19:21:39 +01:00
Flavio Ceolin a6b4af4a93 ia32: irq: Remove unnecessary header
printk is not used anywhere in this file. Just remove it.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2023-01-09 12:07:28 -05:00
Flavio Ceolin 1e83113e4f x86: fatal: Remove possible deadcode
Move esf_get_code to inside CONFIG_EXCEPTION_DEBUG guard.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2023-01-09 12:07:28 -05:00
Flavio Ceolin 816954156c x86: fatal: Remove possible deadcode
esf_get_sp is used only when CONFIG_THREAD_STACK_INFO is enabled.
Just move this function around to be inside an ifdef guard.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2023-01-09 12:07:28 -05:00
Carlo Caione c13d23a43e riscv: Add support for hardware stacking / unstacking
Some RISC-V SoCs implement a mechanism for hardware supported stacking /
unstacking of registers during ISR / exceptions. What happens is that on
ISR / exception entry part of the context is automatically saved by the
hardware on the stack without software intervention, and the same part
of the context is restored by the hardware usually on mret.

This is currently not yet supported by Zephyr, where the full context
must be saved by software in the full fledged ESF. This patcheset is
trying to address exactly this case.

At least three things are needed to support in a general fashion this
problem: (1) a way to store in software only the part of the ESF not
already stacked by hardware, (2) a way to restore in software only the
part of the context that is not going to be restored by hardware and (3)
a way to define a custom ESF.

Point (3) is important because the full ESF frame is now composed by a
custom part depending on the hardware (that can choose which register to
stack / unstack and the order they are saved onto the stack) and a part
defined in software for the remaining part of the context.

In this patch a new CONFIG_RISCV_SOC_HAS_ISR_STACKING is introduced that
enables the code path supporting the three points by the mean of three
macros that must be implemented by the user in a soc_stacking.h file:
SOC_ISR_SW_STACKING, SOC_ISR_SW_UNSTACKING and SOC_ISR_STACKING_ESF
(refer to the symbol help for more details).

This is an example of soc_isr_stacking.h for an hardware that doesn't do
any hardware stacking / unstacking but everything is managed in
software:

    #ifndef __SOC_ISR_STACKING
    #define __SOC_ISR_STACKING

    #if !defined(_ASMLANGUAGE)

    #define SOC_ISR_STACKING_ESF_DECLARE \
    	struct __esf { \
    		unsigned long ra; \
    		unsigned long t0; \
    		unsigned long t1; \
    		unsigned long t2; \
    		unsigned long t3; \
    		unsigned long t4; \
    		unsigned long t5; \
    		unsigned long t6; \
    		unsigned long a0; \
    		unsigned long a1; \
    		unsigned long a2; \
    		unsigned long a3; \
    		unsigned long a4; \
    		unsigned long a5; \
    		unsigned long a6; \
    		unsigned long a7; \
    		unsigned long mepc; \
    		unsigned long mstatus; \
    		unsigned long s0; \
    	} __aligned(16)
    #else

    #define SOC_ISR_SW_STACKING \
    	addi sp, sp, -__z_arch_esf_t_SIZEOF; \
    	DO_CALLER_SAVED(sr);

    #define SOC_ISR_SW_UNSTACKING \
    	DO_CALLER_SAVED(lr);

    #endif /* _ASMLANGUAGE */
    #endif /* __SOC_ISR_STACKING */

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2023-01-09 10:15:07 +01:00
Piotr Jasiński 7fa0af01cf Kconfig: add config for low-priority debug mon isr
Debug monitor needs to be configured to a low priority in order to be
useful for debugging (to prioritize other interrupts when waiting on a
breakpoint).
Added a config that configures the interrupt this way.

Signed-off-by: Piotr Jasiński <piotr.jasinski@nordicsemi.no>
2022-12-28 12:00:46 +01:00
Piotr Jasiński 1fe4b1eb90 linker: make debug monitor isr symbol weak
It seems that currently it's impossible to create a custom
implementation for debug monitor exception without updating the vector
table (z_arm_debug_monitor maps to fault).
My proposition is to make this symbol weak, so that it can be overriden.

Signed-off-by: Piotr Jasiński <piotr.jasinski@nordicsemi.no>
2022-12-28 12:00:46 +01:00
Lucas Tamborrino 9e289c1b20 arch: xtensa: save FPU register in context switching
Save FP user register and FP register file during context switch.

This change enables shared FP registers mode using CONFIG_FPU_SHARING.

Since there is no lazy stacking, the FPU registers will be saved regardless
of whether floating point calculations are performed in the threads when
CONFIG_FPU_SHARING is enabled. This require 72 additional bytes in the
stack memory.

Signed-off-by: Lucas Tamborrino <lucas.tamborrino@espressif.com>
2022-12-27 13:23:17 +01:00
Evgeniy Paltsev 93b98fc2de ARC: introduce reworked irq_offload implementation
Use interrupts (with dedicated interrupt line) for irq_offload
instead of exception-based implementation.

That allows to implement IRQ_OFFLOAD without adding special code
to interrupt / exception path for IRQ_OFFLOAD handling and,
moreover, test the real interrupt code.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
2022-12-20 22:51:24 +01:00
Siyuan Cheng 5854670b98 DSP: add dsp unit test
add dsp context switch test
add complex multiplication test for ARC processor

Signed-off-by: Siyuan Cheng <siyuanc@synopsys.com>
2022-12-19 11:56:55 +01:00
Siyuan Cheng 70ff49af37 DSP: add DSP support for ARC processor
add DSP reg in context switch
add AGU reg in context switch to support XY mem
add thread option and API to dis/enable DSP switch

Signed-off-by: Siyuan Cheng <siyuanc@synopsys.com>
2022-12-19 11:56:55 +01:00
Henri Xavier 45af717a66 arch/arm64: Implement ASID support in ARM64 MMU
Improves context-switch performance.

TLB invalidation and the nG bit are used conservatively. This could
be improved in future work.

Tested with tests/benchmarks/sched_userspace:

BEFORE:
```
Swapping  2 threads: 161562583 cyc & 1000000 rounds ->   1615 ns per ctx
Swapping  8 threads: 161569289 cyc & 1000000 rounds ->   1615 ns per ctx
Swapping 16 threads: 161649163 cyc & 1000000 rounds ->   1616 ns per ctx
Swapping 32 threads: 163487880 cyc & 1000000 rounds ->   1634 ns per ctx
```

AFTER:
```
Swapping  2 threads: 18129207 cyc & 1000000 rounds ->    181 ns per ctx
Swapping  8 threads: 49702891 cyc & 1000000 rounds ->    497 ns per ctx
Swapping 16 threads: 55898650 cyc & 1000000 rounds ->    558 ns per ctx
Swapping 32 threads: 58059704 cyc & 1000000 rounds ->    580 ns per ctx
```

Signed-off-by: Henri Xavier <datacomos@huawei.com>
2022-12-13 17:21:11 +09:00
Duong Vu Nam 84b5d39cb6 arm: aarch32: config static regions with MPU disabled
There is the possibility that when reconfiguring the static regions,
some data that must be accessed is temporarily not accesible due to the
change on the MPU regions configuration. Workaround by disabling MPU
when doing the reconfiguration, same as with dynamic regions, until BR
can be enabled.

Signed-off-by: Duong Vu Nam <duong.vunam@nxp.com>
2022-12-12 10:39:31 +01:00
Duong Vu Nam 47cce0573e arch: update alignment for Cortex-R52
Cortex-R52 doesn't require power of 2 alignment

Signed-off-by: Duong Vu Nam <duong.vunam@nxp.com>
2022-12-12 10:39:31 +01:00
Duong Vu Nam b24f4625f0 arch: support nocache for Cortex-R52
Config NOCACHE_MEMORY depend on ARCH_HAS_NOCACHE_MEMORY_SUPPORT. Enable
ARCH_HAS_NOCACHE_MEMORY_SUPPORT for Cortex-R52 to run NXP S32Z/E with
nocache attibute.

Enable nocache in each driver use it.

Signed-off-by: Duong Vu Nam <duong.vunam@nxp.com>
2022-12-12 10:39:31 +01:00
Duong Vu Nam ee639b64c3 arch: aarch32: add caches capability to Cortex-R52
Support I/D cache for Cortex-R52 to run with cache on NXP S32Z/E.
Make sure no data is present in the D-Cache before initializing mpu

Signed-off-by: Duong Vu Nam <duong.vunam@nxp.com>
2022-12-12 10:39:31 +01:00
Carlo Caione 4b58739034 arc: cache: Rework cache API
And use the new API.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2022-12-01 13:40:56 -05:00
Carlo Caione c96313ec3f arm: cache: Rework cache API
And use the new API.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2022-12-01 13:40:56 -05:00
Carlo Caione 385f06e6a1 arm64: cache: Rework cache API
And use the new API.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2022-12-01 13:40:56 -05:00
Carlo Caione 189cd1f4a2 cache: Rework cache API
The cache operations must be quick, optimized and possibly inlined. The
current API is clunky, functions are not inlined and passing parameters
around that are basically always known at compile time.

In this patch we rework the cache functions to allow us to get rid of
useless parameters and make inlining easier.

In particular this changeset is doing three things:

1. `CONFIG_HAS_ARCH_CACHE` is now `CONFIG_ARCH_CACHE` and
   `CONFIG_HAS_EXTERNAL_CACHE` is now `CONFIG_EXTERNAL_CACHE`

2. The cache API has been reworked.

3. Comments are added.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2022-12-01 13:40:56 -05:00
Evgeniy Paltsev d9b3efa550 ARC: cleanup & modify _st32_huge_offset as it becomes widely used
As we start to use _st32_huge_offset in other places (i.e DSP code)
introduce optimized version with address shifted instruction which
doesn't produce extra instruction.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
2022-11-29 09:48:31 +01:00
Evgeniy Paltsev ae79de1930 ARC: MWDT add TLS support
Add thread local storage support for ARC MWDT toolchain.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
2022-11-29 09:48:23 +01:00
Evgeniy Paltsev d2488d2160 ARC: control shared (common) interrupts via IDU
Control shared interrupts enabling/disabling via IDU.
With that we can easily enable and disable them for all cores
in one place.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
2022-11-28 17:44:54 +01:00
Evgeniy Paltsev 761be25c43 ARC: make ARC_MP_PRIMARY_CPU_ID definition public
Make ARC_MP_PRIMARY_CPU_ID definition public so it can be used in
other ARC code.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
2022-11-28 17:44:54 +01:00
Evgeniy Paltsev 93049141f9 ARC: TLS: cleanup tls register setup
Cleanup tls register setup marcos by using generic one.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
2022-11-28 17:44:20 +01:00
Evgeniy Paltsev cae6696a75 ARC: ARCv2: GCC: restore default TLS behavior for all SDKs
In 0.15.2 SDK we specify r26 as default register for TLS cached
pointer, so it isn't used by compiler even if TLS support isn't
enabled. Restore the previous behavior - so if we don't use
TLS in Zephyr the register isn't stay reserved.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
2022-11-28 17:44:13 +01:00
Wilfred Mallawa 7ac69dc4ca arch/arm/core: fixup typo
Fixup trivial typo.

Signed-off-by: Wilfred Mallawa <wilfred.mallawa@gmail.com>
2022-11-27 12:11:44 +01:00
Henri Xavier b54ba9877f arm64: implement arch_system_halt
When PSCI is enabled, implement `arch_system_halt` using
PSCI_SHUTDOWN.

Signed-off-by: Henri Xavier <datacomos@huawei.com>
2022-11-23 11:37:08 +01:00
Théophile Ranquet 6d8100424a arm: set low exception vector location
The code in prep_c sets VBAR to relocate vector from 0x0, assuming the
low vector bit in SCTLR to be clear. This isn't the case on all
hardware, so set it explicitly to support those.

Signed-off-by: Théophile Ranquet <theophile.ranquet@gmail.com>
2022-11-23 11:36:26 +01:00
honglin leng 676441e0ca xtensa: remove xtensa asm unused header
1. this header is no use for asm

2. if use xclib, this header include xclib stdbool, and expand to typedef

Signed-off-by: honglin leng <a909204013@gmail.com>
2022-11-22 12:45:33 +09:00
Benjamin Gwin b5fe0f7304 arch: arm: mpu: Add support for disabling the background address map
Arm provides a default address map defining default behaviors for
certain address ranges, which can be overlayed with additional regions
in the MPU. Users may also turn off this background map, so that only
regions explicitly programmed in the MPU are allowed.

This provides a Kconfig so that platforms using a non-standard address
map may disable the background address map and provide their own
explicit MPU regions.

Signed-off-by: Benjamin Gwin <bgwin@google.com>
2022-11-16 11:21:06 +01:00
Andrzej Głąbek 6578b218ca arch: arm: aarch32: cortex_m: fault: Fix SCB->CFSR access helpers
This is a follow-up to commit f400c94adf.

Fix typos in names of introduced macros (*STR -> *SR) and cast their
values to uint32_t to avoid warnings reported for messages formatted
with %x.

Signed-off-by: Andrzej Głąbek <andrzej.glabek@nordicsemi.no>
2022-11-11 08:56:18 +00:00
Keith Packard 10294269f7 arch/arc: Detect arcv2 TLS register automatically
The compiler defines __ARC_TLS_REGNO__ as the number of the
register used for TLS variables. Use that instead of hard-coding
a specific register.

Signed-off-by: Keith Packard <keithp@keithp.com>
2022-11-08 10:45:44 +01:00
Stephanos Ioannidis ae0437d2df arch: posix: Declare _posix_zephyr_main with int return type
This commit updates the `_posix_zephyr_main` declaration to use the
return type of `int` instead of `void` when `CONFIG_CPP_MAIN=y` (i.e.
C++-compliant main() support is enabled) so that Zephyr applications
defining their main() in a C++ source file can make use of the proper
main() definition of `int main(void)` as required by the C++ standard.

Note that the forward declaration of `_posix_zephyr_main` is required
if and only if the main() is defined in a C++ source file (i.e. when
`CONFIG_CPP_MAIN=y`).

Signed-off-by: Stephanos Ioannidis <stephanos.ioannidis@nordicsemi.no>
2022-11-05 16:41:45 +09:00
Ederson de Souza 8ac6f74a7d arch/xtensa: Enable code relocation
Besides adding ARCH_HAS_CODE_DATA_RELOCATION, this patch also adds
support for the "sample_controller" SoC (used by qemu_xtensa) as
demonstration.

As Xtensa lacks a common linker script at the arch level, enabling it
for each platform will be a piecemeal effort. This patch adds it to the
`soc/xtensa/sample_controller` SoC. Basically, default RAMABLE_REGION is
set to be called "RAM", and hooks are inserted so that
gen_relocate_app.py can add the relevant linker bits.

Also, `tests/application_developent/code_relocation` was tweaked to
support the qemu_xtensa platform. Basically, add the relevant linker
script and ensure that relevant memory regions have their program header
(PHDR) associated.

Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
2022-11-03 10:25:07 +01:00
Henri Xavier 55df3852bf arm64: Add comment that arch_curr_cpu and get_cpu must be synced
The is code duplication as one is in C, and the other is an assembly
macro. As there is no easy way to find out about this duplication,
adding a comment seems the best way to go.

Signed-off-by: Henri Xavier <datacomos@huawei.com>
2022-11-02 16:08:00 -05:00
Kumar Gala c7fe428df3 arch: arc: Convert to CONFIG_MP_MAX_NUM_CPUS
Convert CONFIG_MP_NUM_CPUS to CONFIG_MP_MAX_NUM_CPUS as we work on
phasing out CONFIG_MP_NUM_CPUS.

Signed-off-by: Kumar Gala <kumar.gala@intel.com>
2022-11-02 14:35:32 -04:00
Evgeniy Paltsev 3d24a07c6b ARC: disallow build for secure configuration with firq
Such configuration isn't supported by HW.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
2022-11-02 15:42:00 +09:00
Henrik Brix Andersen 3c99a1e015 drivers: pcie: reintroduce support for I/O BARs
Reintroduce support for accessing I/O BARs which was removed in
43d84147d9.

Signed-off-by: Henrik Brix Andersen <henrik@brixandersen.dk>
2022-11-01 15:22:31 -04:00
Carles Cufi ab05448307 arch: Add a CONFIG_LITTLE_ENDIAN hidden Kconfig option
In preparation for using CONFIG_*_ENDIAN instead of __BYTE_ORDER__, add
a reflection of CONFIG_BIG_ENDIAN that will allow users to conditionally
compile with #ifdef instead of #ifndef while keeping the default case
(little endian) as the first block in the conditional compilation.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-10-28 19:23:46 +09:00
Keith Packard e802d53900 x86: buf return from gdb_reg_readone is not a string
The buffer contents returned from arch_gdb_reg_readone is a counted array
of bytes, not a C string. Use memcpy instead of strcpy for the failure
return path to avoid compiler warning about missing NUL termination.

Signed-off-by: Keith Packard <keithp@keithp.com>
2022-10-26 12:00:04 +02:00
Nicolas Pitre 74fbf8edf5 Revert "riscv: PMP-based stack guard is incompatible with stack sentinel"
This reverts commit a7b5d606c7.

The assumption behind that commit was wrong. The software-based stack
sentinel writes to the very bottom of the _writable_ stack area i.e.
right next to the actual PMP based guard area. So they are compatible.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-10-25 10:53:34 +02:00
Kumar Gala d60c62e457 arch: arm64: Convert to CONFIG_MP_MAX_NUM_CPUS
Convert CONFIG_MP_NUM_CPUS to CONFIG_MP_MAX_NUM_CPUS as we work on
phasing out CONFIG_MP_NUM_CPUS.

Signed-off-by: Kumar Gala <kumar.gala@intel.com>
2022-10-25 10:52:34 +02:00
Kumar Gala 3369997dd2 arch: riscv: Convert to CONFIG_MP_MAX_NUM_CPUS
Convert CONFIG_MP_NUM_CPUS to CONFIG_MP_MAX_NUM_CPUS as we work on
phasing out CONFIG_MP_NUM_CPUS.

Signed-off-by: Kumar Gala <kumar.gala@intel.com>
2022-10-25 10:52:25 +02:00
Kumar Gala c73b0e28a3 arch: x86: Convert to CONFIG_MP_MAX_NUM_CPUS
Convert CONFIG_MP_NUM_CPUS to CONFIG_MP_MAX_NUM_CPUS as we work on
phasing out CONFIG_MP_NUM_CPUS.

Signed-off-by: Kumar Gala <kumar.gala@intel.com>
2022-10-25 09:52:17 +03:00
Kumar Gala a1195ae39b smp: Move for loops to use arch_num_cpus instead of CONFIG_MP_NUM_CPUS
Change for loops of the form:

for (i = 0; i < CONFIG_MP_NUM_CPUS; i++)
   ...

to

unsigned int num_cpus = arch_num_cpus();
for (i = 0; i < num_cpus; i++)
   ...

We do the call outside of the for loop so that it only happens once,
rather than on every iteration.

Signed-off-by: Kumar Gala <kumar.gala@intel.com>
2022-10-21 13:14:58 +02:00
Kumar Gala fc95ec98dd smp: Convert #if to use CONFIG_MP_MAX_NUM_CPUS
Convert CONFIG_MP_NUM_CPUS to CONFIG_MP_MAX_NUM_CPUS as we work on
phasing out CONFIG_MP_NUM_CPUS.

Signed-off-by: Kumar Gala <kumar.gala@intel.com>
2022-10-20 22:04:10 +09:00
Keith Packard 77abc49952 arch/x86/x86-64: Set TLS before calling z_thread_mark_switched_in
Move the FS_BASE MSR code to to the top of __resume to ensure
that %fs relative addressing run in the thread switching hook
works.

Signed-off-by: Keith Packard <keithp@keithp.com>
2022-10-19 16:00:00 +02:00
Gerard Marull-Paretas 178bdc4afc include: add missing zephyr/irq.h include
Change automated searching for files using "IRQ_CONNECT()" API not
including <zephyr/irq.h>.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2022-10-17 22:57:39 +09:00
Kumar Gala c778eb2a56 smp: Move arrays to use CONFIG_MP_MAX_NUM_CPUS
Move to use CONFIG_MP_MAX_NUM_CPUS for array size declarations instead
of CONFIG_MP_NUM_CPUS.

Signed-off-by: Kumar Gala <kumar.gala@intel.com>
2022-10-17 14:40:12 +09:00
Anas Nashif 3d85ae37b6 arm: add missing includes
Add missing includes that were previously included indirectly.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-10-11 18:37:53 -04:00
Gerard Marull-Paretas ac63bca423 include: add missing sys_clock.h include
Some headers made use of types defined in sys_clock.h (e.g. k_timeout_t)
without including it.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2022-10-11 18:05:17 +02:00
Gerard Marull-Paretas acc8cb4bc8 include: add missing irq.h include
Some modules use the IRQ API without including the necessary headers.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2022-10-11 18:05:17 +02:00
Gerard Marull-Paretas fe32c4e273 arch: x86: core: include kernel.h first
There seems to be an unresolved dependency chain in some x86 arch
headers, this file needs kernel.h to be included before arch_data/func.
This patch is a workaround, but problem should be fixed properly at some
point.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2022-10-11 18:05:17 +02:00
Pieter De Gendt 1694c0d60d arch: posix: Add defaults for recommended stack sizes
The CONFIG_STACK_SENTINEL adds 4 bytes to the stack. Take these
into account for CONFIG_ARCH_POSIX_RECOMMENDED_STACK_SIZE.

Signed-off-by: Pieter De Gendt <pieter.degendt@basalte.be>
2022-10-11 15:22:57 +02:00
Peter Marheine 5eb75b81f7 arm: rename default RAM region from 'SRAM' to 'RAM'
It's useful for RAMABLE_REGION to have a uniform name when
CODE_DATA_RELOCATION is supported, because otherwise the build system
needs to be aware of how the region name differs between architectures.
Since architectures tend to prefer one of 'SRAM' or 'RAM' for that
region, prefer to use 'RAM' as the more general term.

Signed-off-by: Peter Marheine <pmarheine@chromium.org>
2022-10-03 10:09:53 +02:00
Nicolas Pitre 1f362a81f1 riscv: fix crash resulting from touching the initial stack's guard area
The interrupt stack is used as the system stack during kernel
initialization while IRQs are not yet enabled. The sp register is
set to z_interrupt_stacks + CONFIG_ISR_STACK_SIZE.

CONFIG_ISR_STACK_SIZE only represents the desired usable stack size.
This does not take into account the added guard area. Result is a stack
whose pointer is much closer to the trigger zone than expected when
CONFIG_PMP_STACK_GUARD=y, and the SMP configuration in particular pushes
it over the edge during many CI test cases.

Worse: during early init we're not quite ready to handle exceptions
yet and complete havoc ensues with no meaningful debugging output.

Make sure the early assembly code locates the actual top of the stack
by generating a constant with its true size.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-09-28 07:53:56 +00:00
Nicolas Pitre a7b5d606c7 riscv: PMP-based stack guard is incompatible with stack sentinel
The software-based stack sentinel writes to the very bottom of the
stack area triggering the PMP stack protection. Obviously they can't
be used together.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-09-28 07:53:56 +00:00
Nicolas Pitre 4a733694ca riscv: pmp: fix stackguard when used on SMP
The IRQ stack in particular is different on each CPU, and so is its
stack guard PMP entry value. This creates 2 issues:

- The assertion ensuring the last global PMP address is the same
  for each CPU does fail;

- That last global PMP address can't be relied upon to create a
  single-slot per-thread TOR mapping.

Fix both issues by not remembering the actual address for the last
global entry but a dummy address instead that is guaranteed not to
match any opportunistic single-slot TOR mapping.

While at it, lock that IRQ stack guard PMP entry.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-09-28 07:53:56 +00:00
Stephanos Ioannidis 2689590e67 arch: arm64: Disable ldp/stp Qn for consecutive 32-byte loads/stores
GCC may generate ldp/stp instructions with the Advanced SIMD Qn
registers for consecutive 32-byte loads and stores.

This commit disables this GCC behaviour because saving and restoring
the Advanced SIMD context is very expensive, and it is preferable to
keep it turned off by not emitting these instructions for better
context switching performance.

Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
2022-09-23 12:10:25 +02:00
Ruud Derwig 9bccb5cc4b ARC: fx possible memory corruption with userspace
Use  Z_KERNEL_STACK_BUFFER instead of
Z_THREAD_STACK_BUFFER for initial stack.

Fixes #50467

Signed-off-by: Ruud Derwig <Ruud.Derwig@synopsys.com>
2022-09-21 18:46:06 +00:00
Chen Peng1 4c85c84ec2 x86: Kconfig: update dependency for X86_FP_USE_SOFT_FLOAT
Update Kconfig dependency for X86_FP_USE_SOFT_FLOAT.

Signed-off-by: Chen Peng1 <peng1.chen@intel.com>
2022-09-21 18:43:11 +00:00
Nicolas Pitre c76d8c88c0 riscv: smp: fix secondary cpus' initial stack
Z_THREAD_STACK_BUFFER() must not be used here. This is meant for stacks
defined with K_THREAD_STACK_ARRAY_DEFINE() whereas in this case we are
given a stack created with K_KERNEL_STACK_ARRAY_DEFINE().

If CONFIG_USERSPACE=y then K_THREAD_STACK_RESERVED gets defined with
a bigger value than K_KERNEL_STACK_RESERVED. Then Z_THREAD_STACK_BUFFER()
returns a pointer that is more advanced than expected, resulting in a
stack pointer outside its actual stack area and therefore memory
corruption ensues.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-09-21 09:01:58 +00:00
Nicolas Pitre 1c857f37da riscv: pmp: fix SMP build with assertion enabled
Fix SMP build with assertion enabled.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-09-20 09:39:35 +02:00
Enjia Mai d9206aa29b arch: arm: userspace: fix the incorrect ssf under bad syscall
The parameter ssf of the handler_bad_syscall got null pointer
due to that the R1 does not push into the stack in a right
order on cortex-M0. Adjust the pushing order of stack to make
the ssf being passed correctly.

Fixes #50146.

Signed-off-by: Enjia Mai <enjia.mai@intel.com>
2022-09-19 09:17:26 +02:00
Andy Ross 99dd845067 arch/posix: Fix main() renaming trickery
It turns out that SOF is already using a symbol named
"zephyr_app_main()", so this produces a collision.  Pick something
that looks more relevant to "posix", and put an underscore on it (it's
a "system" symbol, after all).

Signed-off-by: Andy Ross <andyross@google.com>
2022-09-15 16:23:11 +00:00
Andy Ross ec44bc435c arch/posix: Fix 32 bit x86 fuzzing
It seems like libfuzzer wants to relocate 32 bit instrumented code
sections at runtime at addresses different than the ones in the ELF
file.  This is problematic, because Zephyr files are compiled
statically and so will crash the first time they try to jump to an
absolute .text address (basically at the first function call after a
fuzzer entry point).

It seems that building with -fPIC is enough to defeat this (we use the
host linker script, which will manage the GOT/PLT entries for us),
which will work as long as the fuzzer isn't playing games with data
other than text.  None of this seems to be documented, so... I guess
it's as good as we can get.  It works, at least.

(x86_64 binaries don't show the same behavior, they run where they
were linked)

Signed-off-by: Andy Ross <andyross@google.com>
2022-09-15 16:23:11 +00:00
Kai Vehmanen 48276fde5c xtensa: use lower-case hex in backtrace output
Align backtrace output with the style used in rest of the codespace.
This makes it more convenient to compare the backtrace to e.g. objdump
output.

Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
2022-09-09 14:09:33 -05:00
Huifeng Zhang 3d81d7f23f arch: arm64: fix the wrong way to send ipi interrupt
On GICv3, when we send an IPI interrupt, aff3, aff2 and aff1 should
be assigned a value corespond to a PE for which interrupt will be
generated. target_list only corresponds to aff0.

On real hardware, aff3, aff2, aff1 and aff0 should be treated as a
whole to determine a PE.

Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
2022-09-09 16:36:37 +00:00
Huifeng Zhang 3ef14cae5e arch: arm64: init VMPIDR_EL2 in z_arm64_el2_init
VMPIDR_EL2 is assigned the value returned by EL2 reads of MPIDR_EL1

MPIDR_EL1 is the register holding the Multiprocessor ID which is to
identify different cores. Because of the virtualization requirements
for AArch64, MPIDR_EL1 should be virtualized (the different virtualized
cores can run on the same physical core). Thus the value of MPIDR_EL1
should be switched when the VM is switched. Setting the VMPIDR_EL2 is
the way to change the value returned by EL1 reads of MPIDR_EL1. Even
without virtualization, we still need to set VMPIDR_EL2 during booting
at EL2 or EL3. Otherwise, all cores' IDs are zero at the EL1 stage
which will break the SMP system.

Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
2022-09-09 16:36:37 +00:00
Mateusz Sierszulski 2ed5763baa arch: riscv: core: Place vectors section through zephyr_linker_sources()
This commit is fixing placing the vectors section through
zephyr_linker_sources(ROM_START ...) (as done in the ARM
architecture port) so its order can be adjusted by SORT_KEY.

Fixes #49903

Signed-off-by: Mateusz Sierszulski <msierszulski@antmicro.com>
2022-09-08 10:39:31 +02:00
Andy Ross b141551cba arch/xtensa: Properly namespace special register API
The Xtensa arch has historically had state/user register accessor
macros with bare three-byte symbol names.  I think this might have
been in the original Cadence-contributed arch integration, but I'm not
sure.  In any case they also exist in the same names in vendor
HAL/toolchain code and are causing collisions.  We never should have
had these symbols exposed in our header.

Put them under an XTENSA_ prefix to decollide.

Signed-off-by: Andy Ross <andyross@google.com>
2022-09-07 20:28:06 -04:00
Gerard Marull-Paretas be38456279 include: types: remove ulong_t
ulong_t was mainly used in MIPS/RISC-V. Just use "unsigned long".

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2022-09-06 18:16:33 +02:00
Gerard Marull-Paretas 79e6b0e0f6 includes: prefer <zephyr/kernel.h> over <zephyr/zephyr.h>
As of today <zephyr/zephyr.h> is 100% equivalent to <zephyr/kernel.h>.
This patch proposes to then include <zephyr/kernel.h> instead of
<zephyr/zephyr.h> since it is more clear that you are including the
Kernel APIs and (probably) nothing else. <zephyr/zephyr.h> sounds like a
catch-all header that may be confusing. Most applications need to
include a bunch of other things to compile, e.g. driver headers or
subsystem headers like BT, logging, etc.

The idea of a catch-all header in Zephyr is probably not feasible
anyway. Reason is that Zephyr is not a library, like it could be for
example `libpython`. Zephyr provides many utilities nowadays: a kernel,
drivers, subsystems, etc and things will likely grow. A catch-all header
would be massive, difficult to keep up-to-date. It is also likely that
an application will only build a small subset. Note that subsystem-level
headers may use a catch-all approach to make things easier, though.

NOTE: This patch is **NOT** removing the header, just removing its usage
in-tree. I'd advocate for its deprecation (add a #warning on it), but I
understand many people will have concerns.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2022-09-05 16:31:47 +02:00
Gerard Marull-Paretas 082043c6e8 drivers: display: intel_multibootfb: convert to DT
Convert the device to be Devicetree based. Adjusted tests and other
areas that were using old Kconfig properties.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2022-09-02 14:16:08 +02:00
Gerard Marull-Paretas 11860face3 drivers: display: framebuffer: rework to make it self-contained
The "framebuf" driver was an incomplete driver expecting _clients_ to
implement missing functionality (i.e. init and device definition)
outside of the driver. This pattern of scattering driver code throughout
the tree is not common (if used at all). If certain drivers share
functionality, one can create a common module within the subsystem (see
e.g. ILI9XXX drivers).

The _generic_ framebuffer code was only used to implement the Intel
Multiboot framebuffer driver. This patch centralizes all the scattered
code in the subsystem and adjusts the driver name to "intel_multibootfb"
to make things clear. If there's ever another framebuffer driver that
shares code, it can be split into multiple modules.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2022-09-02 14:16:08 +02:00
Gerard Marull-Paretas 5954ea1c65 arch: arm64: core: smp: use DT_FOREACH_CHILD_STATUS_OKAY_SEP
Avoid auxiliary macros by using DT_FOREACH_CHILD_STATUS_OKAY_SEP.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2022-08-30 16:19:57 +02:00
Ederson de Souza 4d66eedd10 arch/xtensa/core: Fix timing API issues
Two issues:
 - A unnecessary parentheses pair caused rounding errors (by truncating
   a small value before multiplying it).
 - arch_timing_cycles_to_ns_avg() wasn't actually converting the result
   to nanoseconds.

Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
2022-08-29 16:09:50 -04:00
Stephanos Ioannidis 40bbf78d77 arch: arc: Rename ARC64 output format to elf64-littlearc64
This commit renames the ARC64 output format from `elf64-littlearc` to
`elf64-littlearc64` as required by the updated ARC patches for the GCC
12.1 release.

Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
2022-08-29 16:57:18 +02:00
Carlo Caione 6503795dc1 riscv: Introduce BitManip extensions
Add Zba, Zbb, Zbc and Zbs BitManip extensions.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2022-08-29 16:57:18 +02:00
Carlo Caione 5fece03d7d riscv: Introduce Zicsr and Zifencei extensions
And enable the new extensions on all the SoCs.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2022-08-29 16:57:18 +02:00
Mahesh Mahadevan c029b081cc cmake: Add support to add symbols to nocache section
This PR allows the user to add symbols to the nocache
section. The use for this could be as follows

zephyr_linker_sources_ifdef(CONFIG_NOCACHE_MEMORY
  NOCACHE_SECTION
  nocache.ld
)

nocache.ld (as shown below) can define additional
symbols to  go into the nocache section

. = ALIGN(4);
KEEP(*(NonCacheable))

Signed-off-by: Mahesh Mahadevan <mahesh.mahadevan@nxp.com>
2022-08-29 11:19:48 +02:00
Evgeniy Paltsev 99142065fc ARC: add non-multithreading mode support
Add non-multithreading mode support for all ARC non-SMP
targets.

Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
2022-08-26 21:38:56 -04:00