Commit graph

6012 commits

Author SHA1 Message Date
Marcin Szymczyk
0b20e2afa7 arch: riscv: skip isr.S when SW ISR table is not generated
`isr.S` depends on `CONFIG_GEN_SW_ISR_TABLE`.
Do not build it if SW ISR table is not present.

Signed-off-by: Marcin Szymczyk <marcin.szymczyk@nordicsemi.no>
2024-06-13 16:57:32 -04:00
Yong Cong Sin
61a0f9f1c0 arch: riscv: stop printing symbol name at mepc
Now that the unwind starts from mepc already, the symbol
name at the mepc reg is kinda redundant, so just remove it.

Signed-off-by: Yong Cong Sin <ycsin@meta.com>
2024-06-13 16:46:48 -04:00
Yong Cong Sin
726fefd12c arch: riscv: stacktrace: implement arch_stack_walk()
Created the `arch_stack_walk()` function out from the original
`z_riscv_unwind_stack()`, it's been updated to support
unwinding any thread.

Updated the stack_unwind test case accordingly.

Increased the delay in `test_fatal_on_smp`, to wait
for the the fatal thread to be terminated, as stacktrace can
take a bit more time.

Doubled the kernel/smp testcase timeout from 60 (default) to
120s, as some of the tests can take a little bit more than 60s
to finish.

Signed-off-by: Yong Cong Sin <ycsin@meta.com>
2024-06-13 16:46:48 -04:00
Yong Cong Sin
5121de6e73 arch: introduce arch_stack_walk()
An architecture can indicate that it has an implementation for
the `arch_stack_walk()` function by selecting
`ARCH_HAS_STACKWALK`.

Set the default value of `EXCEPTION_STACK_TRACE_MAX_FRAMES` to
`ARCH_STACKWALK_MAX_FRAMES` if the latter is available.

Signed-off-by: Yong Cong Sin <ycsin@meta.com>
2024-06-13 16:46:48 -04:00
Nicolas Pitre
6cc9d7ec20 tests: arm64: exercize MMU page table allocation and recycling
This code was never formally tested before... and without the preceding
commit it obviously didn't work either.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2024-06-13 05:47:24 -04:00
Nicolas Pitre
f7e11649fd arch/arm64/mmu: fix page table reference counting
Existing code confused table usage and table reference counts together.
This obviously doesn't work. A table with one reference to it and one
populated PTE is not the same as a table with 2 references to it and
no PTe in use.

So split the two concepts and adjust the code accordingly. A page needs
to have its PTE usage count drop to zero before the last reference is
released. When both counts are 0 then the page is free.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2024-06-13 05:47:24 -04:00
Daniel Leung
564ca11631 kernel: mm: rename z_page_fault() to k_mem_page_fault()
This is part of a series of move memory management related
stuff out of Z_ namespace into its own namespace.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2024-06-12 21:13:26 -04:00
Daniel Leung
54af5dda84 kernel: mm: rename z_page_frame_* to k_mem_page_frame_*
Also any demand paging and page frame related bits are
renamed.

This is part of a series to move memory management related
stuff out of the Z_ namespace into its own namespace.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2024-06-12 21:13:26 -04:00
Daniel Leung
7715aa3341 kernel: mm: rename Z_SCRATCH_PAGE to K_MEM_SCRATCH_PAGE
This is part of a series to move memory management related
stuff from Z_ namespace into its own namespace.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2024-06-12 21:13:26 -04:00
Daniel Leung
2d2bbc05d6 kernel: mm: rename Z_VM_KERNEL to K_MEM_IS_VM_KERNEL
This is part of a series to move memory management functions
away from the z_ namespace and into its own namespace.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2024-06-12 21:13:26 -04:00
Daniel Leung
db9d3134c5 kernel: mm: rename Z_MEM_PHYS/VIRT_ADDR to K_MEM_*
This is part of a series to move memory management functions
away from the z_ namespace and into its own namespace. Also
make documentation available via doxygen.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2024-06-12 21:13:26 -04:00
Daniel Leung
50640cb1b8 kernel: mm: rename Z_MEM_VM_OFFSET to K_MEM_VIRT_OFFSET
This is part of a series to move memory management functions
away from the z_ namespace and into its own namespace. Also
make documentation available via doxygen.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2024-06-12 21:13:26 -04:00
Daniel Leung
552e29790d kernel: mm: rename z_phys_un/map to k_mem_*_phys_bare
This renames z_phys_map() and z_phys_unmap() to
k_mem_map_phys_bare() and k_mem_unmap_phys_bare()
respectively. This is part of the series to move memory
management functions away from the z_ namespace.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2024-06-12 21:13:26 -04:00
Wilfried Chauveau
fef0e8a211 arch: arm: cortex_m: update inline comment pointing to isr_wrapper.*
isr_wrapper has been converted to C but this inline comment was not
updated. This fixes the out-of-sync comment.

Signed-off-by: Wilfried Chauveau <wilfried.chauveau@arm.com>
2024-06-12 18:29:12 -04:00
Anas Nashif
c20e798646 arch: call arch_smp_init() directly, do not use SYS_INIT
Move this to a call in the init process. arch_* calls are no services
and should be called consistently during initialization.

Place it between PRE_KERNEL_1 and PRE_KERNEL_2 as some drivers
initialized in PRE_KERNEL_2 might depend on SMP being setup.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-06-12 18:23:54 -04:00
Jérémy LOCHE - MAKEEN Energy
75d65821ce arch: arm: add rom_start_relocation prompts
Add a prompt to ROMSTART_RELOCATION_ROM configs to allow
projects config file selection.
This simplifies the usage in samples/app for board porting.

Signed-off-by: Jérémy LOCHE - MAKEEN Energy <jlh@makeenenergy.com>
2024-06-12 17:11:04 -05:00
Jakub Zymelka
502fcac821 arch: riscv: core: Enable RISCV IRQs for no multithreading
Enable MSTATUS.IEN to allow RISCV interrupts for
non-multithreaded applications.

Signed-off-by: Jakub Zymelka <jakub.zymelka@nordicsemi.no>
2024-06-10 16:57:44 +03:00
Lars-Ove Karlsson
b48aeedf77 arch: common: Removed unnecessary cast
Removed an unnecessary cast to void * from a function that already
had the correct signature.
This makes for more portable code as casting between code and data
pointers are frowned upon by the C standard.

Signed-off-by: Lars-Ove Karlsson <lars-ove.karlsson@iar.com>
2024-06-07 19:05:08 -04:00
Flavio Ceolin
64b1ac831b arm: pm: Don't use deprecated function
Use pm_system_resume instead of z_pm_save_idle_exit

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2024-06-05 17:36:22 -05:00
Flavio Ceolin
a3de27fce9 x86: pm: Don't use deprecated function
Use pm_system_resume instead of z_pm_save_idle_exit

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2024-06-05 17:36:22 -05:00
Flavio Ceolin
9a869ef33a arc: pm: Don't use deprecated function
Use pm_system_resume instead of z_pm_save_idle_exit

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2024-06-05 17:36:22 -05:00
Nikodem Kastelik
12142f72ec arch: arm: core: mpu: allow non-ARM memory attributes
Memory region defined in devicetree can have attributes
that are not intended to be parsed by MPU library,
but might be valid for other components.

Signed-off-by: Nikodem Kastelik <nikodem.kastelik@nordicsemi.no>
2024-06-05 14:42:50 +01:00
Peter Mitsis
0bcdae2c62 kernel: Add CONFIG_ARCH_HAS_DIRECTED_IPIS
Platforms that support IPIs allow them to be broadcast via the
new arch_sched_broadcast_ipi() routine (replacing arch_sched_ipi()).
Those that also allow IPIs to be directed to specific CPUs may
use arch_sched_directed_ipi() to do so.

As the kernel has the capability to track which CPUs may need an IPI
(see CONFIG_IPI_OPTIMIZE), this commit updates the signalling of
tracked IPIs to use the directed version if supported; otherwise
they continue to use the broadcast version.

Platforms that allow directed IPIs may see a significant reduction
in the number of IPI related ISRs when CONFIG_IPI_OPTIMIZE is
enabled and the number of CPUs increases.  These platforms can be
identified by the Kconfig option CONFIG_ARCH_HAS_DIRECTED_IPIS.

Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
2024-06-04 22:35:54 -04:00
Flavio Ceolin
f74a84b251 xtensa: mmu: MMU re-initialization API
With power managment is enabled, depending on the SoC power state
used when idle, the MMU may lose context and may need to be re-initialized.
When re-initializing the MMU, we must not re-create the page table
because it may overwrite changes done during the execution, but we still
need to set the asid and page table for the current context.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2024-06-04 16:27:55 -05:00
Yong Cong Sin
6a3cb93d88 arch: remove the use of z_arch_esf_t completely from internal
Created `GEN_OFFSET_STRUCT` & `GEN_NAMED_OFFSET_STRUCT` that
works for `struct`, and remove the use of `z_arch_esf_t`
completely.

Signed-off-by: Yong Cong Sin <ycsin@meta.com>
2024-06-04 14:02:51 -05:00
Yong Cong Sin
e54b27b967 arch: define struct arch_esf and deprecate z_arch_esf_t
Make `struct arch_esf` compulsory for all architectures by
declaring it in the `arch_interface.h` header.

After this commit, the named struct `z_arch_esf_t` is only used
internally to generate offsets, and is slated to be removed
from the `arch_interface.h` header in the future.

Signed-off-by: Yong Cong Sin <ycsin@meta.com>
2024-06-04 14:02:51 -05:00
Yong Cong Sin
3998e18ec4 arch: rename all esf struct to struct arch_esf
Rename every architecture's esf struct to `struct esf`.

Signed-off-by: Yong Cong Sin <ycsin@meta.com>
2024-06-04 14:02:51 -05:00
Piotr Wojnarowski
0f3fe4daab riscv: Align _isr_wrapper to 64 bytes for CLIC
The CLIC requires that mtvec.base is aligned to 64 bytes.
_isr_wrapper is used as mtvec.base, so align it to 64 bytes.

Signed-off-by: Piotr Wojnarowski <pwojnarowski@antmicro.com>
2024-06-04 13:41:49 +02:00
Ederson de Souza
7f0b5edd8c arch/x86: Make irq_offload SMP-safe on x86_64
The irq_offload mechanism was using the same entry of the IDT vector for
all CPUs on SMP systems. This caused race conditions when two CPUs were
doing irq_offload() calls.

This patch addresses that by adding one indirection layer: the
irq_offload() now sets a per CPU entry with the routine and parameter to
be run. Then a software interrupt is generated, and a default handler
will do the appropriate dispatching.

Finally, test "kernel/smp_abort" is enabled for x86 as it should work
now.

Fixes #72172.

Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
2024-06-04 07:57:06 +02:00
Yong Cong Sin
02770ad963 debug: EXCEPTION_STACK_TRACE should depend on arch Kconfigs
Fix the dependencies of `CONFIG_EXCEPTION_STACK_TRACE`:
- Architecture-specific Kconfig, i.e.
  `X86_EXCEPTION_STACK_TRACE`, will be enabled automatically
  when all the dependencies are met.
- `EXCEPTION_STACK_TRACE` depends on architecture-specific
  Kconfig to be enabled.
- The stack trace implementations should be compiled only if
  user enables `CONFIG_EXCEPTION_STACK_TRACE`.

Signed-off-by: Yong Cong Sin <ycsin@meta.com>
2024-06-03 03:02:04 -07:00
Yong Cong Sin
190777dccf arch: arm64: create ARM64_EXCEPTION_STACK_TRACE
Currently, the stack trace in ARM64 implementation depends on
frame pointer Kconfigs combo to be enabled. Create a dedicated
Kconfig for that instead, so that it is consistent with x86 and
riscv, and update the source accordingly.

Signed-off-by: Yong Cong Sin <ycsin@meta.com>
2024-06-03 03:02:04 -07:00
Yong Cong Sin
ea00e04382 arch: x86: select DEBUG_INFO in X86_EXCEPTION_STACK_TRACE
Select `DEBUG_INFO` instead of depending on it.

Signed-off-by: Yong Cong Sin <ycsin@meta.com>
2024-06-03 03:02:04 -07:00
Yong Cong Sin
8a5823b474 debug: remove !OMIT_FRAME_POINTER from EXCEPTION_STACK_TRACE
Not all stack trace implementation requires frame pointer, move
that dependency to architecture Kconfig.

Signed-off-by: Yong Cong Sin <ycsin@meta.com>
2024-06-03 03:02:04 -07:00
Yong Cong Sin
413b1cf409 debug: remove DEBUG_INFO from EXCEPTION_STACK_TRACE
The `DEBUG_INFO` in the `EXCEPTION_STACK_TRACE` is only
required by x86. Move that to `X86_EXCEPTION_STACK_TRACE`
instead.

Signed-off-by: Yong Cong Sin <ycsin@meta.com>
2024-06-03 03:02:04 -07:00
Wilfried Chauveau
7d7616214b arch: arm: cortex_m: restore comment lost in translation
The comment about ISB in swap.S was lost when translation to C.

Signed-off-by: Wilfried Chauveau <wilfried.chauveau@arm.com>
2024-05-31 09:53:31 -05:00
Yong Cong Sin
94346d2441 arch: arm64: fatal: limit max number of stack traces
In some cases, the `fp` will never be `NULL` and the stack
unwinding can go on and on forever, limit the max depth so that
this will not happen.

Signed-off-by: Yong Cong Sin <ycsin@meta.com>
2024-05-30 03:00:50 -07:00
Yong Cong Sin
c87dc641bc arch: generalize frame pointer via CONFIG_FRAME_POINTER introduction
Enabling `CONFIG_FRAME_POINTER` allows the users to build the
kernel with frame-pointer.

Signed-off-by: Yong Cong Sin <ycsin@meta.com>
2024-05-30 03:00:40 -07:00
Marcin Szymczyk
e17b3fd884 arch: riscv: implement arch_irq_disconnect_dynamic
For SoC with `CONFIG_RISCV_RESERVED_IRQ_ISR_TABLES_OFFSET`,
it should be taken into consideration when disconnecting IRQ.

Signed-off-by: Marcin Szymczyk <marcin.szymczyk@nordicsemi.no>
2024-05-29 11:58:44 +02:00
Yong Cong Sin
6e8d979336 arch: riscv: stacktrace: handle user threads
Handle user threads stack bound validation in
`in_stack_bound()` to get more accurate traces.

If `CONFIG_PMP_POWER_OF_TWO_ALIGNMENT` is enabled:

```
+------------+ <- thread.arch.priv_stack_start
| Guard      | } Z_RISCV_STACK_GUARD_SIZE
+------------+
| Priv Stack | } CONFIG_PRIVILEGED_STACK_SIZE
+------------+ <- thread.arch.priv_stack_start +
                  CONFIG_PRIVILEGED_STACK_SIZE +
                  Z_RISCV_STACK_GUARD_SIZE
```

otherwise:

```
+------------+ <- thread.stack_obj
| Guard      | } Z_RISCV_STACK_GUARD_SIZE
+------------+
| Priv Stack | } CONFIG_PRIVILEGED_STACK_SIZE
+------------+ <- thread.stack_info.start
| Thread     |
| stack      |
|            |
+............|
| TLS        | } thread.stack_info.delta
+------------+ <- thread.stack_info.start +
                  thread.stack_info.size
```

See: zephyr/include/zephyr/arch/riscv/arch.h

Signed-off-by: Yong Cong Sin <ycsin@meta.com>
2024-05-29 08:38:53 +02:00
Yong Cong Sin
602c993799 arch: riscv: stacktrace: fix cpuid type and optimize branch with compiler
Change the type of `cpu_id` to `uint8_t` since that is the type
of `arch_curr_cpu()->id`.

Instead of using precompiler switch (`#ifdef CONFIG_SMP`), use
if-else shorthand instead (`IS_ENABLED(CONFIG_SMP)`).

Signed-off-by: Yong Cong Sin <ycsin@meta.com>
2024-05-29 08:38:53 +02:00
Yong Cong Sin
bbe5e1e6eb build: namespace the generated headers with zephyr/
Namespaced the generated headers with `zephyr` to prevent
potential conflict with other headers.

Introduce a temporary Kconfig `LEGACY_GENERATED_INCLUDE_PATH`
that is enabled by default. This allows the developers to
continue the use of the old include paths for the time being
until it is deprecated and eventually removed. The Kconfig will
generate a build-time warning message, similar to the
`CONFIG_TIMER_RANDOM_GENERATOR`.

Updated the includes path of in-tree sources accordingly.

Most of the changes here are scripted, check the PR for more
info.

Signed-off-by: Yong Cong Sin <ycsin@meta.com>
2024-05-28 22:03:55 +02:00
Yong Cong Sin
5a35037af3 arch: riscv: check esf before calling z_riscv_unwind_stack
Make sure that esf is not NULL before calling
z_riscv_unwind_stack to prevent NULL pointer dereferencing.

Signed-off-by: Yong Cong Sin <ycsin@meta.com>
2024-05-27 06:19:32 -04:00
Flavio Ceolin
4d85f3d91c pm: Deprecate z_pm_save_idle_exit
Deprecate z_pm_save_idle_exit and promote pm_system_resume.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2024-05-27 02:10:03 -07:00
Nikolay Agishev
7180c515c4 ARC: MPU: Add thread stack isolation configs
Regarding recent changes in general MPU configuration
(https://github.com/zephyrproject-rtos/zephyr/pull/71969), add
appropriate configs for isolating thread stacks into ARC MPU.

Signed-off-by: Nikolay Agishev <agishev@synopsys.com>
2024-05-27 07:44:44 +02:00
Yong Cong Sin
7248efcd59 drivers: intc: update to use multi-level API
Update these multi-level interrupt drivers to use the new API.

Signed-off-by: Yong Cong Sin <ycsin@meta.com>
2024-05-25 11:24:32 +03:00
Yong Cong Sin
5d9e266d13 drivers: interrupt_controller: irq_steer: use new multilevel irq impl
Update the NXP's irq_steer driver to use the new multi-level
interrupt implementation.

Signed-off-by: Yong Cong Sin <ycsin@meta.com>
2024-05-25 11:24:32 +03:00
Yong Cong Sin
e2bcedc3ad arch: common: multilevel_irq: simplification with new multilevel IRQ APIs
Use the multi-level interrupt APIs that accepts `level` as an
argument for the code where the level of the interrupt is not
known at build time.

Signed-off-by: Yong Cong Sin <ycsin@meta.com>
2024-05-25 11:24:32 +03:00
Yong Cong Sin
c5f5b964c1 arch: sw_isr: revamp multi-level interrupt architecture
Previously the multi-level irq lookup table is generated by
looping through the devicetree nodes using macros & Kconfig,
which is hard to read and flimsy.

This PR shifts the heavy lifting to devicetree & DT macros such
that an interrupt controller driver, which has its info in the
devicetree, can register itself directly with the multi-level
interrupt architecture, which is more straightforward.

The previous auto-generated look up table with macros is now
moved in a file of its own. A new compatibility Kconfig:
`CONFIG_LEGACY_MULTI_LEVEL_TABLE_GENERATION` is added and
enabled by default to compile the legacy look up table for
interrupt controller drivers that aren't updated to support the
new architecture yet.

Signed-off-by: Yong Cong Sin <ycsin@meta.com>
2024-05-25 11:24:32 +03:00
Yong Cong Sin
c118cd5a13 arch: make the max stack frames configurable
Current on x86 & risc-v that implement stack trace, the
maximum depth of the stack trace is defined by a macro.

Introduce a new Kconfig:EXCEPTION_STACK_TRACE_MAX_FRAMES
so that this is configurable in software.

Signed-off-by: Yong Cong Sin <ycsin@meta.com>
2024-05-23 11:52:08 -04:00
Yong Cong Sin
2a3d9d0d90 arch: arm64: use symtab to print function name in stack trace
Selecting `CONFIG_SYMTAB` will
enable the symtab generation which will be used in the
stack trace to print the function name of the return
address.

Added `arm64` to the `arch.common.stack_unwind.symtab` test.

Signed-off-by: Yong Cong Sin <ycsin@meta.com>
2024-05-23 11:52:08 -04:00