Commit graph

4662 commits

Author SHA1 Message Date
Jim Shu
e3c8b4cae4 arch: riscv: pmp: introduce riscv_pmp_region structure
Using struct riscv_pmp_region to modulize PMP CSR handling, including
PMP NAPOT/TOR mode handling. This patch can make us more easily to
add/remove RISC-V PMP regions without considering register handling.

Signed-off-by: Jim Shu <cwshu@andestech.com>
2022-01-11 11:47:03 +01:00
Jim Shu
e4c5d96a8b arch: riscv: pmp: enable MPU log module for debugging
Cleanup logging API in core_pmp.c. Remove old printf-based debugging API
and change the log module of PMP to individual MPU log module.

Signed-off-by: Jim Shu <cwshu@andestech.com>
2022-01-11 11:47:03 +01:00
Jim Shu
5fc5beabe2 arch: riscv: pmp: fix IRQ handling of PMP stack guard
This commit add 2 minor fixes of IRQ handling:

1. Save caller registers before calling z_riscv_configure_stack_guard()
in RISC-V assembly.

2. reschedule and no_reschdule code paths use different interrupt
return path after supporting of CONFIG_PMP_STACK_GUARD. back-to-back
interrupt checking is in the reschedule code path so that it should
jump to interrupt return path of reschedule.

Signed-off-by: Jim Shu <cwshu09@gmail.com>
2022-01-11 11:47:03 +01:00
Jim Shu
e0329a5525 arch: riscv: pmp: fix return value of arch_mem_domain_partition_remove()
If no thread use this memory domain, there isn't any user PMP region
translated from memory partitions in domain. In this case, memory
partition removal doesn't need to remove user PMP region and
arch_mem_domain_partition_remove() could just successfully return.

Signed-off-by: Jim Shu <cwshu@andestech.com>
2022-01-11 11:47:03 +01:00
Jim Shu
fd1e5aebc0 arch: riscv: fix sp of supervisor thread in _Fault function.
Although CONFIG_USERSPACE is enabled, there are supervisor threads who
don't have privileged stack using exception handler. Only let user
threads to switch to privileged stack in exception handler.

Signed-off-by: Jim Shu <cwshu@andestech.com>
2022-01-11 11:47:03 +01:00
Tomasz Bursztyka
4090962386 drivers/interrupt_controller: Add source id to VT-D interrupt remap
Change the API and apply that change where relevant.

Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
2022-01-07 10:47:27 -05:00
Tomasz Bursztyka
345e122dd2 arch/x86: Add a function to retrieve ID from ACPI's DMAR
This will be necessary to get IOAPIC and HPET source ids for VT-D.

Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
2022-01-07 10:47:27 -05:00
Tomasz Bursztyka
1012e254cc arch/x86: PCIE MSI address and data may be out of remapping
In fact, in case of VT-D being enabled, it will require to get an
address and data for its own MSI based interrupts which cannot be
remapped (i.e.: will directly go to the relevant APIC).

This is necessary to get the Fault event supported in VT-D.

Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
2022-01-07 10:47:27 -05:00
Tomasz Bursztyka
1a1bc0d242 drivers/interrupt_controller: Make VT-D remap generic and handle flags
This will not only be used by MSI remapping but by all relevant
interrupts.

Fix also IRTE settings:
- handle x2apic for destination id
- destination mode is always logical (as for IOAPIC)

Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
2022-01-07 10:47:27 -05:00
Tomasz Bursztyka
4047b793c8 drivers/interrupt_controller: Generate proper MSI address on VT-D
SHV bit depends on the number of vectors allocated.
If it's facing a multi-vector MSI array, it will set the bit.
If not the bit must be 0.

Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
2022-01-07 10:47:27 -05:00
Tomasz Bursztyka
6ed593f861 drivers/pcie: Extending parameters to pcie_msi_map
n_vector will be necessary for VT-D actually.

Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
2022-01-07 10:47:27 -05:00
Tomasz Bursztyka
25b8df0bdb drivers/pcie: Even single MSI based interrupt needs to be remapped
Refactor to handle this case. This is valid only when MSI multi-vector
feature is enabled.

Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
2022-01-07 10:47:27 -05:00
Tomasz Bursztyka
fa34b135f5 arch/x86: Make sure PCIE allocated IRTEs are tighten to irq/vector
As all interruption need to go through VT-D, calling vt-d remap will
happen on lower level as seen next, so make sure all pcie related
irq/vector get tighten to their respective allocated IRTE.

Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
2022-01-07 10:47:27 -05:00
Tomasz Bursztyka
84319db9fe arch/x86: All dynamic IRQ connection need to be remapped
Allocate an IRTE for all irq being connected through
arch_irq_connect_dynamic(). This will be mandatory since VT-D expects to
filter all interruptions (but the one it generates, as we will see
later).

Taking into account CONFIG_INTEL_VTD_ICTL_XAPIC_PASSTHROUGH, which could
help for debugging.

Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
2022-01-07 10:47:27 -05:00
Tomasz Bursztyka
ad8ab01488 arch/x86: On irq remapping, all PCIE MSI/MSI-X need to be remapped
There is no need to differentiate with multi-vector or not, MSI vs
MSI-x: all need to be remapped if Intel VT-D is on.

Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
2022-01-07 10:47:27 -05:00
Tomasz Bursztyka
f0a7f250a0 arch/x86: Fixing MSI vector allocation
Fixing an out of bound issue.

Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
2022-01-07 10:47:27 -05:00
Mark Holden
7803a4e590 arch: riscv: ARCH_EXCEPT macro
Enable ARCH_EXCEPT macro for non-usermode scenario for RISC-V
Macro will now raise an illegal instruction exception so that mepc will
hold expected value in exception handler, and generated coredump can
reconstruct the failing stack

Coredump tests running on renode (for RISC-V) can now utilize fatal error
path through k_panic

Signed-off-by: Mark Holden <mholden@fb.com>
2022-01-01 07:38:20 -05:00
Tomasz Bursztyka
2623315802 arch/x86: PCIE MSI vector allocator can use arch IRQ allocator
Instead of messing up with the PCI bus.

Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
2021-12-22 12:16:52 +01:00
Tomasz Bursztyka
88bac5d0b5 arch/x86: Implement the IRQ allocation and usage interfaces for intel 64
This is the only architecture user for this at the moment.

Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
2021-12-22 12:16:52 +01:00
Tomasz Bursztyka
c76651b9ab arch/x86: Do not call irq controller on dedicated irq/vector function
MSI/MSI-x interrupt do not need any interrupt controller handling
(ioapic/loapic).

Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
2021-12-22 12:16:52 +01:00
TOKITA Hiroshi
2de3133a05 riscv: Add an option for configuring mcause exception mask
GD32V processor core is used non-standard bitmask
for mcause register. Add option to configure the bitmask
to support GD32V.

Signed-off-by: TOKITA Hiroshi <tokita.hiroshi@gmail.com>
2021-12-20 17:51:30 +01:00
Andy Ross
1a2fecec6d soc/intel_adsp: Unify Xtensa CPU reset between cores
Startup on these devices was sort of a mess, with multiple variants of
Xtensa and platform initialization code from multiple ancestries being
invoked at different places for different purposes.  Just use one code
path for everyone.

Bootloader entry starts with a minimal assembly stub that simply sets
WINDOW{START,BASE}, PS and a stack pointer and then jumps to C code.
That then uses the cpu_early_init() implementation from cAVS 2.5's
secondary cores to finish Xtensa initialization, and then flows
directly into the pre-existing bootloader C code to initialize cache
and memory and copy the HP-SRAM image, then it invokes Zephyr via a
simple C function call to z_cstart().

Likewise, remove the "reset vector" from Zephyr.  This was never a
reset vector, reset on these devices goes to a fixed address in a ROM.
CPU initialization is handled explicitly and completely in the
bootloader now, in a way that can be unified between the main and
secondary cores.  Entry from the bootloader now goes directly into
z_cstart() via a C call (via a single jump instruction placed at the
entry point address -- that's going away soon too once we're using a
unified link).

Now that vector table initialization happens in a uniform way, there's
no need to copy the VECBASE value during arch_start_cpu().

Finally note that this also reverts the
CONFIG_RESET_VECTOR_IN_BOOTLOADER kconfig variable added for these
platforms, because it's no longer a tunable and true always.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2021-12-14 18:43:05 -06:00
Lauren Murphy
c1711997bc debug: coredump: add xtensa coredump
Adds Xtensa as supported architecture for coredump. Fixes
a few typos in documentation, Kconfig and a C file. Dumps
minimal set of registers shown by 'info registers' in GDB
for the sample_controller and ESP32 SOCs. Updates tests.

Signed-off-by: Lauren Murphy <lauren.murphy@intel.com>
2021-12-14 07:40:55 -05:00
Carles Cufi
4f64ae383d x86: acpi: Fix address-of-packed-mem warning
The warning below appears once -Waddress-of-packed-mem is enabled:

/home/carles/src/zephyr/zephyr/arch/x86/core/acpi.c: In function
'z_acpi_find_table':
/home/carles/src/zephyr/zephyr/arch/x86/core/acpi.c:190:24: warning:
taking address of packed member of 'struct acpi_xsdt' may result in an
unaligned pointer value [-Waddress-of-packed-member]
  190 |    for (uint64_t *tp = &xsdt->table_ptrs[0]; tp < end; tp++) {

To avoid the warning, use an intermediate void * variable.

More info in #16587.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2021-12-10 14:08:59 +01:00
Sebastian Bøe
1f87642f08 arch: cortex_m: Fix dwt cyccnt assert
Fix the assert that checks for existence of a cycle counter.

The field is named NO CYCCNT, so when it is 1, there is no cycle
counter. But we are asserting the opposite.

Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
2021-12-10 12:27:49 +01:00
Mark Holden
1a697ccf59 coredump: add support for RISC-V
This adds the necessary bits in arch code, and Python scripts
to enable coredump support for RISC-V

Signed-off-by: Mark Holden <mholden@fb.com>
2021-12-08 08:54:32 -05:00
Dmytro Firsov
01a9b117fe xenvm: arm64: add Xen Enlighten and event channel support
This commit adds support of Xen Enlighten page and initial support for
Xen event channels. It is needed for future Xen PV drivers
implementation.

Now enlighten page is mapped to the prepared memory area on
PRE_KERNEL_1 stage. In case of success event channel logic gets
inited and can be used ASAP after Zephyr start. Current implementation
allows to use only pre-defined event channels (PV console/XenBus) and
works only in single CPU mode (without VCPUOP_register_vcpu_info).
Event channel allocation will be implemented in future versions.

Signed-off-by: Dmytro Firsov <dmytro_firsov@epam.com>
2021-12-07 12:15:38 -05:00
Gerard Marull-Paretas
7d1bfb51ae drivers: timer: cortex_m_systick: improve ISR installation
A Cortex-M specific function (sys_clock_isr()) was defined as a weak
function, so in practice it was always available when system clock was
enabled, even if no Cortex-M systick was available. This patch
introduces an auxiliary Kconfig option that, when selected, the ISR
function gets installed. External SysTick drivers can also make use of
this function, thus achieving the same functionality offered today but
in a cleaner way.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2021-12-04 07:34:53 -05:00
Yuguo Zou
abeaf94855 soc: arc: fix ARC_HAS_ACCL_REGS settings
ARC_HAS_ACCL_REGS should set to y to protect ACCL and ACCH registers
during irq. These registers could be used as GPRs by compilers and
therefore need store/restore during irq.

Signed-off-by: Yuguo Zou <yuguo.zou@synopsys.com>
2021-12-02 11:32:14 -06:00
Daniel Leung
dc34f6c84d xtensa: introduce support for GDB stub
This adds basic support for GDB stub on Xtensa. Note that
this only provides the common bits on the architecture side.
SoC support is also required to fully enable GDB stub on
each Xtensa SoC.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-11-30 15:24:00 -05:00
Daniel Leung
650a629b08 debug: gdbstub: remove start argument from z_gdb_main_loop()
Storing the state where this is the first GDB break can be done
in the main GDB stub code. There is no need to store the state
in architecture layer.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-11-30 15:24:00 -05:00
Daniel Leung
e1180c8cee x86: gdbstub: add arch-specific funcs to read/write registers
This adds some architecture-specific functions to read/write
registers for the GDB stub. This is in preparation for the actual
introduction of these functions in the core GDB stub code to
avoid breaking the build in between commits.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-11-30 15:24:00 -05:00
Daniel Leung
1cd7cccbb1 kernel: mem_domain: arch_mem_domain functions to return errors
This changes the arch_mem_domain_*() functions to return errors.
This allows the callers a chance to recover if needed.

Note that:
() For assertions where it can bail out early without side
   effects, these are converted to CHECKIF(). (Usually means
   that updating of page tables or translation tables has not
   been started yet.)
() Other assertions are retained to signal fatal errors during
   development.
() The additional CHECKIF() are structured so that it will bail
   early if possible. If errors are encountered inside a loop,
   it will still continue with the loop so it works as before
   this changes with assertions disabled.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-11-22 12:45:22 -05:00
Julien Massot
36f116b47f scripts/arch: remove usage of deprecated LooseVersion
replace with version.parse from packaging module.

prevent this warning message:
DeprecationWarning: The distutils package is deprecated
and slated for removal in Python 3.12. Use setuptools or
check PEP 632 for potential alternatives

Signed-off-by: Julien Massot <julien.massot@iot.bzh>
2021-11-19 19:16:11 -05:00
Flavio Ceolin
7dd4297214 pm: Remove unused parameter
The number of ticks on z_pm_save_idle_exit is not used and there is
no need to have it.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2021-11-17 11:15:49 -05:00
Michel Haber
9d815e5251 timing: use runtime cycles for cortex-m systick
Use sys_clock_hw_cycles_per_sec() instead of
CONFIG_SYS_CLOCK_HW_CYCLES_PER_SEC to determine clock cycles.

Signed-off-by: Michel Haber <michel-haber@hotmail.com>
2021-11-16 10:43:18 +01:00
Andy Ross
1238410914 arch/x86_64: Add hook for CONFIG_SCHED_THREAD_USAGE accounting in ISRs
Call into z_thread_usage_stop() before ISR entry to avoid including
interrupt handling totals in thread usage stats.

This has to go into the assembly immediately before the callback-based
dispatch.  Note that the dispatch code was putting the vector number
in RCX, which was unfortunate as that's a caller-saved register.
Would be nice to clean this up in the future so it lives in a
preserved register but it's mildly complicated to make work with the
way we do the stack layout right now.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2021-11-08 21:32:20 -05:00
Andy Ross
76b848e38c arch/sparc: Add hook for CONFIG_SCHED_THREAD_USAGE accounting in ISRs
Call into z_thread_usage_stop() before ISR entry to avoid including
interrupt handling totals in thread usage stats.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2021-11-08 21:32:20 -05:00
Andy Ross
c815996606 arch/arc: Add hook for CONFIG_SCHED_THREAD_USAGE accounting in ISRs
Call into z_thread_usage_stop() before ISR entry to avoid including
interrupt handling totals in thread usage stats.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2021-11-08 21:32:20 -05:00
Andy Ross
35af02fe3d arch/arm64: Add hook for CONFIG_SCHED_THREAD_USAGE accounting in ISRs
Call into z_thread_usage_stop() before ISR entry to avoid including
interrupt handling totals in thread usage stats.

This is pretty much exactly where we want it, just after the context
saving steps (which we can't elide since the hook is in C) and
immediately before the tracing hook for ISR entry.  And as I'm reading
the code, this is purely for Zephyr-registered interrupts, meaning
that software exceptions will be accounted for (correctly) as part of
the excepting thread.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2021-11-08 21:32:20 -05:00
Andy Ross
884f1bf39d arch/xtensa: Add hook for CONFIG_SCHED_THREAD_USAGE accounting in ISRs
Call into z_thread_usage_stop() before ISR entry to avoid including
interrupt handling totals in thread usage stats.

Note that this hook is after the register save and stack swap has
happened, so it still incldues some overhead.  But calling out from
the interrupted stack on Xtensa gets really, really hairy due to the
weird intermediate states we leverage (once we've saved enough context
to make a C call safely, we've lost the ability to use register
windows per the C ABI!).

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2021-11-08 21:32:20 -05:00
Torsten Rasmussen
9c74027a7b cmake: CMake linker script generator pass handling
To prepare for linker script creation with flexible number of linker
passes depending on system configuration then the Zephyr CMake linker
script generator has been updated to use pass names instead of pass
numbers.

This allows greater flexibility as a section can now be active based on
the settings on the pass and not the linking pass index number.

As part of this, the `PASS` processing in `linker_script_common.cmake`
has been adjusted so that it properly handles when a linking pass is
handling multiple settings, such as both `LINKER_APP_SMEM_UNALIGNED`
and `DEVICE_HANDLES_PASS1` in same linking pass.

As the number of linking passes are more flexible, then the PASS
argument in `zephyr_linker_section()` and
`zephyr_linker_section_configure()` has been updated to also support
a `NOT <name>` argument, for example: `PASS NOT LINKER_ZEPHYR_FINAL`.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2021-11-08 20:45:07 +01:00
Nikolai Kondrashov
533b8c971a arch: arm: aarch32: Fix spelling of "want"
Fix spelling of "want" in a comment in _arch_isr_direct_mp().

Signed-off-by: Nikolai Kondrashov <spbnick@gmail.com>
2021-11-02 10:46:00 +01:00
Dmytro Firsov
c4ab278688 arm64: xenvm: Add Xen hypercall interface for arm64
This commit adds Xen hypervisor call interface for arm64 architecture.
This is needed for further development of Xen features in Zephyr.

Signed-off-by: Dmytro Firsov <dmytro_firsov@epam.com>
2021-10-29 15:23:33 +02:00
Immo Birnbaum
c6141c49c1 arch: arm: core: aarch32: enable ARMv7-R/Cortex-R code for ARMv7-A/Cortex-A
Modify #ifdefs so that any code that is compiled if CONFIG_ARMV7_R is
set is also compiled if CONFIG_ARMV7_A is set.
Modify #ifdefs so that any code that is compiled if CONFIG_CPU_CORTEX_R
is set is also compiled if CONFIG_CPU_AARCH32_CORTEX_A is set.
Modify source dir inclusion in CMakeLists.txt accordingly.

Brief file descriptions have been updated to include Cortex-A whereever
only Cortex-M and Cortex-R were mentioned so far.

Signed-off-by: Immo Birnbaum <Immo.Birnbaum@weidmueller.com>
2021-10-28 15:26:50 +02:00
Immo Birnbaum
70c403c215 arch: arm: core: aarch32: introduce basic ARMv7 MMU support
An initial implementation for memory management using the ARMv7 MMU.
A single L1 translation table for the whole 4 GB address space is al-
ways present, a configurable number of L2 page tables are linked to
the L1 table based on the static memory area configuration at boot
time, or whenever arch_mem_map/arch_mem_unmap are called at run-time.

Currently, a CPU with the Multiprocessor Extensions and execution at
PL1 are always assumed. Userspace-related features or thread stack
guard pages are not yet supported. Neither are LPAE, PXN or TEX re-
mapping. All mappings are currently assigned to the same domain. Re-
garding the permissions model, access permissions are specified using
the AP[2:1] model rather than the older AP[2:0] model, which, accor-
ding to ARM's documentation, is deprecated and should no longer be
used. The newer model adds some complexity when it comes to mapping
pages as unaccessible (the AP[2:1] model doesn't support explicit
specification of "no R, no W" permissions, it's always at least "RO"),
this is accomplished by invalidating the ID bits of the respective
page's PTE.

Includes sources, Kconfig integration, adjusted CMakeLists and the
modified linker command file (proper section alignment!).

Signed-off-by: Immo Birnbaum <Immo.Birnbaum@weidmueller.com>
2021-10-28 15:26:50 +02:00
Immo Birnbaum
eac90eeb52 arch: arm: core: aarch32: limit ACTLR register access to Cortex-R
The configuration bits ATCMPCEN, B0TCMPCEN and B1TCMPCEN in the ACTLR
register referenced in the function z_arm_tcm_disable_ecc are only de-
fined for Cortex-R CPUs. For Cortex-A CPUs, those bits are declared
as reserved.

Comp.: https://arm-software.github.io/CMSIS_5/Core_A/html/group__CMSIS__ACTLR.html

Signed-off-by: Immo Birnbaum <Immo.Birnbaum@weidmueller.com>
2021-10-28 15:26:50 +02:00
Immo Birnbaum
85f53376dc arch: arm: core: aarch32: Updated brief file description
Updated brief file description so that it also mentions the aarch32
Cortex-A CPUs.

Signed-off-by: Immo Birnbaum <Immo.Birnbaum@weidmueller.com>
2021-10-28 15:26:50 +02:00
Immo Birnbaum
38dc87d4d9 arch: arm: core: aarch32: Add ARMv7-A/Cortex-A(9) related Kconfig items
Add the ARMV7_A, CPU_AARCH32_CORTEX_A and CPU_CORTEX_A9 configuration
items.

Signed-off-by: Immo Birnbaum <Immo.Birnbaum@weidmueller.com>
2021-10-28 15:26:50 +02:00
Immo Birnbaum
305550a775 arch: arm: core: aarch32: Updated brief file description
Updated brief file description so that it also mentions the aarch32
Cortex-A CPUs.

Signed-off-by: Immo Birnbaum <Immo.Birnbaum@weidmueller.com>
2021-10-28 15:26:50 +02:00