Commit graph

6,067 commits

Author SHA1 Message Date
Flavio Ceolin
47e0621bb7 power: Remove not used build option
There is no usage of BOOTLOADER_CONTEXT_RESTORE since quark support
was removed.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2021-01-08 06:49:43 -05:00
Flavio Ceolin
28cf88183a x86: power: Remove dead code
X86 currently has no support for deep sleep states, just removing this
dead code.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2021-01-08 06:49:43 -05:00
Carlo Caione
1d68c48786 aarch64: mmu: Fix typo in mask definition
Fix typo in mask definition (s/UPPER/LOWER)

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2021-01-06 08:18:27 -06:00
Daniel Leung
0d7bdbc876 xtensa: use highest available EPC/EPS regs in restore context
There may be Xtensa SoCs which don't have high enough interrupt
levels for EPC6/EPS6 to exist in _restore_context. So changes
these to those which should be available according to the ISA
config file.

Fixes #30126

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-01-05 10:31:45 -08:00
Carlo Caione
181088600e aarch64: mmu: Avoid creating a new table when not needed
In the current MMU code a new table is created when mapping a memory
region that is overlapping with a block already mapped. The problem is
that the new table is created also when the new and old mappings have
the same attributes.

To avoid using a new table when not needed the attributes of the two
mappings are compared before creating the new table.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2021-01-04 23:53:04 -08:00
Daniel Leung
0aa61793de x86: remove custom switch to main thread function
The original idea of using a custom switch to main thread
function is to make sure the buffer to save floating point
registers are aligned correctly or else exception would be
raised when saving/restoring those registers. Since
the struct of the buffer is defined with alignment hint
to toolchain, the alignment will be enforced by toolchain
as long as the k_thread struct variable is a dedicated,
declared variable. So there is no need for the custom
switch to main thread function anymore.

This also allows the stack usage calculation of
the interrupt stack to function properly as the end of
the interrupt stack is not being used for the dummy
thread anymore.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-01-04 16:59:59 -08:00
Eugeniy Paltsev
14afc02caa isr_tables: adopt _irq_vector_table for using on 64bit architectures
As of today generic _irq_vector_table is used only on 32bit
architectures and 64bit architectures have their own implementation.
Make vectors size adjustable by using uintptr_t instead of uint32_t
for vectors.

The ARCv3 64 bit HS6x processors are going to be first users for
that.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
2021-01-04 16:47:51 -08:00
Watson Zeng
b1adc462be arch: arc: archs using ATOMIC_OPERATIONS_C
ATOMIC_OPERATIONS_BUILTIN still has some problem in mwdt toolchain,
so choosing ATOMIC_OPERATIONS_C instead.

Signed-off-by: Watson Zeng <zhiwei@synopsys.com>
2021-01-04 12:54:05 -05:00
Wojciech Sipak
56c06e852b arch: arm: cortex_r: disable ECC on TCMs
This commit adds possibility to disable ECC in Tightly Coupled
Memory in Cortex-R.
Linker scripts places stacks in this memory and marks it as
.noinit section. With ECC enabled, stack read accesses without
previous write result in Data Abort Exception.

Signed-off-by: Wojciech Sipak <wsipak@antmicro.com>
2020-12-27 18:16:00 +01:00
Watson Zeng
2609101eda arc: stack guard: bug fix with multi push stack situation
accessing the stack below guard_end is always a bug. some
instrustions (like enter_s {r13-r26, fp, blink}) push a collection
of registers on to the stack. In this situation, the fault_addr will
less than guard_end, but sp will greater than guard_end.

|------stack base-------| <--- high address
|                       |
|                       | <--- sp
|------stack top--------|
|------guard_end--------|
|                       | <--- fault_addr
|                       |
|------guard_start------| <--- low address

So we need to remove the SP check. Trade-off here is if we prefer
'false' classifications of MPU stack guard area accesses as stack
error or as general mpu error. The faults get caught anyway, this is
just about classification: don't see a strong need for the extra check
to only report stack pointer accesses to guard area as stack error,
instead of all accesses.

Signed-off-by: Watson Zeng <zhiwei@synopsys.com>
2020-12-20 13:01:22 -05:00
Kumar Gala
63d0a109c0 arch/x86: Convert DEVICE_AND_API_INIT to DEVICE_DEFINE
Convert device to DEVICE_DEFINE instead of DEVICE_AND_API_INIT
so we can deprecate DEVICE_AND_API_INIT in the future.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-12-20 10:19:30 -06:00
Johan Hedberg
8b01efe8e6 arch: x86: Fix early_serial build error when using fixed MMIO address
Fix the following complilation error that happens when specifying a
fixed MMIO address for the UART through X86_SOC_EARLY_SERIAL_MMIO8_ADDR:

arch/x86/core/early_serial.c:30:26: error: #if with no expression
   30 | #if DEVICE_MMIO_IS_IN_RAM

Signed-off-by: Johan Hedberg <johan.hedberg@intel.com>
2020-12-17 11:55:50 -05:00
Peng Fan
cca070c80a arch: arm64: mmu: support using MT_NS attribute
According to CONFIG_ARMV8_A_NS, using MT_SECURE or MT_NS, to simplify
code change, use MT_DEFAULT_SECURE_STATE instead

Signed-off-by: Peng Fan <peng.fan@nxp.com>
2020-12-17 08:08:00 -05:00
Peng Fan
0d0f168460 arm: aarch64: Kconfig: introduce ARMV8_A_NS
Add new Kconfig entry ARMV8_A_NS for zephyr runs in normal world usage.

Signed-off-by: Peng Fan <peng.fan@nxp.com>
2020-12-17 08:08:00 -05:00
Andrew Boie
d2ad783a97 mmu: rename z_mem_map to z_phys_map
Renamed to make its semantics clearer; this function maps
*physical* memory addresses and is not equivalent to
posix mmap(), which might confuse people.

mem_map test case remains the same name as other memory
mapping scenarios will be added in the fullness of time.

Parameter names to z_phys_map adjusted slightly to be more
consistent with names used in other memory mapping functions.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-12-16 08:55:55 -05:00
Luke Starrett
430952f0b2 arch: arm64: GICv2/v3 handling causes abort on spurious interrupt
In _isr_wrapper, the interrupt ID read from the GIC is blindly used to
index into _sw_isr_table, which is only sized based on CONFIG_NUM_IRQ.

It is possible for both GICv2 and GICv3 to return 1023 for a handful
of scenarios, the simplest of which is a level sensitive interrupt
which has subsequently become de-asserted.  Borrowing from the Linux
GIC implementation, a read that returns an interrupt ID of 1023 is
simply ignored.

Minor collateral changes to gic.h to group !_ASMLANGUAGE content
together to allow this header to be used in assembler files.

Signed-off-by: Luke Starrett <luke.starrett@gmail.com>
2020-12-16 08:46:03 -05:00
Andrew Boie
0a791b7a09 x86: mmu: clarify physical/virtual conversions
The page table implementation requires conversion between virtual
and physical addresses when creating and walking page tables. Add
a phys_addr() and virt_addr() functions instead of hard-casting
these values, plus a macro for doing the same in ASM code.

Currently, all pages are identity mapped so VIRT_OFFSET = 0, but
this will now still work if they are not the same.

ASM language was also updated for 32-bit. Comments were left in
64-bit, as long mode semantics don't allow use of Z_X86_PHYS_ADDR
macro; this can be revisited later.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-12-15 14:16:51 -05:00
Alberto Escolar Piedras
6d3476117b posix: Add cpu_hold() function to better emulate code delay
In native_posix and nrf52_bsim add the cpu_hold() function,
which can be used to emulate the time it takes for code
to execute.
It is very similar to arch_busy_wait(), but while
arch_busy_wait() returns when the requested time has passed,
cpu_hold() ensures that the time passes in the callers
context independently of how much time may pass in some
other context.

Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
2020-12-14 12:32:11 +01:00
Anas Nashif
3d33d767f1 twister: rename in code
Some code had reference to sanitycheck, replace with twister.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-11 14:13:02 -05:00
Kumar Gala
7defa02329 x86: Fix compiler warning in z_x86_dump_mmu_flags
Fix compiler warnings associated with 'level' and 'entry' variables
'may be used uninitialized in this function'

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-12-11 17:41:58 +02:00
Carlo Caione
0d1a6dc27f intc: gic: Use SYS_INIT instead of custom init function
The GIC interrupt controller driver is using a custom init function
called directly from the prep_c function. For consistency move that to
use SYS_INIT.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2020-12-11 10:17:27 -05:00
Andrew Boie
3605832682 x86: fix z_x86_thread_page_tables_get()
This was reporting the wrong page tables for supervisor
threads with KPTI enabled.

Analysis of existing use of this API revealed no problems
caused by this issue, but someone may trip over it eventually.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-12-10 18:22:58 -05:00
Andrew Boie
905e6d52a7 x86: improve page table printouts
We now show:

 - Data pages that are paged out in red
 - Pages that are mapped but non-present due to KPTI,
   respectively in cyan or blue if they are identity mapped
   or not.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-12-10 18:22:58 -05:00
Andrei Emeltchenko
34573803a8 arch: x86_64: Rename _exception_stack to z_x86_exception_stack
Rename stack name according to MISRA-C standard.

Signed-off-by: Andrei Emeltchenko <andrei.emeltchenko@intel.com>
2020-12-10 17:06:17 +02:00
Andrei Emeltchenko
a4dbb51e74 arch: x86_64: Rename _nmi_stack to z_x86_nmi_stack
Rename stack name according to MISRA-C standard.

Signed-off-by: Andrei Emeltchenko <andrei.emeltchenko@intel.com>
2020-12-10 17:06:17 +02:00
Andrei Emeltchenko
ddc5139a6e arch: x86_64: Trivial correction
Correct register name in comment.

Signed-off-by: Andrei Emeltchenko <andrei.emeltchenko@intel.com>
2020-12-10 17:06:17 +02:00
Andrei Emeltchenko
e7d3dd1362 arch: x86_64: Using right exception stack with KPTI
With kernel page table isolation (KPTI) we cannot use right exception
stack since after using trampoline stack there was always switch to
7th IST stack (__x86_tss64_t_ist7_OFFSET). Make this configurable as a
parameter in EXCEPT(nr, ist) and EXCEPT_CODE(nr, ist). For the NMI we
would use ist6 (_nmi_stack).

Signed-off-by: Andrei Emeltchenko <andrei.emeltchenko@intel.com>
2020-12-10 17:06:17 +02:00
Andrei Emeltchenko
85db42883f arch/x86: Use NMI stack for NMIs
NMI can be triggered at any time, even when in the process of
switching stacks. Use special stack for it.

Signed-off-by: Andrei Emeltchenko <andrei.emeltchenko@intel.com>
2020-12-10 17:06:17 +02:00
Andrei Emeltchenko
8db06aee69 arch/x86: Add NMI registration API
Add simple NMI registration API.

Signed-off-by: Andrei Emeltchenko <andrei.emeltchenko@intel.com>
2020-12-10 17:06:17 +02:00
Andrew Boie
39dab07f49 x86: provide inline pentry_get()
Non-debug code may need this functionality, create an inline.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-12-09 16:57:01 -08:00
Andrew Boie
d362acb567 x86: mmu: add PTE_LEVEL
For code clarity, add a macro indicating the paging level
for leaf page tables.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-12-09 16:57:01 -08:00
Andrew Boie
d7dc0deae5 x86: mmu: fix ipi comments
Delete an incorrect one and note a limitation of the current
IPI implementation.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-12-09 16:57:01 -08:00
Andrew Boie
e20846eaa7 x86: mmu: add range_map_unlocked()
range_map() now doesn't implicitly hold x86_mmu_lock, allowing
callers to use it if the lock is already held.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-12-09 16:57:01 -08:00
Andrew Boie
6eabe20aee x86: mmu: fix typo
Fix incorrect macro name in comment.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-12-09 16:57:01 -08:00
Andrew Boie
2a139ee95d x86: pte_atomic_update should not return flipped
KPTI gymnastics need to be abstracted away from callers to
page_map_set().

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-12-09 16:57:01 -08:00
Anas Nashif
e3937453a6 power: rename _sys_suspend/_sys_resume
Be consistent in PM namespaces.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-09 15:18:29 -05:00
Anas Nashif
c10d4b377d power: move z_pm_save_idle_exit prototype to power.h
Maintain power prototypes in power.h instead of kernel and arch headers.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-09 15:18:29 -05:00
Anas Nashif
e0f3833bf7 power: remove SYS_ and sys_ prefixes
Remove SYS_ and sys_ from all PM related functions and defines.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-09 15:18:29 -05:00
Anas Nashif
142c3060e7 power: move kconfigs from arch/ to power/
Move all Kconfigs where they belong and in one place.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-09 15:18:29 -05:00
Anas Nashif
dd931f93a2 power: standarize PM Kconfigs and cleanup
- Remove SYS_ prefix
- shorten POWER_MANAGEMENT to just PM
- DEVICE_POWER_MANAGEMENT -> PM_DEVICE

and use PM_ as the prefix for all PM related Kconfigs

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-09 15:18:29 -05:00
Tomasz Bursztyka
da70d20a9c arch/x86: Support PCIE MSI-X
Provide the necessary adjustments to get MSI-X working (with or without
Intel VT-D).

Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
2020-12-08 09:29:20 -05:00
Tomasz Bursztyka
bfbe9b6df5 arch/x86: Implement arch specifics for software MSI multi-vector
Which requires Intel VT-D to work, since we don't allocate contiguous
vectors.

Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
2020-12-08 09:29:20 -05:00
Tomasz Bursztyka
be4c893549 arch/x86: Expose function do get DRHDs from DMAR ACPI table
This is part of Intel VT-D and how to discover capabilities, base
addresses and so on in order to start taking advantage from it.

There is a lot to get from there, but currently we are interested only
by getting the remapping hardware base address. And more specifically
for interrupt remapping usage.

There might be more than one of such hardware so the exposed function is
made to retrieve all of them.

Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
2020-12-08 09:29:20 -05:00
Tomasz Bursztyka
abf65b5e65 arch/x86: Generalize dynamic irq connection on given vector
This will be used by MSI multi-vector implementation to connect the irq
and the vector prior to allocation.

Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
2020-12-08 09:29:20 -05:00
Tomasz Bursztyka
bd57e4cf12 arch/x86: Generalize the vector allocator
This will be used by MSI/MSI-X when multi-vector is requested.

Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
2020-12-08 09:29:20 -05:00
Jingru Wang
5f2aa0409c toolchain: improved toolchain abstraction for compilers and linker
Accodind to c55c64e to update code

Signed-off-by: Jingru Wang <jingru@synopsys.com>
2020-12-05 10:19:50 -05:00
Carlo Caione
7e36bd31fe arch: aarch64: Use SP_EL0 instead of SP_ELx
ARM64 is currently using SP_ELx as stack pointer for kernel and threads
because everything is running in EL1. If support for EL0 is required, it
is necessary to switch to use SP_EL0 instead, that is the only stack
pointer that can be accessed at all exception levels by threads.

While it is not required to keep using SP_EL0 also during the
exceptions, the current code implementation makes it easier to use the
same stack pointer as the one used by threads also during the
exceptions.

This patch moves the code from using SP_ELx to SP_EL0 and fill in the
missing entries in the vector table.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2020-12-04 08:13:42 -05:00
Martin Åberg
53a4acb2dc SPARC: add FPU support
This change adds full shared floating point support for the SPARC
architecture.

All SPARC floating point registers are scratch registers with respect
to function call boundaries. That means we only have to save floating
point registers when switching threads in ISR. The registers are
stored to the corresponding thread stack.

FPU is disabled when calling ISR. Any attempt to use FPU in ISR
will generate the fp_disabled trap which causes Zephyr fatal error.

- This commit adds no new thread state.
- All FPU contest save/restore is synchronous and lazy FPU context
  switch is not implemented.

Signed-off-by: Martin Åberg <martin.aberg@gaisler.com>
2020-12-04 14:33:43 +02:00
Martin Åberg
356d37fb7f SPARC: optimized interrupt stack frame size
With this change we allocate stack space only for the registers we
actually store in the thread interrupt stack frame.

Furthermore, no function is called on with the interrupt context save
frame %sp so no full frame is needed here. ABI functions are called
later in the interrupt trap handler, but that is after the dedicated
interrupt stack has been installed.

This saves 96 bytes of stack space for each interrupted context.

Signed-off-by: Martin Åberg <martin.aberg@gaisler.com>
2020-12-04 14:33:43 +02:00
Martin Åberg
3734465354 SPARC: optimized interrupt trap stores and loads
The input registers (i0..i7) are not modified by the interrupt trap
handler and are preserved by function calls. So we do not need to
store them in the interrupt stack frame.

This saves 48 bytes of stack space for each interrupted context,
and eliminates 4 double word stores and 4 double word loads.

Signed-off-by: Martin Åberg <martin.aberg@gaisler.com>
2020-12-04 14:33:43 +02:00