Commit graph

5896 commits

Author SHA1 Message Date
Daniel Leung
53ac1ee6fa x86_64: add support for thread local storage
Adds the necessary bits to initialize TLS in the stack
area and sets up CPU registers during context switch.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-10-24 10:52:00 -07:00
Daniel Leung
240beb42af kconfig: add configs for thread local storage
Add kconfigs to indicate whether an architecture has support
for thread local storage (TLS), and to enable TLS in kernel.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-10-24 10:52:00 -07:00
Johan Hedberg
1c23a19c36 x86: pcie: Fix array index for bus_segs
This seems like a typo since all other places accessing bus_segs in
this context use i as the index.

Signed-off-by: Johan Hedberg <johan.hedberg@intel.com>
2020-10-23 22:01:59 -04:00
Watson Zeng
3d30f1f60a arc: cpu_idle: remove sleep workround for nsim_hs_smp
In old version nSIM, when cpu is sleeping, no response to
inter-processor interrupt although it's pending and interrupts
are enabled(SNPS JIRA issue P10019563-41294). Now this has
been fixed in nSIM version (2020.09), so we can safely remove it.

Signed-off-by: Watson Zeng <zhiwei@synopsys.com>
2020-10-22 06:17:08 -04:00
Andy Ross
a8d5437799 soc/xtensa: Misc. checkpatch fixups
Code style fixes.  Kept separate from the original changes to permit
easier rebasing.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2020-10-21 06:38:53 -04:00
Andy Ross
0e83961b21 arch/xtensa: soc/xtensa/intel_adsp: Enable KERNEL_COHERENCE
Implement the kernel "coherence" API on top of the linker
cached/uncached mapping work.

Add Xtensa handling for the stack coherence API.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2020-10-21 06:38:53 -04:00
Andy Ross
f6d32ab0a4 kernel: Add cache coherence management framework
Zephyr SMP kernels need to be able to run on architectures with
incoherent caches.  Naive implementation of synchronization on such
architectures requires extensive cache flushing (e.g. flush+invalidate
everything on every spin lock operation, flush on every unlock!) and
is a performance problem.

Instead, many of these systems will have access to separate "coherent"
(usually uncached) and "incoherent" regions of memory.  Where this is
available, place all writable data sections by default into the
coherent region.  An "__incoherent" attribute flag is defined for data
regions that are known to be CPU-local and which should use the cache.
By default, this is used for stack memory.

Stack memory will be incoherent by default, as by definition it is
local to its current thread.  This requires special cache management
on context switch, so an arch API has been added for that.

Also, when enabled, add assertions to strategic places to ensure that
shared kernel data is indeed coherent.  We check thread objects, the
_kernel struct, waitq's, timeouts and spinlocks.  In practice almost
all kernel synchronization is built on top of these structures, and
any shared data structs will contain at least one of them.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-10-21 06:38:53 -04:00
Andy Ross
47940a63d7 arch/xtensa: Don't clear BSS on MP startup when !SMP
It's legal to have CONFIG_MP_NUM_CPUS > 1 and !CONFIG_SMP.  The
tests/kernel/mp test does this as a unit test of the multiprocessor
facilities. Test the right tunable when deciding whether to blow away
static data or not.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2020-10-21 06:38:53 -04:00
Andrew Boie
507ebd541a x86: remove NULL check in arch_user_mode_enter
These days all threads are always a member of a memory domain,
remove this NULL check as it won't ever be false.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-10-20 09:37:49 -07:00
Andrew Boie
f6c64e92ce x86: fix arch_user_mode_enter locking
This function iterates over the thread's memory domain
and updates page tables based on it. We need to be holding
z_mem_domain_lock while this happens.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-10-20 09:37:49 -07:00
Mikkel Jakobsen
8f2d69c4a3 arch: posix: add missing include for cpuhalt.c
Add posix_board_if.h which declares posix_exit().

This fixes implicit declaration of function errors when running
sanitycheck on samples for native_posix that calls sys_reboot().

Signed-off-by: Mikkel Jakobsen <mikkel.aunsbjerg@prevas.dk>
2020-10-20 08:54:59 +02:00
Maximilian Bachmann
498655ae8e arch/x86: fix compilation errors
fixes the following compilation errors
- sys_cache_line_size was undeclared at first use
- there was an assignment to an rvalue in arch_dcache_flush

Signed-off-by: Maximilian Bachmann <m.bachmann@acontis.com>
2020-10-14 13:27:23 -07:00
Andy Ross
55a85771b0 arch/x86: Make EFI copies bytewise
Originally the EFI boot code was written to assume that all sections
in the ELF file were 8-byte aligned and sized (because I thought this
was part of some platform spec somewhere).  This turned out to be
wrong in practice (at least for section sizes), so the requirement was
reduced to 4 bytes.  But now we have a section being generated
somewhere that turns out to violate even that.

There's no particular value in doing those copies in big chunks.
There's at best a mild performance benefit, but if we really cared
we'd be using a more complicated memcpy() implementation anyway.
Replace the loop in the C code with a bytewise copy, change the size
field in the generated header to store bytes, and remove the
assertions (which were the failuers actually being seen in practice)
in the script that were there to detect this misalignment.

Fixes #29095

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2020-10-13 14:07:24 -07:00
Carlo Caione
645082791b arch: aarch64: Catch early errors in EL3 and EL1
Setup the stack as early as possible to catch any possible errors in the
reset routine and handle also EL3 fatal errors.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2020-10-12 12:22:15 -04:00
Carlo Caione
758fb93b0b arch: arm64: Remove useless assembly code
The content of the SCR_EL3 register is overwritten by a later
instruction. Also no need to route SError, IRQs and FIQs to EL3.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2020-10-12 12:22:15 -04:00
Carlo Caione
06adb96c1a arch: arm64: Introduce {inc,dec}_nest_counter macros
The same code is used in several place / files. Make a macro out of it.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2020-10-06 10:25:56 -04:00
Carlo Caione
2f3962534a arch: arm64: Remove new thread entry wrapper
Instead of having some special stack frame when first scheduling new
thread and a new thread entry wrapper to pull out the needed data, we
can reuse the context restore code by adapting the initial stack frame.

This reduces the lines of code and simplify the code at the expense of a
slightly bigger initial stack frame.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2020-10-06 10:25:56 -04:00
Torstein Grindvik
c02eb30bbf gen_isr_tables: Function ptr instead of (void *)
gen_isr_tables.py generates C-code which initializes a table with
values, and these values are structs with members cast to
(const void *) and (void *), respectively.

The actual struct definition has a member of type (const void *)
and another of type void (*)(const void *).

In order to avoid a large amount of reported issues in Coverity,
cast this to the exact type.

Signed-off-by: Torstein Grindvik <torstein.grindvik@nordicsemi.no>
2020-10-02 18:48:46 +02:00
Yuguo Zou
d04ff1af7c arch: arc: Restore MPU registers to its initial states between tests
EMSK boards can't be reset between tests due to hardware configures.
MPU v3 configs in previous test could cause exceptions in the following
tests. This commit fixes this issue by restoring MPU registers initial
states at early init stage.

Signed-off-by: Yuguo Zou <yuguo.zou@synopsys.com>
2020-10-02 11:31:34 +02:00
Yuguo Zou
df4b7803a1 arch: arc: unify different versions of MPU registers
Previously MPU registers macros are only defined within its own header
files and could not be used by other part of program. This commit unify
them together.

Signed-off-by: Yuguo Zou <yuguo.zou@synopsys.com>
2020-10-02 11:31:34 +02:00
Aastha Grover
83b9f69755 code-guideline: Fixing code violation 10.4 Rule
Both operands of an operator in the arithmetic conversions
performed shall have the same essential type category.

Changes are related to converting the integer constants to the
unsigned integer constants

Signed-off-by: Aastha Grover <aastha.grover@intel.com>
2020-10-01 17:13:29 -04:00
Tomasz Bursztyka
2b6c574499 arch/x86: If ACPI is enabled, get the CPU APIC ID from it
The hardcoded APIC ID will be kept as default if the CPU is not found in
ACPI MADT.

Note that ACPI may expose more "CPUs" than there actually are
physically. Thus, make the logic aware of this possibility by checking
the enabled flas. (Non-enabled CPU are ignored).

This fixes up_squared board made of Celeron CPU.

Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
2020-10-01 11:16:40 -07:00
Tomasz Bursztyka
d98f7b1895 arch/x86: Optimize ACPI RSDP lookup
As well as normalizing its signature declaration through header.

Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
2020-10-01 11:16:40 -07:00
Tomasz Bursztyka
4ff1885f69 arch/x86: Move ACPI structures to header file
Let's have all specified ACPI structures in the central header.

Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
2020-10-01 11:16:40 -07:00
Tomasz Bursztyka
9ce08544c5 arch/x86: Fix style issue and code logic in ACPI
Trivial changes.

Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
2020-10-01 11:16:40 -07:00
Tomasz Bursztyka
c7787c623e arch/x86: Cleanup ACPI structure attributes names
No need to mix super short version of names with other structures
having full name. Let's follow a more relevant naming where each and
every attribute name is self-documenting then. (such as s/id/apic_id
etc...)

Also make CONFIG_ACPI usable through IS_ENABLED by enclosing exposed
functions with ifdef CONFIG_ACPI.

Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
2020-10-01 11:16:40 -07:00
Carlo Caione
871bdd0712 arch: arm64: Deprecate booting from EL2
We are deprecating booting and running in EL2.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2020-10-01 10:42:47 -04:00
Carlo Caione
fb2bf23ec1 arch: arm64: Remove EL2/EL3 code
Zephyr is only supposed to be running at EL1 (+ EL0). Now that we drop
in EL1 from ELn at start we can remove all the EL2/EL3 unused code.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2020-10-01 10:42:47 -04:00
Carlo Caione
7d40208ef7 arch: arm64: Remove CONFIG_SWITCH_TO_EL1
Remove the useless CONFIG_SWITCH_TO_EL1 since there should be no reason
to run Zephyr in EL3. So just drop to EL1 by default when booting from
EL3. Remove also non-reachable code.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2020-10-01 10:42:47 -04:00
Luke Starrett
4800b03e56 arch: arm64: cosmetic changes to register dump
- Display full 64-bits register width in crash dumps
- Some values were prefixed 0x, some not.  Made consistent.

Signed-off-by: Luke Starrett <luke.starrett@gmail.com>
2020-10-01 07:29:27 -04:00
Luke Starrett
169e7c5e75 arch: arm64: Fix arm64 crash dump output
- x0/x1 register printing is reversed
- The error stack frame struct (z_arch_esf_t) had the SPSR and ELR in
  the wrong position, inconsistent with the order these regs are pushed
  to the stack in z_arm64_svc.  This caused all register printing to be
  skewed by two.
- Verified by writing known values (abcd0000 -> abcd000f) to x0 - x15
  and then forcing a data abort.

Signed-off-by: Luke Starrett <luke.starrett@gmail.com>
2020-10-01 07:29:27 -04:00
Andrew Boie
c3c7f6c6d3 x86: don't define _image_rom_* unless XIP
Meaningless if we are not a XIP system and are running
from RAM.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-09-30 14:14:07 -07:00
Andrew Boie
f5a7e1a108 kernel: handle thread self-aborts on idle thread
Fixes races where threads on another CPU are joining the
exiting thread, since it could still be running when
the joiners wake up on a different CPU.

Fixes problems where the thread object is still being
used by the kernel when the fn_abort() function is called,
preventing the thread object from being recycled or
freed back to a slab pool.

Fixes a race where a thread is aborted from one CPU while
it self-aborts on another CPU, that was currently worked
around with a busy-wait.

Precedent for doing this comes from FreeRTOS, which also
performs final thread cleanup in the idle thread.

Some logic in z_thread_single_abort() rearranged such that
when we release sched_spinlock, the thread object pointer
is never dereferenced by the kernel again; join waiters
or fn_abort() logic may free it immediately.

An assertion added to z_thread_single_abort() to ensure
it never gets called with thread == _current outside of an ISR.

Some logic has been added to ensure z_thread_single_abort()
tasks don't run more than once.

Fixes: #26486
Related to: #23063 #23062

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-09-30 14:11:59 -04:00
Flavio Ceolin
27fcdaf71e arch: arm: Fix undefined symbol reference
_isr_wrapper is not defined when building with
CONFIG_GEN_SW_ISR_TABLE = n.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2020-09-29 12:36:33 +02:00
Ioannis Glaropoulos
3b89cf173b arch: arm: cortex-m: enable IRQs before main() in single-thread mode
Enable interrupts before switching to main()
in cortex-m builds with single-thread mode
(CONFIG_MULTITHREADING=n).

Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
2020-09-29 10:47:43 +02:00
Andrew Boie
054a4c0ddc x86-64: increase exception stack size
We are not RAM-constrained and there is an open issue where
exception stack overflows are not caught. Increase this size
so that options like CONFIG_NO_OPTIMIZATIONS work without
incident.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-09-28 13:38:41 -07:00
Andrew Boie
a696f8ba27 x86: rename CONFIG_EXCEPTION_STACKS_SIZE
This is an x86-specific definition.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-09-28 13:38:41 -07:00
Johan Hedberg
6f5d8bd2c4 x86: pcie: Fix calling pcie_mm_init()
Commit 5632ee26f3 introduced an issue where in order to use MMIO
configuration:

 - do_pcie_mmio_cfg is required to be true
 - Only set to true in pcie_mm_init()
 - Which is only called from pcie_mm_conf()
 - Which is only called from pcie_conf() if do_pcie_mmio_cfg is
   already true!

The end result is that MMIO configuration will never be used.

Fix the situation by moving the initialization check to pcie_conf().

Signed-off-by: Johan Hedberg <johan.hedberg@intel.com>
2020-09-24 16:08:08 +03:00
Øyvind Rønningstad
407ebf8132 cortex_m: secure_entry_functions.ld: Increase SAU alignment to 32
The spec requires SAU regions to be aligned on 32 bytes.

Signed-off-by: Øyvind Rønningstad <oyvind.ronningstad@nordicsemi.no>
2020-09-23 13:15:38 +02:00
Øyvind Rønningstad
81e7608c03 arm: tz: secure_entry_functions.ld: Fix NSC_ALIGN for nRF devices
If the location counter ('.') is within the area that the veneers
should go, the current solution will give a linker error ("Cannot move
location counter backwards"). This patch places the veneers in the next
SPU region in this case.

Signed-off-by: Øyvind Rønningstad <oyvind.ronningstad@nordicsemi.no>
2020-09-23 13:15:38 +02:00
Øyvind Rønningstad
2b56b86190 arm: tz: secure_entry_functions.ld: Fix NSC_ALIGN redefinition
Allow CONFIG_ARM_NSC_REGION_BASE_ADDRESS to override the nRF-specific
logic for alignment.

Fixes issue https://github.com/zephyrproject-rtos/zephyr/issues/27544

Signed-off-by: Øyvind Rønningstad <oyvind.ronningstad@nordicsemi.no>
2020-09-23 13:15:38 +02:00
Andrew Boie
61d42cf42b x86-64: fix thread tracing
The current instrumentation point for CONFIG_TRACING added in
PR #28512 had two problems:

- If userspace and KPTI are enabled, the tracing point is simply
  never run if we are resuming a user thread as the
  z_x86_trampoline_to_user function is jumped to and calls
  'iret' from there

- Only %rdi is being saved. However, at that location, *all*
  caller-saved registers are in use as they contain the
  resumed thread's context

Simplest solution is to move this up near where we update page
tables. The #ifdefs are used to make sure we don't push/pop
%rdi more than once. At that point in the code only %rdi
is in use among the volatile registers.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-09-22 20:47:48 -04:00
Wentong Wu
50252bbf92 arch: x86: mmu: use z_x86_kernel_ptables as array.
Use z_x86_kernel_ptables as array to make Coverity happy.

Coverity-CID: 212957.
Fixes: 27832.

Signed-off-by: Wentong Wu <wentong.wu@intel.com>
2020-09-22 16:00:03 -05:00
Ioannis Glaropoulos
14f248fe1b arch: arm: cortex_m: cleanup SW_VECTOR_RELAY_CLIENT dependencies
CPU Cortex-M implies Mainline Cortex-M, therfore, the dependency
on ARMV6_M_ARMV8_M_BASELINE is redundant and can be removed. The
change in this commit is a no-op.

We also add the ARMV6_M_ARMV8_M_BASELINE dependency on option
CPU_CORTEX_M0_HAS_VECTOR_TABLE_REMAP to make sure it cannot be
selected for non Cortex-M Baseline SoCs (at least, not without
a warning).

Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
2020-09-21 11:19:22 +02:00
Anas Nashif
75d159bf0d tracing: x86_64: move switched_in to switch function
Tracing switched in threads in C code does not work, it needs to happen
in the arch_switch code. See also Xtensa and ARC.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-09-20 21:27:55 -04:00
Kumar Gala
283a057279 x86_64: Fix memory access size for locore EOI
Newer QEMU (5.1) hangs / timeouts on a number of tests on x86_64.  In
debugging the issue this is related to a fix in QEMU 5.1 that
validates memory region access.  QEMU has the APIC region only allowing
1 to 4 byte access.  64-bit access is treated as an error.

Change the APIC EOI access in locore.S back to just doing a 32-bit
access.

Fixes #	28453

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-09-18 13:29:00 -05:00
Daniel Leung
43b2e291db x86_64: fix size to init stack at boot
The boot code of x86_64 initializes the stack (if enabled)
with a hard-coded size for the ISR stack. However,
the stack being used does not have to be the ISR stack,
and can be any defined stacks. So pass in the actual size
of the stack so the stack can be initialized properly.

Fixes #21843

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-09-17 21:05:45 -04:00
Crist Xu
ac3d9438ed drivers: usb: Fix usb fail when using the on-chip memory
Using SCB_CleanInvalidateDcache instead of SCB_DisableDcache
 & SCB_EnableDcache when config the non-cache area, in case
of the cache will effect the configuration of the non-cache
area

Signed-off-by: Crist Xu <crist.xu@nxp.com>
2020-09-17 16:56:28 -05:00
Andrew Boie
8240d7d07b x86: memory map BIOS Data Area
Changes to paging code ensured that the NULL virtual page is
never mapped. Since RAM is identity mapped, on a PC-like
system accessing the BIOS Data Area in the first 4K requires
a memory mapping. We need to read this to probe the ACPI RSDP.

Additionally check that the BDA has something in it as well
and not a bunch of zeroes.

It is unclear whether this function is truly safe on UEFI
systems, but that is for another day.

Fixes: #27867

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-09-16 20:37:34 -04:00
Daniel Leung
5632ee26f3 x86: pcie: fallback to config via PIO
When probing for PCI-E device resources, it is possible that
configuration via MMIO is not available. This may caused by
BIOS or its settings. So when CONFIG_PCIE_MMIO_CFG=y, have
a fallback path to config devices via PIO. The inability to
config via MMIO has been observed on a couple UP Squared
boards.

Fixes #27339

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-09-16 14:16:50 -05:00