Commit graph

6,067 commits

Author SHA1 Message Date
Luke Starrett
4800b03e56 arch: arm64: cosmetic changes to register dump
- Display full 64-bits register width in crash dumps
- Some values were prefixed 0x, some not.  Made consistent.

Signed-off-by: Luke Starrett <luke.starrett@gmail.com>
2020-10-01 07:29:27 -04:00
Luke Starrett
169e7c5e75 arch: arm64: Fix arm64 crash dump output
- x0/x1 register printing is reversed
- The error stack frame struct (z_arch_esf_t) had the SPSR and ELR in
  the wrong position, inconsistent with the order these regs are pushed
  to the stack in z_arm64_svc.  This caused all register printing to be
  skewed by two.
- Verified by writing known values (abcd0000 -> abcd000f) to x0 - x15
  and then forcing a data abort.

Signed-off-by: Luke Starrett <luke.starrett@gmail.com>
2020-10-01 07:29:27 -04:00
Andrew Boie
c3c7f6c6d3 x86: don't define _image_rom_* unless XIP
Meaningless if we are not a XIP system and are running
from RAM.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-09-30 14:14:07 -07:00
Andrew Boie
f5a7e1a108 kernel: handle thread self-aborts on idle thread
Fixes races where threads on another CPU are joining the
exiting thread, since it could still be running when
the joiners wake up on a different CPU.

Fixes problems where the thread object is still being
used by the kernel when the fn_abort() function is called,
preventing the thread object from being recycled or
freed back to a slab pool.

Fixes a race where a thread is aborted from one CPU while
it self-aborts on another CPU, that was currently worked
around with a busy-wait.

Precedent for doing this comes from FreeRTOS, which also
performs final thread cleanup in the idle thread.

Some logic in z_thread_single_abort() rearranged such that
when we release sched_spinlock, the thread object pointer
is never dereferenced by the kernel again; join waiters
or fn_abort() logic may free it immediately.

An assertion added to z_thread_single_abort() to ensure
it never gets called with thread == _current outside of an ISR.

Some logic has been added to ensure z_thread_single_abort()
tasks don't run more than once.

Fixes: #26486
Related to: #23063 #23062

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-09-30 14:11:59 -04:00
Flavio Ceolin
27fcdaf71e arch: arm: Fix undefined symbol reference
_isr_wrapper is not defined when building with
CONFIG_GEN_SW_ISR_TABLE = n.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2020-09-29 12:36:33 +02:00
Ioannis Glaropoulos
3b89cf173b arch: arm: cortex-m: enable IRQs before main() in single-thread mode
Enable interrupts before switching to main()
in cortex-m builds with single-thread mode
(CONFIG_MULTITHREADING=n).

Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
2020-09-29 10:47:43 +02:00
Andrew Boie
054a4c0ddc x86-64: increase exception stack size
We are not RAM-constrained and there is an open issue where
exception stack overflows are not caught. Increase this size
so that options like CONFIG_NO_OPTIMIZATIONS work without
incident.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-09-28 13:38:41 -07:00
Andrew Boie
a696f8ba27 x86: rename CONFIG_EXCEPTION_STACKS_SIZE
This is an x86-specific definition.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-09-28 13:38:41 -07:00
Johan Hedberg
6f5d8bd2c4 x86: pcie: Fix calling pcie_mm_init()
Commit 5632ee26f3 introduced an issue where in order to use MMIO
configuration:

 - do_pcie_mmio_cfg is required to be true
 - Only set to true in pcie_mm_init()
 - Which is only called from pcie_mm_conf()
 - Which is only called from pcie_conf() if do_pcie_mmio_cfg is
   already true!

The end result is that MMIO configuration will never be used.

Fix the situation by moving the initialization check to pcie_conf().

Signed-off-by: Johan Hedberg <johan.hedberg@intel.com>
2020-09-24 16:08:08 +03:00
Øyvind Rønningstad
407ebf8132 cortex_m: secure_entry_functions.ld: Increase SAU alignment to 32
The spec requires SAU regions to be aligned on 32 bytes.

Signed-off-by: Øyvind Rønningstad <oyvind.ronningstad@nordicsemi.no>
2020-09-23 13:15:38 +02:00
Øyvind Rønningstad
81e7608c03 arm: tz: secure_entry_functions.ld: Fix NSC_ALIGN for nRF devices
If the location counter ('.') is within the area that the veneers
should go, the current solution will give a linker error ("Cannot move
location counter backwards"). This patch places the veneers in the next
SPU region in this case.

Signed-off-by: Øyvind Rønningstad <oyvind.ronningstad@nordicsemi.no>
2020-09-23 13:15:38 +02:00
Øyvind Rønningstad
2b56b86190 arm: tz: secure_entry_functions.ld: Fix NSC_ALIGN redefinition
Allow CONFIG_ARM_NSC_REGION_BASE_ADDRESS to override the nRF-specific
logic for alignment.

Fixes issue https://github.com/zephyrproject-rtos/zephyr/issues/27544

Signed-off-by: Øyvind Rønningstad <oyvind.ronningstad@nordicsemi.no>
2020-09-23 13:15:38 +02:00
Andrew Boie
61d42cf42b x86-64: fix thread tracing
The current instrumentation point for CONFIG_TRACING added in
PR #28512 had two problems:

- If userspace and KPTI are enabled, the tracing point is simply
  never run if we are resuming a user thread as the
  z_x86_trampoline_to_user function is jumped to and calls
  'iret' from there

- Only %rdi is being saved. However, at that location, *all*
  caller-saved registers are in use as they contain the
  resumed thread's context

Simplest solution is to move this up near where we update page
tables. The #ifdefs are used to make sure we don't push/pop
%rdi more than once. At that point in the code only %rdi
is in use among the volatile registers.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-09-22 20:47:48 -04:00
Wentong Wu
50252bbf92 arch: x86: mmu: use z_x86_kernel_ptables as array.
Use z_x86_kernel_ptables as array to make Coverity happy.

Coverity-CID: 212957.
Fixes: 27832.

Signed-off-by: Wentong Wu <wentong.wu@intel.com>
2020-09-22 16:00:03 -05:00
Ioannis Glaropoulos
14f248fe1b arch: arm: cortex_m: cleanup SW_VECTOR_RELAY_CLIENT dependencies
CPU Cortex-M implies Mainline Cortex-M, therfore, the dependency
on ARMV6_M_ARMV8_M_BASELINE is redundant and can be removed. The
change in this commit is a no-op.

We also add the ARMV6_M_ARMV8_M_BASELINE dependency on option
CPU_CORTEX_M0_HAS_VECTOR_TABLE_REMAP to make sure it cannot be
selected for non Cortex-M Baseline SoCs (at least, not without
a warning).

Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
2020-09-21 11:19:22 +02:00
Anas Nashif
75d159bf0d tracing: x86_64: move switched_in to switch function
Tracing switched in threads in C code does not work, it needs to happen
in the arch_switch code. See also Xtensa and ARC.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-09-20 21:27:55 -04:00
Kumar Gala
283a057279 x86_64: Fix memory access size for locore EOI
Newer QEMU (5.1) hangs / timeouts on a number of tests on x86_64.  In
debugging the issue this is related to a fix in QEMU 5.1 that
validates memory region access.  QEMU has the APIC region only allowing
1 to 4 byte access.  64-bit access is treated as an error.

Change the APIC EOI access in locore.S back to just doing a 32-bit
access.

Fixes #	28453

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-09-18 13:29:00 -05:00
Daniel Leung
43b2e291db x86_64: fix size to init stack at boot
The boot code of x86_64 initializes the stack (if enabled)
with a hard-coded size for the ISR stack. However,
the stack being used does not have to be the ISR stack,
and can be any defined stacks. So pass in the actual size
of the stack so the stack can be initialized properly.

Fixes #21843

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-09-17 21:05:45 -04:00
Crist Xu
ac3d9438ed drivers: usb: Fix usb fail when using the on-chip memory
Using SCB_CleanInvalidateDcache instead of SCB_DisableDcache
 & SCB_EnableDcache when config the non-cache area, in case
of the cache will effect the configuration of the non-cache
area

Signed-off-by: Crist Xu <crist.xu@nxp.com>
2020-09-17 16:56:28 -05:00
Andrew Boie
8240d7d07b x86: memory map BIOS Data Area
Changes to paging code ensured that the NULL virtual page is
never mapped. Since RAM is identity mapped, on a PC-like
system accessing the BIOS Data Area in the first 4K requires
a memory mapping. We need to read this to probe the ACPI RSDP.

Additionally check that the BDA has something in it as well
and not a bunch of zeroes.

It is unclear whether this function is truly safe on UEFI
systems, but that is for another day.

Fixes: #27867

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-09-16 20:37:34 -04:00
Daniel Leung
5632ee26f3 x86: pcie: fallback to config via PIO
When probing for PCI-E device resources, it is possible that
configuration via MMIO is not available. This may caused by
BIOS or its settings. So when CONFIG_PCIE_MMIO_CFG=y, have
a fallback path to config devices via PIO. The inability to
config via MMIO has been observed on a couple UP Squared
boards.

Fixes #27339

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-09-16 14:16:50 -05:00
Tomasz Bursztyka
ed98883795 device: Fixing new left over device instance made constant
Recent addition that went under the radar.

Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
2020-09-15 09:18:43 -05:00
Andrew Boie
2bc21ea4de x86: print more detail on non-present pagefaults
The CPU sets the relevant bits on who tried to do what
if the page wasn't present, print them.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-09-11 09:03:01 -04:00
Andrew Boie
aebb9d8a45 aarch64: work around QEMU 'wfi' issue
Work around an issue where the emulator ignores host OS
signals when inside a `wfi` instruction.

This should be reverted once this has been addressed in the
AARCH64 build of QEMU in the SDK.

See https://github.com/zephyrproject-rtos/sdk-ng/issues/255

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-09-10 21:31:15 +02:00
Carlo Caione
d6f608219c arm64: tracing: Fix double tracing
When _arch_switch() API is used, the tracing of the thread swapped out
is done in the C kernel code (in do_swap() for cooperative scheduling
and in set_current() during preemption). In the assembly code we only
have to trace the thread when swapped in.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2020-09-09 15:36:43 -04:00
Ioannis Glaropoulos
394d2912a1 arch: arm: cortex-m: implement timing.c based on DWT
For Cortex-M platforms with DWT we implement
the timing API (timing.c).

Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
2020-09-05 13:28:38 -05:00
Ioannis Glaropoulos
6f84d7d3fd arch: arm: cortex_m: conditionally select ARCH_HAS_TIMING_FUNCTIONS
Cortex-M SoCs implement (optionally) the Data Watchpoint and
Tracing Unit (DWT), which can be used for timing functions.
Select the corresponding ARCH capability if the SoC implements
the DWT.

Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
2020-09-05 13:28:38 -05:00
Anas Nashif
6e27478c3d benchmarking: remove execution benchmarking code
This code had one purpose only, feed timing information into a test and
was not used by anything else. The custom trace points unfortunatly were
not accurate and this test was delivering informatin that conflicted
with other tests we have due to placement of such trace points in the
architecture and kernel code.

For such measurements we are planning to use the tracing functionality
in a special mode that would be used for metrics without polluting the
architecture and kernel code with additional tracing and timing code.

Furthermore, much of the assembly code used had issues.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-09-05 13:28:38 -05:00
Anas Nashif
150c82c8f9 arch: nios2: add timing implementation
Add timing implementation for NIOS2 architecture.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-09-05 13:28:38 -05:00
Anas Nashif
d896decb79 timing: add support for x86
Add initial support for X86 and get timestamps from tsc.

Authored-by: Daniel Leung <daniel.leung@intel.com>
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-09-05 13:28:38 -05:00
Anas Nashif
5dec235196 arch: default timings for all architectures
Use default if architecture does not have a custom timing
implementation.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-09-05 13:28:38 -05:00
Eugeniy Paltsev
a28ec6201f isr_tables: don't whole-archive library
As of today we have a bit weird situation with generated
sw_isr_table / irq_vector_table tables.

On the final linkage stage we pass two files which content
section with sw_isr_table / irq_vector_table. They are
 * libarch__common.a (with an outdated tables from the first
   linkage stage)
 * isr_tables.c.obj (with an actual tables)

The sections where tables are located are marked with
".gnu.linkonce" prefix. That means:
<<<As a GNU extension, if the name begins with .gnu.linkonce,
we only link a single copy of the section.>>>

However the "libarch__common.a" is passed to linker with
"--whole-archive" option which means <<<include every object
file in the archive in the link, rather than searching the archive
for the required object files>>>

That combination confuses MWDT linker and breaks linkage with
MWDT toolchain.

As a simple fix we can move the sw_isr_table / irq_vector_table
sections to their own library and link this library with
"--no-whole-archive" option.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
2020-09-05 10:22:56 -05:00
Wayne Ren
fb05bfce05 ARC: rename arch_switch implementation to z_arc_switch
"arch_switch" is declared as an inline function in kswap.h,
it should be a wrapper of arch level switch. The difference
of declaration and implementation of "arch_swich" causes
warning from MWDT compiler.

Use "arch_switch" with proper declararion (which is just
wraper for "z_arc_switch") to do conext switch for ARC.

Signed-off-by: Wayne Ren <wei.ren@synopsys.com>
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
2020-09-05 10:22:56 -05:00
Wayne Ren
ef224ce1cd ARC: make the assembly codes compatible
Make the assembly codes compatible with both GNU
and Metaware toolchain.

* replace ".balign" with ".align"
  ".align" assembler directive is supposed by all
  ARC toolchains and it is implemented in a same
  way across ARC toolchains.
* replace "mov_s __certain_reg" with "mov __certain_reg"
  Even though GCC encodes those mnemonics and even real
  HW executes them according to PRM these are restricted
  ones for mov_s and CCAC rightfully refuses to accept
  such mnemonics. So for compatibility and clarity sake
  we switch to 32-bit mov instruction which allows use
  of all those instructions.
* Add "%%" prefix while accessing registers from inline
  ASM as it is required by MWDT.
* Drop "@" prefix while accessing symbols (defined in C
  code) from ASM code as it is required by MWDT.

Signed-off-by: Wayne Ren <wei.ren@synopsys.com>

/#
2020-09-05 10:22:56 -05:00
Wayne Ren
d67475ab6e ARC: handle the difference of assembly macro definition
GNU toolchain and MWDT (Metware) toolchain have different style
for accessing arguments in assembly macro. Implement the
preprocessor macro to handle the difference.

Make all ASM macros in swap_macros.h compatible for both ARC
toolchains.

Signed-off-by: Wayne Ren <wei.ren@synopsys.com>
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
2020-09-05 10:22:56 -05:00
Carlo Caione
df4aa230c8 arch: arm64: Use _arch_switch() API
Switch to the _arch_switch() API that is required for an SMP-aware
scheduler instead of using the old arch_swap mechanism.

SMP is not supported yet but this is a necessary step in that direction.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2020-09-05 12:06:38 +02:00
Torsten Rasmussen
c55c64e242 toolchain: improved toolchain abstraction for compilers and linker
First abstraction completed for the toolchains:
- gcc
- clang

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2020-09-04 20:36:59 +02:00
Øyvind Rønningstad
2be0086e87 cortex_m: tz_ns.h: Various fixes (late comments on PR)
Fix dox and restructure ASM.
No functional changes.

Signed-off-by: Øyvind Rønningstad <oyvind.ronningstad@nordicsemi.no>
2020-09-04 19:05:58 +02:00
Pavel Král
06342e3474 arch: arm: mpu: Removal of include path pollution
Removes unnecessary and incorrect directories from include path.

Signed-off-by: Pavel Král <pavel.kral@omsquare.com>
2020-09-04 13:58:38 +02:00
Øyvind Rønningstad
c00f33dcb0 arch: arm: cortex_m: Add tz_ns.h
Provide a TZ_SAFE_ENTRY_FUNC() macro for wrapping non-secure entry
functions in calls to k_sched_lock()/k_sched_unlock()

Provide a __TZ_WRAP_FUNC() macro which helps in creating a function
that "wraps" another in a preface and postface function call.

	int foo(char *arg); // Implemented somewhere else.
	int __attribute__((naked)) foo_wrapped(char *arg)
	{
		WRAP_FUNC(bar, foo, baz);
	}

is equivalent to

	int foo(char *arg); // Implemented somewhere else.
	int foo_wrapped(char *arg)
	{
		bar();
		int res = foo(arg);
		baz();
		return res;
	}

This commit also adds tests for __TZ_WRAP_FUNC().

Signed-off-by: Øyvind Rønningstad <oyvind.ronningstad@nordicsemi.no>
2020-09-04 11:58:41 +02:00
Watson Zeng
1dddbecb35 tracing: swap: bug fix and enhancement for ARC
* Move switched_in into the arch context switch assembly code,
  which will correctly record the switched_in information.

* Add switched_in/switched_out for context switch in irq exit.

Signed-off-by: Watson Zeng <zhiwei@synopsys.com>
2020-09-03 21:54:15 +02:00
Andrew Boie
7d32e9f9a5 mmu: support only identity RAM mapping
We no longer plan to support a split address space with
the kernel in high memory and per-process address spaces.
Because of this, we can simplify some things. System RAM
is now always identity mapped at boot.

We no longer require any virtual-to-physical translation
for page tables, and can remove the dual-mapping logic
from the page table generation script since we won't need
to transition the instruction point off of physical
addresses.

CONFIG_KERNEL_VM_BASE and CONFIG_KERNEL_VM_LIMIT
have been removed. The kernel's address space always
starts at CONFIG_SRAM_BASE_ADDRESS, of a fixed size
specified by CONFIG_KERNEL_VM_SIZE.

Driver MMIOs and other uses of k_mem_map() are still
virtually mapped, and the later introduction of demand
paging will result in only a subset of system RAM being
a fixed identity mapping instead of all of it.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-09-03 14:24:38 -04:00
Flavio Ceolin
7f9cc2359e x86-32: Allow set DPL value for an exception
In order to be possible to debug usermode threads need to be able
issue breakpoint and debug exceptions. To do this it is necessary to
set DPL bits to, at least, the same CPL level.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2020-09-02 20:54:57 -04:00
Flavio Ceolin
5408f3102d debug: x86: Add gdbstub for X86
It implements gdb remote protocol to talk with a host gdb during the
debug session. The implementation is divided in three layers:

1 - The top layer that is responsible for the gdb remote protocol.
2 - An architecture specific layer responsible to write/read registers,
    set breakpoints, handle exceptions, ...
3 - A transport layer to be used to communicate with the host

The communication with GDB in the host is synchronous and the systems
stops execution waiting for instructions and return its execution after
a "continue" or "step" command. The protocol has an exception that is
when the host sends a packet to cause an interruption, usually triggered
by a Ctrl-C. This implementation ignores this instruction though.

This initial work supports only X86 using uart as backend.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2020-09-02 20:54:57 -04:00
Andrew Boie
ffc1da08f9 kernel: add z_thread_single_abort to private hdr
We shouldn't be copy-pasting extern declarations like this.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-09-02 15:02:06 -07:00
Andrew Boie
3425c32328 kernel: move stuff into z_thread_single_abort()
The same code was being copypasted in k_thread_abort()
implementations, just move into z_thread_single_abort().

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-09-02 15:02:06 -07:00
Andrew Boie
e34ac286b7 arm: don't lock irqs during thread abort
This isn't needed; match the vanilla implementation
in kernel/thread_abort.c and do this unlocked. This
should improve system latency.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-09-02 15:02:06 -07:00
Andrew Boie
0a99011357 arm: thread_abort: clarify what's going on
A check was being done that was a more obscure way of
calling arch_is_in_isr(). Add a comment explaining
why we need to trigger PendSV.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-09-02 15:02:06 -07:00
Ioannis Glaropoulos
e08dfec77c arch: arm: cortex-m: add ARM-only API to set all IRQS to Non-Secure
We implement an ARM-only API for ARM Secure Firmware,
to set all NVIC IRQ lines to target the Non-Secure state.

Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
2020-09-02 15:01:30 +02:00
Ioannis Glaropoulos
4ec7725110 arch: arm: cortex-m: Modify ARM-only API for IRQ target state mgmt
we modify the ARM Cortex-M only API for managing the
security target state of the NVIC IRQs. We remove the
internal ASSERT checking allowing to call the API for
non-implemented NVIC IRQ lines. However we still give the
option to the user to check the success of the IRQ target
state setting operation by allowing the API function to
return the resulting target state.

Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
2020-09-02 15:01:30 +02:00