Commit graph

5846 commits

Author SHA1 Message Date
Carlo Caione
2683a1ed97 aarch64: error: Enable recoverable errors
For some kind of faults we want to be able to put in action some
corrective actions and keep executing the code.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2020-11-17 18:52:45 -05:00
Carlo Caione
a054e424e4 aarch64: error: Beautify error printing
Make the printing of errors a bit more descriptive and print the FAR_ELn
register only when strictly required.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2020-11-17 18:52:45 -05:00
Carlo Caione
6d05c57781 arch: aarch64: Branch SError vector table entry
Each vector table entry has 128-bytes to host the vector code. This is
not always enough and in general it's better to branch to the actual
exception handler elsewhere in memory.

Move the SError entry to a branched code.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2020-11-17 18:52:45 -05:00
Martin Åberg
a5293567c0 SPARC: adjust network stack stack sizes
Makes the following tests pass on qemu_leon3:
- net.ipv6
- net.tcp2.simple

Signed-off-by: Martin Åberg <martin.aberg@gaisler.com>
2020-11-17 11:44:16 +02:00
Martin Åberg
feae3249b2 sparc: add support for thread local storage
Adds the necessary bits to initialize TLS in the stack
area and sets up CPU registers during context switch. Register g7 is
used to point to the thread data. Thread data is accessed with negative
offsets from g7.

Signed-off-by: Martin Åberg <martin.aberg@gaisler.com>
2020-11-13 14:53:55 -08:00
Martin Åberg
07160fa153 arch: Add SPARC processor architecture
SPARC is an open and royalty free processor architecture.

This commit provides SPARC architecture support to Zephyr. It is
compatible with the SPARC V8 specification and the SPARC ABI and is
independent of processor implementation.

Functionality specific to SPRAC processor implementations should
go in soc/sparc. One example is the LEON3 SOC which is part of this
patch set.

The architecture port is fully SPARC ABI compatible, including trap
handlers and interrupt context.

Number of implemented register windows can be configured.

Some SPARC V8 processors borrow the CASA (compare-and-swap) atomic
instructions from SPARC V9. An option has been defined in the
architecture port to forward the corresponding code-generation option
to the compiler.

Stack size related config options have been defined in sparc/Kconfig
to match the SPARC ABI.

Co-authored-by: Nikolaus Huber <nikolaus.huber.melk@gmail.com>
Signed-off-by: Martin Åberg <martin.aberg@gaisler.com>
2020-11-13 14:53:55 -08:00
Wentong Wu
bfc7785da0 arch: arm: push ssf to thread privileged stack to complete stack frame
Pushes the seventh argument named ssf to thread's privileged
stack to follow below syscall prototype.

uintptr_t z_mrsh_xx(uintptr_t arg0, uintptr_t arg1, uintptr_t arg2,
		    uintptr_t arg3, uintptr_t arg4, void *more, void *ssf)

Fixes: #29386.

Signed-off-by: Wentong Wu <wentong.wu@intel.com>
2020-11-12 17:12:38 -05:00
Carlo Caione
d770880a71 kconfig: userspace: Select THREAD_STACK_INFO
stack_info is needed for userspace and TLS. Select it when enabling
userspace.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2020-11-12 15:57:40 -05:00
Daniel Leung
270ce0b850 nios2: fix register saving during thread switch instrumentation
This changes to use stack to store registers before calling thread
switch instrumentation functions, instead of using the thread's
register saving struct.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-11-11 23:55:49 -05:00
Daniel Leung
11e6b43090 tracing: roll thread switch in/out into thread stats functions
Since the tracing of thread being switched in/out has the same
instrumentation points, we can roll the tracing function calls
into the one for thread stats gathering functions.
This avoids duplicating code to call another function.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-11-11 23:55:49 -05:00
Daniel Leung
9be37553ee timing: do not repeatedly do init()/start()/stop()
We should not be initializing/starting/stoping timing functions
multiple times. So this changes how the timing functions are
structured to allow only one initialization, only start when
stopped, and only stop when started.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-11-11 23:55:49 -05:00
Carlo Caione
3afb493858 arch: arm64: mmu: Restore SRAM region
In a5f34d85c2 ("soc: arm: qemu_cortex_a53: Remove SRAM region") the
SRAM memory region was removed.

While this is correct when userspace is not enabled, when userspace is
enabled new regions are introduced outside the boundaries of
the mapped [__kernel_ram_start,__kernel_ram_end] region. This means that
we need to map again the whole SRAM to include all the needed regions.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2020-11-11 08:21:53 -05:00
Daniel Leung
c7704d8c66 arc: enable thread local storage
This adds the necessary bits to support thread local storage
in ARC.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-11-11 13:25:29 +01:00
Yuguo Zou
ba2413f544 arch: arc: change to CONFIG_INIT_ARCH_HW_AT_BOOT
align kconfig option CONFIG_ARC_CUSTOM_INIT to
CONFIG_INIT_ARCH_HW_AT_BOOT. Remove unused CONFIG_ARC_CUSTOM_INIT in
kconfig.

Signed-off-by: Yuguo Zou <yuguo.zou@synopsys.com>
2020-11-11 13:20:14 +01:00
Andrew Boie
ea6e4ad098 kernel: support non-identity RAM mapping
Some platforms may have multiple RAM regions which are
dis-continuous in the physical memory map. We really want
these to be in a continuous virtual region, and we need to
stop assuming that there is just one SRAM region that is
identity-mapped.

We no longer use CONFIG_SRAM_BASE_ADDRESS and CONFIG_SRAM_SIZE
as the bounds of kernel RAM, and no longer assume in the core
kernel that these are identity mapped at boot.

Two new Kconfigs, CONFIG_KERNEL_VM_BASE and
CONFIG_KERNEL_RAM_SIZE now indicate the bounds of this region
in virtual memory.

We are currently only memory-mapping physical device driver
MMIO regions so we do not need virtual-to-physical calculations
to re-map RAM yet. When the time comes an architecture interface
will be defined for this.

Platforms which just have one RAM region may continue to
identity-map it.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-11-09 20:19:13 -05:00
Yuguo Zou
3826eb302c arch: arc: add support of ARConnect inter-core debug unit
The Inter-core Debug Unit provides additional debug assist features in
multi-core scenarios.This commit allows ARConnect to conditionally
halt cores during debugging.

Signed-off-by: Yuguo Zou <yuguo.zou@synopsys.com>
2020-11-09 15:52:15 -06:00
Alexandre Mergnat
4b97619b19 riscv: add support for canaries
Kobject test area size must be increased if canary feature
is enabled.

Signed-off-by: Alexandre Mergnat <amergnat@baylibre.com>
2020-11-09 15:37:11 -05:00
Alexandre Mergnat
542a7fa25d arch: riscv: add memory protection support
The IRQ handler has had a major changes to manage syscall, reschedule
and interrupt from user thread and stack guard.

Add userspace support:
- Use a global variable to know if the current execution is user or
  machine. The location of this variable is read only for all user
  thread and read/write for kernel thread.
- Memory shared is supported.
- Use dynamic allocation to optimize PMP slot usage. If the area size
  is a power of 2, only one PMP slot is used, else 2 are used.

Add stack guard support:
- Use MPRV bit to force PMP rules to machine mode execution.
- IRQ stack have a locked stack guard to avoid re-write PMP
  configuration registers for each interruption and then win some
  cycle.
- The IRQ stack is used as "temporary" stack at the beginning of IRQ
  handler to save current ESF. That avoid to trigger write fault on
  thread stack during store ESF which that call IRQ handler to
  infinity.
- A stack guard is also setup for privileged stack of a user thread.

Thread:
- A PMP setup is specific to each thread. PMP setup are saved in each
  thread structure to improve reschedule performance.

Signed-off-by: Alexandre Mergnat <amergnat@baylibre.com>
Reviewed-by: Nicolas Royer <nroyer@baylibre.com>
2020-11-09 15:37:11 -05:00
Alexandre Mergnat
18962e4ab8 arch: riscv: add pmp support
- Set some helper function to write/clear/print PMP config registers.
- Add support for different PMP slot size function to core/board.

Signed-off-by: Alexandre Mergnat <amergnat@baylibre.com>
2020-11-09 15:37:11 -05:00
Alexandre Mergnat
6340dd7a59 arch: riscv: add e31 core support
Introducing core E31 family to link Zephyr features (userspace and
stack protection) to architecture capabilities (PMP).

Signed-off-by: Alexandre Mergnat <amergnat@baylibre.com>
2020-11-09 15:37:11 -05:00
Andrew Boie
d2a72273b7 x86: add support for common page tables
We provide an option for low-memory systems to use a single set
of page tables for all threads. This is only supported if
KPTI and SMP are disabled. This configuration saves a considerable
amount of RAM, especially if multiple memory domains are used,
at a cost of context switching overhead.

Some caching techniques are used to reduce the amount of context
switch updates; the page tables aren't updated if switching to
a supervisor thread, and the page table configuration of the last
user thread switched in is cached.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-11-05 09:33:40 -05:00
Andrew Boie
cd789a7ac7 x86: add logs for tuning page pool size
This will do until we can set up a proper page pool using
all unused ram for paging structures, heaps, and anonymous
mappings.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-11-05 09:33:40 -05:00
Andrew Boie
8e5114b370 x86: update X86_MMU_PAGE_POOL_PAGES documentation
Help users understand how this should be tuned. Rather than
guessing wildly, set the default to 0. This needs to be tuned
on a per-board, per-application basis anyway.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-11-05 09:33:40 -05:00
Andrew Boie
a15be58019 x86: move page table reservation macros
We don't need this for stacks any more and only use this
for pre-calculating the boot page tables size. Move to C
code, this doesn't need to be in headers anywhere.

Names adjusted for conciseness.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-11-05 09:33:40 -05:00
Andrew Boie
b8242bff64 x86: move page tables to the memory domain level
- z_x86_userspace_enter() for both 32-bit and 64-bit now
  call into C code to clear the stack buffer and set the
  US bits in the page tables for the memory range.

- Page tables are now associated with memory domains,
  instead of having separate page tables per thread.
  A spinlock protects write access to these page tables,
  and read/write access to the list of active page
  tables.

- arch_mem_domain_init() implemented, allocating and
  copying page tables from the boot page tables.

- struct arch_mem_domain defined for x86. It has
  a page table link and also a list node for iterating
  over them.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-11-05 09:33:40 -05:00
Andrew Boie
745dd6f931 x86: use unused PTE bits when mapping memory
Page table management for x86 is being revised such that there
will not in many cases be a pristine, master set of page tables.
Instead, when mapping memory, use unused PTE bits to store the
original RW, US, and XD settings when the mapping was made.

This will allow memory domains to alter page tables while still
being able to restore the original mapping permissions.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-11-05 09:33:40 -05:00
Andrew Boie
bd76bdb8ff x86: define some unused PTE bits
These are ignored by the CPU and may be used for OS
accounting.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-11-05 09:33:40 -05:00
Andrew Boie
d31f62a955 x86: smp: add TLB shootdown logic
This will be needed when we support memory un-mapping, or
the same user mode page tables on multiple CPUs. Neither
are implemented yet.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-11-05 09:33:40 -05:00
Carlo Caione
7b7c328f7a aarch64: mmu: Enable support for unprivileged EL0
The current MMU code is assuming that both kernel and threads are both
running in EL1, not supporting EL0. Extend the support to EL0 by adding
the missing attribute to mirror the access / execute permissions to EL0.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2020-11-04 13:58:19 -08:00
Carlo Caione
fd559f16a5 aarch64: mmu: Create new header file
The MMU coded is polluted by a lot of preprocessor code. Move that code
to a proper header file.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2020-11-04 13:58:19 -08:00
Carlo Caione
b9a407b65c aarch64: mmu: Move MMU files in a sub-directory
We are probably going to do more work on the MMU side and more files
will be added. Create a new sub-directory to host all the MMU related
files.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2020-11-04 13:58:19 -08:00
Yuguo Zou
d24c6e5aae arch: arc: use ifdef to replace if define in isr wrapper code
isr wrapper code has mixed usage of #ifdef and #if define macros. Unify
them to more usual #ifdef.

Signed-off-by: Yuguo Zou <yuguo.zou@synopsys.com>
2020-11-02 11:02:47 -06:00
Yuguo Zou
dbd431d2bc arch: arc: fix a reg misuse in leaving tickless idle
There is a register misuse in leaving tickless idle code, which would
destroy exception/interrupt status. This commit fix this issue.

Signed-off-by: Yuguo Zou <yuguo.zou@synopsys.com>
2020-11-02 11:02:47 -06:00
Ioannis Glaropoulos
47e87d8459 arch: arm: cortex_m: implement functionality for ARCH core regs init
Implement the functionality for configuring the
architecture core registers to their warm reset
values upon system initialization. We enable the
support of the feature in the Cortex-M architecture.

Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
2020-11-02 15:02:24 +01:00
Ioannis Glaropoulos
89658dad19 arch: arm: aarch32: cortex_m: improve documentation of z_arm_reset
We enhance the documentation of z_arm_reset, stressing that
the function may either be loaded by the processor coming
out of reset, or by another image, e.g. a bootloader. We
also specify what is required at minimum when executing the
reset function.

Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
2020-11-02 15:02:24 +01:00
Ioannis Glaropoulos
20a9848230 arch: introduce option to force internal architectural state init
We introduce an option that instructs Zephyr to perform
the initialization of internal architectural state (e.g.
ARCH-level HW registers and system control blocks) during
early boot to the reset values. The option is available
to the application developer but shall depend on whether
the architecture supports the functionality.

Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
2020-11-02 15:02:24 +01:00
Carlo Caione
b3ff89bd51 arch: arm64: Remove _BIT suffix
This is redundant and not coherent with the rest of the file. Thus
remove the _BIT suffix from the bit field names.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2020-11-02 12:04:35 +01:00
Carlo Caione
24c907d292 arch: arm64: Add missing vector table entries
The current vector table is missing some (not used) entries. Fill these
in for the sake of completeness.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2020-11-02 12:04:35 +01:00
Carlo Caione
f0b2e3d652 arch: arm64: Use mov_imm when possible in the start code
Instead of relying on mov.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2020-11-02 12:04:35 +01:00
Carlo Caione
673803dc48 arch: arm64: Rename z_arm64_svc to z_arm64_sync_exc
The SVC handler is not only used for the SVC call but in general for all
the synchronous exceptions. Reflect this in the handler name.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2020-11-02 12:04:35 +01:00
Carlo Caione
e738631ddf arch: arm64: Fix indentation
Fix indentation for the ISR wrapper.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2020-11-02 12:04:35 +01:00
Carlo Caione
78b5e5563d arch: arm64: Reword comments
Fix, reword and rework comments.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2020-11-02 12:04:35 +01:00
Carlo Caione
9e897ea2c3 arch: arm64: Remove unused macro parameters
Remove z_arm64_{enter,exit}_exc parameter leftovers.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2020-11-02 12:04:35 +01:00
Andrew Boie
edc5e31d6b x86_64: fix RBX clobber in nested IRQ case
In the code path for nested interrupts, we are not saving
RBX, yet the assembly code is using it as a storage location
for the ISR.

Use RAX. It is backed up in both the nested and non-nested
cases, and the ASM code is not currently using it at that
point.

Fixes: #29594

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-10-28 10:29:32 -07:00
Daniel Leung
f8a909dad1 xtensa: add support for thread local storage
Adds the necessary bits to initialize TLS in the stack
area and sets up CPU registers during context switch.

Note that this does not enable TLS for all Xtensa SoC.
This is because Xtensa SoCs are highly configurable
so that each SoC can be considered a whole architecture.
So TLS needs to be enabled on the SoC level, instead of
at the arch level.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-10-24 10:52:00 -07:00
Daniel Leung
8a79ce1428 riscv: add support for thread local storage
Adds the necessary bits to initialize TLS in the stack
area and sets up CPU registers during context switch.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-10-24 10:52:00 -07:00
Daniel Leung
388725870f arm: cortex_m: add support for thread local storage
Adds the necessary bits to initialize TLS in the stack
area and sets up CPU registers during context switch.

Note that since Cortex-M does not have the thread ID or
process ID register needed to store TLS pointer at runtime
for toolchain to access thread data, a global variable is
used instead.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-10-24 10:52:00 -07:00
Daniel Leung
778c996831 arm: cortex_r: add support for thread local storage
Adds the necessary bits to initialize TLS in the stack
area and sets up CPU registers during context switch.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-10-24 10:52:00 -07:00
Daniel Leung
df77e2af8b arm64: add support for thread local storage
Adds the necessary bits to initialize TLS in the stack
area and sets up CPU registers during context switch.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-10-24 10:52:00 -07:00
Daniel Leung
4b38392ded x86: add support for thread local storage
Adds the necessary bits to initialize TLS in the stack
area and sets up CPU registers during context switch.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-10-24 10:52:00 -07:00