Commit graph

3814 commits

Author SHA1 Message Date
Andrew Boie
e3ab43580c x86: move mmustructs.h
This will be used for both 32-bit and 64-bit mode.
This header gets pulled in by x86's arch/cpu.h, so put
it in include/arch/x86/.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-10-14 11:49:39 -07:00
Ioannis Glaropoulos
a6accb3e75 arch: arm: swap-helper: simplify several assembly expressions
Some assembly simplifications, to make code common for ARMv6
and ARMv7 architecture.

We can use ldrb, directly for reading the SVC encoding; this
removes the need for ANDing the result with 0xff right below.
We remove an immediate value of 0 from an str instruction, as
it's redundant.

Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
2019-10-14 09:25:32 -07:00
Ioannis Glaropoulos
9ea2df1788 arch: arm: documentation improvements in ARM assembly sources
Add more documentation and inline explanatory comments in
assembly sources swap_helper.S and userspace.S and remove
redundant/wrong documentation when applicable.

Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
2019-10-14 09:25:32 -07:00
Ioannis Glaropoulos
237033c61b arch: arm: userspace: z_arm_userspace_enter: reduce push/pop overhead
ARM user space requires ARM_MPU. We can, therefore,
remove the unnecessary #ifdef CONFIG_ARM_MPU blocks
in userspace.S. In addition, we do minor refactoring
in z_arm_userspace_enter(), and z_arm_pendsv(), and
z_arm_svc(), aiming at reducing the push/pop overhead
as much as possible.

Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
2019-10-14 09:25:32 -07:00
Andrew Boie
8ffff144ea kernel: add architecture interface headers
include/sys/arch_inlines.h will contain all architecture APIs
that are used by public inline functions and macros,
with implementations deriving from include/arch/cpu.h.

kernel/include/arch_interface.h will contain everything
else, with implementations deriving from
arch/*/include/kernel_arch_func.h.

Instances of duplicate documentation for these APIs have been
removed; implementation details have been left in place.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-10-11 13:30:46 -07:00
Andrew Boie
e340f8d22e x86: intel64: enable no-execute
Set the NXE bit in the EFER MSR so that the NX bit can
be set in page tables. Otherwise, the NX bit is treated
as reserved and leads to a fault if set.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-10-10 13:41:02 -07:00
Ioannis Glaropoulos
61c60be302 arch: arm: core: add Cortex-R in the files description headers
arch/arm/core is shared between Cortex-M and Cortex-R, so
enhance the file description headers accordingly.

Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
2019-10-09 17:54:16 -05:00
Ioannis Glaropoulos
463d8f7e86 arch: arm: clean up inclusions in assembly files
A clean-up commit that removes unnecessary inclusions from
assembly files in arm/core and arm/core/cortex_m. It also
ogranizes the inclusions based on the following order and
set of rules:
- never include kernel_structs.h
- include toolchain.h and linker/sections.h in all ASM files
- include offsets-short.h, if ASM accesses offset constants
- include arch/cpu.h, if ASM accesses CMSIS constants
  (defined locally in include/arch/arm)
- include file-specific headers, if needed (e.g. vector-table.h)

Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
2019-10-09 17:54:16 -05:00
Wayne Ren
eb2422ed76 arch: arc: enlarge the exception handling stack
after recent changes in zephyr's fault handling, e.g. use log
to repace printk, it requires more stack to exception handling, or
the stack overflow may happen and crash the system.

this commit adds a kconfig option for exception stack size with
a larger default size.

Signed-off-by: Wayne Ren <wei.ren@synopsys.com>
2019-10-09 14:21:56 -04:00
Alberto Escolar Piedras
b237597ab9 arch: posix: isolate arch-soc/board IF from kernel-arch IF
The POSIX ARCH delegates some of the tasks which normally
are taken care of by the ARCH to the SOC or BOARD levels.

To avoid changes in the kernel-arch IF propagating into
the arch-soc and arch-board interfaces (which would break
off-tree posix boards) isolate them.

Also move arch inlined functions into the arch.h header,
and out from the headers which specify the  posix arch-soc
and arch-board interfaces.

Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
2019-10-09 09:14:18 -04:00
Andrew Boie
d1aca7f11b x86_64: fix arch headers
arch/cpu.h and kernel_arch_func.h are expected to define different
functions, per the architecture interface.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-10-09 09:14:18 -04:00
Andrew Boie
7f715bd113 intel64: add inline definition of z_arch_switch()
This just calls the assembly code. Needed to be
inline per the architecture API.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-10-09 09:14:18 -04:00
Andrew Boie
c968422a78 xtensa: fix z_arch_switch()
This was in the wrong header and declared as a macro instead
of an inline function.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-10-09 09:14:18 -04:00
Andrew Boie
747fec226c arm: move z_arch_switch_to_main_thread to C code
There's no compelling reason why this should be inline unlike all
other arches, it's a large function, called exactly once.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-10-09 09:14:18 -04:00
Andrew Boie
54506b5a9b arches: fix z_arch_is_in_isr() defintion
For some reason these were implemented as macros when they
should be inline functions.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-10-09 09:14:18 -04:00
Andrew Boie
f0f700c121 posix: expose z_arch_irq_lock/unlock as inlines
The specification for these arch APIs is to have them inline,
and the bodies were just oneliners calling another function
anyway.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-10-09 09:14:18 -04:00
Stephanos Ioannidis
fe85c2e2e0 arch: arm: Add Cortex-R exception handling documentation.
Add in-line documentation describing the process of register
preservation and exception handling on Cortex-R.

Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
2019-10-08 16:03:32 -05:00
Stephanos Ioannidis
8c4de7e4b0 arch: arm: Remove unnecessary register preservation in Cortex-R port.
The interrupt exit and swap service routines for Cortex-R
unnecessarily preserve r0 and lr registers when making function calls
using bl instruction.

In case of _IntExit in exc_exit.S, the r0 register containing the
caller mode is preserved at the top, and the lr register can safely be
assumed to have been saved into the system mode stack by the interrupt
service routine.

In case of __svc in swap_helper.S, since the function saves lr to the
system mode stack at the top and exits through _IntExit, it is not
necessary to preserve lr register when executing bl instructions.

Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
2019-10-08 16:03:32 -05:00
Charles E. Youse
f7cfb4303b arch/x86: do not assume MP means SMP
It's possible to have multiple processors configured without using the
SMP scheduler, so don't make definitions dependent on CONFIG_SMP.

Signed-off-by: Charles E. Youse <charles.youse@intel.com>
2019-10-07 19:46:55 -04:00
Charles E. Youse
e6a31a9e89 arch/x86: (Intel64) initialize TSS interrupt stack from cpuboot[]
In non-SMP MP situations, the interrupt stacks might not exist, so
do not assume they do. Instead, initialize the TSS IST1 from the
cpuboot[] vector (meaning, on APs, the stack from z_arch_start_cpu).
Eliminates redundancy at the same time.

Signed-off-by: Charles E. Youse <charles.youse@intel.com>
2019-10-07 19:46:55 -04:00
Charles E. Youse
643661cb44 arch/x86: declare z_x86_prep_c() in kernel_arch_func.h
And remove the ad hoc prototype in cpu.c for Intel64.

Signed-off-by: Charles E. Youse <charles.youse@intel.com>
2019-10-07 19:46:55 -04:00
Charles E. Youse
5abab591c2 arch/x86: (Intel64) make z_arch_start_cpu() synchronous
Don't leave z_arch_start_cpu() until the target CPU has been started.

Signed-off-by: Charles E. Youse <charles.youse@intel.com>
2019-10-07 19:46:55 -04:00
Charles E. Youse
3b145c0d4b arch/x86: (Intel64) do not lock interrupts around irq_offload()
This is the Wrong Thing(tm) with SMP enabled. Previously this
worked because interrupts would be re-enabled in the interrupt
entry sequence, but this is no longer the case.

Signed-off-by: Charles E. Youse <charles.youse@intel.com>
2019-10-07 19:46:55 -04:00
Charles E. Youse
66510db98c arch/x86: (Intel64) add scheduler IPI support
Add z_arch_sched_ipi() and such to enable scheduler IPIs when SMP.

Signed-off-by: Charles E. Youse <charles.youse@intel.com>
2019-10-07 19:46:55 -04:00
Charles E. Youse
74e3717af6 arch/x86: (Intel64) fix conditional assembly in locore.S
was ignoring the rest of the expression, though the effect was
harmless (including unreachable code in some builds).

Signed-off-by: Charles E. Youse <charles.youse@intel.com>
2019-10-07 19:46:55 -04:00
Charles E. Youse
f361798cdf arch/x86: limit number of IRQ vectors to 224
Trivial change to the Kconfig: the first 32 vectors are reserved,
so it's not possible to have 256 IRQ vectors. Change max to 224.

Signed-off-by: Charles E. Youse <charles.youse@intel.com>
2019-10-07 19:46:55 -04:00
Charles E. Youse
3eb1a8b59a arch/x86: (Intel64) implement SMP support
Add duplicate per-CPU data structures (x86_cpuboot, tss, stacks, etc.)
for up to 4 total CPUs, add code in locore and z_arch_start_cpu().

The test board, qemu_x86_long, now defaults to 2 CPUs.

Signed-off-by: Charles E. Youse <charles.youse@intel.com>
2019-10-07 19:46:55 -04:00
Charles E. Youse
2808908816 arch/x86: alter signature of z_x86_prep_c() function
Take a dummy first argument, so that the BSP entry point (z_x86_prep_c)
has the same signature as the AP entry point (smp_init_top).

Signed-off-by: Charles E. Youse <charles.youse@intel.com>
2019-10-07 19:46:55 -04:00
Charles E. Youse
f9eaee35b8 arch/x86: (Intel64) use per-CPU parameter struct for CPU startup
A new 'struct x86_cpuboot' is created as well as an instance called
'x86_cpuboot[]' which contains per-CPU boot data (initial stack,
entry function/arg, selectors, etc.). The locore now consults this
table to set up per-CPU registers, etc. during early boot.

Also, rename tss.c to cpu.c as its scope is growing.

Signed-off-by: Charles E. Youse <charles.youse@intel.com>
2019-10-07 19:46:55 -04:00
Charles E. Youse
edf5761c83 arch/x86: (Intel64) rename kernel segment constants
There's no need to qualify the 64-bit CS/DS selectors, and the GS and
TR selectors are renamed CPU0_GS and CPU0_TR as they are CPU-specific.

Signed-off-by: Charles E. Youse <charles.youse@intel.com>
2019-10-07 19:46:55 -04:00
Charles E. Youse
90bf0da332 arch/x86: (Intel64) optimize and re-order startup assembly sequence
In some places the code was being overly pedantic; e.g., there is no
need to load our own 32-bit descriptors because the loader's are fine
for our purposes. We can defer loading our own segments until 64-bit.

The sequence is re-ordered to faciliate code sharing between the BSP
and APs when SMP is enabled (all BSP-specific operations occur before
the per-CPU initialization).

Signed-off-by: Charles E. Youse <charles.youse@intel.com>
2019-10-07 19:46:55 -04:00
Charles E. Youse
17e135bc41 arch/x86: (Intel64) clear BSS before entering long mode
This is really just to facilitate CPU bootstrap code between
the BSP and the APs, moving the clear operation out of the way.

Signed-off-by: Charles E. Youse <charles.youse@intel.com>
2019-10-07 19:46:55 -04:00
Charles E. Youse
a981f51fe6 arch/x86: drivers/loapic_intr.c: move local APIC initialization
In the general case, the local APIC can't be treated as a normal device
with a single boot-time initialization - on SMP systems, each CPU must
initialize its own. Hence the initialization proper is separated from
the device-driver initialization, and said initialization is called
from the early startup-assembly code when appropriate.

Signed-off-by: Charles E. Youse <charles.youse@intel.com>
2019-10-07 19:46:55 -04:00
Charles E. Youse
418e5c1b38 arch/x86: factor out common assembly startup code
The 32-bit and 64-bit assembly startup sequences share quite a
bunch of common code, so it's factored out into one file to avoid
repeating ourselves (and potentially falling out of sync).

Signed-off-by: Charles E. Youse <charles.youse@intel.com>
2019-10-07 19:46:55 -04:00
Charles E. Youse
25a7cc1136 arch/x86: (Intel64) add missing linker symbols
The linker script was missing symbols that defined the boundaries
of kernel memory segments (_image_rom_end, etc.). These are added
so that core/memmap.c can properly account for those segments.

Signed-off-by: Charles E. Youse <charles.youse@intel.com>
2019-10-07 19:46:55 -04:00
Charles E. Youse
8279af76c8 arch/x86: elevate prep_c.c to common code
Elevate the previously 32-bit-only z_x86_prep_c() function to common
code, so both 32-bit and 64-bit arches now enter the kernel this way.
Minor changes to prep_c.c to make it build with the SMP scheduler on.

Signed-off-by: Charles E. Youse <charles.youse@intel.com>
2019-10-07 19:46:55 -04:00
Charles E. Youse
8d97750eef arch/x86: (Intel64) add z_arch_curr_cpu() to enable CONFIG_SMP=y
And set qemu_x86_long board to build with CONFIG_SMP=y by default.
Apparently two benchmark tests - latency_measure and sys_kernel -
do not work with the SMP scheduler, so those tests are disabled.

Signed-off-by: Charles E. Youse <charles.youse@intel.com>
2019-10-07 19:46:55 -04:00
Charles E. Youse
cc9be2e982 arch/x86: (Intel64) start up on _interrupt_stack, not _exception_stack
Simply for consistency with other platforms.

Signed-off-by: Charles E. Youse <charles.youse@intel.com>
2019-10-07 19:46:55 -04:00
Jan Van Winkel
23a866b828 cmake: toolchain abstraction for undefined behaviour sanitizer
Added toolchain abstraction for undefined behaviour sanitizer

Signed-off-by: Jan Van Winkel <jan.van_winkel@dxplore.eu>
2019-10-07 15:00:20 +02:00
Andrew Boie
f0ddbd7eee x86: abstract toplevel page table pointer
This patch is a preparatory step in enabling the MMU in
long mode; no steps are taken to implement long mode support.

We introduce struct x86_page_tables, which represents the
top-level data structure for page tables:

- For 32-bit, this will contain a four-entry page directory
  pointer table (PDPT)
- For 64-bit, this will (eventually) contain a page map level 4
  table (PML4)

In either case, this pointer value is what gets programmed into
CR3 to activate a set of page tables. There are extra bits in
CR3 to set for long mode, we'll get around to that later.

This abstraction will allow us to use the same APIs that work
with page tables in either mode, rather than hard-coding that
the top level data structure is a PDPT.

z_x86_mmu_validate() has been re-written to make it easier to
add another level of paging for long mode, to support 2MB
PDPT entries, and correctly validate regions which span PDPTE
entries.

Some MMU-related APIs moved out of 32-bit x86's arch.h into
mmustructs.h.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-10-04 15:53:49 -07:00
Jan Van Winkel
89e97d13ec cmake: toolchain abstraction for address sanitizer
Added toolchain abstraction for address sanitizer

Signed-off-by: Jan Van Winkel <jan.van_winkel@dxplore.eu>
2019-10-04 13:39:52 -04:00
Andrew Boie
8c98a97581 arm: arch code naming cleanup
This patch re-namespaces global variables and functions
that are used only within the arch/arm/ code to be
prefixed with z_arm_.

Some instances of CamelCase have been corrected.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-10-04 10:46:23 +02:00
Andrew Boie
13433fcc4b x86: fix EXCEPTION_DEBUG dependency
This requires logging, not printk.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-10-01 16:15:55 -05:00
Wayne Ren
56650aff7e arch: arc: fix the bug in prologue of sys call handling
* the old codes may not save the caller saved regs correctly,
  e.g. r7- r12. Because the sys call entry is called in the form
  of static inline function. The compiler optimizations may not save
  all the caller saved regs.

* new codes use the irq stack frame as the sys call frame and gurantee
  all the called saved regs are pushed and popped correctly.

* the side effect of new codes are more stack operations and a little
  overhead.

Signed-off-by: Wayne Ren <wei.ren@synopsys.com>
2019-10-01 09:22:30 -04:00
Andrew Boie
89d4c6928e kernel: add arch abstraction for irq_offload()
This makes it clearer that this is an API that is expected
to be implemented at the architecture level.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-10-01 11:11:42 +02:00
Andrew Boie
bd6d8dc070 arc: rename k_cpu_sleep_mode
This is only used internally by the ARC arch code
and has been renamed to z_arc_cpu_sleep_mode.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-09-30 15:25:55 -04:00
Andrew Boie
2c1fb971e0 kernel: rename __swap
This is part of the core kernel -> architecture API and
has been renamed to z_arch_swap().

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-09-30 15:25:55 -04:00
Andrew Boie
fe031611fd kernel: rename main/idle thread/stacks
The main and idle threads, and their associated stacks,
were being referenced in various parts of the kernel
with no central definition. Expose these in kernel_internal.h
and namespace with z_ appropriately.

The main and idle threads were being defined statically,
with another variable exposed to contain their pointer
value. This wastes a bit of memory and isn't accessible
to user threads anyway, just expose the actual thread
objects.

Redundance MAIN_STACK_SIZE and IDLE_STACK_SIZE defines
in init.c removed, just use the Kconfigs they derive
from.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-09-30 15:25:55 -04:00
Andrew Boie
fa22ecffec x86: remove redunant idle timestamp setting
This metric shows when the system first enters an idle
state, which has already been recorded in the arch-
independent implementation of the idle thread.

Only x86 was doing this.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-09-30 15:25:55 -04:00
Andrew Boie
f6fb634b89 kernel: rename kernel_arch_init()
This is part of the core kernel -> architecture interface and
has been renamed z_arch_kernel_init().

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-09-30 15:25:55 -04:00