This commit fixes incorrect Cortex-R interrupt lock, unlock and state
check function implementations.
The issues can be summarised as follows:
1. The current implementation of 'z_arch_irq_lock' returns the value
of CPSR as the IRQ key and, since CPSR contains many other state
bits, this caused 'z_arch_irq_unlocked' to return false even when
IRQ is unlocked. This problem is fixed by isolating only the I-bit
of CPSR and returning this value as the IRQ key, such that it
returns a non-zero value when interrupt is disabled.
2. The current implementation of 'z_arch_irq_unlock' directly updates
the value of CPSR control field with the IRQ key and this can cause
other state bits in CPSR to be corrupted. This problem is fixed by
conditionally enabling interrupt using CPSIE instruction when the
value of IRQ key is a zero.
3. The current implementation of 'z_arch_is_in_isr' checks the value
of CPSR MODE field and returns true if its value is IRQ or FIQ.
While this does not normally cause an issue, the function can return
false when IRQ offloading is used because the offload function
executes in SVC mode. This problem is fixed by adding check for SVC
mode.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
The callee-saved registers have been separated out and will not
be saved/restored if exception debugging is shut off.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
The context switch implementation forgot to save the current flag
state of the old thread, so on resume the flags would be restored to
whatever value they had at the last interrupt preemption or thread
initialization. In practice this guaranteed that the interrupt enable
bit would always be wrong, becuase obviously new threads and preempted
ones have interrupts enabled, while arch_switch() is always called
with them masked. This opened up a race between exit from
arch_switch() and the final exit path in z_swap().
The other state bits weren't relevant -- the oddball ones aren't used
by Zephyr, and as arch_switch() on this architecture is a function
call the compiler would have spilled the (caller-save) comparison
result flags anyway.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Use of the _current_cpu pointer cannot be done safely in a preemptible
context. If a thread is preempted and migrates to another CPU, the
old CPU record will be wrong.
Add a validation assert to the expression that catches incorrect
usages, and fix up the spots where it was wrong (most important being
a few uses of _current outside of locks, and the arch_is_in_isr()
implementation).
Note that the resulting _current expression now requires locking and
is going to be somewhat slower. Longer term it's going to be better
to augment the arch API to allow SMP architectures to implement a
faster "get current thread pointer" action than this default.
Note also that this change means that "_current" is no longer
expressible as an lvalue (long ago, it was just a static variable), so
the places where it gets assigned now assign to _current_cpu->current
instead.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
The existing stack_analyze APIs had some problems:
1. Not properly namespaced
2. Accepted the stack object as a parameter, yet the stack object
does not contain the necessary information to get the associated
buffer region, the thread object is needed for this
3. Caused a crash on certain platforms that do not allow inspection
of unused stack space for the currently running thread
4. No user mode access
5. Separately passed in thread name
We deprecate these functions and add a new API
k_thread_stack_space_get() which addresses all of these issues.
A helper API log_stack_usage() also added which resembles
STACK_ANALYZE() in functionality.
Fixes: #17852
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
This reverts commit 9987c2e2f9
which spills SoC configs into architecture files and is not
exactly desirable. So revert it.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
All SoCs must now 'select' one of the CONFIG_<arch> symbols. Add an
ARCH_IS_SET helper symbol that's selected by the arch symbols and
checked in CMake, printing a warning otherwise.
Might save people some time until they're used to the new scheme.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
All board defconfig files currently set the architecture in addition to
the board and the SoC, by setting e.g. CONFIG_ARM=y. This spams up
defconfig files.
CONFIG_<arch> symbols currently being set in configuration files also
means that they are configurable (can be changed in menuconfig and in
configuration files), even though changing the architecture won't work,
since other things get set from -DBOARD=<board>. Many boards also allow
changing the architecture symbols independently from the SoC symbols,
which doesn't make sense.
Get rid of all assignments to CONFIG_<arch> symbols and clean up the
relationships between symbols and the configuration interface, like
this:
1. Remove the choice with the CONFIG_<arch> symbols in arch/Kconfig and
turn the CONFIG_<arch> symbols into invisible
(promptless/nonconfigurable) symbols instead.
Getting rid of the choice allows the symbols to be 'select'ed (choice
symbols don't support 'select').
2. Select the right CONFIG_<arch> symbol from the SOC_SERIES_* symbols.
This makes sense since you know the architecture if you know the SoC.
Put the select on the SOC_* symbol instead for boards that don't have
a SOC_SERIES_*.
3. Remove all assignments to CONFIG_<arch> symbols. The assignments
would generate errors now, since the symbols are promptless.
The change was done by grepping for assignments to CONFIG_<arch>
symbols, finding the SOC_SERIES_* (or SOC_*) symbol being set in the
same defconfig file, and putting a 'select' on it instead.
See
https://github.com/ulfalizer/zephyr/commits/hide-arch-syms-unsquashed
for a split-up version of this commit, which will make it easier to see
how stuff was done. This needs to go in as one commit though.
This change is safer than it might seem re. outstanding PRs, because any
assignment to CONFIG_<arch> symbols generates an error now, making
outdated stuff easy to catch.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
CUSTOM_SECTION_ALIGN is already defined within an 'if ARM_MPU', so it
does not need a 'depends on ARM_MPU'.
Flagged by https://github.com/zephyrproject-rtos/ci-tools/pull/128.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Add TRACING_ISR Kconfig to help high latency backend working well.
Currently the ISR tracing hook function is put at the begining and
ending of ISR wrapper, when there is ISR needed in the tracing path
(especially tracing backend), it will cause tracing buffer easily
be exhausted if async tracing method enabled. Also it will increase
system latency if all the ISRs are traced. So add TRACING_ISR to
enable/disable ISR tracing here. Later a filter out mechanism based
on irq number will be added.
Signed-off-by: Wentong Wu <wentong.wu@intel.com>
if USERSPACE is configured, it needs to record the user/kernel mode
of interrupted thread, because the switch of aux_sec_k_sp/aux_user_sp
depends on the aux_irq_act's U bit.
Signed-off-by: Wayne Ren <wei.ren@synopsys.com>
Use BOOTLOADER definition to separate bootloader code. This allows to
use the same file reset-vector.S when building bootloader and when
CONFIG_XTENSA_RESET_VECTOR is enabled.
Signed-off-by: Andrei Emeltchenko <andrei.emeltchenko@intel.com>
There was a bug where double-dispatch of a single thread on multiple
SMP CPUs was possible. This can be mind-bending to diagnose, so when
CONFIG_ASSERT is enabled add an extra instruction to __resume (the
shared code path for both interupt return and context switch) that
poisons the shared RIP of the now-running thread with a recognizable
invalid value.
Now attempts to run the thread again will crash instantly with a
discoverable cookie in their instruction pointer, and this will remain
true until it gets a new RIP at the next interrupt or switch.
This is under CONFIG_ASSERT because it meets the same design goals of
"a cheap test for impossible situations", not because it's part of the
assertion framework.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Enable the shared IRQ for the UART line and enable the remaining tasks
that depends on a separated declaration of the TX/RX/Err/... IRQs.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
The cmsis_rtos tests are failing because the stack size used by CMSIS is
too small. Customize the stack size for the aarch64 architecture and
re-enable the tests.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
ARMv8-A SoCs enter EL3 after reset. Add a new config option
(CONFIG_SWITCH_TO_EL1) to switch from EL3 to EL1 at boot and default it
to 'y'.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
While QEMU's Cortex-A53 emulation by default only emulates a CPU in EL1,
other QEMU forks (for example the QEMU released by Xilinx) and real
hardware starts in EL3.
To support all the ELn we introduce a macro to identify at run-time the
Exception Level and take the correct actions.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
To be able to pass the unit test we need to add a set of defines for the
ARM64 architecture. Fix this.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
To be able to successfully compile the kernel for the ARM64 architecture
we have to tweak the compiler-related files to be able to use the
AArch64 GCC compiler.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Introduce the basic ARM64 architecture support.
A new CONFIG_ARM64 symbol is introduced for the new architecture and new
cmake / Kconfig files are added to switch between ARM and ARM64.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Dynamic MPU regions are used in build configurations with User
mode or MPU-based stack-overflow guards. If these features are
disabled, we skip calling the ARM function for re-programming
the MPU peripheral during context-switch. We also skip doing
this when jumping to main thread (although this brings limited
performace gain as it is called once in the boot cycle)
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
Users are reportedly not able to understand how to debug the following
error message from gen_isr_tables:
gen_isr_tables.py: multiple registrations at table_index 8 for irq 8
Debugging issues these kinds of issues is difficult so we need to give
users as much information as possible.
To make it clearer that it could be an abuse of the 'IRQ_CONNECT' API
that is causing the issue we add this to the error message.
Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
In zephyr_linker_sources().
This is done since the point of the location is to place things at given
offsets. This can only be done consistenly if the linker code is placed
into the _first_ section.
All uses of TEXT_START are replaced with ROM_START.
ROM_START is only supported in some arches, as some arches have several
custom sections before text. These don't currently have ROM_START or
TEXT_START available, but that could be added with a bit of refactoring
in their linker script.
No SORT_KEYs are changed.
This also fixes an error introduced when TEXT_START was added, where
TEXT_SECTION_OFFSET was applied to riscv's common linker.ld instead of
to openisa_rv32m1's specific linker.ld.
Signed-off-by: Øyvind Rønningstad <oyvind.ronningstad@nordicsemi.no>
openocd linker sections are not supposed to be part of the
vector table sections. Place the sections after we define
the _vector_end linker symbol.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
The original intent was that the output handle be written through the
pointer in the second argument, though not all architectures used that
scheme. As it turns out, that write is becoming a synchronization
signal, so it's no longer optional.
Clarify the documentation in arch_switch() about this requirement, and
add an instruction to the x86_64 context switch to implement it as
original envisioned.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Define there options for runtime error handling:
- assert on all errors (ASSERT_ON_ERRORS)
- no runtime checks (no asserts, no runtime error handling)
(NO_RUNTIME_CHECKS)
- full runtime error handling (the default) (RUNTIME_ERROR_CHECKS)
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
This commit adds a Kconfig symbol for specifying whether the SoC
implements the CPU DWT feature.
The Data Watchpoint and Trace (DWT) is an optional debug unit for the
Cortex-M family cores (except ARMv6-M; i.e. M0 and M0+) that provides
watchpoints, data tracing and system profiling capabilities.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
Implement a set of per-cpu trampoline stacks which all
interrupts and exceptions will initially land on, and also
as an intermediate stack for privilege changes as we need
some stack space to swap page tables.
Set up the special trampoline page which contains all the
trampoline stacks, TSS, and GDT. This page needs to be
present in the user page tables or interrupts don't work.
CPU exceptions, with KPTI turned on, are treated as interrupts
and not traps so that we have IRQs locked on exception entry.
Add some additional macros for defining IDT entries.
Add special handling of locore text/rodata sections when
creating user mode page tables on x86-64.
Restore qemu_x86_64 to use KPTI, and remove restrictions on
enabling user mode on x86-64.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
generated_dts_board.h is pretty redundant and confusing as a name. Call
it devicetree.h instead.
dts.h would be another option, but DTS stands for "devicetree source"
and is the source code format, so it's a bit confusing too.
The replacement was done by grepping for 'generated_dts_board' and
'GENERATED_DTS_BOARD'.
Two build diagram and input-output SVG files were updated as well, along
with misc. documentation.
hal_ti, mcuboot, and ci-tools updates are included too, in the west.yml
update.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
KPTI is still work-in-progress on x86_64. Don't allow
user mode to be enabled unless the SOC/board configuration
indicates that the CPU in use is invulnerable to meltdown
attacks.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
See CVE-2019-1125. We mitigate this by adding an 'lfence'
upon interrupt/exception entry after the decision has been
made whether it's necessary to invoke 'swapgs' or not.
Only applies to x86_64, 32-bit doesn't use swapgs.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
- In early boot, enable the syscall instruction and set up
necessary MSRs
- Add a hook to update page tables on context switch
- Properly initialize thread based on whether it will
start in user or supervisor mode
- Add landing function for system calls to execute the
desired handler
- Implement arch_user_string_nlen()
- Implement logic for dropping a thread down to user mode
- Reserve per-CPU storage space for user and privilege
elevation stack pointers, necessary for handling syscalls
when no free registers are available
- Proper handling of gs register considerations when
transitioning privilege levels
Kernel page table isolation (KPTI) is not yet implemented.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
This code:
1) Doesn't work
2) Hasn't ever been enabled by default
3) We mitigate Spectre V2 via Extended IBRS anyway
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
We use a fixed value of 32 as the way interrupts/exceptions
are setup in x86_64's locore.S do not lend themselves to
Kconfig configuration of the vector to use.
HW-based kernel oops is now permanently on, there's no reason
to make it optional that I can see.
Default vectors for IPI and irq offload adjusted to not
collide.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
This is causing problems, as if we create a thread in
a system call we will *not* be using the kernel page
tables if CONFIG_KPTI=n.
Just don't fiddle with this page's permissions; we don't
need it as a guard area anyway since we have a stack
guard placed immediately before it, and this page
is unused if user mode isn't active.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Nothing too fancy here, we try as much as possible to
use the same register layout as the C calling convention.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
These are now common code, all are related to user mode
threads. The rat's nest of ifdefs in ia32's arch_new_thread
has been greatly simplified, there is now just one hook
if user mode is turned on.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
