Modify #ifdefs so that any code that is compiled if CONFIG_ARMV7_R is
set is also compiled if CONFIG_ARMV7_A is set.
Modify #ifdefs so that any code that is compiled if CONFIG_CPU_CORTEX_R
is set is also compiled if CONFIG_CPU_AARCH32_CORTEX_A is set.
Modify source dir inclusion in CMakeLists.txt accordingly.
Brief file descriptions have been updated to include Cortex-A whereever
only Cortex-M and Cortex-R were mentioned so far.
Signed-off-by: Immo Birnbaum <Immo.Birnbaum@weidmueller.com>
An initial implementation for memory management using the ARMv7 MMU.
A single L1 translation table for the whole 4 GB address space is al-
ways present, a configurable number of L2 page tables are linked to
the L1 table based on the static memory area configuration at boot
time, or whenever arch_mem_map/arch_mem_unmap are called at run-time.
Currently, a CPU with the Multiprocessor Extensions and execution at
PL1 are always assumed. Userspace-related features or thread stack
guard pages are not yet supported. Neither are LPAE, PXN or TEX re-
mapping. All mappings are currently assigned to the same domain. Re-
garding the permissions model, access permissions are specified using
the AP[2:1] model rather than the older AP[2:0] model, which, accor-
ding to ARM's documentation, is deprecated and should no longer be
used. The newer model adds some complexity when it comes to mapping
pages as unaccessible (the AP[2:1] model doesn't support explicit
specification of "no R, no W" permissions, it's always at least "RO"),
this is accomplished by invalidating the ID bits of the respective
page's PTE.
Includes sources, Kconfig integration, adjusted CMakeLists and the
modified linker command file (proper section alignment!).
Signed-off-by: Immo Birnbaum <Immo.Birnbaum@weidmueller.com>
The configuration bits ATCMPCEN, B0TCMPCEN and B1TCMPCEN in the ACTLR
register referenced in the function z_arm_tcm_disable_ecc are only de-
fined for Cortex-R CPUs. For Cortex-A CPUs, those bits are declared
as reserved.
Comp.: https://arm-software.github.io/CMSIS_5/Core_A/html/group__CMSIS__ACTLR.html
Signed-off-by: Immo Birnbaum <Immo.Birnbaum@weidmueller.com>
There are two macros for declaring stack arrays:
K_KERNEL_STACK_ARRAY_DEFINE:
Defines the array, allocating storage and setting the section name
K_KERNEL_STACK_ARRAY_EXTERN
Declares the name of a stack array allowing code to reference
the array which must be defined elsewhere
arch/arm/include/aarch32/cortex_m/stack.h was mis-using
K_KERNEL_STACK_ARRAY_DEFINE to declare z_interrupt_stacks by sticking
'extern' in front of the macro use. However, when this macro also set
the object file section for the symbol, having two of those caused a
conflict in the compiler due to the automatic unique name mechanism used
for sections to allow unused symbols to be discarded during linking.
This patch makes the header use the correct macro.
Signed-off-by: Keith Packard <keithp@keithp.com>
The assert log of z_priv_stacks_ram_start failed to build due to passing
&z_priv_stacks_ram_start instead of just z_priv_stacks_ram_start.
Fixes#39190
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
This commit adds the half-precision (16-bit) floating-point
configurations to the ARM AArch32 architectures.
Enabling CONFIG_FP16 has the effect of specifying `-mfp16-format`
option (in case of GCC) which allows using the half-precision floating
point types such as `__fp16` and `_Float16`.
Note that this configuration can be used regardless of whether a
hardware FPU is available or supports half-precision operations.
When an FP16-capable FPU is not available, the compiler will
automatically provide the software emulations.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
ld linker will only resolve undefined symbols inside functions that is
actually being called.
However, not all linkers behaves this way. Certain linkers, for example
armlink, resolves all undefined symbols even if during a later stage at
the linking the function will be pruned.
Therefore `ifdef CONFIG_GEN_ISR_TABLES` has been placed to safeguard
functions that will call undefined symbols when CONFIG_GEN_ISR_TABLES=y.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
z_arm_do_syscall is only defined and used when CONFIG_USERSPACE=y.
Defining the symbol z_arm_do_syscall in assembly without a corresponding
implementation is fine for GNU ld as long as the function is not
actively called, but armlink fails to link in such cases.
Safegaurd GTEXT(z_arm_do_syscall) so the symbol is only referenced when
actively used, that is when CONFIG_USERSPACE=y.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Cleanup and preparation commit for linker script generator.
Zephyr linker scripts provides start and end symbols for each section,
and sometimes even size and LMA start symbols.
Generally, start and end symbols uses the following pattern, as:
Section name: foo
Section start symbol: __foo_start
Section end symbol: __foo_end
However, this pattern is not followed consistently.
To allow for linker script generation and ensure consistent naming of
symbols then the following pattern is introduced consistently to allow
for cleaner linker script generation.
Section name: foo
Section start symbol: __foo_start
Section end symbol: __foo_end
Section size symbol: __foo_size
Section LMA start symbol: __foo_load_start
This commit aligns the symbols for _ramfunc_ram/rom to other symbols and
in such a way they follow consistent pattern which allows for linker
script and scatter file generation.
The symbols are named according to the section name they describe.
Section name is `ramfunc`
The following symbols are aligned in this commit:
- _ramfunc_ram_start -> __ramfunc_start
- _ramfunc_ram_end -> __ramfunc_end
- _ramfunc_ram_size -> __ramfunc_size
- _ramfunc_rom_start -> __ramfunc_load_start
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Related to github #22290. Getting interrupt during mpu buffer validate
is corrupting index register. Fix applied to ARC is to disable
interrupts during the buffer validate operation.
Signed-off-by: Phil Erwin <phil.erwin@lexmark.com>
Cortex-A/R does not have hardware supported nested interrupts, but it is
easily emulatable using the nesting level stored in the kernel
structure.
Signed-off-by: Bradley Bolen <bbolen@lexmark.com>
Add functionality based on Cortex-M that enables recovery from a data
abort using zephyr's exception recovery framework. If there is a
registered z_exc_handle for a function, then use its fixup address if
that function aborts.
Signed-off-by: Bradley Bolen <bbolen@lexmark.com>
With the addition of userspace support, Cortex-R needs to use SVC calls
to handle oops exceptions. Add that support by defining ARCH_EXCEPT to
do a svc call.
Signed-off-by: Bradley Bolen <bbolen@lexmark.com>
The user thread cannot be trusted so do not use the stack pointer it
passes in. Use the thread's privilege stack when in privileged modes to
make sure a user thread does not trick the svc/isr handlers into writing
to memory it should not.
Signed-off-by: Bradley Bolen <bbolen@lexmark.com>
This commit adds the ARMv8.1-M M-Profile Vector Extension (MVE)
configurations as well as the compiler flags to enable it.
The M-Profile Vector Extension consists of the MVE-I and MVE-F
instruction sets which are integer and floating-point vector
instruction sets, respectively.
The MVE-I instruction set is a superset of the ARM DSP instruction
set (ARMv7E-M) and therefore depends on ARMV8_M_DSP, and the MVE-F
instruction set is a superset of the ARM MVE-I instruction set and
therefore depends on ARMV8_1_M_MVEI.
The SoCs that implement the MVE instruction set should select the
following configurations:
select ARMV8_M_DSP
select ARMV8_1_M_MVEI
select ARMV8_1_M_MVEF (if floating-point MVE is supported)
The GCC compiler flags for the MVE instruction set are specified
through the `-mcpu` flag.
In case of the Cortex-M55 (the only supported processor type for
ARMv8.1-M at the time of writing), the `-mcpu=cortex-m55` flag, by
default, enables all the supported extensions which are DSP, MVE-I and
MVE-F.
The extensions that are not supported can be specified by appending
`+no(ext)` to the `-mcpu=cortex-m55` flag:
-mcpu=cortex-m55 Cortex-M55 with DSP + MVE-I + MVE-F
-mcpu=cortex-m55+nomve.fp Cortex-M55 with DSP + MVE-I
-mcpu=cortex-m55+nomve Cortex-M55 with DSP
-mcpu=cortex-m55+nodsp Cortex-M55 without any extensions
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
The TLS global pointer is only set during context switch.
So for the first switch to main thread, the TLS pointer
is NULL which would cause access violation when trying
to access any thread local variables in main thread.
Fix it by setting it before going into main thread.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Cleanup an #ifdef statement in swap_helper.S; use
ARMV6_M_ARMV8_M_BASELINE instead of listing all
Cortex-M baseline implementation variants. This
fixes an issue with Cortex-M23 whose Kconfig
define was not included in the original list.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
When inside an escalated HardFault, we would like to get
more information about the reason for this escalation. We
first check if the reason for thise escalation is an SVC,
which occurs within a priority level that does not allow
it to trigger (e.g. fault or another SVC). If this is true
we set the error reason according to the provided argument.
Only when this is not a synchronous SVC that caused the HF,
do we check the other reasons for HF escalation (e.g. a BF
inside a previous BF).
We also add a case for a debug event, to complete going through
the available flags in HFSR.
Finally we ASSERT if we cannot find the reason for the escalation.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
Move the assessment of a synchronous SVC error into a
separate function. This commit does not introduce any
behavioral changes.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
Clean up a few more hard-coded constants
in swap_helper.S and replace them with
CMSIS-like defines in cpu.h. No behavioral
changes in this commit.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
When locking interrupt in a critical session, it is
safer to do MSR BASEPRI_MAX instead of BASEPRI. The
rationale is that when writing to BASEPRI_MAX, the
writing is conditional, and is only applied if the
change is to a higher priority level. This commit
replaces BASEPRI with BASEPRI_MAX in operations that
aim to lock some specific interrupts:
- irq_lock()
- masking out PendSV
So, for example, it is not possible to actually
unmask any interrupts by doing an irq_lock operation.
The commit does not introduce behavioral changes.
However, it makes irq_lock() more robust against
future changes to the IRQ locking mechanism.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
Baseline Cortex-M requires VTOR to be aligned on 64-word
boundary. That is because bit-7 of VTOR is also RAZ/WI.
The commit updates the vector table section alignment for
Baseline Cortex-M to reflect the implementation constraint.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
Platform specific initialization during early boot
has been a feature supported only by Cortex-M; the
Kconfig symbol is define in arch/arm Kconfig space.
We rename the z_platform_init() function to
z_arm_platform_init(), to indicate more clearly that
this is an internal, private ARM-only API.
This commit does not introduce behavioral changes.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
Platform specific initialization during early boot
has been a feature supported only by Cortex-M; the
Kconfig symbol is defined in arch/arm Kconfig space.
We rename the z_platform_init() function to
z_arm_platform_init(), to indicate more clearly that
this is an internal, private ARM-only API.
This commit does not introduce behavioral changes.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
If the DebugMonitor extension is implemented by the core,
the interrupt may be pended and become active, even if it
is not enabled. Set the priority level of DebugMonitor upon
system initialization to the intended value unconditionally
so we do not end up in undefined behavior, if the exception
is accidentally pended. Since the priority level is set at
init, we can remove resetting the priority in DWT driver
initialization.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
When the SoC implements SysTick, but the system
does not use it as the driver for system timing
we still need to set its interrupt level. This
is because the SysTick IRQ is always enabled,
so we must ensure the interrupt priority is set
to a level lower than the kernel interrupts (for
the assert mechanism to work properly) in case
the SysTick interrupt is accidentaly raised.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
If the PendSV interrupt is not used by Zephyr (this is
the case when we build with single-thread support) we
route the interrupt to z_arm_exc_spurious, instead of
assigning 0 to the vector table entry. This is because
the interrupt is always enabled and always exists, so
it is safer to always get the proper error report, in
case we accidentally pend the PendSV, for any reason.
We also add a comment in the PendSV priority setting,
explaining why it has to be assigned a priority level
even if it is not used.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
Create z_arm_preempted_thread_in_user_mode to abstract the
implementation differences between Cortex-M and R to determine if an
exception came from userspace.
Signed-off-by: Bradley Bolen <bbolen@lexmark.com>
Create z_arm_thread_is_user_mode to abstract the implementation
differences between Cortex-M and R to determine if the current thread is
in user or kernel mode.
Signed-off-by: Bradley Bolen <bbolen@lexmark.com>
Most arch's CMakeLists.txt contain rules to add compiler and linker
flags for coverage if CONFIG_COVERAGE is enabled, but 4 of them were
missing this.
Instead, set the coverage flags in arch/common/CMakeLists.txt which
affects all archs.
Signed-off-by: Jeremy Bettis <jbettis@chromium.org>
Also, this eases readability.
The new API can be used any time all FP registers must be manually
saved and restored for an operation.
Signed-off-by: Øyvind Rønningstad <oyvind.ronningstad@nordicsemi.no>
Most of the code for the three exception functions is identical so use
macros to make things easier to read.
Signed-off-by: Bradley Bolen <bbolen@lexmark.com>
Use the context switch macro for z_arm_cortex_r_svc to be more clear
about the svc call being executed.
Signed-off-by: Bradley Bolen <bbolen@lexmark.com>
Shrink the name of the hidden cortex-m option for the
null-pointer dereference detection feature.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
Reduce the length of the Kconfig defines related to
null-pointed dereference detection in Cortex-M.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
If single thread builds are not supported by the
architecture, the MULTITHREADING option should be
prompt-less to block any modifications to it. We
also introduce an explicit ARCH-level Kconfig that
reflects whether the ARCH is capable of single-thread
Zephyr builds.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
In case CONFIG_NOCACHE_MEMORY=y, the D-Cache need to be clean and
invalidated before enabling the MPU to make sure no data from a
__nocache__ region is present in the D-Cache.
If the D-Cache is disabled, SCB_CleanInvalidateDCache() shall not be
used as it might contains random data for random addresses, and this
might just create a bus fault.
Signed-off-by: Aurelien Jarno <aurelien@aurel32.net>
On reset we do not know what is the status of the D-Cache, nor its
content.
If it is disabled, do not try to clean it, as it might contains random
data for random addresses, and this might just create a bus fault.
Invalidating it is enough.
If it is enabled, it means its content is not random.
SCB_InvalidateDCache() will clean it, invalidate it and disable it.
Signed-off-by: Aurelien Jarno <aurelien@aurel32.net>
This adds code to swap_helper.S which does special handling of LR when
the interrupt came from secure. The LR value is stored to memory, and
put back into LR when swapping back to the relevant thread.
Also, add special handling of FP state when switching from secure to
non-secure, since we don't know whether the original non-secure thread
(which called a secure service) was using FP registers, so we always
store them, just in case.
Signed-off-by: Øyvind Rønningstad <oyvind.ronningstad@nordicsemi.no>
Introduce a Kconfig option to allow Secure function calls to be
pre-empted.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
Signed-off-by: Øyvind Rønningstad <oyvind.ronningstad@nordicsemi.no>
Setup the static MPU regions before PRE_KERNEL_1 and
PRE_KERNEL_2 functions are invoked. This will setup
the MPU for SRAM regions in case code relocated to SRAM
is invoked from any of these functions.
Signed-off-by: Mahesh Mahadevan <mahesh.mahadevan@nxp.com>
Code relocated using CONFIG_CODE_DATA_RELOCATION_SRAM should
be allowed to execute from SRAM
Signed-off-by: Mahesh Mahadevan <mahesh.mahadevan@nxp.com>
1. This will help us identify if the relocation is to
SRAM which is used when setting up the MPU entry
for the SRAM region where code is relocated
2. Move CODE_DATA_RELOCATION configs to ARM specific
folder
Signed-off-by: Mahesh Mahadevan <mahesh.mahadevan@nxp.com>
CONFIG_FPU: The architecture dependency list is redundant.
Having CPU_HAS_FPU being selected by those archs as a dependency
is sufficient and cleaner.
CONFIG_FPU_SHARING: The default should always be y to be on the safe
side here, but as a compromise for not affecting existing config, let's
move the default selection local to those configs that care, again to
avoid a growing list of conditionals here. Adjust the help text which
applies to more than just Cortex-M.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
There is a possibility that the DWT frequency calculation
is divided by zero. So this fixes the issue by repeatedly
trying to get the delta clock cycles and delta DWT cycles
until they both are not zero.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Reboot functionality has nothing to do with PM, so move it out to the
subsys/os folder.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
bus_fault() and hard_fault() were missing final else statement
in the if else if constructs. This commit adds non-empty else {}
to comply with coding guideline 15.7.
Signed-off-by: Jennifer Williams <jennifer.m.williams@intel.com>
z_arm_debug_monitor_event_error_check() was missing final
else statement in the if else if construct so violated guideline
15.7. This commit removes the else if for symmetry in the limited
early-exit conditions, rather than empty final else {}, to comply.
Signed-off-by: Jennifer Williams <jennifer.m.williams@intel.com>
Inline some minor clarifications regarding the
Lazy Stacking feature in the cortex-m pendSV
handler, for ease of understanding. Also, fix
some minor style issues in comments.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
Some of these registers may contain nuggets of information that would be
beneficial when debugging, so include them in the fault dump.
Signed-off-by: Bradley Bolen <bbolen@lexmark.com>
Do not hardcode the array size in the loop for printing out the floating
point registers of the exception stack frame. The size of this array
will change when Cortex-R support is added.
Signed-off-by: Bradley Bolen <bbolen@lexmark.com>
When CONFIG_MULTITHREADING=n kernel specific pendsv is not used. Remove
from vector table.
Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
The GIC can return 0x3ff to indicate a spurious interrupt. Other
interrupt controllers could return something different. Check that the
pending interrupt is valid in order to avoid indexing past the end of
the isr_table.
This fixes#30465 and is based on the aarch64 fix in 9dd2731d.
Signed-off-by: Bradley Bolen <bbolen@lexmark.com>
Flag was present only when ZLI was enabled. That resulted in additional
ifdefs needed whenever code supports ZLI and non-ZLI mode.
Removed ifdefs, added build assert to irq connections to fail at
compile time if IRQ_ZERO_LATENCY is set but ZLI is disabled. Additional
clean up made which resulted from removing the ifdef.
Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
Both operands of an operator in which the usual arithmetic
conversions are performed shall have the same essential
type category.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Add a note in the Kconfig help text that explains why Hard ABI
is not possible on builds with TF-M.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
When building with TFM, the app is linked with libraries built by the
TFM build system. TFM is always built with -msoft-float which is
equivalent to -mfloat-abi=soft. FP_HARDABI adds -mfloat-abi=hard
which gives errors when linking with the libs from TFM since they are
built with a different ABI.
Fixes https://github.com/zephyrproject-rtos/zephyr/issues/33956
Signed-off-by: Øyvind Rønningstad <oyvind.ronningstad@nordicsemi.no>
Split ARM and ARM64 architectures.
Details:
- CONFIG_ARM64 is decoupled from CONFIG_ARM (not a subset anymore)
- Arch and include AArch64 files are in a dedicated directory
(arch/arm64 and include/arch/arm64)
- AArch64 boards and SoC are moved to soc/arm64 and boards/arm64
- AArch64-specific DTS files are moved to dts/arm64
- The A72 support for the bcm_vk/viper board is moved in the
boards/bcm_vk/viper directory
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
This patch introduce new API to enable FPU of thread. This is pair of
existed k_float_disable() API. And also add empty arch_float_enable()
into each architectures that have arch_float_disable(). The arc and
riscv already implemented arch_float_enable() so I do not touch
these implementations.
Motivation: Current Zephyr implementation does not allow to use FPU
on main and other system threads like as work queue. Users need to
create an other thread with K_FP_REGS for floating point programs.
Users can use FPU more easily if they can enable FPU on running
threads.
Signed-off-by: Katsuhiro Suzuki <katsuhiro@katsuster.net>
Add initial support for the Cortex-M55 Core which is an implementation
of the Armv8.1-M mainline architecture and includes support for the
M‑profile Vector Extension (MVE).
The support is based on the Cortex-M33 support that already exists in
Zephyr.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
GCC10 introduced by default calls to out-of-line helpers to implement
atomic operations with the '-moutline-atomic' option. This is breaking
several tests because the embedded calls are trying to access the
zephyr_data region from userspace that is declared as MT_P_RW_U_NA,
triggering a memory fault.
Since there is currently no support for MT_P_RW_U_RO (and probably never
will be), disable the out-of-line helpers disabling the GCC option.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
It is apparently possible for one CPU to change the memory domain
of a thread already being executed on another CPU.
All CPUs must ensure they're using the appropriate mapping after a
thread is newly added to a domain.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Introduce the necessary routines to have the user thread stack correctly
mapped and the functions to swap page tables on context switch.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
The only user of arch_mem_domain_destroy was the deprecated
k_mem_domain_destroy function which has now been removed. So remove
arch_mem_domain_destroy as well.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
There's no need to duplicate the linker section for each architecture.
Instead, move the section declaration to common-rom.ld.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Pretty crude for now, as we always invalidate the entire set.
It remains to be seen if more fined grained TLB flushing is worth
the added complexity given this ought to be a relatively rare event.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Introduce the basic support code for memory domains. To each domain
is associated a top page table which is a copy of the global kernel
one. When a partition is added, corresponding memory range is made
private before its mapping is adjusted.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
We need to protect against concurrent modifications to page tables and
their use counts.
It would have been nice to have one lock per domain, but we heavily
share page tables across domains. Hence the global lock.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Two scenarios are possible.
privatize_page_range:
Affected pages are made private if they're not. This means a whole
new page branch starting from the top may be allocated and content
shared with the reference page tables, except for the private range
where content is duplicated.
globalize_page_range:
That's the reverse operation where pages for given range is shared with
the reference page tables and no longer needed pages are freed.
When changing a domain mapping the range needs to be privatized first.
When changing a global mapping the range needs to be globalized last.
This way page table sharing across domains is maximized and memory
usage remains optimal.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Make the allocation, population and linking of a new table into
a function of its own for easier code reuse.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Can only be written at the highest Exception level implemented.
For example, if EL3 is the highest implemented Exception level,
CNTFRQ_EL0 can only be written at EL3.
Also move z_arm64_el_highest_plat_init to be called when is_el_highest
Signed-off-by: Peng Fan <peng.fan@nxp.com>
This patch adds the code managing the syscalls. The privileged stack
is setup before jumping into the real syscall.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
This leverages the AT (address translation) instruction to test for
given access permission. The result is then provided in the PAR_EL1
register.
Thanks to @jharris-intel for the suggestion.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Introduce the arch_user_string_nlen() assembly routine and the necessary
C code bits.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
User mode is only allowed to induce oopses and stack check failures via
software-triggered system fatal exceptions.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
The arch_is_user_context() function is relying on the content of the
tpidrro_el0 register to determine whether we are in user context or not.
This register is set to '1' when in EL1 and set back to '0' when user
threads are running in userspace.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Introduce the first pieces needed to schedule user threads by defining
two different code paths for kernel and user threads.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
If EL2 is implemented but we're skipping EL2, we should still
do EL2 init. Otherwise we end up with a bunch of things still
at their (unknown) reset values.
This in particular causes problems when different
cores have different virtual timer offsets.
Signed-off-by: James Harris <james.harris@intel.com>
There are several issues with the current implemenation of the
{inc,dec}_nest_counter macros.
The first problem is that it's internally using a call to a misplaced
function called z_arm64_curr_cpu() (for some unknown reason hosted in
irq_manage.c) that could potentially clobber the caller-saved registers
without any notice to the user of the macro.
The second problem is that being a macro the clobbered registers should
be specified at the calling site, this is not possible given the current
implementation.
To fix these issues and make the call quicker, this patch rewrites the
code in assembly leveraging the availability of the _curr_cpu array. It
now clobbers only two registers passed from the calling site.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Null-pointer exception detection using DWT is currently incompatible
with current openocd runner default implementation that leaves debug
mode on by default.
As a consequence, on all targets that use openocd runner, null-pointer
exception detection using DWT will generated an assert.
As a consequence, all tests are failing on such platforms.
Disable this until openocd behavior is fixed (#32984) and enable
the MPU based solution for now.
Signed-off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
With _kernel_offset_to_nested, we only able to access the nested counter
of the first cpu. Since we are going to support SMP, we need accessing
nested from per cpu.
To get the current cpu, introduce z_arm64_curr_cpu for asm usage,
because arch_curr_cpu could not be compiled in asm code.
Signed-off-by: Peng Fan <peng.fan@nxp.com>
There is no strict reason to use assembly for the reset routine. Move as
much code as possible to C code using the proper helpers.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
The name for registers and bit-field in the cpu.h file is incoherent and
messy. Refactor the whole file using the proper suffixes for bits,
shifts and masks.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Each vector slot has room for 32 instructions. The exception context
saving needs 15 instructions already. Rather than duplicating those
instructions in each out-of-line exception routines, let's store
them directly in the vector table. That vector space is otherwise
wasted anyway. Move the z_arm64_enter_exc macro into vector_table.S
as this is the only place where it should be used.
To further reduce code size, let's make z_arm64_exit_exc into a
function of its own to avoid code duplication again. It is put in
vector_table.S as this is the most logical location to go with its
z_arm64_enter_exc counterpart.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Assert if the null pointer de-referencing detection (via DWT) is
enabled when the processor is in debug mode, because the debug
monitor exception can not be triggered in debug mode (i.e. the
behavior is unpredictable). Add a note in the Kconfig definition
of the null-pointer detection implementation via DWT, stressing
that the solution requires the core be in normal mode.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
We introduce build time asserts for
CONFIG_CORTEX_M_DEBUG_NULL_POINTER_EXCEPTION_PAGE_SIZE
to catch that the user-supplied value has, as requested
by the Kconfig symbol specification, a power of 2 value.
For the MPU-based implementation of null-pointer detection
we can use an existing macro for the build time assert,
since the region for catching null-pointer exceptions
is a regular MPU region, with different restrictions,
depending on the MPU architecture. For the DWT-based
implementation, we introduce a custom build-time assert.
We add also a run-time ASSERT for the MPU-based
implementation in ARMv8-M platforms, which require
that the null pointer exception detection page is
already mapped by the MPU.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
By design, the DebugMonitor exception is only employed
for null-pointer dereferencing detection, and enabling
that feature is not supported in Non-Secure builds. So
when enabling the DebugMonitor exception, assert that
it is not targeting the Non Secure domain.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
Enable the null-pointer dereferencing detection by default
throughout the test-suite. Explicitly disable this for the
gen_isr_table test which needs to perform vector table reads.
Disable null-pointer exception detection on qemu_cortex_m3
board, as DWT it is not emulated by QEMU on this platform.
Additionally, disable null-pointer exception detection on
mps2_an521 (QEMU target), as DWT is not present and the MPU
based solution won't work, since the target does not have
the area 0x0 - 0x400 mapped, but the QEMU still permits
read access.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
Implementation for null pointer exception detection feature
using the MPU on Cortex-M. Null-pointer detection is implemented
by programming an MPU to guard a limited area starting at
address 0x0. on non ARMv8-M we program an MPU region with
No-access policy. On ARMv8-M we program a region with any
permissions, assuming the region will overlap with fixed
FLASH0 region. We add a compile-time message to warn the
user if the MPU-based null-pointer exception solution can
not be used (ARMv8-M only).
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
Padding inserted after the (first-stage) vector table,
so that the Zephyr image does not attempt to use the
area which we reserve to detect null pointer dereferencing
(0x0 - <size>). If the end of the vector table section is
higher than the upper end of the reserved area, no padding
will be added. Note also that the padding will be added
only once, to the first stage vector table, even if the current
snipped is included multiple times (this is for a corner case,
when we want to use this feature together with SW Vector Relaying
on MCUs without VTOR but with an MPU present).
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
Additions to the null-pointer exception detection mechanism
for ARMv8-M Mainline MCUs.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
Implement the functionality to detect null pointer dereference
exceptions via the DWT unit in the ARMv7-M Mainline MCUs.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
When we enable the null pointer exceptino feature (using DWT)
we include debug.c in the build. debug.c contains the functions
to configure and enable null pointer detection using the Data
Watchdog and Trace unit.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
Extend the debug monitor exception handler to
- return recoverable faults when the debug monitor
is enabled but we do not get an expected DWT event,
- call a debug monitor routine to check for null pointer
exceptions.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
Move the DWT utility functions, present in timing.c
in an internal cortex-m header.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
Introduce the required Kconfig symbol framework for the
Cortex-M-specific null pointer dereferencing detection
feature. There are two implementations (based on DWT and
MPU) so we introduce the corresponding choice symbols,
including a choice symbol to signify that the feature
is to be disabled.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
The most common secure monitor firmware in the ARM world is TF-A. The
current release allows up to 8 64-bit values to be returned from a
SMC64 call from AArch64 state.
Extend the number of possible return values from 4 to 8.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Instead of relying on hardcoded offset in the assembly code, introduce
the offset macros to make the code more clear.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
The current code is assuming that the SMC/HVC helpers can only be used
by the PSCI driver. This is wrong because a mechanism to call into the
secure monitor should be made available regardless of using PSCI or not.
For example several SoCs relies on SMC calls to read/write e-fuses,
retrieve the chip ID, control power domains, etc...
This patch introduces a new CONFIG_HAS_ARM_SMCCC symbol to enable the
SMC/HVC helpers support and export that to drivers that require it.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
This is fundamental enough that it better be initialized ASAP.
Many other things get initialized soon afterwards assuming the MMU
is already operational.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Location of __kernel_ram_start is too far and _app_smem .bss areas
are not covered. Use _image_ram_start instead.
Location of __kernel_ram_end is also way too far. We should stop at
_image_ram_end where the expected unmapped area starts.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
This is easier to cover multiple segments this way. Especially since
not all boundary symbols from the linker script come with a size
derrivative.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The MT_OVERWRITE case is much more common. Redefine that flag as
MT_NO_OVERWRITE instead for those fewer cases where it is needed.
One such case is platform provided mappings. Apply them after the
common kernel mappings and use the MT_NO_OVERWRITE on them.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
There is no real reason for keeping page tables into separate pools.
Make it global which allows for more efficient memory usage and
simplifies the code.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Introduce a remove_map() to ... remove a mapping.
Add a use count to the page table pool so pages can be dynamically
allocated, deallocated and reused.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Add a newer, much smaller and simpler implementation of abort and
join. No need to involve the idle thread. No need for a special code
path for self-abort. Joining a thread and waiting for an aborting one
to terminate elsewhere share an implementation. All work in both
calls happens under a single locked path with no unexpected
synchronization points.
This fixes a bug with the current implementation where the action of
z_sched_single_abort() was nonatomic, releasing the lock internally at
a point where the thread to be aborted could self-abort and confuse
the state such that it failed to abort at all.
Note that the arm32 and native_posix architectures, which have their
own thread abort implementations, now see a much simplified
"z_thread_abort()" internal API.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
We need to form the ASSERT expression inside the MemManage
fault handler for the case we building without USERSPACE
and STACK GUARD support, in the same way it is formed for
the case with USERSPACE or MPU STACK GUARD support, that
is, we only assert if we came across a stacking error.
Data access violations can still occur even without user
mode or guards, e.g. when trying to write to Read-only
memory (such as the code region).
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
Add the missing pieces to enable XIP for AArch64. Try to simulate the
XIP using QEMU using the '-bios' parameter.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
The call to sys_trace_idle() is potentially clobbering x0 resulting in a
wrong value being used by the following code. Save and restore x0 before
and after the call to sys_trace_idle() to avoid any issue.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Suggested-by: James Harris <james.harris@intel.com>
Additional stack for tests when building with FPU_SHARING
enabled is required, because the option may increase ESF
stacking requirements for threads.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
When VTOR is implemented on the Cortex-M SoC, we can
basically use any address (properly aligned) for the
vector table starting address. We fix the setting of
VTOR in prep_c.c for non-XIP images, in this commit,
so we do not need to always have the vector table be
present at the start of RAM (CONFIG_SRAM_BASE_ADDRESS)
and allow for extra linker sections being placed before
the vector table section.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
If CONFIG_EXTRA_EXCEPTION_INFO is enabled, log
the value of EXC_RETURN in the fault handler.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
Under FPU sharing mode, any thread is allowed to generate
a Floating Point context (use FP registers in FP instructions),
regardless of whether threads are pre-tagged with K_FP_REGS
option when they are created.
When building with MPU stack guard feature enabled,
a large MPU stack guard is required to catch stack
overflows, if lazy FP stacking is enabled. When lazy
FP stacking is not enabled, a default 32 byte guard is
sufficient.
If lazy stacking is enabled by default, all threads may
potentially generate FP context, so they would need to
program a large MPU guard, carved out of their reserved
stack memory.
To avoid this memory waste, we modify the behavior, and make
lazy stacking a dynamically enabled feature, implemented as
follows:
- threads that are not pre-tagged with K_FP_REGS, and have
not generated an FP context use a default MPU guard and disable
lazy stacking. As long as the threads do not have an active FP
context, they won't stack FP registers, anyway, on ISRs and
exceptions, while they will benefit from reserving a small
MPU guard size
- as soon as a thread starts using FP registers, ISR might
temporarily experience some increased ISR latency due to lazy
stacking being disabled. This will be the case until the next
context switch, where the threads that have active FP context
will be tagged with K_FP_REGS, enable lazy stacking, and
program a wide MPU guard.
The implementation is a tradeoff between performance (ISR
latency) and memory consumption.
Note that when MPU STACK GUARD feature is not enabled, lazy
FP stacking is always activated.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
For the standard multi-theading builds, we will
enforce FP context stacking only when FPU_SHARING
is set. For the single-threading use case we enable
context stacking by default.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
If CONTROL register is done in reset.S we can skip
clearing the FPCA when enabling the floating point
support, to save a few instructions. The CONTROL
register is cleared right after boot, if the symbol
CONFIG_INIT_ARCH_HW_AT_BOOT is enabled.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
Major changes:
- move related functions together
- optimize add_map() not to walk the page tables *twice* on
every loop
- properly handle leftover size when a range is already mapped
- don't overwrite existing mappings by default
- return an error when the mapping fails
and make the code clearer overall.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Both _IRQ_VECTOR_TABLE_SECTION_NAME and _SW_ISR_TABLE_SECTION_NAME
are defined with asterisk at the end in an attempt to include
all related symbols in the linker script. However, these two
macros are also being used in the source code to specify
the destination sections for variables. Asterisks in the name
results in older GCC (4.x) complaining about those asterisks.
So create new macros for use in linker script, and keep
the names asterisk free.
Fixes#29936
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
In the current interrupt nesting implementation, if an ISR is
interrupted while executing inside a branch, the lr_svc register will
be corrupted, and the branch of the interrupted ISR will exit to the
return address of the final branch of the interrupting ISR, which may
or may not correspond to the intended return address.
This commit fixes the aforementioned bug by storing the lr_svc register
in the stack at the ISR entry, and restoring its value before exiting
the ISR.
For more details, refer to the issue #30517.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
This commit fixes the following bugs in the AArch32 z_arm_exc_exit
routine:
1. Invalid return address when calling `z_arm_pendsv` from the
exception-specific mode
2. Caller-saved register is referenced after a call to `z_arm_pendsv`
For more details, refer to the issue #31511.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
This commit updates the stale references to the `_IntExit` function in
the in-line documentation.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
Image header is compatible with Linux aarch64 boot protocol,
so zephyr can be booted with U-boot or Xen loader.
Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
This removes the z_ prefix those (functions, enums, etc.) that
are being used outside the coredump subsys. This aligns better
with the naming convention.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Firmware implementing the PSCI functions described in ARM document
number ARM DEN 0022A ("Power State Coordination Interface System
Software on ARM processors") can be used by Zephyr to initiate various
CPU-centric power operations.
It is needed for virtualization, it is used to coordinate OSes and
hypervisors and it provides the functions used for SMP bring-up such as
CPU_ON and CPU_OFF.
A new PSCI driver is introduced to setup a proper subsystem used to
communicate with the PSCI firmware, implementing the basic operations:
get_version, cpu_on, cpu_off and affinity_info.
The current implementation only supports PSCI 0.2 and PSCI 1.0
The PSCI conduit (SMC or HVC) is setup reading the corresponding
property in the DTS node.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Using newlibc with AArch64 is causing an alignement fault in
z_bss_zero() when the code is run on real hardware (on QEMU the problem
is not reproducible).
The main cause is that the memset() function exported by newlibc is
using 'DC ZVA' to zero out memory.
While this is often a nice optimization, this is causing the issue on
AArch64 because memset() is being used before the MMU is enabled, and
when the MMU is disabled all data accesses will be treated as
Device_nGnRnE.
This is a problem because quoting from the ARM reference manual: "If the
memory region being zeroed is any type of Device memory, then DC ZVA
generates an Alignment fault which is prioritized in the same way as
other alignment faults that are determined by the memory type".
newlibc tries to be a bit smart about this reading the DCZID_EL0
register before deciding whether using 'DC ZVA' or not. While this is a
good idea for code running in EL0, currently the Zephyr kernel is
running in EL1. This means that the value of the DCZID_EL0 register is
actually retrieved from the HCR_EL2.TDZ bit, that is always 0 because
EL2 is not currently supported / enabled. So the 'DC ZVA' instruction is
unconditionally used in the newlibc memset() implementation.
The "standard" solution for this case is usually to use a different
memset routine to be specifically used for two cases: (1) against IO
memory or (2) against normal memory but with MMU disabled (which means
all memory is considered device memory for data accesses).
To fix this issue in Zephyr we avoid calling memset() when clearing the
bss, and instead we use a simple loop to zero the memory region.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
In rare cases when a thread may overflow its stack, the
core will not report a Stacking Error. This is the case
when a large stack array is created, making the PSP cross
beyond the stack guard; in this case a MemManage fault
won't cause a stacking error (but only a Data Access
Violation error). We fix the fault handling logic so
such errors are reported as stack overflows and not as
generic CPU exceptions.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
When the MMARVALID bit is not set, do not read the MMFAR
register to get the fault address in a MemManage fault.
This change prevents the fault handler to erroneously
assume MMFAR contains a valid address.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
Before hooking up the MMU driver code to the Zephyr MMU core code it's
better to match the expected variable types of the two parts.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
The MMU code is currently assuming that Zephyr only uses one single set
of page tables shared by kernel and user threads. This could possibly be
not longer true in the future when multiple set of page tables can be
present and swapped at run-time.
With this patch a new arm_mmu_ptables struct is introduced that is used
to host a buffer pointing to the memory region containing the page
tables and the helper variables used to manage the page tables. This new
struct is then used by the ARM64 MMU code instead of assuming that the
kernel page tables are the only ones present.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
The ARM64 MMU code used to create the page tables is strictly tied to
the custom arm_mmu_region struct. To be able to hook up this code to the
Zephyr MMU APIs we need to make it more generic.
This patch makes the mapping function more generic and creates a new
helper function add_arm_mmu_region() to map the regions defined by the
old arm_mmu_region structs using this new generic function.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
In the current code the base xlat table is a standalone array. This is
done because we know at compile time the size of this table so we can
allocate the correct size and save a bit of memory. All the other xlat
tables are statically allocated in a different array with full size.
With this patch we move all the page tables in one single array,
including the base table. This is probably going to waste a bit of space
but it makes easier to:
- have all the page tables mapped in one single contiguous memory region
instead of having to take care of two different arrays in two
different locations
- duplicate the page tables more quickly if we need to
- use a pre-allocated space to host the page tables
- use a pre-computed set of page tables saved in a contiguous memory
region
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
In the current MMU code a new table is created when mapping a memory
region that is overlapping with a block already mapped. The problem is
that the new table is created also when the new and old mappings have
the same attributes.
To avoid using a new table when not needed the attributes of the two
mappings are compared before creating the new table.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
This commit adds possibility to disable ECC in Tightly Coupled
Memory in Cortex-R.
Linker scripts places stacks in this memory and marks it as
.noinit section. With ECC enabled, stack read accesses without
previous write result in Data Abort Exception.
Signed-off-by: Wojciech Sipak <wsipak@antmicro.com>
According to CONFIG_ARMV8_A_NS, using MT_SECURE or MT_NS, to simplify
code change, use MT_DEFAULT_SECURE_STATE instead
Signed-off-by: Peng Fan <peng.fan@nxp.com>
In _isr_wrapper, the interrupt ID read from the GIC is blindly used to
index into _sw_isr_table, which is only sized based on CONFIG_NUM_IRQ.
It is possible for both GICv2 and GICv3 to return 1023 for a handful
of scenarios, the simplest of which is a level sensitive interrupt
which has subsequently become de-asserted. Borrowing from the Linux
GIC implementation, a read that returns an interrupt ID of 1023 is
simply ignored.
Minor collateral changes to gic.h to group !_ASMLANGUAGE content
together to allow this header to be used in assembler files.
Signed-off-by: Luke Starrett <luke.starrett@gmail.com>
The GIC interrupt controller driver is using a custom init function
called directly from the prep_c function. For consistency move that to
use SYS_INIT.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
- Remove SYS_ prefix
- shorten POWER_MANAGEMENT to just PM
- DEVICE_POWER_MANAGEMENT -> PM_DEVICE
and use PM_ as the prefix for all PM related Kconfigs
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
ARM64 is currently using SP_ELx as stack pointer for kernel and threads
because everything is running in EL1. If support for EL0 is required, it
is necessary to switch to use SP_EL0 instead, that is the only stack
pointer that can be accessed at all exception levels by threads.
While it is not required to keep using SP_EL0 also during the
exceptions, the current code implementation makes it easier to use the
same stack pointer as the one used by threads also during the
exceptions.
This patch moves the code from using SP_ELx to SP_EL0 and fill in the
missing entries in the vector table.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
On Cortex M7, we need to check the optional presence of
Lock Access Register (LAR) which is indicated in
Lock Status Register (LSR).
When present, a special access token must be written to unlock DWT
registers.
Signed-off-by: Alexandre Bourdiol <alexandre.bourdiol@st.com>
The 2K alignment assembler directives should be under
'SECTION_SUBSEC_FUNC(exc_vector_table,_vector_table_section,_vector_table)'
Otherwise the _vector_table is actually 0x80 bytes aligned.
Signed-off-by: Peng Fan <peng.fan@nxp.com>
Most of kernel files where declaring os module without providing
log level. Because of that default log level was used instead of
CONFIG_KERNEL_LOG_LEVEL.
Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
Adds a new CONFIG_MPU which is set if an MPU is enabled. This
is a menuconfig will some MPU-specific options moved
under it.
MEMORY_PROTECTION and SRAM_REGION_PERMISSIONS have been merged.
This configuration depends on an MMU or MPU. The protection
test is updated accordingly.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
k_mem_partition is part of the CONFIG_USERSPACE abstraction,
but some older MPU code was depending on it even if user mode
isn't enabled. Use a new structure z_arm_mpu_partition instead,
which will insulate this code from any changes to the core
kernel definition of k_mem_partition.
The logic in z_arm_configure_dynamic_mpu_regions has been
adjusted to copy the necessary information out of the
memory domain instead of passing the addresses of the domain
structures directly to the lower-level MPU code.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Every time I try to decode all the defines in this driver what I get is
only a huge headache. This patch:
- adds a few sensible comments
- remove the redundant defines
- rename the defines to be more self-explanatory
- reorder the defines
- try to make sense of some mysterious derived values
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
As done already for other structs, use the macro-generated offsets when
referencing register in the ESF.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
The init_stack_frame is the same as the the ESF. No need to have two
separate structs. Consolidate everything into one single struct and make
register entries explicit.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Use GEN_OFFSET_SYM macro to genarate absolute symbols for the
_callee_saved struct and use these new symbols in the assembly code.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
For some kind of faults we want to be able to put in action some
corrective actions and keep executing the code.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Make the printing of errors a bit more descriptive and print the FAR_ELn
register only when strictly required.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Each vector table entry has 128-bytes to host the vector code. This is
not always enough and in general it's better to branch to the actual
exception handler elsewhere in memory.
Move the SError entry to a branched code.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Since the tracing of thread being switched in/out has the same
instrumentation points, we can roll the tracing function calls
into the one for thread stats gathering functions.
This avoids duplicating code to call another function.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
We should not be initializing/starting/stoping timing functions
multiple times. So this changes how the timing functions are
structured to allow only one initialization, only start when
stopped, and only stop when started.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
In a5f34d85c2 ("soc: arm: qemu_cortex_a53: Remove SRAM region") the
SRAM memory region was removed.
While this is correct when userspace is not enabled, when userspace is
enabled new regions are introduced outside the boundaries of
the mapped [__kernel_ram_start,__kernel_ram_end] region. This means that
we need to map again the whole SRAM to include all the needed regions.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
The current MMU code is assuming that both kernel and threads are both
running in EL1, not supporting EL0. Extend the support to EL0 by adding
the missing attribute to mirror the access / execute permissions to EL0.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
We are probably going to do more work on the MMU side and more files
will be added. Create a new sub-directory to host all the MMU related
files.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Implement the functionality for configuring the
architecture core registers to their warm reset
values upon system initialization. We enable the
support of the feature in the Cortex-M architecture.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
We enhance the documentation of z_arm_reset, stressing that
the function may either be loaded by the processor coming
out of reset, or by another image, e.g. a bootloader. We
also specify what is required at minimum when executing the
reset function.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
This is redundant and not coherent with the rest of the file. Thus
remove the _BIT suffix from the bit field names.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
The current vector table is missing some (not used) entries. Fill these
in for the sake of completeness.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
The SVC handler is not only used for the SVC call but in general for all
the synchronous exceptions. Reflect this in the handler name.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Adds the necessary bits to initialize TLS in the stack
area and sets up CPU registers during context switch.
Note that since Cortex-M does not have the thread ID or
process ID register needed to store TLS pointer at runtime
for toolchain to access thread data, a global variable is
used instead.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Adds the necessary bits to initialize TLS in the stack
area and sets up CPU registers during context switch.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Adds the necessary bits to initialize TLS in the stack
area and sets up CPU registers during context switch.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Setup the stack as early as possible to catch any possible errors in the
reset routine and handle also EL3 fatal errors.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
The content of the SCR_EL3 register is overwritten by a later
instruction. Also no need to route SError, IRQs and FIQs to EL3.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Instead of having some special stack frame when first scheduling new
thread and a new thread entry wrapper to pull out the needed data, we
can reuse the context restore code by adapting the initial stack frame.
This reduces the lines of code and simplify the code at the expense of a
slightly bigger initial stack frame.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Both operands of an operator in the arithmetic conversions
performed shall have the same essential type category.
Changes are related to converting the integer constants to the
unsigned integer constants
Signed-off-by: Aastha Grover <aastha.grover@intel.com>
Zephyr is only supposed to be running at EL1 (+ EL0). Now that we drop
in EL1 from ELn at start we can remove all the EL2/EL3 unused code.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Remove the useless CONFIG_SWITCH_TO_EL1 since there should be no reason
to run Zephyr in EL3. So just drop to EL1 by default when booting from
EL3. Remove also non-reachable code.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
- Display full 64-bits register width in crash dumps
- Some values were prefixed 0x, some not. Made consistent.
Signed-off-by: Luke Starrett <luke.starrett@gmail.com>
- x0/x1 register printing is reversed
- The error stack frame struct (z_arch_esf_t) had the SPSR and ELR in
the wrong position, inconsistent with the order these regs are pushed
to the stack in z_arm64_svc. This caused all register printing to be
skewed by two.
- Verified by writing known values (abcd0000 -> abcd000f) to x0 - x15
and then forcing a data abort.
Signed-off-by: Luke Starrett <luke.starrett@gmail.com>
Fixes races where threads on another CPU are joining the
exiting thread, since it could still be running when
the joiners wake up on a different CPU.
Fixes problems where the thread object is still being
used by the kernel when the fn_abort() function is called,
preventing the thread object from being recycled or
freed back to a slab pool.
Fixes a race where a thread is aborted from one CPU while
it self-aborts on another CPU, that was currently worked
around with a busy-wait.
Precedent for doing this comes from FreeRTOS, which also
performs final thread cleanup in the idle thread.
Some logic in z_thread_single_abort() rearranged such that
when we release sched_spinlock, the thread object pointer
is never dereferenced by the kernel again; join waiters
or fn_abort() logic may free it immediately.
An assertion added to z_thread_single_abort() to ensure
it never gets called with thread == _current outside of an ISR.
Some logic has been added to ensure z_thread_single_abort()
tasks don't run more than once.
Fixes: #26486
Related to: #23063#23062
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Enable interrupts before switching to main()
in cortex-m builds with single-thread mode
(CONFIG_MULTITHREADING=n).
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
If the location counter ('.') is within the area that the veneers
should go, the current solution will give a linker error ("Cannot move
location counter backwards"). This patch places the veneers in the next
SPU region in this case.
Signed-off-by: Øyvind Rønningstad <oyvind.ronningstad@nordicsemi.no>
CPU Cortex-M implies Mainline Cortex-M, therfore, the dependency
on ARMV6_M_ARMV8_M_BASELINE is redundant and can be removed. The
change in this commit is a no-op.
We also add the ARMV6_M_ARMV8_M_BASELINE dependency on option
CPU_CORTEX_M0_HAS_VECTOR_TABLE_REMAP to make sure it cannot be
selected for non Cortex-M Baseline SoCs (at least, not without
a warning).
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
Using SCB_CleanInvalidateDcache instead of SCB_DisableDcache
& SCB_EnableDcache when config the non-cache area, in case
of the cache will effect the configuration of the non-cache
area
Signed-off-by: Crist Xu <crist.xu@nxp.com>
Work around an issue where the emulator ignores host OS
signals when inside a `wfi` instruction.
This should be reverted once this has been addressed in the
AARCH64 build of QEMU in the SDK.
See https://github.com/zephyrproject-rtos/sdk-ng/issues/255
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
When _arch_switch() API is used, the tracing of the thread swapped out
is done in the C kernel code (in do_swap() for cooperative scheduling
and in set_current() during preemption). In the assembly code we only
have to trace the thread when swapped in.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Cortex-M SoCs implement (optionally) the Data Watchpoint and
Tracing Unit (DWT), which can be used for timing functions.
Select the corresponding ARCH capability if the SoC implements
the DWT.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
This code had one purpose only, feed timing information into a test and
was not used by anything else. The custom trace points unfortunatly were
not accurate and this test was delivering informatin that conflicted
with other tests we have due to placement of such trace points in the
architecture and kernel code.
For such measurements we are planning to use the tracing functionality
in a special mode that would be used for metrics without polluting the
architecture and kernel code with additional tracing and timing code.
Furthermore, much of the assembly code used had issues.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Switch to the _arch_switch() API that is required for an SMP-aware
scheduler instead of using the old arch_swap mechanism.
SMP is not supported yet but this is a necessary step in that direction.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Provide a TZ_SAFE_ENTRY_FUNC() macro for wrapping non-secure entry
functions in calls to k_sched_lock()/k_sched_unlock()
Provide a __TZ_WRAP_FUNC() macro which helps in creating a function
that "wraps" another in a preface and postface function call.
int foo(char *arg); // Implemented somewhere else.
int __attribute__((naked)) foo_wrapped(char *arg)
{
WRAP_FUNC(bar, foo, baz);
}
is equivalent to
int foo(char *arg); // Implemented somewhere else.
int foo_wrapped(char *arg)
{
bar();
int res = foo(arg);
baz();
return res;
}
This commit also adds tests for __TZ_WRAP_FUNC().
Signed-off-by: Øyvind Rønningstad <oyvind.ronningstad@nordicsemi.no>
The same code was being copypasted in k_thread_abort()
implementations, just move into z_thread_single_abort().
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
This isn't needed; match the vanilla implementation
in kernel/thread_abort.c and do this unlocked. This
should improve system latency.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
A check was being done that was a more obscure way of
calling arch_is_in_isr(). Add a comment explaining
why we need to trigger PendSV.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
We implement an ARM-only API for ARM Secure Firmware,
to set all NVIC IRQ lines to target the Non-Secure state.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
