The implementation of `z_arm_clear_arm_mpu_config` was compiled for all
ARM cores that declare to have an MPU. However, we only want to compile
it if the MPU is actually enabled.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
Where the bottom is the only one which interacts with
the host operating system.
And the top the only one that interacts or is aware
of the hosted operating system (Zephyr).
The bottom uses the native simulator CPU
start/stop emulation.
By now we replicate its code as a provisional measure,
until the native simulator becomes standard.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
Where the bottom is the only one which interacts with
the host operating system, and does not necessarily
need to know about the Zephyr OS.
This is in preparation for the native simulator,
which which the bottom is also fully Zephy agnostic.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
The issue due to which we set the default compiler C std version
to C11 is not specific to any particular POSIX arch board,
but to all. Instead of setting this property for each board,
let's set it at the architecture level.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
This adds support for 32 double-precision registers in the context
switching of aarch32 architecture.
Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
`fpscr` is assigned from `struct __fpu_sf.fpscr` in `vfp_restore`, but it
wasn't saved into `struct __fpu_sf.fpscr` in the svc and isr handler, So
it may be a dirty value.
- Fix it by saving `fpscr` in the svc hand isr handler.
- Jump out if FPU isn't enabled
Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
The code_relocation feature creates generic section names that sometimes
conflict with already existing names.
This patch adds a '_reloc_' word to the created names to reduce the risk
of conflict.
This solves #54785.
Signed-off-by: Björn Stenberg <bjorn@haxx.se>
This adds a few line use zephyr_syscall_header() to include
headers containing syscall function prototypes.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Some RISCV platforms shipping a CLIC have a peculiar interrupt ID
ordering / mapping.
According to the "Core-Local Interrupt Controller (CLIC) RISC-V
Privileged Architecture Extensions" Version 0.9-draft at paragraph 16.1
one of these ordering recommendations is "CLIC-mode interrupt-map for
systems retaining interrupt ID compatible with CLINT mode" that is
described how:
The CLINT-mode interrupts retain their interrupt ID in CLIC mode.
[...]
The existing CLINT software interrupt bits are primarily intended for
inter-hart interrupt signaling, and so are retained for that purpose.
[...]
CLIC interrupt inputs are allocated IDs beginning at interrupt ID
17. Any fast local interrupts that would have been connected at
interrupt ID 16 and above should now be mapped into corresponding
inputs of the CLIC.
That is a very convoluted way to say that interrupts 0 to 15 are
reserved for internal use and CLIC only controls interrupts reserved for
platform use (16 up to n + 16, where n is the maximum number of
interrupts supported).
Let's now take now into consideration this situation in the DT:
clic: interrupt-controller {
...
};
device0: some-device {
interrupt-parent = <&clic>;
interrupts = <0x1>;
...
};
and in the driver for device0:
IRQ_CONNECT(DT_IRQN(node), ...);
From the hardware prospective:
(1a) device0 is using the first IRQ line of the CLIC
(2a) the interrupt ID / exception code of the `MSTATUS` register
associated to this IRQ is 17, because the IDs 0 to 15 are reserved
From the software / Zephyr prospective:
(1b) Zephyr is installing the IRQ vector into the SW ISR table (and into
the IRQ vector table for DIRECT ISRs in case of CLIC vectored mode)
at index 0x1.
(2b) Zephyr is using the interrupt ID of the `MSTATUS` register to index
into the SW ISR table (or IRQ vector table)
It's now clear how (2a) and (2b) are in contrast with each other.
To fix this problem we have to take into account the offset introduced
by the reserved interrupts. To do so we introduce
CONFIG_RISCV_RESERVED_IRQ_ISR_TABLES_OFFSET as hidden option for the
platforms to set.
This Kconfig option is used to shift the interrupt numbers when
installing the IRQ vector into the SW ISR table and/or IRQ vector table.
So for example in the previous case and using
CONFIG_RISCV_RESERVED_IRQ_ISR_TABLES_OFFSET == 16, the IRQ vector
associated to the device0 would be correctly installed at index 17 (16 +
1), matching what is reported by the `MSTATUS` register.
CONFIG_NUM_IRQS must be increased accordingly.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Zephyr currently only supports CLINT direct mode and CLINT vectored
mode. Add support for CLIC vectored mode as well.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Before adding support for the CLIC vectored mode, rename
CONFIG_RISCV_MTVEC_VECTORED_MODE to CONFIG_RISCV_VECTORED_MODE to be
more generic and eventually include also the CLIC vectored mode.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
With all the stacks and TSS (etc), the x86_64 arch code can only
support maximum of 4 CPUs at the moment. So add a build assert
if more CPUs are specified via CONFIG_MP_MAX_NUM_CPUS, also
overwrite the range value for CONFIG_MP_MAX_NUM_CPUS.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
We are missing setting of switch_handle for the thread which
is aborting due to exception (i.e. in case of k_panic or
__ASSERT triggered). This may cause livelock in SMP code
after a08e23f68e commit ("kernel/sched: Fix SMP
must-wait-for-switch conditions in abort/join").
Fix that.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
Currently the lazy fpu saving algorithm in arm64 is using the fpu_owner
pointer from the cpu structure to understand the owner of the context
in the cpu and save it in case someone different from the owner is
accessing the fpu.
The semantics for memory consistency across smp systems is quite prone
to errors and reworks on the current code might miss some barriers that
could lead to inconsistent state across cores, so to overcome the issue,
use atomics to hide the complexity and be sure that the code will behave
as intended.
While there, add some isb barriers after writes to cpacr_el1, following
the guidance of ARM ARM specs about writes on system registers.
Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
This will avoid unconditionally pulling z_riscv_switch() into the build
as it is not used, reducing the resulting binary some more.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
With Zephyr now always using `int main(void)`, there's no longer any need
for this definition. The last remaining use which gated the declaration of
_posix_zephyr_main isn't necessary as adding that declaration
unconditionally is harmless.
Signed-off-by: Keith Packard <keithp@keithp.com>
cpu_node_list does not hold the corrent mapping of cpu id and mpid when
core booting sequence does not follow the DTS cpu node sequence. This
will cause an issue that sgi cannot deliver to the right target.
Add the cpu_map array to hold the corrent mapping between cpu id and
mpid.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Each core should init their own stack during the reset when SMP enabled,
but do not touch others. The current init results in each core starting
init the stack from the same address which will break others.
Fix the issue by setting a correct start address.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
LOG system has unalignment access instruction which will cause an
alignment exception before MPU is enabled. Remove the LOG print before
MPU is enabled to avoid this issue.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
This trick turns out also to be needed by the abort/join code.
Promote it to a more formal-looking internal API and clean up the
documentation to (hopefully) clarify the exact behavior and better
explain the need.
This is one of the more... enchanted bits of the scheduler, and while
the trick is IMHO pretty clean, it remains a big SMP footgun.
Signed-off-by: Andy Ross <andyross@google.com>
For secure EL2 to be entered the EEL2 bit in SCR_EL3 must be set. This
should only be set if Zephyr has not been configured for NS mode only,
if the device is currently in secure EL3, and if secure EL2 is supported
via the SEL2 bit in AA64PFRO_EL1. Added logic to enable EEL2 if all
conditions are met.
Signed-off-by: Chad Karaginides <quic_chadk@quicinc.com>
This reverts commit f0b458a619.
This is a pointless change that simply increases footprint.
Existing code already supports compilation without multithreading.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Per the ARMv8 architecture document, modification of the system control
register is a context-changing operation. Context-changing operations are
only guaranteed to be seen after a context synchronization event.
An ISB is a context synchronization event. One has been placed after
each SCTLR modification. Issue was found running full speed on target.
Signed-off-by: Chad Karaginides <quic_chadk@quicinc.com>
Allow builds which has CONFIG_MULTITHREADING disabled.
This is reduce code footprint which is handy for
constrained targets as bootloaders.
Signed-off-by: Marek Matej <marek.matej@espressif.com>
Allow builds which has CONFIG_MULTITHREADING disabled.
This is reduce code footprint which is handy for
constrained targets as bootloaders.
Signed-off-by: Marek Matej <marek.matej@espressif.com>
This make MCUboot build as Zephyr application.
Providing optinal 2nd stage bootloader to the
IDF bootloader, which is used by default.
This provides more flexibility when building
and loading multiple images and aims to
brings better DX to users by using the sysbuild.
MCUboot and applications has now separate
linker scripts.
Signed-off-by: Marek Matej <marek.matej@espressif.com>
Let's consider CPU1 waiting on a spinlock already taken by CPU2.
It is possible for CPU2 to invoke the FPU and trigger an FPU exception
when the FPU context for CPU2 is not live on that CPU. If the FPU context
for the thread on CPU2 is still held in CPU1's FPU then an IPI is sent
to CPU1 asking to flush its FPU to memory.
But if CPU1 is spinning on a lock already taken by CPU2, it won't see
the pending IPI as IRQs are disabled. CPU2 won't get its FPU state
restored and won't complete the required work to release the lock.
Let's prevent this deadlock scenario by looking for pending FPU IPI from
the spinlock loop using the arch_spin_relax() hook.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Let's consider CPU1 waiting on a spinlock already taken by CPU2.
It is possible for CPU2 to invoke the FPU and trigger an FPU exception
when the FPU context for CPU2 is not live on that CPU. If the FPU context
for the thread on CPU2 is still held in CPU1's FPU then an IPI is sent
to CPU1 asking to flush its FPU to memory.
But if CPU1 is spinning on a lock already taken by CPU2, it won't see
the pending IPI as IRQs are disabled. CPU2 won't get its FPU state
restored and won't complete the required work to release the lock.
Let's prevent this deadlock scenario by looking for a pending FPU IPI
from the arch_spin_relax() hook and honor it.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The compiler is not able to emit a proper DSB operation for ARM64. Move
to the arch-specific implementation and use assembly code instead.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Enhance for cases when call z_float_enable() with NULL thread.
Signed-off-by: Dong Wang <dong.d.wang@intel.com>
Signed-off-by: Qipeng Zha <qipeng.zha@intel.com>
This adds code to always map data TLB for VECBASE so that
we would be dealing with fewer data TLB misses during
exception handling. With VECBASE always mapped, there is
no need to pre-load anymore.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This moves the TLB miss handling to the C exception handler.
This also allows us to handle page faults (for example,
unmapped pages) during this time as any more exceptions
handled in the C handler will not trigger the double
exception handler but the same C handler.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Instead of being able to arbitrarily set the PTEVADDR for page
table, this provides choices (currently just one). This is in
preparation to enable handling memory management exception in
C code. For that to work, we will need to pre-load the page
table address (PTEVADDR) for the memory page containing
exception code and data (containing jump addresses), and
various stacks. This is to prempt any TLB misses during handling
the level 1 interrupt code. If a TLB miss is encountered during
handling of level 1 interrupt, we will be thrown into double
exception handling code where we will get stuck in infinite
loop. However, in order to pre-load the page table entries,
PTEVADDR needs to be calculated. This requires the use of
PTEVADDR base which cannot be loaded via l32r, as we may cause
a data TLB miss. So we must be able to grab the PTEVADDR base
address strictly within code, and must be without any data
load. So changing CONFIG_XTENSA_MMU_PTEVADDR to be based on
choice so we can have pre-defined bit shift value for shift
operation. This shift value will be used in exception handling
code.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Add a build option to tell if memory should be mapped in cached
and uncachedr regions.
If the memory is neither in cached nor uncached region it is not double
mapped.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Initial support for Xtensa MMU version 3. It is using a two level page
table based on fact that the page table is in the virtual space. Only
the top level (page directory) is wired mapped in the TLB to avoid
second level page miss.
The mapped memory is completely fragmented in multiple sections, maybe
we find a better way in future.
The exception handler is where we effectively map the memory, the way it
works is:
1) SW try to access some memory address
2) The address is not mapped, so the MMU will try the auto-refill,
looking the page table
3) The page table contents is not mapped (remember, just the top-level page
is mapped)
4) An exception will be triggered, in the exception we try to read the
portion of the page table that maps the original address
5) The address is not mapped, so the MMU will try again the auto-refill.
This time though, the address is mapped by the top level page that is
properly mapped. (The top-level page maps the page table itself).
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Unlike tracing module mainly for debug usage, this is
to allow runtime profiling IRQ performance data, and
target to enable it in product release since platform
can choose to make it work with low weight protocol.
Enable this option and implement runtime_irq_stats()
in platform code, such as Intel ISH platform implement
with SHMI protocol to allow host profiling irq stats.
Signed-off-by: Qipeng Zha <qipeng.zha@intel.com>
Until now iterable sections APIs have been part of the toolchain
(common) headers. They are not strictly related to a toolchain, they
just rely on linker providing support for sections. Most files relied on
indirect includes to access the API, now, it is included as needed.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
The commit 434ca63e2f introduced the
Cortex-A and Cortex-R CPU type dependency to `CONFIG_FP16` based on
the reasoning that the hardware half-precision support is only
available on them.
While it is true that the _hardware_ half-precision support is limited
to these targets, the compiler will provide the _software_ emulation
for the targets that lack the hardware half-precision support, as
mentioned in 41fd6e003c (the original
commit that introduced `CONFIG_FP16`).
Signed-off-by: Stephanos Ioannidis <stephanos.ioannidis@nordicsemi.no>
In z_xtensa_backtrace_print the parameter depth is checked for <= 0.
There is no need to check it again later, also, since the variable is
not used after the while loop we can use directly the parameter without
an additional variable.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Enable single-threading support for the riscv architecture.
Add z_riscv_switch_to_main_no_multithreading function for
supporting single-threading.
The single-threading does not work with enabling PMP_STACK_GUARD.
It is because single-threading does not use context-switching.
But the privileged mode transition that PMP depends on implicitly
presupposes using context-switching. It is a contradiction.
Thus, disable PMP_STACK_GUARD when MULTITHREADING is not enabled.
Signed-off-by: TOKITA Hiroshi <tokita.hiroshi@fujitsu.com>
Intel ISH SoC can't reboot via RST_CNT register,
so make sys_arch_reboot as weak function to allow
implement different arch reboot in SoC layer.
Signed-off-by: Dong Wang <dong.d.wang@intel.com>
Signed-off-by: Qipeng Zha <qipeng.zha@intel.com>
The "cross stack call" mechanism has intermediate states where the
stack frames are not valid for our own interrupt entry code, which
causes corruption if an interrupt races at exactly the right time.
Leave interrupts masked until just before the call.
The fix is midly complicated by the fact that we RELY on nested window
exception frames to spill registers from the interruptee, so have to
do the masking with PS.INTLEVEL, which requires a register to save its
contents, which we don't have since everything needs to happen in one
4-register window. But thankfully our Zephyr-reserved EPS register is
guaranteed to be available through this process.
Fixes#57009
Signed-off-by: Andy Ross <andyross@google.com>
With paging config, need to use physical address as
paging is not enabled here.
From IA manual, LDMXCSR instruction description is,
Loads the source operand into the MXCSR control/status
register, the source operand is a 32-bit memory location.
Signed-off-by: Qipeng Zha <qipeng.zha@intel.com>
Use the common exit() provided by libc so we get standard behavior
across all architectures. So only implement a special exit when
XT_SIMULATOR is defined.
Signed-off-by: Kumar Gala <kumar.gala@intel.com>
Local variables in ASM macro works differently for GNU and MWDT
toolchains. In case of GNU toolchain they are local per each macro
instance, but in case of MWDT they are local per file where macro
is used.
To avoid issues when macro is used multiple times in one file let's
align _st32_huge_offset to have same behaviour with GNU & MWDT
toolchains.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
The backtrace requires a valid stack pointer to start
printing backtraces. So if there is no stack pointer
being passed in, skip printing backtraces.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
use helper macros from csr.h instead of inline assembly which results
in cleaner and more maintainable code
Signed-off-by: Manojkumar Subramaniam <manoj@electrolance.com>
Commit 408472673e added inline
assembly to lock interrupt. However, XCC doesn't like the syntax
using STRINGIFY, and also an empty clobber section. So parameterize
the second argument to rsil, and remove the last colon.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This adds some structs for interrupt stack frames to make it
easier to access individual elements, and ultimately getting
rid of magic array element numbers in the code. Hopefully,
this would aid in debugging where you can view the whole
struct in debugger.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Align the comment with the new int main(void) convention
used in Zephyr.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
Some functions such as the ISH power management function needs to
reserve GDT entry place holders which are filled during runtime.
Add config option to define the number of GDT entry place holders
reserved and change the ia32 linker script accordingly.
Signed-off-by: Leifu Zhao <leifu.zhao@intel.com>
The init infrastructure, found in `init.h`, is currently used by:
- `SYS_INIT`: to call functions before `main`
- `DEVICE_*`: to initialize devices
They are all sorted according to an initialization level + a priority.
`SYS_INIT` calls are really orthogonal to devices, however, the required
function signature requires a `const struct device *dev` as a first
argument. The only reason for that is because the same init machinery is
used by devices, so we have something like:
```c
struct init_entry {
int (*init)(const struct device *dev);
/* only set by DEVICE_*, otherwise NULL */
const struct device *dev;
}
```
As a result, we end up with such weird/ugly pattern:
```c
static int my_init(const struct device *dev)
{
/* always NULL! add ARG_UNUSED to avoid compiler warning */
ARG_UNUSED(dev);
...
}
```
This is really a result of poor internals isolation. This patch proposes
a to make init entries more flexible so that they can accept sytem
initialization calls like this:
```c
static int my_init(void)
{
...
}
```
This is achieved using a union:
```c
union init_function {
/* for SYS_INIT, used when init_entry.dev == NULL */
int (*sys)(void);
/* for DEVICE*, used when init_entry.dev != NULL */
int (*dev)(const struct device *dev);
};
struct init_entry {
/* stores init function (either for SYS_INIT or DEVICE*)
union init_function init_fn;
/* stores device pointer for DEVICE*, NULL for SYS_INIT. Allows
* to know which union entry to call.
*/
const struct device *dev;
}
```
This solution **does not increase ROM usage**, and allows to offer clean
public APIs for both SYS_INIT and DEVICE*. Note that however, init
machinery keeps a coupling with devices.
**NOTE**: This is a breaking change! All `SYS_INIT` functions will need
to be converted to the new signature. See the script offered in the
following commit.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
init: convert SYS_INIT functions to the new signature
Conversion scripted using scripts/utils/migrate_sys_init.py.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
manifest: update projects for SYS_INIT changes
Update modules with updated SYS_INIT calls:
- hal_ti
- lvgl
- sof
- TraceRecorderSource
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
tests: devicetree: devices: adjust test
Adjust test according to the recently introduced SYS_INIT
infrastructure.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
tests: kernel: threads: adjust SYS_INIT call
Adjust to the new signature: int (*init_fn)(void);
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
rename the function that sets the handler for the nmi.
It should be namespaced and not camel-case:
z_NmiHandlerSet to z_arm_nmi_set_handler
Signed-off-by: Thomas Stranger <thomas.stranger@outlook.com>
In case of recoverable fatal errors the execution should
switch to another thread. This will ensure the current_cpu nested
count is reset when there is a context switch.
Signed-off-by: Aastha Grover <aastha.grover@intel.com>
We get the following error when building with arm-clang:
error: non-ASM statement in naked function is not supported
__TZ_WRAP_FUNC(preface, foo1, postface);
^
tests/arch/arm/arm_tz_wrap_func/src/main.c:69:25: note: attribute is here
uint32_t __attribute__((naked)) wrap_foo1(uint32_t arg1, uint32_t arg2,
^
1 error generated.
Remove the do/while wrapper to make this a true naked function.
Signed-off-by: Kumar Gala <kumar.gala@intel.com>
Revert commit 44628735b8
This commit broke the ability for nxp rt series to
reset except with power cycle
Signed-off-by: Declan Snyder <declan.snyder@nxp.com>
Current implementation of cache management APIs for ARM only applies to
Cortex-M, so move it to its own directory.
Signed-off-by: Manuel Argüelles <manuel.arguelles@nxp.com>
Fixes unneeded chain of includes. Since zefi is built separately
(using python script), any dependency creates include chain with
possibly missing configuration options.
Signed-off-by: Andrei Emeltchenko <andrei.emeltchenko@intel.com>
Update current stack limit on every context switch, including switching
to irq stack and switching back to thread stack.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
This commit mainly enable the safe exception stack including the stack
switch. Init the safe exception stack by calling
z_arm64_safe_exception_stack during the boot stage on every core. Also,
tweaks several files to properly switch the mode with different cases.
1) The same as before, when executing in userspace, SP_EL0 holds the
user stack and SP_EL1 holds the privileged stack, using EL1h mode.
2) When entering exception from EL0 then SP_EL0 will be saved in the
_esf_t structure. SP_EL1 will be the current SP, then retrieves the safe
exception stack to SP_EL0, making sure the always pointing to safe
exception stack as long as the system running in kernel space.
3) When exiting exception from EL1 to EL0 then SP_EL0 will be restored
from the stack value previously saved in the _esf_t structure. Still at
EL1h mode.
4) Either entering or exiting exception from EL1 to EL1, SP_EL0 will
keep holding the safe exception stack unchanged as memtioned above.
5) Do a quick stack check every time entering the exception from EL1 to
EL1. If check fail, set SP_EL1 to safe exception stack, and then handle
the fatal error.
Overall, the exception from user mode will be handled with kernel stack
at the assumption that it is impossible the stackoverflow happens at the
entry of exception from EL0 to EL1. However the exception from kernel
mode will be firstly checked with the safe exception stack to see if the
kernel stack overflows, because the exception might be triggered by
stack invalid accessing.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Add safe exception stack init function which does several things:
1) setting current cpu safe exception stack pointer to its corresponding
stack top.
2) init sp_el0 with the above safe exception stack.
That makes sure the sp_el0 points to per-cpu safe_stack in the kernel
space.
3) init the current_stack_limit and corrupted_sp with 0
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
As the preparation for enabling safe exception stack, add a variable in
_esf_t to save the user stack held by sp_el0 at the point of the
exception happening from EL0. The newly added variable in _esf_t is
named sp from which the user stack will be restored when exceptions eret
to EL0.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Add three per-cpu variables for the convenience of quickly accessing.
The safe_exception_stack stores the top of safe exception stack pointer.
The current_stack_limit stores the current thread's priv stack limit.
The corrputed_sp stores the priv sp or irq sp for the stack overflow
case, or 0 for the normal case.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Introduce two configs to prepare to enable the safe exception stack for
the kernel space. This is the preparation for enabling hardware stack
guard. Also define the safe exception stack for kernel exception stack
check.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
If so this is most certainly a bug. arch_mem_unmap() should be
used before mapping the same area again.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
First, we have commit 7d27bd0b85 ("arch: arm64: Disable infinite
recursion warning for `discard_table`") that blindly shut up a compiler
warning that did actually highlighted a real bug. Revert that and fix
the bug properly. And yes, mea culpa for having been the first to
approve that commit, or even creating the bug in the first place.
Then let's add proper table usage cound handling for discard_table() to
work properly and avoid leaking table pages.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
For RISCV arch, enable FLASH_SIZE and FLASH_BASE_ADDRESS config.
To avoid duplicated work, remove flash config from RISCV soc.
Signed-off-by: Jonas Otto <jonas@jonasotto.com>
The image header is compatible for zImage(32) protocol.
Offset Value Description
0x24 0x016F2818 Magic number to identify ARM Linux zImage
0x28 start address The address the zImage starts at
0x2C end address The address the zImage ends at
As Zephyr can be built with a fixed load address, Xen/Uboot can read
the image header and decide where to copy the Zephyr image.
Also, it is to be noted that for AArch32 A/R, the vector table should
be aligned to 0x20 address. Refer ARM DDI 0487I.a ID081822, G8-9815,
G8.2.168, VBAR, Vector Base Address Register :-
Bits[4:0] = RES0.
For AArch32 M (Refer DDI0553B.v ID16122022, D1.2.269, VTOR, Vector Table
Offset Register), Bits [6:0] = RES0.
As zImage header occupies 0x30 bytes, thus it is necessary to align
the vector table base address to 0x80 (which satisfies both VBAR and
VTOR requirements).
Also, it is to be noted that not all the AArch32 M class have VTOR, thus
ARM_ZIMAGE_HEADER header depends on
CPU_AARCH32_CORTEX_R || CPU_AARCH32_CORTEX_A || CPU_CORTEX_M_HAS_VTOR.
The reason being the processors which does not have VBAR or VTOR, needs
to have exception vector table at a fixed address in the beginning of
ROM (Refer the comment in arch/arm/core/aarch32/cortex_m/CMakeLists.txt)
. They cannot support any headers.
Also, the first instruction in zImage header is to branch to the kernel
start address. This is to support booting in situations where the zImage
header need not be parsed.
In case of Arm v8M, the first two entries in the reset vector should be
"Initial value for the main stack pointer on reset" and "Start address
for the reset handler" (Refer Armv8M DDI0553B.vID16122022, B3.30,
Vector tables).
In case of Armv7M (ARM DDI 0403E. ID021621, B1.5.3 The vector table),
the first entry is "SP_main. This is the reset value of the Main stack
pointer.".
Thus when v7M or v8M starts from reset, it expects to see these values
at the default reset vector location.
See the following text from Armv7M (ARM DDI 0403E. ID021621, B1-526)
"On powerup or reset, the processor uses the entry at offset 0 as the
initial value for SP_main..."
Signed-off-by: Ayan Kumar Halder <ayan.kumar.halder@amd.com>
Add missing include to prevent `'EINVAL' undeclared` when
using `CONFIG_NULL_POINTER_EXCEPTION_DETECTION_DWT=y`
Signed-off-by: George Ruinelli <caco3@ruinelli.ch>
FP16 isn't something that is supported on Cortex-M so limit the
Kconfig feature to Cortex-A or Cortex-R.
Signed-off-by: Kumar Gala <kumar.gala@intel.com>
The CONFIG_ROM_START_OFFSET is supposed to be added to
the current when linking, instead of having the current
address set to it. So fix that.
Not sure why it worked up to this point, but llvm/clang/lld
complained that it could not move location counter backward.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Introduce an optional hook to be called when the CPU is made idle.
If needed, this hook can be used to prevent the CPU from actually
entering sleep by skipping the WFE/WFI instruction.
Signed-off-by: Andrzej Głąbek <andrzej.glabek@nordicsemi.no>
Looks like some implementors decided not to implement the full set of
PMP range matching modes. Let's rearrange the code so that any of those
modes can be disabled.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Let's honor CONFIG_MPU_REQUIRES_POWER_OF_TWO_ALIGNMENT even for kernel
stacks. This saves one global PMP slot when creating the guard area for
the IRQ stack, and some hw implementations might require that anyway.
With this changes, arch_mem_domain_max_partitions_get() becomes much
more reliable and tests/kernel/mem_protect is more likely to pass even
with the stack guard enabled.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Additional privileged stack space is used by peripheral emulators when
userspace is enabled. This is largely due to additional function calls and
data structures allocated on the stack. This can potentially lead to stack
smashing if the privileged stack size isn't high enough, causing an
exception.
Increase the privileged stack space when userspace and peripheral emulation
are enabled.
Signed-off-by: Aaron Massey <aaronmassey@google.com>
When CONFIG_SOC_ISR_SW_UNSTACKING is defined, it's up to the custom soc
code to remove the ESF, because the software-managed part of the ESF is
depending on the hardware. Fix this in the ISR code.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Some implementations may not capture the faulting instruction in mtval
and set it to zero when an illegal instruction fault is raised This is
notably the case with QEMU version 7.0.0 when a CSR instruction is
involved.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The FRCSR, FSCSR, FRRM, FSRM, FSRMI, FRFLAGS, FSFLAGS and FSFLAGSI
are in fact CSR instructions targeting the fcsr, frm and fflags
registers. They should be caught as FPU instructions as well.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
- IRQ state for the interrupted context corresponds to the PIE bit not
the IE bit.
- Restoring the saved FPU state should clear the entire field before
or'ing wanted bits in.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
For RISC-V, the reg property of a cpu node in the devicetree describes
the low level unique ID of each hart. Using devicetree macro's, a list
of all cpus with status "okay" can be generated.
Using devicetree overlays, a hart or multiple harts can be marked as
"disabled", thus excluding them from the list. This allows platforms
that have non-zero indexed SMP capable harts to be functionally mapped
to Zephyr's sequential CPU numbering scheme.
On kernel init, if the application has MP_MAX_NUM_CPUS greater than 1,
generate the list of cpu nodes from the device tree with status "okay"
and map the unique hartid's to zephyr cpu's
While we are at it, as the hartid is the value that gets passed to
z_riscv_secondary_cpu_init, use that as the variable name instead of
cpu_num
Signed-off-by: Conor Paxton <conor.paxton@microchip.com>
RISC-V multi-hart systems that employ a heterogeneous core complex are
not guaranteed to have the smp capable harts starting with a unique id
of zero, matching Zephyr's sequential zero indexed cpu numbering scheme.
Add option, RV_BOOT_HART to choose the hart to boot from.
On reset, check the current hart equals RV_BOOT_HART: if so, boot first
core. else, loop in the boot secondary core and wait to be brought up.
For multi-hart systems that are not running a Zephyr mp or smp
application, park the non zephyr related harts in a wfi loop.
Signed-off-by: Conor Paxton <conor.paxton@microchip.com>
Add an option to generate simplified error codes instead of more
specific architecture specific error codes. Enable this by default in
tests to make exception tests more generic across hardware.
Fixes#54053.
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
Disables allowing the python argparse library from automatically
shortening command line arguments, this prevents issues whereby
a new command is added and code that wrongly uses the shortened
command of an existing argument which is the same as the new
command being added will silently change script behaviour.
Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
Commit 4f9b547ebd ("riscv: smp: prepare for more than one IPI type")
didn't clear pending IPI flags.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
We can leverage the FPU dirty state as an indicator for preemptively
reloading the FPU content when a thread that did use the FPU before
being scheduled out is scheduled back in. This avoids the FPU access
trap overhead when switching between multiple threads with heavy FPU
usage.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
FPU context switching is always performed on demand through the FPU
access exception handler. Actual task switching only grants or denies
FPU access depending on the current FPU owner.
Because RISC-V doesn't have a dedicated FPU access exception, we must
catch the Illegal Instruction exception and look for actual FP opcodes.
There is no longer a need to allocate FPU storage on the stack for every
exception making esf smaller and stack overflows less likely.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Instead of saving/restoring FPU content on every exception and task
switch, this replaces FPU sharing support with a "lazy" (on-demand)
context switching algorithm similar to the one used on ARM64.
Every thread starts with FPU access disabled. On the first access the
FPU trap is invoked to:
- flush the FPU content to the previous thread's memory storage;
- restore the current thread's FPU content from memory.
When a thread loads its data in the FPU, it becomes the FPU owner.
FPU content is preserved across task switching, however FPU access is
either allowed if the new thread is the FPU owner, or denied otherwise.
A thread may claim FPU ownership only through the FPU trap. This way,
threads that don't use the FPU won't force an FPU context switch.
If only one running thread uses the FPU, there will be no FPU context
switching to do at all.
It is possible to do FP accesses in ISRs and syscalls. This is not the
norm though, so the same principle is applied here, although exception
contexts may not own the FPU. When they access the FPU, the FPU content
is flushed and the exception context is granted FPU access for the
duration of the exception. Nested IRQs are disallowed in that case to
dispense with the need to save and restore exception's FPU context data.
This is the core implementation only to ease reviewing. It is not yet
hooked into the build.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Right now this is hardcoded to z_sched_ipi(). Make it so that other IPI
services can be added in the future.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
If running under Xtensa simulator, it is good to tell simulator
to stop execution once we reach double exception, as the current
double exception handler is simply an endless loop. If we turn
on tracing in the simulator, the output file would contain
an infinite iteration of this endless loop, and the simulator
needs to be stopped manually before the file size goes out of
control. So we need to tell the simulator to stop once
we reach this point instead of doing an endless loop.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Given the Zephyr CPU number is no longer tied to the hartid, we must
consider the actual hartid when sending an IPI to a given CPU. Since
those hartids can be anything, let's just save them in the cpu structure
as each CPU is brought online.
While at it, throw in some `get_hart_msip()` cleanups.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Currently it is assumed that Zephyr CPU numbers match their hartid
value one for one. This assumption was relied upon to efficiently
retrieve the current CPU's `struct _cpu` pointer.
People are starting to have systems with a mix of different usage for
each CPU and such assumption may no longer be true.
Let's completely decouple the hartid from the Zephyr CPU number by
stuffing each CPU's `struct _cpu` pointer in their respective scratch
register instead. `arch_curr_cpu()` becomes more efficient as well.
Since the scratch register was previously used to store userspace's
exception stack pointer, that is now moved into `struct _cpu_arch`
which implied minor user space entry code cleanup and rationalization.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Ultil now Cortex A/R aarch32 implementation for context
switching expects that interrupts was disabled. This is
true if a context switching happens at thread context.
But if a context switching happens at last action during
interrupt context, this assumption is not true because the
interrupts are still enabled (to allow nesting interrupts).
Disable interrupts at the last interrupt action to ensure
the interrupts are always disabled before context switching
is processed
Signed-off-by: Dat Nguyen Duy <dat.nguyenduy@nxp.com>
In platforms where the linker is capable of doing global optimizations,
like relaxing address mode and synthesize new instructions, Zephyr has to
disable it when enabling USERSPACE since the build expects that address
don't change after the first stage build.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
This commit updates all in-tree code to use `CONFIG_CPP` instead of
`CONFIG_CPLUSPLUS`, which is now deprecated.
Signed-off-by: Stephanos Ioannidis <stephanos.ioannidis@nordicsemi.no>
Return specific fault reasons instead of the generic
`K_ERR_CPU_EXCEPTION`, which provides minimal debugging aid.
Fixes#53093.
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
This patchset is fixing two things:
1. The proper sys_* functions are used for cache mainteinance
operations.
2. To check the status of the L1 cache the SCB registers are probed so
the code is assuming a core architecture cache is present, thus make
the code conditionally compiled on CONFIG_ARCH_CACHE.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
arch_dcache_range() function does not exist anymore, nor K_CACHE_WB
macro. Removing it entirely.
arch_dcache_flush_range() signature changed, so relevantly applying
these.
Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
RISC-V has a modular design. Some hardware with a custom interrupt
controller needs a bit more work to lock / unlock IRQs.
Account for this hardware by introducing a set of new
z_soc_irq_* functions that can override the default behaviour.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
esf_get_sp is used only when CONFIG_THREAD_STACK_INFO is enabled.
Just move this function around to be inside an ifdef guard.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Some RISC-V SoCs implement a mechanism for hardware supported stacking /
unstacking of registers during ISR / exceptions. What happens is that on
ISR / exception entry part of the context is automatically saved by the
hardware on the stack without software intervention, and the same part
of the context is restored by the hardware usually on mret.
This is currently not yet supported by Zephyr, where the full context
must be saved by software in the full fledged ESF. This patcheset is
trying to address exactly this case.
At least three things are needed to support in a general fashion this
problem: (1) a way to store in software only the part of the ESF not
already stacked by hardware, (2) a way to restore in software only the
part of the context that is not going to be restored by hardware and (3)
a way to define a custom ESF.
Point (3) is important because the full ESF frame is now composed by a
custom part depending on the hardware (that can choose which register to
stack / unstack and the order they are saved onto the stack) and a part
defined in software for the remaining part of the context.
In this patch a new CONFIG_RISCV_SOC_HAS_ISR_STACKING is introduced that
enables the code path supporting the three points by the mean of three
macros that must be implemented by the user in a soc_stacking.h file:
SOC_ISR_SW_STACKING, SOC_ISR_SW_UNSTACKING and SOC_ISR_STACKING_ESF
(refer to the symbol help for more details).
This is an example of soc_isr_stacking.h for an hardware that doesn't do
any hardware stacking / unstacking but everything is managed in
software:
#ifndef __SOC_ISR_STACKING
#define __SOC_ISR_STACKING
#if !defined(_ASMLANGUAGE)
#define SOC_ISR_STACKING_ESF_DECLARE \
struct __esf { \
unsigned long ra; \
unsigned long t0; \
unsigned long t1; \
unsigned long t2; \
unsigned long t3; \
unsigned long t4; \
unsigned long t5; \
unsigned long t6; \
unsigned long a0; \
unsigned long a1; \
unsigned long a2; \
unsigned long a3; \
unsigned long a4; \
unsigned long a5; \
unsigned long a6; \
unsigned long a7; \
unsigned long mepc; \
unsigned long mstatus; \
unsigned long s0; \
} __aligned(16)
#else
#define SOC_ISR_SW_STACKING \
addi sp, sp, -__z_arch_esf_t_SIZEOF; \
DO_CALLER_SAVED(sr);
#define SOC_ISR_SW_UNSTACKING \
DO_CALLER_SAVED(lr);
#endif /* _ASMLANGUAGE */
#endif /* __SOC_ISR_STACKING */
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Debug monitor needs to be configured to a low priority in order to be
useful for debugging (to prioritize other interrupts when waiting on a
breakpoint).
Added a config that configures the interrupt this way.
Signed-off-by: Piotr Jasiński <piotr.jasinski@nordicsemi.no>
It seems that currently it's impossible to create a custom
implementation for debug monitor exception without updating the vector
table (z_arm_debug_monitor maps to fault).
My proposition is to make this symbol weak, so that it can be overriden.
Signed-off-by: Piotr Jasiński <piotr.jasinski@nordicsemi.no>
Save FP user register and FP register file during context switch.
This change enables shared FP registers mode using CONFIG_FPU_SHARING.
Since there is no lazy stacking, the FPU registers will be saved regardless
of whether floating point calculations are performed in the threads when
CONFIG_FPU_SHARING is enabled. This require 72 additional bytes in the
stack memory.
Signed-off-by: Lucas Tamborrino <lucas.tamborrino@espressif.com>
Use interrupts (with dedicated interrupt line) for irq_offload
instead of exception-based implementation.
That allows to implement IRQ_OFFLOAD without adding special code
to interrupt / exception path for IRQ_OFFLOAD handling and,
moreover, test the real interrupt code.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
add DSP reg in context switch
add AGU reg in context switch to support XY mem
add thread option and API to dis/enable DSP switch
Signed-off-by: Siyuan Cheng <siyuanc@synopsys.com>
There is the possibility that when reconfiguring the static regions,
some data that must be accessed is temporarily not accesible due to the
change on the MPU regions configuration. Workaround by disabling MPU
when doing the reconfiguration, same as with dynamic regions, until BR
can be enabled.
Signed-off-by: Duong Vu Nam <duong.vunam@nxp.com>
Config NOCACHE_MEMORY depend on ARCH_HAS_NOCACHE_MEMORY_SUPPORT. Enable
ARCH_HAS_NOCACHE_MEMORY_SUPPORT for Cortex-R52 to run NXP S32Z/E with
nocache attibute.
Enable nocache in each driver use it.
Signed-off-by: Duong Vu Nam <duong.vunam@nxp.com>
Support I/D cache for Cortex-R52 to run with cache on NXP S32Z/E.
Make sure no data is present in the D-Cache before initializing mpu
Signed-off-by: Duong Vu Nam <duong.vunam@nxp.com>
The cache operations must be quick, optimized and possibly inlined. The
current API is clunky, functions are not inlined and passing parameters
around that are basically always known at compile time.
In this patch we rework the cache functions to allow us to get rid of
useless parameters and make inlining easier.
In particular this changeset is doing three things:
1. `CONFIG_HAS_ARCH_CACHE` is now `CONFIG_ARCH_CACHE` and
`CONFIG_HAS_EXTERNAL_CACHE` is now `CONFIG_EXTERNAL_CACHE`
2. The cache API has been reworked.
3. Comments are added.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
As we start to use _st32_huge_offset in other places (i.e DSP code)
introduce optimized version with address shifted instruction which
doesn't produce extra instruction.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
Control shared interrupts enabling/disabling via IDU.
With that we can easily enable and disable them for all cores
in one place.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
Make ARC_MP_PRIMARY_CPU_ID definition public so it can be used in
other ARC code.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
In 0.15.2 SDK we specify r26 as default register for TLS cached
pointer, so it isn't used by compiler even if TLS support isn't
enabled. Restore the previous behavior - so if we don't use
TLS in Zephyr the register isn't stay reserved.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
The code in prep_c sets VBAR to relocate vector from 0x0, assuming the
low vector bit in SCTLR to be clear. This isn't the case on all
hardware, so set it explicitly to support those.
Signed-off-by: Théophile Ranquet <theophile.ranquet@gmail.com>
1. this header is no use for asm
2. if use xclib, this header include xclib stdbool, and expand to typedef
Signed-off-by: honglin leng <a909204013@gmail.com>
Arm provides a default address map defining default behaviors for
certain address ranges, which can be overlayed with additional regions
in the MPU. Users may also turn off this background map, so that only
regions explicitly programmed in the MPU are allowed.
This provides a Kconfig so that platforms using a non-standard address
map may disable the background address map and provide their own
explicit MPU regions.
Signed-off-by: Benjamin Gwin <bgwin@google.com>
This is a follow-up to commit f400c94adf.
Fix typos in names of introduced macros (*STR -> *SR) and cast their
values to uint32_t to avoid warnings reported for messages formatted
with %x.
Signed-off-by: Andrzej Głąbek <andrzej.glabek@nordicsemi.no>
The compiler defines __ARC_TLS_REGNO__ as the number of the
register used for TLS variables. Use that instead of hard-coding
a specific register.
Signed-off-by: Keith Packard <keithp@keithp.com>
This commit updates the `_posix_zephyr_main` declaration to use the
return type of `int` instead of `void` when `CONFIG_CPP_MAIN=y` (i.e.
C++-compliant main() support is enabled) so that Zephyr applications
defining their main() in a C++ source file can make use of the proper
main() definition of `int main(void)` as required by the C++ standard.
Note that the forward declaration of `_posix_zephyr_main` is required
if and only if the main() is defined in a C++ source file (i.e. when
`CONFIG_CPP_MAIN=y`).
Signed-off-by: Stephanos Ioannidis <stephanos.ioannidis@nordicsemi.no>
Besides adding ARCH_HAS_CODE_DATA_RELOCATION, this patch also adds
support for the "sample_controller" SoC (used by qemu_xtensa) as
demonstration.
As Xtensa lacks a common linker script at the arch level, enabling it
for each platform will be a piecemeal effort. This patch adds it to the
`soc/xtensa/sample_controller` SoC. Basically, default RAMABLE_REGION is
set to be called "RAM", and hooks are inserted so that
gen_relocate_app.py can add the relevant linker bits.
Also, `tests/application_developent/code_relocation` was tweaked to
support the qemu_xtensa platform. Basically, add the relevant linker
script and ensure that relevant memory regions have their program header
(PHDR) associated.
Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
The is code duplication as one is in C, and the other is an assembly
macro. As there is no easy way to find out about this duplication,
adding a comment seems the best way to go.
Signed-off-by: Henri Xavier <datacomos@huawei.com>
In preparation for using CONFIG_*_ENDIAN instead of __BYTE_ORDER__, add
a reflection of CONFIG_BIG_ENDIAN that will allow users to conditionally
compile with #ifdef instead of #ifndef while keeping the default case
(little endian) as the first block in the conditional compilation.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
The buffer contents returned from arch_gdb_reg_readone is a counted array
of bytes, not a C string. Use memcpy instead of strcpy for the failure
return path to avoid compiler warning about missing NUL termination.
Signed-off-by: Keith Packard <keithp@keithp.com>
This reverts commit a7b5d606c7.
The assumption behind that commit was wrong. The software-based stack
sentinel writes to the very bottom of the _writable_ stack area i.e.
right next to the actual PMP based guard area. So they are compatible.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Change for loops of the form:
for (i = 0; i < CONFIG_MP_NUM_CPUS; i++)
...
to
unsigned int num_cpus = arch_num_cpus();
for (i = 0; i < num_cpus; i++)
...
We do the call outside of the for loop so that it only happens once,
rather than on every iteration.
Signed-off-by: Kumar Gala <kumar.gala@intel.com>
Move the FS_BASE MSR code to to the top of __resume to ensure
that %fs relative addressing run in the thread switching hook
works.
Signed-off-by: Keith Packard <keithp@keithp.com>
Change automated searching for files using "IRQ_CONNECT()" API not
including <zephyr/irq.h>.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
Some headers made use of types defined in sys_clock.h (e.g. k_timeout_t)
without including it.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
There seems to be an unresolved dependency chain in some x86 arch
headers, this file needs kernel.h to be included before arch_data/func.
This patch is a workaround, but problem should be fixed properly at some
point.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
The CONFIG_STACK_SENTINEL adds 4 bytes to the stack. Take these
into account for CONFIG_ARCH_POSIX_RECOMMENDED_STACK_SIZE.
Signed-off-by: Pieter De Gendt <pieter.degendt@basalte.be>
It's useful for RAMABLE_REGION to have a uniform name when
CODE_DATA_RELOCATION is supported, because otherwise the build system
needs to be aware of how the region name differs between architectures.
Since architectures tend to prefer one of 'SRAM' or 'RAM' for that
region, prefer to use 'RAM' as the more general term.
Signed-off-by: Peter Marheine <pmarheine@chromium.org>
The interrupt stack is used as the system stack during kernel
initialization while IRQs are not yet enabled. The sp register is
set to z_interrupt_stacks + CONFIG_ISR_STACK_SIZE.
CONFIG_ISR_STACK_SIZE only represents the desired usable stack size.
This does not take into account the added guard area. Result is a stack
whose pointer is much closer to the trigger zone than expected when
CONFIG_PMP_STACK_GUARD=y, and the SMP configuration in particular pushes
it over the edge during many CI test cases.
Worse: during early init we're not quite ready to handle exceptions
yet and complete havoc ensues with no meaningful debugging output.
Make sure the early assembly code locates the actual top of the stack
by generating a constant with its true size.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The software-based stack sentinel writes to the very bottom of the
stack area triggering the PMP stack protection. Obviously they can't
be used together.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The IRQ stack in particular is different on each CPU, and so is its
stack guard PMP entry value. This creates 2 issues:
- The assertion ensuring the last global PMP address is the same
for each CPU does fail;
- That last global PMP address can't be relied upon to create a
single-slot per-thread TOR mapping.
Fix both issues by not remembering the actual address for the last
global entry but a dummy address instead that is guaranteed not to
match any opportunistic single-slot TOR mapping.
While at it, lock that IRQ stack guard PMP entry.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
GCC may generate ldp/stp instructions with the Advanced SIMD Qn
registers for consecutive 32-byte loads and stores.
This commit disables this GCC behaviour because saving and restoring
the Advanced SIMD context is very expensive, and it is preferable to
keep it turned off by not emitting these instructions for better
context switching performance.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
Z_THREAD_STACK_BUFFER() must not be used here. This is meant for stacks
defined with K_THREAD_STACK_ARRAY_DEFINE() whereas in this case we are
given a stack created with K_KERNEL_STACK_ARRAY_DEFINE().
If CONFIG_USERSPACE=y then K_THREAD_STACK_RESERVED gets defined with
a bigger value than K_KERNEL_STACK_RESERVED. Then Z_THREAD_STACK_BUFFER()
returns a pointer that is more advanced than expected, resulting in a
stack pointer outside its actual stack area and therefore memory
corruption ensues.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The parameter ssf of the handler_bad_syscall got null pointer
due to that the R1 does not push into the stack in a right
order on cortex-M0. Adjust the pushing order of stack to make
the ssf being passed correctly.
Fixes#50146.
Signed-off-by: Enjia Mai <enjia.mai@intel.com>
It turns out that SOF is already using a symbol named
"zephyr_app_main()", so this produces a collision. Pick something
that looks more relevant to "posix", and put an underscore on it (it's
a "system" symbol, after all).
Signed-off-by: Andy Ross <andyross@google.com>
It seems like libfuzzer wants to relocate 32 bit instrumented code
sections at runtime at addresses different than the ones in the ELF
file. This is problematic, because Zephyr files are compiled
statically and so will crash the first time they try to jump to an
absolute .text address (basically at the first function call after a
fuzzer entry point).
It seems that building with -fPIC is enough to defeat this (we use the
host linker script, which will manage the GOT/PLT entries for us),
which will work as long as the fuzzer isn't playing games with data
other than text. None of this seems to be documented, so... I guess
it's as good as we can get. It works, at least.
(x86_64 binaries don't show the same behavior, they run where they
were linked)
Signed-off-by: Andy Ross <andyross@google.com>
Align backtrace output with the style used in rest of the codespace.
This makes it more convenient to compare the backtrace to e.g. objdump
output.
Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
On GICv3, when we send an IPI interrupt, aff3, aff2 and aff1 should
be assigned a value corespond to a PE for which interrupt will be
generated. target_list only corresponds to aff0.
On real hardware, aff3, aff2, aff1 and aff0 should be treated as a
whole to determine a PE.
Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
VMPIDR_EL2 is assigned the value returned by EL2 reads of MPIDR_EL1
MPIDR_EL1 is the register holding the Multiprocessor ID which is to
identify different cores. Because of the virtualization requirements
for AArch64, MPIDR_EL1 should be virtualized (the different virtualized
cores can run on the same physical core). Thus the value of MPIDR_EL1
should be switched when the VM is switched. Setting the VMPIDR_EL2 is
the way to change the value returned by EL1 reads of MPIDR_EL1. Even
without virtualization, we still need to set VMPIDR_EL2 during booting
at EL2 or EL3. Otherwise, all cores' IDs are zero at the EL1 stage
which will break the SMP system.
Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
This commit is fixing placing the vectors section through
zephyr_linker_sources(ROM_START ...) (as done in the ARM
architecture port) so its order can be adjusted by SORT_KEY.
Fixes#49903
Signed-off-by: Mateusz Sierszulski <msierszulski@antmicro.com>
The Xtensa arch has historically had state/user register accessor
macros with bare three-byte symbol names. I think this might have
been in the original Cadence-contributed arch integration, but I'm not
sure. In any case they also exist in the same names in vendor
HAL/toolchain code and are causing collisions. We never should have
had these symbols exposed in our header.
Put them under an XTENSA_ prefix to decollide.
Signed-off-by: Andy Ross <andyross@google.com>
As of today <zephyr/zephyr.h> is 100% equivalent to <zephyr/kernel.h>.
This patch proposes to then include <zephyr/kernel.h> instead of
<zephyr/zephyr.h> since it is more clear that you are including the
Kernel APIs and (probably) nothing else. <zephyr/zephyr.h> sounds like a
catch-all header that may be confusing. Most applications need to
include a bunch of other things to compile, e.g. driver headers or
subsystem headers like BT, logging, etc.
The idea of a catch-all header in Zephyr is probably not feasible
anyway. Reason is that Zephyr is not a library, like it could be for
example `libpython`. Zephyr provides many utilities nowadays: a kernel,
drivers, subsystems, etc and things will likely grow. A catch-all header
would be massive, difficult to keep up-to-date. It is also likely that
an application will only build a small subset. Note that subsystem-level
headers may use a catch-all approach to make things easier, though.
NOTE: This patch is **NOT** removing the header, just removing its usage
in-tree. I'd advocate for its deprecation (add a #warning on it), but I
understand many people will have concerns.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
Convert the device to be Devicetree based. Adjusted tests and other
areas that were using old Kconfig properties.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
The "framebuf" driver was an incomplete driver expecting _clients_ to
implement missing functionality (i.e. init and device definition)
outside of the driver. This pattern of scattering driver code throughout
the tree is not common (if used at all). If certain drivers share
functionality, one can create a common module within the subsystem (see
e.g. ILI9XXX drivers).
The _generic_ framebuffer code was only used to implement the Intel
Multiboot framebuffer driver. This patch centralizes all the scattered
code in the subsystem and adjusts the driver name to "intel_multibootfb"
to make things clear. If there's ever another framebuffer driver that
shares code, it can be split into multiple modules.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
Two issues:
- A unnecessary parentheses pair caused rounding errors (by truncating
a small value before multiplying it).
- arch_timing_cycles_to_ns_avg() wasn't actually converting the result
to nanoseconds.
Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
This commit renames the ARC64 output format from `elf64-littlearc` to
`elf64-littlearc64` as required by the updated ARC patches for the GCC
12.1 release.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
This PR allows the user to add symbols to the nocache
section. The use for this could be as follows
zephyr_linker_sources_ifdef(CONFIG_NOCACHE_MEMORY
NOCACHE_SECTION
nocache.ld
)
nocache.ld (as shown below) can define additional
symbols to go into the nocache section
. = ALIGN(4);
KEEP(*(NonCacheable))
Signed-off-by: Mahesh Mahadevan <mahesh.mahadevan@nxp.com>
Add support for LLVM's libfuzzer utility. This works by building an
executable with a "LLVMFuzzerTestOneInput()" entry point (which is
external to Zephyr, running in the host process environment!), which
it drives out of its own main() routine. The toolchain API is exposed
as just another sanitizer variant, which is clean.
Signed-off-by: Andy Ross <andyross@google.com>
GCC 12 performs bounds checking on the pointer arguments specified like
an array (e.g. `int arg[]`) and treats such arguments with an empty
length as having the length of 0, resulting in the compiler printing
out `stringop-overread' warning when they are accessed.
This commit corrects any pointer arguments declared using the array
expression to use the pointer expression instead.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
This implements support for relocating code to chosen memory regions via
the `zephyr_code_relocate` CMake function for RISC-V SoCs. ARM-specific
assumptions that were made by gen_relocate_app.py need to be corrected,
in particular not assuming any particular name for the default RAM
section (which is 'SRAM' for most ARM pltaforms) and not assuming 32-bit
pointers (so the test works on RV64).
Signed-off-by: Peter Marheine <pmarheine@chromium.org>
Support for CODE_DATA_RELOCATION is not inherently limited to ARM, so
move the Kconfig definition to top-level so it can be used by other
architectures. Since support is opt-in (requiring linker script
support), add a helper symbol enabled by architecture config that gates
whether CODE_DATA_RELOCATION is available instead of listing all
supported systems inline.
Signed-off-by: Peter Marheine <pmarheine@chromium.org>
When a cache API function is called from userspace, this results on
ARM64 in an OOPS (bad syscall error). This is due to at least two
different factors:
- the location of the cache handlers is preventing the linker to
actually find the handlers
- specifically for ARM64 and ARC some cache handling functions are not
implemented (when userspace is not used the compiler simply optimizes
out these calls)
Fix the problem by:
- moving the userspace cache handlers to a their logical and proper
location (in the drivers directory)
- adding the missing handlers for ARM64 and ARC
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Some platforms have the possibility to cancel the powering off until the
very latest moment (for example if an IRQ is received). Deal with this
kind of failures.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
In case of ARCv3 64 bit we have only one 64bit accumulator
register instead of register pair, so fixup register
save & restore code.
While we at it also make ARC_HAS_ACCL_REGS option (which
controls accumulator reg/regs save & restore) default
for HS5x and HS6x as well - as it should be.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
It is frequent to find variable definitions like this:
```c
static const struct device *dev = DEVICE_DT_GET(...)
```
That is, module level variables that are statically initialized with a
device reference. Such value is, in most cases, never changed meaning
the variable can also be declared as const (immutable). This patch
constifies all such cases.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
Wire this up the same way ASAN works. Right now it's support only by
recent clang versions (not gcc), and only in 64 bit mode. But it's
capable of detecting uninitialized data reads, which ASAN is not.
This support is wired into the sys_heap (and thus k_heap/k_malloc)
layers, allowing detection of heap misuse like use-after-free. Note
that there is one false negative lurking: due to complexity, in the
case where a sys_heap_realloc() call is able to shrink memory in
place, the now-unused suffix is not marked uninitialized immediately,
making it impossible to detect use-after-free of those particular
bytes. But the system will recover cleanly the next time the memory
gets allocated.
Also no attempt was made to integrate this handling into the newlib or
picolibc allocators, though that should hopefully be possible via
similar means.
Signed-off-by: Andy Ross <andyross@google.com>
This had bitrotten a bit, and didn't build as shipped. Current
libasan implementations want -fsanitize=address passed as a linker
argument too. We have grown a "lld" linker variant that needs the
same cmake treatment as the "ld" binutils one, but never got it. But
the various flags had been cut/pasted around to different places, with
slightly different forms. That's really sort of a mess, as sanitizer
support was only ever support with host toolchains for native_posix
(and AFAICT no one anywhere has made this work on cross compilers in
an embedded environment). And the separate "gcc" vs. "llvm" layers
were silly, as there has only ever been one API for this feature (from
LLVM, then picked up compatibly by gcc).
Pull this stuff out and just do it in one place in the posix arch for
simplicity.
Also recent sanitizers are trying to add instrumentation padding
around data that we use linker trickery to pack tightly
(c.f. SYS_INIT, STRUCT_SECTION_ITERABLE) and we need a way
("__noasan") to turn that off. Actually for gcc, it was enough to
just make the records const (already true for most of them, except a
native_posix init struct), but clang apparently isn't smart enough.
Finally, add an ASAN_RECOVER kconfig that enables the use of
"halt_on_error=0" in $ASAN_OPTIONS, which continues execution past the
first error.
Signed-off-by: Andy Ross <andyross@google.com>
Changing $(ARCH_DIR)/common/Kconfig to arch/common/Kconfig.
The use of ARCH_DIR at this place is wrong, as it suddenly requires out
of tree archs to support a common/Kconfig file, which may make no sense
to them.
If an out of tree arch wants to place common Kconfig code in a common
Kconfig file, that's their choice and they should source such file
themselves.
Instead just source the Zephyr arch common file directly.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
We have now:
- CPU_HAS_{D,I}CACHE: when the CPU has support for d-cache and i-cache
- {D,I}CACHE: to enable / disable d-cache and i-cache
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
A Cortex-M BusFault often arises from the execution of a function
pointer that got corrupted.
The Zephyr Cortex-M fault handler de-references the `$pc` in
`z_arm_is_synchronous_svc()` to determine if the fault was due to a
kernel oops (ARCH_EXCEPT). This can cause a BusFault if the pc itself
was corrupt. A BusFault from a HardFault will trigger ARM Cortex-M
"Lockup" preventing the Zephyr fault handler from running to
completion. This in turn, results in no fault handling information
getting dumped by the Zephyr fault handler.
To fix the issue, we can simply set the `CCR.BFHFNMIGN` bit prior to
the instruction address dereference which will cause the processor to
ignore the BusFault and return a value of 0x0 instead of entering
lockup. After the operation is complete, we clear `CCR.BFHFNMIGN` as
it would be unexpected for any other code in the fault handler to
trigger a fault.
The issue can be reproduced programmatically with:
```
void (*unaligned_func)(void) = (void (*)(void))0x50000001;
unaligned_func();
```
I bumped into this problem while debugging an issue on the nRF9160DK
(`west build --board nrf9160dk_nrf9160ns`) and confirmed that after
making this change I now see the full fault handler print:
```
[00:00:45.582,214] <err> os: Exception occurred in Secure State
[00:00:45.582,244] <err> os: ***** HARD FAULT *****
[...]
[00:00:45.583,984] <err> os: Current thread: 0x2000d340 (shell_uart)
[00:00:45.829,498] <err> fatal_error: Resetting system
```
Signed-off-by: Chris Coleman <chris@memfault.com>
Allow enabling FPU with TF-M with the following limitations:
- Only IPC mode is supported by TF-M.
- Disallow FPU hard ABI when building the NS application, the TF-M build
system does not pass the flags correctly to all dependencies.
Signed-off-by: Joakim Andersson <joakim.andersson@nordicsemi.no>
QEMU requires that the semihosting trap instruction sequence, which
consists of three uncompressed instructions, lie in the same page, and
refuses to interpret the trap sequence if these instructions are placed
across two different pages.
This commit adds 16-byte alignment requirement to the `semihost_exec`
function, which occupies 12 bytes, to ensure that the three trap
sequence instructions in this function are never placed across two
different pages.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
All SOC_ERET definitions expand to the mret instruction (used to return
from a trap: exception or interruption). The 'eret' instruction existed
in previous RISC-V privileged specs, but it doesn't seem to be used in
Zephyr (ref. RISC-V Privileged Architectures 3.2.2).
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
Some processors support Dual-redundant Core Lock-step
DCLS) topology but the processor still can be ran in
split-lock mode (by default or changed at flash time).
So, introduce config DCLS that is enabled by default if
config CPU_HAS_DCLS is set, it should be disabled if
processor is used in split-lock mode.
Signed-off-by: Dat Nguyen Duy <dat.nguyenduy@nxp.com>
ICI (Inter-Core Interrupt Unit) interrupts and priorities were hardcoded
in C files. This patch moves this information to Devicetree and updates
code to make use of it.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
Transfer the entry point and initial parameters in the callee_saved
struct rather than on the stack. This saves 48 byte stack per thread
and simplifies the logic.
Signed-off-by: Julius Barendt <julius.barendt@gaisler.com>
Execute data and instruction sync barriers after writing to SCTLR
to disable the MPU, to ensure the registers are set before
proceeding and that the new changes are seen by the instructions
that follow.
Signed-off-by: Manuel Arguelles <manuel.arguelles@nxp.com>
Execute data and instruction sync barriers after writing to SCTLR
to enable the MPU, to ensure the registers are set before
proceeding and that the new changes are seen by the instructions
that follow.
Signed-off-by: Manuel Arguelles <manuel.arguelles@nxp.com>
The simulator seems to drop garbage addresses (somewhere in the ROM it
looks like) into this SR at arbitrary times. I don't know if this is
a hardware exception handler that we can't turn off, or a simulator
bug, or what. But our code that assumes it will be cleared to zero or
valid is breaking. Set it every time in every context switch for now
pending someone figuring out what's going wrong.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
When compiling OpenAMP with Zephyr Cache Management, undefined references
are listed for all functions called with in the cache management
Signed-off-by: Ryan McClelland <ryanmcclelland@fb.com>
Move those defines and values back to headers. Kconfig is not a good
place for this, later this should move to DTS.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
MISRA C:2012 Rule 14.4 (The controlling expression of an if statement
and the controlling expression of an iteration-statement shall have
essentially Boolean type.)
Use `do { ... } while (false)' instead of `do { ... } while (0)'.
Use comparisons with zero instead of implicitly testing integers.
Use comparisons with NULL instead of implicitly testing pointers.
Use comparisons with NUL instead of implicitly testing plain chars.
This commit is a subset of the original auditable-branch commit:
5d02614e34a86b549c7707d3d9f0984bc3a5f22a
Signed-off-by: Simon Hein <SHein@baumer.com>
In interrupt chandler code we don't save full current task context
on stack (we don't save callee regs) before z_get_next_switch_handle()
call, but we passing _current to it, so z_get_next_switch_handle
saves current task to switch_handle, which means that this CPU
current task can be picked by other CPU before we fully store it
context on this CPU.
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Any project with Kconfig option CONFIG_LEGACY_INCLUDE_PATH set to n
couldn't be built because some files were missing zephyr/ prefix in
includes
Re-run the migrate_includes.py script to fix all legacy include paths
Signed-off-by: Tomislav Milkovic <milkovic@byte-lab.com>
The use of spsr_hyp is "UNPREDICTABLE" for the ARM Cortex-R52.
Some implements choose to implement the behavior, but it
should not be assumed.
Fixes#47330
Signed-off-by: Tobias Röhmel <tobias.roehmel@rwth-aachen.de>
We can use definitions provided by "standard CMSIS" to access
MEMFAULT/BUSFAULT/USGFAULT fields in CFSR.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
irq_lock() returns an unsigned integer key.
Generated by spatch using semantic patch
scripts/coccinelle/irq_lock.cocci
Signed-off-by: Johann Fischer <johann.fischer@nordicsemi.no>
Move scripts needed by the build system and not designed to be run
individually or standalone into the build subfolder.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Quoting from the SiFive Interrupt Cookbook [0]
CLIC vectored mode has a similar concept to CLINT vectored mode, where
an interrupt vector table is used for specific interrupts. However, in
CLIC vectored mode, the handler table contains the address of the
interrupt handler instead of an opcode containing a jump instruction.
When an interrupt occurs in CLIC vectored mode, the address of the
handler entry from the vector table is loaded and then jumped to in
hardware
So, when CLIC is present we must use IRQ_VECTOR_TABLE_JUMP_BY_ADDRESS
instead of IRQ_VECTOR_TABLE_JUMP_BY_CODE.
[0] https://starfivetech.com/uploads/sifive-interrupt-cookbook-v1p2.pdf
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
This commit adds icache and dcache maintenance functions
for aarch32.
Signed-off-by: Jamie Iles <quic_jiles@quicinc.com>
Signed-off-by: Dave Aldridge <quic_daldridg@quicinc.com>
Add a new API used by arch to implement suspend-to-RAM (S2RAM).
The API is composed by a single function to save the CPU context on
suspend.
A CPU context is the arch-specific set of registers that must be
preserved on power-off (in retained RAM) to be able to resume the
execution from the point it was suspended without going through the
whole kernel startup stage.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Buffer size must be decreased by one when non-zero to calculate the
right end address, and this must be checked for overflows.
Variables for region limit renamed for clarity since they may be
understood as the raw register values.
Signed-off-by: Manuel Arguelles <manuel.arguelles@nxp.com>
ARMv8-R aarch32 processor has support for
ARM PMSAv8-32. To add support for ARMv8-R we reuse the
ARMv8-M effort and change access to the different registers
such as rbar, rlar, mair, prselr.
Signed-off-by: Julien Massot <julien.massot@iot.bzh>
Signed-off-by: Manuel Arguelles <manuel.arguelles@nxp.com>
When CONFIG_INIT_STACKS is enabled all stacks should be filled with 0xaa
so that the thread analyzer can measure stack utilization, but the IRQ
stack was not filled and so `kernel stacks` on the shell would show that
the stack had been fully used and inferring an IRQ stack overflow
regardless of the IRQ stack size.
Fill the IRQ stack before it gets used so that we can have precise usage
reports.
Signed-off-by: Jamie Iles <quic_jiles@quicinc.com>
Signed-off-by: Dave Aldridge <quic_daldridg@quicinc.com>
There is no reason to have this script in a different place than all the
other python scripts. Move it.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
For vectored interrupts use the generated IRQ vector table instead of
relying on a custom-generated table.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
The whole mechanism of IRQ table generation is build around the
assumption that the IRQ vector table contains an array of addresses the
PC will be assigned to when the corresponding interrupt is triggered.
While this is correct for the majority of architectures (ARM, RISCV with
CLIC in vectored mode, etc...) this is not valid in general (for example
RISCV with CLINT/HLINT in vectored mode).
In this alternative format for the IRQ vector table, the pc will get
assigned by the hardware to the address of the vector table index
corresponding to the interrupt ID. From the vector table index, a
subsequent jump will occur from there to service the interrupt.
This means that the IRQ vector table contains an opcode that is a jump
instruction to a specific location instead of the address of the
location itself.
This patch is introducing support for this alternative IRQ vector table
format. The user can now select one format or the other one by acting on
IRQ_VECTOR_TABLE_JUMP_BY_ADDRESS or IRQ_VECTOR_TABLE_JUMP_BY_CODE
Kconfig symbols.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Removes the ability to enable the FPU with TF-M -- added in
PR #45906, and which is causing CI failures -- until a more
robust solution can be implemented for FPU support w/TF-M.
Signed-off-by: Kevin Townsend <kevin.townsend@linaro.org>
Following zephyr's style guideline, all if statements, including single
line statements shall have braces.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Following zephyr's style guideline, all if statements, including single
line statements shall have braces.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
The EFI console output call return with interrput enabled, it is a
firmware bug. And there was a solution that disabled interrupt it
return right away. But in some case the interrupt could happen
during the efi call context. If an interrupt was handled, a printk
call again will make it re-entried, or a swap might be happens.
This is suggested solution appiled for EFI console output:
1. Skip printk call when it is called in interrupt context.
2. Disable the schedule during the EFI call window.
Signed-off-by: Enjia Mai <enjia.mai@intel.com>
Add a minimal EFI console driver to support printf, this console driver
only supports console output. Otherwise the printf will not work.
Signed-off-by: Enjia Mai <enjia.mai@intel.com>
Some early RISC-V SoCs have a problem when an `mret` instruction is used
outside a trap handler.
After the latest Zephyr RISC-V huge rework, the arch_switch code is
indeed calling `mret` when not in handler mode, breaking some early
RISC-V platforms.
Optionally restore the old behavior by adding a new
CONFIG_RISCV_ALWAYS_SWITCH_THROUGH_ECALL symbol.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Targets with text or data addresses above the 4GB boundary may need to use
the large code model to ensure relocations in the linker work correctly.
Signed-off-by: Keith Packard <keithp@keithp.com>
This is really useful only for one case i.e. when testing against zero.
Do that test inline where it is needed and make the rest of the code
independent from the actual numerical value being tested to make code
maintenance easier if/when new cases are added.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
MISRA C:2012 Rule 21.13 (Any value passed to a function in <ctype.h>
shall be representable as an unsigned char or be the value EOF).
Functions in <ctype.h> have undefined behavior if they are called with
any other value. Callers affected by this change are not prepared to
handle EOF anyway. The addition of these casts avoids the issue
and does not result in any performance penalty.
Signed-off-by: Abramo Bagnara <abramo.bagnara@bugseng.com>
Signed-off-by: Simon Hein <SHein@baumer.com>
Allow the application to enable the FPU when TF-M has been enabled.
Pass the correct compilation flags according to the TF-M integration
guide.
Signed-off-by: Joakim Andersson <joakim.andersson@nordicsemi.no>
Enable single-threaded support for the arm64 archtecture.
This mode of execution is supported on an soc under
development and is validated regularly.
Signed-off-by: Eugene Cohen <quic_egmc@quicinc.com>
In performing a double check of Zephyr arm64 MMU config
against edk2, a different in the programming of the
Translation Control Register (TCR) was found. TCR.TG[1]
should be set to address Cortex-A57 erratum 822227:
"Using unsupported 16K translation granules might cause
Cortex-A57 to incorrectly trigger a domain fault"
Signed-off-by: Eugene Cohen <quic_egmc@quicinc.com>
By default ARCH_IRQ_VECTOR_TABLE_ALIGN and ARCH_SW_ISR_TABLE_ALIGN are
set to 0. Use a more proper value.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
The generation of the software ISR table and the IRQ vector table
(respectively generated by CONFIG_GEN_SW_ISR_TABLE and
CONFIG_GEN_IRQ_VECTOR_TABLE) should (in theory) go through three stages:
1. A placeholder table is generated in arch/common/isr_tables.c and
placed in an orphaned .gnu.linkonce.{irq_vector_table, sw_isr_table}
section
2. The real table is generated by arch/common/gen_isr_tables.py (creating
the build/zephyr/isr_tables.c file)
3. The real table is un-orphaned by moving it in a proper section with a
proper alignment
While all the steps are done automatically for the software ISR table,
for the IRQ vector table each architectures must take care of modiying
its own linker script to place somewhere the generated IRQ vector table
(basically step 3 is missing).
This is currently only done for 2 architectures: Cortex-M (ARMv7) and
ARC. But when another architecture tries to use the IRQ vector table,
the linker complains about that. For example:
Linking C executable zephyr/zephyr.elf
riscv64-zephyr-elf/bin/ld.bfd: warning: orphan section
`.gnu.linkonce.irq_vector_table' from
`zephyr/CMakeFiles/zephyr_final.dir/isr_tables.c.obj' being placed in
section `.gnu.linkonce.irq_vector_table'
In this patch we introduce a new CONFIG_ARCH_IRQ_VECTOR_TABLE_ALIGN to
support the architectures requiring a special alignment for the IRQ
vector table and we also introduce a way to automatically place the IRQ
vector table in place in the same way it is done for the ISR software
table.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
On platforms where reset vector catch is not possible
it is useful to have a compile-time option to spin
at the reset vector allowing a debugger to be attached
and then to manually resume execution.
Define a config option for arm64 to spin at the
reset vectdor so a debugger can be attached.
Signed-off-by: Eugene Cohen <quic_egmc@quicinc.com>
Under no circumstances the generated IRQ vector table can and should
contain NULL values. This is correctly enforced at generation time by
the gen_isr_tables.py script making the existence of the ISR_WRAPPER
define useless.
The enforced behaviour is:
- When the ISR software table exists defaults to _isr_wrapper
- Otherwise defaults to z_irq_spurious
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Retrieve the pmpaddr value matching the last global PMP slot and add it
to the per-thread m-mode and u-mode entry array. Even if that value is
not written out again on thread context switch, that value can still be
used by set_pmp_entry() to attempt a single-slot TOR mapping with it.
Nicely abstract this with the new z_riscv_pmp_thread_init() where the
PMP_M_MODE(thread) and PMP_U_MODE(thread) argument generators can be
used.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Adds compatibility with Intel ADSP GDB from Zephyr SDK and
from Cadence toolchain to coredump_gdbserver.py.
Adds CAVS 15-25 (APL) register definitions. Implements
handle_register_single_read_packet to serve ADSP GDB
p packets.
Prevents BSA from changing between stack dump printout
and coredump by taking lock. Observed to be necessary for
accurate results on slower simulated platforms.
Signed-off-by: Lauren Murphy <lauren.murphy@intel.com>
Triggers CPU exception with illegal instruction when z_except_reason
is called (e.g. in k_panic, k_oops). Creates exception stack frame
for use by coredump. Adds unique cause code for ARCH_EXCEPT. Disables
test case failure for qemu_xtensa.
Without an ARCH_EXCEPT implementation, z_except_reason calls
z_fatal_error directly with a null ESF and bypasses
xtensa_excint1_c's error logging. An ESF is required to coredump.
Signed-off-by: Lauren Murphy <lauren.murphy@intel.com>
A QEMU bug may create bad transient PMP representations causing
false access faults to be reported. Work around it by setting
pmp registers to zero from the update start point to the end
before updating them with new values.
The QEMU fix is here with more details about this bug:
https://lists.gnu.org/archive/html/qemu-devel/2022-06/msg02800.html
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
This reverts the bulk of commit c8bfc2afda ("riscv: make
arch_is_user_context() SMP compatible") and replaces it with a flag
stored in the thread local storage (TLS) area, therefore making TLS
mandatory for userspace support on RISC-V.
This has many advantages:
- The tp (x4) register is already dedicated by the standard for this
purpose, making TLS support almost free.
- This is very efficient, requiring only a single instruction to clear
and 2 instructions to set.
- This makes the SMP case much more efficient. No need for funky
exception code any longer.
- SMP and non-SMP now use the same implementation making maintenance
easier.
- The is_user_mode variable no longer requires a dedicated PMP mapping
and therefore freeing one PMP slot for other purposes.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
5f65dbcc9dab3d39473b05397e05.
The tp (x4) register is neither caller nor callee saved according to
the RISC-V standard calling convention. It only has to be set on thread
context switching and is otherwise read-only.
To protect the kernel against a possible rogue user thread, the tp is
also re-set on exception entry from u-mode.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Logging v1 has been removed and log_strdup wrapper function is no
longer needed. Removing the function and its use in the tree.
Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
MISRA C:2012 Rule 8.2 (Function types shall be in prototype form with
named parameters.)
Added missing parameter names.
Signed-off-by: Abramo Bagnara <abramo.bagnara@bugseng.com>
For some reasons RISCV is the only arch where the vector table entry is
called __irq_wrapper instead of _isr_wrapper. This is not only a
cosmetic change but Zephyr expects the common ISR handler to be called
_isr_wrapper (for example when generating the IRQ vector table).
Change it.
find ./ -type f -exec sed -i 's/__irq_wrapper/_isr_wrapper/g' {} \;
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
This commit updates all deprecated `K_KERNEL_STACK_ARRAY_EXTERN` macro
usages to use the `K_KERNEL_STACK_ARRAY_DECLARE` macro instead.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
This commit corrects all `extern K_THREAD_STACK_DEFINE` macro usages
to use the `K_THREAD_STACK_DECLARE` macro instead.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
This commit corrects all `extern K_KERNEL_STACK_ARRAY_DEFINE` macro
usages to use the `K_KERNEL_STACK_ARRAY_DECLARE` macro instead.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
This commit selectively disables the infinite recursion warning
(`-Winfinite-recursion`), which may be reported by GCC 12 and above,
for the `disable_table` function because no actual infinite recursion
will occur under normal circumstances.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
arch_mem_map() on ARM64 is currently not supporting the K_MEM_PERM_USER
parameter so we cannot allocate userspace accessible memory using the
memory helpers. Fix this.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Expose the Xtenesa CCOUNT timing register (the lowest level CPU cycle
counter) using the arch_timing_*() API.
This is the simplest possible way to get this working. Future work
might focus on moving the rate configuration into devicetree in a
standard way, integrating with the platform clock driver on intel_adsp
such that the reported cycle rate tracks runtime changes (though IIRC
this is not a SOF requirement), and adding better test coverage to the
timing layer, which right now isn't exercised anywhere but in
benchmarks.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
<soc.h> has been traditionally been used as a proxy to HAL headers,
register definitions, etc. Nowadays, <soc.h> is anarchy. It serves a
different purpose depending on the SoC. In some cases it includes HALs,
in some others it works as a header sink/proxy (for no good reason), as
a register definition when there's no HAL... To make things worse, it is
being included in code that is, in theory, non-SoC specific.
This patch is part of a series intended to improve the situation by
removing <soc.h> usage when not needed, and by eventually removing it.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
<soc.h> has been traditionally been used as a proxy to HAL headers,
register definitions, etc. Nowadays, <soc.h> is anarchy. It serves a
different purpose depending on the SoC. In some cases it includes HALs,
in some others it works as a header sink/proxy (for no good reason), as
a register definition when there's no HAL... To make things worse, it is
being included in code that is, in theory, non-SoC specific.
This patch is part of a series intended to improve the situation by
removing <soc.h> usage when not needed, and by eventually removing it.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
The NS16550 UART base address was hardcoded in <soc.h> headers. This
bypasses the console choice defined in Devicetree. Hardcoded hardware
choices must be avoided now that DT is in place.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
In preparation for the support of RV32E optimize a bit the t* registers
usage limiting that to t{0-2}.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
This patch is doing several things:
- Core ISA and extension Kconfig symbols have now a formalized name
(CONFIG_RISCV_ISA_* and CONFIG_RISCV_ISA_EXT_*)
- a new Kconfig.isa file was introduced with the full set of extensions
currently supported by the v2.2 spec
- a new Kconfig.core file was introduced to host all the RISCV cores
(currently only E31)
- ISA and extensions settings are moved to SoC configuration files
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
Adds few missing zephyr/ prefixes to leftover #include statements that
either got added recently or were using double quote format.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
When returning from z_riscv_switch, depending on whether the thread that
has just been swapped in was earlier swapped out synchronously (i.e. via
regular function call) or asynchronously (i.e. via exception/irq) we
will return to arch_switch() or __irq_wrapper respectively. Comment this
fact for clarity.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
After the introduction of arch_switch() in #43085, ECALL is no longer
used for context switching by default, so remove the comment stating so.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
The xtensa interrupt return path was forgetting to check the nested
interrupt state and calling into the scheduler to select the context
to which to return, which of course is completely wrong. We MUST
return to the ISR we interrupted.
In fact in practice this was only visible in the case of a nested
interrupt that causes a context switch, otherwise the "interrupted"
argument just gets returned and things work. In particular, it can
happen when the nested context is a fatal exception that aborts the
current thread, which is how this was discovered. The timing required
to see this on live interrupts on real applications is likely to have
been extremely difficult to detect.
Fixes#45779
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
ARCH_HAS_USERSPACE and ARCH_HAS_STACK_PROTECTION are direct functions
of RISCV_PMP regardless of the SoC.
PMP_STACK_GUARD is a function of HW_STACK_PROTECTION (from
ARCH_HAS_STACK_PROTECTION) and not the other way around.
This allows for tests/kernel/fatal/exception to test protection against
various stack overflows based on the PMP stack guard functionality.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
_current_cpu->irq_stack is not yet initialized when this is executed on
CPU 0. Also the guard area is outside of CONFIG_ISR_STACK_SIZE now
e.g. it is within the K_KERNEL_STACK_RESERVED area at the start of
the buffer. So simply use z_interrupt_stacks[] directly instead.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
A separate privileged stack is used when CONFIG_GEN_PRIV_STACKS=y. The
main stack guard area is no longer needed and can be made available to
the application upon transitioning to user mode. And that's actually
required if we want a naturally aligned power-of-two buffer to let the
PMP map a NAPOT entry on it which is the whole point of having this
CONFIG_PMP_POWER_OF_TWO_ALIGNMENT option in the first place.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The StackGuard area is used to save the esf and run the exception code
resulting from a StackGuard trap. Size it appropriately.
Remove redundancy, clarify documentation, etc.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The init stack of the secondary core should use KERNEL_STACK_BUFFER + sz
Using Z_THREAD_STACK_BUFFER will calculate the wrong stack size.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
The current SMP boot code doesn't consider that the cores can boot at
the same time. Possibly, more than one core can boot into primary core
boot sequence. Fix it by using the atomic operation to make sure only
one core act as the primary core.
Correspondingly, sgi_raise_ipi should transfer CPU id to mpidr.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Add the ability to have multiple irq priority levels which are not
masked by irq_lock() by adding CONFIG_ZERO_LATENCY_LEVELS.
If CONFIG_ZERO_LATENCY_LEVELS is set to a value > 1 then multiple zero
latency irqs are reserved by the kernel (and not only one). The priority
of the zero-latency interrupt can be configured by IRQ_CONNECT.
To be backwards compatible the prio argument in IRQ_CONNECT is still
ignored and the target prio set to zero if CONFIG_ZERO_LATENCY_LEVELS
is 1 (default).
Implements #45276
Signed-off-by: Christoph Coenen <ccoenen@baumer.com>
Ensure callee registers included in coredump.
Push callee registers onto stack and pass as param to
z_do_kernel_oops for CONFIG_ARMV7_M_ARMV8_M_MAINLINE
when CONFIG_EXTRA_EXCEPTION_INFO enabled.
Signed-off-by: Mark Holden <mholden@fb.com>
Now we define PROPERTY_OUTPUT_FORMAT (which is used for
binutils) only for ARCv3 32 bit. Let's define it for all
ARC elf formats instead of relying on default values.
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Provide required compiler/assembler options for building with mwdt
toolchain for ARCv3 64 bit.
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Debugger plugins use the `z_sys_post_kernel` variable to detect whether
the kernel is currently running, and hence whether any threads exist. As
this is just a standard variable however, after a reset the initial
value of this variable is whatever it was before reset (true) until the
bss section is zeroed halfway through `z_arm_prep_c`. Debuggers are
therefore unable to differentiate between a normally running application
and the very first stages of the boot process.
Clearing this variable as the first action upon reset allows debuggers
to display the correct thread state after the first 3 instructions have
run.
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
Fix writing of ICC_SRE_EL3 to or-in bits to align
with original intent to read-modify-write this
register.
Also disable FIQ and IRQ bypass so interrupt delivery
occurs through GIC. Platforms may choose to override
this behavior in z_arm64_el3_plat_init implementations.
Remove ICC_SRE_EL3 config from viper and qemu since
this is now handled in the arm64 arch core.
Signed-off-by: Eugene Cohen <quic_egmc@quicinc.com>
The gen_usr_tables scripts were not updated to make use of the
<zephyr/...> include prefix, fix this.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
Assembler files were not migrated with the new <zephyr/...> prefix.
Note that the conversion has been scripted, refer to #45388 for more
details.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
In order to bring consistency in-tree, migrate all arch code to the new
prefix <zephyr/...>. Note that the conversion has been scripted, refer
to zephyrproject-rtos#45388 for more details.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
When legacy mode is enabled, Zephyr includes both include/ and
include/zephyr. Allow the zefi.py script to accept multiple include
paths to cover this scenario.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
This adds lazy floating point context switching. On svc/irq entrance,
the VFP is disabled and a pointer to the exception stack frame is saved
away. If the esf pointer is still valid on exception exit, then no
other context used the VFP so the context is still valid and nothing
needs to be restored. If the esf pointer is NULL on exception exit,
then some other context used the VFP and the floating point context is
restored from the esf.
The undefined instruction handler is responsible for saving away the
floating point context if needed. If the handler is in the first
irq/svc context and the current thread uses the VFP, then the float
context needs to be saved. Also, if the handler is in a nested context
and the previous context was using the FVP, save the float context.
Signed-off-by: Bradley Bolen <bbolen@lexmark.com>
This commit updates the Cortex-R reset routine to initialise
(synchronise) the VFP D16-D31 registers when Dual-redundant Core
Lock-step (DCLS) is enabled.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
Grouping the FPU registers together will make adding FPU support for
Cortex-A/R easier later. It provides the ability to get the sizeof and
offsetof FPU registers easier.
Signed-off-by: Bradley Bolen <bbolen@lexmark.com>
Cortex-A/R use a descending stack frame and the hardware does not help
with the stacking. This led to some less than desirable workarounds in
the exception code where the basic stack frame was saved twice.
Rearranging the order of the exception stack frame removes that problem
and provides a clearer path to saving CPU context in a fully descending
manner.
Signed-off-by: Bradley Bolen <bbolen@lexmark.com>
This commit adds the unified floating-point configuration symbols for
the ARM architectures.
These configuration symbols allow specification of the floating-point
coprocessors, such as VFP (also known as FP for Cortex-M) and NEON,
for the ARM architectures.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
When building with CONFIG_SCHED_CPU_MASK_PIN_ONLY we can assume that a
thread will always be executed in a same CPU and consequently skip the
cache invalidation.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Where we have access to a bootstrap UEFI environment, it's productive
to use that console as the default printk handler. That avoids the
bringup hassle of trying to configure UART settings blindly, as has
been customary. It also emits nice text to the framebuffer on devices
with no serial port or other debug harness at all.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
The NAPOT mode isn't computed properly in qemu when the full address
range is covered. Let's hardcode the value that the qemu code checks
explicitly until the appropriate fix is applied to qemu itself.
For reference, here's the qemu patch:
https://lists.gnu.org/archive/html/qemu-devel/2022-04/msg00961.html
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Overall diffstat with the new PMP code in place:
18 files changed, 866 insertions(+), 1372 deletions(-)
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Add the appropriate hooks effectively replacing the old implementation
with the new one.
Also the stackguard wasn't properly enforced especially with the
usermode combination. This is now fixed.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The idea here is to compute the PMP register set on demand i.e. upon
scheduling in the affected threads, and only if changes occurred.
A simple sequence number is used to stay in sync with the latest update.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Stackguard uses the PMP to prevents many types of stack overflow by
making any access to the bottom stack area raise a CPU exception. Each
thread has its set of precomputed PMP entries and those are written to
PMP registers at context switch time.
This is the code to set it up. It will be connected later.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
This is the core code to manage PMP entries with only the global entries
initialisation for now. It is not yet linked into the build.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
MDB debugger may modify debug_select and debug_mask registers
on start, so we can't rely on debug_select reset value.
Let's set correct value on primary CPU without reading initial
value from debug_select.
Internal ID: P10019563-50516
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
Set TP in exception context so that it gets loaded into the CPU when
first running the thread. Set TP for secondary cores to related idle TLS
area.
Signed-off-by: Keith Packard <keithp@keithp.com>
V7-A also supports TPIDRURO, so go ahead and use that for TLS, enabling
thread local storage for the other ARM architectures.
Add __aeabi_read_tp function in case code was compiled to use that.
Signed-off-by: Keith Packard <keithp@keithp.com>
Making context switch cache-coherent in SMP is hard. The
KERNEL_COHERENCE handling was conservatively invalidating the stack
region of a thread that was being switched in. This was because it
might have (1) run on this CPU in the past, but (2) run most recently
on a different CPU. In that case we might have stale data still in
our local dcache!
But this has performance impact in the (very common!) case of a thread
being switched out briefly and then back in (e.g. k_sleep() for a
small duration). It will come back having lost all of its cached
stack context, and will have to fetch all that information back from
shared SRAM!
Treat this by tracking a "last_cpu" for each thread in the arch part
of the thread struct. If we're coming back to the same CPU we left,
we know we can skip the invalidate.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Commit d8f186aa4a ("arch: common: semihost: add semihosting
operations") encapsulated semihosting invocation in a per-arch
semihost_exec() function. There is a fixed register variable declaration
for the return value but this variable is not listed as an output
operand to respective inline assembly segments which is an error.
This is not reported as such by gcc and the generated code is still OK
in those particular instances but this is not guaranteed, and clang
does complain about such cases.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
This will generate profile data that can be analyzed using gprof. When
you build the application (currently for native_posix only), after
running the application you will get a file "gmon.out" with the call
graph which can be processed with gprof:
gprof build/zephyr/zephyr.exe gmon.out > analysis.txt
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Add an API that utilizes the ARM semihosting mechanism to interact with
the host system when a device is being emulated or run under a debugger.
RISCV is implemented in terms of the ARM implementation, and therefore
the ARM definitions cross enough architectures to be defined 'common'.
Functionality is exposed as a separate API instead of syscall
implementations (`_lseek`, `_open`, etc) due to various quirks with
the ARM mechanisms that means function arguments are not standard.
For more information see:
https://developer.arm.com/documentation/dui0471/m/what-is-semihosting-
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
impl
Control the usage of semihosting with a dedicated symbol, instead of
implying semihosting from the usage of `SEMIHOST_CONSOLE`. This allows
semihosting to be used without the semihost console.
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
This PR allows the user to add symbols to the ramfunc
section. The use for this could be as follows:
zephyr_linker_sources_ifdef(CONFIG_ARCH_HAS_RAMFUNC_SUPPORT
RAMFUNC_SECTION
quick_access_code.ld
)
quick_access_code.ld (as shown below) can define additional
symbols to go into the ramfunc section
. = ALIGN(4);
KEEP(*(CodeQuickAccess))
Signed-off-by: Mahesh Mahadevan <mahesh.mahadevan@nxp.com>
When "eager FPU sharing" mode is enabled, FPU registers must be
initialised at the time of thread creation because the floating-point
context is always active and no further FPU initialisation is performed
later.
Note that, in case of the "lazy FPU sharing" mode, floating-point
context is inactive by default and the FPU is initialised when the
first floating-point instruction is executed.
Refer to the issue #44902 for more details.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
With GCC 11 now supporting low overhead branching in ARMv8.1, ASM "LE"
(loop-end) instructions would trigger an INVSTATE hard-fault after
FPSCR was set to 0. This was due to the FPSCR getting a new field in
ARMv8.1. LTPSIZE is now set to it's reset value of Tail predication not
applied.
Signed-off-by: Ryan McClelland <ryanmcclelland@fb.com>
The Cache is an optional configuration of both the ARM Cortex-M7 and
Cortex-M55. Previously, it was just checking that it was just an M7
rather than knowing that the CPU actually was built with the cache.
Signed-off-by: Ryan McClelland <ryanmcclelland@fb.com>
Platform specific functions necessary to enable this feature were
implemented (z_xtensa_ptr_executable() and
z_xtensa_stack_ptr_is_sane() for Intel ADSP platforms.
Current implementation just ensures stack pointer and program counter
are within relevant areas defined in the linker scripts, without going
too fine grained.
Also, `.iram1` section, used by the backtrace code, also added to
Intel ADSP linker script.
Finally, update west manifest to use up-to-date SOF, which contains a
patch to fix build issues related to the linker changes.
Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
This commit changes the CODE_DATA_RELOCATON dependency by
adding CPU_AARCH32_CORTEX_R next to CPU_CORTEX_M.
Signed-off-by: Mateusz Sierszulski <msierszulski@antmicro.com>
Cortex-M code is the only flavor that supports switching between secure
and non-secure state so make sure this kconfig only applies to it.
Signed-off-by: Bradley Bolen <bbolen@lexmark.com>
Commit a2cfb8431d ("arch: arm: Add code for swapping threads between
secure and non-secure") changed the mode variable in the _thread_arch to
be defined by ARM_STORE_EXC_RETURN or USERSPACE. The generated offset
define for mode was enabled by FPU_SHARING or USERSPACE. This broke
Cortex-R with FPU, but with ARM_STORE_EXC_RETURN disabled. Reconcile
the checks.
Signed-off-by: Bradley Bolen <bbolen@lexmark.com>
Under QEMU and SeaBIOS, everything gets to be printed
immediately after "Booting from ROM.." as there is no newline.
This prevents parsing QEMU console output for the very first
line where it needs to match from the beginning of the line.
So add a dummy newline here so the next output is at
the beginning of a line.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Moving this option to the subdirectory for boards might make it easier
to find, and will keep it next to some other board-related Kconfig
options set in the same file.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
In ARM parlance, the subroutine call return address is stored in the
"link register" or simply lr. Refer to it as lr which is clearer than
the anonymous x30 designation.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
ARM64 supports more memory mapping types for device memory (nGnRnE,
nGnRE, GRE), add these mapping support for os common mapping API
function z_phys_map().
Signed-off-by: Jiafei Pan <Jiafei.Pan@nxp.com>
This is a strange one: The printing code pushes a floating point
register, and is called during the mpu falt. If the floating point
registers are lazily stacked, this fp push can cause another mpu
fault to be pending during the current mpu fault, and tail chained
without returning to PendSV. Since we're already cleaning up the
fp execption reason, we might as well also clean up thisp pending,
spurious mpu exception.
Signed-off-by: Jimmy Brisson <jimmy.brisson@linaro.org>
If an SVC was pending during the stack overflow, it will run
after the return of the memory manage fault. To the SVC's misfortune of
the SVC handler, the it's invariant, that PSP point to the
hardware-stacked context is no longer valid. When the user has a
k_sys_fatal_error_handler that tries to kill the thread that caused a
stack overflow, this manifests as the svc reading the memory of whatever
is on the stack after being adjusted by the mem manage fault handler, and
that leads to unending, spurious hard faults, locking up the system.
This patch prevents that.
Signed-off-by: Jimmy Brisson <jimmy.brisson@linaro.org>
New KConfig options for 'A' and 'M' RISC-V extensions have been
added. These are used to configure the '-march' string used by GCC
to produce a compatible binary for the requested RISC-V variant.
In order to maintain compatibility with all currently defined SoC,
default the options for HW mul / Atomics support to 'y', but allow
them to be overridden for any SoC which does not support these.
I tested this change locally via twister agaisnt a few RISC-V platforms
including some 32bit and 64bit. To verify the 4 possibilities of Atomics
& HW Mul: (No, No), (No, Yes), (Yes, No), (Yes, Yes -- current behavior),
I used an out-of-tree GCC (xPack RISC-V GCC) which has multilib support
for rv32i, rv32ia, rv32ima to test against our out-of-tree Intel Nios V/m
processor in HW. The Zephyr SDK RISCV GCC currently does not contain
multilib support for all variants exposed by these new KConfig options.
Signed-off-by: Nathan Krueger <nathan.krueger@intel.com>
In case of EFI, efi_init must be called before initializing early
serial: if that one as X86_SOC_EARLY_SERIAL_PCIDEV defined, its pcie
access will try to initialise pcie mmio access which one will try to
find an ACPI table. At this point, calling ACPI API prior to initialize
EFI will make RSDP looked up already... and since it cannot find it
without EFI being initialized first, ACPI is then broken.
Just moving early serial to initialize after multiboot/efi being setup.
Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
If such table pointer is present with EFI system table, this will speed
up ACPI initialization later on.
Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
As for Multiboot, let prep_c be aware of EFI boot.
In the futur, EFI will pass an argument to it.
Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
In order to mitigate at runtime whether it booted on multiboot or EFI,
let's introduce a dedicated x86 cpu argument structure which holds the
type and the actual pointer delivered by the method (multiboot_info, or
efi_system_table)
Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
Just a dummy function will do.
When enabled, the code does not need the #ifdef as cmake is handling
this properly already. This was also the wrong CONFIG_ used there
anyway.
Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
The incorrect sequence will cause the thread cannot be aborted in the
ISR context. The following test case failed:
tests/kernel/fatal/exception/kernel.common.stack_sentinel.
The stack sentinel detects the stack overflow as normal during a timer
ISR exit. Note that, currently, the stack overflow detection is behind
the context switch checking, and then the detection will call svc to
raise a fatal error resulting in increasing the nested counter(+1). At
this point, it needs a context switch to finally abort the thread.
However, after the fatal error handling, the program cannot do a context
switch either during the svc exit[1], or during the timer ISR exit[2].
[1] is because the svc context is in an interrupt nested state (the
nested counter is 2).
[2] is because the current point (after svc context pop out) is right
behind the switch checking.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
This is painful. There is no way for u-mode code to know if we're
currently executing in u-mode without generating a fault, besides
stealing a general purpose register away from the standard ABI
that is. And a global variable doesn't work on SMP as this must be
per-CPU and we could be migrated to another CPU just at the right
moment to peek at the wrong CPU variable (and u-mode can't disable
preemption either).
So, given that we'll have to pay the price of an exception entry
anyway, let's at least make it free to privileged threads by using
the mscratch register as the non-user context indicator (it must
be zero in m-mode for exception entry to work properly). In the
case of u-mode we'll simulate a proper return value in the
exception trap code. Let's settle on the return value in t0
and omit the volatile to give the compiler a chance to cache
the result.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
To do so efficiently on systems without the mul instruction, we use
shifts and adds which is faster and sometimes smaller than a plain loop.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Stop using &_kernel as this is not SMP friendly. Let's use s0 (after
preserving its content) to hold ¤t_cpu instead so it won't have
to be reloaded each time it is needed. This will be even more relevant
when SMP support is added.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Rely on mstatus rather than thread->base.user_options since it is always
up to date (updated by z_riscv_switch) to simplify the code and be SMP
proof. Also carry over SF_INIT to the mstatus being restored in case
it was changed in the mean time.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The move to arch_switch() is a prerequisite for SMP support.
Make it optimal without the need for an ECALL roundtrip on every
context switch. Performance numbers from tests/benchmarks/sched:
Before:
unpend 107 ready 102 switch 188 pend 218 tot 615 (avg 615)
After:
unpend 107 ready 102 switch 170 pend 217 tot 596 (avg 595)
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
This is a per-thread register that gets updated only when context
switching. No need to load and save it on every exception entry.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The minimum stack alignment is 16. Therefore, the stack space to store
a struct __esf object must be rounded up to the next 16-byte boundary.
It is not sufficient to do the rounding on the __z_arch_esf_t_SIZEOF
definition. When the stack is constructed in arch_new_thread() it is
also necessary to do the rounding there too.
Let's make the structure itself carry the alignment attribute instead to
make it work in all cases.
While at it, remove the unused _K_THREAD_NO_FLOAT_SIZEOF definition.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Get rid of all those global variables and IRQ locking.
Use the regular IRQ exit path to let tests validate preemption properly.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Complete revamp of the exception entry code, including syscall handling.
Proper syscall frame exception trigger. Many correctness fixes, hacks
removal, etc. etc.
I tried to make this into several commits, but this stuff is all
inter-related and a pain to split.
The diffstat summary:
14 files changed, 250 insertions(+), 802 deletions(-)
Binary size (before):
text data bss dec hex filename
1104 0 0 1104 450 isr.S.obj
64 0 0 64 40 userspace.S.obj
Binary size (after):
text data bss dec hex filename
600 0 0 600 258 isr.S.obj
36 0 0 36 24 userspace.S.obj
Run of samples/userspace/syscall_perf (before):
*** Booting Zephyr OS build zephyr-v3.0.0-325-g3748accae018 ***
Main Thread started; qemu_riscv32
Supervisor thread started
User thread started
Supervisor thread(0x80010048): 384 cycles 509 instructions
User thread(0x80010140): 77312 cycles 77437 instructions
Run of samples/userspace/syscall_perf (after):
*** Booting Zephyr OS build zephyr-v3.0.0-326-g4c877a2753b3 ***
Main Thread started; qemu_riscv32
Supervisor thread started
User thread started
Supervisor thread(0x80010048): 384 cycles 509 instructions
User thread(0x80010138): 7040 cycles 7165 instructions
Yes, that's more than a 10x speed-up!
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Same rationale as preceding commit. Let's create pseudo-instructions in
assembly scope to make the code more uniform and readable.
Furthermore the definition of COPY_ESF_FP() was wrong as the width of
floating point registers vary not according to CONFIG_64BIT but
CONFIG_CPU_HAS_FPU_DOUBLE_PRECISION. It is therefore wrong to use
lr/sr (previously RV_OP_LOADREG/RV_OP_STOREREG) and a regular temporary
register to transfer such content.
Note: There are far more efficient ways to copy FP context around but
such optimisations will come separately.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Those are prominent enough that having RV_OP_LOADREG and RV_OP_STOREREG
shouting at you all over the place is rather unpleasant and bad taste.
Let's create pseudo-instructions of our own with assembler macros
rather than preprocessor defines and only in assembly scope.
This makes the asm code way more uniform and readable.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The thread->base.user_options field is an uint8_t. Access it using lb.
A "copy" of it is made into __esf.fp_state. Make that field an uint8_t
too and access it with lb/sb.
_callee_saved.fcsr is an uint32_t. Access it with lw/sw.
Ditto for is_user_mode.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
This reverts commit 8686ab5472.
The purpose of this commit will be reintroduced later on top of
a cleaner codebase.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
This reverts commit be28de692c.
The purpose of this commit will be reintroduced later on top of
a cleaner codebase.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
This reverts commit b0458201cc.
The purpose of this commit will be reintroduced later on top of
a cleaner codebase.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
ARMv8-R allows to set the vector table address using VBAR
register, so there is no need to relocate it.
Move away vector_table setting from reset.S and move it to
relocate vector table function as it's done for Cortex-M
CPU.
Signed-off-by: Julien Massot <julien.massot@iot.bzh>
It is not necessary to go through the full exception exit code.
This is simpler, smaller and faster.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Make it optimal without the need for an SVC/exception roundtrip on
every context switch. Performance numbers from tests/benchmarks/sched:
Before:
unpend 85 ready 58 switch 258 pend 231 tot 632 (avg 699)
After:
unpend 85 ready 59 switch 115 pend 138 tot 397 (avg 478)
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Get rid of all those global variables and scheduler locking.
Use the reguler IRQ exit path to let tests properly validate preemption.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Add CONFIG_SMP to fvp_baser_aemv8r_smp board.
Fix compile warnings by adding missing header file in arm_mpu.c.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
This commit mainly fixes the broadcast_ipi issue when one core broadcast
ipi to other cores using gic_raise_sgi. The issue doesn't affect the
functionality of Zephyr SMP but will happen when Zephyr runs on Xen.
Suppose Xen provides 4 CPUs to the Zephyr guest, for example, when cpu0
broadcasts ipi to the rest of the cores, the mask should be 0xE(0b1110),
but for now, Zephyr will send 0xFFFE. So for Xen, it will receive a
target list containing many invalid CPUs which don't exist. My solution
is: to generate the target list according to the online CPUs.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
The ARMv8-R processors always boot into Hyp mode (EL2)
To enter EL1:
Program the HACTLR register because it defaults
to only allowing EL2 accesses. HACTLR controls
whether EL1 can access memory region registers and CPUACTLR.
Program the SPSR before entering EL1.
Other registers default to allowing accesses at EL1 from reset.
Set VBAR to the correct location for the vector table.
Set ELR to point to the entry point of the EL1 code and call ERET.
Signed-off-by: Julien Massot <julien.massot@iot.bzh>
Improve code by using DEVICE_DT_GET_ONE instead of device_get_binding,
since the intel_vt_d device instance can be obtained at compile time.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
According to Kconfig guidelines, boolean prompts must not start with
"Enable...". The following command has been used to automate the changes
in this patch:
sed -i "s/bool \"[Ee]nables\? \(\w\)/bool \"\U\1/g" **/Kconfig*
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
In the Armv8R AArch64 profile[1], the Armv8R AArch64 is always in secure
mode. But the FVP_BaseR_AEMv8R before version 11.16.16 doesn't strictly
follow this rule. It still has some non-secure registers
(e.g. CNTHP_CTL_EL2).
Since version 11.16.16, the FVP_BaseR_AEMv8R has fixed this issue. The
CNTHP_XXX_EL2 registers have been changed to CNTHPS_XXX_EL2. So the
FVP_BaseR_AEMv8R (version >= 11.16.16) cannot boot Zephyr. This patch
will fix it.
[1] https://developer.arm.com/documentation/ddi0600/latest/
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Change-Id: If986f34dc080ae7a8b226bba589b6fe616a4260b
Use CLINT to send interrupts to another CPU. SMP support is kinda
incomplete without it.
This patch only enables it for riscv-privilege platforms - specifically,
"virt" one.
Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
Secondary CPUs are now initialised and made available to the system. If
the system has more CPUs than configured via CONFIG_MP_NUM_CPUS, those
are still left looping as before.
Some implementations of `soc_interrupt_init` also changed to use
`arch_irq_lock` instead of `irq_lock`.
Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
Enable `arch_switch()` as preparation for SMP support. This patch
doesn't try to keep support for old style context swap - only switch
based swap is supported, to keep things simple.
A fair amount of refactoring was done in this patch, specially regarding
the code that decides what to do about the ISR. In RISC-V, ECALL
instructions are used to signalize several events, such as user space
system calls, forced syscall, IRQ offload, return from syscall and
context switch. All those handled by the ISR - which also handles
interrupts. After refactor, this "dispatching" step is done at the
beginning of ISR (just after saving generic registers).
As with other platforms, the thread object itself is used as the thread
"switch handle" for the context swap.
Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
isr.S code currently gets CPU information from global `_kernel` assuming
there's only one CPU. In order to prepare for upcoming SMP support,
change code to actually get current CPU information.
Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
Change the CPU_CORTEX_R kconfig option to CPU_AARCH32_CORTEX_R to
distinguish the armv7 version from the armv8 version of Cortex-R.
Signed-off-by: Bradley Bolen <bbolen@lexmark.com>
When Zephyr runs directly on actual hardware, it will be always
directing MSI messages to BSP (BootStrap Processor). This was fine until
Zephyr could be ran on virtualizor that may NOT run it on BSP.
So directing MSI messages on current processor. If Zephyr runs on actual
hardware, it will be BSP since such setup is always made at boot time by
the BSP. On other use case it will be whatever is relevant at that time.
Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
Depending on whether X2APIC is enabled or not, it will be safer to grab
such ID from the right place.
Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
This will centralize CPUID related accessors. There was no need for it
so far, but this is going to change.
Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
When XIP is not enabled, z_data_copy() already falls back to an empty
function. No need to ifdef it.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
The x86 and xtensa implementations of irq_offload() invoke synchronous
interrupts on the local CPU, and are therefore safe to use from within
an interrupt context. This is a cheap and portable way to exercise
nested interrupts, which are otherwise highly platform-dependent to
test. Add a kconfig to signal the capability.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
The Xtensa implementation of arch_irq_offload() required that the user
select the correct interrupt manually, and would race with itself if
invoked from separate CPUs (it was saved here by the main
irq_offload() function which has a semaphore to serialize access).
Use the new gen_zsr.py script to automatically detect the highest
available software interrupt, and keep a per-CPU set of
callback/parameter pointers.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Currently, the DCACHE range invalidation can cause data corruption when
the ends of the given range is not aligned to a full cache line.
Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
Avoid executing ISRs using the thread stack as it might not be sized
for that. Plus, we do have IRQ stacks already set up for us.
The non-nested IRQ context is still (and has to be) saved on the thread
stack as the thread could be preempted.
The irq_offload case is never nested and always invoked with the
sched_lock held so it can be simplified a bit.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
This is an uint32_t so the proper register width must be used, otherwise
the adjacent structure member will be overwritten (didn't happen in
practice because of struct member alignment but still). This makes the
inc_nest_counter and dec_nest_counter macros rather unwieldy, especially
with upcoming changes, so let's just remove them.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Let's provide our own z_early_memset() and z_early_memcpy() rather than
making our own .bss clearing function that risk missing out on updates
to the main version.
Also remove extra stuff already provided by kernel_internal.h.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
This was introduced when trying to fix a previous merge conflict. It
broke userspace tests on nucleo_l073rz.
Fixes#42627
Signed-off-by: Bradley Bolen <bbolen@lexmark.com>
These functions help the code to be more self-documenting. Use them to
make the code's intent clearer.
Signed-off-by: Bradley Bolen <bbolen@lexmark.com>
Replace CONFIG_CPU_CORTEX_R with CONFIG_ARMV7_R since it is clearer with
respect to the difference between v7 and v8 Cortex-R.
Signed-off-by: Bradley Bolen <bbolen@lexmark.com>
When calling a syscall, the SVC routine will now elevate the thread to
privileged mode and exit the SVC setting the return address to the
syscall handler. When the thread is swapped back in, it will be running
z_do_arm_syscall in system mode. That function will run the syscall
then automatically return the thread to usr mode.
This allows running the syscall in sys mode on a thread so that we can
use syscalls that sleep without doing unnatural things. The previous
implementation would enable interrupts while still in the SVC call and
do weird things with the nesting count. An interrupt could happen
during this time when the syscall was still in the exception state, but
the nested count had been decremented too soon. Correctness of the
nested count is important for future floating point unit work.
The Cortex-R behavior now matches that of Cortex-M.
Signed-off-by: Bradley Bolen <bbolen@lexmark.com>
Non-standard `jalr rd, rs` pseudo-instructions are used.
This commit changes them to `ret` for standard return pseudo-instruction
or `jalr rd, rs, 0` for no offset jump register and link.
Fixes#41100.
Signed-off-by: Henry Hsieh <r901042004@yahoo.com.tw>
Some XCC toolchains do not provide atexit() which results
in undefined reference error. So add a weak dummy atexit()
for this siutation.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Turns out that xt-xcc will bail when faced with a real core-isa.h (it
wants you to rely on the builtins in the compiler). Undefine __XCC__
to force it to actually parse and emit declarations for its own
header.
(Also adds a newline to the generated one-line C file to silence a warning)
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
We had a similar sequence for interrupt return, where we were
selecting (actually only for the benefit of qemu) the highest priority
EPCn/EPSn registers for our RFI instruction. That works much better
in python the preprocessor.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
The kernel coherence cache flush code was using a scratch register to
mark the top of the stack. Likewise a good candidate for ZSR use.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
This is actually Cadence-authored code, but its use of EXCSAVE1 as a
sideband input to the exception handler is very much in the same
family of tricks. Use ZSR assignments here too.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Zephyr likes to use the various Xtensa scratch registers for its own
purposes in several places. Unfortunately, owing to the
configurability of the architecture, we have to use different
registers for different platforms. This has been done so far with a
collection of different tricks, some... less elegant than others.
Put it all in one place. This is a python script that emites a
"zsr.h" header with register assignments for all the existing users.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
MIPS (Microprocessor without Interlocked Pipelined Stages) is a
instruction set architecture (ISA) developed by MIPS Computer
Systems, now MIPS Technologies.
This commit provides MIPS architecture support to Zephyr. It is
compatible with the MIPS32 Release 1 specification.
Signed-off-by: Antony Pavlov <antonynpavlov@gmail.com>
This moves CONFIG_MMU and its children from arch/Kconfig into
kernel/Kconfig. These are used to enable kernel support of MMU
so they should be under kernel/.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Currently, is_user_mode is 8-byte in riscv64 and it breaks a 4-byte PMP
region protecting it. Because is_user_mode is a single flag, we could
just fix it's size to 4-byte in both riscv32 and riscv64.
Signed-off-by: Jim Shu <cwshu09@gmail.com>
In RV64, all general-purpose registers and pmpcfg CSR are 64-bit
instead of 32-bit. Fix these registers and related C variables/literals
to be 32/64-bit compatible.
Signed-off-by: Jim Shu <cwshu09@gmail.com>
Avoid setting the switch_handler in the z_get_next_switch_handle() code
when the context is not fully saved yet to avoid a race against other
cores waiting on wait_for_switch().
See issue #40795 and discussion in #41840
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
For functions returning nothing, there is no need to document
with @return, as Doxgen complains about "documented empty
return type of ...".
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
For functions returning nothing, there is no need to document
with @return, as Doxgen complains about "documented empty
return type of ...".
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
For functions returning nothing, there is no need to document
with @return, as Doxgen complains about "documented empty
return type of ...".
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
For functions returning nothing, there is no need to document
with @return, as Doxgen complains about "documented empty
return type of ...".
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
For functions returning nothing, there is no need to document
with @return, as Doxgen complains about "documented empty
return type of ...".
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
For functions returning nothing, there is no need to document
with @return, as Doxgen complains about "documented empty
return type of ...".
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
For functions returning nothing, there is no need to document
with @return, as Doxgen complains about "documented empty
return type of ...".
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
For functions returning nothing, there is no need to document
with @return, as Doxgen complains about "documented empty
return type of ...".
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This is trick (mapping RAM twice so you can use alternate Region
Protection Option addresses to control cacheability) is something any
Xtensa hardware designer might productively choose to do. And as it
works really well, we should encourage that by making this a generic
architecture feature for Zephyr.
Now everything works by setting two kconfig values at the soc level
defining the cached and uncached regions. As long as these are
correct, you can then use the new arch_xtensa_un/cached_ptr() APIs to
convert between them and a ARCH_XTENSA_SET_RPO_TLB() macro that
provides much smaller initialization code (in C!) than the HAL
assembly macros. The conversion routines have been generalized to
support conversion between any two regions.
Note that full KERNEL_COHERENCE still requires support from the
platform linker script, that can't be made generic given the way
Zephyr does linkage.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
This commit enable PMP-based memory protection of code and rodata
instead of relying on non-writable real HW (e.g. flash). Use static
PMP region with PMP Lock bit to protect them in both user/supervisor
mode.
Signed-off-by: Jim Shu <cwshu@andestech.com>
Implement new mechanism of arch_buffer_validate() to support checking
static PMP regions. This is preparation patch for code/rodate protection
via RISC-V PMP.
Signed-off-by: Jim Shu <cwshu@andestech.com>
Thread init related to PMP & userspace contains 5 parts:
1. User/supervisor thread clear PMP context
2. User thread clear it's context
3. User/supervisor thread assign to different entry
4. Supervisor thread assign mstatus.MPRV for M-mode PMP protection
5. User/supervisor thread setup PMP regions of stack guard if enabled
Signed-off-by: Jim Shu <cwshu@andestech.com>
Reorder the memory domain async functions to:
arch_mem_domain_partition_add()
arch_mem_domain_partition_remove()
arch_mem_domain_thread_add()
arch_mem_domain_thread_remove()
Signed-off-by: Jim Shu <cwshu@andestech.com>
Simplify multiple ifdef case in computing region number. Also move these
macros to core_pmp.c because they are only used in one file.
Signed-off-by: Jim Shu <cwshu@andestech.com>
Using struct riscv_pmp_region to modulize PMP CSR handling, including
PMP NAPOT/TOR mode handling. This patch can make us more easily to
add/remove RISC-V PMP regions without considering register handling.
Signed-off-by: Jim Shu <cwshu@andestech.com>
Cleanup logging API in core_pmp.c. Remove old printf-based debugging API
and change the log module of PMP to individual MPU log module.
Signed-off-by: Jim Shu <cwshu@andestech.com>
This commit add 2 minor fixes of IRQ handling:
1. Save caller registers before calling z_riscv_configure_stack_guard()
in RISC-V assembly.
2. reschedule and no_reschdule code paths use different interrupt
return path after supporting of CONFIG_PMP_STACK_GUARD. back-to-back
interrupt checking is in the reschedule code path so that it should
jump to interrupt return path of reschedule.
Signed-off-by: Jim Shu <cwshu09@gmail.com>
If no thread use this memory domain, there isn't any user PMP region
translated from memory partitions in domain. In this case, memory
partition removal doesn't need to remove user PMP region and
arch_mem_domain_partition_remove() could just successfully return.
Signed-off-by: Jim Shu <cwshu@andestech.com>
Although CONFIG_USERSPACE is enabled, there are supervisor threads who
don't have privileged stack using exception handler. Only let user
threads to switch to privileged stack in exception handler.
Signed-off-by: Jim Shu <cwshu@andestech.com>
In fact, in case of VT-D being enabled, it will require to get an
address and data for its own MSI based interrupts which cannot be
remapped (i.e.: will directly go to the relevant APIC).
This is necessary to get the Fault event supported in VT-D.
Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
This will not only be used by MSI remapping but by all relevant
interrupts.
Fix also IRTE settings:
- handle x2apic for destination id
- destination mode is always logical (as for IOAPIC)
Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
SHV bit depends on the number of vectors allocated.
If it's facing a multi-vector MSI array, it will set the bit.
If not the bit must be 0.
Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
Refactor to handle this case. This is valid only when MSI multi-vector
feature is enabled.
Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
As all interruption need to go through VT-D, calling vt-d remap will
happen on lower level as seen next, so make sure all pcie related
irq/vector get tighten to their respective allocated IRTE.
Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
Allocate an IRTE for all irq being connected through
arch_irq_connect_dynamic(). This will be mandatory since VT-D expects to
filter all interruptions (but the one it generates, as we will see
later).
Taking into account CONFIG_INTEL_VTD_ICTL_XAPIC_PASSTHROUGH, which could
help for debugging.
Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
There is no need to differentiate with multi-vector or not, MSI vs
MSI-x: all need to be remapped if Intel VT-D is on.
Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
Enable ARCH_EXCEPT macro for non-usermode scenario for RISC-V
Macro will now raise an illegal instruction exception so that mepc will
hold expected value in exception handler, and generated coredump can
reconstruct the failing stack
Coredump tests running on renode (for RISC-V) can now utilize fatal error
path through k_panic
Signed-off-by: Mark Holden <mholden@fb.com>
GD32V processor core is used non-standard bitmask
for mcause register. Add option to configure the bitmask
to support GD32V.
Signed-off-by: TOKITA Hiroshi <tokita.hiroshi@gmail.com>
Startup on these devices was sort of a mess, with multiple variants of
Xtensa and platform initialization code from multiple ancestries being
invoked at different places for different purposes. Just use one code
path for everyone.
Bootloader entry starts with a minimal assembly stub that simply sets
WINDOW{START,BASE}, PS and a stack pointer and then jumps to C code.
That then uses the cpu_early_init() implementation from cAVS 2.5's
secondary cores to finish Xtensa initialization, and then flows
directly into the pre-existing bootloader C code to initialize cache
and memory and copy the HP-SRAM image, then it invokes Zephyr via a
simple C function call to z_cstart().
Likewise, remove the "reset vector" from Zephyr. This was never a
reset vector, reset on these devices goes to a fixed address in a ROM.
CPU initialization is handled explicitly and completely in the
bootloader now, in a way that can be unified between the main and
secondary cores. Entry from the bootloader now goes directly into
z_cstart() via a C call (via a single jump instruction placed at the
entry point address -- that's going away soon too once we're using a
unified link).
Now that vector table initialization happens in a uniform way, there's
no need to copy the VECBASE value during arch_start_cpu().
Finally note that this also reverts the
CONFIG_RESET_VECTOR_IN_BOOTLOADER kconfig variable added for these
platforms, because it's no longer a tunable and true always.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Adds Xtensa as supported architecture for coredump. Fixes
a few typos in documentation, Kconfig and a C file. Dumps
minimal set of registers shown by 'info registers' in GDB
for the sample_controller and ESP32 SOCs. Updates tests.
Signed-off-by: Lauren Murphy <lauren.murphy@intel.com>
The warning below appears once -Waddress-of-packed-mem is enabled:
/home/carles/src/zephyr/zephyr/arch/x86/core/acpi.c: In function
'z_acpi_find_table':
/home/carles/src/zephyr/zephyr/arch/x86/core/acpi.c:190:24: warning:
taking address of packed member of 'struct acpi_xsdt' may result in an
unaligned pointer value [-Waddress-of-packed-member]
190 | for (uint64_t *tp = &xsdt->table_ptrs[0]; tp < end; tp++) {
To avoid the warning, use an intermediate void * variable.
More info in #16587.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Fix the assert that checks for existence of a cycle counter.
The field is named NO CYCCNT, so when it is 1, there is no cycle
counter. But we are asserting the opposite.
Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
This commit adds support of Xen Enlighten page and initial support for
Xen event channels. It is needed for future Xen PV drivers
implementation.
Now enlighten page is mapped to the prepared memory area on
PRE_KERNEL_1 stage. In case of success event channel logic gets
inited and can be used ASAP after Zephyr start. Current implementation
allows to use only pre-defined event channels (PV console/XenBus) and
works only in single CPU mode (without VCPUOP_register_vcpu_info).
Event channel allocation will be implemented in future versions.
Signed-off-by: Dmytro Firsov <dmytro_firsov@epam.com>
A Cortex-M specific function (sys_clock_isr()) was defined as a weak
function, so in practice it was always available when system clock was
enabled, even if no Cortex-M systick was available. This patch
introduces an auxiliary Kconfig option that, when selected, the ISR
function gets installed. External SysTick drivers can also make use of
this function, thus achieving the same functionality offered today but
in a cleaner way.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
ARC_HAS_ACCL_REGS should set to y to protect ACCL and ACCH registers
during irq. These registers could be used as GPRs by compilers and
therefore need store/restore during irq.
Signed-off-by: Yuguo Zou <yuguo.zou@synopsys.com>
This adds basic support for GDB stub on Xtensa. Note that
this only provides the common bits on the architecture side.
SoC support is also required to fully enable GDB stub on
each Xtensa SoC.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Storing the state where this is the first GDB break can be done
in the main GDB stub code. There is no need to store the state
in architecture layer.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This adds some architecture-specific functions to read/write
registers for the GDB stub. This is in preparation for the actual
introduction of these functions in the core GDB stub code to
avoid breaking the build in between commits.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This changes the arch_mem_domain_*() functions to return errors.
This allows the callers a chance to recover if needed.
Note that:
() For assertions where it can bail out early without side
effects, these are converted to CHECKIF(). (Usually means
that updating of page tables or translation tables has not
been started yet.)
() Other assertions are retained to signal fatal errors during
development.
() The additional CHECKIF() are structured so that it will bail
early if possible. If errors are encountered inside a loop,
it will still continue with the loop so it works as before
this changes with assertions disabled.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
replace with version.parse from packaging module.
prevent this warning message:
DeprecationWarning: The distutils package is deprecated
and slated for removal in Python 3.12. Use setuptools or
check PEP 632 for potential alternatives
Signed-off-by: Julien Massot <julien.massot@iot.bzh>
Use sys_clock_hw_cycles_per_sec() instead of
CONFIG_SYS_CLOCK_HW_CYCLES_PER_SEC to determine clock cycles.
Signed-off-by: Michel Haber <michel-haber@hotmail.com>
Call into z_thread_usage_stop() before ISR entry to avoid including
interrupt handling totals in thread usage stats.
This has to go into the assembly immediately before the callback-based
dispatch. Note that the dispatch code was putting the vector number
in RCX, which was unfortunate as that's a caller-saved register.
Would be nice to clean this up in the future so it lives in a
preserved register but it's mildly complicated to make work with the
way we do the stack layout right now.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Call into z_thread_usage_stop() before ISR entry to avoid including
interrupt handling totals in thread usage stats.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Call into z_thread_usage_stop() before ISR entry to avoid including
interrupt handling totals in thread usage stats.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Call into z_thread_usage_stop() before ISR entry to avoid including
interrupt handling totals in thread usage stats.
This is pretty much exactly where we want it, just after the context
saving steps (which we can't elide since the hook is in C) and
immediately before the tracing hook for ISR entry. And as I'm reading
the code, this is purely for Zephyr-registered interrupts, meaning
that software exceptions will be accounted for (correctly) as part of
the excepting thread.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Call into z_thread_usage_stop() before ISR entry to avoid including
interrupt handling totals in thread usage stats.
Note that this hook is after the register save and stack swap has
happened, so it still incldues some overhead. But calling out from
the interrupted stack on Xtensa gets really, really hairy due to the
weird intermediate states we leverage (once we've saved enough context
to make a C call safely, we've lost the ability to use register
windows per the C ABI!).
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
To prepare for linker script creation with flexible number of linker
passes depending on system configuration then the Zephyr CMake linker
script generator has been updated to use pass names instead of pass
numbers.
This allows greater flexibility as a section can now be active based on
the settings on the pass and not the linking pass index number.
As part of this, the `PASS` processing in `linker_script_common.cmake`
has been adjusted so that it properly handles when a linking pass is
handling multiple settings, such as both `LINKER_APP_SMEM_UNALIGNED`
and `DEVICE_HANDLES_PASS1` in same linking pass.
As the number of linking passes are more flexible, then the PASS
argument in `zephyr_linker_section()` and
`zephyr_linker_section_configure()` has been updated to also support
a `NOT <name>` argument, for example: `PASS NOT LINKER_ZEPHYR_FINAL`.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
This commit adds Xen hypervisor call interface for arm64 architecture.
This is needed for further development of Xen features in Zephyr.
Signed-off-by: Dmytro Firsov <dmytro_firsov@epam.com>
Modify #ifdefs so that any code that is compiled if CONFIG_ARMV7_R is
set is also compiled if CONFIG_ARMV7_A is set.
Modify #ifdefs so that any code that is compiled if CONFIG_CPU_CORTEX_R
is set is also compiled if CONFIG_CPU_AARCH32_CORTEX_A is set.
Modify source dir inclusion in CMakeLists.txt accordingly.
Brief file descriptions have been updated to include Cortex-A whereever
only Cortex-M and Cortex-R were mentioned so far.
Signed-off-by: Immo Birnbaum <Immo.Birnbaum@weidmueller.com>
An initial implementation for memory management using the ARMv7 MMU.
A single L1 translation table for the whole 4 GB address space is al-
ways present, a configurable number of L2 page tables are linked to
the L1 table based on the static memory area configuration at boot
time, or whenever arch_mem_map/arch_mem_unmap are called at run-time.
Currently, a CPU with the Multiprocessor Extensions and execution at
PL1 are always assumed. Userspace-related features or thread stack
guard pages are not yet supported. Neither are LPAE, PXN or TEX re-
mapping. All mappings are currently assigned to the same domain. Re-
garding the permissions model, access permissions are specified using
the AP[2:1] model rather than the older AP[2:0] model, which, accor-
ding to ARM's documentation, is deprecated and should no longer be
used. The newer model adds some complexity when it comes to mapping
pages as unaccessible (the AP[2:1] model doesn't support explicit
specification of "no R, no W" permissions, it's always at least "RO"),
this is accomplished by invalidating the ID bits of the respective
page's PTE.
Includes sources, Kconfig integration, adjusted CMakeLists and the
modified linker command file (proper section alignment!).
Signed-off-by: Immo Birnbaum <Immo.Birnbaum@weidmueller.com>
The configuration bits ATCMPCEN, B0TCMPCEN and B1TCMPCEN in the ACTLR
register referenced in the function z_arm_tcm_disable_ecc are only de-
fined for Cortex-R CPUs. For Cortex-A CPUs, those bits are declared
as reserved.
Comp.: https://arm-software.github.io/CMSIS_5/Core_A/html/group__CMSIS__ACTLR.html
Signed-off-by: Immo Birnbaum <Immo.Birnbaum@weidmueller.com>
There are two macros for declaring stack arrays:
K_KERNEL_STACK_ARRAY_DEFINE:
Defines the array, allocating storage and setting the section name
K_KERNEL_STACK_ARRAY_EXTERN
Declares the name of a stack array allowing code to reference
the array which must be defined elsewhere
arch/arm/include/aarch32/cortex_m/stack.h was mis-using
K_KERNEL_STACK_ARRAY_DEFINE to declare z_interrupt_stacks by sticking
'extern' in front of the macro use. However, when this macro also set
the object file section for the symbol, having two of those caused a
conflict in the compiler due to the automatic unique name mechanism used
for sections to allow unused symbols to be discarded during linking.
This patch makes the header use the correct macro.
Signed-off-by: Keith Packard <keithp@keithp.com>
In some drivers, noncache memory need to be used for dma coherent
memroy, so add nocache memory segment mapping and support for ARM64
platforms.
The following variables definition example shows they will use nocache
memory allocation:
int var1 __nocache;
int var2 __attribute__((__section__(".nocache")));
Signed-off-by: Jiafei Pan <Jiafei.Pan@nxp.com>
Don't allow to enable multiple register banks / fast
interrupts if we have only one interrupt priority level.
NOTE: we duplicate some checks by adding dependencies to ARC
Kconfig and adding build-time checks in C code. We do it
intentionally as for some reason we can violate dependencies
in architecture-level Kconfig by adding incorrect default in
SoC-level Kconfig. Such violation happens without any
warnings / errors from the Kconfig.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
Add the arm64 MMU arch_virt_region_align() implementation used
to return a possible virtual addres alignment in order to
optimize the MMU table layout and possibly avoid using L3 tables
and use some L1 & L3 blocks instead for most of the mapping.
Suggested-by: Nicolas Pitre <npitre@baylibre.com>
Signed-off-by: Neil Armstrong <narmstrong@baylibre.com>
This reverts commit 67d290540e.
The script is actually used to generate the _soc_inthandlers.h
file when introducing a new Xtensa SoC. So restore it.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
The assert log of z_priv_stacks_ram_start failed to build due to passing
&z_priv_stacks_ram_start instead of just z_priv_stacks_ram_start.
Fixes#39190
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
When mapping the following:
device_map(&base0, DEVA_BASE, DEVA_SIZE, K_MEM_CACHE_NONE);
device_map(&base1, DEVB_BASE , DEVB_SIZE, K_MEM_CACHE_NONE);
with:
- DEVA_SIZE not multiple of a 4KB granule L2 block size (0x200000)
- DEVB_SIZE more than 2 x 4KB granule L2 block size
The mmu code will fill the first device_map() in a L3 table, then
on the second mapping the mmu code will complete the previous L3
table.
At the end of this table, the actual code will select an L2 block
instead of a table because the *virtual address* is multiple with
the L2 block size.
But if the physical address is not, the virtual block offset will
be ORed to the physical address, and not added.
Leading to a weird scenario where virtual memory is duplicated
resulting of the addresses ORing and not addition.
Example:
device_map(&base0, DEVA_BASE, 0x20000, K_MEM_CACHE_NONE);
device_map(&base1, 0x44000000 , 0x400000, K_MEM_CACHE_NONE);
First will result in VA 0x5ffe0000 and second in VA 0x5fbe0000.
The MMU code will use a table to map 0x5ffe0000 to 0x5fbfffff.
For 0x5fc00000 to 0x5fdfffff, since the VA is multiple of an L2
block size, the L3 table is not used.
But the L2 block description entry address is 0x44060000, meaning
that for each access in this L2 block, the following will be done:
0x44060000 | (VA & 1FFFFF)
This is working for the 0x5fc40000 to 0x5fc5ffff access, but for the
0x5fbc60000 (0x5fbe0000 + 0x80000) access the PA gets calculated as :
0x44060000 | (0x5fc60000 & 1FFFFF) = 0x44060000 | 0x60000 = 0x44060000
Instead of the expected 0x44080000.
The solution is to check if the PA descriptor is aligned with the
level block size, if not move to the next level.
Signed-off-by: Neil Armstrong <narmstrong@baylibre.com>
Some Xtensa SoCs may not have that many levels of interrupts.
So limit the call to DEF_INT_C_HANDLER() to only supported
levels to avoid calling non-existent functions.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Add dynamic_areas_init. It will mark a mpu region as a dynamic region
area. The dynamic region areas is designed to be the background
regions, so that the system could re-program the thread regions on
the backgroud regions.
Add configure_dynamic_mpu_regions to re-program the thread regions on
the backgroud regions. The configure_dynamic_mpu_regions function is
the core function of implementing the userspace for the MPU. This
function is used in thread creation and context switch.
During context switch, the pre thread's regions should be disabled,
and the new thread's regions will be re-programed. Since the thread's
stack region will also be switched, there will be a problem before
new thread's region being re-programed which is the new thread's
stack usage. To avoid the exception generated by stack usage caused by
unprogramed new thread's stack region, I disable mpu first before
flush_dynamic_regions_to_mpu and then enable it.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Add a new macro MEM_DOMAIN_ALIGN_AND_SIZE for mmu and mpu mem
alignment.
MEM_DOMAIN_ALIGN_AND_SIZE is
- CONFIG_MMU_PAGE_SIZE, when mmu is enabled.
- CONFIG_ARM_MPU_REGION_MIN_ALIGN_AND_SIZE when mpu enabled.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Include the new introduced include/arch/arm64/mm.h instead of the
arm_mmu.h or arm_mpu.h.
Unify function names z_arm64_thread_pt_init/z_arm64_swap_ptables with
z_arm64_thread_mem_domains_init/z_arm64_swap_mem_domains for mmu and
mpu, because:
1. mmu and mpu have almost the same logic.
2. mpu doesn't have ptables.
3. using the function names help reducing "#if define" macros.
Similarly, change z_arm64_ptable_ipi to z_arm64_domain_sync_ipi
And fix a log bug in arm_mmu.c.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
This patch mainly moves mpu related code from
arch/arm64/core/cortex_r/mpu/ to arch/arm64/core/cortex_r/ and moves
the mpu header files from include/arch/arm64/cortex_r/mpu/ to
include/arch/arm64/cortex_r/
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
