* Add support for coredump on ARM64 architectures.
* Add the script used for post-processing coredump output.
Signed-off-by: Marcelo Ruaro <marcelo.ruaro@huawei.com>
Signed-off-by: Rodrigo Cataldo <rodrigo.cataldo@huawei.com>
Signed-off-by: Roberto Medina <roberto.medina@huawei.com>
Currently the lazy fpu saving algorithm in arm64 is using the fpu_owner
pointer from the cpu structure to understand the owner of the context
in the cpu and save it in case someone different from the owner is
accessing the fpu.
The semantics for memory consistency across smp systems is quite prone
to errors and reworks on the current code might miss some barriers that
could lead to inconsistent state across cores, so to overcome the issue,
use atomics to hide the complexity and be sure that the code will behave
as intended.
While there, add some isb barriers after writes to cpacr_el1, following
the guidance of ARM ARM specs about writes on system registers.
Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
cpu_node_list does not hold the corrent mapping of cpu id and mpid when
core booting sequence does not follow the DTS cpu node sequence. This
will cause an issue that sgi cannot deliver to the right target.
Add the cpu_map array to hold the corrent mapping between cpu id and
mpid.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Each core should init their own stack during the reset when SMP enabled,
but do not touch others. The current init results in each core starting
init the stack from the same address which will break others.
Fix the issue by setting a correct start address.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
LOG system has unalignment access instruction which will cause an
alignment exception before MPU is enabled. Remove the LOG print before
MPU is enabled to avoid this issue.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
This trick turns out also to be needed by the abort/join code.
Promote it to a more formal-looking internal API and clean up the
documentation to (hopefully) clarify the exact behavior and better
explain the need.
This is one of the more... enchanted bits of the scheduler, and while
the trick is IMHO pretty clean, it remains a big SMP footgun.
Signed-off-by: Andy Ross <andyross@google.com>
For secure EL2 to be entered the EEL2 bit in SCR_EL3 must be set. This
should only be set if Zephyr has not been configured for NS mode only,
if the device is currently in secure EL3, and if secure EL2 is supported
via the SEL2 bit in AA64PFRO_EL1. Added logic to enable EEL2 if all
conditions are met.
Signed-off-by: Chad Karaginides <quic_chadk@quicinc.com>
Per the ARMv8 architecture document, modification of the system control
register is a context-changing operation. Context-changing operations are
only guaranteed to be seen after a context synchronization event.
An ISB is a context synchronization event. One has been placed after
each SCTLR modification. Issue was found running full speed on target.
Signed-off-by: Chad Karaginides <quic_chadk@quicinc.com>
Let's consider CPU1 waiting on a spinlock already taken by CPU2.
It is possible for CPU2 to invoke the FPU and trigger an FPU exception
when the FPU context for CPU2 is not live on that CPU. If the FPU context
for the thread on CPU2 is still held in CPU1's FPU then an IPI is sent
to CPU1 asking to flush its FPU to memory.
But if CPU1 is spinning on a lock already taken by CPU2, it won't see
the pending IPI as IRQs are disabled. CPU2 won't get its FPU state
restored and won't complete the required work to release the lock.
Let's prevent this deadlock scenario by looking for pending FPU IPI from
the spinlock loop using the arch_spin_relax() hook.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The init infrastructure, found in `init.h`, is currently used by:
- `SYS_INIT`: to call functions before `main`
- `DEVICE_*`: to initialize devices
They are all sorted according to an initialization level + a priority.
`SYS_INIT` calls are really orthogonal to devices, however, the required
function signature requires a `const struct device *dev` as a first
argument. The only reason for that is because the same init machinery is
used by devices, so we have something like:
```c
struct init_entry {
int (*init)(const struct device *dev);
/* only set by DEVICE_*, otherwise NULL */
const struct device *dev;
}
```
As a result, we end up with such weird/ugly pattern:
```c
static int my_init(const struct device *dev)
{
/* always NULL! add ARG_UNUSED to avoid compiler warning */
ARG_UNUSED(dev);
...
}
```
This is really a result of poor internals isolation. This patch proposes
a to make init entries more flexible so that they can accept sytem
initialization calls like this:
```c
static int my_init(void)
{
...
}
```
This is achieved using a union:
```c
union init_function {
/* for SYS_INIT, used when init_entry.dev == NULL */
int (*sys)(void);
/* for DEVICE*, used when init_entry.dev != NULL */
int (*dev)(const struct device *dev);
};
struct init_entry {
/* stores init function (either for SYS_INIT or DEVICE*)
union init_function init_fn;
/* stores device pointer for DEVICE*, NULL for SYS_INIT. Allows
* to know which union entry to call.
*/
const struct device *dev;
}
```
This solution **does not increase ROM usage**, and allows to offer clean
public APIs for both SYS_INIT and DEVICE*. Note that however, init
machinery keeps a coupling with devices.
**NOTE**: This is a breaking change! All `SYS_INIT` functions will need
to be converted to the new signature. See the script offered in the
following commit.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
init: convert SYS_INIT functions to the new signature
Conversion scripted using scripts/utils/migrate_sys_init.py.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
manifest: update projects for SYS_INIT changes
Update modules with updated SYS_INIT calls:
- hal_ti
- lvgl
- sof
- TraceRecorderSource
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
tests: devicetree: devices: adjust test
Adjust test according to the recently introduced SYS_INIT
infrastructure.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
tests: kernel: threads: adjust SYS_INIT call
Adjust to the new signature: int (*init_fn)(void);
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
Update current stack limit on every context switch, including switching
to irq stack and switching back to thread stack.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
This commit mainly enable the safe exception stack including the stack
switch. Init the safe exception stack by calling
z_arm64_safe_exception_stack during the boot stage on every core. Also,
tweaks several files to properly switch the mode with different cases.
1) The same as before, when executing in userspace, SP_EL0 holds the
user stack and SP_EL1 holds the privileged stack, using EL1h mode.
2) When entering exception from EL0 then SP_EL0 will be saved in the
_esf_t structure. SP_EL1 will be the current SP, then retrieves the safe
exception stack to SP_EL0, making sure the always pointing to safe
exception stack as long as the system running in kernel space.
3) When exiting exception from EL1 to EL0 then SP_EL0 will be restored
from the stack value previously saved in the _esf_t structure. Still at
EL1h mode.
4) Either entering or exiting exception from EL1 to EL1, SP_EL0 will
keep holding the safe exception stack unchanged as memtioned above.
5) Do a quick stack check every time entering the exception from EL1 to
EL1. If check fail, set SP_EL1 to safe exception stack, and then handle
the fatal error.
Overall, the exception from user mode will be handled with kernel stack
at the assumption that it is impossible the stackoverflow happens at the
entry of exception from EL0 to EL1. However the exception from kernel
mode will be firstly checked with the safe exception stack to see if the
kernel stack overflows, because the exception might be triggered by
stack invalid accessing.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Add safe exception stack init function which does several things:
1) setting current cpu safe exception stack pointer to its corresponding
stack top.
2) init sp_el0 with the above safe exception stack.
That makes sure the sp_el0 points to per-cpu safe_stack in the kernel
space.
3) init the current_stack_limit and corrupted_sp with 0
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
As the preparation for enabling safe exception stack, add a variable in
_esf_t to save the user stack held by sp_el0 at the point of the
exception happening from EL0. The newly added variable in _esf_t is
named sp from which the user stack will be restored when exceptions eret
to EL0.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Add three per-cpu variables for the convenience of quickly accessing.
The safe_exception_stack stores the top of safe exception stack pointer.
The current_stack_limit stores the current thread's priv stack limit.
The corrputed_sp stores the priv sp or irq sp for the stack overflow
case, or 0 for the normal case.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Introduce two configs to prepare to enable the safe exception stack for
the kernel space. This is the preparation for enabling hardware stack
guard. Also define the safe exception stack for kernel exception stack
check.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
If so this is most certainly a bug. arch_mem_unmap() should be
used before mapping the same area again.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
First, we have commit 7d27bd0b85 ("arch: arm64: Disable infinite
recursion warning for `discard_table`") that blindly shut up a compiler
warning that did actually highlighted a real bug. Revert that and fix
the bug properly. And yes, mea culpa for having been the first to
approve that commit, or even creating the bug in the first place.
Then let's add proper table usage cound handling for discard_table() to
work properly and avoid leaking table pages.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The cache operations must be quick, optimized and possibly inlined. The
current API is clunky, functions are not inlined and passing parameters
around that are basically always known at compile time.
In this patch we rework the cache functions to allow us to get rid of
useless parameters and make inlining easier.
In particular this changeset is doing three things:
1. `CONFIG_HAS_ARCH_CACHE` is now `CONFIG_ARCH_CACHE` and
`CONFIG_HAS_EXTERNAL_CACHE` is now `CONFIG_EXTERNAL_CACHE`
2. The cache API has been reworked.
3. Comments are added.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
The is code duplication as one is in C, and the other is an assembly
macro. As there is no easy way to find out about this duplication,
adding a comment seems the best way to go.
Signed-off-by: Henri Xavier <datacomos@huawei.com>
Change for loops of the form:
for (i = 0; i < CONFIG_MP_NUM_CPUS; i++)
...
to
unsigned int num_cpus = arch_num_cpus();
for (i = 0; i < num_cpus; i++)
...
We do the call outside of the for loop so that it only happens once,
rather than on every iteration.
Signed-off-by: Kumar Gala <kumar.gala@intel.com>
Change automated searching for files using "IRQ_CONNECT()" API not
including <zephyr/irq.h>.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
GCC may generate ldp/stp instructions with the Advanced SIMD Qn
registers for consecutive 32-byte loads and stores.
This commit disables this GCC behaviour because saving and restoring
the Advanced SIMD context is very expensive, and it is preferable to
keep it turned off by not emitting these instructions for better
context switching performance.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
On GICv3, when we send an IPI interrupt, aff3, aff2 and aff1 should
be assigned a value corespond to a PE for which interrupt will be
generated. target_list only corresponds to aff0.
On real hardware, aff3, aff2, aff1 and aff0 should be treated as a
whole to determine a PE.
Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
VMPIDR_EL2 is assigned the value returned by EL2 reads of MPIDR_EL1
MPIDR_EL1 is the register holding the Multiprocessor ID which is to
identify different cores. Because of the virtualization requirements
for AArch64, MPIDR_EL1 should be virtualized (the different virtualized
cores can run on the same physical core). Thus the value of MPIDR_EL1
should be switched when the VM is switched. Setting the VMPIDR_EL2 is
the way to change the value returned by EL1 reads of MPIDR_EL1. Even
without virtualization, we still need to set VMPIDR_EL2 during booting
at EL2 or EL3. Otherwise, all cores' IDs are zero at the EL1 stage
which will break the SMP system.
Signed-off-by: Huifeng Zhang <Huifeng.Zhang@arm.com>
When a cache API function is called from userspace, this results on
ARM64 in an OOPS (bad syscall error). This is due to at least two
different factors:
- the location of the cache handlers is preventing the linker to
actually find the handlers
- specifically for ARM64 and ARC some cache handling functions are not
implemented (when userspace is not used the compiler simply optimizes
out these calls)
Fix the problem by:
- moving the userspace cache handlers to a their logical and proper
location (in the drivers directory)
- adding the missing handlers for ARM64 and ARC
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
When CONFIG_INIT_STACKS is enabled all stacks should be filled with 0xaa
so that the thread analyzer can measure stack utilization, but the IRQ
stack was not filled and so `kernel stacks` on the shell would show that
the stack had been fully used and inferring an IRQ stack overflow
regardless of the IRQ stack size.
Fill the IRQ stack before it gets used so that we can have precise usage
reports.
Signed-off-by: Jamie Iles <quic_jiles@quicinc.com>
Signed-off-by: Dave Aldridge <quic_daldridg@quicinc.com>
Following zephyr's style guideline, all if statements, including single
line statements shall have braces.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Targets with text or data addresses above the 4GB boundary may need to use
the large code model to ensure relocations in the linker work correctly.
Signed-off-by: Keith Packard <keithp@keithp.com>
Enable single-threaded support for the arm64 archtecture.
This mode of execution is supported on an soc under
development and is validated regularly.
Signed-off-by: Eugene Cohen <quic_egmc@quicinc.com>
In performing a double check of Zephyr arm64 MMU config
against edk2, a different in the programming of the
Translation Control Register (TCR) was found. TCR.TG[1]
should be set to address Cortex-A57 erratum 822227:
"Using unsupported 16K translation granules might cause
Cortex-A57 to incorrectly trigger a domain fault"
Signed-off-by: Eugene Cohen <quic_egmc@quicinc.com>
On platforms where reset vector catch is not possible
it is useful to have a compile-time option to spin
at the reset vector allowing a debugger to be attached
and then to manually resume execution.
Define a config option for arm64 to spin at the
reset vectdor so a debugger can be attached.
Signed-off-by: Eugene Cohen <quic_egmc@quicinc.com>
This commit selectively disables the infinite recursion warning
(`-Winfinite-recursion`), which may be reported by GCC 12 and above,
for the `disable_table` function because no actual infinite recursion
will occur under normal circumstances.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
arch_mem_map() on ARM64 is currently not supporting the K_MEM_PERM_USER
parameter so we cannot allocate userspace accessible memory using the
memory helpers. Fix this.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
<soc.h> has been traditionally been used as a proxy to HAL headers,
register definitions, etc. Nowadays, <soc.h> is anarchy. It serves a
different purpose depending on the SoC. In some cases it includes HALs,
in some others it works as a header sink/proxy (for no good reason), as
a register definition when there's no HAL... To make things worse, it is
being included in code that is, in theory, non-SoC specific.
This patch is part of a series intended to improve the situation by
removing <soc.h> usage when not needed, and by eventually removing it.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
The init stack of the secondary core should use KERNEL_STACK_BUFFER + sz
Using Z_THREAD_STACK_BUFFER will calculate the wrong stack size.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
The current SMP boot code doesn't consider that the cores can boot at
the same time. Possibly, more than one core can boot into primary core
boot sequence. Fix it by using the atomic operation to make sure only
one core act as the primary core.
Correspondingly, sgi_raise_ipi should transfer CPU id to mpidr.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Fix writing of ICC_SRE_EL3 to or-in bits to align
with original intent to read-modify-write this
register.
Also disable FIQ and IRQ bypass so interrupt delivery
occurs through GIC. Platforms may choose to override
this behavior in z_arm64_el3_plat_init implementations.
Remove ICC_SRE_EL3 config from viper and qemu since
this is now handled in the arm64 arch core.
Signed-off-by: Eugene Cohen <quic_egmc@quicinc.com>
Assembler files were not migrated with the new <zephyr/...> prefix.
Note that the conversion has been scripted, refer to #45388 for more
details.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
In order to bring consistency in-tree, migrate all arch code to the new
prefix <zephyr/...>. Note that the conversion has been scripted, refer
to zephyrproject-rtos#45388 for more details.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
Commit d8f186aa4a ("arch: common: semihost: add semihosting
operations") encapsulated semihosting invocation in a per-arch
semihost_exec() function. There is a fixed register variable declaration
for the return value but this variable is not listed as an output
operand to respective inline assembly segments which is an error.
This is not reported as such by gcc and the generated code is still OK
in those particular instances but this is not guaranteed, and clang
does complain about such cases.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Add an API that utilizes the ARM semihosting mechanism to interact with
the host system when a device is being emulated or run under a debugger.
RISCV is implemented in terms of the ARM implementation, and therefore
the ARM definitions cross enough architectures to be defined 'common'.
Functionality is exposed as a separate API instead of syscall
implementations (`_lseek`, `_open`, etc) due to various quirks with
the ARM mechanisms that means function arguments are not standard.
For more information see:
https://developer.arm.com/documentation/dui0471/m/what-is-semihosting-
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
impl
In ARM parlance, the subroutine call return address is stored in the
"link register" or simply lr. Refer to it as lr which is clearer than
the anonymous x30 designation.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
ARM64 supports more memory mapping types for device memory (nGnRnE,
nGnRE, GRE), add these mapping support for os common mapping API
function z_phys_map().
Signed-off-by: Jiafei Pan <Jiafei.Pan@nxp.com>
It is not necessary to go through the full exception exit code.
This is simpler, smaller and faster.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Make it optimal without the need for an SVC/exception roundtrip on
every context switch. Performance numbers from tests/benchmarks/sched:
Before:
unpend 85 ready 58 switch 258 pend 231 tot 632 (avg 699)
After:
unpend 85 ready 59 switch 115 pend 138 tot 397 (avg 478)
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Get rid of all those global variables and scheduler locking.
Use the reguler IRQ exit path to let tests properly validate preemption.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Add CONFIG_SMP to fvp_baser_aemv8r_smp board.
Fix compile warnings by adding missing header file in arm_mpu.c.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
This commit mainly fixes the broadcast_ipi issue when one core broadcast
ipi to other cores using gic_raise_sgi. The issue doesn't affect the
functionality of Zephyr SMP but will happen when Zephyr runs on Xen.
Suppose Xen provides 4 CPUs to the Zephyr guest, for example, when cpu0
broadcasts ipi to the rest of the cores, the mask should be 0xE(0b1110),
but for now, Zephyr will send 0xFFFE. So for Xen, it will receive a
target list containing many invalid CPUs which don't exist. My solution
is: to generate the target list according to the online CPUs.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
In the Armv8R AArch64 profile[1], the Armv8R AArch64 is always in secure
mode. But the FVP_BaseR_AEMv8R before version 11.16.16 doesn't strictly
follow this rule. It still has some non-secure registers
(e.g. CNTHP_CTL_EL2).
Since version 11.16.16, the FVP_BaseR_AEMv8R has fixed this issue. The
CNTHP_XXX_EL2 registers have been changed to CNTHPS_XXX_EL2. So the
FVP_BaseR_AEMv8R (version >= 11.16.16) cannot boot Zephyr. This patch
will fix it.
[1] https://developer.arm.com/documentation/ddi0600/latest/
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Change-Id: If986f34dc080ae7a8b226bba589b6fe616a4260b
Currently, the DCACHE range invalidation can cause data corruption when
the ends of the given range is not aligned to a full cache line.
Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
Avoid executing ISRs using the thread stack as it might not be sized
for that. Plus, we do have IRQ stacks already set up for us.
The non-nested IRQ context is still (and has to be) saved on the thread
stack as the thread could be preempted.
The irq_offload case is never nested and always invoked with the
sched_lock held so it can be simplified a bit.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
This is an uint32_t so the proper register width must be used, otherwise
the adjacent structure member will be overwritten (didn't happen in
practice because of struct member alignment but still). This makes the
inc_nest_counter and dec_nest_counter macros rather unwieldy, especially
with upcoming changes, so let's just remove them.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Let's provide our own z_early_memset() and z_early_memcpy() rather than
making our own .bss clearing function that risk missing out on updates
to the main version.
Also remove extra stuff already provided by kernel_internal.h.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Avoid setting the switch_handler in the z_get_next_switch_handle() code
when the context is not fully saved yet to avoid a race against other
cores waiting on wait_for_switch().
See issue #40795 and discussion in #41840
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
For functions returning nothing, there is no need to document
with @return, as Doxgen complains about "documented empty
return type of ...".
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This commit adds support of Xen Enlighten page and initial support for
Xen event channels. It is needed for future Xen PV drivers
implementation.
Now enlighten page is mapped to the prepared memory area on
PRE_KERNEL_1 stage. In case of success event channel logic gets
inited and can be used ASAP after Zephyr start. Current implementation
allows to use only pre-defined event channels (PV console/XenBus) and
works only in single CPU mode (without VCPUOP_register_vcpu_info).
Event channel allocation will be implemented in future versions.
Signed-off-by: Dmytro Firsov <dmytro_firsov@epam.com>
This changes the arch_mem_domain_*() functions to return errors.
This allows the callers a chance to recover if needed.
Note that:
() For assertions where it can bail out early without side
effects, these are converted to CHECKIF(). (Usually means
that updating of page tables or translation tables has not
been started yet.)
() Other assertions are retained to signal fatal errors during
development.
() The additional CHECKIF() are structured so that it will bail
early if possible. If errors are encountered inside a loop,
it will still continue with the loop so it works as before
this changes with assertions disabled.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Call into z_thread_usage_stop() before ISR entry to avoid including
interrupt handling totals in thread usage stats.
This is pretty much exactly where we want it, just after the context
saving steps (which we can't elide since the hook is in C) and
immediately before the tracing hook for ISR entry. And as I'm reading
the code, this is purely for Zephyr-registered interrupts, meaning
that software exceptions will be accounted for (correctly) as part of
the excepting thread.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
This commit adds Xen hypervisor call interface for arm64 architecture.
This is needed for further development of Xen features in Zephyr.
Signed-off-by: Dmytro Firsov <dmytro_firsov@epam.com>
In some drivers, noncache memory need to be used for dma coherent
memroy, so add nocache memory segment mapping and support for ARM64
platforms.
The following variables definition example shows they will use nocache
memory allocation:
int var1 __nocache;
int var2 __attribute__((__section__(".nocache")));
Signed-off-by: Jiafei Pan <Jiafei.Pan@nxp.com>
Add the arm64 MMU arch_virt_region_align() implementation used
to return a possible virtual addres alignment in order to
optimize the MMU table layout and possibly avoid using L3 tables
and use some L1 & L3 blocks instead for most of the mapping.
Suggested-by: Nicolas Pitre <npitre@baylibre.com>
Signed-off-by: Neil Armstrong <narmstrong@baylibre.com>
When mapping the following:
device_map(&base0, DEVA_BASE, DEVA_SIZE, K_MEM_CACHE_NONE);
device_map(&base1, DEVB_BASE , DEVB_SIZE, K_MEM_CACHE_NONE);
with:
- DEVA_SIZE not multiple of a 4KB granule L2 block size (0x200000)
- DEVB_SIZE more than 2 x 4KB granule L2 block size
The mmu code will fill the first device_map() in a L3 table, then
on the second mapping the mmu code will complete the previous L3
table.
At the end of this table, the actual code will select an L2 block
instead of a table because the *virtual address* is multiple with
the L2 block size.
But if the physical address is not, the virtual block offset will
be ORed to the physical address, and not added.
Leading to a weird scenario where virtual memory is duplicated
resulting of the addresses ORing and not addition.
Example:
device_map(&base0, DEVA_BASE, 0x20000, K_MEM_CACHE_NONE);
device_map(&base1, 0x44000000 , 0x400000, K_MEM_CACHE_NONE);
First will result in VA 0x5ffe0000 and second in VA 0x5fbe0000.
The MMU code will use a table to map 0x5ffe0000 to 0x5fbfffff.
For 0x5fc00000 to 0x5fdfffff, since the VA is multiple of an L2
block size, the L3 table is not used.
But the L2 block description entry address is 0x44060000, meaning
that for each access in this L2 block, the following will be done:
0x44060000 | (VA & 1FFFFF)
This is working for the 0x5fc40000 to 0x5fc5ffff access, but for the
0x5fbc60000 (0x5fbe0000 + 0x80000) access the PA gets calculated as :
0x44060000 | (0x5fc60000 & 1FFFFF) = 0x44060000 | 0x60000 = 0x44060000
Instead of the expected 0x44080000.
The solution is to check if the PA descriptor is aligned with the
level block size, if not move to the next level.
Signed-off-by: Neil Armstrong <narmstrong@baylibre.com>
Add dynamic_areas_init. It will mark a mpu region as a dynamic region
area. The dynamic region areas is designed to be the background
regions, so that the system could re-program the thread regions on
the backgroud regions.
Add configure_dynamic_mpu_regions to re-program the thread regions on
the backgroud regions. The configure_dynamic_mpu_regions function is
the core function of implementing the userspace for the MPU. This
function is used in thread creation and context switch.
During context switch, the pre thread's regions should be disabled,
and the new thread's regions will be re-programed. Since the thread's
stack region will also be switched, there will be a problem before
new thread's region being re-programed which is the new thread's
stack usage. To avoid the exception generated by stack usage caused by
unprogramed new thread's stack region, I disable mpu first before
flush_dynamic_regions_to_mpu and then enable it.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Add a new macro MEM_DOMAIN_ALIGN_AND_SIZE for mmu and mpu mem
alignment.
MEM_DOMAIN_ALIGN_AND_SIZE is
- CONFIG_MMU_PAGE_SIZE, when mmu is enabled.
- CONFIG_ARM_MPU_REGION_MIN_ALIGN_AND_SIZE when mpu enabled.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Include the new introduced include/arch/arm64/mm.h instead of the
arm_mmu.h or arm_mpu.h.
Unify function names z_arm64_thread_pt_init/z_arm64_swap_ptables with
z_arm64_thread_mem_domains_init/z_arm64_swap_mem_domains for mmu and
mpu, because:
1. mmu and mpu have almost the same logic.
2. mpu doesn't have ptables.
3. using the function names help reducing "#if define" macros.
Similarly, change z_arm64_ptable_ipi to z_arm64_domain_sync_ipi
And fix a log bug in arm_mmu.c.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
This patch mainly moves mpu related code from
arch/arm64/core/cortex_r/mpu/ to arch/arm64/core/cortex_r/ and moves
the mpu header files from include/arch/arm64/cortex_r/mpu/ to
include/arch/arm64/cortex_r/
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Referring the Arm Generic Interrupt Controller Architecture
Specification GIC architecture version 3 and version 4 document
(see 2.2.1 Special INTIDs paragraph), these INTIDs are reserved
for special purposes and should be ignored for now.
For the ITS implementation, the INTID 1023 must be ignored since this
special INTID will trigger after an LPI acknowledge, thus triggering
the spurious interrupt handler.
The GICv3 Linux implementation ignores these INTIDs the same way.
Signed-off-by: Neil Armstrong <narmstrong@baylibre.com>
In case we enable a large number of IRQs, like when enabling LPIs using
interrupts > 8192, we hit an assembler error where the immediate value
is too large.
Copy the IRQ number into x1 to permit using a large IRQ number.
Signed-off-by: Neil Armstrong <narmstrong@baylibre.com>
Cleanup and preparation commit for linker script generator.
Zephyr linker scripts provides start and end symbols for each larger
areas in the linker script.
The symbols _image_text_start and _image_text_end sometimes includes
linker/kobject-text.ld. This mean there must be both the regular
__text_start and __text_end symbols for the pure text section, as well
as <group>_start and <group>_end symbols.
The symbols describing the text region which covers more than just the
text section itself will thus be changed to:
_image_text_start -> __text_region_start
_image_text_end -> __text_region_end
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Cleanup and preparation commit for linker script generator.
Zephyr linker scripts provides start and end symbols for each larger
areas in the linker script.
The symbols _image_rom_start and _image_rom_end corresponds to the group
ROMABLE_REGION defined in the ld linker scripts.
The symbols _image_rodata_start and _image_rodata_end is not placed as
independent group but covers common-rom.ld, thread-local-storage.ld,
kobject-rom.ld and snippets-rodata.ld.
This commit align those names and prepares for generation of groups in
linker scripts.
The symbols describing the ROMABLE_REGION will be renamed to:
_image_rom_start -> __rom_region_start
_image_rom_end -> __rom_region_end
The rodata will also use the group symbol notation as:
_image_rodata_start -> __rodata_region_start
_image_rodata_end -> __rodata_region_end
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
This prevent the new thread to attempt accessing cached ptable entries
which are no longer valid.
Signed-off-by: Manuel Argüelles <manuel.arguelles@coredumplabs.com>
