There are two macros for declaring stack arrays:
K_KERNEL_STACK_ARRAY_DEFINE:
Defines the array, allocating storage and setting the section name
K_KERNEL_STACK_ARRAY_EXTERN
Declares the name of a stack array allowing code to reference
the array which must be defined elsewhere
arch/arm/include/aarch32/cortex_m/stack.h was mis-using
K_KERNEL_STACK_ARRAY_DEFINE to declare z_interrupt_stacks by sticking
'extern' in front of the macro use. However, when this macro also set
the object file section for the symbol, having two of those caused a
conflict in the compiler due to the automatic unique name mechanism used
for sections to allow unused symbols to be discarded during linking.
This patch makes the header use the correct macro.
Signed-off-by: Keith Packard <keithp@keithp.com>
In some drivers, noncache memory need to be used for dma coherent
memroy, so add nocache memory segment mapping and support for ARM64
platforms.
The following variables definition example shows they will use nocache
memory allocation:
int var1 __nocache;
int var2 __attribute__((__section__(".nocache")));
Signed-off-by: Jiafei Pan <Jiafei.Pan@nxp.com>
Don't allow to enable multiple register banks / fast
interrupts if we have only one interrupt priority level.
NOTE: we duplicate some checks by adding dependencies to ARC
Kconfig and adding build-time checks in C code. We do it
intentionally as for some reason we can violate dependencies
in architecture-level Kconfig by adding incorrect default in
SoC-level Kconfig. Such violation happens without any
warnings / errors from the Kconfig.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
Add the arm64 MMU arch_virt_region_align() implementation used
to return a possible virtual addres alignment in order to
optimize the MMU table layout and possibly avoid using L3 tables
and use some L1 & L3 blocks instead for most of the mapping.
Suggested-by: Nicolas Pitre <npitre@baylibre.com>
Signed-off-by: Neil Armstrong <narmstrong@baylibre.com>
This reverts commit 67d290540e.
The script is actually used to generate the _soc_inthandlers.h
file when introducing a new Xtensa SoC. So restore it.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
The assert log of z_priv_stacks_ram_start failed to build due to passing
&z_priv_stacks_ram_start instead of just z_priv_stacks_ram_start.
Fixes#39190
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
When mapping the following:
device_map(&base0, DEVA_BASE, DEVA_SIZE, K_MEM_CACHE_NONE);
device_map(&base1, DEVB_BASE , DEVB_SIZE, K_MEM_CACHE_NONE);
with:
- DEVA_SIZE not multiple of a 4KB granule L2 block size (0x200000)
- DEVB_SIZE more than 2 x 4KB granule L2 block size
The mmu code will fill the first device_map() in a L3 table, then
on the second mapping the mmu code will complete the previous L3
table.
At the end of this table, the actual code will select an L2 block
instead of a table because the *virtual address* is multiple with
the L2 block size.
But if the physical address is not, the virtual block offset will
be ORed to the physical address, and not added.
Leading to a weird scenario where virtual memory is duplicated
resulting of the addresses ORing and not addition.
Example:
device_map(&base0, DEVA_BASE, 0x20000, K_MEM_CACHE_NONE);
device_map(&base1, 0x44000000 , 0x400000, K_MEM_CACHE_NONE);
First will result in VA 0x5ffe0000 and second in VA 0x5fbe0000.
The MMU code will use a table to map 0x5ffe0000 to 0x5fbfffff.
For 0x5fc00000 to 0x5fdfffff, since the VA is multiple of an L2
block size, the L3 table is not used.
But the L2 block description entry address is 0x44060000, meaning
that for each access in this L2 block, the following will be done:
0x44060000 | (VA & 1FFFFF)
This is working for the 0x5fc40000 to 0x5fc5ffff access, but for the
0x5fbc60000 (0x5fbe0000 + 0x80000) access the PA gets calculated as :
0x44060000 | (0x5fc60000 & 1FFFFF) = 0x44060000 | 0x60000 = 0x44060000
Instead of the expected 0x44080000.
The solution is to check if the PA descriptor is aligned with the
level block size, if not move to the next level.
Signed-off-by: Neil Armstrong <narmstrong@baylibre.com>
Some Xtensa SoCs may not have that many levels of interrupts.
So limit the call to DEF_INT_C_HANDLER() to only supported
levels to avoid calling non-existent functions.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Add dynamic_areas_init. It will mark a mpu region as a dynamic region
area. The dynamic region areas is designed to be the background
regions, so that the system could re-program the thread regions on
the backgroud regions.
Add configure_dynamic_mpu_regions to re-program the thread regions on
the backgroud regions. The configure_dynamic_mpu_regions function is
the core function of implementing the userspace for the MPU. This
function is used in thread creation and context switch.
During context switch, the pre thread's regions should be disabled,
and the new thread's regions will be re-programed. Since the thread's
stack region will also be switched, there will be a problem before
new thread's region being re-programed which is the new thread's
stack usage. To avoid the exception generated by stack usage caused by
unprogramed new thread's stack region, I disable mpu first before
flush_dynamic_regions_to_mpu and then enable it.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Add a new macro MEM_DOMAIN_ALIGN_AND_SIZE for mmu and mpu mem
alignment.
MEM_DOMAIN_ALIGN_AND_SIZE is
- CONFIG_MMU_PAGE_SIZE, when mmu is enabled.
- CONFIG_ARM_MPU_REGION_MIN_ALIGN_AND_SIZE when mpu enabled.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Include the new introduced include/arch/arm64/mm.h instead of the
arm_mmu.h or arm_mpu.h.
Unify function names z_arm64_thread_pt_init/z_arm64_swap_ptables with
z_arm64_thread_mem_domains_init/z_arm64_swap_mem_domains for mmu and
mpu, because:
1. mmu and mpu have almost the same logic.
2. mpu doesn't have ptables.
3. using the function names help reducing "#if define" macros.
Similarly, change z_arm64_ptable_ipi to z_arm64_domain_sync_ipi
And fix a log bug in arm_mmu.c.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
This patch mainly moves mpu related code from
arch/arm64/core/cortex_r/mpu/ to arch/arm64/core/cortex_r/ and moves
the mpu header files from include/arch/arm64/cortex_r/mpu/ to
include/arch/arm64/cortex_r/
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Referring the Arm Generic Interrupt Controller Architecture
Specification GIC architecture version 3 and version 4 document
(see 2.2.1 Special INTIDs paragraph), these INTIDs are reserved
for special purposes and should be ignored for now.
For the ITS implementation, the INTID 1023 must be ignored since this
special INTID will trigger after an LPI acknowledge, thus triggering
the spurious interrupt handler.
The GICv3 Linux implementation ignores these INTIDs the same way.
Signed-off-by: Neil Armstrong <narmstrong@baylibre.com>
In case we enable a large number of IRQs, like when enabling LPIs using
interrupts > 8192, we hit an assembler error where the immediate value
is too large.
Copy the IRQ number into x1 to permit using a large IRQ number.
Signed-off-by: Neil Armstrong <narmstrong@baylibre.com>
For some platforms, like NXP's IMX8 or Mediatek's MT8195,
the size of an interrupt vector table entry is 0x1C bytes,
less than usual (0x30 for Intel's platforms).
So, the interrupt handlers don't fit in the vector table
entries.
I've added a small indirection to bypass this size
constraint and moved the default handlers to the end
of vector table, renaming them to
_Level\LVL\()VectorHelper.
For this, I've added a generic configuration -
XTENSA_SMALL_VECTOR_TABLE_ENTRY.
Signed-off-by: Iuliana Prodan <iuliana.prodan@nxp.com>
In some cases the 'reschedule' code path is executed when the current
thread is the same as the next thread in the ready Q. If this happens,
the swap_return_value of the thread is ifalsely being reset to -EAGAIN.
This commit prevents the rescheduling code to run if the current thread
is the same as the thread in the ready Q.
Signed-off-by: Wolfgang Reißnegger <gnagflow@fb.com>
This adds arch_float_enable() and arch_float_disable() to x86-64.
As x86-64 always has FP/SSE enabled, these operations are basically
no-ops. These are added just for the completeness of arch interface.
Fixes#38022
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
A simple WAITI isn't sufficient in all cases. The cAVS 2.5 hardware
uses WAITI as the entry state for per-core power gating, which is very
difficult to debug. Provide a fallback that simply spins in the idle
loop waiting for interrupts to provide a stable system while this
feature stabilizes.
Also, the SOF code for those platforms references a known bug with the
Xtensa LX6 core IP (or at least some versions), and will prefix the
WAIT instruction with 128 NOP.N's followed by an ISYNC and EXTW. This
bug hasn't been seen under Zephyr yet, and details are sketchy. But
the code is simply enough to import and works correctly.
Place both workaround under new kconfig variables and select them both
(even though they're actually mutually exclusive -- if you select both
CPU_IDLE_SPIN overrides) for cavs_v25.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
On CPU startup, When we reach the cache flush code in arch_switch(),
the outgoing thread is a dummy. The behavior of the existing code was
to leave the existing value in the SR unchanged (probably NULL at
startup). Then the context switch would walk from that address up to
the top of the outgoing stack, flushing everything in between. That's
wrong, because the outgoing stack is a real pointer (generally the
interrupt stack of the current CPU), and we're flushing everything in
memory underneath it.
This also reverts commit 29abc8adc0 ("xtensa: fix booting secondary
cores on the dummy thread"), which appears to have been an early
attempt to address this issue. It worked (modulo all the extra and
potentially incorrect flushing) on cavs v1.5/1.8 because of the way
the entry code worked there. But on 2.5 we now hit the first context
switch in a case where those extra lines are in address space already
marked unwritable by the CPU, so the flush explodes.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
__cxa_atexit implementation provided by MWDT startup code calls
malloc which isn't supported right now. As we don't support
calling static destructors in Zephyr let's provide our own
__cxa_atexit stub and get rid of MWDT startup libs
entirely.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
This commit adds the half-precision (16-bit) floating-point
configurations to the ARM AArch32 architectures.
Enabling CONFIG_FP16 has the effect of specifying `-mfp16-format`
option (in case of GCC) which allows using the half-precision floating
point types such as `__fp16` and `_Float16`.
Note that this configuration can be used regardless of whether a
hardware FPU is available or supports half-precision operations.
When an FP16-capable FPU is not available, the compiler will
automatically provide the software emulations.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
This commit specifies the intList section in the IDT_LIST region in the
arch/common CMakeLists.txt file.
It uses zephyr_linker_section to setup the intList section for first
pass linker file and configures the section to hold irq_info and
intList input section.
For second pass linker file, the irq_info and intList input sections are
placed in the /DISCARD/ section.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Converted existing ld script templates into CMake files.
This commit takes the common-ram.ld, common-rom.ld, debug-sections.ld,
and thread-local-storage.ld and creates corresponding CMake files for
the linker script generator.
The CMake files uses the new Zephyr CMake functions:
- zephyr_linker_section()
- zephyr_linker_section_configure()
- zephyr_linker_section_obj_level()
to generate the same linker result as the existing C preprocessor based
scheme.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
For IMX, for timer interrupt, the interrupt handler
was not the correct one executed and that’s because
the handlers were not at the expected address.
For IMX the size constraint of the interrupt vector
table entry is 0x1C bytes of code, less than usual.
I've added a small indirection to bypass this size
constraint and moved the default handlers to the end
of vector table, renaming them to
_Level\LVL\()VectorHelper.
Signed-off-by: Iuliana Prodan <iuliana.prodan@nxp.com>
ld linker will only resolve undefined symbols inside functions that is
actually being called.
However, not all linkers behaves this way. Certain linkers, for example
armlink, resolves all undefined symbols even if during a later stage at
the linking the function will be pruned.
Therefore `ifdef CONFIG_GEN_ISR_TABLES` has been placed to safeguard
functions that will call undefined symbols when CONFIG_GEN_ISR_TABLES=y.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
z_arm_do_syscall is only defined and used when CONFIG_USERSPACE=y.
Defining the symbol z_arm_do_syscall in assembly without a corresponding
implementation is fine for GNU ld as long as the function is not
actively called, but armlink fails to link in such cases.
Safegaurd GTEXT(z_arm_do_syscall) so the symbol is only referenced when
actively used, that is when CONFIG_USERSPACE=y.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Cleanup and preparation commit for linker script generator.
Zephyr linker scripts provides start and end symbols for each larger
areas in the linker script.
The symbols _image_text_start and _image_text_end sometimes includes
linker/kobject-text.ld. This mean there must be both the regular
__text_start and __text_end symbols for the pure text section, as well
as <group>_start and <group>_end symbols.
The symbols describing the text region which covers more than just the
text section itself will thus be changed to:
_image_text_start -> __text_region_start
_image_text_end -> __text_region_end
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Cleanup and preparation commit for linker script generator.
Zephyr linker scripts provides start and end symbols for each larger
areas in the linker script.
The symbols _image_rom_start and _image_rom_end corresponds to the group
ROMABLE_REGION defined in the ld linker scripts.
The symbols _image_rodata_start and _image_rodata_end is not placed as
independent group but covers common-rom.ld, thread-local-storage.ld,
kobject-rom.ld and snippets-rodata.ld.
This commit align those names and prepares for generation of groups in
linker scripts.
The symbols describing the ROMABLE_REGION will be renamed to:
_image_rom_start -> __rom_region_start
_image_rom_end -> __rom_region_end
The rodata will also use the group symbol notation as:
_image_rodata_start -> __rodata_region_start
_image_rodata_end -> __rodata_region_end
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Cleanup and preparation commit for linker script generator.
Zephyr linker scripts provides start and end symbols for each section,
and sometimes even size and LMA start symbols.
Generally, start and end symbols uses the following pattern, as:
Section name: foo
Section start symbol: __foo_start
Section end symbol: __foo_end
However, this pattern is not followed consistently.
To allow for linker script generation and ensure consistent naming of
symbols then the following pattern is introduced consistently to allow
for cleaner linker script generation.
Section name: foo
Section start symbol: __foo_start
Section end symbol: __foo_end
Section size symbol: __foo_size
Section LMA start symbol: __foo_load_start
This commit aligns the symbols for _ramfunc_ram/rom to other symbols and
in such a way they follow consistent pattern which allows for linker
script and scatter file generation.
The symbols are named according to the section name they describe.
Section name is `ramfunc`
The following symbols are aligned in this commit:
- _ramfunc_ram_start -> __ramfunc_start
- _ramfunc_ram_end -> __ramfunc_end
- _ramfunc_ram_size -> __ramfunc_size
- _ramfunc_rom_start -> __ramfunc_load_start
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Add support of ARC mpu v6
* minimal region size down to 32 bytes
* maximal region number up to 32
* not support uncacheable region and volatile uncached region
* clean up mpu code for better readablity
Signed-off-by: Yuguo Zou <yuguo.zou@synopsys.com>
This function should be pinned in memory instead of simply
putting it in the boot section, as this function will be
used when new threads are created at runtime.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
If generic section is not present at boot, the thread stack
may not be in physical memory. Unconditionally page in the stack
instead of relying on page fault to speed up a little bit
on starting the thread.
Also, this prevents a double fault during thread setup when
setting up stack permission in z_x86_userspace_enter().
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
When converting the address and size arguments for extra mappings,
the script assumes they are always base 16. This is not always
the case. So let Python's own int() decides how to interpret
the values as it supports "0x" prefix also.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
With demand paging, it is possible for data pages to not be
present in physical memory. The gen_mmu.py script is updated
so that, if so desired, the generic sections are marked
non-present so the paging mechanism can bring them in
if needed.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
If the BSS section is not present in physical memory at boot,
do not zero the section, or else page faults would occur.
The zeroing of BSS will be done once the paging mechanism
has been initialized.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This adds the kconfig to allow reserving a number of page frames
which do not count towards free memory. This is to ensure that
there are enough page frames available for paging code and data.
Or else, it would be possible to exhaust all page frames via
anonymous memory mappings.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Enable RISC-V GP relative addressing by linker relaxation to reduce
the code size. It optimizes addressing of globals in small data section
(.sdata).
The gp initialization at program start needs each SoC support. Also,
if RISC-V SoC has custom linker script, SoC should provide
__global_pointer$ symbol in it's linker script.
Signed-off-by: Jim Shu <cwshu@andestech.com>
This prevent the new thread to attempt accessing cached ptable entries
which are no longer valid.
Signed-off-by: Manuel Argüelles <manuel.arguelles@coredumplabs.com>
Increases the default CONFIG_TEST_EXTRA_STACKSIZE for the 32-bit RISC-V
architecture. This fixes the portability.posix.fs test on the
qemu_riscv32 platform.
Signed-off-by: Maureen Helm <maureen.helm@nxp.com>
This reverts commit 7b09d031fa. Because
context save of GP register is removed, we don't need to initialize GP
at thread init. GP will be a constant value so that it could only be
initialized at program start.
Signed-off-by: Jim Shu <cwshu@andestech.com>
RISC-V global pointer (GP) register is neither caller nor callee
register, and it's a constant value in the single ELF file. Thus, we
don't need to save/restore GP at ISR enter/exit. Remove it to optimize
context switch performance.
Signed-off-by: Jim Shu <cwshu@andestech.com>
Related to github #22290. Getting interrupt during mpu buffer validate
is corrupting index register. Fix applied to ARC is to disable
interrupts during the buffer validate operation.
Signed-off-by: Phil Erwin <phil.erwin@lexmark.com>
Cortex-A/R does not have hardware supported nested interrupts, but it is
easily emulatable using the nesting level stored in the kernel
structure.
Signed-off-by: Bradley Bolen <bbolen@lexmark.com>
Add functionality based on Cortex-M that enables recovery from a data
abort using zephyr's exception recovery framework. If there is a
registered z_exc_handle for a function, then use its fixup address if
that function aborts.
Signed-off-by: Bradley Bolen <bbolen@lexmark.com>
