In order to bring consistency in-tree, migrate all arch code to the new
prefix <zephyr/...>. Note that the conversion has been scripted, refer
to zephyrproject-rtos#45388 for more details.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
When legacy mode is enabled, Zephyr includes both include/ and
include/zephyr. Allow the zefi.py script to accept multiple include
paths to cover this scenario.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
Where we have access to a bootstrap UEFI environment, it's productive
to use that console as the default printk handler. That avoids the
bringup hassle of trying to configure UART settings blindly, as has
been customary. It also emits nice text to the framebuffer on devices
with no serial port or other debug harness at all.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
When "eager FPU sharing" mode is enabled, FPU registers must be
initialised at the time of thread creation because the floating-point
context is always active and no further FPU initialisation is performed
later.
Note that, in case of the "lazy FPU sharing" mode, floating-point
context is inactive by default and the FPU is initialised when the
first floating-point instruction is executed.
Refer to the issue #44902 for more details.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
Under QEMU and SeaBIOS, everything gets to be printed
immediately after "Booting from ROM.." as there is no newline.
This prevents parsing QEMU console output for the very first
line where it needs to match from the beginning of the line.
So add a dummy newline here so the next output is at
the beginning of a line.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
In case of EFI, efi_init must be called before initializing early
serial: if that one as X86_SOC_EARLY_SERIAL_PCIDEV defined, its pcie
access will try to initialise pcie mmio access which one will try to
find an ACPI table. At this point, calling ACPI API prior to initialize
EFI will make RSDP looked up already... and since it cannot find it
without EFI being initialized first, ACPI is then broken.
Just moving early serial to initialize after multiboot/efi being setup.
Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
If such table pointer is present with EFI system table, this will speed
up ACPI initialization later on.
Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
As for Multiboot, let prep_c be aware of EFI boot.
In the futur, EFI will pass an argument to it.
Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
In order to mitigate at runtime whether it booted on multiboot or EFI,
let's introduce a dedicated x86 cpu argument structure which holds the
type and the actual pointer delivered by the method (multiboot_info, or
efi_system_table)
Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
Just a dummy function will do.
When enabled, the code does not need the #ifdef as cmake is handling
this properly already. This was also the wrong CONFIG_ used there
anyway.
Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
Improve code by using DEVICE_DT_GET_ONE instead of device_get_binding,
since the intel_vt_d device instance can be obtained at compile time.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
According to Kconfig guidelines, boolean prompts must not start with
"Enable...". The following command has been used to automate the changes
in this patch:
sed -i "s/bool \"[Ee]nables\? \(\w\)/bool \"\U\1/g" **/Kconfig*
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
When Zephyr runs directly on actual hardware, it will be always
directing MSI messages to BSP (BootStrap Processor). This was fine until
Zephyr could be ran on virtualizor that may NOT run it on BSP.
So directing MSI messages on current processor. If Zephyr runs on actual
hardware, it will be BSP since such setup is always made at boot time by
the BSP. On other use case it will be whatever is relevant at that time.
Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
Depending on whether X2APIC is enabled or not, it will be safer to grab
such ID from the right place.
Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
This will centralize CPUID related accessors. There was no need for it
so far, but this is going to change.
Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
When XIP is not enabled, z_data_copy() already falls back to an empty
function. No need to ifdef it.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
For functions returning nothing, there is no need to document
with @return, as Doxgen complains about "documented empty
return type of ...".
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
In fact, in case of VT-D being enabled, it will require to get an
address and data for its own MSI based interrupts which cannot be
remapped (i.e.: will directly go to the relevant APIC).
This is necessary to get the Fault event supported in VT-D.
Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
This will not only be used by MSI remapping but by all relevant
interrupts.
Fix also IRTE settings:
- handle x2apic for destination id
- destination mode is always logical (as for IOAPIC)
Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
SHV bit depends on the number of vectors allocated.
If it's facing a multi-vector MSI array, it will set the bit.
If not the bit must be 0.
Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
Refactor to handle this case. This is valid only when MSI multi-vector
feature is enabled.
Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
As all interruption need to go through VT-D, calling vt-d remap will
happen on lower level as seen next, so make sure all pcie related
irq/vector get tighten to their respective allocated IRTE.
Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
Allocate an IRTE for all irq being connected through
arch_irq_connect_dynamic(). This will be mandatory since VT-D expects to
filter all interruptions (but the one it generates, as we will see
later).
Taking into account CONFIG_INTEL_VTD_ICTL_XAPIC_PASSTHROUGH, which could
help for debugging.
Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
There is no need to differentiate with multi-vector or not, MSI vs
MSI-x: all need to be remapped if Intel VT-D is on.
Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
The warning below appears once -Waddress-of-packed-mem is enabled:
/home/carles/src/zephyr/zephyr/arch/x86/core/acpi.c: In function
'z_acpi_find_table':
/home/carles/src/zephyr/zephyr/arch/x86/core/acpi.c:190:24: warning:
taking address of packed member of 'struct acpi_xsdt' may result in an
unaligned pointer value [-Waddress-of-packed-member]
190 | for (uint64_t *tp = &xsdt->table_ptrs[0]; tp < end; tp++) {
To avoid the warning, use an intermediate void * variable.
More info in #16587.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Storing the state where this is the first GDB break can be done
in the main GDB stub code. There is no need to store the state
in architecture layer.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This adds some architecture-specific functions to read/write
registers for the GDB stub. This is in preparation for the actual
introduction of these functions in the core GDB stub code to
avoid breaking the build in between commits.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This changes the arch_mem_domain_*() functions to return errors.
This allows the callers a chance to recover if needed.
Note that:
() For assertions where it can bail out early without side
effects, these are converted to CHECKIF(). (Usually means
that updating of page tables or translation tables has not
been started yet.)
() Other assertions are retained to signal fatal errors during
development.
() The additional CHECKIF() are structured so that it will bail
early if possible. If errors are encountered inside a loop,
it will still continue with the loop so it works as before
this changes with assertions disabled.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
replace with version.parse from packaging module.
prevent this warning message:
DeprecationWarning: The distutils package is deprecated
and slated for removal in Python 3.12. Use setuptools or
check PEP 632 for potential alternatives
Signed-off-by: Julien Massot <julien.massot@iot.bzh>
Call into z_thread_usage_stop() before ISR entry to avoid including
interrupt handling totals in thread usage stats.
This has to go into the assembly immediately before the callback-based
dispatch. Note that the dispatch code was putting the vector number
in RCX, which was unfortunate as that's a caller-saved register.
Would be nice to clean this up in the future so it lives in a
preserved register but it's mildly complicated to make work with the
way we do the stack layout right now.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
This adds arch_float_enable() and arch_float_disable() to x86-64.
As x86-64 always has FP/SSE enabled, these operations are basically
no-ops. These are added just for the completeness of arch interface.
Fixes#38022
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Cleanup and preparation commit for linker script generator.
Zephyr linker scripts provides start and end symbols for each larger
areas in the linker script.
The symbols _image_text_start and _image_text_end sometimes includes
linker/kobject-text.ld. This mean there must be both the regular
__text_start and __text_end symbols for the pure text section, as well
as <group>_start and <group>_end symbols.
The symbols describing the text region which covers more than just the
text section itself will thus be changed to:
_image_text_start -> __text_region_start
_image_text_end -> __text_region_end
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Cleanup and preparation commit for linker script generator.
Zephyr linker scripts provides start and end symbols for each larger
areas in the linker script.
The symbols _image_rom_start and _image_rom_end corresponds to the group
ROMABLE_REGION defined in the ld linker scripts.
The symbols _image_rodata_start and _image_rodata_end is not placed as
independent group but covers common-rom.ld, thread-local-storage.ld,
kobject-rom.ld and snippets-rodata.ld.
This commit align those names and prepares for generation of groups in
linker scripts.
The symbols describing the ROMABLE_REGION will be renamed to:
_image_rom_start -> __rom_region_start
_image_rom_end -> __rom_region_end
The rodata will also use the group symbol notation as:
_image_rodata_start -> __rodata_region_start
_image_rodata_end -> __rodata_region_end
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
This function should be pinned in memory instead of simply
putting it in the boot section, as this function will be
used when new threads are created at runtime.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
If generic section is not present at boot, the thread stack
may not be in physical memory. Unconditionally page in the stack
instead of relying on page fault to speed up a little bit
on starting the thread.
Also, this prevents a double fault during thread setup when
setting up stack permission in z_x86_userspace_enter().
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
When converting the address and size arguments for extra mappings,
the script assumes they are always base 16. This is not always
the case. So let Python's own int() decides how to interpret
the values as it supports "0x" prefix also.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
With demand paging, it is possible for data pages to not be
present in physical memory. The gen_mmu.py script is updated
so that, if so desired, the generic sections are marked
non-present so the paging mechanism can bring them in
if needed.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
If the BSS section is not present in physical memory at boot,
do not zero the section, or else page faults would occur.
The zeroing of BSS will be done once the paging mechanism
has been initialized.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
add .S file extension suffix into CMAKE_ASM_SOURCE_FILE_EXTENSIONS,
because clang from OneApi can't recongnize them as asm files on
windows, then they won't be added into build system.
Signed-off-by: Chen Peng1 <peng1.chen@intel.com>
Correct the wrong operand of clflush instruction. The old operand
points to a location inside stack and doesn't work. The new one
works well by taking linux kernel code as reference.
End address instead of size should get round up
Add Kconfig option to disable the usage of mfence intruction for
SoC that has clfulsh but no mfence supported.
Signed-off-by: Dong Wang <dong.d.wang@intel.com>
The code depends on the order of evaluation 'z_x86_check_stack_bounds'
function arguments.
The solution is to assign these values to variables and then pass
them in.
The fix would be to make 2 local variables, assign them the values
of _df_esf.esp and .cs, and then call the function with those 2 local
variables as arguments.
Found as a coding guideline violation (MISRA R13.2) by static
coding scanning tool.
Change "int reason" to "unsigned reason" like in other functions.
Signed-off-by: Maksim Masalski <maksim.masalski@intel.com>
According to the Zephyr Coding Guideline all switch statements
shall be well-formed.
Add a comment to the empty default case.
Add a LOG_ERR to the default case.
Found as a coding guideline violation (MISRA R16.1) by static
coding scanning tool.
Signed-off-by: Maksim Masalski <maksim.masalski@intel.com>
commit 5e9c583c24 ("arch/x86_64: Terrible, awful hackery to
bootstrap entry") introduced a terrible trick which begins execution
at the bottom of .locore with a jump, which then gets replaced with
NOP instructions for the benefit of 16 bit real mode startup of the
other CPUs later on.
But I forgot that EFI enters in 64 bit code natively, and so never
hits that path. And moving it to the 64 bit setup code doesn't work,
because at that point when we are NOT loaded from EFI, we already have
the Zephyr page tables in place that disallow writes to .locore.
So do it in the EFI loader, which while sort of a weird place, has the
benefit of being in C instead of assembly.
Really all this code needs to go away. A proper x86 entry
architecture would enter somewhere in the main blob, and .locore
should be a tiny stub we copy in at runtime.
Fixes#36107
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Since physical memory is no longer wholly identity mapped,
it is not needed to set the VM size to be larger than
physical memory size. The VM size was 2GB (max physical
memory size of x86 boards) + 1GB (for memory mappings).
So simply shrink the size to 1GB, as the kernel size is
small and we still have a large chunk of space to do
memory mapping.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
With ACPI doing dynamic memory mapping and unmapping
to access ACPI tables, there is no need to identity
map all the physical memory anymore. So remove
the "select" statement in ACPI kconfig.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Instead of accessing ACPI tables through physical address, do
memory mapping/unmapping so they can be accessed via virtual
addresses. This allows us to avoid identity mapping all
physical memory, and thus no need for a page table large enough
to map everything.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This limits the search for Extended BIOS Data Area (EBDA) to
0x80000 to 0x100000 as this is usually the area for it.
If 0000:040e has an address not pointing to this area, it is
probably an invalid address, and should not be de-referenced
to avoid segfault.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Most arch's CMakeLists.txt contain rules to add compiler and linker
flags for coverage if CONFIG_COVERAGE is enabled, but 4 of them were
missing this.
Instead, set the coverage flags in arch/common/CMakeLists.txt which
affects all archs.
Signed-off-by: Jeremy Bettis <jbettis@chromium.org>
Essential type of RHS operand (64 bit) is wider than essential
type of composite expression in LHS operand (32 bit).
LHS entry_val is 32 bit, and RHS (phys+offset) is 64 bit.
Cast RHS composite expression to the (pentry_t) type.
Found as a coding guideline violation (MISRA R10.7) by static
coding scanning tool.
Signed-off-by: Maksim Masalski <maksim.masalski@intel.com>
The 16 bit bootstrap code for SMP CPUs was using the 286-era "lmsw"
instruction (load machine status word) to set the protected bit in CR0
(which is the modern evolution of the same register), presumably
because this is 16 bit code and we can't move a dword into CR0.
But that's wrong, because the full instruction set *is* available in
real mode on a 386, you just have to use a operand size prefix to get
to it, which the assembler emits for you automatically when you use
the .code16 directive.
Write this conventionally and use modern (e.g. 1986-era) instructions.
It also has the advantage of not confusing much more modern
hypervisors like ACRN by issuing instructions they (and I!) never knew
existed.
Fixes#35076
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Because of a historical misunderstanding, by default the ACRN
hypervisor wants to load Zephyr at address 0x1000 and enter the binary
at that same address. This entry point corresponds to the __start
symbol of the build they were given, which is a 1-cpu non-SMP
configuration. Unfortunately, when we build with
CONFIG_MP_NUM_CPUS=1, the code in locore.S #if's out the 16 bit entry
point for the auxiliary CPUs at the start of the section. So in the
build ACRN received, the start address happened to be 0x7000, the same
address we need to launch the AP processors from.
That's right: under ACRN, the SAME ADDRESS used to enter the OS in 32
bit mode needs to be used later to boot CPUs running in 16 bit real
mode!
The solution, such as it is, is to put a 32 bit jump at the entry
address which hops to the 32 bit OS entry code, and then scribble NOP
instructions over that jump once we get there so that the next time we
reach that address (in real mode) we fall through to the correct
entry.
This patch should be considered a temporary workaround. While it
works on all x86 hardware, it's not really needed. A much better
solution would be to eliminate the locore linker region entirely
(which causes other headaches) and enter the Zephyr binary in a 32 bit
address somewhere in the contiguous high memory area. All that locore
is needed for is the 16 bit bootstrap code for SMP processors, which
is ~6 instructions and can be copied in from the kernel at runtime.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
From the point of checking the info pointer value all code in the
z_multiboot_init() function depends on it being non-NULL. Therefore,
simply return from the function if it's NULL.
Fixes#33084
Signed-off-by: Johan Hedberg <johan.hedberg@intel.com>
When loaded via EFI, we obviously don't have a multiboot info pointer
available (we might have an EFI system table, but zefi doesn't pass
that through yet). Don't try to parse the "whatever garbage was in
%rbp" as a multiboot table.
The configuration is a little clumsy, as strictly our EFI kconfig just
says we're "building for" EFI but not that we'll boot that way. And
tests like arch/x86/info are trying to set CONFIG_MULTIBOOT=n
unconditionally, when it really should be something they detect from
devicetree or wherever.
Fixes#33545
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
This marks code and data within x86/ia32 so they are going to
reside in boot and pinned regions. This is a step to enable
demand paging for whole kernel.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This adds both boot and pinned sections to the linker
script for ia32. This is required for enabling demand
paging for kernel and data.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
There is exactly one function being defined with TEXT_START
macro so the x86-32 __start can appear at the beginning of
text section. Since no one else is using it, better remove
TEXT_START to simplify things.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This implements arch_page_phys_get() to translate mapped
virtual addresses back to physical addresses.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
In z_mem_manage_init(), z_free_page_count is only manipulated
after all reserved pages are marked, and will reflect
the actual number of page frames being added to the free page
frame list. Manipulating z_free_page_count before this is
going to mess up the accounting, so remove the code to
decrement z_free_page_count in arch_reserved_pages_update()
under x86.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
There was a restriction that KERNEL_VM_OFFSET must equal to
SRAM_OFFSET so that page directory pointer (PDP) or page
directory (PD) can be reused. This is not very practical in
real world due to various hardware designs, especially those
where SRAM is not aligned to PDP or PD. So rework those bits.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Remove the config BOOT_TIME_MEASUREMENT and corresponding #ifdef'd code
throughout (kernel/init.c, idle.c, core/common.S , reset.S, ... ) which
hold the extern hooks for z_timestamp_main and z_timestamp_idle in the
removed boot_time test suite.
Signed-off-by: Jennifer Williams <jennifer.m.williams@intel.com>
This adds the bits to the gen_mmu.py script so that extra mappings
can be added with caching disabled. This is useful for mapping
MMIO regions where caching is not desired.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
There is a possibility that the TSC frequency calculation
is divided by zero. So this fixes the issue by repeatedly
trying to get the delta clock cycles and delta TSC cycles
until they both are not zero.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Rephrasing away from ain't, which is informal, uncommon, and can
be viewed as substandard or 'slang'.
Signed-off-by: Jennifer Williams <jennifer.m.williams@intel.com>
Reboot functionality has nothing to do with PM, so move it out to the
subsys/os folder.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
Both operands of an operator in which the usual arithmetic
conversions are performed shall have the same essential
type category.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
The variable tsc_freq is not accessible in user thread
and is thus preventing user threads to convert cycles to ns.
So make tsc_freq available globally in default memory
domain so conversion is possible.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Tests of a value against zero should be made explicit, unless the
operand is effectively Boolean. This is based on MISRA rule 14.4.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
This changes the assert when a large page is encountered to
copying the page directory entry to the new page directory.
This is needed when a large page entry is generated by
gen_mmu.py. Note that this still asserts when there are entries
of large page at higher level.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This makes the gen_mmu.py script to error out if the reserved space
for page table in zephyr_prebuilt.elf is not large enough to
accommodate the generated page table. Let catch this at build time
instead of mysterious hangs when loading the page table at boot.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
The whole page table is pre-allocated at build time and is
dependent on the range of address space. This kconfig allows
reserving extra pages (of size CONFIG_MMU_PAGE_SIZE) to
the page table so that gen_mmu.py can make use of these
extra pages.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This patch replaces ENOSYS into ENOTSUP to keep consistency with
the return value specification of k_float_enable().
Signed-off-by: Katsuhiro Suzuki <katsuhiro@katsuster.net>
This patch introduce new API to enable FPU of thread. This is pair of
existed k_float_disable() API. And also add empty arch_float_enable()
into each architectures that have arch_float_disable(). The arc and
riscv already implemented arch_float_enable() so I do not touch
these implementations.
Motivation: Current Zephyr implementation does not allow to use FPU
on main and other system threads like as work queue. Users need to
create an other thread with K_FP_REGS for floating point programs.
Users can use FPU more easily if they can enable FPU on running
threads.
Signed-off-by: Katsuhiro Suzuki <katsuhiro@katsuster.net>
The only user of arch_mem_domain_destroy was the deprecated
k_mem_domain_destroy function which has now been removed. So remove
arch_mem_domain_destroy as well.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
This unifies all the display of region size in hex.
Some of them are there to aid in figuring out the end of
a memory region so it is easier if they are already in hex.
This also fixes the display of address range where the end
is off by one and should be (base + size - 1).
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
After page table is load, we should be executing in virtual
address space. Therefore we need to set ESP to the virtual
address of interrupt stack for the boot process.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This reverts commit d40e8ede8e.
This fixes triple faults after wiping the identity mapping of
physical memory when running entering userspace.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This reverts commit 7d32e9f9a5.
We now allow the kernel to be linked virtually. This patch:
- Properly converts between virtual/physical addresses
- Handles early boot instruction pointer transition
- Double-maps SRAM to both virtual and physical locations
in boot page tables to facilitate instruction pointer
transition, with logic to clean this up after completed.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This reuses the page directory pointer table (PAE=y) or page
directory (PAE=n) to point to next level page directory table
(PAE=y) or page tables (PAE=n) to identity map the physical
memory. This gets rid of the extra memory needed to host
the extra mappings which are only used at boot. Following
patches will have code to actual unmap physical memory
during the boot process, so this avoids some wasting of
memory.
Since no extra memory needs to be reserved, this also reverts
commit ee3d345c09
("x86: mmu: reserve more space for page table if linking in virt").
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This allows specifying second --verbose in command line to
enable more messages. Two new ones have been added to aid
in debugging code for mapping and setting permission to
a single page.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
There is no need to use this kconfig, as the phys-to-virt
offset is enough to figure out if the kernel is linked in
virtual address space in gen_mmu.py.
For code, use Z_VM_KERNEL instead.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
With the introduction of Z_MEM_*_ADDR for physical<->virtual
address translation, there is no need to have x86 specific
versions.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Since the removal of Quark-based boards, there are no user of
Minute-IA. Also, the generic x86 SoC is not exactly Minute-IA
so change it to use a fairly safe CPU_ATOM.
Fixes#14442
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
For some unknown reason, the pagetable address for _df_tss.cr3
did not get translated from virtual to physical. However,
the translation is done if the pointer to pagetable is obtained
through reference to the first array element (instead of simply
through the name of array). Without CR3 pointing to the page
table via physical address, double fault does not work. So
fixing this by being explicit with the page table pointer.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
When adding a new thread to memory domain, there is a NULL check
to figure out if a thread is being migrated to another memory
domain. However, the NULL check is AFTER physical-to-virtual
address translation which means (NULL + offset) != NULL anymore.
This results in calling reset_region() with an invalid page table
pointer. Fix this by doing the NULL check before address
translation.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
When linking in virtual address space, we still need physical
addresses in SRAM to be mapped so platform can boot from physical
memory and to access structure necessary for boot (e.g. GDT and
IDT). So we need to enlarge the reserved space for page table
to accommodate this.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
We have been having the assumption that the physical memory
is identity-mapped to virtual address space. However, with
the ability to set CONFIG_KERNEL_VM_BASE separately from
CONFIG_SRAM_BASE_ADDRESS, the assumption is no longer valid.
This changes the boot code in x86 32-bit, so that once
the page table is loaded, we can proceed with executing in
the virtual address space. So do a long jump to virtual
address just before calling z_x86_prep_c. From this point on,
code execution is in virtual address space.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
When linking in virtual address space, we still need physical
addressed in SRAM to be mapped so platform can boot from physical
memory and to access structure necessary for boot (e.g. GDT and
IDT). So identity maps the kernel in SRAM.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
When the kernel is mapped into virtual address space
that is different than the physical address space,
the dynamic GDT generation uses the virtual addresses.
However, the GDT table is required at boot before
page table is loaded where the virtual addresses are
invalid. So make sure GDT generation is using
physical addresses.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
There is an assumption made in the page table generation code
that the kernel would occupy the same physical and virtual
addresses. However, we may want to map the kernel into
a virtual address space which differs from kernel's physical
address space. For example, with demand paging enabled on
kernel code and data, we can accommodate kernel that is
larger than physical memory size, and may want to utilize
a bigger virtual address space. So add address translation
in the gen_mmu.py script for this.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This adds virtual address translation to a few variables
used in crt0.S. This is needed as they are linked at
virtual addresses but before page table is loaded,
they are not available at virtual addresses and must be
referred via physical addresses.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
When feeding &z_shared_kernel_page_start directly to
Z_X86_PHYS_ADDR(), the compiler would complain array subscript
out of bound if linking in virtual address space. So cast it
into uintptr_t first before translation.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Add a newer, much smaller and simpler implementation of abort and
join. No need to involve the idle thread. No need for a special code
path for self-abort. Joining a thread and waiting for an aborting one
to terminate elsewhere share an implementation. All work in both
calls happens under a single locked path with no unexpected
synchronization points.
This fixes a bug with the current implementation where the action of
z_sched_single_abort() was nonatomic, releasing the lock internally at
a point where the thread to be aborted could self-abort and confuse
the state such that it failed to abort at all.
Note that the arm32 and native_posix architectures, which have their
own thread abort implementations, now see a much simplified
"z_thread_abort()" internal API.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
This changes x86 to use CONFIG_SRAM_OFFSET instead of
arch-specific CONFIG_X86_KERNEL_OFFSET. This allows the common
MMU macro Z_BOOT_VIRT_TO_PHYS() and Z_BOOT_PHYS_TO_VIRT() to
function properly if we ever need to map the kernel into
virtual address space that does not have the same starting
physical address.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This adds a new kconfig CONFIG_SRAM_OFFSET to specify the offset
from beginning of SRAM where the kernel begins. On x86 and
PC compatible platforms, the first 1MB of RAM is reserved and
Zephyr should not link anything there. However, this 1MB still
needs to be mapped by the MMU to access various platform related
information. CONFIG_SRAM_OFFSET serves similar function as
CONFIG_KERNEL_VM_OFFSET and is needed for proper phys/virt
address translations.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Instead of doing these in assembly, use the common z_bss_zero()
and z_data_copy() C functions instead. This simplifies code
a bit and we won't miss any additions to these two functions
(if any) under x86 in the future (as x86_64 was actually not
clearing gcov bss area).
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This moves calling z_loapic_enable() from crt0.S into
z_x86_prep_c(). This is done so we can move BSS clearing
and data section copying inside z_x86_prep_c() as
these are needed before calling z_loapic_enable().
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This adds a new kconfig to enable the use of memory map.
This map can be populated automatically if
CONFIG_MULTIBOOT_MEMMAP=y or can be manually defined
via x86_memmap[].
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This is an hidden option to indicate we are building for
PC-compatible devices (where there are BIOS, ACPI, etc.
which are standard on such devices).
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Before accessing the multiboot data passed by the bootloader,
we need to map the memory first. This adds the code to map
the memory if necessary.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
We assume that all x86 CPUs do have clflush instructions.
And the cache line size is now provided through DTS.
So detecting clflush instruction as well as the cache line size is no
longer required at runtime and thus removed.
Signed-off-by: Tomasz Bursztyka <tomasz.bursztyka@linux.intel.com>
This adds X86 keyword to the kconfigs to indicate these are
for x86. The old options are still there marked as
deprecated.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
With x86, there are usually memory regions that are reserved
for firmware and device MMIOs. We don't want to use these
pages for memory mapping so mark them as reserved at boot.
The weakly defined x86_memmap contains the list of memory
regions which can be overriden by SoC or board configurations.
Also, with CONFIG_MULTIBOOT_MEMMAP=y, the memory regions
are populated from multiboot provided data.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
The x86_64 SysV ABI requires 16 byte alignment for the stack pointer
during execution of normal code. That means that on entry to an
ABI-compatible C function (which is reached via a CALL instruction
that pushes the return address) the RSP register must be MISaligned by
exactly 8 bytes. The kernel mode thread setup got this right, but we
missed the equivalent condition in userspace entry.
The end result was a misaligned stack, which is surprisingly robust
for most use. But recent toolchains have starting doing some more
elaborate vectorization, and the resulting SSE instructions started
failing in userspace on the misaliged loads.
Note that there's a comment about optimization: we're doing the stack
alignment in the "wrong place" and are needlessly wasting bytes in
some cases. We should see the raw stack boundaries where we are
setting up RSP values. Add a FIXME to this effect, but don't touch
anything as this patch is a targeted bugfix.
Also fix a somewhat embarassing 32-bit-ism that would have truncated
the address of a userspace stack that we tried to put above 4G.
Fixes#31018
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
This adds a new GEN_ABSOLUTE_SYM_KCONFIG() specifically for
generating absolute symbols in assembly for kconfig values.
This is needed as the existing GEN_ABSOLUTE_SYM() with
constraints in extended assembly parses the "value" as
signed 32-bit integers. An unsigned 32-bit integer with
MSB set results in a negative number in the final binary.
This also prevents integers larger than 32-bit. So this
new macro simply puts the value inline within the assembly
instrcution instead of having it as parameter.
Fixes#31562
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
On Intel processors, if GS is not zero and is being set to
zero, GS_BASE is also being set to zero. This would interfere
with the actual use of GS_BASE for usespace. To avoid accidentally
clearing GS_BASE, simply set GS to 0 at boot, so any subsequent
clearing of GS will not clear GS_BASE.
The clearing of GS_BASE was discovered while trying to figure out
why the mem_protect test would hang within 10-20 repeated runs.
GDB revealed that both GS and GS_BASE was set to zero when the tests
hanged. After setting GS to zero at boot, the mem_protect tests
were running repeated for 5,000+ times without hanging.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
We've already enabled full RAM mapping if ACPI is enabled, also
set a large 3GB address space size, these systems are not RAM-
constrained (they are PC platforms) and they have large MMIO
config spaces for PCIe.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
All arch_ APIs and macros are implemented, and the page fault
handling code will call into the core kernel.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Pre-allocation of paging structures is now required, such that
no allocations are ever needed when mapping memory.
Instantiation of new memory domains may still require allocations
unless a common page table is used.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
We no longer use a page pool to draw memory pages when doing
memory map operations. We now preallocate the entire virtual
address space so no allocations are ever necessary when mapping
memory.
We still need memory to clone page tables, but this is now
expressed by a new Kconfig X86_MAX_ADDITIONAL_MEM_DOMAINS
which has much clearer semantics than specifying the number
of pages in the pool.
The default address space size is now 8MB, but this can be
tuned by the application.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
A more comprehensive solution would use E820 enumeration, but we
are unlikely to ever care that much, as we intend to use demand
paging on microcontrollers and not PC-like hardware. This is
really to just prevent QEMU from crashing.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
This is only needed if the base address of SRAM doesn't
have the same alignment as the base address of the virtual
address space.
Fix the calculations on X86 where this is the case.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
When zefi.py was changed to pass compiler and objcopy the flag to
objcopy for the EFI target was dropped. This is because the current
SDK (0.12.1) doesn't support that target type for objcopy. However,
target is necessary for the images to be created correctly and boot.
Switch back to use the host objcopy as a stop gap fix, until the SDK
can support target for EFI.
Fixes: #31517
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
This changes the timing functions to use TSC to gather
timing information instead of using the timer for
scheduling as it provides higher resolution for timing
information.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
