Commit graph

47967 commits

Author SHA1 Message Date
Andrew Boie
6c97ab3167 mmu: promote public APIs
These are application facing and are prefixed with k_.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-23 19:47:23 -05:00
Andrew Boie
79d73063af tests: context: disable if DEMAND_PAGING
Until #31333 is resolved, the periodic timer in the eviction
algorithm interacts with this test in such a way that the system
deadlocks.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-23 19:47:23 -05:00
Andrew Boie
c7be5dddda mmu: backing stores reserve page fault room
If we evict enough pages to completely fill the backing store,
through APIs like k_mem_map(), z_page_frame_evict(), or
z_mem_page_out(), this will produce a crash the next time we
try to handle a page fault.

The backing store now always reserves a free storage location
for actual page faults.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-23 19:47:23 -05:00
Andrew Boie
cad944e259 tests: demand_paging: add more API tests
Add remaining APIs in mem_manage.h even though not all have
been promoted to public (yet).

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-23 19:47:23 -05:00
Andrew Boie
60d306642e kernel: add z_num_pagefaults_get()
Simple counter of number of successfully handled page faults by
the core kernel.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-23 19:47:23 -05:00
Andrew Boie
9a12d6028b CODEOWNERS: add demand paging subdir
Adding myself as original author.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-23 19:47:23 -05:00
Andrew Boie
ef65597b73 tests: add intial demand paging testcase
More to be added, but for now show that we can map more
anonymous memory than we physically have, and that reading/
writing to it works as expected.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-23 19:47:23 -05:00
Andrew Boie
146eaf4335 tests: add basic k_mem_map() test
Show we can measure free memory properly and map a page of
anonymous memory, which has been zeroed and is writable.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-23 19:47:23 -05:00
Andrew Boie
611b626b39 mmu: pin the whole kernel
This will enable testing of the implementation until the
critical set of pages is identified and known to the
kernel.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-23 19:47:23 -05:00
Andrew Boie
4d6c20b9d7 qemu_x86_tiny: enable demand paging
This target is specifically for simulating x86 micro-
controllers with limited memory.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-23 19:47:23 -05:00
Andrew Boie
ed22064e27 x86: implement demand paging APIs
All arch_ APIs and macros are implemented, and the page fault
handling code will call into the core kernel.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-23 19:47:23 -05:00
Andrew Boie
b0d608e3c9 demand_paging: add RAM-based demo backing store
Will be used by QEMU targets for testing purposes.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-23 19:47:23 -05:00
Andrew Boie
367cfa4946 demand_paging: add NRU algorithm
Simple textbook Not Recently Used eviction algorithm.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-23 19:47:23 -05:00
Andrew Boie
d50b2419b8 tests: mem_map: pin test pages
These get mapped to multiple virtual addresses and must be
pinned.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-23 19:47:23 -05:00
Andrew Boie
a5cb878144 kernel: add demand paging implementation
Implement runtime APIs for pinning, paging in, and evicting
memory, as well as the page fault hook called from architecture
code.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-23 19:47:23 -05:00
Andrew Boie
b9bbef2a2c kernel: add app-facing demand paging APIs
Routines to evict memory, page-in memory, and set pinned state
at runtime.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-23 19:47:23 -05:00
Andrew Boie
431b7c0fe5 kernel: add demand paging internal interfaces
APIs used by backing store and eviction algorithms.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-23 19:47:23 -05:00
Andrew Boie
993cf9f8eb demand_paging: add infra for demand paging modules
Backing stores and eviction algorithms will be included here.
Exactly one must be chosen, with a default option to leave
the implementation to the application.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-23 19:47:23 -05:00
Andrew Boie
a6eca9fab6 kernel: add demand paging arch interfaces
Architecture layer hooks for demand paging. See
doxygen for these API definitions for more details.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-23 19:47:23 -05:00
Andrew Boie
56a9e7b91e arch: add CONFIG_DEMAND_PAGING
Indicates at the kernel level that demand paging is active.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-23 19:47:23 -05:00
Andrew Boie
ecb25fec51 mmu: ensure gperf data is mapped
Page tables created at build time may not include the
gperf data at the very end of RAM. Ensure this is mapped
properly at runtime to work around this.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-23 19:47:23 -05:00
Andrew Boie
299a2cf62e mmu: arch_mem_map() may no longer fail
Pre-allocation of paging structures is now required, such that
no allocations are ever needed when mapping memory.

Instantiation of new memory domains may still require allocations
unless a common page table is used.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-23 19:47:23 -05:00
Andrew Boie
b0b7756756 x86: pre-allocate address space
We no longer use a page pool to draw memory pages when doing
memory map operations. We now preallocate the entire virtual
address space so no allocations are ever necessary when mapping
memory.

We still need memory to clone page tables, but this is now
expressed by a new Kconfig X86_MAX_ADDITIONAL_MEM_DOMAINS
which has much clearer semantics than specifying the number
of pages in the pool.

The default address space size is now 8MB, but this can be
tuned by the application.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-23 19:47:23 -05:00
Andrew Boie
ea10c98c08 qemu_x86_tiny: don't use first megabyte at all
Just tell the kernel that RAM starts 1MB in, period.
Better simulation of a low-memory microcontroller as
we're not managing a very large number of page frames
we'll never use.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-23 19:47:23 -05:00
Andrew Boie
5c47bbc501 x86: only map the kernel image
The policy is changed and we no longer map all page frames.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-23 19:47:23 -05:00
Andrew Boie
e2b62b8abb newlib: clamp max heap size on MMU systems
Previously, newlib claimed all free physical memory in the
system.

Now, the kernel manages this, allowing for memory to be
used via k_mem_map() calls.

Establish an upper bound to how much newlib will try to
claim on system startup, instead of trying to take all
of it, allowing other parts of the system to also map
anonymous memory.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-23 19:47:23 -05:00
Andrew Boie
b01177156f newlib: memory-map the heap, cleanups
We now draw heap memory from an anonymous memory mapping
instead of a hard-coded region past the kernel image,
which is no longer mapped by default.

Some readability cleanups were made to a particuarly
horrible set of nested ifdefs. A few types were adjusted.

sbrk()'s count argument is an intptr_t, not an int.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-23 19:47:23 -05:00
Andrew Boie
893822fbda arch: remove KERNEL_RAM_SIZE
We don't map all RAM at boot any more, just the kernel image.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-23 19:47:23 -05:00
Andrew Boie
61651976c2 x86: tests: pagetables: fix assumptions
All RAM may not be mapped. Check the mapping for the main kernel
image and the locore if it exists.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-23 19:47:23 -05:00
Andrew Boie
f3e9b61a91 x86: reserve the first megabyte
A more comprehensive solution would use E820 enumeration, but we
are unlikely to ever care that much, as we intend to use demand
paging on microcontrollers and not PC-like hardware. This is
really to just prevent QEMU from crashing.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-23 19:47:23 -05:00
Andrew Boie
5db615bb38 mmu: add k_mem_free_get()
Return the amount of physical anonymous memory remaining.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-23 19:47:23 -05:00
Andrew Boie
8ccec8eba6 kernel: add k_mem_map() interface
Allows applications to increase the data space available to Zephyr
via anonymous memory mappings. Loosely based on mmap().

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-23 19:47:23 -05:00
Andrew Boie
e35f179db3 kernel: add page frame management
Initialize the page frame ontology at boot and update it
when we do memory mappings.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-23 19:47:23 -05:00
Andrew Boie
73a3e05e40 kernel: add CONFIG_ARCH_HAS_RESERVED_PAGE_FRAMES
We will need this to run on x86 with PC-like hardware.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-23 19:47:23 -05:00
Andrew Boie
35ec07dfe0 tests: x86: pagetables: pass if userspace disabled
_app_smem doesn't exist in this case.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-23 19:47:23 -05:00
Andrew Boie
951e4a8a9a arm64: linker: define z_mapped_* symbols
Add linker symbols corresponding to the start and end of the
mapped Zephyr image. This is not used by the ARM arch yet, but
is required to compile the core kernel MMU code.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-23 19:47:23 -05:00
Andrew Boie
94aedfbc77 x86: linker: define z_mapped_* symbols
We will use this to map the kernel instead of all RAM.
The end of the kernel is always page-aligned, regardless
of CONFIG_SRAM_REGION_PERMISSIONS as it must be mapped.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-23 19:47:23 -05:00
Andrew Boie
5791434ee5 linker-defs: add syms for kernel image bounds
These are needed on MMU systems and define where the kernel
image resides in virtual memory at boot so that it may be
memory-mapped.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-23 19:47:23 -05:00
Andrew Boie
69355d13a8 arch: add KERNEL_VM_OFFSET
This is only needed if the base address of SRAM doesn't
have the same alignment as the base address of the virtual
address space.

Fix the calculations on X86 where this is the case.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-23 19:47:23 -05:00
Peter Bigot
3746ebc305 Revert "device: inject device dependency information from devicetree nodes"
This reverts commit b61515320e.

Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
2021-01-23 18:01:03 -05:00
Peter Bigot
affa7a1c7e Revert "device: add post-process of elf file to manage device handles"
This reverts commit 40d3653758.

Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
2021-01-23 18:01:03 -05:00
Peter Bigot
56d6b071fb Revert "tests: devicetree: add test for devicetree node data"
This reverts commit 58b035ad3b.

Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
2021-01-23 18:01:03 -05:00
Peter Bigot
6a6cf2183e Revert "shell: device_service: add dependency information"
This reverts commit a3af137c26.

Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
2021-01-23 18:01:03 -05:00
Peter Bigot
9c70fb1e18 Revert "linker: common-rom: tweak section naming to feet all linkers"
This reverts commit fd5fa203b6.

Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
2021-01-23 18:01:03 -05:00
Kumar Gala
4ac2c420f9 ci: Bump up number of builds
To get the daily build to hopefully run completely w/o timeouts lets
increase the number of builders.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2021-01-23 10:36:55 -06:00
Ying ming
9625576b3c test: scheduling :add new conf file
Add a conf file to make sure the kernel will use simple linked-list
ready queue as scheduling algorithm. This operation will increase module
testcase coverage and z_priq_dumb_add z_prj_dum_remove function are
called.

Signed-off-by: Ying ming <mingx.ying@intel.com>
2021-01-23 11:04:51 -05:00
Shubham Kulkarni
9f139b5d83 soc/esp32: Move full logging library to IRAM
This commit updates linker script to move full logging module into
internal RAM

Signed-off-by: Shubham Kulkarni <shubham.kulkarni@espressif.com>
2021-01-23 08:43:10 -05:00
Shubham Kulkarni
d621dad21b soc/esp32: Move logging library to IRAM with CONFIG_LOG_MINIMAL
This commit updates linker script to move minimal log module to internal
RAM

Signed-off-by: Shubham Kulkarni <shubham.kulkarni@espressif.com>
2021-01-23 08:43:10 -05:00
Shubham Kulkarni
ca39c7edd7 soc: esp32: linker: Fix backtrace for Invalid cache access exception
This change moves .rodata for panic handler and fatal.c into DRAM
Moves panic handler and its dependent functions into IRAM

Signed-off-by: Shubham Kulkarni <shubham.kulkarni@espressif.com>
2021-01-23 08:43:10 -05:00
Shubham Kulkarni
8b7da334d5 arch: xtensa: Print backtrace from panic handler
This change uses stack frame to print backtrace once exception occurs
Printing backtrace helps to identify the cause of exception

Signed-off-by: Shubham Kulkarni <shubham.kulkarni@espressif.com>
2021-01-23 08:43:10 -05:00