Commit graph

56,357 commits

Author SHA1 Message Date
Daniel Leung
c3669c045a tests: debug: add new coredump_backends
This adds a new coredump_backends test for coredump backends,
and currently tests both the logging and flash partition
backends.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-01-21 22:08:59 -05:00
Daniel Leung
2b53f3ded2 debug: coredump: add new backend using flash partition
This adds another simple backend to use flash partition as
backing storage.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-01-21 22:08:59 -05:00
Daniel Leung
6c569142ea debug: coredump: add shell commands for logging backend
This adds some shell commands to the logging backend. Since
this is a simple backend, only get/clear errors are
implemented.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-01-21 22:08:59 -05:00
Daniel Leung
1dc813853a debug: coredump: add query and command APIs
This adds two new APIs to the coredump subsystem to perform
query and command. These can be used to query coredump subsys
for information, and to perform commands such as finding
out if there is a stored coredump.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-01-21 22:08:59 -05:00
Daniel Leung
c4de586fbe debug: coredump: no need to track errors at frontend
The coredump frontend (mostly) consists of wrappers of backend
functions so there is really no need to track errors at
the frontend level. Let the backends deal with their own errors
and this simplifies the code a bit.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-01-21 22:08:59 -05:00
Daniel Leung
236e7c7e33 debug: coredump: remove unused code
Originally there was a null backend but it was removed before
code was merged. However, some leftover code still refers to
this null backend. So remove the leftovers.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-01-21 22:08:59 -05:00
Daniel Leung
1387e7c7c2 scripts: coredump: fix crashing on non-existent memory region
get_mem_region() may return None for a non-existent memory region
so we need to check first before looking at its properties.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-01-21 22:08:59 -05:00
Andy Ross
e932a1537c arch: tests: Document interrupt delivery behavior after arch_irq_disable()
Upcoming changes from Andrew that add a global timeout to the kernel
broke because of some voodoo behavior in the kernel/context test.  It
will use arch_irq_disable() on the timer interrupt directly to prevent
interrupts and measure timekeeping in their absence.  But some
architectures[1] don't reliably deliver interrupts that arrive, which
means that a running timeout that exists across this period will
result in a corrupt timeout queue.

Document that rule for architectures, move the offending test to the
end of the test suite (to minimize the chance of interacting with
other test code) and put a giant warning about the situation on it.
Long term, we may want to rework this test to do its job in other
ways.

[1] On x86, the interrupt disable happens at the IO-APIC level, while
interrupt latching and delivery is downstream in each CPU's Local
APIC.  An IO-APIC masked interrupt is completely invisible to the APIC
and can never be delivered once the line goes low.

Fixes #31333

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2021-01-21 17:50:01 -05:00
Martí Bolívar
e8f12344b0 doc: dts: improve 'get zephyr.dts' HOWTO
Tweak the language to try to be more clear in the HOWTO entry for
looking at a board's devicetree.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-01-21 17:21:13 -05:00
Flavio Ceolin
c408765d70 timer: legacy_api: Fix a build dependency in legacy_api header
z_timer_idle_enter is declared only when CONFIG_TICKLESS_IDLE is
selected. This function is not implemented anywhere, but the only
driver including this header is not TICKLESS_CAPABLE. So, no undefined
reference will happen.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2021-01-21 17:20:32 -05:00
Flavio Ceolin
1dd2d414b8 kernel: build: Make TICKLESS_CAPABLE a hidden option
TICKLESS_CAPABLE is an option that has to be selected by the target.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2021-01-21 17:20:32 -05:00
Flavio Ceolin
d6d4a832a4 kernel: build: Make TICKLESS_KERNEL depends on TICKLESS_CAPABLE
Tickless kernel option depends on tickless capable being selected by
the target.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2021-01-21 17:20:32 -05:00
Flavio Ceolin
c2b2462e70 doc: build: Change a header position
The pre-build header was after its introduction paragraph.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2021-01-21 17:20:15 -05:00
Flavio Ceolin
2185c5b399 doc: build: Missing italic
Just using italic when referencing a script. Following this
documentation pattern.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2021-01-21 17:20:15 -05:00
Armando Visconti
45d0508bdf drivers/sensor: iis2dlpc: Add multi-instance support
Make this driver multi-instance and use the new API.

Signed-off-by: Armando Visconti <armando.visconti@st.com>
2021-01-21 16:49:55 -05:00
Andy Ross
8ccd506966 doc: Clean up docs on thread termination behavior
Discussion about how to re-spawn threads led to the discovery that our
documentation on exactly when that was legal was ambiguous and
confusing.  Rewrite it to be explicit.

Fixes #28970

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2021-01-21 16:48:01 -05:00
Johann Fischer
ecc57a12a6 include: power: include missing header for slist
Include missing header for slist.

Signed-off-by: Johann Fischer <johann.fischer@nordicsemi.no>
2021-01-21 16:47:29 -05:00
Andrew Boie
63fc93e21f mmu: promote public APIs
These are application facing and are prefixed with k_.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-21 16:47:00 -05:00
Andrew Boie
6af87480d0 tests: context: disable if DEMAND_PAGING
Until #31333 is resolved, the periodic timer in the eviction
algorithm interacts with this test in such a way that the system
deadlocks.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-21 16:47:00 -05:00
Andrew Boie
7a642f81ab mmu: backing stores reserve page fault room
If we evict enough pages to completely fill the backing store,
through APIs like k_mem_map(), z_page_frame_evict(), or
z_mem_page_out(), this will produce a crash the next time we
try to handle a page fault.

The backing store now always reserves a free storage location
for actual page faults.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-21 16:47:00 -05:00
Andrew Boie
7370895c0e tests: demand_paging: add more API tests
Add remaining APIs in mem_manage.h even though not all have
been promoted to public (yet).

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-21 16:47:00 -05:00
Andrew Boie
d7e6bc3e84 kernel: add z_num_pagefaults_get()
Simple counter of number of successfully handled page faults by
the core kernel.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-21 16:47:00 -05:00
Andrew Boie
5f0a7201c7 CODEOWNERS: add demand paging subdir
Adding myself as original author.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-21 16:47:00 -05:00
Andrew Boie
060462d5c6 tests: add intial demand paging testcase
More to be added, but for now show that we can map more
anonymous memory than we physically have, and that reading/
writing to it works as expected.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-21 16:47:00 -05:00
Andrew Boie
0f8dc1c109 tests: add basic k_mem_map() test
Show we can measure free memory properly and map a page of
anonymous memory, which has been zeroed and is writable.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-21 16:47:00 -05:00
Andrew Boie
a45486e1d5 mmu: pin the whole kernel
This will enable testing of the implementation until the
critical set of pages is identified and known to the
kernel.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-21 16:47:00 -05:00
Andrew Boie
cd0a50d5c9 qemu_x86_tiny: enable demand paging
This target is specifically for simulating x86 micro-
controllers with limited memory.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-21 16:47:00 -05:00
Andrew Boie
7711c9a82d x86: implement demand paging APIs
All arch_ APIs and macros are implemented, and the page fault
handling code will call into the core kernel.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-21 16:47:00 -05:00
Andrew Boie
f8fd9932cd demand_paging: add RAM-based demo backing store
Will be used by QEMU targets for testing purposes.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-21 16:47:00 -05:00
Andrew Boie
200ea42437 demand_paging: add NRU algorithm
Simple textbook Not Recently Used eviction algorithm.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-21 16:47:00 -05:00
Andrew Boie
24eb50d7f4 tests: mem_map: pin test pages
These get mapped to multiple virtual addresses and must be
pinned.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-21 16:47:00 -05:00
Andrew Boie
2fe1fc53c8 kernel: add demand paging implementation
Implement runtime APIs for pinning, paging in, and evicting
memory, as well as the page fault hook called from architecture
code.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-21 16:47:00 -05:00
Andrew Boie
d5b8fe16ad kernel: add app-facing demand paging APIs
Routines to evict memory, page-in memory, and set pinned state
at runtime.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-21 16:47:00 -05:00
Andrew Boie
3e51a7a775 kernel: add demand paging internal interfaces
APIs used by backing store and eviction algorithms.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-21 16:47:00 -05:00
Andrew Boie
1c2bd343ff demand_paging: add infra for demand paging modules
Backing stores and eviction algorithms will be included here.
Exactly one must be chosen, with a default option to leave
the implementation to the application.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-21 16:47:00 -05:00
Andrew Boie
b8ae437967 kernel: add demand paging arch interfaces
Architecture layer hooks for demand paging. See
doxygen for these API definitions for more details.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-21 16:47:00 -05:00
Andrew Boie
48cc63b4a3 arch: add CONFIG_DEMAND_PAGING
Indicates at the kernel level that demand paging is active.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-21 16:47:00 -05:00
Andrew Boie
e9bfd64110 mmu: ensure gperf data is mapped
Page tables created at build time may not include the
gperf data at the very end of RAM. Ensure this is mapped
properly at runtime to work around this.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-21 16:47:00 -05:00
Andrew Boie
db56722729 mmu: arch_mem_map() may no longer fail
Pre-allocation of paging structures is now required, such that
no allocations are ever needed when mapping memory.

Instantiation of new memory domains may still require allocations
unless a common page table is used.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-21 16:47:00 -05:00
Andrew Boie
64f05d443a x86: pre-allocate address space
We no longer use a page pool to draw memory pages when doing
memory map operations. We now preallocate the entire virtual
address space so no allocations are ever necessary when mapping
memory.

We still need memory to clone page tables, but this is now
expressed by a new Kconfig X86_MAX_ADDITIONAL_MEM_DOMAINS
which has much clearer semantics than specifying the number
of pages in the pool.

The default address space size is now 8MB, but this can be
tuned by the application.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-21 16:47:00 -05:00
Andrew Boie
d2b7261076 qemu_x86_tiny: don't use first megabyte at all
Just tell the kernel that RAM starts 1MB in, period.
Better simulation of a low-memory microcontroller as
we're not managing a very large number of page frames
we'll never use.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-21 16:47:00 -05:00
Andrew Boie
3660040e22 x86: only map the kernel image
The policy is changed and we no longer map all page frames.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-21 16:47:00 -05:00
Andrew Boie
e966244de2 newlib: clamp max heap size on MMU systems
Previously, newlib claimed all free physical memory in the
system.

Now, the kernel manages this, allowing for memory to be
used via k_mem_map() calls.

Establish an upper bound to how much newlib will try to
claim on system startup, instead of trying to take all
of it, allowing other parts of the system to also map
anonymous memory.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-21 16:47:00 -05:00
Andrew Boie
bf91269738 newlib: memory-map the heap, cleanups
We now draw heap memory from an anonymous memory mapping
instead of a hard-coded region past the kernel image,
which is no longer mapped by default.

Some readability cleanups were made to a particuarly
horrible set of nested ifdefs. A few types were adjusted.

sbrk()'s count argument is an intptr_t, not an int.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-21 16:47:00 -05:00
Andrew Boie
73561be500 arch: remove KERNEL_RAM_SIZE
We don't map all RAM at boot any more, just the kernel image.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-21 16:47:00 -05:00
Andrew Boie
a7d8b3385a x86: tests: pagetables: fix assumptions
All RAM may not be mapped. Check the mapping for the main kernel
image and the locore if it exists.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-21 16:47:00 -05:00
Andrew Boie
51e3c9efa5 x86: reserve the first megabyte
A more comprehensive solution would use E820 enumeration, but we
are unlikely to ever care that much, as we intend to use demand
paging on microcontrollers and not PC-like hardware. This is
really to just prevent QEMU from crashing.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-21 16:47:00 -05:00
Andrew Boie
9111ec2c19 mmu: add k_mem_free_get()
Return the amount of physical anonymous memory remaining.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-21 16:47:00 -05:00
Andrew Boie
69d39af5e6 kernel: add k_mem_map() interface
Allows applications to increase the data space available to Zephyr
via anonymous memory mappings. Loosely based on mmap().

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-21 16:47:00 -05:00
Andrew Boie
2ca5fb7e06 kernel: add page frame management
Initialize the page frame ontology at boot and update it
when we do memory mappings.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2021-01-21 16:47:00 -05:00