Commit graph

3196 commits

Author SHA1 Message Date
Daniel Leung
2dfae4a0f7 kernel: demand_paging: allow reserving page frames
This adds the kconfig to allow reserving a number of page frames
which do not count towards free memory. This is to ensure that
there are enough page frames available for paging code and data.
Or else, it would be possible to exhaust all page frames via
anonymous memory mappings.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-08-26 21:16:22 -04:00
Daniel Leung
2117a2a44b kernel: app_smem: allowing pinning memory partitions
This allows memory partitions to be put into the pinned
section so they are available during boot. For example,
the stack guard (in libc partition) is needed during boot
but before the paging mechanism is initialized. Without
pinning it in physical memory, it would fault in early
boot process.

A new cmake property app_smem,pinned_partitions is
introduced so that additional partitions can be pinned
if needed.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-08-26 21:16:22 -04:00
Daniel Leung
e88afd2c37 kernel: mmu: pin/unpin boot sections during boot process
During boot process, the boot sections need to be pinned in
memory to prevent them from being paged out (to avoid
pages being paged out and immediately paged in again).
Once the boot process is completed (just before calling main()),
the boot sections can be unpinned so the memory can be
used for demand paging for paging in data pages.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-08-26 21:16:22 -04:00
Daniel Leung
f32ea4433c kernel: demand_paging: clear BSS after paging is initialized
If BSS section is not present in memory at boot, it would not
have been cleared as the data pages are not in physical memory.
Manipulating those pages would result in page faults.
In this scenario, zeroing BSS can only be done once the paging
mechanism has been initialized. So do it there.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-08-26 21:16:22 -04:00
Daniel Leung
7771d27525 kernel: mmu: move when page fault is counted
The beginning of code in do_page_fault() is to pin the page
in memory if it is already present in physical memory.
It is there so that if a page is not present, it can proceed
to perform page-in and then pin it. So the counting of
page faults needs to be moved after the pinning code so
it actually counts page faults, and not counting pinning
operations when the page is already present.

Also clarify the comment on the goto statement as it is not
correct.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-08-26 21:16:22 -04:00
Daniel Leung
ebbfde9742 kernel: move z_main_stack into pinned section
The z_main_stack is needed before paging mechanism is initialized
so put the stack into the pinned section to avoid page faults.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-08-26 21:16:22 -04:00
Daniel Leung
c38634fa33 kernel: mmu: fix assigning unaligned addr to page frame
In do_page_fault(), the incoming page fault address is not
aligned, and it was unconditionally assigned to the page
frame virtual address field. If the backing store simply
returns the virtual address without processing in
k_mem_paging_backing_store_location_get(), this unaligned
address will be passed to arch_mem_page_out(). On x86,
it is further passed to range_map() which asserts if
the physical address is not page aligned. So align
the address to page size before assigning it to the page
frame virtual address field.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-08-26 21:16:22 -04:00
Flavio Ceolin
d9aa414831 kernel: work_q: Add an init function
k_work_queue_start receives a struct that is expected to be
uninitialized (zeroed). Otherwise the behavior is undefined.

Following the Zephyr semantics, this pr introduce a new init function
for this struct.

Fixes #36865

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2021-08-25 22:07:04 -04:00
Jordan Yates
b01e41ccdd device: supported devices visitor API
Adds an API to query and visit supported devices. Follows the example
set by the required devices API.

Implements #37793.

Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
2021-08-25 19:43:54 -04:00
Daniel Leung
45f549dee9 kernel: allow linking a prebuilt library instead of compiling
This adds a very primitive logic to allow linking a prebuilt
static library of kernel code instead of building the kernel
from source. Note that the library is built with a specific
set of kconfigs, and they must match when building applications,
or else there would be mysterious crashes.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-08-25 18:01:14 -04:00
Naiyuan Tian
bc3fda491f kernel: userspace: fix typo in the comments
While reading the code, found some typos in the code comments,
line 226 and 668.
Fix comments to make it more solid.

Signed-off-by: Naiyuan Tian <naiyuan.tian@intel.com>
2021-08-24 07:31:49 -04:00
Maksim Masalski
8ed9cddbc5 lib: add explanation comment in "work->handler" case
Coding scanning tool raises a violation that happens dereferencing
of the "work" pointer in the expression "work->handler"

As were discussed before in PR #35664 it is not true.
Add explanation comment, because static code analysis tool
raised false-positive violation there.

Signed-off-by: Maksim Masalski <maksim.masalski@intel.com>
2021-08-20 18:48:16 -04:00
Fabio Baltieri
7f36f90167 kernel: drop unused priority related definitions
There are no reference for either K_NUM_PRIORITIES or
K_NUM_PRIO_BITMAPS, with the former being dropped in:

  1acd8c2996 kernel: Scheduler rewrite

Dropping both of these, and also the two comments about extra priorities
taking extra RAM space, as those do not seem to apply either.

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2021-08-17 17:52:17 -04:00
Fabio Baltieri
f88a420d69 toolchain: migrate iterable sections calls to the external API
This migrates all the current iterable section usages to the external
API, dropping the "Z_" prefix:

Z_ITERABLE_SECTION_ROM
Z_ITERABLE_SECTION_ROM_GC_ALLOWED
Z_ITERABLE_SECTION_RAM
Z_ITERABLE_SECTION_RAM_GC_ALLOWED
Z_STRUCT_SECTION_ITERABLE
Z_STRUCT_SECTION_ITERABLE_ALTERNATE
Z_STRUCT_SECTION_FOREACH

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2021-08-12 17:47:04 -04:00
Flavio Ceolin
8eceeee798 pm: device: Add wakeup source API
Introduce a new API to allow devices capable of wake up the system
register themselves was wake up sources. This permits applications to
select the most appropriate way to wake up the system when it is
suspended.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2021-08-11 19:39:13 -04:00
Evgeniy Paltsev
497cb2e587 CPP: fix static objects init for MWDT toolchain
The constructors of static objects are stored in ".ctors"
section. In case of MWDT toolchain we have incompatible
".ctors" section format with GNU toolchain. So let's use
initialization code provided by MWDT instead of Zephyr one
in case of MWDT toolchain usage.

As it is done for GNU toolchain We call constructors of
static objects but we don't call destructors for them.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
2021-08-09 22:47:22 -04:00
Gerard Marull-Paretas
c2cf1ad203 pm: device: remove usage of local states
The device PM subsystem already holds the device state, so there is no
need to keep duplicates inside the device. The pm_device_state_get has
been refactored to just return the device state. Note that this is still
not safe, but the same applied to the previous implementation. This
problem will be addressed later.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2021-08-04 08:23:01 -04:00
Flavio Ceolin
f2e323f06c futex: Avoid unnecessary lock
It is not necessary the spin lock to protect atomic operation.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2021-07-30 20:21:04 -04:00
Andrew Boie
f07df42d49 kernel: make k_current_get() work without syscall
We cache the current thread ID in a thread-local variable
at thread entry, and have k_current_get() return that,
eliminating system call overhead for this API.

DL: changed _current to use z_current_get() as it is
    being used during boot where TLS is not available.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-07-30 20:16:47 -04:00
Gerard Marull-Paretas
70322853a8 pm: device: move device busy APIs to pm subsystem
The following device busy APIs:

- device_busy_set()
- device_busy_clear()
- device_busy_check()
- device_any_busy_check()

were used for device PM, so they have been moved to the pm subsystem.
This means they are now prefixed with `pm_` and are defined in
`pm/device.h`.

If device PM is not enabled dummy functions are now provided that do
nothing or return `-ENOSYS`, meaning that the functionality is not
available.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2021-07-30 09:28:42 -04:00
Guennadi Liakhovetski
339a6bdafb kernel: fix several typos in a comment in timeout.c
Fix several simple typos.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
2021-07-23 16:06:54 -04:00
Guennadi Liakhovetski
45b70e1500 smp: limit the scope of some SMP-only functions
z_smp_init() is only available if CONFIG_SMP is defined,
smp_timer_init() also depends on two Kconfig parameters. Also make it
conditional in cavs_timer.c. Also clarify some SMP-related comments
there.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
2021-07-23 16:06:54 -04:00
Daniel Leung
dbc0be487f kernel: use proper macro to declare extern interrupt stacks
The z_interrupt_stacks was declared extern in the kernel internal
header file using the same macro which defines the same stack
array but with an added "extern" in front. This macro adds
alignment and section attribute which are actually not the same
as the actual stack array defined in kernel/init.c. The section
name used in the section attribute contains the file name where
the stack array is defined or extern declared. So the same
symbol, in this case z_interrupt_stacks, has different
attributes in two places, and GCC 11 starts to complain about
this. So use the newly introduced macro to extern declare
the stack array without adding/replacing any symbol attributes.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-07-22 07:24:11 -05:00
David Palchak
043507dbd6 kernel: native_posix: Don't run global C++ constructors
On the native_posix board global object constructors
are run by the underlying OS runtime init prior to
Zephyr kernel init. Thus Zephyr should not run global
object constructors a second time. Doing so breaks
application behavior that relies on global
constructors doing work that must be done only once.
See bug #36858 for more information.

Signed-off-by: David Palchak <palchak@google.com>
2021-07-12 19:51:16 -04:00
Chih Hung Yu
0ef77d4ea4 kernel: Fix negative mutex lock_count value
If you try to unlock an unlocked mutex, it will incorrectly
succeeds and decreases the lock count to -1.

Fixes #36572

Signed-off-by: Chih Hung Yu <chyu313@gmail.com>
2021-07-06 19:19:41 -04:00
Anas Nashif
8b3f36c656 kernel: move internal headers into include/kernel
Move 2 headers that are internal to the kernel into include/kernel.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-06-16 20:38:55 -04:00
Jennifer Williams
ae85da1c90 kernel: poll: fix coding guideline 15.7 missing comment
The final else {} in the if...else if is missing required
comment (non-empty, ';' is not sufficient). This adds a comment
to comply with CG 15.7.

Signed-off-by: Jennifer Williams <jennifer.m.williams@intel.com>
2021-06-04 16:22:50 -05:00
Maksim Masalski
e9600a75fc kernel: thread: add default case, remove unused break
According to the Zephyr Coding Guideline all switch statements
shall be well-formed. Add a default case with break and comment
to avoid static analysis tool to raise a violation that there is no
default case.
Also, I think, in all cases above no need to use "break",
because they already are using "return".

Found as a coding guideline violation (MISRA R16.1) by static
coding scanning tool.

Signed-off-by: Maksim Masalski <maksim.masalski@intel.com>
2021-06-04 16:21:01 -05:00
Maksim Masalski
78ba2ec830 coding guidelines: add to function prototypes form named parameters
Function types shall be in prototype form with named parameters

Found as a coding guideline violation (MISRA R8.2) by static
coding scanning tool.

Signed-off-by: Maksim Masalski <maksim.masalski@intel.com>
2021-06-04 16:20:06 -05:00
Torbjörn Leksell
70d721c1bb Tracing: Incorrect Unlock Mutex Trace Hook Fix
Changed location of the last k_mutex_unlock trace hook since it was
being called after k_sched_unlock, which could result in tracing
scenarios (other thread waiting for lock) where it appeared that a
mutex was being locked again before becoming unlocked.

Signed-off-by: Torbjörn Leksell <torbjorn.leksell@percepio.com>
2021-06-03 07:10:05 -05:00
Ioannis Glaropoulos
dd574f6ec7 kernel: stack_sentinel: disable in single-threaded builds
Add a dependency on MULTITHREADING for the
STACK_SENTINEL feature, so it may not get
enabled in single-thread Zephyr builds.

Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
2021-05-28 10:41:46 -05:00
Daniel Leung
dfa4b7e375 kernel: mmu: z_backing_store* to k_mem_paging_backing_store*
These functions are those that need be implemented by backing
store outside kernel. Promote them from z_* so these can be
included in documentation.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-05-28 11:33:22 -04:00
Daniel Leung
31c362d966 kernel: mmu: rename z_eviction* to k_mem_paging_eviction*
These functions and data structures are those that need
to be implemented by eviction algorithm and application
outside kernel. Promote them from z_* so these can be
included in documentation.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-05-28 11:33:22 -04:00
Lauren Murphy
4c85b4606b kernel: k_sleep: fix return value for absolute timeout
Fixes calculation of remaining ticks returned from z_tick_sleep
so that it takes absolute timeouts into account.

Fixes #32506

Signed-off-by: Lauren Murphy <lauren.murphy@intel.com>
2021-05-26 18:11:52 -05:00
Ioannis Glaropoulos
4084242a71 kernel: make MULTITHREADING promptless if single-thread not supported
If single thread builds are not supported by the
architecture, the MULTITHREADING option should be
prompt-less to block any modifications to it. We
also introduce an explicit ARCH-level Kconfig that
reflects whether the ARCH is capable of single-thread
Zephyr builds.

Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
2021-05-26 11:03:22 -05:00
Flavio Ceolin
97281b3862 pm: device_runtime: get rid of the spinlock
Protect critical sections using the mutex.
The mutex is required to use the conditional variable and since we
need to atomically check the pm state and the workqueue before wait
the condition, it is necessary to protect them using the same mutex.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2021-05-26 10:56:55 -04:00
Flavio Ceolin
378a19d2a8 pm: device_runtime: Add helper to wait on async ops
Add a function that properly uses a mutex to check a condition before
wait on the conditional variable.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2021-05-26 10:56:55 -04:00
Maksim Masalski
970820e92d sched: create unique function name
In file include/kernel/thread.h in "struct _thread_base" is a member
called "_wait_q_t *pended_on"
At the same time in file kernel/sched.c is function called
"static _wait_q_t *pended_on()"

Coding scanning tool assigns violation (MISRA R5.9) that static
object reused, because thread.h is included in struct.c file.

I think we can rename function to avoid misreading in the future.

Signed-off-by: Maksim Masalski <maksim.masalski@intel.com>
2021-05-25 19:06:21 -04:00
Andrzej Głąbek
59b21a29aa kernel: timeout: Fix adding of an absolute timeout
Correct the way the relative ticks value is calculated for an absolute
timeout. Previously, elapsed() was called twice and the returned value
was first subtracted from and then added to the ticks value. It could
happen that the HW counter value read by elapsed() changed between the
two calls to this function. This caused the test_timeout_abs test case
from the timer_api test suite to occasionally fail, e.g. on certain nRF
platforms.

Signed-off-by: Andrzej Głąbek <andrzej.glabek@nordicsemi.no>
2021-05-24 23:53:18 -04:00
Andy Ross
851d14afc8 kernel/sched: Remove "cooperative scheduling only" special cases
The scheduler has historically had an API where an application can
inform the kernel that it will never create a thread that can be
preempted, and the kernel and architecture layer would use that as an
optimization hint to eliminate some code paths.

Those optimizations have dwindled to almost nothing at this point, and
they're now objectively a smaller impact than the special casing that
was required to handle the idle thread (which, obviously, must always
be preemptible).

Fix this by eliminating the idea of "cooperative only" and ensuring
that there will always be at least one preemptible priority with value
>=0.  CONFIG_NUM_PREEMPT_PRIORITIES now specifies the number of
user-accessible priorities other than the idle thread.

The only remaining workaround is that some older architectures (and
also SPARC) use the CONFIG_PREEMPT_ENABLED=n state as a hint to skip
thread switching on interrupt exit.  So detect exactly those platforms
and implement a minimal workaround in the idle loop (basically "just
call swap()") instead, with a big explanation.

Note that this also fixes a bug in one of the philosophers samples,
where it would ask for 6 cooperative priorities but then use values -7
through -2.  It was assuming the kernel would magically create a
cooperative priority for its idle thread, which wasn't correct even
before.

Fixes #34584

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2021-05-24 23:38:16 -04:00
Maksim Masalski
d6c9d40ee0 userspace: remove dead code
File userspace.c contains dead code in function char *otype_to_str()
Remove "return NULL" and replace with "ret = NULL".

Found as a coding guideline violation (MISRA R2.1) by static
coding scanning tool.

Signed-off-by: Maksim Masalski <maksim.masalski@intel.com>
2021-05-24 22:35:03 -04:00
Armando Visconti
4b4068c948 kernel/device: add arg checking in z_device_ready()
If this call receives an invalid device pointer as argument it
assumes that the `device` is not ready for usage.

This routine is currently called by two device specific APIs:

    - device_usable_check(const struct device *dev)
    - device_is_ready(const struct device *dev)

The device-specific APIs documentation claims that these two
routines must be called with a device pointer captured from
DEVICE_DT_GET(). So passing NULL is a violation of the rule.

Nevertheless, is quite common in drivers to assign NULL to
a device pointer if the corresponding DT property has not been
found (e.g. a not used gpio interrupt declaration for a given
device instance) and seems legit to interpret this condition
same as the device is not ready for usage.

Signed-off-by: Armando Visconti <armando.visconti@st.com>
2021-05-18 11:29:46 -05:00
Peter Bigot
656c09589a kernel: work: fix race condition with cancel before work runs
The original state management solution involved separate locks for a
work queue and each work item.  To avoid inter-lock dependencies a
window was left between the point where the work item was removed from
the queue (protected by queue lock) and the point where the work item
state was updated to mark the work item running.

This introduced a bug: If a cancellation was issued during this window
it would succeed, and the work item would appear to be idle even
though in fact the work queue thread was about to run it.

Since there is now only one lock, move the work item state updates
into the mutex regions associated with dequeuing the work item and
clearing the work queue busy flag.

Note that removing the window between queue and work mutex regions
eliminates the potential of having a dequeued work item be cancelled
before its QUEUED flag is cleared, simplifying the work item state
update.

Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
2021-05-18 15:02:08 +02:00
Maksim Masalski
929956df70 coding guidelines rule 14_3_j: add explicit case check
Violation of the [MISRAC2012-RULE_14_3-j]:
Boolean operations whose results are invariant
shall not be permitted

Probably in that part of code is a misprint.
Added to check _OBJ_INIT_FALSE case explicitly

Signed-off-by: Maksim Masalski <maksim.masalski@intel.com>
2021-05-18 08:36:57 -04:00
Anas Nashif
7b9084cb4f ztest: set thread name to test name
Use the actual test name and not a hardcoded thread name. This is useful
for tracing test cases.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-05-17 18:45:57 -04:00
Andy Ross
bd077561d0 kernel/swap: Add assertion to catch lock-breaking context switches
Our z_swap() API takes a key returned from arch_irq_lock() and
releases it atomically with the context switch.  Make sure that the
action of the unlocking is to unmask interrupts globally.  If
interrupts would still be masked then that means there is an OUTER
interrupt lock still held, and the code that locked it surely doesn't
expect the thread to be suspended and interrupts unmasked while it's
held!

Unfortunately, this kind of mistake is very easy to make.  We should
catch that with a simple assertion.  This is essentially a crude
Zephyr equivalent of the extremely common "BUG: scheduling while
atomic" error in Linux drivers (just google it).

The one exception made is the circumstance where a thread has already
aborted itself.  At that stage, whatever upthread lock state might
have existed will have already been messed up, so there's no value in
our asserting here.  We can't catch all bugs, and this can actually
happen in error handling and/or test frameworks.

Fixes #33319

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2021-05-17 15:27:37 -04:00
Daniel Leung
216dc5ddfe kernel: mmu: remove un-needed call to virt_to_bitmap_offset
When marking the reserved region at the end of virtual address
space, call virt_to_bitmap_offset() is not needed as we already
know the offset. So remove it.

Coverity-CID: 235930
Fixes #35160

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-05-13 09:00:54 -05:00
Daniel Leung
660d1478c6 kernel: init.c: tag source for boot/pinned sections
This adds the tags for functions and variables so they
can be put into boot/pinned sections.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-05-10 16:00:43 -05:00
Daniel Leung
1310ad6b0e linker: add bits for pinned regions
This adds the necessary bits for linker scripts and source code
to specify which symbols need to be pinned in memory. This is
needed for demand paging as some functions and data must reside
in memory all the time and cannot be paged out (e.g. paging,
scheduler, and interrupt routines for functionality).

This is up to the arch/SoC/board to define the sections in
their linker scripts as the pinned section may need special
alignment which cannot be done in common script snippets.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-05-10 16:00:43 -05:00
Daniel Leung
d812728ec4 linker: add bits for boot regions
This adds the necessary bits for linker scripts and source code
to specify which symbols are needed for boot process so they
can be grouped together.

One use of this is to group boot related code and data so these
won't interval with other kernel and application for better
caching.

This is a must for demand paging as some functions and data
must be available during the boot process and before the memory
manager is initialized. During this time, paging cannot be used
so symbols linked in virtual memory space are unavailable.

This is up to the arch/SoC/board to define the sections in
their linker scripts as section may need special alignment
which cannot be done in common script snippets.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-05-10 16:00:43 -05:00