Commit graph

522 commits

Author SHA1 Message Date
Johan Hedberg
3fbf12487c kernel: Introduce a way to specify minimum system heap size
There are several subsystems and boards which require a relatively large
system heap (used by k_malloc()) to function properly. This became even
more notable with the recent introduction of the ACPICA library, which
causes ACPI-using boards to require a system heap of up to several
megabytes in size.

Until now, subsystems and boards have tried to solve this by having
Kconfig overlays which modify the default value of HEAP_MEM_POOL_SIZE.
This works ok, except when applications start explicitly setting values
in their prj.conf files:

$ git grep CONFIG_HEAP_MEM_POOL_SIZE= tests samples|wc -l
     157

The vast majority of values set by current sample or test applications
is much too small for subsystems like ACPI, which results in the
application not being able to run on such boards.

To solve this situation, we introduce support for subsystems to specify
their own custom system heap size requirement. Subsystems do
this by defining Kconfig options with the prefix HEAP_MEM_POOL_ADD_SIZE_.
The final value of the system heap is the sum of the custom
minimum requirements, or the value existing HEAP_MEM_POOL_SIZE option,
whichever is greater.

We also introduce a new HEAP_MEM_POOL_IGNORE_MIN Kconfig option which
applications can use to force a lower value than what subsystems have
specficied, however this behavior is disabled by default.

Whenever the minimum is greater than the requested value a CMake warning
will be issued in the build output.

This patch ends up modifying several places outside of kernel code,
since the presence of the system heap is no longer detected using a
non-zero CONFIG_HEAP_MEM_POOL_SIZE value, rather it's now detected using
a new K_HEAP_MEM_POOL_SIZE value that's evaluated at build.

Signed-off-by: Johan Hedberg <johan.hedberg@intel.com>
2023-12-20 11:01:42 +01:00
Daniel Leung
fa561ccd59 kernel: mmu: no need to expose z_free_page_count
z_free_page_count is only used in one file, so there is
no need to expose it, even to other part of kernel.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-12-12 18:46:21 +00:00
Daniel Leung
34c6b17680 doc: fixed already renamed _Cstart to z_cstart
The function _Cstart has already been renamed to z_cstart,
so change the remaining references of it in various docs.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-11-30 21:01:47 -05:00
Daniel Leung
c972ef1a0f kernel: mm: move kernel mm functions under kernel includes
This moves the k_* memory management functions from sys/ into
kernel/ includes, as there are kernel public APIs. The z_*
functions are further separated into the kernel internal
header directory.

Also made a quick change to doxygen to group sys_mem_* into
the OS Memory Management group so they will appear in doc.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-11-20 09:19:14 +01:00
Flavio Ceolin
f9c7a5e6fb kernel: random: Rename early random get function
Rename z_early_boot_rand_get with z_early_rand_get to get consistent
with other early functions.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2023-10-13 10:03:53 +03:00
Daniel Leung
2c2d53c7e5 kernel: remove deprecate wording on arch_kernel_init()
The wording on deprecating arch_kernel_init() in favor of prep_c()
has never been materialized. Various architectures are using it to
perform initialization. So remove the wording.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-10-09 10:15:49 +02:00
Daniel Leung
dd6a7eb77d kernel: demand_paging: add doc to enum arch_page_location
This adds doc to enum arch_page_location.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-10-09 10:15:49 +02:00
Daniel Leung
77dc74136c kernel: tls: fix doc on arch_tls_stack_setup()
There was a typo there so fix it.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-10-09 10:15:49 +02:00
Anas Nashif
a1c7bfbc63 kernel: remove unused z_init_thread_base from kernel.h
This API is internal and not used in any way in kernel.h, so move it
back to where it is needed.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-09-30 18:43:28 +02:00
Anas Nashif
e19f21cb27 kernel: move z_is_thread_essential out of public kernel header
This is a private API to the kernel, so move out of kernel.h

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-09-30 18:43:28 +02:00
Anas Nashif
f0c7fbf0f1 kernel: move sched_priq.h to internal/ folder
This header is internal to the kernel and shall not be included directly.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-09-30 18:43:28 +02:00
Peter Mitsis
e6f1090553 kernel: Integrate object core statistics
Integrates object core statistics framework into the following
kernel objects:
  sys_mem_blocks, k_mem_slab
  threads, _cpu, z_kernel

Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
2023-09-30 08:04:14 +03:00
Anas Nashif
8dc2746c0e kernel: z_handle_obj_poll_events is internal not kernel.h material
This internal kernel API is misplaced in a public kernel header. Just
make it available to the code using it in the kernel.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-09-17 07:59:30 -04:00
Peter Mitsis
c7255cf374 kernel: Remove references to _EXPIRED
The _EXPIRED macro is no longer necessary. It is a relic of an older
timeout processing algorithm from several years ago.

Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
2023-09-15 19:16:20 -04:00
Anas Nashif
fcf50ed6e7 kernel: move timeout_q.h to kernel/include
This is a private kernel header with private kernel APIs, it should not
be exposed in the public zephyr include directory.

Once sample remains to be fixed (metairq_dispatch), which currently uses
private APIs from that header, it should not be the case.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-09-12 12:55:36 -04:00
Anas Nashif
8634c3b444 kernel: move wait_q.h header to be internal
This header does not expose any public APIs, so move it under
kernel/include and change files including it.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-09-12 12:55:36 -04:00
Peter Mitsis
bd5839ec9e kernel: Fix wrap-around check in kernel/mmu.h
Fixes the buffer wrap-around check so that it will not be ignored
by the GNU C compiler.

Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
2023-08-01 09:51:33 +02:00
Florian Grandel
e256b7d244 kernel: spinlock: LOCKED -> K_SPINLOCK
Let the kernel use the new K_SPINLOCK macro and remove the alias.

Signed-off-by: Florian Grandel <fgrandel@code-for-humans.de>
2023-07-10 09:27:21 +02:00
Florian Grandel
816d09c453 kernel: spinlock: expose LOCKED macro as public API
While the LOCKED pattern is universally useful it can be misused. This
change therefore exposes the LOCKED pattern with extensive usage
documentation to reduce the risk of abuse or unintended deadlock.

Signed-off-by: Florian Grandel <fgrandel@code-for-humans.de>
2023-07-10 09:27:21 +02:00
Evgeniy Paltsev
16b8191be0 SMP: fix build failure if SMP=y and SYS_CLOCK_EXISTS=n
Fix build failure for the SMP configurations without sysclock.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
2023-06-22 06:17:27 -04:00
Gerard Marull-Paretas
48b201cc53 device: make device dependencies optional
Device dependencies are not always required, so make them optional via
CONFIG_DEVICE_DEPS. When enabled, the gen_device_deps script will run so
that dependencies are collected and part of the final image. Related
APIs will be also made available. Since device dependencies are used in
just a few places (power domains), disable the feature by default. When
not enabled, a second linking pass will not be required.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2023-06-21 09:32:05 +02:00
Gerard Marull-Paretas
5982d83e2a device: s/struct device.handles/struct device.deps
Rename struct device `handles` member to `deps`, in line with previous
renamings in the device API.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2023-06-21 09:32:05 +02:00
Andy Ross
c3046f417a kernel/sched: Use new barrier and spin APIs
The switch_handle field in the thread struct is used as an atomic flag
between CPUs in SMP, and has been known for a long time to technically
require memory barriers for correct operation.  We have an API for
that now, so put them in:

* The code immediately before arch_switch() needs a write barrier to
  ensure that thread state written by the scheduler is seen to happen
  before the outgoing thread is flagged with a valid switch handle.

* The loop in z_sched_switch_spin() needs a read barrier at the end,
  to make sure the calling context doesn't load state from before the
  other CPU stored the switch handle.

Also, that same spot in switch_spin was spinning with interrupts held,
which means it needs a call to arch_spin_relax() to avoid a FPU state
deadlock on some architectures.

Signed-off-by: Andy Ross <andyross@google.com>
2023-05-26 17:09:35 -04:00
Andy Ross
b89e427bd6 kernel/sched: Rename/redocument wait_for_switch() -> z_sched_switch_spin()
This trick turns out also to be needed by the abort/join code.
Promote it to a more formal-looking internal API and clean up the
documentation to (hopefully) clarify the exact behavior and better
explain the need.

This is one of the more... enchanted bits of the scheduler, and while
the trick is IMHO pretty clean, it remains a big SMP footgun.

Signed-off-by: Andy Ross <andyross@google.com>
2023-05-26 17:09:35 -04:00
Flavio Ceolin
df4be07b26 kernel: mmu: Fix Xtensa memory alignment issue
z_page_frame can't be packed on Xtensa due memory alignment
constraints. When this is struct is packed it is 5 bytes long it will
cause an memory alignment problem on Xtensa.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2023-05-23 08:54:29 +02:00
Peter Mitsis
e1aa1396d8 kernel: Remove unused absolute symbols
Removes unused absolute symbols that are defined via the
GEN_ABSOLUTE_SYM() macro in the kernel directory.

Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
2023-04-18 10:51:28 -04:00
Aastha Grover
5537776898 kernel: Add z_sched_wake_thread API
This API wakes up a given thread and is also called from
z_thread_timeout()

Signed-off-by: Aastha Grover <aastha.grover@intel.com>
2023-03-09 09:22:21 +01:00
Andy Ross
f3afd5a4c9 kernel/sched: Use kernel timeouts for timeslice expirations
Rework the fragile and ad-hoc computation of timeslice expirations
into per-CPU struct _timeout objects with regular callbacks.  The
expiration callbacks themselves simply set a per-cpu flag (they might
run on any CPU), which gets checked at the end of the timer ISR on
every CPU.

This simplifies logic and removes a bunch of code.  It also fixes at
least three bugs:

1. As @npitre discovered: On SMP, the number of ticks announced on any
given CPU is going to be a subset of all expired ticks.  This broke
the accounting of timeslice ticks, and effectively meant that
timeslicing only worked on SMP on systems where one CPU could hog all
the announcements, and only on that CPU.

2. The bootstrap path to arm the timer driver after setting the first
timeout in an empty list couldn't take into account
sys_clock_elapsed() ticks, as it didn't know whether it was being
called underneath an existing announce loop.  Now this code is no
longer responsible for knowing anything about time slicing at all.

3. Also on SMP, there was a case where two CPUs timeslicing
simultaneously could stomp on each others' timeouts in
z_set_timeout_expiry(), as neither had a way of knowing what the
other's state was.  CPUs could miss their own expiration and have to
wait for the slice expiration on the other CPU.  Now, timeouts are
global objects with simple expiration times, and there's no need for
that function at all.

Signed-off-by: Andy Ross <andyross@google.com>
2023-03-09 09:21:12 +01:00
Peter Mitsis
a06f9ac418 kernel: Remove unused derived offset symbols
Some of the offset symbols that are derived from the macro
GEN_OFFSET_SYM() are not used anywhere in the Zephyr codebase.
Remove them as part of a cleanup effort.

Instances of an associated GEN_OFFSET_SYM() have also been
removed when the resulting macro is no longer referenced.

Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
2023-02-22 16:01:32 +01:00
Peter Mitsis
fa23eb49b3 kernel: Remove unused generated offset symbols
Some of the offset symbols generated via the macro GEN_OFFSET_SYM()
are not used anywhere in the Zephyr codebase. Remove them as part of
a cleanup effort.

Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
2023-02-22 16:01:32 +01:00
Peter Mitsis
ca58339e16 kernel: Add routine to walk a wait queue
Adds a routine to safely walk a specified wait queue and invoke a
custom callback function on each waiting thread.

Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
2023-02-11 06:45:13 +09:00
Kumar Gala
c778eb2a56 smp: Move arrays to use CONFIG_MP_MAX_NUM_CPUS
Move to use CONFIG_MP_MAX_NUM_CPUS for array size declarations instead
of CONFIG_MP_NUM_CPUS.

Signed-off-by: Kumar Gala <kumar.gala@intel.com>
2022-10-17 14:40:12 +09:00
Nicolas Pitre
1f362a81f1 riscv: fix crash resulting from touching the initial stack's guard area
The interrupt stack is used as the system stack during kernel
initialization while IRQs are not yet enabled. The sp register is
set to z_interrupt_stacks + CONFIG_ISR_STACK_SIZE.

CONFIG_ISR_STACK_SIZE only represents the desired usable stack size.
This does not take into account the added guard area. Result is a stack
whose pointer is much closer to the trigger zone than expected when
CONFIG_PMP_STACK_GUARD=y, and the SMP configuration in particular pushes
it over the edge during many CI test cases.

Worse: during early init we're not quite ready to handle exceptions
yet and complete havoc ensues with no meaningful debugging output.

Make sure the early assembly code locates the actual top of the stack
by generating a constant with its true size.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-09-28 07:53:56 +00:00
Hu Zhenyu
57487622f5 kernel: Init the base.slice_ticks for dummy_thread
Fixes #46324
Set dummy_thread->base.slice_ticks to 0 when
CONFIG_TIMESLICE_PER_THREAD is set. To avoid
_current_cpu->slice_ticks be a big number.

Signed-off-by: Hu Zhenyu <zhenyu.hu@intel.com>
2022-08-04 19:44:24 -04:00
Simon Hein
02cfbfea51 kernel: comply to coding guidelines MISRA C:2012 Rule 14.4
MISRA C:2012 Rule 14.4 (The controlling expression of an if statement
and the controlling expression of an iteration-statement shall have
essentially Boolean type.)

Use `bool' instead of `int' to represent Boolean values.
Use `do { ... } while (false)' instead of `do { ... } while (0)'.
Use comparisons with zero instead of implicitly testing integers.

This commit is a subset of the original commit:
5d02614e34a86b549c7707d3d9f0984bc3a5f22a

Signed-off-by: Simon Hein <SHein@baumer.com>
2022-07-21 06:16:16 -04:00
Stephanos Ioannidis
360f810704 kernel: Migrate to K_KERNEL_PINNED_STACK_ARRAY_DECLARE
This commit updates all deprecated `K_KERNEL_PINNED_STACK_ARRAY_EXTERN`
macro usages to use the `K_KERNEL_PINNED_STACK_ARRAY_DECLARE` macro
instead.

Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
2022-06-20 10:25:52 +02:00
Gerard Marull-Paretas
cffefc818d kernel: migrate includes to <zephyr/...>
In order to bring consistency in-tree, migrate all kernel code to the
new prefix <zephyr/...>. Note that the conversion has been scripted,
refer to zephyrproject-rtos#45388 for more details.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2022-05-09 09:26:20 +02:00
Bradley Bolen
88ba97fea4 arch: arm: aarch32: cortex_a_r: Add shared FPU support
This adds lazy floating point context switching.  On svc/irq entrance,
the VFP is disabled and a pointer to the exception stack frame is saved
away.  If the esf pointer is still valid on exception exit, then no
other context used the VFP so the context is still valid and nothing
needs to be restored.  If the esf pointer is NULL on exception exit,
then some other context used the VFP and the floating point context is
restored from the esf.

The undefined instruction handler is responsible for saving away the
floating point context if needed.  If the handler is in the first
irq/svc context and the current thread uses the VFP, then the float
context needs to be saved.  Also, if the handler is in a nested context
and the previous context was using the FVP, save the float context.

Signed-off-by: Bradley Bolen <bbolen@lexmark.com>
2022-05-05 12:03:27 +09:00
Flavio Ceolin
d02a1e9879 pm: Only resize power domains
Instead of resizing all devices handles, we just resize devices that are
power domains. This means that a power domain has to be declared as
compatbile with "power-domain" in device tree node.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2022-04-18 17:25:01 -07:00
Nazar Kazakov
f483b1bc4c everywhere: fix typos
Fix a lot of typos

Signed-off-by: Nazar Kazakov <nazar.kazakov.work@gmail.com>
2022-03-18 13:24:08 -04:00
Andy Ross
3e696896bf kernel: Add "per thread" timeslice mechanism
Zephyr's timeslice implementation has always been somewhat primitive.
You get a global timeslice that applies broadly to the whole bottom of
the priority space, with no ability (beyond that one priority
threshold) to tune it to work on certain threads, etc...

This adds an (optionally configurable) API that allows timeslicing to
be controlled on a per-thread basis: any thread at any priority can be
set to timeslice, for a configurable per-thread slice time, and at the
end of its slice a callback can be provided that can take action.
This allows the application to implement things like responsiveness
heuristics, "fair" scheduling algorithms, etc... without requiring
that facility in the core kernel.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2022-03-09 13:49:44 -05:00
Nicolas Pitre
678b76e4b0 kernel/init.c: allow for memset/memcpy alternatives during early boot
Zeroing the BSS and copying data to RAM with regular memset/memcpy may
cause problems when those functions are assuming a fully initialized
system for their optimizations to work e.g. some instructions require
an active MMU, but turning the MMU on needs the .bss section to be
cleared first, etc.

Commit c5b898743a ("aarch64: Fix alignment fault on z_bss_zero()")
provides a detailed explanation of such a case.

Replacing z_bss_zero() with an architecture specific one is problematic
as the former may see new sections added to it that would be missed by
the later. The same reasoning goes for z_data_copy().

Let's make maintenance much easier by providing weak versions of
memset/memcpy that can be overridden by architecture-specific safe
versions when needed.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2022-02-21 21:00:12 -05:00
Krzysztof Chruscinski
1da97e1374 kernel: Add function for calculating stack usage
Extracting stack usage calculation from k_thread_stack_space_get to
z_stack_space_get so it can be used also for interrupt stack.

Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
2022-02-21 20:57:17 -05:00
Daniel Leung
88cfd3343d kernel: arch: no need for #ifdef MMU in header
There is no need to use conditional compilation for the function
prototypes in the kernel architecture header file. So remove it.
Added bouns is that these functions can appear in documentation
without explicitly enabled in pre-defines during doc build.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2022-01-18 19:18:30 -05:00
Peter Mitsis
4eb1dd02cc kernel: extend CPU runtime stats
Extends the CPU usage runtime stats to track current, total, peak
and average usage (as bounded by the scheduling of the idle thread).
This permits a developer to obtain more system information if desired
to tune the system.

Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
2022-01-10 10:38:06 -05:00
Peter Mitsis
5deaffb2ee kernel: update z_sched_thread_usage()
This commit does two things to the z_sched_thread_usage(). First,
it updates the API so that it accepts a pointer to the runtime
stats instead of simply returning the usage cycles. This gives it
the flexibility to retrieve additional statistics in the future.

Second, the runtime stats are only updated if the specified thread
is the current thread running on the current core.

Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
2022-01-10 10:38:06 -05:00
Berend Ozceri
b208e5811e kernel/swap: Initialize dummy thread's resource pool
The resource pool of the short-lived dummy thread "stub" may be
inherited by other threads created during system initialization. This
commit initializes this resource pool to NULL or the system pool to
ensure that a well-defined resource pool propagates to other threads
that inherit it from the dummy thread.

Fixes #41482.

Signed-off-by: Berend Ozceri <berend@recogni.com>
2022-01-06 11:57:18 -05:00
Daniel Leung
650a629b08 debug: gdbstub: remove start argument from z_gdb_main_loop()
Storing the state where this is the first GDB break can be done
in the main GDB stub code. There is no need to store the state
in architecture layer.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-11-30 15:24:00 -05:00
Flavio Ceolin
623ed5ae29 pm: Remove invalid comments
Remove comments referencing an old function / behavior.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2021-11-29 19:37:55 -05:00
Krzysztof Chruscinski
c0808e3f59 logging: Minimal mode configuration cleanup
Remove LOG_MINIMAL kconfig option which was confusing
since LOG_MODE_MINIMAL existed. LOG_MINIMAL was used to
force minimal mode but because of invalid dependencies
it was leading to issues.

Refactored code to use LOG_MODE_MINIMAL everywhere and
renamed LOG_MINIMAL to LOG_DEFAULT_MINIMAL which has impact
on defualt logging mode (which still can be later changed
in conf file or in menuconfig).

Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
2021-11-20 11:58:40 -05:00