Each kernel object type that participates in the object core framework
previously supplied its own SYS_INIT routine to initialize its
k_obj_type and to walk its static-object linker section, linking each
object core. These routines were near-identical across 11 object types
and differed only in the type id, the object struct and the obj_core
offset.
Replace that per-type boilerplate with a declarative K_OBJ_TYPE_DEFINE()
macro that emits a const descriptor into a new iterable ROM section, and
walk those descriptors once from a single SYS_INIT in obj_core.c. The
descriptor captures the type storage, type id, obj_core offset, the
static object section bounds and the object stride, which is all the
shared loop needs to initialize and link every statically defined
object.
Converted: condvar, event, fifo, lifo, mailbox, msgq, mutex, pipe, sem,
stack and timer. The non-uniform initializers (thread, mem_slab and the
internal cpu/kernel objects) are left unchanged and will be dealt with
in followup commits.
Footprint on qemu_cortex_m3 (tests/kernel/obj_core/obj_core, all types
enabled): flash 32012 -> 30880 (-1132 B), RAM unchanged. With
CONFIG_OBJ_CORE disabled the image is byte-for-byte identical.
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Bringing the kernel/ sources into the Doxygen requirements-traceability
INPUT (for @satisfies links) surfaces a handful of latent documentation
warnings that would break the -W docs build. Fix them.
With these, the kernel/ tree is clean under doxygen 1.17.
Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
The comment made it seem like we do two passes on the "threads that need
wakeup" list: a first one to unpend and another to ready. In reality,
we do only one path and perform both operations at one using
`z_sched_wake_thread_locked()`.
Update the comment explaining the algorithm so it reflects more
accurately what the code actually does.
Signed-off-by: Mathieu Choplain <mathieu.choplain-ext@st.com>
Migrate scheduler-internal callers of z_abort_thread_timeout() to the
new z_try_abort_thread_timeout(). This covers the abort sites in
sched.c, thread.c, scheduler.c (z_sched_wake), events.c, and pipe.c.
The patterns used:
z_unpend_thread (sched.c) retries on -EAGAIN: if the timeout
handler is in flight on another CPU, drop _sched_spinlock so the
handler can run to completion and retry. This preserves 1b8c7a3's
unpend+abort atomicity from the caller's perspective.
halt_thread (sched.c) takes the caller's sched-lock key as a pointer
so its direct abort on the dying thread can retry on -EAGAIN.
Waiting for the handler is mandatory: a caller may free the thread's
storage as soon as halt_thread() returns, and without waiting, the
still-in-flight handler would later dereference freed memory.
_THREAD_DEAD is set before the abort, so the handler bails via the
killed check in z_sched_wake_thread_locked().
For next_up() (the scheduler-context caller), the key is not cleanly
available: K_SPINLOCK in z_get_next_switch_handle and do_swap's
(void)k_spin_lock both discard it. halt_thread is invoked on
_current with NULL key and no abort is performed -- _current is
running, so its base.timeout cannot be linked. The gap is closed at
the other end: z_thread_halt() spins on z_try_abort_thread_timeout()
outside any lock after the halt-queue wait completes, before
returning to the caller of k_thread_abort().
z_unpend_all_locked / unpend_all (sched.c) skip the local
ready_thread() on -EAGAIN and let the still-blocked handler ready
the thread when _sched_spinlock drops. The threads being woken are
not freed, so no UAF risk; end state is identical.
z_impl_k_wakeup (thread.c), z_sched_wake (scheduler.c) and
event_walk_op (events.c) perform the wake entirely under
_sched_spinlock, so a (void) abort is race-free -- a racing in-flight
handler is blocked on the same lock during the wake.
copy_to_pending_readers (pipe.c) is restructured to also wake the
reader under the scheduler lock instead of after it, so the
return-value set, unpend, abort, and ready all happen atomically.
The dticks-cancel check in z_thread_timeout() is preserved for now
because other callers (sem.c, mutex.c, ... via z_unpend_first_thread())
still use z_abort_thread_timeout() and rely on it for race protection.
A follow-up commit migrates those, and a final commit drops the
cancel check and removes z_abort_thread_timeout() itself.
Add z_try_abort_thread_timeout() as an inline wrapper.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
kernel.h implies kernel_structs.h via kernel_includes.h, making
explicit inclusion of kernel_structs.h unnecessary whenever kernel.h
is already included in the same translation unit.
Remove the redundant includes across arch, boards, drivers, kernel,
lib, samples, subsys, and tests trees.
in include/zephyr/kernel_structs.h:
* 2. kernel.h shall imply kernel_structs.h, such that it shall not be
* necessary to include kernel_structs.h explicitly when kernel.h is
* included.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Migrate scheduler API implementations (k_sched_lock/unlock,
z_reschedule, z_yield_current, etc.) and their private declarations
from ksched.h/sched.c into scheduler.c and scheduler.h.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Use z_abort_thread_timeout() instead of the lower-level z_abort_timeout().
The thread-flavoured version also has a stub fallback when
CONFIG_SYS_CLOCK_EXISTS=n removing the need for preprocessor checks.
Signed-off-by: Mathieu Choplain <mathieu.choplain-ext@st.com>
When CONFIG_WAITQ_SCALABLE=y, wake up all threads from a post-waitq-walk
callback which is invoked while the scheduler spinlock is still held. This
solves the race condition that was worked around via `no_wake_in_timeout`
flag in k_thread and `is_timeout` parameter of z_sched_wake_thread_locked()
which can now both be dropped.
Signed-off-by: Mathieu Choplain <mathieu.choplain-ext@st.com>
Modify z_sched_waitq_walk() to accept an optional callback invoked after
the walk while still holding the scheduler spinlock. This can be used to
perform post-walk operations "atomically". Update all callers to work with
this new function signature.
While at it, create dedicated (private) typedefs for the callbacks and
clean up/improve the routine and callbacks' documentation.
Signed-off-by: Mathieu Choplain <mathieu.choplain-ext@st.com>
When CONFIG_WAITQ_SCALABLE=n, the callback invoked by z_sched_waitq_walk()
is allowed to remove the thread provided as argument from the wait queue
(an operation implicitly performed when waking up a thread).
Use this to our advantage when waking threads pending on a k_event by
waking threads as part of the waitq walk callback instead of building
a list of threads to wake and performing the wake outside the callback.
When CONFIG_WAITQ_SCALABLE=n, this allows removing a pointer-sized field
from the thread structure which reduces the overhead of CONFIG_EVENTS=y.
The old implementation (build list in callback and wake outside callback)
is retained and used when CONFIG_WAITQ_SCALABLE=y since we can't modify
the wait queue as part of the walk callback in this situation. This is now
documented above the corresponding field in k_thread structure.
Signed-off-by: Mathieu Choplain <mathieu.choplain-ext@st.com>
Don't acquire the _sched_spinlock in z_sched_wake_thread(). This allows
calling the function from callbacks which already own the spinlock. The
function is renamed to z_sched_wake_thread_locked() to reflect this new
behavior, and all existing callers are updated to ensure they hold the
_sched_spinlock as is now required.
Signed-off-by: Mathieu Choplain <mathieu.choplain-ext@st.com>
Add conditional compilation guards around timeout operations in
kernel/events.c to ensure compatibility with timer-less configurations.
Signed-off-by: Mohamed Moawad <moawad@synopsys.com>
This is variation of the PR to handle phantom events and hopefully
this get merged into the PR to land.
See-also: https://github.com/zephyrproject-rtos/zephyr/pull/89624
Signed-off-by: Charles Hardin <ckhardin@gmail.com>
Most kernel objects should be initialized well before being
manipulated in ISR context.
Event objects are no exception. Initializing a k_event object in
ISR context would implicitly be racey and introduce an element of
non-determinism.
Assert that k_event_init() is not called from ISR context.
Signed-off-by: Chris Friedt <cfriedt@tenstorrent.com>
Namespaced the generated headers with `zephyr` to prevent
potential conflict with other headers.
Introduce a temporary Kconfig `LEGACY_GENERATED_INCLUDE_PATH`
that is enabled by default. This allows the developers to
continue the use of the old include paths for the time being
until it is deprecated and eventually removed. The Kconfig will
generate a build-time warning message, similar to the
`CONFIG_TIMER_RANDOM_GENERATOR`.
Updated the includes path of in-tree sources accordingly.
Most of the changes here are scripted, check the PR for more
info.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Add a closing comment to the endif with the configuration
information to which the endif belongs too.
To make the code more clearer if the configs need adaptions.
Signed-off-by: Simon Hein <Shein@baumer.com>
Move the syscall_handler.h header, used internally only to a dedicated
internal folder that should not be used outside of Zephyr.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
The original idea of z_current_get() was to be the counterpart
of k_current_get() when thread local variable for current has
not been initialized if TLS is enabled, otherwise they are
the same function. Now since z_current_get() is being used
outside of core kernel, rename it under kernel namespace so
other subsystem can conceptually use them too.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This header does not expose any public APIs, so move it under
kernel/include and change files including it.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Update the return value of functions that modify the internal event
state from `void` to `uint32_t`, so that calling code can determine
whether the event was already in a given state, or if the call modified
it.
This simplifies the usage of `struct k_event` as an alternative to
`atomic_t` that users can block on.
Implements #57216
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
Updates events to prevent a timeout from corrupting the list of
threads that needs to be waken up.
Signed-off-by: Aastha Grover <aastha.grover@intel.com>
Fixes race condition for k_event_post_internal() in an
SMP environment while walking the waitq. Uses z_sched_waitq_walk()
to safely walk the waitq by using a sched_spinlock.
It should be noted that since walking the wait queue is an
operation of indeterminant length, there exists the possibility
that the sched_spinlock (which is a highly used and contended-for
lock) may be locked for an indeterminant amount of time. However,
it is expected that few threads will be waiting on any given kernel
event object, which should ameliorate this risk.
Fixes#54317
Signed-off-by: Aastha Grover <aastha.grover@intel.com>
There is no easy way to clear event bits without
the potential for a race to exist between producer(s)
and consumer(s). The result of this race is that events
can be lost through the various resetting mechanisms
available (flag to k_event_wait(), or k_event_set()).
Add k_event_set_masked() which permits bits to be set or cleared.
This allows consumers to clear just the bits that they have read
without (accidentally) discarding any new bits.
Update unit tests to verify the functionality.
Partly Fixes#46117.
Signed-off-by: Andrew Jackson <andrew.jackson@amd.com>
Although there is nothing wrong with the existing code,
it doesn't permit individual bits to be set (or cleared).
This makes further changes slightly awkward.
Use a mask to restrict the bits set in an event.
Signed-off-by: Andrew Jackson <andrew.jackson@amd.com>
In order to bring consistency in-tree, migrate all kernel code to the
new prefix <zephyr/...>. Note that the conversion has been scripted,
refer to zephyrproject-rtos#45388 for more details.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
Threads may wait on an event object such that any events posted to
that event object may wake a waiting thread if the posting satisfies
the waiting threads' event conditions.
The configuration option CONFIG_EVENTS is used to control the inclusion
of events in a system as their use increases the size of
'struct k_thread'.
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
