All we really want here is to set default parameters. However
k_sched_time_slice_set() also calls z_reset_time_slice(_current)
which expects `_current` to be fully initialized.
Simply initialize `slice_ticks` and `slice_max_prio` with default values
directly. Unfortunately the compiler isn't smart enough to expand
k_ms_to_ticks_ceil32(CONFIG_TIMESLICE_SIZE) to a constant expression
at build time so we must do the conversion by hand (and it shouldn't
overflow due to the nature of the value).
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Slice expirations are now based on the same timeout mechanism as
regular timers which have been recently fixed and proven to work with
single-tick periods.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The reason for arch_num_cpus() is to be able to dynamically adapt to
the actual number of available CPUs at run time.
In the z_sched_init() case, it is not the number of active CPUs that
we need but rather the total number of potential CPUs, and that is
represented by CONFIG_MP_MAX_NUM_CPUS not arch_num_cpus().
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Add the `zephyr,pm-device-runtime-auto` flag to `pm.yaml` and
`struct pm_device`.
This flag is intended to signify to the boot system that device runtime
PM should be automatically enabled on the device after the init function
has run.
Only run `pm_device_runtime_auto_enable` function on a device if
initialisation succeeded. This prevents actions being run on devices
that are not ready.
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
Make sliceable() the actual condition for a sliceable thread. Avoid
creating a slice timeout for non sliceable threads. Always reset
slice_expired even if the next thread is not sliceable. Fold
slice_expired_locked() into z_time_slice() to avoid the hidden
unlock/lock. Change `curr` to `thread` as this is not necessarily
the current thread (yet) being set. Make variables static.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Updates events to prevent a timeout from corrupting the list of
threads that needs to be waken up.
Signed-off-by: Aastha Grover <aastha.grover@intel.com>
Fixes race condition for k_event_post_internal() in an
SMP environment while walking the waitq. Uses z_sched_waitq_walk()
to safely walk the waitq by using a sched_spinlock.
It should be noted that since walking the wait queue is an
operation of indeterminant length, there exists the possibility
that the sched_spinlock (which is a highly used and contended-for
lock) may be locked for an indeterminant amount of time. However,
it is expected that few threads will be waiting on any given kernel
event object, which should ameliorate this risk.
Fixes#54317
Signed-off-by: Aastha Grover <aastha.grover@intel.com>
Moving timeslice events to timeouts isn't quite enough on SMP, as it's
still possible for systems that don't broadcast their timer interrupts
to end up handling an expiration for a foreign CPU. There, we need an
IPI, and a symmetric call to z_time_slice() (which is itempotent and
fast) in the IPI ISR.
Signed-off-by: Andy Ross <andyross@google.com>
Rework the fragile and ad-hoc computation of timeslice expirations
into per-CPU struct _timeout objects with regular callbacks. The
expiration callbacks themselves simply set a per-cpu flag (they might
run on any CPU), which gets checked at the end of the timer ISR on
every CPU.
This simplifies logic and removes a bunch of code. It also fixes at
least three bugs:
1. As @npitre discovered: On SMP, the number of ticks announced on any
given CPU is going to be a subset of all expired ticks. This broke
the accounting of timeslice ticks, and effectively meant that
timeslicing only worked on SMP on systems where one CPU could hog all
the announcements, and only on that CPU.
2. The bootstrap path to arm the timer driver after setting the first
timeout in an empty list couldn't take into account
sys_clock_elapsed() ticks, as it didn't know whether it was being
called underneath an existing announce loop. Now this code is no
longer responsible for knowing anything about time slicing at all.
3. Also on SMP, there was a case where two CPUs timeslicing
simultaneously could stomp on each others' timeouts in
z_set_timeout_expiry(), as neither had a way of knowing what the
other's state was. CPUs could miss their own expiration and have to
wait for the slice expiration on the other CPU. Now, timeouts are
global objects with simple expiration times, and there's no need for
that function at all.
Signed-off-by: Andy Ross <andyross@google.com>
Some of the offset symbols that are derived from the macro
GEN_OFFSET_SYM() are not used anywhere in the Zephyr codebase.
Remove them as part of a cleanup effort.
Instances of an associated GEN_OFFSET_SYM() have also been
removed when the resulting macro is no longer referenced.
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
Some of the offset symbols generated via the macro GEN_OFFSET_SYM()
are not used anywhere in the Zephyr codebase. Remove them as part of
a cleanup effort.
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
Commit 3e729b2b1c ("kernel/timer: Correctly clamp period argument")
increased the lower limit to 1 so that it wouldn't conflict with a
K_NO_WAIT. But in doing so it enforced a minimum period of 2 ticks.
And the subtraction must obviously be avoided if the period is zero, etc.
Instead of doing this masquerade in k_timer_start(), let's move the
subtraction and clamping in z_timer_expiration_handler() right before
registering a new timeout. It makes the code cleaner, and then it is
possible to have single-tick periods again.
Whith this, timer_jitter_drift in tests/kernel/timer/timer_behavior does
pass with any CONFIG_SYS_CLOCK_TICKS_PER_SEC value, even when the tick
period is equal or larger than the specified timer period for the test
which failed the test before.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The call to unschedule_locked() would return true ("successfully
unscheduled") even in the case where the underlying z_abort_timeout()
failed (because the callback was already unpended and
in-progress/complete/about-to-be-run, remember that timeout callbacks
are unsynchronized), leading to state bugs and races against the
callback behavior.
Correctly detect that case and propagate the error to the caller.
Fixes#51872
Signed-off-by: Andy Ross <andyross@google.com>
Fixes sporadic data access violations that were occuring when pipes
were being used from an ISR. The ISR was incorrectly using the pipe
descriptor belonging to the interrupted thread. This led to corrupted
pipe meta-data. The solution proposed here is to perform a run-time
check and if use a pipe descriptor on the ISR's stack if called from
an ISR.
For additional information, see:
https://github.com/zephyrproject-rtos/zephyr/issues/52812
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
Adds a spin lock/unlock barrier pair after a pipe thread wakes.
After the list of waiting threads is generated, it is possible for
threads on that list to timeout and be removed from the wait queue.
However, since that list was generated before the timeout occurred,
the timed-out thread must wait until the copying is done (the
pipe's spin-lock has been released).
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
By the time the working list of readers/writers is processed, it is
possible that waiting reader/writer being processed had timed out
and is no longer on the wait queue. As such, we can not blindly
wake the next thread as that next thread might not be the thread we
had just been processing.
To address this, the calls to z_sched_wake() have been replaced
with z_unpend_thread() and z_ready_thread() so that a specific
thread can be safely targeted for waking.
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
Uses the new z_sched_waitq_walk() routine to walk the pipe's wait
queue to build a list of waiting threads that will be used for
the data transfer.
This method is preferred over the previous as it ensures that
wait queue is safely traversed.
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
Adds a routine to safely walk a specified wait queue and invoke a
custom callback function on each waiting thread.
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
When a timer is restarted from a high priority interrupt, it may
happen that the timer is re-added to the timeout list right after
it is removed from that list prior to execution of its expiration
handler but before that execution actually occurs. This leads to
an assertion failure reported for `z_add_timeout()` because then
that function, called from `z_timer_expiration_handler()` for
periodic timers, turns out to be adding a timeout that is already
added to the timeout list.
This commit detects such situation in `z_timer_expiration_handler()`
and makes that function exit immediately when that occurs (as the
timer was restared, its expiration handler should not be executed).
Signed-off-by: Andrzej Głąbek <andrzej.glabek@nordicsemi.no>
Most of the time, z_cstart() is running on an arbitrary region
of memory as stack, where the necessary stack setup has not been
performed. This prevents stack protection to work correctly,
as the stack canary has not been populated. So mark z_cstart()
to have no stack protection at all inside the function to avoid
raising exception during boot.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
This commit updates all in-tree code to use `CONFIG_CPP` instead of
`CONFIG_CPLUSPLUS`, which is now deprecated.
Signed-off-by: Stephanos Ioannidis <stephanos.ioannidis@nordicsemi.no>
At least one static analysis tool is flagging a potential NULL
derefence in sys_clock_announce()'s tick processing loop where the
routine 'first()' is concerned. In practice, this does not occur as
...
1. The code in question is protected by a spinlock.
2. 'first()' does not change the contents of anything.
The code has consequently been tweaked to prevent similar such false
positives in the future.
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
Accurate timekeeping is something that is often taken for granted.
However, reliability of timekeeping code is critical for most core
and subsystem code. Furthermore, Many higher-level timekeeping
utilities in Zephyr work off of ticks but there is no way to modify
ticks directly which would require either unnecessary delays in
test code or non-ideal compromises in test coverage.
Since timekeeping is so critical, there should be as few barriers
to testing timekeeping code as possible, while preserving
integrity of the kernel's public interface.
With this, we expose `sys_clock_tick_set()` as a system call only
when `CONFIG_ZTEST` is set, declared within the ztest framework.
Signed-off-by: Chris Friedt <cfriedt@meta.com>
The following testcases fail with qemu_cortex_r5 caused by main stack
overflow.
tests/kernel/workq/work_queue/kernel.workqueue
tests/ztest/base/testing.ztest.base.verbose_0_userspace
The main stack size is 512 for qemu_cortex_r5(a Cortex-A/R aarch32
platform) with CONFIG_ZTEST=y. The Cortex-M platforms are already set to
1024. Likely 512 will fail for most aarch32 platforms soon.
Fix the issue by increasing the CONFIG_MAIN_STACK_SIZE to 1024.
Also, remove 'default 1024 if TEST_ARM_CORTEX_M' since Cortex-M is no
longer an exception of default 1024.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
- Logging supports printing 64-bit values now. Cast to unsigned long and
use %lu all times.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
In z_phys_unmap(), the call to virt_region_free() is not using
aligned virtual address and space. This can result in freeing
smaller region that allocated given that inputs to z_phys_unmap()
may not be aligned. So use the already calculated aligned
virtual address and size as input to virt_region_free().
Note that the assertion and if-block in virt_region_free() to
check whether the to-be-unmapped region is within the virtual
memory region needs to be trimmed by one byte at the end.
The assertion and if-block are checking against the region
end address but (start + size) is just one byte over the end.
So subtract one.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
The C++ standard requires the main() function to have the return type
of 'int' and does not allow the main() to be defined with the 'void'
return type. Moreover, GCC goes as far as to emit a hard error when the
'::main()' has the return type of `void`.
This commit introduces an option to instruct the Zephyr kernel to call
the 'int main(void)' instead of the 'void main(void)' in case a Zephyr
application defines main() in a C++ source file.
Signed-off-by: Stephanos Ioannidis <stephanos.ioannidis@nordicsemi.no>
Move runtime code to use arch_num_cpus() instead of CONFIG_MP_NUM_CPUS
and use CONFIG_MP_MAX_NUM_CPUS for ifdef and BUILD_ASSERT macros.
Signed-off-by: Kumar Gala <kumar.gala@intel.com>
Cleanup the mess of duplicate function definitions, unnecessary
variables and duplicate strings. All banner strings are now constant in
ROM. Also fixes a double space between the end of the version string and
the trailing `***` when there is no boot delay.
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
The BOOT_DELAY option does nothing in code if MULTITHREADING is not
enabled. Move the dependency to Kconfig instead.
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
Change for loops of the form:
for (i = 0; i < CONFIG_MP_NUM_CPUS; i++)
...
to
unsigned int num_cpus = arch_num_cpus();
for (i = 0; i < num_cpus; i++)
...
We do the call outside of the for loop so that it only happens once,
rather than on every iteration.
Signed-off-by: Kumar Gala <kumar.gala@intel.com>
Continue to phase out MP_NUM_CPUS, change Kconfig to be
MP_MAX_NUM_CPUS and make MP_MAX_NUM_CPUS the main Kconfig symbol.
Signed-off-by: Kumar Gala <kumar.gala@intel.com>
The dummy thread doesn't include a TLS area, so any thread local variables
will fail to work if used in the switched_out tracing hook. Skip the hook
in this case, as it's not really accurate anyways; the dummy thread is
only used to set context for the initial switch for each core.
Signed-off-by: Keith Packard <keithp@keithp.com>
Using char pointers for %p should be avoided in log messages. It will
cause issues in configurations where logging strings are removed from
the binary and they are not inspected when cbprintf packages from
logging string are built. In that case any char pointers are treated as
strings and copied into the pacakge body.
Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
Introduce a Kconfig (MP_MAX_NUM_CPUS) and an api arch_num_cpus() to
allow for systems that might determine the number of CPUs available to
Zephyr at runtime.
CONFIG_MP_MAX_NUM_CPUS is intented to be use for any array initialization
and such that need to occur at build time. For most systems
arch_num_cpus() will just report the value of CONFIG_MP_MAX_NUM_CPUS.
The intent is to phase out CONFIG_NP_NUM_CPUS.
Signed-off-by: Kumar Gala <kumar.gala@intel.com>
Warnings being treated as errors when building :
Error this 'for' clause does not guard...
[-Werror=misleading-indentation]
Signed-off-by: Francois Ramu <francois.ramu@st.com>
The _SYS_INIT_LEVEL* definitions were used to indicate the index entry
into the levels array defined in init.c (z_sys_init_run_level). init.c
uses this information internally, so there is no point in exposing this
in a public header. It has been replaced with an enum inside init.c. The
device shell was re-using the same defines to index its own array. This
is a fragile design, the shell needs to be responsible of its own data
indexing. A similar situation happened with some unit tests.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
The function in charge of calling all init function was defined in
device.c, had a public prototype and was just used in init.c. Since this
is really an internal function tied to Kernel init code, move it to
init.c and make it static, there's no need to expose it publicly.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
The `ARCH` init level was added to solve a specific problem, call init
code (SYS_INIT/devices) before `z_cstart` in the `intel_adsp` platform.
The documentation claims it runs before `z_cstart`, but this is only
true if the SoC/arch takes care of calling:
```c
z_sys_init_run_level(_SYS_INIT_LEVEL_ARCH);
```
Which is only true for `intel_adsp` nowadays. So in practice, we now
have a platform specific init level. This patch proposes to do things in
a slightly different way. First, level name is renamed to `EARLY`, to
emphasize it runs in the early stage of the boot process. Then, it is
handled by the Kernel (inside `z_cstart()` before calling
`arch_kernel_init()`). This means that any platform can now use this
level. For `intel_adsp`, there should be no changes, other than
`gcov_static_init()` will be called before (I assume this will allow to
obtain coverage for code called in EARLY?).
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
For historical reasons[1] suspending threads would release the
scheduler lock between pend() (which places the current thread onto a
wait queue) and z_swap() (which effects the context swtich). This
process happens with the caller's lock held, so local interrupts are
masked. But on SMP this opens a tiny race where another CPU could
grab the pended thread and switch to it while we were still executing
on its stack!
Fix this by elevating the "lock swap" code that already exists in the
(portable/switch-based) z_swap() code one level so that it happens in
z_pend_curr() also. Now we hold the scheduler lock between pend and
the final context switch.
Note that this technique can't work for the older z_swap_irqlock()
implementation, which exists to vestigially support a few bits of arch
code (mostly direct interrupts) that don't work on SMP anyway.
Address with an assert to prevent future misuse.
[1] z_swap() is a historical API implemented in per-arch assembly for
older architectures (like ARM32!). It was designed to be called
with what at the time was a global IRQ lock, so it doesn't
understand the idea of a separate scheduler lock. When we finally
get all archictures on arch_switch() this design can be cleaned up
quite a bit.
Signed-off-by: Andy Ross <andyross@google.com>
We have cases where some devices needs to be initialized very early and
before c_start is call, i.e. to setup very early console or to setup
memory. Traditionally this would be hardcoded as part of the soc layer
and not using device model or the init levels.
This patch adds a new level ARCH, which will be called in early
architecture code and before we jump to the kernel code.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
k_heap_aligned_alloc was not handling K_FOREVER timeout
correctly due to unsigned return value. Added explicit
K_FOREVER handling of end time.
Fixes#50611.
Signed-off-by: Jay Shoen <jay.shoen@perceive.io>
