Commit graph

3183 commits

Author SHA1 Message Date
Anas Nashif
3dc0c4544b kernel: move float operations out of thread.c
Move float operation out and add missing vrfy hook for enabling float.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-03-06 19:27:28 -05:00
Anas Nashif
6c003bdbcf kernel: remove unused code in headers
List of functions defined in headers and not being used anywhere.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-03-06 19:27:28 -05:00
Anas Nashif
9e83413542 kernel: split thread monitor
Move thread monitor related functions, not enabled in most cases outside
of thread.c and cleanup headers.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-03-06 19:27:28 -05:00
Anas Nashif
5e591c38f1 kernel: do not export z_thread_priority_set
This function is only being used by a test, so instead of reimplementing
a syscall in the test, provide a Kconfig option to provide the
functionality that only works with tests and remove some of the
duplication and extra code.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-03-06 19:27:28 -05:00
Anas Nashif
3ca50f5060 kernel: move z_init_static_threads to where it is being used
Move out of thread and put directly in init.c where it is being used.
Also remove definition from kernel.h, this is an internal function and
should not be in a public header.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-03-06 19:27:28 -05:00
Anas Nashif
a6ce422b10 kernel: remove cmsis-rtos layering violation
We shouldn't be calling hooks from optional and upper layer subsystems
in the kernel, instead, just call the hook to set thread status in the
API where it is needed.

This now clears related bit in cmsis thread status bitarray when
terminating a thread in the cmsis rtos v1 layer directly and not in the
kenrel code.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-03-06 19:27:28 -05:00
Anas Nashif
077222c975 kernel: split cpu_mask handling into own file
In an effort to cleanup sched.c, move sections of code that can be
compiled in based on options into own files. CPU mask here is managed by
a kconfig and is not widely used (SMP affinity on multicore systems).

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-03-06 19:27:28 -05:00
Anas Nashif
6e95bdeca6 kernel: reorg Kconfigs and split them
The kernel kconfig is becoming too big and unmanageable with too many
options scattered across the file. Move some areas out and reorg main
Kconfig slightly.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-03-06 19:27:28 -05:00
Anas Nashif
8791012ed1 kernel: move essential flag related routines out
The functions to manipulate the essential flag indeed operate on
threads, but they are misplaced in the thread implementation file. Put
them alongside other routines setting other thread flags and cleanup
headers a bit.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-03-06 19:27:28 -05:00
Anas Nashif
2c31db4cc8 kernel: split irq_offload ccode into own file
Move irq_offload code out of thread.c into own file. This code is not
related to threads.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-03-06 19:27:28 -05:00
Anas Nashif
a7d74f80ce kernel: move spinlock validation to own file
Move spin_lock validation outside of thread.c into own file. This code
really has nothing to do with the thread code.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-03-06 19:27:28 -05:00
Ederson de Souza
4440d6a189 kernel/userspace: Fix dynamic thread stack allocation at userspace
It wasn't saving adjusted stack size at either the private stack or the
k_object, thus failing subsequent checks.

Test added to check for this case and prevent regressions.

Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
2024-03-06 14:17:53 +01:00
Anas Nashif
46484da502 kernel: move priority queue handling to own file/header
clean up headers under include/ and move handling of priority queue to
own file/header.
No need for the header  include/zephyr/kernel/internal/sched_priq.h
anymore. Move the relevant structures where they are being used in
kernel_structs.h.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-03-02 15:06:45 +01:00
Peter Mitsis
ee9c44fee6 linker: Round TLS sizes up in linker script
Instead of rounding up both __tdata_size and __tbss_size at runtime,
perform the calculation when the image is built.

Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
2024-02-25 22:28:56 -05:00
Nguyen Minh Thien
8188be57d3 kernel: fix spelling errors
Fix spelling errors found in comment of the kernel source code.

Signed-off-by: Nguyen Minh Thien <nguyenmthien@live.com>
2024-02-25 20:53:37 -05:00
Peter Mitsis
51ae993c12 kernel: Update k_wakeup()
This commit does two things to k_wakeup():

1. It locks the scheduler before marking the thread as not suspended.
As the the clearing of the _THREAD_SUSPENDED bit is not atomic, this
helps ensure that neither another thread nor ISR interrupts this
action (resulting in a corrupted thread_state).

2. The call to flag_ipi() has been removed as it is already being
made within ready_thread().

Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
2024-02-25 20:50:03 -05:00
Guennadi Liakhovetski
09cf3e0910 smp: fix a race when starting / resuming multiple CPUs
cpu_start_fn is global, it's used by the initiator CPU to start or
resume secondary CPUs. However it's possible, that the initiator CPU
goes ahead and starts a second secondary CPU before the first one has
finished using the object. Fix this by creating a local copy of the
global object.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
2024-02-16 07:27:04 +01:00
Flavio Ceolin
d622533ca7 kernel: thread: Allow stack in coherent memory
When allowing dynamic thread stack allocation the stack may come from
the heap in coherent memory, trying to use cached memory is over
complicated because of heap meta data and cache line sizes.
Also when userspace is enabled, stacks have to be page aligned and the
address of the stack is used to track kernel objects.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2024-02-09 14:23:14 -05:00
Daniel Leung
caacc27d37 kernel: smp: CPU start may result in null pointer access
It is observed that starting up CPU may result in other CPUs
crashing due to de-referencing NULL pointers. Note that this
happened on the up_squared board, but there was no way to
attach debugger to verify. It started working again after
moving z_dummy_thread_init() before smp_timer_init(), which
was the old behavior before commit
eefaeee061 where the issue
first appeared. So mimic the old behavior to workaround
the issue.

Fixes #68115

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2024-02-04 19:55:14 -06:00
Andy Ross
840db7858e kernel/thread: Detect in-kernel "reserved" stack overflow
Traditionally, k_thread_create() has required that the application
size the stack correctly.  Zephyr doesn't detect or return errors and
treats stack overflow as an application bug (though obviously some
architectures have runtime features to trap on overflows).

At this one spot though, it's possible for the kernel to adjust the
stack for K_THREAD_STACK_RESERVED in such a way that the arch layer's
own stack initialization overflows.  That failure can be seen by
static analysis, so we can't just sweep it under the rug as an
application failure.

Unfortunately there aren't any good options for handling it here (no
way to return failure, can't be a build assert as the size is a
runtime argument).  A panic will have to do.

Fixes: #67106
Fixes: #65584

Signed-off-by: Andy Ross <andyross@google.com>
2024-02-04 10:23:25 -05:00
Krzysztof Chruściński
25173f71cd pm: device_runtime: Extend with synchronous runtime PM
In many cases suspending or resuming of a device is limited to
just a few register writes. Current solution assumes that those
operations may be blocking, asynchronous and take a lot of time.
Due to this assumption runtime PM API cannot be effectively used
from the interrupt context. Zephyr has few driver APIs which
can be used from an interrupt context and now use of runtime PM
is limited in those cases.

Patch introduces a new type of PM device - synchronous PM. If
device is specified as capable of synchronous PM operations then
device runtime getting and putting is executed in the critical
section. In that case, runtime API can be used from an interrupt
context. Additionally, this approach reduces RAM needed for
PM device (104 -> 20 bytes of RAM on ARM Cortex-M).

Signed-off-by: Krzysztof Chruściński <krzysztof.chruscinski@nordicsemi.no>
2024-02-01 15:03:42 +01:00
Christopher Friedt
9f3d7776ab kernel: dynamic: reduce verbosity in degenerate case
k_thread_stack_free() is designed to be called with any pointer
value. We return -EINVAL when an attempt is made to free an
invalid stack pointer.

This change reduces the verbosity in the degenerate case, when
the pointer is not obtained via k_thread_stack_alloc(), but
otherwise does not affect functionality.

If debug log verbosity is not enabled, we save a few bytes in
.text / .rodata / .strtab.

Signed-off-by: Christopher Friedt <cfriedt@meta.com>
2024-01-26 06:50:11 -05:00
Daniel Leung
2cdd44801e kernel: move z_init_cpu to private kernel headers
z_init_cpu() is a private kernel API so move it under
kernel/include.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2024-01-17 11:57:20 -05:00
Daniel Leung
35a1814c4d kernel: smp: remove z_smp_thread_init/_swap
This removes z_smp_thread_init() and z_smp_thread_swap() as
SOF has been updated to use k_smp_cpu_custom_start() instead.
This removes the need for SOF to mirror part of the SMP
power up code, and thus these two functions are no longer
needed.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2024-01-17 11:57:20 -05:00
Daniel Leung
eefaeee061 kernel: smp: introduce k_smp_cpu_resume
This provides a path to resume a previously suspended CPU.
This differs from k_smp_cpu_start() where per-CPU kernel
structs are not initialized such that execution context
can be saved during suspend and restored during resume.
Though the actual context saving and restoring are
platform specific.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2024-01-17 11:57:20 -05:00
Daniel Leung
fe66e35db0 kernel: smp: put comment on SMP code
Adds some comments to the SMP code to, hopefully, make it
a bit more clear to future readers.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2024-01-17 11:57:20 -05:00
Daniel Leung
89b231e7e2 kernel: smp: introduce k_smp_cpu_start
This renames z_smp_cpu_start() to k_smp_cpu_start().
This effectively promotes z_smp_cpu_start() into a public API
which allows out of tree usage. Since this is a new API,
we can afford to change it signature, where it allows
an additional initialization steps to be done before a newly
powered up CPU starts participating in scheduling threads
to run.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2024-01-17 11:57:20 -05:00
Daniel Leung
803e0e452f kernel: amend wording on CONFIG_SMP_BOOT_DELAY
This extends the wording so that not only architecture code can
start secondary CPUs at a later time. Also adds a missing 'to'.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2024-01-17 11:57:20 -05:00
Gerson Fernando Budke
b8188e54a4 kernel: Implement k_sleep for Single Thread
The current z_tick_sleep return directly when building kernel for Single
Thread model. This reorganize the code to use k_busy_wait() to be time
coherent since subsystems may depend on it.

In the case of a K_FOREVER timeout is selected the Single Thread the
implementation will invoke k_cpu_idle() and the system will wait for
an interrupt saving power.

Signed-off-by: Gerson Fernando Budke <gerson.budke@ossystems.com.br>
2024-01-10 15:10:16 +01:00
Gaetan Perrot
68581caa74 kernel: need_swap zephyrproject-rtos#66299
Enhancement on void z_reschedule_irqlock(uint32_t key)
to avoid useless context switch

signed-off-by: Gaetan Perrot <gaetanperrotpro@gmail.com>
2024-01-04 09:42:12 +01:00
Junfan Song
4ae558c505 kernel: work: Fix race in workqueue thread
After a call to k_work_flush returns the sync variable
may still be modified by the workq.  This is because
the work queue thread continues to modify the flag in
sync even after k_work_flush returns.  This commit adds
K_WORK_FLUSHING_BIT, and with this bit, we moved the
logic of waking up the caller from handle_flush to the
finalize_flush_locked in workq, so that after waking up
the caller, the workqueue will no longer operate on sync.

Fixes: #64530

Signed-off-by: Junfan Song <sjf221100@gmail.com>
2024-01-03 10:20:19 +01:00
Anas Nashif
05315ea6af kernel: fatal: remove LCOV exclusion
line is already excluded as part of the function.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-12-21 09:18:44 +01:00
Daniel Leung
a7dccc4475 kernel: mmu: mitigate range check overflow issue
It is possible that address + size will overflow the available
address space and the pointer wraps around back to zero. Some
of these have been fixed in previous commits. This fixes
the remaining ones with regard to Z_PHYS_RAM_START/_END,
and Z_VIRT_RAM_START/_END.

Fixes #65542

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-12-20 11:37:17 -05:00
Johan Hedberg
7bb16c779b posix: mqueue: Remove custom default for HEAP_MEM_POOL_SIZE
Use the new HEAP_MEM_POOL_ADD_SIZE_ prefix to construct a minimum
requirement for posix message queue usage. This way we can remove the
"special case" default values from the HEAP_MEM_POOL_SIZE Kconfig
definition.

Signed-off-by: Johan Hedberg <johan.hedberg@intel.com>
2023-12-20 11:01:42 +01:00
Johan Hedberg
3fbf12487c kernel: Introduce a way to specify minimum system heap size
There are several subsystems and boards which require a relatively large
system heap (used by k_malloc()) to function properly. This became even
more notable with the recent introduction of the ACPICA library, which
causes ACPI-using boards to require a system heap of up to several
megabytes in size.

Until now, subsystems and boards have tried to solve this by having
Kconfig overlays which modify the default value of HEAP_MEM_POOL_SIZE.
This works ok, except when applications start explicitly setting values
in their prj.conf files:

$ git grep CONFIG_HEAP_MEM_POOL_SIZE= tests samples|wc -l
     157

The vast majority of values set by current sample or test applications
is much too small for subsystems like ACPI, which results in the
application not being able to run on such boards.

To solve this situation, we introduce support for subsystems to specify
their own custom system heap size requirement. Subsystems do
this by defining Kconfig options with the prefix HEAP_MEM_POOL_ADD_SIZE_.
The final value of the system heap is the sum of the custom
minimum requirements, or the value existing HEAP_MEM_POOL_SIZE option,
whichever is greater.

We also introduce a new HEAP_MEM_POOL_IGNORE_MIN Kconfig option which
applications can use to force a lower value than what subsystems have
specficied, however this behavior is disabled by default.

Whenever the minimum is greater than the requested value a CMake warning
will be issued in the build output.

This patch ends up modifying several places outside of kernel code,
since the presence of the system heap is no longer detected using a
non-zero CONFIG_HEAP_MEM_POOL_SIZE value, rather it's now detected using
a new K_HEAP_MEM_POOL_SIZE value that's evaluated at build.

Signed-off-by: Johan Hedberg <johan.hedberg@intel.com>
2023-12-20 11:01:42 +01:00
Andrei Emeltchenko
9d3a3e96e1 kernel: threads: Do not use string compare instead of bit ops
Remove converting bit to string and comparing the string instead of
ready helpers. The "Check if thread is in use" seems to check only
that parameters state_buf and sizeof(state_buf) not zero.

Signed-off-by: Andrei Emeltchenko <andrei.emeltchenko@intel.com>
2023-12-18 12:24:53 +01:00
Peter Mitsis
4b2bf5abcc kernel: Apply const to k_pipe_put() parameter
The pointer parameter 'data' in the function 'k_pipe_put()' ought to
use the const modifier as the contents of the buffer to which it
points never change. Internally, that const modifier is dropped as
both 'k_pipe_get()' and 'k_pipe_put()' share common code for copying
data; however 'k_pipe_put()' never takes a path that modifies those
contents.

Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
2023-12-15 14:51:35 -05:00
Anas Nashif
a7e8391e31 debug: coredump: handle xtensa coredump like everyone else
There should not be any special handling for coredump with xtensa.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-12-14 09:32:27 +01:00
Andrej Butok
cc4ffb2a54 kernel: cmake: fix k_sleep compilation error for single thread
Fix k_sleep compilation error:
[build] ... syscalls/kernel.h:135: undefined reference to `z_impl_k_sleep'
for single thread applications (CONFIG_MULTITHREADING = n).
The shed.c contains source code which must be present also
in single thread applications.

Signed-off-by: Andrej Butok <andrey.butok@nxp.com>
2023-12-14 09:31:38 +01:00
Daniel Leung
fa561ccd59 kernel: mmu: no need to expose z_free_page_count
z_free_page_count is only used in one file, so there is
no need to expose it, even to other part of kernel.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-12-12 18:46:21 +00:00
Daniel Leung
22447c9736 kernel: mmu: fix typo K_DIRECT_MAP to K_MEM_DIRECT_MAP
Fix typo in comment to reflect the actual macro named
K_MEM_DIRECT_MAP.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-12-08 08:31:15 -05:00
Peter Mitsis
a3e5af95de kernel: Update k_sleep() and k_usleep() return values
Updates both the k_sleep() and k_usleep() return values so that if
the thread was woken up prematurely, they will return the time left
to sleep rounded up to the nearest millisecond (for k_sleep) or
microsecond (for k_usleep) instead of rounding down. This removes
ambiguity should there be a non-zero number of remaining ticks
that correlate to a time of less than 1 millisecond or 1 microsecond.

Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
2023-12-07 10:41:00 +00:00
Keith Packard
9f42537fc7 kernel: Use k_us_to_cyc_ceil32 in k_busy_wait
Replace open-coded time conversion with the macro which as that will
usually use a constant divide or multiply.

Signed-off-by: Keith Packard <keithp@keithp.com>
2023-12-05 09:24:28 +01:00
Guennadi Liakhovetski
69cdc32892 llext: export some symbols
Export some symbols for loadable modules. Also add an
EXPORT_SYSCALL() helper macro for exporting system calls by their
official names.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
2023-12-01 10:08:12 -05:00
Daniel Leung
34c6b17680 doc: fixed already renamed _Cstart to z_cstart
The function _Cstart has already been renamed to z_cstart,
so change the remaining references of it in various docs.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-11-30 21:01:47 -05:00
Qipeng Zha
3d29c9fe54 kernel: timeout: fix issue with z_timeout_expires
- issue found with Ztest case of test_thread_timeout_remaining_expires
  on Intel ISH platform when adjust CONFIG_SYS_CLOCK_TICKS_PER_SEC
  to 10k.
- timeout_rem() return exact remaining ticks which is calibrated by
  decrease elapsed(), while z_timeout_expires try to get expire ticks
  to be timeout using current tick as base, so need get exact current
  ticks by plus elasped().

Signed-off-by: Qipeng Zha <qipeng.zha@intel.com>
2023-11-30 12:22:54 +01:00
Peter Mitsis
91a6af3e8f kernel: mmu: Fix static analysis issue
Instead of performing a set of relative address comparisons using
pointers of type 'uint8_t *', we leverage the existing IN_RANGE()
macro and perform the comparisons with 'uintptr_t'.

Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
2023-11-28 16:44:16 -05:00
Christopher Friedt
afc59112a9 device: support for mutable devices
Add support for mutable devices. Mutable devices are those which
can be modified after declaration, in-place, in kernel mode.

In order for a device to be mutable, the following must be true

  * `CONFIG_DEVICE_MUTABLE` must be y-selected
  * the Devicetree bindings for the device must include
    `mutable.yaml`
  * the Devicetree node must include the `zephyr,mutable` property

Signed-off-by: Christopher Friedt <cfriedt@meta.com>
2023-11-28 15:35:39 +01:00
Daniel Leung
40ba4015e3 kernel: mm: only include demand_paging.h if needed
This moves including of demand_paging.h out of kernel/mm.h,
so that users of demand paging APIs must include the header
explicitly. Since the main user is kernel itself, we can be
more discipline about header inclusion.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-11-23 10:01:45 +01:00
Flavio Ceolin
8679c58644 kernel: Option to not use tls to get current thread
Add a Kconfig option to tell whether or not using thread
local storage to store current thread.

The function using it can be called from ISR and using
TLS variables in this context may (should ???) not be
allowed

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2023-11-21 15:49:48 +01:00