Commit graph

334 commits

Author SHA1 Message Date
Andy Ross
b11e796c36 kernel/sched: Add CONFIG_CPU_MASK_PIN_ONLY
Some SMP applications have threading designs where every thread
created is always assigned to a specific CPU, and never want to
schedule them symmetrically across CPUs under any circumstance.

In this situation, it's possible to optimize the run queue design a
bit to put a separate queue in each CPU struct instead of having a
single global one.  This is probably good for a few cycles per
scheduling event (maybe a bit more on architectures where cache
locality can be exploited) in circumstances where there is more than
one runnable thread.  It's a mild optimization, but a basically simple
one.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2021-09-28 20:15:05 -04:00
Fabio Baltieri
7f36f90167 kernel: drop unused priority related definitions
There are no reference for either K_NUM_PRIORITIES or
K_NUM_PRIO_BITMAPS, with the former being dropped in:

  1acd8c2996 kernel: Scheduler rewrite

Dropping both of these, and also the two comments about extra priorities
taking extra RAM space, as those do not seem to apply either.

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2021-08-17 17:52:17 -04:00
Ioannis Glaropoulos
4084242a71 kernel: make MULTITHREADING promptless if single-thread not supported
If single thread builds are not supported by the
architecture, the MULTITHREADING option should be
prompt-less to block any modifications to it. We
also introduce an explicit ARCH-level Kconfig that
reflects whether the ARCH is capable of single-thread
Zephyr builds.

Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
2021-05-26 11:03:22 -05:00
Andy Ross
851d14afc8 kernel/sched: Remove "cooperative scheduling only" special cases
The scheduler has historically had an API where an application can
inform the kernel that it will never create a thread that can be
preempted, and the kernel and architecture layer would use that as an
optimization hint to eliminate some code paths.

Those optimizations have dwindled to almost nothing at this point, and
they're now objectively a smaller impact than the special casing that
was required to handle the idle thread (which, obviously, must always
be preemptible).

Fix this by eliminating the idea of "cooperative only" and ensuring
that there will always be at least one preemptible priority with value
>=0.  CONFIG_NUM_PREEMPT_PRIORITIES now specifies the number of
user-accessible priorities other than the idle thread.

The only remaining workaround is that some older architectures (and
also SPARC) use the CONFIG_PREEMPT_ENABLED=n state as a hint to skip
thread switching on interrupt exit.  So detect exactly those platforms
and implement a minimal workaround in the idle loop (basically "just
call swap()") instead, with a big explanation.

Note that this also fixes a bug in one of the philosophers samples,
where it would ask for 6 cooperative priorities but then use values -7
through -2.  It was assuming the kernel would magically create a
cooperative priority for its idle thread, which wasn't correct even
before.

Fixes #34584

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2021-05-24 23:38:16 -04:00
Anas Nashif
7b9084cb4f ztest: set thread name to test name
Use the actual test name and not a hardcoded thread name. This is useful
for tracing test cases.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-05-17 18:45:57 -04:00
Guennadi Liakhovetski
8d07b7751a smp: add a Kconfig option to delay booting secondary CPUs
Usually Zephyr boots all secondary CPUs as a part of system
boot. Some applications however need an ability to boot on
the main CPU only and enable secondary CPUs selectively at
run-time. Add a Kconfig option to support this behaviour.
When booting CPUs on demand applications also need helpers
to initialise a dummy thread and begin threaded execution
on those CPUs, add two such helpers.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
2021-05-03 17:13:01 -04:00
Anas Nashif
6df4405cca doc: fix typos
Fix various typos in the docs.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-04-30 16:03:08 -04:00
Krzysztof Chruscinski
2165e8c585 Revert "kernel: Deprecate CONFIG_MULTITHREADING"
This reverts commit 28cb9dab64.
2021-04-29 14:50:35 +02:00
Daniel Leung
1559712b22 timing: add kconfig CONFIG_TIMING_FUNCTIONS_NEED_AT_BOOT
This adds a new kconfig CONFIG_TIMING_FUNCTIONS_NEED_AT_BOOT so
that the timing subsystem can be initialized at boot, instead of
being #ifdef under thread runtime statistics. This will allow
other part of kernel and other subsystems to utilize the timing
functions.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-04-06 16:43:55 -04:00
Anas Nashif
068e0872d7 kernel: remove EXPERIMENTAL from some Kconfigs
both thread monitor and thread names are not EXPERIMENTAL any more. They
have been used across the tree and lots depend on those features
already.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-03-23 13:01:08 +01:00
Anas Nashif
c076d94eec kernel: remove tickless idle
This feature predated the tickless kernel and has been in legacy mode
for a while. We now have no drivers or systems that do not support
tickless, so remove this option and cleanup the code to only use
tickless.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-03-19 11:22:17 -04:00
Kumar Gala
7d35a8c93d kernel: remove arch_mem_domain_destroy
The only user of arch_mem_domain_destroy was the deprecated
k_mem_domain_destroy function which has now been removed.  So remove
arch_mem_domain_destroy as well.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2021-03-18 16:30:47 +01:00
Andy Ross
820c94e5dd arch/xtensa: Inline atomics
The xtensa atomics layer was written with hand-coded assembly that had
to be called as functions.  That's needlessly slow, given that the low
level primitives are a two-instruction sequence.  Ideally the compiler
should see this as an inline to permit it to better optimize around
the needed barriers.

There was also a bug with the atomic_cas function, which had a loop
internally instead of returning the old value synchronously on a
failed swap.  That's benign right now because our existing spin lock
does nothing but retry it in a tight loop anyway, but it's incorrect
per spec and would have caused a contention hang with more elaborate
algorithms (for example a spinlock with backoff semantics).

Remove the old implementation and replace with a much smaller inline C
one based on just two assembly primitives.

This patch also contains a little bit of refactoring to address the
scheme has been split out into a separate header for each, and the
ATOMIC_OPERATIONS_CUSTOM kconfig has been renamed to
ATOMIC_OPERATIONS_ARCH to better capture what it means.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2021-03-08 11:14:27 -05:00
Andy Ross
6400bb54d6 kernel/idle: Clean up and refactoring / remove TICKLESS_IDLE_THRESH
While I'm in the idle code, let's clean this loop up.  It was a really
bad #ifdef hell:

* Remove the CONFIG_TICKLESS_IDLE_THRESH logic (and the kconfig),
  which never did anything but needlessly increase latency.

* Move the needed timeout logic from the main loop into
  pm_save_idle(), which eliminates the special case for
  !SYS_CLOCK_EXISTS.

Behavior (modulo that one kconfig) should be completely unchanged, and
now the inner part of the idle loop looks like:

    while (true) {
        (void) arch_irq_lock();

        if (IS_ENABLED(CONFIG_PM)) {
            pm_save_idle();
        } else {
            k_cpu_idle();
        }
    }

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2021-03-04 14:31:12 -05:00
Peter Bigot
b706a5e999 kernel: remove old work queue implementation
Now that the old API has been reimplemented with the new API remove
the old implementation and its tests.

Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
2021-03-03 20:06:00 -05:00
Peter Bigot
d1affd9118 kernel: default to new work API implementation
Switch the default and clean up some test workarounds.  This will enable
final conversions necessary to transition to the new API.

Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
2021-03-03 20:06:00 -05:00
Peter Bigot
dc34e7c6f6 kernel: add new work queue implementation
This commit provides a complete reimplementation of the work queue
infrastructure intended to eliminate the race conditions and feature
gaps in the existing implementation.

Both bare and delayable work structures are supported.  Items can be
submitted; delayable items can be scheduled for submission at a future
time.  Items can be delayed, queued, and running all at the same time.
A running item can also be canceling.

The new implementation:
* replaces "pending" with "busy" which identifies the active states;
* supports canceling delayed and submitted items;
* prevents resubmission of a item being canceled until cancellation
  completes;
* supports waiting for cancellation to complete;
* supports flushing a work item (waiting for the last submission to
  complete without preventing resubmission);
* supports waiting for a queue to drain (only allows resubmission from
  the work thread);
* supports stopping a work queue in conjunction with draining it;
* prevents handler-reentrancy during resubmission.

Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
2021-03-03 20:06:00 -05:00
Peter Bigot
44539ed645 kernel: select work queue implementation
Attempts to reimplement the existing work API using a new work
implementation failed, primarily due to heavy use of whitebox testing
in validating the original API.  Add a temporary Kconfig that will
select between the two implementations so we can use the same
identifiers but select which implementation they reference.

This commit just adds the selection infrastructure and uses it to
conditionalize the existing implementation in anticipation of the new
one in the next commit.

Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
2021-03-03 20:06:00 -05:00
Martin Åberg
612dad264c kernel: Decouple TICKS_PER_SEC from TICKLESS_CAPABLE
The SYS_CLOCK_TICKS_PER_SEC default may depend on the kernel config
for tickless, rather than the capability.

Signed-off-by: Martin Åberg <martin.aberg@gaisler.com>
2021-02-04 12:34:23 -05:00
Alexandre Bourdiol
8925af94f2 kernel: Kconfig: increase test default MAIN_STACK_SIZE for ARM Cortex M
There are more and more tests that fail due to stackoverflow.
Increasing MAIN_STACK_SIZE to fix those issues.

Signed-off-by: Alexandre Bourdiol <alexandre.bourdiol@st.com>
2021-02-02 10:05:46 -05:00
Flavio Ceolin
d6d4a832a4 kernel: build: Make TICKLESS_KERNEL depends on TICKLESS_CAPABLE
Tickless kernel option depends on tickless capable being selected by
the target.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2021-01-21 17:20:32 -05:00
Nicolas Pitre
7a22a4bdf6 heap: clean up some size related issues
First, the maximum heap size must fit in 31 bits worth of chunks
because the internal 32-bit field holding the size is shared with
the `used` bit.

Then the mention of a 256-byte block in the doc is no longer
relevant. That pertained to the previous allocator implementation.

And ditto for the HEAP_MEM_POOL_MIN_SIZE kconfig option.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2021-01-15 12:08:20 -05:00
Andy Ross
e956639dd6 kernel: Remove CONFIG_LEGACY_TIMEOUT_API
This was a fallback for an API change several versions ago.  It's time
for it to go.

Fixes: #30893

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2021-01-14 21:33:16 -05:00
Anas Nashif
dd931f93a2 power: standarize PM Kconfigs and cleanup
- Remove SYS_ prefix
- shorten POWER_MANAGEMENT to just PM
- DEVICE_POWER_MANAGEMENT -> PM_DEVICE

and use PM_ as the prefix for all PM related Kconfigs

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-09 15:18:29 -05:00
Anas Nashif
e1d42724e5 kernel: make KERNEL_COHERENCE depend on ARCH_HAS_COHERENCE
We can't enable KERNEL_COHERENCE is architecture does not support it.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-08 09:30:02 -05:00
Andy Ross
8a6aee9cac kernel: Make the "heap" backend to mem_pool default
Remove the MEM_POOL_HEAP_BACKEND kconfig, treating it as true always.
Now the legacy mem_pool cannot be enabled and all usage uses the
k_heap/sys_heap backend.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2020-12-07 21:50:14 -05:00
Andrew Boie
3f7ae0d749 x86: increase default idle stack size
With MMU features enabled, we are using 248 out of 256
available bytes on 32-bit. This is extremely uncomfortable, relax
to a larger value like several other arches.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-12-03 17:33:39 -05:00
Kamil Lazowski
104f100749 kernel: memslab: Add maximum mem_slab utilization trace
Add new function to mem_slab API that enables user
to get maximum number of slabs used so far.

Signed-off-by: Kamil Lazowski <Kamil.Lazowski@nordicsemi.no>
2020-11-18 22:33:27 -05:00
Daniel Leung
fd7a68dbe9 kernel: use timing functions to gather thread runtime stats
This uses the timing functions to gather execution cycles of
threads. This provides greater details if arch/SoC/board
uses timer with higher resolution.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-11-11 23:55:49 -05:00
Daniel Leung
fc577c4bd1 kernel: gather basic thread runtime statistics
This adds the bits to gather the first thread runtime statictic:
thread execution time. It provides a rough idea of how much time
a thread is spent in active execution. Currently it is not being
used, pending following commits where it combines with the trace
points on context switch as they instrument the same locations.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-11-11 23:55:49 -05:00
Spoorthy Priya Yerabolu
e9ef078a11 kernel: kconfig: changing recommendations for SYS_CLOCK_TICKS_PER_SEC
Documentation for kconfig SYS_CLOCK_TICKS_PER_SEC has some outdated
recommendations. Changing them to align with other documentation
under kernel timing.

Fixes: #25482

Signed-off-by: Spoorthy Priya Yerabolu <spoorthy.priya.yerabolu@intel.com>
2020-11-11 23:55:10 -05:00
Daniel Leung
2d152ab019 kernel: limit thread local storage to Zephyr SDK
Toolchains other than Zephyr SDK may not support generating
code with thread local storage. So limit TLS to Zephyr SDK
for now, and only enable TLS on other toolchains as needed.

Fixes #29541

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-10-30 15:59:06 +01:00
Daniel Leung
62cf1960ad kernel: support using thread local storage for errno
This enables storing errno in the thread local storage area.
With this enabled, a syscall to access errno can be avoided
when userspace is also enabled.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-10-24 10:52:00 -07:00
Daniel Leung
388725870f arm: cortex_m: add support for thread local storage
Adds the necessary bits to initialize TLS in the stack
area and sets up CPU registers during context switch.

Note that since Cortex-M does not have the thread ID or
process ID register needed to store TLS pointer at runtime
for toolchain to access thread data, a global variable is
used instead.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-10-24 10:52:00 -07:00
Daniel Leung
240beb42af kconfig: add configs for thread local storage
Add kconfigs to indicate whether an architecture has support
for thread local storage (TLS), and to enable TLS in kernel.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-10-24 10:52:00 -07:00
Andy Ross
f6d32ab0a4 kernel: Add cache coherence management framework
Zephyr SMP kernels need to be able to run on architectures with
incoherent caches.  Naive implementation of synchronization on such
architectures requires extensive cache flushing (e.g. flush+invalidate
everything on every spin lock operation, flush on every unlock!) and
is a performance problem.

Instead, many of these systems will have access to separate "coherent"
(usually uncached) and "incoherent" regions of memory.  Where this is
available, place all writable data sections by default into the
coherent region.  An "__incoherent" attribute flag is defined for data
regions that are known to be CPU-local and which should use the cache.
By default, this is used for stack memory.

Stack memory will be incoherent by default, as by definition it is
local to its current thread.  This requires special cache management
on context switch, so an arch API has been added for that.

Also, when enabled, add assertions to strategic places to ensure that
shared kernel data is indeed coherent.  We check thread objects, the
_kernel struct, waitq's, timeouts and spinlocks.  In practice almost
all kernel synchronization is built on top of these structures, and
any shared data structs will contain at least one of them.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-10-21 06:38:53 -04:00
Andrew Boie
f5a7e1a108 kernel: handle thread self-aborts on idle thread
Fixes races where threads on another CPU are joining the
exiting thread, since it could still be running when
the joiners wake up on a different CPU.

Fixes problems where the thread object is still being
used by the kernel when the fn_abort() function is called,
preventing the thread object from being recycled or
freed back to a slab pool.

Fixes a race where a thread is aborted from one CPU while
it self-aborts on another CPU, that was currently worked
around with a busy-wait.

Precedent for doing this comes from FreeRTOS, which also
performs final thread cleanup in the idle thread.

Some logic in z_thread_single_abort() rearranged such that
when we release sched_spinlock, the thread object pointer
is never dereferenced by the kernel again; join waiters
or fn_abort() logic may free it immediately.

An assertion added to z_thread_single_abort() to ensure
it never gets called with thread == _current outside of an ISR.

Some logic has been added to ensure z_thread_single_abort()
tasks don't run more than once.

Fixes: #26486
Related to: #23063 #23062

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-09-30 14:11:59 -04:00
Carles Cufi
28cb9dab64 kernel: Deprecate CONFIG_MULTITHREADING
Deprecate the Kconfig option for the time being. Unless a contributor
volunteers to take over the work to maintain the option, it will be
removed after 2 releases.

Relates to #27415.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2020-09-23 15:50:32 -05:00
Daniel Leung
b5c2ff9397 kernel: add kconfig CONFIG_KERNEL_MEM_POOL
Adds a kconfig CONFIG_KERNEL_MEM_POOL to decide whether
kernel memory pool related code is compiled. This option
can be disabled to shrink code size. If k_heap is not
being used at all, kheap.c will also not be compiled,
resulting in further smaller code size.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-09-19 05:49:13 -04:00
Anas Nashif
6e27478c3d benchmarking: remove execution benchmarking code
This code had one purpose only, feed timing information into a test and
was not used by anything else. The custom trace points unfortunatly were
not accurate and this test was delivering informatin that conflicted
with other tests we have due to placement of such trace points in the
architecture and kernel code.

For such measurements we are planning to use the tracing functionality
in a special mode that would be used for metrics without polluting the
architecture and kernel code with additional tracing and timing code.

Furthermore, much of the assembly code used had issues.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-09-05 13:28:38 -05:00
Daniel Leung
0ffcfa9633 timing: introduce timing functions as a generic feature
Add timing functions and APIs.  This is now used with some of the tests
we have for performance and metrics and will be used whereever timing
informations are needed, for example for tracing, profiling and other
operations where timing info is critical.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-09-05 13:28:38 -05:00
Andrew Boie
00f71b0d63 kernel: add CONFIG_ARCH_MEM_DOMAIN_SYNCHRONOUS_API
Saves us a few bytes of program text on arches that don't need
these implemented, currently all uniprocessor MPU-based systems.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-08-26 20:35:02 -04:00
Anas Nashif
3b75a08e3f Kconfig: move power management Kconfig into subsys/power
Consolidate all PM related Kconfigs in one place.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-08-24 10:24:30 +02:00
Andrew Boie
bc35b0b780 userspace: add optional member to memory domains
Some systems may need to associate arch-specific data to
a memory domain. Add a Kconfig and `arch` field for this,
and a new arch API to initialize it.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-08-20 13:58:54 -04:00
Andrew Boie
b0c155f3ca kernel: overhaul stack specification
The core kernel computes the initial stack pointer
for a thread, properly aligning it and subtracting out
any random offsets or thread-local storage areas.
arch_new_thread() no longer needs to make any calculations,
an initial stack frame may be placed at the bounds of
the new 'stack_ptr' parameter passed in. This parameter
replaces 'stack_size'.

thread->stack_info is now set before arch_new_thread()
is invoked, z_new_thread_init() has been removed.
The values populated may need to be adjusted on arches
which carve-out MPU guard space from the actual stack
buffer.

thread->stack_info now has a new member 'delta' which
indicates any offset applied for TLS or random offset.
It's used so the calculations don't need to be repeated
if the thread later drops to user mode.

CONFIG_INIT_STACKS logic is now performed inside
z_setup_new_thread(), before arch_new_thread() is called.

thread->stack_info is now defined as the canonical
user-accessible area within the stack object, including
random offsets and TLS. It will never include any
carved-out memory for MPU guards and must be updated at
runtime if guards are removed.

Available stack space is now optimized. Some arches may
need to significantly round up the buffer size to account
for page-level granularity or MPU power-of-two requirements.
This space is now accounted for and used by virtue of
the Z_THREAD_STACK_SIZE_ADJUST() call in z_setup_new_thread.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-07-30 21:11:14 -04:00
David Leach
db36b146e1 kernel: Remove supported size comments from HEAP_MEM_POOL_SIZE
The mempool implementation doesn't require specific sizes and can
support arbitrary sizes up to the limit of available memory. The
Kconfig documentation on this configuration was confusing user.

Fixes #20418

Signed-off-by: David Leach <david.leach@nxp.com>
2020-07-24 10:19:47 +02:00
Anas Nashif
568211738d doc: replace lifo/fifo with LIFO/FIFO
Replace all occurances of lifo/fifo with LIFO/FIFO to be consistent.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-07-15 14:01:33 -04:00
Enjia Mai
7ac40aabc0 tests: adding test cases for arch-dependent SMP function
Add one another test case for testing both arch_curr_cpu() and
arch_sched_ipi() architecture layer interface.

Signed-off-by: Enjia Mai <enjiax.mai@intel.com>
2020-07-02 08:42:53 -04:00
Anas Nashif
2c5d40437b kernel: logging: convert K_DEBUG to LOG_DBG
Move K_DEBUG to use LOG_DBG instead of plain printk.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-06-25 16:12:36 -05:00
Kumar Gala
a1b77fd589 zephyr: replace zephyr integer types with C99 types
git grep -l 'u\(8\|16\|32\|64\)_t' | \
		xargs sed -i "s/u\(8\|16\|32\|64\)_t/uint\1_t/g"
	git grep -l 's\(8\|16\|32\|64\)_t' | \
		xargs sed -i "s/s\(8\|16\|32\|64\)_t/int\1_t/g"

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-06-08 08:23:57 -05:00