There's no need for a system call for this; futexes live in
user memory and the initialization bit is ignored.
It's sufficient to just do an atomic_set().
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
This is an oddball API. It's untested. In fact testing its proper
behavior requires very elaborate automation (you need a device outside
the Zephyr hardware to measure real world time, and a mechanism for
getting the device into and out of idle without using the timer
driver). And this makes for needless difficulty managing code
coverage metrics.
It was always just a hint anyway. Mark the old API deprecated and
replace it with a kconfig tunable. The effect of that is just to
change the timeout value passed to the timer driver, where we can
manage code coverage metrics more easily (only one driver cares to
actually support this feature anyway).
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
move tracing.h to debug/tracing.h and
create a shim for backward-compatibility.
No functional changes to the headers.
A warning in the shim can be controlled with CONFIG_COMPAT_INCLUDES.
Related to #16539
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
The first word is used as a pointer, meaning it is 64 bits on 64-bit
systems. To reserve it, it has to be either a pointer, a long, or an
intptr_t. Not an int nor an u32_t.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Folks found the use of @rststar/@endrststar non-intuitive (wanted to use
@rststart). The "star" was there indicating the doxygen comment lines
had a leading asterisk that needed to be stripped, but since our
commenting convention is to use the leading asterisk on continuation
lines, the leading asterisk is always there. So, change the doxygen
alias to the more expected @rst/@endrst.
Signed-off-by: David B. Kinder <david.b.kinder@intel.com>
A k_futex is a lightweight mutual exclusion primitive designed
to minimize kernel involvement. Uncontended operation relies
only on atomic access to shared memory. k_futex structure lives
in application memory. And when using futexes, the majority of
the synchronization operations are performed in user mode. A
user-mode thread employs the futex wait system call only when
it is likely that the program has to block for a longer time
until the condition becomes true. When the condition comes true,
futex wake operation will be used to wake up one or more threads
waiting on that futex.
This patch implements two futex operations: k_futex_wait and
k_futex_wake. For k_futex_wait, the comparison with the expected
value, and starting to sleep are performed atomically to prevent
lost wake-ups. If different context changed futex's value after
the calling use-mode thread decided to block himself based on
the old value, the comparison will help observing the value
change and will not start to sleep. And for k_futex_wake, it
will wake at most num_waiters of the waiters that are sleeping
on that futex. But no guarantees are made on which threads are
woken, that means scheduling priority is not taken into
consideration.
Fixes: #14493.
Signed-off-by: Wentong Wu <wentong.wu@intel.com>
In z_sys_mem_pool_block_alloc() the size of the first level block
allocation is rounded up to the next 4-bite boundary. This means one
or more of the trailing blocks could overlap the free block bitmap.
Let's consider this code from kernel.h:
#define K_MEM_POOL_DEFINE(name, minsz, maxsz, nmax, align) \
char __aligned(align) _mpool_buf_##name[_ALIGN4(maxsz * nmax) \
+ _MPOOL_BITS_SIZE(maxsz, minsz, nmax)]; \
The static pool allocation rounds up the product of maxsz and nmax not
size of individual blocks. If we have, say maxsz = 10 and nmax = 20,
the result of _ALIGN4(10 * 20) is 200. That's the offset at which the
free block bitmap will be located.
However, because z_sys_mem_pool_block_alloc() does this:
lsizes[0] = _ALIGN4(p->max_sz);
Individual level 0 blocks will have a size of 12 not 10. That means
the 17th block will extend up to offset 204, 18th block up to 216, 19th
block to 228, and 20th block to 240. So 4 out of the 20 blocks are
overflowing the static pool area and 3 of them are even located
completely outside of it.
In this example, we have only 20 blocks that can't be split so there is
no extra free block bitmap allocation beyond the bitmap embedded in the
sys_mem_pool_lvl structure. This means that memory corruption will
happen in whatever data is located alongside the _mpool_buf_##name
array. But even with, say, 40 blocks, or larger blocks, the extra bitmap
size would be small compared to the extent of the overflow, and it would
get corrupted too of course.
And the data corruption will happen even without allocating any memory
since z_sys_mem_pool_base_init() stores free_list pointer nodes into
those blocks, which in turn may get corrupted if that other data is
later modified instead.
Fixing this issue is simple: rounding on the static pool allocation is
"misparenthesized". Let's turn
_ALIGN4(maxsz * nmax)
into
_ALIGN4(maxsz) * nmax
But that's not sufficient.
In z_sys_mem_pool_base_init() we have:
size_t buflen = p->n_max * p->max_sz, sz = p->max_sz;
u32_t *bits = (u32_t *)((u8_t *)p->buf + buflen);
Considering the same parameters as above, here we're locating the extra
free block bitmap at offset `buflen` which is 20 * 10 = 200, again below
the reach of the last 4 memory blocks. If the number of blocks gets past
the size of the embedded bitmap, it will overlap memory blocks.
Also, the block_ptr() call used here to initialize the free block linked
list uses unrounded p->max_sz, meaning that it is initially not locating
dlist nodes within the same block boundaries as what is expected from
z_sys_mem_pool_block_alloc(). This opens the possibility for allocated
adjacent blocks to overwrite dlist nodes, leading to random crashes in
the future.
So a complete fix must round up p->max_sz here too.
Given that runtime usage of max_sz should always be rounded up, it is
then preferable to round it up once at compile time instead and avoid
further mistakes of that sort. The existing _ALIGN4() usage on p->max_sz
at run time are then redundant.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
There is no point allowing smaller alignments. And on 64-bit systems the
minimum becomes 8 rather than 4, so let's adjust things automatically.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Found a few annoying typos and figured I better run script and
fix anything it can find, here are the results...
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
The k_stack data type cannot be u32_t on a 64-bit system as it is
often used to store pointers. Let's define a dedicated type for stack
data values, namely stack_data_t, which can be adjusted accordingly.
For now it is defined to uintptr_t which is the integer type large
enough to hold a pointer, meaning it is equivalent to u32_t on 32-bit
systems and u64_t on 64-bit systems.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
We introduce k_float_disable() system call, to allow threads to
disable floating point context preservation. The system call is
to be used in FP Sharing Registers mode (CONFIG_FP_SHARING=y).
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
Enable generation of doxygen documentation for kernel APIs that are
behind Kconfig options and add a note about the option needed to enable
the APIs.
Enable both CONFIG_SCHED_CPU_MASK and CONFIG_SCHED_DEADLINE in doxygen
config file.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Doxygen comments can include doxygen-specific markup tags. If other
markup tags are used (e.g., restructuredText) we need to indicate that
in the doxygen comments (via @rststar/@endrststar tags).
Signed-off-by: David B. Kinder <david.b.kinder@intel.com>
This convenience macro wraps Z_DECL_ALIGN() and __in_section() to
simplify static definitions of structure instances gathered in dedicated
sections. Most of the time those go together, and the section name is
already closely related to the struct type, so abstracting things behind
a simpler interface reduces probability of mistakes and makes the code
clearer. A few input section names have been adjusted accordingly.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The alignment fix on struct device definitions should be done to all
such linker list tricks. Let's abstract the declaration plus alignment
with a macro and apply it to all concerned cases.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The fifo/lifo API is implemented on top of the queue API with macros
that blindly force a cast to struct k_queue. Providing a reference to
the _queue member from the k_fifo structure is much cleaner as it let
the compiler perform pointer type checking. Generated code is identical.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
Architectures that lack implementations of synchronous traps (via
Z_ARCH_EXCEPT()) end up using a z_except_reason() implementation that
doesn't actually trap at all. It just invokes
z_NanoFatalErrorHandler() in the current thread context.
That has two problems:
First, it was just blindly assuming that the error handling invoked
would abort the current thread, swap away, and never return. But that
can be application code in z_SysFatalErrorHandler that we can't
control.
Second, it was too broad with this assumption and stuff a
CODE_UNREACHABLE hint in for the compiler. But in fact
z_except_reason() may be invoked in interrupt context (for example the
stackprot check) where it may NOT swap away and WILL return
synchronously from the call. This doesn't seem to have caused a
miscompilation in production code, but it made a total voodoo hash out
of my debugging around this macro for an hour or so until I figured
out why my logging was being optimized out.
Do the abort unconditionally instead of relying on the app, and remove
the incorrect compiler hint.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Add k_usleep() API, analogous to k_sleep(), excepting that the argument
is in microseconds rather than milliseconds.
Signed-off-by: Charles E. Youse <charles.youse@intel.com>
k_poll_signal_raise() returns an error code to indicate that the raise
was too late to notify an expiring poll. Make clear that this does not
mean that the signal was lost: a subsequent poll will find it and expire
immediately.
Signed-off-by: Peter A. Bigot <pab@pabigot.com>
The struct _caller_saved is not used. Most architectures put
automatically the registers onto stack, in others architectures the
exception code does it.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
This macro is slated for complete removal, as it's not possible
on arches with an MPU stack guard to know the true buffer bounds
without also knowing the runtime state of its associated thread.
As removing this completely would be invasive to where we are
in the 1.14 release, demote to a private kernel Z_ API instead.
The current way that the macro is being used internally will
not cause any undue harm, we just don't want any external code
depending on it.
The final work to remove this (and overhaul stack specification in
general) will take place in 1.15 in the context of #14269Fixes: #14766
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Rename reserved function names in arch/ subdirectory. The Python
script gen_priv_stacks.py was updated to follow the 'z_' prefix
naming.
Signed-off-by: Patrik Flykt <patrik.flykt@intel.com>
Rename reserved function names in drivers/ subdirectory. Update
function macros concatenatenating function names with '##'. As
there is a conflict between the existing gpio_sch_manage_callback()
and _gpio_sch_manage_callback() names, leave the latter unmodified.
Signed-off-by: Patrik Flykt <patrik.flykt@intel.com>
This is used to have each arch canonically state how much
room in the stack object is reserved for non-thread use.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
This is a trivial change to satisfy C++, which requires that designated
initializers appear in the same order as the members they initialize.
Fixes: #14540
Signed-off-by: Charles E. Youse <charles.youse@intel.com>
There was a detected user error in the code where racing insertions of
k_delayed_work items into different queues would be detected and
flagged as an error (honestly I don't see much value there -- Zephyr
doesn't as a general rule protect against errors like this, and
work_q's are inherently kernel things that don't require
userspace-style checking).
This got broken with spinlockification, where each work_q object got
its own lock, so the single lock wouldn't protect against the other
insert function any more. As it happens, that was needless. The core
synchronization on a work_q is in the internal k_queue object anyway
-- the lock in this file was only ever used for (very fast,
noncontending) delayed work insertion. So go back to a global lock to
preserve the original behavior.
Fixes#14104
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Update reserved function names starting with one underscore, replacing
them as follows:
'_k_' with 'z_'
'_K_' with 'Z_'
'_handler_' with 'z_handl_'
'_Cstart' with 'z_cstart'
'_Swap' with 'z_swap'
This renaming is done on both global and those static function names
in kernel/include and include/. Other static function names in kernel/
are renamed by removing the leading underscore. Other function names
not starting with any prefix listed above are renamed starting with
a 'z_' or 'Z_' prefix.
Function names starting with two or three leading underscores are not
automatcally renamed since these names will collide with the variants
with two or three leading underscores.
Various generator scripts have also been updated as well as perf,
linker and usb files. These are
drivers/serial/uart_handlers.c
include/linker/kobject-text.ld
kernel/include/syscall_handler.h
scripts/gen_kobject_list.py
scripts/gen_syscall_header.py
Signed-off-by: Patrik Flykt <patrik.flykt@intel.com>
You can't cancel what hasn't been submitted. Clarification added
following minor bike shed in github. Fixes#14105
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Nothing in the code actually returns -EINPROGRESS, and in the case of
k_work_init() I don't see how that can even be done in a reliable way.
Don't claim we do what we don't. Fixes#14109.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
In some circumstances (e.g., a tickless kernel), k_timer_remaining_get()
would not account for time passed that didn't involve clock interrupts.
This adds a simple fix for that, and adds a test case. In addition, the
return value of k_timer_remaining_get() is clamped at 0 in the case of
overdue timers and the API description is adjusted to reflect this.
Fixes: #13353
Signed-off-by: Charles E. Youse <charles.youse@intel.com>
One spinlock per pipe object. Also removed some vestigial locking
around _ready_thread(). That call is internally synchronized now.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Straightforward port. Each struct k_queue object gets a spinlock to
control obvious data ownership.
Note that this port actually discovered a preexisting bug: the -ENOMEM
case in queue_insert() was failing to release the lock. But because
the tests that hit that path didn't rely on other threads being
scheduled, they ran to successful completion even with interrupts
disabled. The spinlock API detects that as a recursive lock when
asserts are enabled.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Each work_q object gets a separate spinlock to synchronize access
instead of the global lock. Note that there was a recursive lock
condition in k_delayed_work_cancel(), so that's been split out into an
internal unlocked version and the API entry point that wraps it with a
lock.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
This was never a long-term solution, more of a gross hack
to get test cases working until we could figure out a good
end-to-end solution for memory domains that generated
appropriate linker sections. Now that we have this with
the app shared memory feature, and have converted all tests
to remove it, delete this feature.
To date all userspace APIs have been tagged as 'experimental'
which sidesteps deprecation policies.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
While k_uptime_get() and k_uptime_get32() return time in
milliseconds, they don't need to have millisecond resolution.
Resolution with default Zephyr settings in 10ms.
Signed-off-by: Paul Sokolovsky <paul.sokolovsky@linaro.org>
This adds a simple implementation of SMP CPU affinity to Zephyr. The
API is simple and doesn't try to invent abstractions like "cpu sets".
Each thread has an enable/disable flag associated with each CPU in the
system, and the bits can be turned on and off (for threads that are
not currently runnable, of course) using an easy three-function API.
Because the implementation picked requires enumerating runnable
threads in priority order looking for one that match the current CPU,
this is not a good fit for the SCALABLE or MULTIQ scheduler backends,
so it currently can be enabled only for SCHED_DUMB (which is the
default anyway). Fancier algorithms do exist, but even the best of
them scale as O(N_CPUS), so aren't quite constant time and often
require significant memory overhead to keep separate lists for
different cpus/sets.
The intended use here is for apps that want to "pin" threads to
specific CPUs for latency control, or conversely to prevent certain
threads from taking time on specific CPUs to leave them free for fast
response.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Added cpu_idle APIs to a doxygen group, otherwise there were missing
from the project documentation.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Whether a timeout is linked into the timeout queue can be determined
from the corresponding sys_dnode_t linked state. This removes the need
to use a special flag value in dticks to determine that the timeout is
inactive.
Update _abort_timeout to return an error code, rather than the flag
value, when the timeout to be aborted was not active.
Remove the _INACTIVE flag value, and replace its external uses with an
internal API function that checks whether a timeout is inactive.
Signed-off-by: Peter A. Bigot <pab@pabigot.com>
Although sys_dnode_t and sys_dlist_t are aliases, their roles are
different and they appear in different positions in dlist API calls.
Signed-off-by: Peter A. Bigot <pab@pabigot.com>
