Commit graph

1899 commits

Author SHA1 Message Date
Christoph Winklhofer
63d33e631c json: support parsing and serializing of strings with char arrays
Support parsing and serializing of struct fields that are defined as a
char array.

Use the token JSON_TOK_STRING_BUF to parse and serialize a string for a
char array, for example:

struct foo {
  const char *str;
  char str_buf[30];
};

struct json_obj_descr foo_descr[] = {
  JSON_OBJ_DESCR_PRIM(struct foo, str, JSON_TOK_STRING),
  JSON_OBJ_DESCR_PRIM(struct foo, str_buf, JSON_TOK_STRING_BUF),
};

The struct 'json_obj_descr' now has an additional union member 'field'
to store the size of the struct field, which is used with the token
'JSON_TOK_STRING_BUF' to determine the element size.

Fixes: #65200
Signed-off-by: Christoph Winklhofer <cj.winklhofer@gmail.com>
2025-04-11 06:32:50 +02:00
Christoph Winklhofer
fc37c02eb1 json: improve parsing and serializing of 'float' and 'double'
Up to now, the handling of type float was offloaded to the users of the
JSON utility, with the token JSON_TOK_FLOAT.

Improve handling of floating point types and support the types 'float'
and 'double' in a built-in way so that they can be directly parsed to
and serialized from variables (of type float or double).

The types are serialized in the shortest representation, either as a
decimal number or in scientific notation:
  * float (with JSON_TOK_FLOAT_FP): encoded with maximal 9 digits
  * double (with JSON_TOK_DOUBLE_FP): encoded with maximal 16 digits
  * NaN, Infinity, -Infinity: encoded and decoded as:
    {"nan_val":NaN,"inf_pos":Infinity,"inf_neg":-Infinity}

Enable the floating point functionality with the Kconfig option:
  JSON_LIBRARY_FP_SUPPORT=y

It requires a libc implementation with support for floating point
functions: strtof(), strtod(), isnan() and isinf().

Fixes: #59412
Signed-off-by: Christoph Winklhofer <cj.winklhofer@gmail.com>
2025-04-09 22:05:14 +02:00
Christoph Winklhofer
c7b7ec2faf json: Support floating point values NaN and Infinity for numbers
Add support to decode the special floating point values NaN, Infinity
and -Infinity. For example:
  {"nan_val":NaN,"inf_pos":Infinity,"inf_neg":-Infinity}

Note that this commit is a preparation for the built-in support of
floating point values and these are only accepted when compiled with
the flag -DCONFIG_JSON_LIBRARY_FP_SUPPORT.

Signed-off-by: Christoph Winklhofer <cj.winklhofer@gmail.com>
2025-04-09 22:05:14 +02:00
Christoph Winklhofer
f07f2da8dc json: support scientific notation for numbers
Add support to decode floating point numbers in scientific notation,
e.g. 3.40282347e+38. Only the lower-case specifier 'e' is allowed.

Signed-off-by: Christoph Winklhofer <cj.winklhofer@gmail.com>
2025-04-09 22:05:14 +02:00
Christoph Winklhofer
061e345972 json: Fix calculation of object size
The calculation of the object size may be incorrect when the size of
a field is smaller than the struct alignment. When such a struct is
used in an array field, the decoded object contains wrong values.

The alignment influences the object size. For example the following
struct has a calculated object size of 8 bytes, however due to
alignment the real size of the struct is 12 bytes:

struct test_bool {
  bool b1; /* offset 0, size 1 */
           /* 3-byte padding */
  int i1;  /* offset 4, size 4 */
  bool b2; /* offset 8, size 1 */
           /* 3-byte padding */
};

This commit changes the object size calculation and computes the size
with the offset and size of the last field in the struct (rounded up
by the struct alignment).

Fixes: #85121
Signed-off-by: Christoph Winklhofer <cj.winklhofer@gmail.com>
2025-04-08 11:44:51 +02:00
Nicolas Pitre
11021cdd4f kernel: sys_heap: decouple realloc from aligned_realloc
When sys_heap_realloc() is expressed in terms of sys_heap_aligned_realloc()
it invokes a longer aligned allocation code path with an extra runtime
overhead even though no alignment is necessary.

Let's reference and invoke the aligned allocation code path only when an
actual aligned allocation is requested. This opens the possibility for
the linker to garbage-collect the aligning code otherwise.

Improve realloc documentation while at it.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2025-04-01 22:13:04 +02:00
Nicolas Pitre
457fa60bfc kernel: mempool: decouple simple alloc from aligned_alloc
When k_malloc() is expressed in terms of k_aligned_alloc() it invokes a
longer aligned allocation code path with an extra runtime overhead even
though no alignment is necessary.

Let's reference and invoke the aligned allocation code path only when an
actual aligned allocation is requested. This opens the possibility for
the linker to garbage-collect the aligning code otherwise.

Also bypass k_heap_malloc() and friends given they're invoked with
K_NO_WAIT. Go directly to sys_heap_*() instead to cut some more unneeded
overhead.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2025-04-01 22:13:04 +02:00
Nicolas Pitre
9da06456f2 kernel: kheap: decouple simple alloc from aligned_alloc
When k_heap_alloc() is expressed in terms of k_heap_aligned_alloc()
it invokes a longer aligned allocation code path with an extra runtime
overhead even though no alignment is necessary.

Let's reference and invoke the aligned allocation code path only when an
actual aligned allocation is requested. This opens the possibility for
the linker to garbage-collect the aligning code otherwise.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2025-04-01 22:13:04 +02:00
Andre Heinemans
ad65b81024 lib: open-amp: optional copy of rsc_table to memory region
Option OPENAMP_COPY_RSC_TABLE is added for cases where the placement of the
.resource_table section is not controlled by the remote. This option lets
zephyr do the copying of the .resource_table section to a specified memory
region.

Signed-off-by: Andre Heinemans <andre.heinemans@nxp.com>
2025-03-24 12:17:53 +01:00
Henrik Brix Andersen
4d01efbf90 lib: net_buf: remove deprecated net_buf_put() and net_buf_get() functions
Remove the deprecated net_buf_put() and net_buf_get() functions.

Signed-off-by: Henrik Brix Andersen <hebad@vestas.com>
2025-03-12 19:04:19 +01:00
Chris Friedt
7623a18c2e posix: remove deprecated kconfig options
Remove deprecated Kconfig options. These were given an extra
release cycle of soak time, but generally were OK to remove
after Zephyr v4.0.0 was released.

Signed-off-by: Chris Friedt <cfriedt@tenstorrent.com>
2025-03-12 19:03:52 +01:00
Chris Friedt
4c28582224 posix: threads: use non-deprecated kconfig
MAX_PTHREAD_COUNT was deprecated in favour of
POSIX_THREAD_THREADS_MAX prior to v3.7.0.

Use CONFIG_POSIX_THREAD_THREADS_MAX going forward.

Signed-off-by: Chris Friedt <cfriedt@tenstorrent.com>
2025-03-12 19:03:52 +01:00
Chris Friedt
33fb827603 posix: key: do not use deprecated kconfig
Use CONFIG_POSIX_THREAD_KEYS_MAX instead of
CONFIG_MAX_PTHREAD_KEY_COUNT.

Signed-off-by: Chris Friedt <cfriedt@tenstorrent.com>
2025-03-12 19:03:52 +01:00
Alberto Escolar Piedras
ca5b7506e8 lib/os/fdtable: Remove ifdef on NATIVE_POSIX
This board does not exist anymore, so this is just dead code at this
point.

Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
2025-03-12 02:27:36 +01:00
Nicolas Pitre
e1eead3925 ring_buffer: shrink size of struct ring_buf
Make struct ring_buf 12 bytes smaller by default. This comes with a 32KB
buffer size limit which covers almost all cases. The previous limit of
2GB can be restored with CONFIG_RING_BUFFER_LARGE=y.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2025-03-11 08:59:05 +01:00
Nicolas Pitre
b97f11a752 ring_buffer: simplify code some more
Some variable renaming made the code clearer, which in turn allowed for
minor optimizations.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2025-03-11 08:59:05 +01:00
Nicolas Pitre
fa73b8b061 ring_buffer: update Kconfig help text
Make it more informative and useful.

Buffer sizes that are a power of 2 didn't provide any advantage since
commit 099850e916 ("ring_buffer: the great simplification").

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2025-03-11 08:59:05 +01:00
Helmut Lord
f31b4a67be lib: utils: add consistent overhead byte stuffing
Implementation uses netbufs.

Signed-off-by: Helmut Lord <kellyhlord@gmail.com>
2025-03-11 05:36:49 +01:00
Jari Tervonen
3b1e60da90 lib: posix: Fix NULL pointer to strcmp in posix mqueue.
When a queue has been unlinked but not removed the name pointer is set to
NULL. The queue is still in the list when adding new queues and the name
pointer is passed to strcmp.

Signed-off-by: Jari Tervonen <jari.tervonen@nordicsemi.no>
2025-03-08 03:37:48 +01:00
Benjamin Cabé
0c2c0c4328 libc: add comment to empty sys/cdefs.h file
This documents the reason to exist for this otherwise odd-looking empty
header file.

Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
2025-02-21 15:13:20 +00:00
Chris Friedt
bca2ef672e posix: mem: remove imply MMU from POSIX options
With MMU being converted to a non-user-configurable option, we can
rely on that exclusively to know whether arch_mem_map() and
arch_mem_unmap() are available and we can remove the workarounds
in POSIX at the Kconfig level.

Signed-off-by: Chris Friedt <cfriedt@tenstorrent.com>
2025-02-19 04:59:33 +01:00
Damian Krolik
73802412a8 lib: os: mpsc_pbuf: take spinlock in utilization getters
Two public functions do not take the spinlock even though
they access mutable, non-atomic members of the buffer:
- mpsc_pbuf_get_utilization()
- mpsc_pbuf_get_max_utilization()

Take the spinlock to avoid possible data races.

Signed-off-by: Damian Krolik <damian.krolik@nordicsemi.no>
2025-02-18 13:30:29 +01:00
Luis Ubieda
ec45b29ea3 p4wq: Add Kconfig to perform early init on threads
In order to make them functional for devices during init. Default
behavior is to keep late initialization, as before.

Signed-off-by: Luis Ubieda <luisf@croxel.com>
2025-02-18 05:32:58 +01:00
Robin Kastberg
9ab06ec667 toolchain: iar: Add experimental IAR support
This adds experimental support for the IAR toolchain.

Signed-off-by: Robin Kastberg <robin.kastberg@iar.com>
2025-02-14 19:12:44 +00:00
Keith Packard
dc0fd3af17 picolibc: Use common abort(), call from assert when !__ASSERT_ON
Switch to the common abort implementation so that we can use it
from the assert hooks to avoid undefined behavior.

Closes: 84824

Signed-off-by: Keith Packard <keithp@keithp.com>
2025-02-14 10:42:16 +01:00
Florian Weber
093b29fdb5 lib: os: p4wq: add done handler
Add an optional handler to the p4wq to give the submitting code
(e.g. rtio workq) a possibility execute code after the work was
succesfully executed.

Signed-off-by: Florian Weber <Florian.Weber@live.de>
2025-02-12 16:03:17 +01:00
Yuval Peress
a455440570 cpp: Add c++ version number
Several downstream libraries need to specify that they work with c++ 17+
or 20+ and there's no easy way to do this without constantly updating
the Kconfig file and supporting different Kconfig versions for different
Zephyr versions (since I can't add a dependency on a new c++ standard
and have it backwards compatible.

Signed-off-by: Yuval Peress <peress@google.com>
2025-02-10 15:57:26 +00:00
Marek Matej
65d2139dc6 lib: libc: malloc.c: fix calculation of sys heap
Change the condition so that ESP32 SoCs are using `_heap_sentry` to
get actual run-time heap size.

Signed-off-by: Marek Matej <marek.matej@espressif.com>
2025-02-05 17:49:54 +01:00
Chris Friedt
f4259c384f posix: options: add xsi realtime option group
Add a POSIX Option Group called XSI_REALTIME (with Kconfig
option CONFIG_XSI_REALTIME).

When XSI_REALTIME is selected (or when required POSIX Options
are enabled), define _XOPEN_REALTIME to be something other
than -1 (_XOPEN_VERSION seemed appropriate).

Signed-off-by: Chris Friedt <cfriedt@tenstorrent.com>
2025-02-03 09:05:09 +01:00
Chris Friedt
d08d63676b posix: options: split kconfig.xsi into separate option groups
Since XSI is composed of several distinct POSIX Option Groups
split Kconfig.xsi into separate files - one for each Option
Group.

Signed-off-by: Chris Friedt <cfriedt@tenstorrent.com>
2025-02-03 09:05:09 +01:00
Chris Friedt
0ac931b386 posix: options: profiles: uncomment options that should be selected
For the REALTIME_MINIMAL Application Environment Profile, uncomment
the POSIX_MEMLOCK, POSIX_MEMLOCK_RANGE, POSIX_MEMORY_PROTECTION,
POSIX_MAPPED_FILES, and POSIX_SHARED_MEMORY_OBJECTS options.

These should have been uncommented back when Kconfig options for
those features were added, so this can probably be called a
Kconfig bug.

Signed-off-by: Chris Friedt <cfriedt@tenstorrent.com>
2025-02-03 09:05:09 +01:00
Chris Friedt
8ab6e29cf1 posix: shm: ensure addend is less than INTPTR_MAX
Although it's quite unlikely that we will see shared memory
objects in practice that are greater than 2GiB, there is a
possibility of integer overflow when comparing intptr_t and
size_t, since the former is signed and the latter is unsigned.

Explicitly check to ensure that the addend is less than
INTPTR_MAX before subtraction.

This fixes CID 487734

Signed-off-by: Chris Friedt <cfriedt@tenstorrent.com>
2025-01-30 20:27:59 +01:00
Chris Friedt
f81293d0dc posix: semaphore: explicitly ignore return of k_mutex_unlock()
The nsem_list_unlock() static inline function looks as though it
was designed to always ignore the return value of
k_mutex_unlock(), presumably for performance reasons.

A quick audit of the code looks as though the call should
always succeed.

Prepend (void) to the call to avoid the coverity defect in the
future.

CID 444386

Signed-off-by: Chris Friedt <cfriedt@tenstorrent.com>
2025-01-30 20:27:48 +01:00
Krzysztof Chruściński
3e52109590 lib: os: mpsc_pbuf: Fix infinite loop during allocation
It was possible that allocation was continuously attempting to find
space. It could happen if allocation interrupted consuming a packet
(which is marked as busy) and there is not enough space to allocate
requested packet but there would be if busy packet was freed.
Algorithm in that case was continuously going through the buffer
in search for packets that can be dropped to find space for the
new packet.

Signed-off-by: Krzysztof Chruściński <krzysztof.chruscinski@nordicsemi.no>
2025-01-29 17:55:56 +01:00
Pisit Sawangvonganan
e3a4a16594 lib: fix typo in multiple directories
Utilize a code spell-checking tool to scan for and correct spelling errors
in various files within the `lib` directory.

Signed-off-by: Pisit Sawangvonganan <pisit@ndrsolution.com>
2025-01-28 00:06:18 +01:00
Joel Holdsworth
1b399d78fc picolibc: Replace hard coded -fno-lto flag
GCC versions before 4.5 do not have the -fno-lto flag and give an error if
it is used. The flag may also be unavailable on various other non-GCC
compilers.

Previously, the picolibc CMakeList.txt script assumed that the flag was
available, and hard-coded it as an addition to the compiler flags.

This patch improves compatibility by making use of the already existing
"prohibit_lto" CMake target compiler property, which is set to the
appropriate flag to disable LTO support, or unset if the compiler lacks
such an option.

Signed-off-by: Joel Holdsworth <jholdsworth@nvidia.com>
2025-01-27 13:24:52 +01:00
Nicolas Pitre
3075a7d906 ring_buffer: factorize almost identical code
Factorize almost identical code. May even improve performance due to
CPU cache locality.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2025-01-25 01:40:03 +01:00
Nicolas Pitre
83279d1524 ring_buffer: optimize the partial buffer loop
It is more efficient to break early when size of claimed area becomes zero.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2025-01-25 01:40:03 +01:00
Nicolas Pitre
67978f8ffc ring_buffer: delete redundant code
No need to clamp requested size as ring_buf_get_claim() does it already.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2025-01-25 01:40:03 +01:00
Geoffrey Hunter
fa795f4912 lib: smf: Fix handled bug causing events to not propagate.
When using the SMF for a project discovered that events would sometimes
 not propagate to parent states correctly. Could not create a minimum
reproducable test case for this, but it was found that these changes fixed
the bug. This commit creates a new function to reset internal state,
which is called on entry to smf_set_initial() and smf_set_state().

Closes #81300.

Signed-off-by: Geoffrey Hunter <gbmhunter@gmail.com>
2025-01-23 19:24:08 +01:00
Ivan Pankratov
257d9d45ba kernel: sys_heap: stats: save heap pointers to an array during init
To request heap statistics, a pointer to a heap structure is required.
This is straightforward for a user-defined heap. However, such a pointer
is not known for heaps created by other components or libraries, like
libc. Therefore, it is not possible to calculate the total heap memory.

The proposed solution is to use an array of pointers, which is filled in
on every sys_heap_init() call. One can then iterate through it to sum up
the total memory allocated for all heaps.

The array size is configurable. The default array size is zero,
which means the feature is disabled. Any other integer greater then zero
defines the array size and enables the feature.

A list of pointers instead of an array could be another approach,
but it requeres a heap, which is not always available.

Signed-off-by: Ivan Pankratov <ivan.pankratov@silabs.com>
2025-01-23 16:37:33 +01:00
Håvard Reierstad
e13e893dfc net: buf: revert disallowing blocking in syswq
Reverts the change made in d4d53010f00cadbb4f89c6d41391937646fc1740
(The changes was moved to another file in a restructuring)

The commit incorrectly assumed that no blocking would be allowed in
the syswq. This has caused issues observed among others
in #77241 and #80167

Signed-off-by: Håvard Reierstad <haavard.reierstad@nordicsemi.no>
2025-01-23 10:14:59 +01:00
Chris Friedt
dba7598517 posix: options: shm: use truncation flag that has been added
Originally, when the POSIX_SHARED_MEMORY_OBJECTS option was
added, the O_TRUNC flag was not consistently available.

Now that it is consistently available, ensure it is used
within the shm_open() function.

Signed-off-by: Chris Friedt <cfriedt@tenstorrent.com>
2025-01-23 00:12:58 +01:00
Maochen Wang
f36e2e0664 lib: net_buf: support counting max used buf
Support counting the max used net_buf when CONFIG_NET_BUF_POOL_USAGE
is defined.

Signed-off-by: Maochen Wang <maochen.wang@nxp.com>
2025-01-20 11:16:18 +01:00
Chris Friedt
8609a05236 posix: options: mlock: refine imply for MMU and DEMAND_PAGING
POSIX mlock() and munlock() require an MMU as well as
DEMAND_PAGING.

Signed-off-by: Chris Friedt <cfriedt@tenstorrent.com>
2025-01-11 18:31:02 +01:00
Daniel Leung
f81add2f65 posix: pin init functions and data for demand paging
Boot time initialization functions and data used there must be
available at boot. With demand paging, these may not exist in
memory when they are being used, resulting in page faults.
So pin these functions and data in linker sections to make
sure they are in memory at boot time.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2025-01-11 04:36:38 +01:00
Nicolas Pitre
46aa6717ff Revert "arch: deprecate _current"
Mostly a revert of commit b1def7145f ("arch: deprecate `_current`").

This commit was part of PR #80716 whose initial purpose was about providing
an architecture specific optimization for _current. The actual deprecation
was sneaked in later on without proper discussion.

The Zephyr core always used _current before and that was fine. It is quite
prevalent as well and the alternative is proving rather verbose.
Furthermore, as a concept, the "current thread" is not something that is
necessarily architecture specific. Therefore the primary abstraction
should not carry the arch_ prefix.

Hence this revert.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2025-01-10 07:49:08 +01:00
Chris Friedt
d58a5ce580 posix: options: mmap: disable xtensa support due to linker issue
The Xtensa linker scripts seem to be injecting syntax errors when
MMU is enabled. Disable the implication in Kconfig.mem for Xtensa
until linker issues are resolved.

Signed-off-by: Chris Friedt <cfriedt@tenstorrent.com>
2025-01-09 15:50:26 +01:00
Chris Friedt
cee9166dee posix: options: mlockall: include toolchain header
Include the toolchain header because ARG_UNUSED() is
not defined.

Signed-off-by: Chris Friedt <cfriedt@tenstorrent.com>
2025-01-09 15:49:57 +01:00
Chris Friedt
0acea8d5e6 posix: options: mprotect: include toolchain header
Include zephyr/toolchain.h to get access to ARG_UNUSED().

Signed-off-by: Chris Friedt <cfriedt@tenstorrent.com>
2025-01-09 15:49:42 +01:00