Add noncallable printf with string used for ztest assert/assume
macros to ensure printf-like argument validation at compile time.
Signed-off-by: Krzysztof Chruściński <krzysztof.chruscinski@nordicsemi.no>
When CONFIG_ZTEST_FAIL_ON_ASSUME is set, a failed assumption will
cause the suite to fail, but the individual test will be marked as
SKIPPED. We should fail the test so it's clear what's going on.
Fixes#86611
Signed-off-by: Yuval Peress <peress@google.com>
if CONFIG_PM=y, board will enter low power,
which will cause problem for debugger.
So for ztest cases, we need enable this to avoid problem.
This used to apply to soc/platform level, now remove them.
Signed-off-by: Hake Huang <hake.huang@nxp.com>
test_main can call ztest_run_all multiple times. For example
tests/bluetooth/host/keys/bt_keys_get_addr/src/main.c does so. If the
first call skips some tests and then they run in the second call,
everything is fine. However if they passed or failed in the first run
and skipped in the second run, then the test output (json, xml, and
printed summary) show the test as skipped even though it ran.
The solution is to not clear the test stats.
__ztest_init_unit_test_result_for_suite was added in commit
996c8457d7 by nashif.
Signed-off-by: Jeremy Bettis <jbettis@google.com>
Ensure all pending log messages are processed by the log processing
thread when log_flush is called, blocking the caller until done.
Signed-off-by: Tom Burdick <thomas.burdick@intel.com>
Obviously, everyone knows that there are 8 bits per byte, so
there isn't a lot of magic happening, per se, but it's also
helpful to clearly denote where the magic number 8 is referring
to the number of bits in a byte.
Occasionally, 8 will refer to a field size or offset in a
structure, MMR, or word. Occasionally, the number 8 will refer
to the number of bytes in a 64-bit value (which should probably
be replaced with `sizeof(uint64_t)`).
For converting bits to bytes, or vice-versa, let's use
`BITS_PER_BYTE` for clarity (or other appropriate `BITS_PER_*`
macros).
Signed-off-by: Chris Friedt <cfriedt@tenstorrent.com>
The ztest.c calls atoi, which is provided by the C standard library.
Remove the conditional inclusion of the C standard library with the
definition of CONFIG_ZTEST_SHUFFLE.
Signed-off-by: Firas Sammoura <fsammoura@google.com>
Add the missing #ifndef CONFIG_ZTEST_SHUFFLE in ztest.c file.
This fixes multiple definitions of NUM_ITER_PER_SUITE
and NUM_ITER_PER_TEST.
Signed-off-by: Arkadiusz Cholewinski <arkadiuszx.cholewinski@intel.com>
Expanding the ZTEST shell to include a command that
accepts additional parameters, such as ztest run_testcase
<suite:test> -p any_value.
Adding the new ZTEST_P macro, to create the
parameterized test.
Signed-off-by: Arkadiusz Cholewinski <arkadiuszx.cholewinski@intel.com>
Expand ztest_shell subcommands `run-testcase`
and `run-testsuite` to include additional parameter `-r`
repeat-iter. This allows to run a particular testcase or
testsuite several times.
Signed-off-by: Arkadiusz Cholewinski <arkadiuszx.cholewinski@intel.com>
Replace `shell_fprintf(sh, SHELL_ERROR, ...)` with
`shell_error(sh, ...)` since it brings the same underlying
action with less typing.
Note: `shell_error` already concatenates `\n` to format strings,
so we remove `\n` from the original code.
Signed-off-by: Pisit Sawangvonganan <pisit@ndrsolution.com>
Utilize a code spell-checking tool to scan for and correct spelling errors
in `subsys/testsuite/ztest/src/ztest.c` file.
Signed-off-by: Pisit Sawangvonganan <pisit@ndrsolution.com>
Undefine _POSIX_C_SOURCE before defining it to avoid double
definition
The previous version, changed in this PR
https://github.com/zephyrproject-rtos/zephyr/pull/70363, did the undef.
Make sure to do the same for all files in the current dir.
Signed-off-by: Dawid Niedzwiecki <dawidn@google.com>
The testsuite subsystem provides the macro PRINT_DATA() which can be
overridden using CONFIG_ZTEST_TC_UTIL_USER_OVERRIDE. Switch all calls of
PRINT() to PRINT_DATA withing the testsuite subsystem.
Signed-off-by: Keith Short <keithshort@google.com>
Fix compilation error implicit declaration of function 'free'
in case of `CONFIG_NEWLIB_LIBC`.
Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
Many ztest macros were not indexed correctly as part of the ztest_test
group, for example ZTEST_SUITE was completely missing in the
documentation.
In addition, document a few macros that did not have any documentation.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Add an option to reset gcov counters before running tests.
This ensures that only code lines triggered by test itself are counted in
the coverage report and all the board initialization code and pre-test
bootstrap is not counted. This is useful when, for example, you are
testing code that is also executed during bootup
Test Plan:
west build -p always -b qemu_x86_64 tests/ztest/base/ -- \
-DCONFIG_COVERAGE=y -DCONFIG_COVERAGE_GCOV=y -DCONFIG_COVERAGE_DUMP=y \
-DCONFIG_ZTEST_COVERAGE_RESET_BEFORE_TESTS=y
ninja -Cbuild run | tee log.log
Signed-off-by: Roman Studenikin <srv@meta.com>
Make `struct arch_esf` compulsory for all architectures by
declaring it in the `arch_interface.h` header.
After this commit, the named struct `z_arch_esf_t` is only used
internally to generate offsets, and is slated to be removed
from the `arch_interface.h` header in the future.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Namespaced the generated headers with `zephyr` to prevent
potential conflict with other headers.
Introduce a temporary Kconfig `LEGACY_GENERATED_INCLUDE_PATH`
that is enabled by default. This allows the developers to
continue the use of the old include paths for the time being
until it is deprecated and eventually removed. The Kconfig will
generate a build-time warning message, similar to the
`CONFIG_TIMER_RANDOM_GENERATOR`.
Updated the includes path of in-tree sources accordingly.
Most of the changes here are scripted, check the PR for more
info.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
The `cpu.h` is meant to be a hack around Zephyr's dependencies
for unit-test, relocate it to a folder that's included only by
unit-test.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
When z_test_1cpu_start() is called to ensure that only a single CPU
on an SMP system is available for use in a test, this commit will
ensure that that CPU is the primary CPU--CPU0. This is done because
some timer drivers only have the timer interrupt processed by one CPU.
A bit of a song and dance is performed to achieve this without enabling
the CPU mask/affinity pinning API. If the cpuhold thread is found to
be executing on CPU0, then a new copy of cpuhold thread is created. Once
the new copy is executing (incidentally guaranteed to be on another CPU)
then it informs the original copy and busy waits until it the original
copy is switched out of CPU0. At this point, we can create the next
cpuhold thread to occupy another CPU if needed.
During this song and dance, it is critical that the 'copy' not pend. If
it pends, we can not guarantee which CPU it will execute on when it
unpends. As the cpuhold threads have the highest priority, nothing is
going to cause them to execute on another CPU for as long as they do
not pend.
Signed-off-by: Peter Mitsis <peter.mitsis@intel.com>
arch_interface.h is for architecture and should not be
under sys/. So move it under include/zephyr/arch/.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Add declaration of the _POSIX_C_SOURCE < 200809L for ztest library for
declaration of strtok_r and strdup functions.
Signed-off-by: Dawid Niedzwiecki <dawidn@google.com>
As we are replacing native_posix with native_sim, let's
refer to native_sim instead of native_posix in the comments.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
As per current definition of the Z_TEST macro, defined in
include/zephyr/ztest_test_new.h, the passed parameter is named
'fixture', not 'this'.
Signed-off-by: Reto Schneider <reto.schneider@husqvarnagroup.com>
This commit increases the default value of `ZTEST_TEST_DELAY_MS` from 3000
to 5000 milliseconds because the current value of 3000ms may not be
sufficient for the hosts with lower CPU clock frequency (e.g. new Zephyr CI
runners with cost-effective processors).
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
This is obviously a copy & paste error. It was introduced by
107cb86bb3 (ztest: Add initial zexpect API
for delayed failing).
Signed-off-by: Reto Schneider <reto.schneider@husqvarnagroup.com>
This file uses strtok_r which is an extension to the the
std C library. Let's explicity select one of the extensions
which includes it instead of relaying on somebody having
set it for this file somewhere else.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
Export the minimal set of ztest_test_* symbols to llexts so that the
zassert_* macros can be used by the extension code to affect test
execution. Calling these also requires vprintk() to be exported.
Signed-off-by: Luca Burelli <l.burelli@arduino.cc>
- Support for listing both testcases and testsuites
- Support for running single suites or single test cases
- Support shuffling tests and repeating execution based on command line
arguments.
For example, build with
west build -p -b qemu_cortex_m3 tests/kernel/sleep -t run -- \
-DCONFIG_ZTEST_SHUFFLE=y -DCONFIG_ZTEST_SHELL=y
Following commands are available:
uart:~$ ztest
ztest - Ztest commands
Subcommands:
run-all :Run all tests
shuffle :Shuffle tests
list-testsuites :List all test suites
list-testcases :List all test cases
run-testsuite :Run test suite
run-testcase :Run testcase
shuffle accepts two arguments --suite_iter and --case_iter which allows
repeated exercution of testcases or suites.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Many functions return non-zero return codes on errors. I added an assert
for the case, when a function is expected to fail. It is just a
shorthand for `zassert_not_equal(0, ret);`, analogous to
`zassert_ok`, introduced in c5d85e175f.
I also added the corresponding `zassume_nok` and `zexpect_nok`.
Signed-off-by: Greter Raffael <rgreter@baumer.com>
