The signal handler needs to return gracefully. When multiple
tests assert, the first assert will raise SIGABRT and the signal
handler will run and the test will stop running.
The second assert will raise SIGABRT but the signal handler
will not be called, therefore the test PASS/FAIL status is not
updated correctly.
Signed-off-by: Al Semjonovs <asemjonovs@google.com>
A test case may invoke run_all multiple times, causing
logic for listing tests to run tests on second call.
Signed-off-by: Al Semjonovs <asemjonovs@google.com>
Make the option TEST_EXTRA_STACK_SIZE depend on Tests, otherwise it will
appear in each generated .config file.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
This config option is only used with ztest, so move it under ztest.
Otherwise it appears in .config when you build ANY application.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Add a bunch of missing "zephyr/" prefixes to #include statements in
various test and test framework files.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
This commit removes the NULL check on the idle thread array
(`idle_tid`) because it is statically declared and will never be NULL.
This fixes the "comparison will always evaluate as ‘false’" warning
generated by the GCC 12.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
Add test summary after all test suites finish running.
The summary can be one-line or verbose, which is configured
with CONFIG_ZTEST_VERBOSE_SUMMARY. The one-line summary covers
overall suite stats. The verbose summary covers each test
function within the suite besides the one-line summary.
Signed-off-by: Ming Shao <ming.shao@intel.com>
Support verbose or one-line summary at test suite level.
Support verbose or no output at test function level.
Totally 4 combinations configurable:
- function verbose + suite verbose
- function verbose + suite oneline
- no function output + suite verbose
- no function output + suite oneline
Signed-off-by: Ming Shao <ming.shao@intel.com>
Documentation now includes:
- How to create a test suite
- How to use predicates
- How to use setup/before/after/teardown
- How to use test rules
- Direct people to use FFF instead of ztest mock
Fixes#47420
Signed-off-by: Yuval Peress <peress@google.com>
Add macro Z_TEST_SKIP_IFNDEF which is used when you want to skip test
if configuration option is not enabled.
Signed-off-by: Andrei Emeltchenko <andrei.emeltchenko@intel.com>
The addition of the assumption API introduced a bug where a skipped test
would subsequently skip later tests within the same suite. A skipped test
was being treated as an entire skipped test suite.
Keep track of tests being skipped due to skipping in the suite setup versus
being skipped within their test function.
TEST=west build -p -b native_posix tests/kernel/condvar/condvar_api/ -t run
TEST=twister -T tests/ztest -p native_posix
Signed-off-by: Aaron Massey <aaronmassey@google.com>
Following zephyr's style guideline, all if statements, including single
line statements shall have braces.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Add an assume API which works like JUnit's. Assumptions can be made
at any point before your test returns (setup, before, and during the
test). If an assumption fails, the test will be marked as skipped.
This helps avoid a cascading affect of failed tests when a base
feature breaks. The feature is expected to have tests and the tests
which depend on it should be skipped (not failed) if that feature
is broken.
Issue #42472
Signed-off-by: Yuval Peress <peress@google.com>
When system clock is set globally for the test which is executed
on various qemu platforms it may happen that clock is set too
high compared to CPU clock frequency. In that case test may stuck.
Added check and test skipping in that case.
Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
The malloc partition may actually be empty, if so, don't try to add it to
the domain as that will generate an error.
Signed-off-by: Keith Packard <keithp@keithp.com>
On X86 platforms, the interrupt trigger method has been
changed to use APIC IPI, we don't use INT command to trigger
interrupt, so remove this unused macro.
Signed-off-by: Chen Peng1 <peng1.chen@intel.com>
On X86 platforms, the interrupt trigger method has been changed
from using INT command to using APIC IPI, we need to make sure
the IPI interrupt is handled before do our check, so add some
nop operations.
Signed-off-by: Chen Peng1 <peng1.chen@intel.com>
Added test command line arguments to filter
which tests are executed. Filtered tests should follow
suiteA::test1,suiteB::test2 format.
Signed-off-by: Al Semjonovs <asemjonovs@google.com>
Adds few missing zephyr/ prefixes to leftover #include statements that
either got added recently or were using double quote format.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
Update the new API to use K_USER as the flags for both
CONFIG_USERSPACE and CONFIG_TEST_USERSPACE. Also, fix the linker
script to properly include the suites, tests, and rules.
Fixes#44108
Signed-off-by: Yuval Peress <peress@google.com>
output of ztest is not easy to ready with many testsuites running in
sequence, make it a bit more readable.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Added macro to test that a value falls within a defined range, including
the provided end points.
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
Enable ZTEST_DO_THE_SHUFFLE to shuffle the order tests are ran.
Additional configs ZTEST_DO_THE_SHUFFLE_SUITE_REPEAT_COUNT
ZTEST_DO_THE_SHUFFLE_TEST_REPEAT_COUNT specify the number of times
the test or suite is executed.
Signed-off-by: Al Semjonovs <asemjonovs@google.com>
In order to bring consistency in-tree, migrate all subsystems code to
the new prefix <zephyr/...>. Note that the conversion has been scripted,
refer to zephyrproject-rtos#45388 for more details.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
The ztest unittest module relies on some headers that fake the real
ones, e.g. <zephyr/arch/cpu.h>. When new header location was introduced,
their path was not updates. This patch fixes this problem and provides
ztest with both, legacy and new include paths so that it can keep
working on both scenarios.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
Update the new API to use K_USER as the flags for both
CONFIG_USERSPACE and CONFIG_TEST_USERSPACE. Also, fix the linker
script to properly include the suites, tests, and rules.
Fixes#44108
Signed-off-by: Yuval Peress <peress@google.com>
define test run id and make it available to ztest. This will be used to
verify we are evaluating the same test we have just built and flashed
and that we are not looking at some old output.
Existing code in ztest will use this, and twister will define the run
id.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Make sure that the test rules' `before` function runs before the
suite's. This allows the suite to override any defaults set by the
rule.
Signed-off-by: Yuval Peress <peress@google.com>
The idle thread got an index suffix in #23536 to make it easier to
identify different idle threads on different cores. This looks out of
place on single-core devices when the idle thread is listed next to
other kernel threads, such as main.
Remove the idle thread index on single-core platforms, and replace all
references to this format in tests and documentation.
Signed-off-by: Trond Einar Snekvik <Trond.Einar.Snekvik@nordicsemi.no>
Delay start of threads and timer to ensure that setup
is completed. Especially, vital on multiple CPUs.
Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
