- Support for listing both testcases and testsuites
- Support for running single suites or single test cases
- Support shuffling tests and repeating execution based on command line
arguments.
For example, build with
west build -p -b qemu_cortex_m3 tests/kernel/sleep -t run -- \
-DCONFIG_ZTEST_SHUFFLE=y -DCONFIG_ZTEST_SHELL=y
Following commands are available:
uart:~$ ztest
ztest - Ztest commands
Subcommands:
run-all :Run all tests
shuffle :Shuffle tests
list-testsuites :List all test suites
list-testcases :List all test cases
run-testsuite :Run test suite
run-testcase :Run testcase
shuffle accepts two arguments --suite_iter and --case_iter which allows
repeated exercution of testcases or suites.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Move the syscall_handler.h header, used internally only to a dedicated
internal folder that should not be used outside of Zephyr.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
This test cannot be run in this architecture as it does not
support userspace.
Today it is filtered by kconfig, which works but spends
time running cmake.
As native_posix is a default test platform it is better
to filter it alltogether by arch, which saves quite a lot
of time.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
Skip `test_catch_fatal_error` test case for Armv8-R because
divide by zero trapping is not supported on this architecture.
Fixes#63268
Signed-off-by: Manuel Argüelles <manuel.arguelles@nxp.com>
The SW div implementation from libgcc doesn't handle division by
zero case, so the HW exception isn't triggered.
So, skip this test case (as we are already doing for some other
ARC targets).
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
Twister now supports using YAML lists for all fields that were written
as space-separated lists. Used twister_to_list.py script. Some artifacts
on string length are due to how ruamel dumps content.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
Adding the new Kconfig (enabled by default) to make a failed assumption
mark the final result as failed. This change has the following benefits
which have been asked for by the Zephyr community:
1. A failed assumption does not go silent. In this example, the failed
assumption will still mark the test as skipped, but the final result
will be to mark the full test run as failed. This would allow
blocking the CI when an assume fails.
2. Normal test skipping via the ztest_test_skip() is unaffected by this
change. Those tests will be marked as skipped, but the binary will
still pass.
Signed-off-by: Yuval Peress <peress@google.com>
This commit enables the assert test mode (`CONFIG_ASSERT_TEST`) for the
ztest error hook test because it implements a custom post assert fail
hook (`ztest_post_assert_fail_hook`) that returns without aborting to
faciliate the testing of the assert functions.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
Add a bunch of missing "zephyr/" prefixes to #include statements in
various test and test framework files.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
Add an assume API which works like JUnit's. Assumptions can be made
at any point before your test returns (setup, before, and during the
test). If an assumption fails, the test will be marked as skipped.
This helps avoid a cascading affect of failed tests when a base
feature breaks. The feature is expected to have tests and the tests
which depend on it should be skipped (not failed) if that feature
is broken.
Issue #42472
Signed-off-by: Yuval Peress <peress@google.com>
1.Remove filter to allow platforms(e.g. qemu_x86) supporting
CONFIG_ARCH_HAS_USERSPACE run testcases when userspace is disabled.
2.Define all testcases in the yaml for consistency and issue parsing
them during setup.
Signed-off-by: Guo Lixin <lixinx.guo@intel.com>
In order to bring consistency in-tree, migrate all tests to the new
prefix <zephyr/...>. Note that the conversion has been scripted, refer
to #45388 for more details.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
Change the CPU_CORTEX_R kconfig option to CPU_AARCH32_CORTEX_R to
distinguish the armv7 version from the armv8 version of Cortex-R.
Signed-off-by: Bradley Bolen <bbolen@lexmark.com>
Because RISC-V arch (M extension) doesn't trigger exception for
division-by-zero, this test can't support RISC-V and is disabled in
RISC-V.
Signed-off-by: Jim Shu <cwshu@andestech.com>
Fix the trigger_fault_access. The fix is similar to cortex m.
Fix the trigger_fault_divide_zero. The fix is similar to qemu, i.e.
skip this testcase, because FVP would not trigger the exception for
dividing zero either.
Signed-off-by: Jaxson Han <jaxson.han@arm.com>
Move to CMake 3.20.0.
At the Toolchain WG it was decided to move to CMake 3.20.0.
The main reason for increasing CMake version is better toolchain
support.
Better toolchain support is added in the following CMake versions:
- armclang, CMake 3.15
- Intel oneAPI, CMake 3.20
- IAR, CMake 3.15 and 3.20
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
The Cortex-R platforms may no use XIP which means userspace code with
have read and execute access to some of the RAM so that it can execute
code. However, userspace will not have access the the protected
read/write parts of RAM so it is safe to use the Cortex-M version of
trigger_fault_access for this test.
Signed-off-by: Bradley Bolen <bbolen@lexmark.com>
On the Cortex M0+, M0, M23 the div-by-zero is not mapped
as an hardware exception but triggers a HardFault.
The test case is skipped for any ARMV6_M_ARMV8_M_BASELINE.
Signed-off-by: Francois Ramu <francois.ramu@st.com>
This compile test should be checking if a symbol has been defined,
otherwise it is using the kconfig value directly. This fixes a warning
../src/main.c:115:37: warning: "CONFIG_SOC_QEMU_ARC" is not defined,
evaluates to 0 [-Wundef]
when using the -Wundef flag.
Signed-off-by: Bradley Bolen <bbolen@lexmark.com>
qemu arc has limited support for DIV/REM, no DIVZERO exception
throw, so we need to skip trigger_fault_divide_zero test case.
Signed-off-by: Watson Zeng <zhiwei@synopsys.com>
Put the testcase test_catch_assert_in_isr() to execute last, to prevent
it affects other test cases. Because when we caught an assert failure
in the ISR handler, it cannot be guaranteed that all the current
program status would be recovered.
Fixes#34844.
Signed-off-by: Enjia Mai <enjiax.mai@intel.com>
Make the divide instruction not being optimized of the ztest example
test case. Do this because it will be replaced by invalid opcode
exception instruction generated by GCC.
Fixes#33403.
Signed-off-by: Enjia Mai <enjiax.mai@intel.com>
em_starterkit has ICCM at 0x0 address, access to 0x0
address doesn't generate any exception, so we access
to 0xFFFFFFFF address instead to trigger exception.
Signed-off-by: Watson Zeng <zhiwei@synopsys.com>
When running the test on Cortex-M platforms, use access to
_current to trigger a memory access fault, as this address
is guaranteed to be in kernel ram and will not trigger
SecureFault exceptions.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
This functions is being called across the tree, no reason why it should
not be a public API.
The current usage violates a few MISRA rules.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
The quirky behavior when accessing of 0x0 address is related to
exact ARC SoCs/boards and not to entire ARC EM CPUs family. So
check for exact SoCs configs (CONFIG_SOC_ARC_IOT and CONFIG_SOC_NSIM)
instead of checking against CONFIG_CPU_ARCEM.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Minor fixes to the sample output in the README file,
reflecting the correct error reason (3: oops) instead
of reason 0 (generic CPU error).
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
test_catch_z_oops test case should not run in user mode,
since Z_OOPS needs to be called in supervisor mode, otherwise
we will be getting a reguser user memory access error that is
irrelevant to the test case (and simply is triggered by Z_OOPS
accessing the thread's syscall frame pointer).
In addition to that, we fix the argument of Z_OOPS, because, it
was triggering a null-pointer dereferencing, which results in an
error thrown before Z_OOPS is even executed (if null-pointer
exception is caught).
We also need to generate a dummy thread->syscall_frame, that
Z_OOPS implementation will access. We fix this to
_image_ram_start, because this memory is always part of the
image memory and is always available to supervisor mode. So,
accessing it won't trigger a security fault, for example.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
The test case suggests that "For most arch which support
userspace, derefencing NULL pointer will be caught by
exception.". This is certainly not true for Cortex-M, where
read access to address 0x0 is generally allowed. The reason
the test had been passing was either 1) because in many
Cortex-M platforms, including QEMU, address 0x0 is unmapped,
or 2) GCC is generating an undefining instruction. However,
now that we have activated the null pointer derefrencing
detection, we may end up with two exceptions and the test
would fail.
Change illegal access to something outside the mapped memory
area, e.g. 0xFFFFFFFF.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
Enlarge the test thread stack size, to fix test case fail
in some board which need more stack size for testing, when
CONFIG_EXCEPTION_DEBUG is enabled.
Signed-off-by: Enjia Mai <enjiax.mai@intel.com>
Fix issue #31339 and #31419, test case fail due to access NULL did not
trigger a fatal error in some platform such as nsim_em and iotdk.
Signed-off-by: Enjia Mai <enjiax.mai@intel.com>
