Somehow this change was dropped (rebase?), this is needed to parse new
ztest test correctly, otherwise will have many tests missing from the
plan.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Add option to prepare for device testing by only building and storing
the needed data that can then later be executed and run on DUT using
--test-only.
This adds --prep-artifacts-for-testing which is the same as --build-only
and --device-testing and other options that were needed to make this
operational and produce the right binaries and files.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
using serial as the group name is misleading, so call it 'device' which
is what all those options about.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
This is all static functions that can maintained outside of the class
and called only in one place, so move this to the testsuite file.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Get options from passed environment class instead of using command line
options to the init function.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Move all code related to instance filtering and loading into the class
instead of having it in the main script. Simplify option passing.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
We have been scanning code for tests for every scenario defined in the
yaml file. This needs to only be parsed once and not more. We are
dealing with the same files for each scenario.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
If we already have recorded the reason during the qemu run, do not
override it at the final stage of status processing.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
below command line need has fixture configured in map.yml,
if not, it shall be skipped
twister -p frdm_k64f --device-testing \
--hardware-map map.yml -T tests/subsys/modbus/ -vv
add harness check with testcase harness by default
will fix this issue
Fixing: #46635
Signed-off-by: Hake Huang <hake.huang@oss.nxp.com>
Option --runtime-artifact-cleanup/-M allows to clean build artifacts
during runtime to save space usage. However, if one wanted to combine
it with --build-only and then --test-only, they would get an error.
This commit expands the list of files stored with -M with files
required to made --test-only also to work afterwards. Such change can
be useful for setups where building is done on one node and then
minimal amount of artifacts are transfered to another one.
Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
add_missing_case_status is more appropriate for this function.
We are adding missing status, not cases.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Add a filter type to distinguish between testsuite filters that are
maintained in the testcase.yaml file and those provided on the command
line.
The issue was that when something is excluded deliberately on the command
line twister would report an error if that filter would filter out an
integration platform. We do not want that, because the filtering is
being done by the caller knowing that some tests would be not run, even
if those are integration platforms.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Some tests can cause serial leftover logs buffered on board.
A simple ser.flush() is not enough. So add explicit readline()
to drain such logs which prepares a clean serial context for
the case that follows.
Signed-off-by: Ming Shao <ming.shao@intel.com>
The fallback testcase, which is the scenario name from the yaml file is
added if we did not discover any testcases via parsing. It can be
removed if we discover testcaes at runtime or otherwise it is added as
skipped...
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Status was set for testcases after a timeout on flash, which does not
abort the process and continues with test execution and eventually
passes the test.
Setting status on the testcases was done too early and before we have
executed the tests.
Also increase the flash timeout to 60s.
Fixes#45845
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
