Using --platform-reports will also generate json reports
with results from a single platform. Needed for on-target
results publishing process.
Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
Measure both cmake and make/ninja build times and log them in debug mode
and additionally put the result in the json report for tracking build
times.
Cleanup what build/cmake function return and remove unused keys in the
result map.
Remove some excessive logging of launched jobs or default platforms that
gets in the way when in debug mode.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
When pytest scenario fails, then 'handler.log' is printed.
Changed to print 'twister_harness.log' that is a log from
pytest execution. That file tells much more when test fails.
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
Test suites names are not being handled uniformly for tests not
in zephyr tree. Their names depend on -T arg used in twister's CLI.
The newly added options allow to select if twister should add paths
to suite names. This is needed if test plans are to be used for tests
outside of zephyr tree.
Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
"Path" is a path (relative to zephyr) to a directory with test suite's
definition. Such entry helps to locate a given test.
Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
When using the hardware map and testing on device, print out the name of
the dut (from the hardware map) on the script and report it in the json
output as well.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Stop counting errors as failures and report them as is. This fixes
issues with counting when retrying failures and errors.
We have been adding errors to failures and then double count them when
displaying information on the screen instead of keep an accurate count
in the counter class.
This is now fixed and we do report errors seperately, ie. build errors.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Take an instance that has failed and not whatever was last processed in
the for loop.
Also, change instructions to use `west twister` rather than
`./scripts/twister` to be consistent.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
If errors are found and tests are failing, print a brief summary showing
the top 10 items failing with instructions on how to reproduce either
with twister or west.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
This commit calculate memory footprint from build.log and
proposes an alternative approach to #2465.
Signed-off-by: Katarzyna Giądła <katarzyna.giadla@nordicsemi.no>
Version is not enough, we need the run date but also the commit date, to
ease reporting and use in dashboards.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
When twister is set to retry any failures, count the number of retries
and record the number in the json file. This will help us identify
unstable tests or tests requiring attention.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
