Commit graph

822 commits

Author SHA1 Message Date
Anas Nashif
c0317fba1f twister: handler quotes in extra_args
Escape quotes provided in extra arguments on the command line.

example: ... -x "CONFIG_COMPILER_OPT=\"-fanalyzer\""

Fixes #46382

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-29 05:58:15 -04:00
Piotr Golyzniak
7ce914b7f4 twister: verify all platform names
Fix bug in verify_platforms_existence method - make it possible to
verify all platform names from list - not only first.

Fixes: #48321

Signed-off-by: Piotr Golyzniak <piotr.golyzniak@nordicsemi.no>
2022-07-26 20:42:20 -04:00
Stephanos Ioannidis
0f9588e51f scripts: twisterlib: Fix default value for --ninja argument
This commit makes the `--ninja` argument default to `true` when neither
`-k` nor `--make` is specified (i.e. when Make generator is not
selected).

Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
2022-07-26 20:17:59 -04:00
Stephanos Ioannidis
1f81d2cd97 scripts: twisterlib: Fix --short-build-path argument handling
When `--short-build-path` argument is specified to the twister, the
following error message is displayed because the Python argparse module
does not allow specifying a mutually exclusive argument as required:

  ValueError: mutually exclusive arguments must be optional

This commit removes the `required` parameter when adding the `--ninja`
argument, which is a mutually exclusive argument, and adds a manual
check to validate this condition instead.

Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
2022-07-26 20:17:59 -04:00
Bartosz Bilas
3322aad03b scripts: pylib: twister: twisterlib: fix typo
s/enviornment/environment

Signed-off-by: Bartosz Bilas <b.bilas@grinn-global.com>
2022-07-19 11:41:54 -04:00
Jason Wright
c6c7dde969 scripts: better logging for --coverage-formats option
Adds additional log messages for coverage reports
beyond HTML format.

Signed-off-by: Jason Wright <jwright@synchron.com>
2022-07-18 15:03:13 -04:00
Jason Wright
60c4d20548 scripts: restrict --coverage-formats option to gcovr
Generates an error if --coverage-formats option is supplied without
coverage tool being set to gcovr.

Signed-off-by: Jason Wright <jwright@synchron.com>
2022-07-18 15:03:13 -04:00
Jason Wright
786f28f162 scripts: add --coverage-formats option for use with gcovr
Adds optional twister flag to support gcovr output report formats.

Signed-off-by: Jason Wright <jwright@synchron.com>
2022-07-18 15:03:13 -04:00
Chen Peng1
e7088743df scripts: twister: only load connected hardwares.
only load hardwares with connected is true in
hardware map files.

Signed-off-by: Chen Peng1 <peng1.chen@intel.com>
2022-07-13 06:18:08 -04:00
Chen Peng1
1d5c37ab96 script: twister: fix hardware runner_params parsing.
The specific hardware instance is already got from
device_is_available function, we don't need to
iterate all hardwares again.

Signed-off-by: Chen Peng1 <peng1.chen@intel.com>
2022-07-13 06:18:08 -04:00
Anas Nashif
a2fbb783dd twister: fix new ztest parsing
Somehow this change was dropped (rebase?), this is needed to parse new
ztest test correctly, otherwise will have many tests missing from the
plan.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-13 10:13:28 +02:00
Anas Nashif
5f146dcaf9 twister: prep for device testing
Add option to prepare for device testing by only building and storing
the needed data that can then later be executed and run on DUT using
--test-only.

This adds --prep-artifacts-for-testing which is the same as --build-only
and --device-testing and other options that were needed to make this
operational and produce the right binaries and files.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-13 10:13:28 +02:00
Anas Nashif
68a8613a92 twister: fix --build-only argument help
Minor fix to the --build-only option help.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-13 10:13:28 +02:00
Anas Nashif
398e9902b8 twister: rename argument group: serial -> device
using serial as the group name is misleading, so call it 'device' which
is what all those options about.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-13 10:13:28 +02:00
Anas Nashif
e1e43015ec twister: also parse cpp files in app root
Fix wildcard to also scan cpp files in root folder of test application.

Fixes #47220

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-12 23:24:41 -04:00
Anas Nashif
9e3d697f15 twister: fix calculation of testsuite names
We were omitting one element of the path by mistake.
Also fix some naming issues in the testsuite.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-05 15:07:35 -04:00
Anas Nashif
1463f4bdc5 twister: rework testsuite parsing
Move testsuite setup and parsing to the testsuite class. Simplify
reading data from yaml.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-05 12:06:27 -04:00
Anas Nashif
4789f07a8f twister: fixups to various files
Various fixups, will be folded in correct commits eventually.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-05 12:06:27 -04:00
Anas Nashif
087f1e4a63 twister: fixed various pylint issues
Fix various pylint issues.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-05 12:06:27 -04:00
Anas Nashif
3d1971dee3 twister: adapt and fix testsuite tests
Re-add testsuite tests, include file scanning.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-05 12:06:27 -04:00
Anas Nashif
1a6948ac26 twister: move file scanning out of class
This is all static functions that can maintained outside of the class
and called only in one place, so move this to the testsuite file.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-05 12:06:27 -04:00
Anas Nashif
9437e6d963 twister: fix twister testsuite
Make testsuite work again.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-05 12:06:27 -04:00
Anas Nashif
08caa71de7 twister: testplan: get options directly
Get options from passed environment class instead of using command line
options to the init function.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-05 12:06:27 -04:00
Anas Nashif
91b2cce64e twister: fix subset processing
Process subsets was done in the wrong step, so make it apply and
generarte subsets correctly.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-05 12:06:27 -04:00
Anas Nashif
79d7d0188b twister: make ninja the default
Using ninja is up to 50% faster, make it the default and add an option
to build with Make.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-05 12:06:27 -04:00
Anas Nashif
7149173627 twister: simplify handler options
Depend on global options instead of passing them around for each class.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-05 12:06:27 -04:00
Anas Nashif
f2b5a6674a twister: move harness class to class folder
harness class moved along size other classes.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-05 12:06:27 -04:00
Anas Nashif
57dbaa3da3 twister: move more code from main script
Move code from twister script into respective classes.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-05 12:06:27 -04:00
Anas Nashif
f539f1dc48 twister: move size report into size calc class
Move from main script into size_calc class.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-05 12:06:27 -04:00
Anas Nashif
806722cdbf twister: move device testing code
Move device handling code into hardware map class.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-05 12:06:27 -04:00
Anas Nashif
f7d501447d twister: move cleanup and classification of code
Move code out of the main script and use global options.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-05 12:06:27 -04:00
Anas Nashif
396b8574eb twister: testplan: move calls to json loading/filter into class
Move all code related to instance filtering and loading into the class
instead of having it in the main script. Simplify option passing.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-05 12:06:27 -04:00
Anas Nashif
f85d1eb4e1 twister: rework testplan and main twister script
Move all testplan related code to the testplan class and cleanup main
twister script.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-05 12:06:27 -04:00
Anas Nashif
b8735b3300 twister: more cleanup to option passing
Use global options rather than passing them individually.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-05 12:06:27 -04:00
Anas Nashif
facc685ae9 twister: move runner code into runner class
Move all code related to test execution into runner class.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-05 12:06:27 -04:00
Anas Nashif
20f257a97d twister: move execution code out of testplan
Move all code related to execution into runner class.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-05 12:06:27 -04:00
Anas Nashif
d43e03e19e twister: isolate testplan class
Move testplan class into own file and rename the original twisterlib.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-05 12:06:27 -04:00
Anas Nashif
96b9ff6b5e twister: move runner classes out
Move runners into own file.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-05 12:06:27 -04:00
Anas Nashif
e5c2e797c9 twister: move test instance class out
Move test instance class into own file.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-05 12:06:27 -04:00
Anas Nashif
adfa6c7777 twister: move coverage classes out
Move coverage related classes out into own file.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-05 12:06:27 -04:00
Anas Nashif
78d9dab7f3 twister: cleanup imports and move size classes out
More python import optimizations and cleanup and move size_calc class
out into own file.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-05 12:06:27 -04:00
Anas Nashif
75eee4b1f3 twister: move platform class out
Move both platform and config parser into own file.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-05 12:06:27 -04:00
Anas Nashif
0779ccc5fc twister: import cleanup
python import cleanup and simplification.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-05 12:06:27 -04:00
Anas Nashif
f490b059ad twister: move handlers class out
Split handlers into own file.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-05 12:06:27 -04:00
Anas Nashif
2b213d3231 twister: split hardwaremap class out
Split hardware map class into own file.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-05 12:06:27 -04:00
Anas Nashif
d5424d5116 twister: split report classes out
Move report classes out into own file and split out of testplan class.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-05 12:06:27 -04:00
Anas Nashif
d8d217d2ea twister: move testsuite, testcase classes out
Move testsuite/testcase and supporting classes into own files.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-05 12:06:27 -04:00
Anas Nashif
521ce42161 twister: move cmake cache class
Move cmake cache classes into own file.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-05 12:06:27 -04:00
Anas Nashif
e01cbbe31f twister: add environment class
Add environment class to discover toolchain, zephyr version and other
items.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-05 12:06:27 -04:00
Anas Nashif
acda94be2d twister: optimize file scanning
We have been scanning code for tests for every scenario defined in the
yaml file. This needs to only be parsed once and not more. We are
dealing with the same files for each scenario.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-05 12:06:27 -04:00