Adds an option to inform twister a testsuite should only be built and
run for platforms with unique sets of attributes. This enables
for example keying on unique (arch, simulation) platforms to run the test
suite on.
The most common usage may be test suites configured to run once per
(arch, simulation) pair as being enough. Additional information about
platforms may enable running a test once per hardware IP block or once
per soc family or soc avoiding duplicated effort in building and running
tests when once suffices.
Signed-off-by: Tom Burdick <thomas.burdick@intel.com>
We want to be able to have platform or architecture extra configs
without having to duplicate a whole section of the test specification.
This adds support for namespacing of extra configs, for example:
arch:nios2:CONFIG_SYS_CLOCK_TICKS_PER_SEC=1000
or
platform:qemu_x86:CONFIG_FOO=y
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Previous commit dealing with cleanup (c304db56f1) was not taking
--prep-artifacts-for-testing into consideration. Fix this and make sure
we do cleanup when this option is enabled.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Have boards define what binaries should be kept for device testing. The
purpose of this is to reduce the amount of artifacts and just keep the
binaries needed for flashing a device, reducing the size of the final
payload.
A board can have twister keep multiple binaries needed for testing when
--prep-artifacts-for-testing option is used.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Add SDO block transfer support for the CANopen program download west
runner. SDO block transfers are experimental in the underlying CANopen for
Python.
Reduce the program download chunk size to half the program download buffer
size to avoid blocking IO.
Signed-off-by: Henrik Brix Andersen <hebad@vestas.com>
When re-generating hardware map for DKs with more than 1 serial port (more
enries in the hardware map), then we keep enfo only about the last entry.
For boards like nrf5340 it works, because the last serial port is used,
but for nrf52840 with two ports, the first one is important, so after
re-generating the hardware map it does not work.
The solution is to keep all entries after re-generating the hardware map.
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
Expands kconfigfunctions to include checks for value existence in
an array property by nodelabel, or a chosen's boolean prop value.
Signed-off-by: Stephen Stauts <stephen.stauts@nordicsemi.no>
Add a YAMLLint compliance check that uses the yamllint package to report
linting error on YAML files.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
When scanning for available elfs, twister discards ones with `_pre`
in their name, to not include prebuilds like 'zephyr_pre0.elf`.
However, the scan looks at the whole test name. If _pre is in the name
(e.g. _precision) it would be wrongly discarded ending with twister
reporting error that no elfs were found. This commit makes the discard
to look only at the last part of the name (i.e. not including test name
but only a lowest level, where zephyr.elf etc are located.)
Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
Fix all comments-indentation errors detected by yamllint:
yamllint -f parsable -c .yamllint $( find -regex '.*\.y[a]*ml' ) | \
grep '(comments-indentation)'
This checks that the comment is aligned with the content.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
Fix all thruthy errors detected by yamllint:
yamllint -f parsable -c .yamllint $( find -regex '.*\.y[a]*ml' ) | \
grep '(truthy)'
This only accepts true/false for boolean properties. Seems like python
takes all sort of formats:
https://github.com/yaml/pyyaml/blob/master/lib/yaml/constructor.py#L224-L235
But the current specs only mention "true" or "false"
https://yaml.org/spec/1.2.2/#10212-boolean
Which is the standard yamllint config.
Excluding codeconv and workflow files, as some are using yes/no instead
in the respective documentation.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
When using llvm we need to set the gcov-tool to "llvm-cov gcov" but
the lcov tool is incapable of passing arguments to the gcov-tool. i.e.
the following cannot work:
$ lcov --gcov-tool "llvm-cov gcov" ...
Instead, create a symlink to llvm-cov prefixed as `gcov` which by the
documentation of llvm-cov will alias to `llvm-cov gcov` subcommand.
Signed-off-by: Yuval Peress <peress@google.com>
Add a compliance check that tries to load MAINTAINERS.yml with
get_maintainer.Maintainers() if it's been modified by the CL, and fail
compliance if it fails to be parsed.
Example output:
```
ERROR : Test MaintainersFormat failed:
Error parsing MAINTAINERS.yml: MAINTAINERS.yml: YAML error: while
scanning a simple key
in "MAINTAINERS.yml", line 976, column 1
could not find expected ':'
in "MAINTAINERS.yml", line 977, column 3
```
```
ERROR : Test MaintainersFormat failed:
Error parsing MAINTAINERS.yml: MAINTAINERS.yml: glob pattern
'drivers/regulator' in 'files' in area 'Drivers: Regulators' matches a
directory, but has no trailing '/'
```
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
Make sure that this runner can handle multiple --tool-opt arguments,
and that the options specified for pyocd are passed to that tool in
order.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Change the bindings file filter to check for "dts/bindings/" as a
substring rather than prefix. This makes the check catch files in
subdirectories as well (for sample tests and samples), and makes the
check usable on external projects where the bindings may not live in the
project root.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
Move the file iteration loops of the bindings compliance check in the
callers, so that if we add more checks we don't have iterate on each one
of them and a check works on a single file.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
Add a check to validate and limit the size of images in the repository,
setting stricter limit for files in boards/.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
Git normally only works on files, but can include directories if there's
any submodule in the repository. This is currently checked in
filter_py(), move the check into get_files() so it does not have to be
copied around.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
DevicetreeBindingsCheck is missing the splitlines() call to process the
get_files() output, hence cycling through each character of the output
rather than each file, which causes the check to never run on anything.
Fix it by moving the splitline call into get_files() itself, since every
caller is going to use it anyway.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
This commit expands Twister's CLI so that arguments passed after
a double dash (`--`) get forwarded to the ztest test executable when
run. When built for native_posix, ztest provides a useful CLI that
allows filtering individual suites and tests, adjusting timing settings,
and controlling the shuffling seed. Currently there is no easy way to
use these (with the exception of `--seed`) without needing to dig out
the resulting `zephyr.exe` binary from the build dir and invoke it
manually.
### Examples
Run a specific ztest suite only (useful when writing tests and you don't
want to run a long testcase in its entirety)
```
$ scripts/twister \
-p native_posix \
-s
zephyr/tests/lib/cmsis_dsp/complexmath/libraries.cmsis_dsp.complexmath \
-- -test=complexmath_q31::*
```
Unrecognized arguments that precede the double dash will result in an
error message:
```
$ scripts/twister \
-p native_posix \
--foobar 123 \
-- -test=basic_math_q7::*
...
Unrecognized arguments found: '--foobar 123'. Use -- to delineate extra
arguments for test binary or pass -h for help.
```
Signed-off-by: Tristan Honscheid <honscheid@google.com>
Zephyr's coding guidelines require braces on every code block body.
Enable InsertBraces option so clang-format automatically adds these
braces.
Signed-off-by: Keith Short <keithshort@google.com>
This commit adds the requirement for the `TIMER_RANDOM_GENERATOR`
Kconfig to be set to `n` because it enables timer-based pseudo-random
number generator, which is only intended to be used for testing
purposes.
Signed-off-by: Stephanos Ioannidis <stephanos.ioannidis@nordicsemi.no>
This commit updates the KconfigCheck to add the Kconfig symbols from
both "config" and "choice" symbol lists to the defined symbol list, as
opposed to the "config" list only, in order to prevent any references
to the choice symbols outside the Kconfig files (e.g. in documentation)
from being reported as undefined.
Signed-off-by: Stephanos Ioannidis <stephanos.ioannidis@nordicsemi.no>
This PR clarifies the violation messages emitted by gitlint when
checking the commit message. For example:
* Before: `43: UC4 Line exceeds max length (N>75): "..."`
* After: `43: UC4 Commit line exceeds max length (N>75): "..."`
This makes it easier to identify the source of the error since there is
currently no additional context besides the error code UC*. I recently
pushed a commit that had some sample code as part of the commit body
that exceeded the lenght limit, and thought the error was referring to
one of my source files based on the line it showed. (feel free to laugh
at me, but let's make it better for the next person)
Signed-off-by: Tristan Honscheid <honscheid@google.com>
Generates XML file containing VIF policies by reading the device tree
using EDT.pickle generated during build
This script writes a subset of general and sink-pdo VIF policies in
output file
This script gets invoked during build if enabled through kconfig
The generated XML containing USB-C VIF policies could be used by
USB PD/Type-C analysers/testers to understand USB-C properties and
perform tests accordingly
Signed-off-by: Madhurima Paruchuri <mparuchuri@google.com>
path for posix was not being matched correctly due to a missing /. So
some posix tests were excluded and not excercised.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
The current docs are incorrect as comma separated paths
are not working. Providing a separate --exclude argement
per path does however work.
Signed-off-by: Håkon Øye Amundsen <haakon.amundsen@nordicsemi.no>
Ignore unit tests and EFI binaries when calculating footprints with
--show-footprint, or we will get a build error.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
The --snr (nrfjprog) --id (jlink) and --board-id (pyocd) options were
deprecated a long time ago in favor of --dev-id. It is time to remove
them.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
This commit calculate memory footprint from build.log and
proposes an alternative approach to #2465.
Signed-off-by: Katarzyna Giądła <katarzyna.giadla@nordicsemi.no>
non-simulation platforms set key to 'na', we need to check for that
instead of just checking for existance of key.
Fixes#52595
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Both get_defined_syms and UNDEF_KCONFIG_WHITELIST are used exclusively
by the Kconfig check, so move them to the class.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Instead of globally resolving ZEPHYR_BASE, move it to _main() so we can
catch potential exceptions when doing so and for consistency with the
rest of globals, like GIT_TOP. This also includes documenting a new
"<zephyr-base>" path hint for classes.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
The global variable ZEPHYR_BASE is always set in the script, so no need
to check for its existence.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Some errors are attributable to a specific file, but not a specific line
within that file. Allow for formatted errors that refer to a file but
not a line.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Many tests invoke `git diff` independently with very similar parameters
to obtain a filtered list of files.
Consolidate those into a new `get_files()` function.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
The workaround for old pytest bug
https://github.com/PyCQA/pylint/issues/2906
is not required anymore after more than 3 years, so remove it.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
