Commit graph

5615 commits

Author SHA1 Message Date
Jordan Yates
29942475c5 scripts: gen_handles: output dependency graph
Output the final dependency graph as a `.dot` file, which when rendered
by graphviz can be easier to comprehend than the text descriptions.

This output is optional in that it will not be generated if `graphviz`
is not installed.

Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
2022-07-30 08:29:29 -05:00
Jordan Yates
8d17e857a1 scripts: gen_handles: use elf_parser.py
Use the new elf_parser module to simplify the process of generating the
final device handle arrays.

Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
2022-07-30 08:29:29 -05:00
Jordan Yates
06b0d5a4fb scripts: elf_parser: added
Adds a python module intended to simplify other scripts that need to
work with device information compiled into the first pass `.elf` file.

Scripts utilizing this module can focus on iterating over dependency
graphs to generate the desired output, instead of also needing to
extract and build the graphs in the first place.

Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
2022-07-30 08:29:29 -05:00
Jordan Yates
cfb8c5e70f pm: device: rename placeholder variable
Rename the placeholder variable generated for PM slots so that the
prefix doesn't colide with the PM structs declared by devices. This
simplifies the process of searching for symbols in `.elf` files.

Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
2022-07-30 08:29:29 -05:00
Anas Nashif
c0317fba1f twister: handler quotes in extra_args
Escape quotes provided in extra arguments on the command line.

example: ... -x "CONFIG_COMPILER_OPT=\"-fanalyzer\""

Fixes #46382

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-29 05:58:15 -04:00
Tristan Honscheid
4a8ffee1d8 twister: Don't run coverage report when --build-only is used
When doing a split build and test with coverage enabled, (i.e.
`twister --coverage --build-only` followed by `twister --coverage
--test-only`), Twister attempts to run coverage reports immediately
after building, which fails because tests haven't actually run yet
and causes several error messages to print in to the log.

This change causes twister to skip the call to `run_coverage()` if
`--build-only` is set and print an info message. This suppresses the
error messages from the coverage report tool complaining about missing
coverage files, but still instruments the built code for coverage data
collection. When twister is invoked again, but with `--test-only`
instead, the coverage files will be emitted and the reports
successfully generated.

 #### Testing

I ran...

```
$ zephyr/scripts/twister -T zephyr/tests/lib/cmsis_dsp/basicmath/
--coverage -p native_posix --build-only
$ zephyr/scripts/twister -T zephyr/tests/lib/cmsis_dsp/basicmath/
--coverage -p native_posix --test-only
```

... followed by a combined build and test ...

```
$ zephyr/scripts/twister -T zephyr/tests/lib/cmsis_dsp/basicmath/
--coverage -p native_posix
```

... as a control and diffed the HTML coverage reports, which were
identical other than a timestamp.

Signed-off-by: Tristan Honscheid <honscheid@google.com>
2022-07-28 05:58:52 -04:00
Piotr Golyzniak
7ce914b7f4 twister: verify all platform names
Fix bug in verify_platforms_existence method - make it possible to
verify all platform names from list - not only first.

Fixes: #48321

Signed-off-by: Piotr Golyzniak <piotr.golyzniak@nordicsemi.no>
2022-07-26 20:42:20 -04:00
Stephanos Ioannidis
0f9588e51f scripts: twisterlib: Fix default value for --ninja argument
This commit makes the `--ninja` argument default to `true` when neither
`-k` nor `--make` is specified (i.e. when Make generator is not
selected).

Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
2022-07-26 20:17:59 -04:00
Stephanos Ioannidis
1f81d2cd97 scripts: twisterlib: Fix --short-build-path argument handling
When `--short-build-path` argument is specified to the twister, the
following error message is displayed because the Python argparse module
does not allow specifying a mutually exclusive argument as required:

  ValueError: mutually exclusive arguments must be optional

This commit removes the `required` parameter when adding the `--ninja`
argument, which is a mutually exclusive argument, and adds a manual
check to validate this condition instead.

Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
2022-07-26 20:17:59 -04:00
Anas Nashif
5432cac806 ci: testplan: still run default tests with global changes
In cases of global changes where 100s of nodes are launched, i.e. on
samples and tests (more than 20 tests/samples changed), do a full
covrage run.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-22 11:00:23 -04:00
Kumar Gala
ccb9b6b5ba scripts: check_compliance: generate Kconfig.dts
Generate a Kconfig.dts to allow check_compliance to run.

Signed-off-by: Kumar Gala <galak@kernel.org>
2022-07-19 09:39:49 -07:00
Kumar Gala
b109a0a9f7 dts: Add simple script to generate Kconfig.dts based on bindings
Given binding dirs the script will generate a Kconfig.dts of the
form:

    DT_COMPAT_ADI_ADT7420 := adi,adt7420

    config DT_HAS_ADI_ADT7420_ENABLED
            depends on DTS_HAS_ADI_ADT7420
            def_bool $(dt_compat_enabled,$(DT_COMPAT_ADI_ADT7420))

Than a driver Kconfig can use these Kconfig symbols as follows:

    menuconfig ADT7420
            bool "ADT7420 Temperature Sensor"
            default y
            depends on DT_HAS_ADI_ADT7420_ENABLED
            ...

Signed-off-by: Kumar Gala <galak@kernel.org>
2022-07-19 09:39:49 -07:00
Bartosz Bilas
3322aad03b scripts: pylib: twister: twisterlib: fix typo
s/enviornment/environment

Signed-off-by: Bartosz Bilas <b.bilas@grinn-global.com>
2022-07-19 11:41:54 -04:00
Alex Kolosov
f59e9dd857 scripts: runners: improvements for spi_burn west runner
Following changes has been made by this commit:
    - fix issue when a binary is not able to be found by relative path
    - implement "west debug" command
    - implement --dt-flash option

Signed-off-by: Alex Kolosov <rikorsev@gmail.com>
2022-07-19 12:23:47 +00:00
Anas Nashif
6027152408 scripts: set_assignee: handle exceptions
Deal with exceptions that happen when a user does not exist on GH.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-18 21:41:22 -04:00
Martí Bolívar
d6f68f0bbb scripts: gen_defines: fix tokenization of array elements
This fixes commit
3d5cc38cf6.

That commit only works if removing the quotes from e.g. a string in an
array actually results in a token. If the string's value is something
like

  "foo,bar"

though, it will result in the "token"

  foo,bar

in the generated output.

This is wrong; fix it using the new edtlib.str_as_token() API
introduced to allow callers to perform the same procedure as that
library for fixing up strings.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2022-07-18 17:50:44 -05:00
Martí Bolívar
7b2a728584 scripts: gen_defines: add a missing comment
Adding a comment at this point in the header file splits these macros
into their own section. They are presently grouped together with the
previous section, which is unrelated. This is confusing; fix it.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2022-07-18 17:50:44 -05:00
Martí Bolívar
4ef8c8e447 edtlib: expose str_as_token() API
Some callers need to be able to convert strings to tokens in the same
way edtlib does. Make this possible by exposing the internal helper
function used to do that under a suitable name.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2022-07-18 17:50:44 -05:00
Martí Bolívar
19bb6b330c edtlib: fix error handling in an internal helper
The error message emitted by _interrupt_parent() is wrong; it
mistakenly says:

  node None has an 'interrupts' property, but [...]

This 'None' is appearing because the same routine overwrites the
'node' argument that the caller is asking about with node parents
until it hits the root, at which point root.parent is None.

Fix it by caching the original node and using that in the error
message instead.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2022-07-18 17:50:44 -05:00
Jason Wright
c6c7dde969 scripts: better logging for --coverage-formats option
Adds additional log messages for coverage reports
beyond HTML format.

Signed-off-by: Jason Wright <jwright@synchron.com>
2022-07-18 15:03:13 -04:00
Jason Wright
60c4d20548 scripts: restrict --coverage-formats option to gcovr
Generates an error if --coverage-formats option is supplied without
coverage tool being set to gcovr.

Signed-off-by: Jason Wright <jwright@synchron.com>
2022-07-18 15:03:13 -04:00
Jason Wright
786f28f162 scripts: add --coverage-formats option for use with gcovr
Adds optional twister flag to support gcovr output report formats.

Signed-off-by: Jason Wright <jwright@synchron.com>
2022-07-18 15:03:13 -04:00
Christopher Friedt
3783cf8353 scripts: release: list_backports: use older python dict merge method
In Python versions >= 3.9, dicts can be merged with the `|` operator.

This is not the case for python versions < 3.9, and the simplest way
is to use `dict_c = {**dict_a, **dict_b}`.

Signed-off-by: Christopher Friedt <cfriedt@fb.com>
2022-07-19 00:20:13 +09:00
Christopher Friedt
aa4e437573 ci: backports: check if a backport PR has a valid issue
This is an automated check for the Backports project to
require one or more `Fixes #<issue>` items in the body
of the pull request.

Fixes #46164

Signed-off-by: Christopher Friedt <cfriedt@fb.com>
2022-07-18 20:09:44 +09:00
Christopher Friedt
57762ca12c scripts: release: list_backports.py
Created list_backports.py to examine prs applied to a backport
branch and extract associated issues. This is helpful for
adding to release notes.

The script may also be used to ensure that backported changes
also have one or more associated issues.

Signed-off-by: Christopher Friedt <cfriedt@fb.com>
2022-07-18 20:09:44 +09:00
Christopher Friedt
3b3fc27860 scripts: release: use GITHUB_TOKEN and start_date in scripts
Updated bug_bash.py and list_issues.py to use the GITHUB_TOKEN
environment variable for consistency with other scripts.

Updated bug_bash.py to use `-s / --start-date` instead of
`-b / --begin-date`.

Signed-off-by: Christopher Friedt <cfriedt@fb.com>
2022-07-18 20:09:44 +09:00
Torsten Rasmussen
6066ab43b3 kconfig: add support for warnings when enabling deprecated features
This adds two new Kconfig settings.

The first setting `DEPRECATED` which is a promptless symbol.
This symbol must be selected by any deprecated setting when enabled.

The second setting is `WARN_DEPRECATED` which is a user controlled
setting that configures the build system to print a warning when a
deprecated feature is enabled.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2022-07-15 09:53:08 -05:00
Kumar Gala
b200b9a9e0 dts: bindings: bus labels are now optional
All in tree device drivers on a bus use some form of DEVICE_DT_GET
so we no longer need to require label properties.

Signed-off-by: Kumar Gala <galak@kernel.org>
2022-07-14 09:19:35 -07:00
Chen Peng1
e7088743df scripts: twister: only load connected hardwares.
only load hardwares with connected is true in
hardware map files.

Signed-off-by: Chen Peng1 <peng1.chen@intel.com>
2022-07-13 06:18:08 -04:00
Chen Peng1
1d5c37ab96 script: twister: fix hardware runner_params parsing.
The specific hardware instance is already got from
device_is_available function, we don't need to
iterate all hardwares again.

Signed-off-by: Chen Peng1 <peng1.chen@intel.com>
2022-07-13 06:18:08 -04:00
Anas Nashif
a0c3bdafbf twister: do not modify testplan on subsequent runs
Do not modify testplan.json if it already exists. Testplan is only
created once and any work in the same worskpace should not change it.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-13 10:13:28 +02:00
Anas Nashif
a2fbb783dd twister: fix new ztest parsing
Somehow this change was dropped (rebase?), this is needed to parse new
ztest test correctly, otherwise will have many tests missing from the
plan.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-13 10:13:28 +02:00
Anas Nashif
5f146dcaf9 twister: prep for device testing
Add option to prepare for device testing by only building and storing
the needed data that can then later be executed and run on DUT using
--test-only.

This adds --prep-artifacts-for-testing which is the same as --build-only
and --device-testing and other options that were needed to make this
operational and produce the right binaries and files.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-13 10:13:28 +02:00
Anas Nashif
68a8613a92 twister: fix --build-only argument help
Minor fix to the --build-only option help.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-13 10:13:28 +02:00
Anas Nashif
398e9902b8 twister: rename argument group: serial -> device
using serial as the group name is misleading, so call it 'device' which
is what all those options about.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-13 10:13:28 +02:00
Anas Nashif
e1e43015ec twister: also parse cpp files in app root
Fix wildcard to also scan cpp files in root folder of test application.

Fixes #47220

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-12 23:24:41 -04:00
Gerard Marull-Paretas
0a4b248047 scripts: checkpatch: add Atmel Pwm typedef
Add Atmel Pwm typedef so that when doing `Pwm * const pwm` definition
checkpatch doesn't complain about the position of the '*'. If not added,
we must define `Pwm *const pwm` to make checkpatch happy, but,
clang-format results in `Pwm * const pwm` (correct).

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2022-07-12 17:51:23 +02:00
Anas Nashif
01582f80b7 scripts: move gen_cfb_font_header.py to scripts/build
Move scripts needed by the build system and not designed to be run
individually or standalone into the build subfolder.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-12 10:03:45 +02:00
Anas Nashif
61f672607a scripts: move file2hex.py to scripts/build/file2hex.py
Move scripts needed by the build system and not designed to be run
individually or standalone into the build subfolder.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-12 10:03:45 +02:00
Anas Nashif
92575fdac8 scripts: move parse_syscalls.py to scripts/build/
Move scripts needed by the build system and not designed to be run
individually or standalone into the build subfolder.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-12 10:03:45 +02:00
Anas Nashif
98ab67d7dc scripts: move user_wordsize.py to scripts/build/user_wordsize.py
Move scripts needed by the build system and not designed to be run
individually or standalone into the build subfolder.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-12 10:03:45 +02:00
Anas Nashif
a8a976647e scripts: move uf2conv.py to scripts/build/uf2conv.py
Move scripts needed by the build system and not designed to be run
individually or standalone into the build subfolder.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-12 10:03:45 +02:00
Anas Nashif
9ee1e32b57 scripts: move subfolder_list.py to scripts/build/subfolder_list.py
Move scripts needed by the build system and not designed to be run
individually or standalone into the build subfolder.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-12 10:03:45 +02:00
Anas Nashif
d859997994 scripts: move process_gperf.py to scripts/build
Move scripts needed by the build system and not designed to be run
individually or standalone into the build subfolder.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-12 10:03:45 +02:00
Anas Nashif
72e7fa8176 scripts: move mergehex.py to scripts/build
Move scripts needed by the build system and not designed to be run
individually or standalone into the build subfolder.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-12 10:03:45 +02:00
Anas Nashif
09b4becaee scripts: move gen_image_info.py to scripts/build
Move scripts needed by the build system and not designed to be run
individually or standalone into the build subfolder.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-12 10:03:45 +02:00
Anas Nashif
967f9b437b scripts: move dir_is_writeable.py to scripts/build
Move scripts needed by the build system and not designed to be run
individually or standalone into the build subfolder.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-12 10:03:45 +02:00
Anas Nashif
c74d20e433 scripts: move gen_syscalls.py to scripts/build
Move scripts needed by the build system and not designed to be run
individually or standalone into the build subfolder.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-12 10:03:45 +02:00
Anas Nashif
4f65bf6412 scripts: move gen_strerror_table.py to scripts/build
Move scripts needed by the build system and not designed to be run
individually or standalone into the build subfolder.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-12 10:03:45 +02:00
Anas Nashif
d589e7f04e scripts: move gen_relocate_app.py to scripts/build
Move scripts needed by the build system and not designed to be run
individually or standalone into the build subfolder.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-07-12 10:03:45 +02:00