Commit graph

3879 commits

Author SHA1 Message Date
Håkon Øye Amundsen
325e35036b checkpatch: update --exclude docs
The current docs are incorrect as comma separated paths
are not working. Providing a separate --exclude argement
per path does however work.

Signed-off-by: Håkon Øye Amundsen <haakon.amundsen@nordicsemi.no>
2022-12-01 15:48:30 +01:00
Anas Nashif
1217d850f5 twister: instance: fix layout
Fix indentation of return.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-12-01 06:07:30 -05:00
Anas Nashif
3677a10834 twister: ignore unit tests/EFI images with --show-footprint
Ignore unit tests and EFI binaries when calculating footprints with
--show-footprint, or we will get a build error.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-12-01 06:07:30 -05:00
Gerard Marull-Paretas
2cee5ff519 scripts: west_commands: runners: remove deprecated options
The --snr (nrfjprog) --id (jlink) and --board-id (pyocd) options were
deprecated a long time ago in favor of --dev-id. It is time to remove
them.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2022-11-30 14:15:30 +01:00
Katarzyna Giądła
14358aee1c twister: get information about memory footprint from build.log
This commit calculate memory footprint from build.log and
proposes an alternative approach to #2465.

Signed-off-by: Katarzyna Giądła <katarzyna.giadla@nordicsemi.no>
2022-11-29 16:17:32 -05:00
Evgeniy Paltsev
ae79de1930 ARC: MWDT add TLS support
Add thread local storage support for ARC MWDT toolchain.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
2022-11-29 09:48:23 +01:00
Anas Nashif
ef7d0d6c01 twister: fix simulation conditional
non-simulation platforms set key to 'na', we need to check for that
instead of just checking for existance of key.

Fixes #52595

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-11-28 10:47:32 -08:00
Carles Cufi
76499a6127 scripts: compliance: Move Kconfig-related items to the class
Both get_defined_syms and UNDEF_KCONFIG_WHITELIST are used exclusively
by the Kconfig check, so move them to the class.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-29 01:40:33 +09:00
Carles Cufi
b8a6a510aa scripts: compliance: Resolve ZEPHYR_BASE in _main()
Instead of globally resolving ZEPHYR_BASE, move it to _main() so we can
catch potential exceptions when doing so and for consistency with the
rest of globals, like GIT_TOP. This also includes documenting a new
"<zephyr-base>" path hint for classes.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-29 01:40:33 +09:00
Carles Cufi
e60c1e024b scripts: compliance: Clean up ZEPHYR_BASE usage
The global variable ZEPHYR_BASE is always set in the script, so no need
to check for its existence.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-29 01:40:33 +09:00
Carles Cufi
5f094fbbf5 scripts: compliance: Make line optional in fmtd_failure()
Some errors are attributable to a specific file, but not a specific line
within that file. Allow for formatted errors that refer to a file but
not a line.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-29 01:40:33 +09:00
Carles Cufi
28c2932a1d scripts: compliance: Consolidate 'git diff' calls in get_files()
Many tests invoke `git diff` independently with very similar parameters
to obtain a filtered list of files.
Consolidate those into a new `get_files()` function.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-29 01:40:33 +09:00
Carles Cufi
5fbfd72f9d scripts: compliance: Remove old TODO
This TODO doesn't seem to apply anymore based on its description, so
remove it altogether.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-29 01:40:33 +09:00
Carles Cufi
bafec6bbde scripts: compliance: Remove unneeded workaround
The workaround for old pytest bug
https://github.com/PyCQA/pylint/issues/2906
is not required anymore after more than 3 years, so remove it.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-29 01:40:33 +09:00
Anas Nashif
808266a493 tests: use ignore_fault field instead of tags
Use dedicated field in the yaml file instead of mixing this testing
feature with tags.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-11-25 06:38:05 -05:00
Anas Nashif
a90d711c7d twister: add simulation handler
Simplify the code a bit by introducing a simulator handler class.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-11-25 06:09:02 -05:00
Anas Nashif
dae4720703 boards: native: mark as simulator
Mark native_posix platform as a simulator.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-11-25 06:09:02 -05:00
Anas Nashif
3bd819c09d twister: treat default platforms without a mean to run as any other
If a default platform is not able to run for whatever reason, treat it
as any other platform. Default platforms shall ALWAYS be runnable in CI,
or else we are just re-building for no good reason.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-11-25 06:09:02 -05:00
Anas Nashif
3e870495f2 boards: add simulation executable for 3rd party sims
Provide the information as part of the board yaml file, this way we will
know what binary to check for.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-11-25 06:09:02 -05:00
Hake Huang
70553a64a3 test: twister: add more exception protection
the ser.in_wait sometime meet issue like serial handler error.
This is because the serial port is in reset status.
add final protection to retry later

error log:

Traceback (most recent call last):
  File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "zephyr/scripts/pylib/twister/twisterlib/handlers.py", line 345,
    in monitor_serial
    if not ser.in_waiting:
  File "~/python3.10/site-packages/serial/serialposix.py", line 549,
     in in_waiting
    s = fcntl.ioctl(self.fd, TIOCINQ, TIOCM_zero_str)
OSError: [Errno 9] Bad file descriptor

Signed-off-by: Hake Huang <hake.huang@oss.nxp.com>
2022-11-25 10:46:30 +01:00
Carles Cufi
a4600b0470 scripts: compliance: Use f-strings when appropriate
There were many instances in the code where f-strings were more
appropriate or fitting, change those to use them.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-24 09:34:25 +01:00
Carles Cufi
ba295755ce scripts: compliance: Add Nordic copyright
Add Nordic Semi as a contributor to this file.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-24 09:34:25 +01:00
Carles Cufi
4654bb6a1e scripts: compliance: Clean up import statements
Sort them alphabetically and by stdlib vs 3rd-party modules.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-24 09:34:25 +01:00
Carles Cufi
d9aa61f74e scripts: compliance: Use subprocess.run()
As per the current Python documentation, use subprocess.run() since the
other, older methods are not recommended anymore.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-24 09:34:25 +01:00
Carles Cufi
b7cb3af443 scripts: compliance: checkpatch: Fix regex
The checkpatch regex matched too greedily, so that descriptions with ":"
would match too much, example:

-:20: ERROR:SPACING: spaces required around that '=' (ctx:VxW)

Would match "SPACING: spaces required around that '=' (ctx" instead of
just "SPACING".

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-24 09:34:25 +01:00
Andrzej Głąbek
6c21c9635d scripts: gen_relocate_app: Fix a typo in parse_input_string()
This is a follow-up to commit b1a3ce4016.

`parse_input_string()` needs to use `:NOCOPY:`, not `:NOCOPY`, when
partitioning input lines, otherwise, when a line contains the NOCOPY
flag, the file name returned by the function starts with `:` and the
file cannot be then found by the script.
Such problem can be observed in the code_relocation_nocopy sample,
which without this fix does not actually relocate code from ext_code.c.

Signed-off-by: Andrzej Głąbek <andrzej.glabek@nordicsemi.no>
2022-11-23 16:49:03 -05:00
Fabio Baltieri
7b57d3c5bb scripts: compliance: check for binary files
Add a compliance check that fails if the diff is adding any binary file
to prevent committing them by mistake.

Only check for added files and explicitly allow image files in doc/ and
boards/.

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2022-11-21 16:12:04 -05:00
Carles Cufi
49b01b39b6 scripts: compliance: Remove unconditional debug output
This prevents us from parsing the scripts' output correctly.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-21 11:02:39 -05:00
Carles Cufi
66ad0e68a6 scripts: compliance: Use consistent naming convention
Tests are named with CamelCase in general, use that in the names.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-21 11:02:39 -05:00
Carles Cufi
8c12efe2e5 scripts: compliance: Make test filtering case insensitive
To simplify using the -m and -e command-line options make the test
filtering case insensitive.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-21 11:02:39 -05:00
Carles Cufi
95053c8358 scripts: compliance: Fix DT Bindings check name
Check names must not contain spaces, since they are passed via `-m` in
the command-line.

Also update .gitignore and compliance.yml.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-21 11:02:39 -05:00
Carles Cufi
8275457709 scripts: compliance: Fix wrong call
Fix call to parse_dt_bindings() in the Devicetree Bindings check.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-21 11:02:39 -05:00
Veijo Pesonen
606515c6c4 scripts: requirements: extras: add zcbor
Add ZCBOR to the requirements-extras.txt and require at least version
0.6.0 which is the current version of the ZCBOR C library.

Signed-off-by: Veijo Pesonen <veijo.pesonen@nordicsemi.no>
2022-11-21 16:11:04 +01:00
Cong Nguyen Huu
c304db56f1 twister: add option cleanup all tests
add 2 choices for '--runtime-artifact-cleanup' option,
'passing' to delete artifacts of passing tests,
'all' to delete artifacts of both passing tests
and failing tests

keep testcase_extra.config, it contains some extra Kconfig
needed for the tests and they is only generated first time
when run retry failed

Signed-off-by: Cong Nguyen Huu <cong.nguyenhuu@nxp.com>
2022-11-21 08:39:13 -05:00
Daniel DeGrasse
6aa7194d48 scripts: kconfig: add dt_node_has_compat kconfig function
Add dt_node_has_compat kconfig function, to check if a node path has a
given compatible string while parsing kconfig files

Signed-off-by: Daniel DeGrasse <daniel.degrasse@nxp.com>
2022-11-21 13:03:26 +01:00
Daniel DeGrasse
db9bcd90ce scripts: kconfig: add dt_node_parent kconfig function
Add dt_node_parent kconfig function, to get parent path for a given
devicetree path when parsing kconfig files

Signed-off-by: Daniel DeGrasse <daniel.degrasse@nxp.com>
2022-11-21 13:03:26 +01:00
Chris Friedt
5e7a86e1f9 scripts: compliance: add devicetree binding compliance check
Add `DevicetreeBindingsCheck` and `required_false_check()`, which
checks to make sure that no new Devicetree bindings add the
redundant `required: false` property.

Signed-off-by: Chris Friedt <cfriedt@meta.com>
2022-11-20 13:12:44 -05:00
Anas Nashif
ae24329daa twister: show duplicates correctly
Find duplicate test identifiers and report them with
--list-test-duplicates.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-11-18 09:19:47 -05:00
Gerard Marull-Paretas
537133fe44 devicetree: edtlib: tests: add coverage for child includes
Add test coverage for the child-binding include feature. It includes
verification of included properties as well as usage of allow/blocklist.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2022-11-17 06:57:46 -06:00
Joe Perches
6cdeefcabb checkpatch: improve the indented label test
checkpatch identifies a label only when a terminating colon
immediately follows an identifier.

Bitfield definitions can appear to be labels so ignore any
spaces between the identifier terminating colon and any digit
that may be used to define a bitfield length.

Miscellanea:

o Improve the initial checkpatch comment
o Use the more typical '&&' instead of 'and'
o Require the initial label character to be a non-digit
  (Can't use $Ident here because $Ident allows ## concatenation)
o Use $sline instead of $line to ignore comments
o Use '$sline !~ /.../' instead of '!($line =~ /.../)'

Link: https://lkml.kernel.org/r/b54d673e7cde7de5de0c9ba4dd57dd0858580ca4.camel@perches.com
Signed-off-by: Joe Perches <joe@perches.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Manikishan Ghantasala <manikishanghantasala@gmail.com>
Cc: Alex Elder <elder@ieee.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Stephanos Ioannidis <stephanos.ioannidis@nordicsemi.no>
2022-11-17 01:02:03 +09:00
Daniel DeGrasse
a766e4c4fb scripts: Fix header ifdef guard for gen_image_info.py
gen_image_info.py was generating invalid include guard. Properly strip
all special characters from filename to generate include guard #ifdef

Signed-off-by: Daniel DeGrasse <daniel.degrasse@nxp.com>
2022-11-15 14:46:02 +01:00
Daniel DeGrasse
c303c6fae7 scripts: pylib: remove west log dependency from domains.py
Convert domains.py to use basic python logging module implementation to
remove west logging dependency. This removes hard dependency on west from
twister.

Fixes #51731

Signed-off-by: Daniel DeGrasse <daniel.degrasse@nxp.com>
2022-11-14 10:12:31 +00:00
Maciej Perkowski
45e1ff94cd twister: tests: Add unit tests for names of external and module tests
Add test cases verifing that test id for external and module tests is
not --testsuite-root dependant.

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2022-11-13 17:45:09 -05:00
Maciej Perkowski
21eb27c5c0 twister: Update path structure for tests
With the patch a relative path between a test and west's project top
directory is always calculated instead of being relative to zephyr
base. The test names and results structur change by addition of one
level. This way the names/structure conventon for tests from zephyr and
modules is alligned.
For external tests ".." parts are replaced with "external".
This allows to maintain unified folder structure also for results of
external tests.
Without this patch the folder/name allignment is lost for
external and module test becaming depandant on a way how
twister was called.
.

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2022-11-13 17:45:09 -05:00
Maciej Perkowski
e27c9fe6b9 twister: tests: Fix os.path.join usage in twister unit test
From docs "If a component is an absolute path, all previous
components are thrown away and joining continues from the
absolute path component." This commit fixes the args and
corrects the expected value.
https://docs.python.org/3/library/os.path.html#os.path.join

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2022-11-13 17:45:09 -05:00
Carles Cufi
1e203ea15e scripts: compliance: Ensure lxml is installed
See https://github.com/weiwei/junitparser/issues/99.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-13 06:25:59 -05:00
Carles Cufi
1951071f14 requirements: Add lxml as a requirement
junitparser has a bug that prevents it from working correctly when lxml
is not installed.

See this GH issue for more information:
https://github.com/weiwei/junitparser/issues/99

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-13 06:25:59 -05:00
Carles Cufi
1ee0d60990 scripts: compliance: Parse the pylint output
Parse the output generated by pylint and store it as FmtdFailure
instances, so that they can be annotated later.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-13 06:25:59 -05:00
Carles Cufi
edd2c83dd4 scripts: compliance: Modernize pylint invocation
The current recommended API call for subprocess instantiation is run(),
so replace the low-level Popen and communicate() with it.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-13 06:25:59 -05:00
Carles Cufi
8275d21a6c scripts: compliance: Add GitHub-compatible annotation support
Add support for GitHub workflow annotations, so that errors are
displayed inline with the Pull Request diff.

More info here:
https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#about-workflow-commands

Enable the new option in the compliance GitHub action.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-13 06:25:59 -05:00