Commit graph

249 commits

Author SHA1 Message Date
Fabio Baltieri
9d3681c10c scripts: compliance: add support for YAMLLint
Add a YAMLLint compliance check that uses the yamllint package to report
linting error on YAML files.

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2023-01-04 17:29:23 +01:00
Fabio Baltieri
0daa3a78a9 yamllint: indentation: fix files in scripts/
Fix the YAML files indentation for files in scripts/.

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2023-01-04 14:23:53 +01:00
Fabio Baltieri
7dd902d035 yamllint: fix all yamllint comments-indentation errors
Fix all comments-indentation errors detected by yamllint:

yamllint -f parsable -c .yamllint $( find -regex '.*\.y[a]*ml' ) | \
  grep '(comments-indentation)'

This checks that the comment is aligned with the content.

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2023-01-04 01:16:45 +09:00
Fabio Baltieri
f33d43795d scripts: compliance: add a check for MAINTAINERS.yml format
Add a compliance check that tries to load MAINTAINERS.yml with
get_maintainer.Maintainers() if it's been modified by the CL, and fail
compliance if it fails to be parsed.

Example output:

```
ERROR   : Test MaintainersFormat failed:
Error parsing MAINTAINERS.yml: MAINTAINERS.yml: YAML error: while
scanning a simple key
  in "MAINTAINERS.yml", line 976, column 1
could not find expected ':'
  in "MAINTAINERS.yml", line 977, column 3
```

```
ERROR   : Test MaintainersFormat failed:
Error parsing MAINTAINERS.yml: MAINTAINERS.yml: glob pattern
'drivers/regulator' in 'files' in area 'Drivers: Regulators' matches a
directory, but has no trailing '/'
```

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2022-12-29 14:42:02 +01:00
Fabio Baltieri
e7a4aec3c9 scripts: compliance: check for bindings in subdirectories
Change the bindings file filter to check for "dts/bindings/" as a
substring rather than prefix. This makes the check catch files in
subdirectories as well (for sample tests and samples), and makes the
check usable on external projects where the bindings may not live in the
project root.

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2022-12-28 19:01:57 +01:00
Fabio Baltieri
5bf79ed9cd scripts: compliance: use get_files filter for dts checks
Drop the custom logic to skip deleted files, use get_files filter
instead.

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2022-12-28 19:01:57 +01:00
Fabio Baltieri
afc2ad34bd scripts: compliance: move binding file cycle in run()
Move the file iteration loops of the bindings compliance check in the
callers, so that if we add more checks we don't have iterate on each one
of them and a check works on a single file.

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2022-12-28 19:01:57 +01:00
Fabio Baltieri
ab4d87dd41 scripts: compliance: add an ImageSize check
Add a check to validate and limit the size of images in the repository,
setting stricter limit for files in boards/.

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2022-12-22 17:01:54 +01:00
Fabio Baltieri
30fd28d9f0 scripts: compliance: move isfile() check in get_files()
Git normally only works on files, but can include directories if there's
any submodule in the repository. This is currently checked in
filter_py(), move the check into get_files() so it does not have to be
copied around.

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2022-12-22 17:01:54 +01:00
Fabio Baltieri
5d23505336 scripts: compliance: fix DevicetreeBindingsCheck
DevicetreeBindingsCheck is missing the splitlines() call to process the
get_files() output, hence cycling through each character of the output
rather than each file, which causes the check to never run on anything.

Fix it by moving the splitline call into get_files() itself, since every
caller is going to use it anyway.

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2022-12-22 17:01:54 +01:00
Carles Cufi
c0455b836f scripts: ci: compliance: Remove Codeowners check
This check is no longer used and will be replaced by the maintainers
file, remove it.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-12-20 17:34:35 +01:00
Stephanos Ioannidis
0e5762e56e scripts: compliance: Fix misdetection of Kconfig choices as undefined
This commit updates the KconfigCheck to add the Kconfig symbols from
both "config" and "choice" symbol lists to the defined symbol list, as
opposed to the "config" list only, in order to prevent any references
to the choice symbols outside the Kconfig files (e.g. in documentation)
from being reported as undefined.

Signed-off-by: Stephanos Ioannidis <stephanos.ioannidis@nordicsemi.no>
2022-12-13 17:20:10 +09:00
Anas Nashif
bbf03fd991 ci: twister: fixed path for tag based filtering
path for posix was not being matched correctly due to a missing /. So
some posix tests were excluded and not excercised.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-12-02 07:31:37 -05:00
Katarzyna Giądła
14358aee1c twister: get information about memory footprint from build.log
This commit calculate memory footprint from build.log and
proposes an alternative approach to #2465.

Signed-off-by: Katarzyna Giądła <katarzyna.giadla@nordicsemi.no>
2022-11-29 16:17:32 -05:00
Evgeniy Paltsev
ae79de1930 ARC: MWDT add TLS support
Add thread local storage support for ARC MWDT toolchain.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
2022-11-29 09:48:23 +01:00
Carles Cufi
76499a6127 scripts: compliance: Move Kconfig-related items to the class
Both get_defined_syms and UNDEF_KCONFIG_WHITELIST are used exclusively
by the Kconfig check, so move them to the class.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-29 01:40:33 +09:00
Carles Cufi
b8a6a510aa scripts: compliance: Resolve ZEPHYR_BASE in _main()
Instead of globally resolving ZEPHYR_BASE, move it to _main() so we can
catch potential exceptions when doing so and for consistency with the
rest of globals, like GIT_TOP. This also includes documenting a new
"<zephyr-base>" path hint for classes.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-29 01:40:33 +09:00
Carles Cufi
e60c1e024b scripts: compliance: Clean up ZEPHYR_BASE usage
The global variable ZEPHYR_BASE is always set in the script, so no need
to check for its existence.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-29 01:40:33 +09:00
Carles Cufi
5f094fbbf5 scripts: compliance: Make line optional in fmtd_failure()
Some errors are attributable to a specific file, but not a specific line
within that file. Allow for formatted errors that refer to a file but
not a line.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-29 01:40:33 +09:00
Carles Cufi
28c2932a1d scripts: compliance: Consolidate 'git diff' calls in get_files()
Many tests invoke `git diff` independently with very similar parameters
to obtain a filtered list of files.
Consolidate those into a new `get_files()` function.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-29 01:40:33 +09:00
Carles Cufi
5fbfd72f9d scripts: compliance: Remove old TODO
This TODO doesn't seem to apply anymore based on its description, so
remove it altogether.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-29 01:40:33 +09:00
Carles Cufi
bafec6bbde scripts: compliance: Remove unneeded workaround
The workaround for old pytest bug
https://github.com/PyCQA/pylint/issues/2906
is not required anymore after more than 3 years, so remove it.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-29 01:40:33 +09:00
Carles Cufi
a4600b0470 scripts: compliance: Use f-strings when appropriate
There were many instances in the code where f-strings were more
appropriate or fitting, change those to use them.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-24 09:34:25 +01:00
Carles Cufi
ba295755ce scripts: compliance: Add Nordic copyright
Add Nordic Semi as a contributor to this file.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-24 09:34:25 +01:00
Carles Cufi
4654bb6a1e scripts: compliance: Clean up import statements
Sort them alphabetically and by stdlib vs 3rd-party modules.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-24 09:34:25 +01:00
Carles Cufi
d9aa61f74e scripts: compliance: Use subprocess.run()
As per the current Python documentation, use subprocess.run() since the
other, older methods are not recommended anymore.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-24 09:34:25 +01:00
Carles Cufi
b7cb3af443 scripts: compliance: checkpatch: Fix regex
The checkpatch regex matched too greedily, so that descriptions with ":"
would match too much, example:

-:20: ERROR:SPACING: spaces required around that '=' (ctx:VxW)

Would match "SPACING: spaces required around that '=' (ctx" instead of
just "SPACING".

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-24 09:34:25 +01:00
Fabio Baltieri
7b57d3c5bb scripts: compliance: check for binary files
Add a compliance check that fails if the diff is adding any binary file
to prevent committing them by mistake.

Only check for added files and explicitly allow image files in doc/ and
boards/.

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2022-11-21 16:12:04 -05:00
Carles Cufi
49b01b39b6 scripts: compliance: Remove unconditional debug output
This prevents us from parsing the scripts' output correctly.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-21 11:02:39 -05:00
Carles Cufi
66ad0e68a6 scripts: compliance: Use consistent naming convention
Tests are named with CamelCase in general, use that in the names.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-21 11:02:39 -05:00
Carles Cufi
8c12efe2e5 scripts: compliance: Make test filtering case insensitive
To simplify using the -m and -e command-line options make the test
filtering case insensitive.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-21 11:02:39 -05:00
Carles Cufi
95053c8358 scripts: compliance: Fix DT Bindings check name
Check names must not contain spaces, since they are passed via `-m` in
the command-line.

Also update .gitignore and compliance.yml.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-21 11:02:39 -05:00
Carles Cufi
8275457709 scripts: compliance: Fix wrong call
Fix call to parse_dt_bindings() in the Devicetree Bindings check.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-21 11:02:39 -05:00
Chris Friedt
5e7a86e1f9 scripts: compliance: add devicetree binding compliance check
Add `DevicetreeBindingsCheck` and `required_false_check()`, which
checks to make sure that no new Devicetree bindings add the
redundant `required: false` property.

Signed-off-by: Chris Friedt <cfriedt@meta.com>
2022-11-20 13:12:44 -05:00
Carles Cufi
1e203ea15e scripts: compliance: Ensure lxml is installed
See https://github.com/weiwei/junitparser/issues/99.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-13 06:25:59 -05:00
Carles Cufi
1ee0d60990 scripts: compliance: Parse the pylint output
Parse the output generated by pylint and store it as FmtdFailure
instances, so that they can be annotated later.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-13 06:25:59 -05:00
Carles Cufi
edd2c83dd4 scripts: compliance: Modernize pylint invocation
The current recommended API call for subprocess instantiation is run(),
so replace the low-level Popen and communicate() with it.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-13 06:25:59 -05:00
Carles Cufi
8275d21a6c scripts: compliance: Add GitHub-compatible annotation support
Add support for GitHub workflow annotations, so that errors are
displayed inline with the Pull Request diff.

More info here:
https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#about-workflow-commands

Enable the new option in the compliance GitHub action.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-13 06:25:59 -05:00
Carles Cufi
89c3885c28 scripts: compliance: Add a new formatted failure method and class
Format the JUnit failure according to the standard:
https://www.ibm.com/docs/en/developer-for-zos/14.2?topic=formats-junit-xml-format

This also includes a new FmtdFailure class, which will be used in
subsequent commits to implement GitHub-compatible annotations. A list of
FmtdFailure instances is kept in the ComplianceTest instance because
the TestCase instance cannot restore classes derived from Result that
the junitparser module doesn't know about.

Use the new function to format checkpatch errors.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-13 06:25:59 -05:00
Carles Cufi
e9415cf929 scripts: compliance: Rename add_failure to failure
For consistency with error() and skip(), rename add_failure() to
failure().

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-13 06:25:59 -05:00
Carles Cufi
2e66fac6d3 scripts: compliance: Fix inheritance of KconfigBasicCheck
KconfigBasic should not be inheriting from both KconfigCheck and
ComplianceTest. This was a workaround to find all inheritors of
ComplianceTest, so fix the inheritors search instead.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-08 07:20:21 -05:00
Carles Cufi
aed9622a1e scripts: compliance: Allow for custom message and type
Instead of arbitrarily storing the string provided by the caller in
either the message attribute or the element text, let the reporter
choose instead.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-08 07:20:21 -05:00
Carles Cufi
72fde1b9b3 scripts: compliance: Generate one Result per issue encountered
Instead of accumulating failure information in a single Result instance,
located at index 0, generate one Result sub-class instance per
failure/error/skip as it is intended in the original library.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-08 07:20:21 -05:00
Carles Cufi
5ae70b9b77 scripts: compliance: Use standard TestCase class
No need to define a custom class anymore.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-08 07:20:21 -05:00
Carles Cufi
f090efa793 scripts: compliance: Remove unused info message handling
No need to keep this in the tree if it's not used at all.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-08 07:20:21 -05:00
Carles Cufi
c3933561b0 scripts: compliance: Use text setter/getters
The junitparser Python module now comes with a setter and getter for
Result.text, use them instead of peeking into the underlying XML.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-08 07:20:21 -05:00
Anas Nashif
e2cd634185 ci: also upload results of twiser a week full builds
Upload results of full runs we do twice a week to opensearch.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-11-07 14:20:02 -06:00
Anas Nashif
662aed81d2 actions: Upload test results to opensearch service
On push, upload test results to opensearch for analysis and reporting.
Goal is to use this data to understand test coverage better and use this
services for all test reporting, also for results coming from testing on
hardware.
Opensearch is currenly being used for evaluation, we are considering
the switch to elasticsearch later.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-11-02 10:26:43 -04:00
Torsten Rasmussen
36ea41fad3 samples: introduce 'with_mcuboot' sample to show sysbuild feature
This sample is intended to show users how they can create a sample which
has dedicated configuration files for extra images included in a build.

MCUboot is used as example on how its default can be adjusted and
MCUboot itself automatically be included in the build when using
sysbuild.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2022-10-18 14:13:27 +02:00
Anas Nashif
d8197f7535 ci: testplan: error out if we find mis-configured tests
If we find errors during test plan creation, stop execution and report
out the issues.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-10-13 06:23:46 -04:00