Commit graph

196 commits

Author SHA1 Message Date
Fabio Baltieri
9de541b9c2 scripts: ci: check_compliance: add a no-modules Kconfig check
Add a variation of the basic Kconfig check that runs with no modules,
catches symbols that are used in the main repository but are defined
only in modules, which are potentially problematic if a downstream
project is not using the specific module.

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2023-07-05 09:11:02 +02:00
Carles Cufi
ff2d9cfcb6 scripts: compliance: Fix handling of integer node items
Some node items in Kconfig can be kconfiglib.MENU or kconfiglib.COMMENT.
Those are integers and thus do not contain a node.item.name field.
Handle those separately to avoid hitting the followig exception:

Traceback (most recent call last):
  File "/home/runner/work/zephyr/zephyr/./scripts/ci/check_compliance.py",\
  line 1307, in main
    n_fails = _main(args)
  File "/home/runner/work/zephyr/zephyr/./scripts/ci/check_compliance.py",\
  line 1242, in _main
    test.run()
  File "/home/runner/work/zephyr/zephyr/./scripts/ci/check_compliance.py",\
  line 277, in run
    self.check_no_redefined_in_defconfig(kconf)
  File "/home/runner/work/zephyr/zephyr/./scripts/ci/check_compliance.py",\
  line 445, in check_no_redefined_in_defconfig
    Kconfig node '{node.item.name}' found with prompt or help in\
    {node.filename}.
AttributeError: 'int' object has no attribute 'name'

Seen in #58454.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2023-07-04 08:23:02 +02:00
Jamie McCrae
65c9058ed4 sysbuild: Fix value propagation of signature type to MCUboot
Fixes an issue whereby the type of the signature was not passed
to MCUboot.

Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
2023-06-08 06:51:33 -04:00
Torsten Rasmussen
db89e7699d sysbuild: generate .config.sysbuild for controlling build settings
Sysbuild now generates a .config.sysbuild config file which specifies
settings controlled by sysbuild.
Any setting specified in this .config will overrule user provided
setting, and a warning will be raised if the sysbuild controlled value
is different from the value specified by the user.

This has the following benefits:
- Allow sysbuild to control any build setting without adjustments to
  the existing Kconfig tree
- Allow sysbuild to adjust settings based on knowledge regarding enabled
  images / bootloaders.
- Cleanup CMake code, as settings in sysbuild no longer needs to be
  propagated using CMake cache variables.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2023-05-25 14:57:41 +02:00
Filip Kokosinski
442a452903 scripts/ci/check_compliance: add GitDiffCheck compliance check
This commit adds a new `GitDiffCheck` compliance check that checks the
newly added commits with `git diff --check` and reports them back if an
error is found.

This check is needed for some files (e.g. Kconfig) as they are not checked
by `Kconfig` and `KconfigBasic` checks on every commit in a pull request.

Signed-off-by: Filip Kokosinski <fkokosinski@antmicro.com>
2023-05-19 14:52:49 +00:00
Jamie McCrae
96a9b7e1ff scripts: ci: check_compliance: Ignore mcuboot Kconfigs
Adds mcuboot Kconfig options to the ignore list as these are
defined in mcuboot but used in documentation and samples which
utilise sysbuild to build mcuboot.

Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
2023-04-24 13:27:53 +02:00
Magdalena Kasenberg
661f07ba0f bluetooth: tester: Add CONFIG_BTTESTER_LOG_LEVEL
to set a different log level than the default.

Signed-off-by: Magdalena Kasenberg <magdalena.kasenberg@codecoup.pl>
2023-04-12 13:05:32 +02:00
Grzegorz Chwierut
482c94819f tests: boot: test_mcuboot: Add mcuboot log level
Added logging of MCUboot messages to fix twister test scenario.
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
2023-04-06 18:51:23 +02:00
Daniel Leung
4d914f4f89 toolchain: move CONFIG_LLVM_USE_LD into cmake/toolchain/llvm
This moves CONFIG_LLVM_USE_LD into cmake/toolchain/llvm as this
is a toolchain kconfig. Also make it a choice to allow the use
of LLVM's lld as linker.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-03-29 19:48:47 -04:00
Jamie McCrae
ef146c3dbd scripts: compliance: Add MCUboot Kconfig to allow list
Adds a Kconfig for MCUboot which is used in the documentation to
the CI Kconfig check allow list.

Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
2023-03-29 11:30:36 +02:00
Jason Kridner
f36be35600 scripts: compliance: allow webp image files
WebP images are smaller for similar resolutions and quality while being
compatible with web browsers and other image processing tools.

For higher resolution images, be sure to limit the display size to make
the rendered pages look reasonable.

This was approved by Benjamin Cabé (@kartben) in this discussion thread:
https://github.com/zephyrproject-rtos/zephyr/pull/55488#issuecomment-1461792751

Signed-off-by: Jason Kridner <jkridner@beagleboard.org>
2023-03-21 16:03:43 -04:00
Grant Ramsay
090b13cf1e scripts: ci: check_compliance: Ignore undefined CONFIG_CMD_CACHE
CONFIG_CMD_CACHE is a U-Boot config, mentioned in docs.

Signed-off-by: Grant Ramsay <gramsay@enphaseenergy.com>
2023-02-24 18:11:56 +01:00
Fabio Baltieri
317b3c6897 scripts: compliance: pass argv explicitly to parse_args
Change the check_compliance main invocation to pass an explicit argv
list from main to parse_args(). This does not change anything for direct
invocation, but allows wrapping the compliance script and using it as a
library while controlling its behavior by passing the intended flags to
main().

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2023-02-01 17:17:34 +09:00
Jamie McCrae
d72e135b2b scripts: ci: Enable pylint check for argument parser abbreviations
Enables running CI checks which will now search for initialisation
of argument parser where abbreviations are not disabled.

Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
2023-01-26 20:12:36 +09:00
Jamie McCrae
ec7044437e treewide: Disable automatic argparse argument shortening
Disables allowing the python argparse library from automatically
shortening command line arguments, this prevents issues whereby
a new command is added and code that wrongly uses the shortened
command of an existing argument which is the same as the new
command being added will silently change script behaviour.

Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
2023-01-26 20:12:36 +09:00
Fabio Baltieri
d428c8c836 scripts: compliance: support not writing output to files
Support running the compliance check script without writing the output
to any file.

This adds a check for the --output flag to not write to the output file
if the value is empty, and add a second flag for inhibiting the
individual case files.

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2023-01-26 10:10:58 +00:00
Fabio Baltieri
9d3681c10c scripts: compliance: add support for YAMLLint
Add a YAMLLint compliance check that uses the yamllint package to report
linting error on YAML files.

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2023-01-04 17:29:23 +01:00
Fabio Baltieri
f33d43795d scripts: compliance: add a check for MAINTAINERS.yml format
Add a compliance check that tries to load MAINTAINERS.yml with
get_maintainer.Maintainers() if it's been modified by the CL, and fail
compliance if it fails to be parsed.

Example output:

```
ERROR   : Test MaintainersFormat failed:
Error parsing MAINTAINERS.yml: MAINTAINERS.yml: YAML error: while
scanning a simple key
  in "MAINTAINERS.yml", line 976, column 1
could not find expected ':'
  in "MAINTAINERS.yml", line 977, column 3
```

```
ERROR   : Test MaintainersFormat failed:
Error parsing MAINTAINERS.yml: MAINTAINERS.yml: glob pattern
'drivers/regulator' in 'files' in area 'Drivers: Regulators' matches a
directory, but has no trailing '/'
```

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2022-12-29 14:42:02 +01:00
Fabio Baltieri
e7a4aec3c9 scripts: compliance: check for bindings in subdirectories
Change the bindings file filter to check for "dts/bindings/" as a
substring rather than prefix. This makes the check catch files in
subdirectories as well (for sample tests and samples), and makes the
check usable on external projects where the bindings may not live in the
project root.

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2022-12-28 19:01:57 +01:00
Fabio Baltieri
5bf79ed9cd scripts: compliance: use get_files filter for dts checks
Drop the custom logic to skip deleted files, use get_files filter
instead.

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2022-12-28 19:01:57 +01:00
Fabio Baltieri
afc2ad34bd scripts: compliance: move binding file cycle in run()
Move the file iteration loops of the bindings compliance check in the
callers, so that if we add more checks we don't have iterate on each one
of them and a check works on a single file.

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2022-12-28 19:01:57 +01:00
Fabio Baltieri
ab4d87dd41 scripts: compliance: add an ImageSize check
Add a check to validate and limit the size of images in the repository,
setting stricter limit for files in boards/.

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2022-12-22 17:01:54 +01:00
Fabio Baltieri
30fd28d9f0 scripts: compliance: move isfile() check in get_files()
Git normally only works on files, but can include directories if there's
any submodule in the repository. This is currently checked in
filter_py(), move the check into get_files() so it does not have to be
copied around.

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2022-12-22 17:01:54 +01:00
Fabio Baltieri
5d23505336 scripts: compliance: fix DevicetreeBindingsCheck
DevicetreeBindingsCheck is missing the splitlines() call to process the
get_files() output, hence cycling through each character of the output
rather than each file, which causes the check to never run on anything.

Fix it by moving the splitline call into get_files() itself, since every
caller is going to use it anyway.

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2022-12-22 17:01:54 +01:00
Carles Cufi
c0455b836f scripts: ci: compliance: Remove Codeowners check
This check is no longer used and will be replaced by the maintainers
file, remove it.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-12-20 17:34:35 +01:00
Stephanos Ioannidis
0e5762e56e scripts: compliance: Fix misdetection of Kconfig choices as undefined
This commit updates the KconfigCheck to add the Kconfig symbols from
both "config" and "choice" symbol lists to the defined symbol list, as
opposed to the "config" list only, in order to prevent any references
to the choice symbols outside the Kconfig files (e.g. in documentation)
from being reported as undefined.

Signed-off-by: Stephanos Ioannidis <stephanos.ioannidis@nordicsemi.no>
2022-12-13 17:20:10 +09:00
Evgeniy Paltsev
ae79de1930 ARC: MWDT add TLS support
Add thread local storage support for ARC MWDT toolchain.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
2022-11-29 09:48:23 +01:00
Carles Cufi
76499a6127 scripts: compliance: Move Kconfig-related items to the class
Both get_defined_syms and UNDEF_KCONFIG_WHITELIST are used exclusively
by the Kconfig check, so move them to the class.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-29 01:40:33 +09:00
Carles Cufi
b8a6a510aa scripts: compliance: Resolve ZEPHYR_BASE in _main()
Instead of globally resolving ZEPHYR_BASE, move it to _main() so we can
catch potential exceptions when doing so and for consistency with the
rest of globals, like GIT_TOP. This also includes documenting a new
"<zephyr-base>" path hint for classes.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-29 01:40:33 +09:00
Carles Cufi
e60c1e024b scripts: compliance: Clean up ZEPHYR_BASE usage
The global variable ZEPHYR_BASE is always set in the script, so no need
to check for its existence.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-29 01:40:33 +09:00
Carles Cufi
5f094fbbf5 scripts: compliance: Make line optional in fmtd_failure()
Some errors are attributable to a specific file, but not a specific line
within that file. Allow for formatted errors that refer to a file but
not a line.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-29 01:40:33 +09:00
Carles Cufi
28c2932a1d scripts: compliance: Consolidate 'git diff' calls in get_files()
Many tests invoke `git diff` independently with very similar parameters
to obtain a filtered list of files.
Consolidate those into a new `get_files()` function.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-29 01:40:33 +09:00
Carles Cufi
5fbfd72f9d scripts: compliance: Remove old TODO
This TODO doesn't seem to apply anymore based on its description, so
remove it altogether.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-29 01:40:33 +09:00
Carles Cufi
bafec6bbde scripts: compliance: Remove unneeded workaround
The workaround for old pytest bug
https://github.com/PyCQA/pylint/issues/2906
is not required anymore after more than 3 years, so remove it.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-29 01:40:33 +09:00
Carles Cufi
a4600b0470 scripts: compliance: Use f-strings when appropriate
There were many instances in the code where f-strings were more
appropriate or fitting, change those to use them.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-24 09:34:25 +01:00
Carles Cufi
ba295755ce scripts: compliance: Add Nordic copyright
Add Nordic Semi as a contributor to this file.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-24 09:34:25 +01:00
Carles Cufi
4654bb6a1e scripts: compliance: Clean up import statements
Sort them alphabetically and by stdlib vs 3rd-party modules.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-24 09:34:25 +01:00
Carles Cufi
d9aa61f74e scripts: compliance: Use subprocess.run()
As per the current Python documentation, use subprocess.run() since the
other, older methods are not recommended anymore.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-24 09:34:25 +01:00
Carles Cufi
b7cb3af443 scripts: compliance: checkpatch: Fix regex
The checkpatch regex matched too greedily, so that descriptions with ":"
would match too much, example:

-:20: ERROR:SPACING: spaces required around that '=' (ctx:VxW)

Would match "SPACING: spaces required around that '=' (ctx" instead of
just "SPACING".

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-24 09:34:25 +01:00
Fabio Baltieri
7b57d3c5bb scripts: compliance: check for binary files
Add a compliance check that fails if the diff is adding any binary file
to prevent committing them by mistake.

Only check for added files and explicitly allow image files in doc/ and
boards/.

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2022-11-21 16:12:04 -05:00
Carles Cufi
49b01b39b6 scripts: compliance: Remove unconditional debug output
This prevents us from parsing the scripts' output correctly.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-21 11:02:39 -05:00
Carles Cufi
66ad0e68a6 scripts: compliance: Use consistent naming convention
Tests are named with CamelCase in general, use that in the names.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-21 11:02:39 -05:00
Carles Cufi
8c12efe2e5 scripts: compliance: Make test filtering case insensitive
To simplify using the -m and -e command-line options make the test
filtering case insensitive.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-21 11:02:39 -05:00
Carles Cufi
95053c8358 scripts: compliance: Fix DT Bindings check name
Check names must not contain spaces, since they are passed via `-m` in
the command-line.

Also update .gitignore and compliance.yml.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-21 11:02:39 -05:00
Carles Cufi
8275457709 scripts: compliance: Fix wrong call
Fix call to parse_dt_bindings() in the Devicetree Bindings check.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-21 11:02:39 -05:00
Chris Friedt
5e7a86e1f9 scripts: compliance: add devicetree binding compliance check
Add `DevicetreeBindingsCheck` and `required_false_check()`, which
checks to make sure that no new Devicetree bindings add the
redundant `required: false` property.

Signed-off-by: Chris Friedt <cfriedt@meta.com>
2022-11-20 13:12:44 -05:00
Carles Cufi
1e203ea15e scripts: compliance: Ensure lxml is installed
See https://github.com/weiwei/junitparser/issues/99.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-13 06:25:59 -05:00
Carles Cufi
1ee0d60990 scripts: compliance: Parse the pylint output
Parse the output generated by pylint and store it as FmtdFailure
instances, so that they can be annotated later.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-13 06:25:59 -05:00
Carles Cufi
edd2c83dd4 scripts: compliance: Modernize pylint invocation
The current recommended API call for subprocess instantiation is run(),
so replace the low-level Popen and communicate() with it.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-13 06:25:59 -05:00
Carles Cufi
8275d21a6c scripts: compliance: Add GitHub-compatible annotation support
Add support for GitHub workflow annotations, so that errors are
displayed inline with the Pull Request diff.

More info here:
https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#about-workflow-commands

Enable the new option in the compliance GitHub action.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-11-13 06:25:59 -05:00