Commit graph

301 commits

Author SHA1 Message Date
Fabio Baltieri
98d9a7f86f scripts: compliance: add a compliance check to keep blocks of code sorted
This allows defining a list within two marker (zephyr-keep-sorted-start
and zephyr-keep-sorted-stop), and have the CI validate that the block is
kept sorted every time stuff gets added to it.

This is mainly for Kconfig and CMake include lists so that there's no
ambiguity on where to add new stuff.

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2023-11-13 09:41:05 +01:00
Maciej Perkowski
aa3e79a88a scripts: Add test_plan.py to twister_ignore.txt
There is no point in running full twister scope when
test_plan.py script is modified.

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2023-10-31 09:02:16 +01:00
Maciej Perkowski
3314b8e0c3 twister: bugfix: Fix infinite loop in test_plan.py script
Commit 72f416f382 added a horizontal
scan for .yaml files when a modification was made in "common" folder.
If yamls were found in such way, the loop ended. However, the
implementation didn't address what happens if such yamls are not found.
This made the script going into an infinite loop. If yamls are not
found next to "common", the script should proceed as before, i.e. go
to the directory above an start looking there.

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2023-10-30 10:54:57 -04:00
Maciej Perkowski
72f416f382 scripts: Add workflow for "common" directories in find_tests()
Some tests use a "common" directory to store pieces of code which
are reused by different scenarios. In those cases, no test yaml is
found within such director nor within its parents.
If no test yaml is found in a directory, and the directory is called
common, also look in collocated directories if they have test yamls.
If so, add all those locations to the scope.
E.g. tests/bluetooth/controller/common

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2023-10-27 09:13:16 -04:00
Maciej Perkowski
9f85001061 scripts: Make workflow of test_plan.py script more robust
The script was not resolving all detected changes uniformly:
find_excludes() could skip or not certain patterns based on
required testing scope. The idea was to not include files
that were already handled by find_test() and find_boards() workflows.
However, only boards and tests folders could be removed but
not samples. This also led to blind spots: changes in some files
were not triggering any tests. E.g. change in a test/common, where
no corresponding yaml can be found by find_tests() which is also
ignored by find_excludes(). In the new workflow a list of resolved
files (for which find_arch(), find_tests() or find_boards()
found scope) is created. Instead of using skip in find_excludes,
files are excluded only if they were resolved.

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2023-10-27 09:13:16 -04:00
Krishna T
85e3a4ca6a scripts: ci: Fix for compliance with multi-user machine
If in a server multiple users are running compliance then it throws
errors as the filename "Kconfig.modules" and "Kconfig.dts" is owned by
someone else.

Fix this by creating a randomized Kconfig file and delete after use.

Signed-off-by: Krishna T <krishna.t@nordicsemi.no>
2023-10-26 16:47:30 +02:00
Maciej Perkowski
0752d60443 scripts: Add arg to test_plan.py for alternative test locations
The arg --testsuite-root was copied from twister. When it is used
for test_plan.py it will be propagated to twister calls. This allows
to make alternative test locations (e.g. from another repo) to work
with test_plan.py

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2023-10-24 09:04:09 +02:00
Maciej Perkowski
7692610004 scripts: Add arg to set_plan.py for alternative tag relation list
Add --alt-tag arg for test_plan.py script. User can use it and point
to an alternative file with tag-directories relations. If so, such
file will be used instead of the default one.

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2023-10-24 09:04:09 +02:00
Maciej Perkowski
47405a114a scripts: Allow using alternative ignore-patters in test_plan.py
Expand test_plan.py args with --ignore-path. This allows to provide an
alternative lists of patterns for the script.

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2023-10-24 09:04:09 +02:00
Maciej Perkowski
f7c4d5f9d4 scripts: Allow test_plan.py to work with other than zephyr repos
The test_plan.py script has a path to repository to be scanned for
changes hard coded to zephyr. This patch separates zephyr path from
such repository's path and adds an arg to pass repo to scan

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2023-10-24 09:04:09 +02:00
Maciej Perkowski
f9178926a9 scripts: Add --no-detailed-test-id arg to test_plan.py script
An option --no-detailed-test-id was added to twister to help align
names for test outside of zephyr tree. This commit add this arg
to test_plan.py script which is then propagated to twister.

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2023-10-24 09:04:09 +02:00
Anas Nashif
b643692fa5 ci: test_plan: use integration mode for architectures
Reduce the scope to integration only.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-10-23 11:20:57 -04:00
Anas Nashif
99bb4b9581 ci: testplan: fix include path
Add zephyr to the include path detection.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-10-23 11:20:57 -04:00
Anas Nashif
2ab8c1b779 ci: do not run twister on set_assignees.py changes
No need to run twister when changing this script.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-10-20 15:10:11 +02:00
Anas Nashif
22042e7b4e ci: tags: include wifi/net drivers in the net group
Changes to networking and wifi drivers should trigger networking tests.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-10-10 16:22:54 +03:00
Yonatan Schachter
726e14e475 doc: bindesc: Add documentation for binary descriptors
Add documentation for binary descriptors under "OS Services"

Signed-off-by: Yonatan Schachter <yonatan.schachter@gmail.com>
2023-09-28 07:39:09 -04:00
Jamie McCrae
ce65806cd4 tests: boot: Add MCUboot data sharing test
Adds a test for the data sharing retention feature of MCUboot
configuration to an application.

Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
2023-09-27 20:18:31 -04:00
Jamie McCrae
db4febc584 retention: Add bootloader configuration interface
Adds a bootloader configuration interface which allows for a
bootloader (e.g. MCUboot) to set configuration in a shared data
area which is then read by the application.

Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
2023-09-27 20:18:31 -04:00
Jamie McCrae
97123c4f4a scripts: ci: check_compliance: Add sysbuild Kconfig exceptions
Adds exceptions for Kconfigs which sysbuild uses when configuring
MCUboot.

Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
2023-09-25 12:45:44 +02:00
Fabio Baltieri
183b84d0e2 scripts: compliance: add a check for missing west area maintainer enties
Add a check to ensure that every module has a corresponding maintainers
file entry, ensure modules are not added with no recorded point of
contact.

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2023-09-14 08:13:57 -07:00
Anas Nashif
948ab60eb3 ci: test_plan: also filter on platforms when manifest changes
Add platform filtering when generating tests for manifest modules.
In the clang workflow, we do select to run only on one platform, so this
needs to be applied as well or we end up building unwanted tests using
the wrong toolchain.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-08-24 06:36:00 -04:00
Anas Nashif
d4169c93fd ci: test_plan: add tags based on manifest change
When manifest changes, add tags based on the changed project names.t

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-08-22 18:08:30 +02:00
Fabian Blatz
29d07d6020 scripts: ci: check_compliance: Replace dead Kconfig doc link
Update the link to the Kconfig documentation page.

Signed-off-by: Fabian Blatz <fabianblatz@gmail.com>
2023-08-09 12:47:12 +00:00
Patryk Duda
4b94fc3da2 llvm: Add support for selecting runtime library
This patch adds Kconfig options to select either GNU libgcc or LLVM
compiler-rt. The 'rtlib' flag is provided in a config file, so this
patch introduces 'clang_libgcc.cfg' and 'clang_compiler_rt.cfg' which
enable appropriate library. The file is selected by concatenating
the 'clang_' prefix with library name.

Signed-off-by: Patryk Duda <pdk@semihalf.com>
2023-08-03 19:35:11 +02:00
Fabio Baltieri
68f514fc38 scripts: compliance: always run the MaintainersFormat check
The check currently only runs if the maintainers file itself is changed,
but that means that the check is going to miss every PR that moves
directory or delete files that can potentially trigger an error.

This check is cheap to run, just run it unconditionally.

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2023-07-25 18:49:34 -04:00
Jamie McCrae
d52b634610 doc: mgmt: mcumgr: Add details on making handlers
Adds a guide on how to make out-of-tree MCUmgr function handlers
and groups, with an example showing a test implementation.

Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
2023-07-20 16:22:31 +02:00
Fabio Baltieri
9de541b9c2 scripts: ci: check_compliance: add a no-modules Kconfig check
Add a variation of the basic Kconfig check that runs with no modules,
catches symbols that are used in the main repository but are defined
only in modules, which are potentially problematic if a downstream
project is not using the specific module.

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2023-07-05 09:11:02 +02:00
Carles Cufi
ff2d9cfcb6 scripts: compliance: Fix handling of integer node items
Some node items in Kconfig can be kconfiglib.MENU or kconfiglib.COMMENT.
Those are integers and thus do not contain a node.item.name field.
Handle those separately to avoid hitting the followig exception:

Traceback (most recent call last):
  File "/home/runner/work/zephyr/zephyr/./scripts/ci/check_compliance.py",\
  line 1307, in main
    n_fails = _main(args)
  File "/home/runner/work/zephyr/zephyr/./scripts/ci/check_compliance.py",\
  line 1242, in _main
    test.run()
  File "/home/runner/work/zephyr/zephyr/./scripts/ci/check_compliance.py",\
  line 277, in run
    self.check_no_redefined_in_defconfig(kconf)
  File "/home/runner/work/zephyr/zephyr/./scripts/ci/check_compliance.py",\
  line 445, in check_no_redefined_in_defconfig
    Kconfig node '{node.item.name}' found with prompt or help in\
    {node.filename}.
AttributeError: 'int' object has no attribute 'name'

Seen in #58454.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2023-07-04 08:23:02 +02:00
Anas Nashif
7f0aff9045 ci: testplan: do not start more runners than needed
We have been launching 10 runners up a certain number of tests, although
we only neeed half of that for some scenarios.
Too many runners started that have to execute just a small number of
tests wastes times on setup and blocks the queue. Just start the number
of nodes needed based on initial calculation.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-06-08 07:33:56 -04:00
Jamie McCrae
65c9058ed4 sysbuild: Fix value propagation of signature type to MCUboot
Fixes an issue whereby the type of the signature was not passed
to MCUboot.

Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
2023-06-08 06:51:33 -04:00
Anas Nashif
432eefca05 ci: tags: add mcumgr to tags file
Filter based on changes in pull request and do not build mcumgr on
unrelated changes.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-06-02 04:47:06 -04:00
Anas Nashif
9cef0451ba ci: tags: add networking to tags file
If we are not changing the network subsystem, do not build samples and
tests.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-06-02 04:47:06 -04:00
Anas Nashif
cec8fd1c6e tests: ztest: fix tags for testsuite and unify them
Use test_framework as unified tag so we can filter tests in a consistent
way.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-06-02 04:47:06 -04:00
Anas Nashif
4e8cdb8bc3 ci: manage when to build testsuite tests
Do not build test framework tests on every change, use tag filtering to
improve CI performance.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-06-02 04:47:06 -04:00
Torsten Rasmussen
db89e7699d sysbuild: generate .config.sysbuild for controlling build settings
Sysbuild now generates a .config.sysbuild config file which specifies
settings controlled by sysbuild.
Any setting specified in this .config will overrule user provided
setting, and a warning will be raised if the sysbuild controlled value
is different from the value specified by the user.

This has the following benefits:
- Allow sysbuild to control any build setting without adjustments to
  the existing Kconfig tree
- Allow sysbuild to adjust settings based on knowledge regarding enabled
  images / bootloaders.
- Cleanup CMake code, as settings in sysbuild no longer needs to be
  propagated using CMake cache variables.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2023-05-25 14:57:41 +02:00
Filip Kokosinski
442a452903 scripts/ci/check_compliance: add GitDiffCheck compliance check
This commit adds a new `GitDiffCheck` compliance check that checks the
newly added commits with `git diff --check` and reports them back if an
error is found.

This check is needed for some files (e.g. Kconfig) as they are not checked
by `Kconfig` and `KconfigBasic` checks on every commit in a pull request.

Signed-off-by: Filip Kokosinski <fkokosinski@antmicro.com>
2023-05-19 14:52:49 +00:00
Jamie McCrae
96a9b7e1ff scripts: ci: check_compliance: Ignore mcuboot Kconfigs
Adds mcuboot Kconfig options to the ignore list as these are
defined in mcuboot but used in documentation and samples which
utilise sysbuild to build mcuboot.

Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
2023-04-24 13:27:53 +02:00
Magdalena Kasenberg
661f07ba0f bluetooth: tester: Add CONFIG_BTTESTER_LOG_LEVEL
to set a different log level than the default.

Signed-off-by: Magdalena Kasenberg <magdalena.kasenberg@codecoup.pl>
2023-04-12 13:05:32 +02:00
Grzegorz Chwierut
482c94819f tests: boot: test_mcuboot: Add mcuboot log level
Added logging of MCUboot messages to fix twister test scenario.
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
2023-04-06 18:51:23 +02:00
Daniel Leung
4d914f4f89 toolchain: move CONFIG_LLVM_USE_LD into cmake/toolchain/llvm
This moves CONFIG_LLVM_USE_LD into cmake/toolchain/llvm as this
is a toolchain kconfig. Also make it a choice to allow the use
of LLVM's lld as linker.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-03-29 19:48:47 -04:00
Jamie McCrae
ef146c3dbd scripts: compliance: Add MCUboot Kconfig to allow list
Adds a Kconfig for MCUboot which is used in the documentation to
the CI Kconfig check allow list.

Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
2023-03-29 11:30:36 +02:00
Jason Kridner
f36be35600 scripts: compliance: allow webp image files
WebP images are smaller for similar resolutions and quality while being
compatible with web browsers and other image processing tools.

For higher resolution images, be sure to limit the display size to make
the rendered pages look reasonable.

This was approved by Benjamin Cabé (@kartben) in this discussion thread:
https://github.com/zephyrproject-rtos/zephyr/pull/55488#issuecomment-1461792751

Signed-off-by: Jason Kridner <jkridner@beagleboard.org>
2023-03-21 16:03:43 -04:00
Grant Ramsay
090b13cf1e scripts: ci: check_compliance: Ignore undefined CONFIG_CMD_CACHE
CONFIG_CMD_CACHE is a U-Boot config, mentioned in docs.

Signed-off-by: Grant Ramsay <gramsay@enphaseenergy.com>
2023-02-24 18:11:56 +01:00
Anas Nashif
59b3e5ba11 ci: elasticsearch: do not set id for documentats.
Do not set id for docuemnts, lets elasticsearch do that on its own.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-02-21 13:06:29 -05:00
Anas Nashif
39324048eb ci: elasticsearch: set run date when uploading data
To get consistent and 1 single rundate per CI run, set run_date when
we upload the data. This is optional mostly for CI and to avoid
duplication when we re-run jobs.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-02-21 13:06:29 -05:00
Anas Nashif
66fa169623 ci: do not hardcode es server in script
Use environment variable instead of hardcoding server in script. This
will allow reuse.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-02-03 20:53:20 +09:00
Anas Nashif
fcd60cf789 ci: make any arch/ changes trigger kernel testing as well
This should fix issue like #54337 where architecture code changes were
filtering out kernel tests, where those changes are being tested. Kernel
and arch code is tightly coupled and we need to run kernel tests.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-02-03 10:09:26 +09:00
Fabio Baltieri
317b3c6897 scripts: compliance: pass argv explicitly to parse_args
Change the check_compliance main invocation to pass an explicit argv
list from main to parse_args(). This does not change anything for direct
invocation, but allows wrapping the compliance script and using it as a
library while controlling its behavior by passing the intended flags to
main().

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2023-02-01 17:17:34 +09:00
Anas Nashif
fd702ebb34 ci: push test results to elasticsearch instance
Move away from opensearch and switch to elasticsearch for reporting.
The results will be available at https://kibana.zephyrproject.io/

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-01-27 22:57:26 +09:00
Jamie McCrae
d72e135b2b scripts: ci: Enable pylint check for argument parser abbreviations
Enables running CI checks which will now search for initialisation
of argument parser where abbreviations are not disabled.

Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
2023-01-26 20:12:36 +09:00