Commit graph

4869 commits

Author SHA1 Message Date
ec401102ab west: add a minichlink runner 2024-07-21 14:03:21 +00:00
Piotr Kosycarz
b8ae3a416b scripts: ci: test_plan: use find_modules only when commits are provided
Current implementation will not work if comits were not provided.
ie. use case with list of changed files will fail as args.commits is None.

Signed-off-by: Piotr Kosycarz <piotr.kosycarz@nordicsemi.no>
2024-07-18 12:34:10 -04:00
Anas Nashif
4139f2ed60 MAINTAINERS: add bugs scripts to CI
Add script to an area and ignore in twister CI.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-07-12 16:10:13 -04:00
Anas Nashif
47aa08026f ci: scripts: do not count PRs labeled as bug in snapshot
Doing duplicates count of bugs, a PR fixing a bug is not a bug report.
Many PRs fixing an open bug are labeled with 'bug' and thuse are being
counted twice.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-07-12 16:10:13 -04:00
Anas Nashif
913426e585 ci: set_assignee: pick next area when submitter = assignee
Address the cases where submitter is also the maintainer of the code
changed and other areas are being changed. In this case, assign to the
next area maintainers instead of assigning to submitter.

Example: maintainer of component A introduced significant changes to
area A but also makes changes to other areas B and C. Right now
maintainers of B and C are added as reviewers.

This change will assign to the next area after A, i.e. B in cases where the
submitter is also the maintainer of area A.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-07-10 19:14:17 -04:00
Grzegorz Swiderski
9b682308b3 scripts: list_hardware: Find all SoCs before checking runner configs
When validating the flash runner configurations in `soc.yml`, the only
SoCs that were considered had to be defined under this structure:

   family:
     - series:
         - socs:
             - name: ...

However, the `family` and `series` keys are optional, so the `soc.yml`
files can also be arranged like this:

   family:
     - socs:
         - name: ...
   series:
     - socs:
         - name: ...
   socs:
     - name: ...

The solution is to move the validation code further down, so that it can
reuse the SoC data that was already correctly parsed while initializing
a `Systems` instance.

Signed-off-by: Grzegorz Swiderski <grzegorz.swiderski@nordicsemi.no>
2024-07-10 11:46:52 -04:00
Jamie McCrae
5b6b850fb9 scripts: requirements: Bump imgtool to 2.1.0
Updates the minimum version of imgtool to 2.1.0, which is one year
newer than the 2.0.0 release

Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
2024-07-10 11:38:07 +02:00
Nick Kraus
29084dc06b west: Regex based version finding in OpenOCD runner
Previous OpenOCD version finding would fail when additional tokens were
prepended to the 'openocd --version' output, as happens with some third-
party OpenOCD repackages (xPack for one).

Fixes: #71955

Signed-off-by: Nick Kraus <nick@nckraus.com>
2024-07-09 19:02:13 +02:00
Jordan Yates
af0cbb19c0 scripts: twisterlib: coverage: ignore __ASSERT branches
Disable branch coverage for the `__ASSERT` family of macros. Covering
all of the assertion branches by definition means triggering the
assertion, which can be either challenging or impossible to exercise,
and in either case results in the immediate termination of the test.

Signed-off-by: Jordan Yates <jordan@embeint.com>
2024-07-08 20:41:31 -04:00
Jordan Yates
53bd9ead64 scripts: twisterlib: coverage: fix multiple branch excludes
Multiple values for `--exclude-branches-by-pattern` will result in only
the last value taking effect. Resolve this by merging all the provided
regex patterns into a single pattern with the `|` operator.

Signed-off-by: Jordan Yates <jordan@embeint.com>
2024-07-08 20:41:31 -04:00
Sylvio Alves
e04964aebd requirements: bump pyelftools to >=0.29
The minimum version of pyelftools is 0.29 to make it working
with scripts/footprint/size_report

Fixes #75605

Signed-off-by: Sylvio Alves <sylvio.alves@espressif.com>
2024-07-08 17:18:29 -04:00
Hake Huang
e0f772e880 test: pytest: add linkserver support in pytest
pytest supporting add linkserver for nxp platforms

Signed-off-by: Hake Huang <hake.huang@oss.nxp.com>
2024-07-08 09:23:48 +02:00
Grzegorz Swiderski
7a10c7dff1 scripts: west_commands: Fix west boards for SoC and arch in modules
Fixes #71761

The `west boards` command parses extra BOARD_ROOTs from Zephyr modules,
so that the boards defined in those modules are automatically listed.

In HWMv2, OOT boards can be described in terms of OOT SoCs, which means
that extra SOC_ROOTs must also be provided. Otherwise, an error message
will be displayed when attempting to list all boards. Therefore, every
Zephyr module SOC_ROOT should be included as well.

In HWMv1 (deprecated), OOT boards can be defined in terms of OOT archs,
but module ARCH_ROOTs had never been included automatically. The fix for
this is long overdue, but it's included for symmetry.

Signed-off-by: Grzegorz Swiderski <grzegorz.swiderski@nordicsemi.no>
2024-07-05 14:55:08 +02:00
Yuval Peress
1b51740fa3 twister: Fix gTest harness
The gTest harness asssumed that the lines end with the test name, but
some gTest implementations include the test duration in the line. Update
both the tests and regex to allow this and also avoid capturing
characters into the `test_name` that cannot be valid test name chars.

Fixes #72318

Signed-off-by: Yuval Peress <peress@google.com>
2024-07-03 08:52:13 +02:00
Henrik Brix Andersen
e5aad45bb1 west: commands: runners: canopen: increase default SDO timeout
Increase the default SDO timeout for the CANopen program download west
runner from 0.3 seconds to 1 second. Depending on the flash size and speed,
a full erase may take slightly longer than 300 ms.

The timeout can be customized by using the --sdo-timeout runner parameter.

Fixes: #73987

Signed-off-by: Henrik Brix Andersen <hebad@vestas.com>
2024-07-02 09:20:57 +02:00
Gerson Fernando Budke
768b8bbca3 scripts: west robot & simulation: Fix OOT
The current version of scipts do not consider OOT boards use cases and
the tests with robot now are strict to only one robot file, which is
not realistic for real environment. This address those issues and allow
multiple testsuits at command line and lists at tests entries. It add
another test parameter to allow configure robotframework options.

Fixes: #74563

Signed-off-by: Gerson Fernando Budke <gerson.budke@ossystems.com.br>
2024-07-01 16:10:35 -04:00
Henrik Brix Andersen
c4c5f168e8 ci: ignore changes to the CANopen program download runner
Ignore changes to the CANopen program download west runner in CI.

This script is only executed when performing a DFU using the CANopen
protocol via CAN, which is never triggered in CI anyways.

Signed-off-by: Henrik Brix Andersen <hebad@vestas.com>
2024-07-01 16:05:44 -04:00
Maciej Perkowski
a033ba65d2 twister: Add warning to the "-footprint-from-buildlog" flag
The flag won't work with sysbuild since there is no way to
reliably tell to a parser which data came from which image.

fixes: #74092

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2024-06-28 21:12:57 -04:00
Daniel Leung
2eb0ffb7ea scripts: logging/dictionary: fix annoying pylint warning
pylint keeps failing and complaining about arg_data_type is
used before assignment. So assign it to None to silence
the warning.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2024-06-28 21:14:14 +02:00
Daniel Leung
f5762e07f2 logging: dictionary: new verion 3 format
The package_len has been extended from 10 bits to 11 bits
in the log message header. So the format for dictionary
logging also needs to be updated.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2024-06-28 21:14:14 +02:00
Daniel Leung
0e33c7a976 scripts: logging/dictionary: replace %#llx too
Python does not really support long long double, so %llx cannot
be formatted correctly, so we replace it with a simple %lx.
There is another variant %#llx and we also need replace it to
%#lx.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2024-06-28 21:14:14 +02:00
Daniel Leung
ecc642b2df scripts: logging/dictionary: refactor common parser functions...
... and put them into the LogParser class file instead of
the verisoned parser. This is in preparation for introducing
a new parser version.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2024-06-28 21:14:14 +02:00
Daniel Leung
3da42fb547 scripts: logging/dictionary: fix long long alignment
Printing long long requires alignment on 64-bit before parsing
the actual argument. Or else the parser would be looking at
some unrelated bits. So fix it.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2024-06-28 21:14:14 +02:00
Daniel Leung
eacf71d92a scripts: logging/dictionary: extract DataTypes into its own file
This extracts the DataTypes class into its own file. This is in
preparation to add a new version of parser which can reuse this
class.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2024-06-28 21:14:14 +02:00
Louis Feller
5b4c8945fd west : runners : Fix error message when runners.yaml is not found.
Adjust error message so that it clearly states runners.yaml is
missing from <build_dir>/zephyr, instead of referencing CMake cache
variable ZEPHYR_RUNNERS_YAML, which is no longer used (since
3124c02987 ).
Also clean up that variable in CMake since it is no longer used
(0 other references in entire tree).
Fixes #70605

Signed-off-by: Louis Feller <louis.feller@st.com>
2024-06-28 12:23:18 -04:00
Gerard Marull-Paretas
4753005c55 scripts: get_maintainer: check maintenance status
Error if an area is set as maintained but it has no maintainers
assigned.

Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
2024-06-28 07:19:49 -04:00
Jordan Yates
243eb36b18 everywhere: reindent .overlay files with tabs
`checkpatch.pl` requires that dts sources are indented with tabs,
fix all the spaces that slipped in while checkpatch wasn't watching.

Signed-off-by: Jordan Yates <jordan@embeint.com>
2024-06-26 15:59:44 +02:00
Félix Turgeon
5cd580ce44 scripts: coredump: Add register write handler to arm cortex-m gdbstubs.
When working with coredumps, it is useful to be able to modify base
registers. Adding this capability allows implementing scripts to
inspect backtrace of threads other than current the current thread.

Signed-off-by: Félix Turgeon <felixturgeon@meta.com>
2024-06-25 19:14:20 -04:00
Lingao Meng
be96512d82 scripts: spelling.txt: add double prepositions
Add double prepositions to the default spelling check list.

Signed-off-by: Lingao Meng <menglingao@xiaomi.com>
2024-06-25 06:05:35 -04:00
Lingao Meng
302422ad9d everywhere: replace double words
import os
import re

common_words = set([
    'about', 'after', 'all', 'also', 'an', 'and',
     'any', 'are', 'as', 'at',
    'be', 'because', 'but', 'by', 'can', 'come',
    'could', 'day', 'do', 'even',
    'first', 'for', 'get', 'give', 'go', 'has',
    'have', 'he', 'her',
    'him', 'his', 'how', 'I', 'in', 'into', 'it',
    'its', 'just',
    'know', 'like', 'look', 'make', 'man', 'many',
    'me', 'more', 'my', 'new',
    'no', 'not', 'now', 'of', 'one', 'only', 'or',
    'other', 'our', 'out',
    'over', 'people', 'say', 'see', 'she', 'so',
    'some', 'take', 'tell', 'than',
    'their', 'them', 'then', 'there', 'these',
    'they', 'think',
    'this', 'time', 'two', 'up', 'use', 'very',
    'want', 'was', 'way',
    'we', 'well', 'what', 'when', 'which', 'who',
    'will', 'with', 'would',
    'year', 'you', 'your'
])

valid_extensions = set([
    'c', 'h', 'yaml', 'cmake', 'conf', 'txt', 'overlay',
    'rst', 'dtsi',
    'Kconfig', 'dts', 'defconfig', 'yml', 'ld', 'sh', 'py',
    'soc', 'cfg'
])

def filter_repeated_words(text):
    # Split the text into lines
    lines = text.split('\n')

    # Combine lines into a single string with unique separator
    combined_text = '/*sep*/'.join(lines)

    # Replace repeated words within a line
    def replace_within_line(match):
        return match.group(1)

    # Regex for matching repeated words within a line
    within_line_pattern =
	re.compile(r'\b(' +
		'|'.join(map(re.escape, common_words)) +
		r')\b\s+\b\1\b')
    combined_text = within_line_pattern.
		sub(replace_within_line, combined_text)

    # Replace repeated words across line boundaries
    def replace_across_lines(match):
        return match.group(1) + match.group(2)

    # Regex for matching repeated words across line boundaries
    across_lines_pattern = re.
		compile(r'\b(' + '|'.join(
			map(re.escape, common_words)) +
			r')\b(\s*[*\/\n\s]*)\b\1\b')
    combined_text = across_lines_pattern.
		sub(replace_across_lines, combined_text)

    # Split the text back into lines
    filtered_text = combined_text.split('/*sep*/')

    return '\n'.join(filtered_text)

def process_file(file_path):
    with open(file_path, 'r', encoding='utf-8') as file:
        text = file.read()

    new_text = filter_repeated_words(text)

    with open(file_path, 'w', encoding='utf-8') as file:
        file.write(new_text)

def process_directory(directory_path):
    for root, dirs, files in os.walk(directory_path):
        dirs[:] = [d for d in dirs if not d.startswith('.')]
        for file in files:
            # Filter out hidden files
            if file.startswith('.'):
                continue
            file_extension = file.split('.')[-1]
            if
	file_extension in valid_extensions:  # 只处理指定后缀的文件
                file_path = os.path.join(root, file)
                print(f"Processed file: {file_path}")
                process_file(file_path)

directory_to_process = "/home/mi/works/github/zephyrproject/zephyr"
process_directory(directory_to_process)

Signed-off-by: Lingao Meng <menglingao@xiaomi.com>
2024-06-25 06:05:35 -04:00
Jordan Yates
1ab4fe68d1 scripts: spelling.txt: add double prepositions
Add double prepositions to the default spelling check list.

Signed-off-by: Jordan Yates <jordan@embeint.com>
2024-06-22 05:40:22 -04:00
Jordan Yates
07870934e3 everywhere: replace double words
Treewide search and replace on a range of double word combinations:
    * `the the`
    * `to to`
    * `if if`
    * `that that`
    * `on on`
    * `is is`
    * `from from`

Signed-off-by: Jordan Yates <jordan@embeint.com>
2024-06-22 05:40:22 -04:00
Jordan Yates
79f81f6eb8 scripts: checkpatch.pl: treat .overlay files as dts
Treat devicetree overlay files as DTS source for the purposes of
checkpatch.

Fixes #74570.

Signed-off-by: Jordan Yates <jordan@embeint.com>
2024-06-21 08:46:12 -04:00
Zhani Baramidze
6ced73e552 scripts: fixed missing argument in parse_from_attribute call.
In size_report script, if the DWARF section of ELF file contains both
debug_loc and a debug_loclists sections, LocationListsPair class is
used to track locations. In that case, parse_from_attribute was missing
one argument which was causing the script to fail.

Signed-off-by: Zhani Baramidze <jbaramidze@meta.com>
2024-06-20 17:03:21 -04:00
Yong Cong Sin
7adac57943 scripts: checkpatch: add stdio.h FILE typedef
Add `FILE` typedef in the `stdio.h` so that when doing
`FILE *file` definition checkpatch doesn't complain about
the position of the '*' and fail in CI.

Signed-off-by: Yong Cong Sin <ycsin@meta.com>
2024-06-20 14:07:14 -04:00
Carles Cufi
4fecad4fe0 scripts: runners: nrf_common: Fix recover on 54H20
In order for the --recover option to work properly on the nRF54H20, it
requires executing it for both cores, the radio and the application one.
Extend the recover_target() function so that it does so for both 53 and
54H20.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2024-06-20 12:05:39 -04:00
Anas Nashif
1a46b7ed0e ci: assignees: set multiple assignees of same area
Noticing many PRs that wait too long in the queue although once of the
maintainers approved with asignees set to other maintainers.

This changes the current behavior of picking the first maintainer in the
list and assigning to them only, instead we assign to all maintainers of
the main area being changed.

Who ends up driving the PR to a mergeable state is then to the
maintainers and they can unassign/assign based on availability.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-06-18 14:39:22 -04:00
Yong Cong Sin
88c96f7986 drivers: counter: stm32: remove deprecated clock source Kconfigs
These clock selection Kconfigs should have been deprecated for
more than 2 releases, remove them:

- `CONFIG_COUNTER_RTC_STM32_CLOCK_SRC`
- `CONFIG_COUNTER_RTC_STM32_CLOCK_LSI`
- `CONFIG_COUNTER_RTC_STM32_CLOCK_LSE`

Signed-off-by: Yong Cong Sin <ycsin@meta.com>
2024-06-18 14:33:58 -04:00
Luca Burelli
38b0460de3 check_compliance: allow _MODULE variant of Kconfig options
Since b53a792ff0 Zephyr has the ability to define tristate Kconfig
options. When a tristate option FOO is selected as a "module", this
results in autoconf.h defining CONFIG_FOO_MODULE, not CONFIG_FOO.

This patch allows the check_compliance script to also accept references
to a Kconfig symbol ending in _MODULE if the prefix is defined.

Signed-off-by: Luca Burelli <l.burelli@arduino.cc>
2024-06-18 15:54:39 +02:00
Marcin Niestroj
efcc0ed27f scripts: pytest: execute 'zephyr.exe' in application build directory cwd
Execute 'zephyr.exe' in application build directory as 'current working
directory' (cwd). This makes sure that native_sim specific drivers (like
flash simulator with file backend in 'flash.bin') are using unique context
for external resources with relative paths.

This fixes executing native_sim tests in twister with flash simulator.
Previously a shared 'flash.bin' was used for all executed 'zephyr.exe'
processes in twister. After this patch a unique 'flash.bin' file is used
for each tested sample, since those 'flash.bin' is placed in application
build directory instead of twister root directory.

Signed-off-by: Marcin Niestroj <m.niestroj@emb.dev>
2024-06-18 15:53:00 +02:00
Torsten Rasmussen
a9efb1dae6 west: update build extension command to use APP_DIR
`west build` can be invoked without specifying the source directory when
being invoked from the source directory itself.
When using `west build` for incremental builds, then the build command
will examine the CMake cache to determine the application dir by using
the value of CMAKE_HOME_DIRECTORY.

With sysbuild, this leads to the wrong assumption that the sysbuild
itself is the application to build.

Instead, have west build look for APP_DIR which points to the correct
source dir when sysbuild is used. Use APPLICATION_SOURCE_DIR when
APP_DIR is not set, as this indicates a no-sysbuild build.

Keep CMAKE_HOME_DIRECTORY behavior as last fallback mechanism.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2024-06-17 12:09:45 -04:00
Dmitrii Golovanov
1ccd972563 twister: Fix on output directory reaching backup limit
Stop Twister if there are too many backup copies of the output
directory already.
Before this fix, Twister silently kept artifacts from the last run,
unless `--clobber-output` was explicitly given.

Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
2024-06-17 12:08:20 -04:00
Jordan Yates
25fa41c209 scripts: twisterlib: coverage: handle multiple gcov dumps
Extend the coverage tool to handle applications that generate multiple
gcov dumps in a single execution. This can happen when the application
calls `sys_reboot`.

Handling multiple dumps enables coverage testing of exception handlers.

Signed-off-by: Jordan Yates <jordan@embeint.com>
2024-06-15 04:39:53 -04:00
Marcin Niestroj
4835658b46 west: runners: openocd: handle empty --elf-file
Make this param truly optional, so that hex file alone is enough for
flashing.

Signed-off-by: Marcin Niestroj <m.niestroj@emb.dev>
2024-06-14 21:05:49 -04:00
Thomas Gagneret
f5df063cbd scripts: zephyr_module: Move SPDX name normalization to writer.py
Since `writer.py` is the one writting the SPDX file, it should normalize
the name field and not `walker.py` which generates the SBOM components.

Signed-off-by: Thomas Gagneret <thomas.gagneret@hexploy.com>
2024-06-14 19:07:48 -04:00
Thomas Gagneret
0d05318c96 scripts: zephyr_module: Add URL, version to SPDX
Improve the SPDX with the current values:
 - URL: extracted from `git remote`. If more than one remote, URL is not
 set.
 - Version: extracted from `git rev-parse` (commit id).
 - PURL and CPE for Zephyr: generated from URL and version.

For zephyr, the tag is extracted, if present, and replace the commit id for
the version field.
Since official modules does not have tags, tags are not yet extracted for
modules.

To track vulnerabilities from modules dependencies, a new SBOM,
`modules-deps.spdx` was created. It contains the `external-references`
provided by the modules. It allows to easily track vulnerabilities from
these external dependencies.

Signed-off-by: Thomas Gagneret <thomas.gagneret@hexploy.com>
2024-06-14 19:07:48 -04:00
Anas Nashif
252ca99d0c ci: testplan: do not run tests on all platforms when a test changes
With many tests having 10s or scenarios and variants, anytime we make a
change to a test right now, we end up building all scenarios on all
platforms which ends up in multiple 10s of the thousands of instances
that need to run on 30 or 40 runners blocking CI for hours. We do not
really need that, a test needs to be smart about its coverage and not
rely on boiling the ocean to catch, mostly build errors that are
platform specific.

Change this to do the normal coverage we get on push events.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-06-14 13:59:24 -04:00
Rubin Gerritsen
470a0fcaa5 cocinelle: Add a sematic patch file for ztest string assertions
ztest now provides functionality to compare strings.
These are simpler to use than the strcmp ways.

The semantic patch transforms many of the commonly used patterns.
It does not handle variable length macros.

Signed-off-by: Rubin Gerritsen <rubin.gerritsen@nordicsemi.no>
2024-06-14 19:25:42 +02:00
Rubin Gerritsen
fc377363e4 cocinelle: Parse ztest test functions
We want to able to use cocinelle on ztest functions as
well when transforming APIs.

Provide a simple macro so that test functions are recognized.

Signed-off-by: Rubin Gerritsen <rubin.gerritsen@nordicsemi.no>
2024-06-14 19:25:42 +02:00
Dmitrii Golovanov
68bb87bd50 twister: pytest: Log output passthrough
Pass Twister pytest plugin's log output, as well as output from
a test image running with pytest, up to Twister log irregardless
of the current verbosity level set at Twister.
This allows to collect full test output and also ensures that
recording data embedded in the test log is passed to the Twister
pytest harness for export.

Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
2024-06-14 09:41:52 -04:00