Commit graph

5778 commits

Author SHA1 Message Date
Chen Xingyu
0095c224fa scripts: west_commands: runners: Add support for probe-rs
probe-rs is a new programming and debugging tool written in Rust, supports
many probes and targets.

This commit introduces initial support for probe-rs to Zephyr.

Signed-off-by: Chen Xingyu <hi@xingrz.me>
2024-08-01 12:39:28 +02:00
TOKITA Hiroshi
f99f862a79 scripts: kconfig: Add integer arithmetic functions
Added functions for integer arithmetic operations
(add, sub, mul, div, mod, inc, dec)

Signed-off-by: TOKITA Hiroshi <tokita.hiroshi@gmail.com>
2024-07-31 12:56:25 +02:00
Christophe Dufaza
2b2a0e04b2 west: blobs: verify fetched blobs after downloading
Running 'west blobs fetch' does not verify the digest of downloaded files:
1. if the checksum of the previously downloaded file does match
   that in the blob metadata (status BLOB_PRESENT), do nothing
2. if the checksum of the previously downloaded file does not match
   that in the blob metadata (status BLOB_OUTDATED),
   download the "up to date" file
3. if the blob has not yet been downloaded (status BLOB_NOT_PRESENT),
   download it

None of the 2) and 3) code paths will verify that the checksum of the file
just downloaded actually matches the digest in the blob's metadata.

In the event that the metadata of a module is incorrect, then the user
will not notice anything, and may rely on an unexpected binary,
e.g. a static library for a different architecture.

According to the Binary Blobs documentation [1], the expected
behavior is to check the blob digest after downloading.

[1] Fetching blobs, Zephyr 3.6.0 (still applies to Zephyr 3.7.0rc3)
docs.zephyrproject.org/3.6.0/contribute/bin_blobs.html#fetching-blobs

Signed-off-by: Christophe Dufaza <chris@openmarl.org>
2024-07-30 18:29:39 +01:00
Christophe Dufaza
ae9326c920 west: blobs: prefer constants to literals for blob status
Replace blob status literals with the corresponding
constants defined in zephyr_module.

Signed-off-by: Christophe Dufaza <chris@openmarl.org>
2024-07-30 18:29:39 +01:00
Théo Battrel
b764238e0d west: Update fish completion for boards
With the recent change to hwmv2, loading the
boards is extremely slow and can take several
seconds.

To solve that problem this commit add a cache
of the boards. The cache is updated based on:
    - when the latest commit of the manifest
    file directory is updated or;
    - when the manifest file directory is not
    a git directory, when the manifest file
    content itself is updated.

At the same time:
    - update how the board completion is
    displayed by including the board vendor to it;
    - add missing `--board` and `--board-dir`
    options;
    - remove `hwmv1` board completion code.

Signed-off-by: Théo Battrel <theo.util@protonmail.ch>
2024-07-30 18:25:28 +01:00
Théo Battrel
377eed6251 west: add vendor to boards format
Add the new `vendor` formatter argument to display board vendor.

Signed-off-by: Théo Battrel <theo.util@protonmail.ch>
2024-07-30 18:25:28 +01:00
Carles Cufi
539f3a1600 scripts: west_commands: nrfjprog: Change tool-opt help text
The current option used as an example, --recover, is actually a separate
option in the runner itself. Instead use --clockspeed as an example,
which is applicable to all nrfjprog commands.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2024-07-27 15:19:39 +03:00
Carles Cufi
7eb364b082 scripts: west_commands: nrfjprog: Add support for --qspiini
Allow for users to provide a --qspiini parameter that is passed directly to
the nrfjprog executable but only in the --program operation. This is
required since e073210ec2 enabled the
-O/--tool-opt for all operations, but --qspiini is only allowed combined
with --program.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2024-07-27 15:19:39 +03:00
Marcin Niestroj
35059a1b3a west: runners: esp32: respect --esp-device during erase
When --erase was specified, esp32 runner was autodetecting serial port to
be used, regardless of --esp-device argument.

Append '--port SERIAL_DEVICE' parameter earlier, so that erase command
invocation uses explicitly specified serial device.

Signed-off-by: Marcin Niestroj <m.niestroj@emb.dev>
2024-07-27 15:19:01 +03:00
Daniel Leung
b2f3d801b4 twister: pytest: use test timeout as the default base timeout
The default base timeout for pytest is statically set by
the TwisterHarnessConfig class to be 60 seconds. However,
sometimes it takes longer than 60s before the app starts
to run, especially on emulator/simulator where it takes
quite some time to start. So pass the test timeout as
the base timeout via pytest command line argument.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2024-07-27 15:18:01 +03:00
Piotr Kosycarz
b8ae3a416b scripts: ci: test_plan: use find_modules only when commits are provided
Current implementation will not work if comits were not provided.
ie. use case with list of changed files will fail as args.commits is None.

Signed-off-by: Piotr Kosycarz <piotr.kosycarz@nordicsemi.no>
2024-07-18 12:34:10 -04:00
Anas Nashif
4139f2ed60 MAINTAINERS: add bugs scripts to CI
Add script to an area and ignore in twister CI.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-07-12 16:10:13 -04:00
Anas Nashif
47aa08026f ci: scripts: do not count PRs labeled as bug in snapshot
Doing duplicates count of bugs, a PR fixing a bug is not a bug report.
Many PRs fixing an open bug are labeled with 'bug' and thuse are being
counted twice.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-07-12 16:10:13 -04:00
Anas Nashif
913426e585 ci: set_assignee: pick next area when submitter = assignee
Address the cases where submitter is also the maintainer of the code
changed and other areas are being changed. In this case, assign to the
next area maintainers instead of assigning to submitter.

Example: maintainer of component A introduced significant changes to
area A but also makes changes to other areas B and C. Right now
maintainers of B and C are added as reviewers.

This change will assign to the next area after A, i.e. B in cases where the
submitter is also the maintainer of area A.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-07-10 19:14:17 -04:00
Grzegorz Swiderski
9b682308b3 scripts: list_hardware: Find all SoCs before checking runner configs
When validating the flash runner configurations in `soc.yml`, the only
SoCs that were considered had to be defined under this structure:

   family:
     - series:
         - socs:
             - name: ...

However, the `family` and `series` keys are optional, so the `soc.yml`
files can also be arranged like this:

   family:
     - socs:
         - name: ...
   series:
     - socs:
         - name: ...
   socs:
     - name: ...

The solution is to move the validation code further down, so that it can
reuse the SoC data that was already correctly parsed while initializing
a `Systems` instance.

Signed-off-by: Grzegorz Swiderski <grzegorz.swiderski@nordicsemi.no>
2024-07-10 11:46:52 -04:00
Jamie McCrae
5b6b850fb9 scripts: requirements: Bump imgtool to 2.1.0
Updates the minimum version of imgtool to 2.1.0, which is one year
newer than the 2.0.0 release

Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
2024-07-10 11:38:07 +02:00
Nick Kraus
29084dc06b west: Regex based version finding in OpenOCD runner
Previous OpenOCD version finding would fail when additional tokens were
prepended to the 'openocd --version' output, as happens with some third-
party OpenOCD repackages (xPack for one).

Fixes: #71955

Signed-off-by: Nick Kraus <nick@nckraus.com>
2024-07-09 19:02:13 +02:00
Jordan Yates
af0cbb19c0 scripts: twisterlib: coverage: ignore __ASSERT branches
Disable branch coverage for the `__ASSERT` family of macros. Covering
all of the assertion branches by definition means triggering the
assertion, which can be either challenging or impossible to exercise,
and in either case results in the immediate termination of the test.

Signed-off-by: Jordan Yates <jordan@embeint.com>
2024-07-08 20:41:31 -04:00
Jordan Yates
53bd9ead64 scripts: twisterlib: coverage: fix multiple branch excludes
Multiple values for `--exclude-branches-by-pattern` will result in only
the last value taking effect. Resolve this by merging all the provided
regex patterns into a single pattern with the `|` operator.

Signed-off-by: Jordan Yates <jordan@embeint.com>
2024-07-08 20:41:31 -04:00
Sylvio Alves
e04964aebd requirements: bump pyelftools to >=0.29
The minimum version of pyelftools is 0.29 to make it working
with scripts/footprint/size_report

Fixes #75605

Signed-off-by: Sylvio Alves <sylvio.alves@espressif.com>
2024-07-08 17:18:29 -04:00
Hake Huang
e0f772e880 test: pytest: add linkserver support in pytest
pytest supporting add linkserver for nxp platforms

Signed-off-by: Hake Huang <hake.huang@oss.nxp.com>
2024-07-08 09:23:48 +02:00
Grzegorz Swiderski
7a10c7dff1 scripts: west_commands: Fix west boards for SoC and arch in modules
Fixes #71761

The `west boards` command parses extra BOARD_ROOTs from Zephyr modules,
so that the boards defined in those modules are automatically listed.

In HWMv2, OOT boards can be described in terms of OOT SoCs, which means
that extra SOC_ROOTs must also be provided. Otherwise, an error message
will be displayed when attempting to list all boards. Therefore, every
Zephyr module SOC_ROOT should be included as well.

In HWMv1 (deprecated), OOT boards can be defined in terms of OOT archs,
but module ARCH_ROOTs had never been included automatically. The fix for
this is long overdue, but it's included for symmetry.

Signed-off-by: Grzegorz Swiderski <grzegorz.swiderski@nordicsemi.no>
2024-07-05 14:55:08 +02:00
Yuval Peress
1b51740fa3 twister: Fix gTest harness
The gTest harness asssumed that the lines end with the test name, but
some gTest implementations include the test duration in the line. Update
both the tests and regex to allow this and also avoid capturing
characters into the `test_name` that cannot be valid test name chars.

Fixes #72318

Signed-off-by: Yuval Peress <peress@google.com>
2024-07-03 08:52:13 +02:00
Henrik Brix Andersen
e5aad45bb1 west: commands: runners: canopen: increase default SDO timeout
Increase the default SDO timeout for the CANopen program download west
runner from 0.3 seconds to 1 second. Depending on the flash size and speed,
a full erase may take slightly longer than 300 ms.

The timeout can be customized by using the --sdo-timeout runner parameter.

Fixes: #73987

Signed-off-by: Henrik Brix Andersen <hebad@vestas.com>
2024-07-02 09:20:57 +02:00
Gerson Fernando Budke
768b8bbca3 scripts: west robot & simulation: Fix OOT
The current version of scipts do not consider OOT boards use cases and
the tests with robot now are strict to only one robot file, which is
not realistic for real environment. This address those issues and allow
multiple testsuits at command line and lists at tests entries. It add
another test parameter to allow configure robotframework options.

Fixes: #74563

Signed-off-by: Gerson Fernando Budke <gerson.budke@ossystems.com.br>
2024-07-01 16:10:35 -04:00
Henrik Brix Andersen
c4c5f168e8 ci: ignore changes to the CANopen program download runner
Ignore changes to the CANopen program download west runner in CI.

This script is only executed when performing a DFU using the CANopen
protocol via CAN, which is never triggered in CI anyways.

Signed-off-by: Henrik Brix Andersen <hebad@vestas.com>
2024-07-01 16:05:44 -04:00
Maciej Perkowski
a033ba65d2 twister: Add warning to the "-footprint-from-buildlog" flag
The flag won't work with sysbuild since there is no way to
reliably tell to a parser which data came from which image.

fixes: #74092

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2024-06-28 21:12:57 -04:00
Daniel Leung
2eb0ffb7ea scripts: logging/dictionary: fix annoying pylint warning
pylint keeps failing and complaining about arg_data_type is
used before assignment. So assign it to None to silence
the warning.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2024-06-28 21:14:14 +02:00
Daniel Leung
f5762e07f2 logging: dictionary: new verion 3 format
The package_len has been extended from 10 bits to 11 bits
in the log message header. So the format for dictionary
logging also needs to be updated.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2024-06-28 21:14:14 +02:00
Daniel Leung
0e33c7a976 scripts: logging/dictionary: replace %#llx too
Python does not really support long long double, so %llx cannot
be formatted correctly, so we replace it with a simple %lx.
There is another variant %#llx and we also need replace it to
%#lx.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2024-06-28 21:14:14 +02:00
Daniel Leung
ecc642b2df scripts: logging/dictionary: refactor common parser functions...
... and put them into the LogParser class file instead of
the verisoned parser. This is in preparation for introducing
a new parser version.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2024-06-28 21:14:14 +02:00
Daniel Leung
3da42fb547 scripts: logging/dictionary: fix long long alignment
Printing long long requires alignment on 64-bit before parsing
the actual argument. Or else the parser would be looking at
some unrelated bits. So fix it.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2024-06-28 21:14:14 +02:00
Daniel Leung
eacf71d92a scripts: logging/dictionary: extract DataTypes into its own file
This extracts the DataTypes class into its own file. This is in
preparation to add a new version of parser which can reuse this
class.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2024-06-28 21:14:14 +02:00
Louis Feller
5b4c8945fd west : runners : Fix error message when runners.yaml is not found.
Adjust error message so that it clearly states runners.yaml is
missing from <build_dir>/zephyr, instead of referencing CMake cache
variable ZEPHYR_RUNNERS_YAML, which is no longer used (since
3124c02987 ).
Also clean up that variable in CMake since it is no longer used
(0 other references in entire tree).
Fixes #70605

Signed-off-by: Louis Feller <louis.feller@st.com>
2024-06-28 12:23:18 -04:00
Gerard Marull-Paretas
4753005c55 scripts: get_maintainer: check maintenance status
Error if an area is set as maintained but it has no maintainers
assigned.

Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
2024-06-28 07:19:49 -04:00
Jordan Yates
243eb36b18 everywhere: reindent .overlay files with tabs
`checkpatch.pl` requires that dts sources are indented with tabs,
fix all the spaces that slipped in while checkpatch wasn't watching.

Signed-off-by: Jordan Yates <jordan@embeint.com>
2024-06-26 15:59:44 +02:00
Félix Turgeon
5cd580ce44 scripts: coredump: Add register write handler to arm cortex-m gdbstubs.
When working with coredumps, it is useful to be able to modify base
registers. Adding this capability allows implementing scripts to
inspect backtrace of threads other than current the current thread.

Signed-off-by: Félix Turgeon <felixturgeon@meta.com>
2024-06-25 19:14:20 -04:00
Lingao Meng
be96512d82 scripts: spelling.txt: add double prepositions
Add double prepositions to the default spelling check list.

Signed-off-by: Lingao Meng <menglingao@xiaomi.com>
2024-06-25 06:05:35 -04:00
Lingao Meng
302422ad9d everywhere: replace double words
import os
import re

common_words = set([
    'about', 'after', 'all', 'also', 'an', 'and',
     'any', 'are', 'as', 'at',
    'be', 'because', 'but', 'by', 'can', 'come',
    'could', 'day', 'do', 'even',
    'first', 'for', 'get', 'give', 'go', 'has',
    'have', 'he', 'her',
    'him', 'his', 'how', 'I', 'in', 'into', 'it',
    'its', 'just',
    'know', 'like', 'look', 'make', 'man', 'many',
    'me', 'more', 'my', 'new',
    'no', 'not', 'now', 'of', 'one', 'only', 'or',
    'other', 'our', 'out',
    'over', 'people', 'say', 'see', 'she', 'so',
    'some', 'take', 'tell', 'than',
    'their', 'them', 'then', 'there', 'these',
    'they', 'think',
    'this', 'time', 'two', 'up', 'use', 'very',
    'want', 'was', 'way',
    'we', 'well', 'what', 'when', 'which', 'who',
    'will', 'with', 'would',
    'year', 'you', 'your'
])

valid_extensions = set([
    'c', 'h', 'yaml', 'cmake', 'conf', 'txt', 'overlay',
    'rst', 'dtsi',
    'Kconfig', 'dts', 'defconfig', 'yml', 'ld', 'sh', 'py',
    'soc', 'cfg'
])

def filter_repeated_words(text):
    # Split the text into lines
    lines = text.split('\n')

    # Combine lines into a single string with unique separator
    combined_text = '/*sep*/'.join(lines)

    # Replace repeated words within a line
    def replace_within_line(match):
        return match.group(1)

    # Regex for matching repeated words within a line
    within_line_pattern =
	re.compile(r'\b(' +
		'|'.join(map(re.escape, common_words)) +
		r')\b\s+\b\1\b')
    combined_text = within_line_pattern.
		sub(replace_within_line, combined_text)

    # Replace repeated words across line boundaries
    def replace_across_lines(match):
        return match.group(1) + match.group(2)

    # Regex for matching repeated words across line boundaries
    across_lines_pattern = re.
		compile(r'\b(' + '|'.join(
			map(re.escape, common_words)) +
			r')\b(\s*[*\/\n\s]*)\b\1\b')
    combined_text = across_lines_pattern.
		sub(replace_across_lines, combined_text)

    # Split the text back into lines
    filtered_text = combined_text.split('/*sep*/')

    return '\n'.join(filtered_text)

def process_file(file_path):
    with open(file_path, 'r', encoding='utf-8') as file:
        text = file.read()

    new_text = filter_repeated_words(text)

    with open(file_path, 'w', encoding='utf-8') as file:
        file.write(new_text)

def process_directory(directory_path):
    for root, dirs, files in os.walk(directory_path):
        dirs[:] = [d for d in dirs if not d.startswith('.')]
        for file in files:
            # Filter out hidden files
            if file.startswith('.'):
                continue
            file_extension = file.split('.')[-1]
            if
	file_extension in valid_extensions:  # 只处理指定后缀的文件
                file_path = os.path.join(root, file)
                print(f"Processed file: {file_path}")
                process_file(file_path)

directory_to_process = "/home/mi/works/github/zephyrproject/zephyr"
process_directory(directory_to_process)

Signed-off-by: Lingao Meng <menglingao@xiaomi.com>
2024-06-25 06:05:35 -04:00
Jordan Yates
1ab4fe68d1 scripts: spelling.txt: add double prepositions
Add double prepositions to the default spelling check list.

Signed-off-by: Jordan Yates <jordan@embeint.com>
2024-06-22 05:40:22 -04:00
Jordan Yates
07870934e3 everywhere: replace double words
Treewide search and replace on a range of double word combinations:
    * `the the`
    * `to to`
    * `if if`
    * `that that`
    * `on on`
    * `is is`
    * `from from`

Signed-off-by: Jordan Yates <jordan@embeint.com>
2024-06-22 05:40:22 -04:00
Jordan Yates
79f81f6eb8 scripts: checkpatch.pl: treat .overlay files as dts
Treat devicetree overlay files as DTS source for the purposes of
checkpatch.

Fixes #74570.

Signed-off-by: Jordan Yates <jordan@embeint.com>
2024-06-21 08:46:12 -04:00
Zhani Baramidze
6ced73e552 scripts: fixed missing argument in parse_from_attribute call.
In size_report script, if the DWARF section of ELF file contains both
debug_loc and a debug_loclists sections, LocationListsPair class is
used to track locations. In that case, parse_from_attribute was missing
one argument which was causing the script to fail.

Signed-off-by: Zhani Baramidze <jbaramidze@meta.com>
2024-06-20 17:03:21 -04:00
Yong Cong Sin
7adac57943 scripts: checkpatch: add stdio.h FILE typedef
Add `FILE` typedef in the `stdio.h` so that when doing
`FILE *file` definition checkpatch doesn't complain about
the position of the '*' and fail in CI.

Signed-off-by: Yong Cong Sin <ycsin@meta.com>
2024-06-20 14:07:14 -04:00
Carles Cufi
4fecad4fe0 scripts: runners: nrf_common: Fix recover on 54H20
In order for the --recover option to work properly on the nRF54H20, it
requires executing it for both cores, the radio and the application one.
Extend the recover_target() function so that it does so for both 53 and
54H20.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2024-06-20 12:05:39 -04:00
Anas Nashif
1a46b7ed0e ci: assignees: set multiple assignees of same area
Noticing many PRs that wait too long in the queue although once of the
maintainers approved with asignees set to other maintainers.

This changes the current behavior of picking the first maintainer in the
list and assigning to them only, instead we assign to all maintainers of
the main area being changed.

Who ends up driving the PR to a mergeable state is then to the
maintainers and they can unassign/assign based on availability.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-06-18 14:39:22 -04:00
Yong Cong Sin
88c96f7986 drivers: counter: stm32: remove deprecated clock source Kconfigs
These clock selection Kconfigs should have been deprecated for
more than 2 releases, remove them:

- `CONFIG_COUNTER_RTC_STM32_CLOCK_SRC`
- `CONFIG_COUNTER_RTC_STM32_CLOCK_LSI`
- `CONFIG_COUNTER_RTC_STM32_CLOCK_LSE`

Signed-off-by: Yong Cong Sin <ycsin@meta.com>
2024-06-18 14:33:58 -04:00
Luca Burelli
38b0460de3 check_compliance: allow _MODULE variant of Kconfig options
Since b53a792ff0 Zephyr has the ability to define tristate Kconfig
options. When a tristate option FOO is selected as a "module", this
results in autoconf.h defining CONFIG_FOO_MODULE, not CONFIG_FOO.

This patch allows the check_compliance script to also accept references
to a Kconfig symbol ending in _MODULE if the prefix is defined.

Signed-off-by: Luca Burelli <l.burelli@arduino.cc>
2024-06-18 15:54:39 +02:00
Marcin Niestroj
efcc0ed27f scripts: pytest: execute 'zephyr.exe' in application build directory cwd
Execute 'zephyr.exe' in application build directory as 'current working
directory' (cwd). This makes sure that native_sim specific drivers (like
flash simulator with file backend in 'flash.bin') are using unique context
for external resources with relative paths.

This fixes executing native_sim tests in twister with flash simulator.
Previously a shared 'flash.bin' was used for all executed 'zephyr.exe'
processes in twister. After this patch a unique 'flash.bin' file is used
for each tested sample, since those 'flash.bin' is placed in application
build directory instead of twister root directory.

Signed-off-by: Marcin Niestroj <m.niestroj@emb.dev>
2024-06-18 15:53:00 +02:00
Torsten Rasmussen
a9efb1dae6 west: update build extension command to use APP_DIR
`west build` can be invoked without specifying the source directory when
being invoked from the source directory itself.
When using `west build` for incremental builds, then the build command
will examine the CMake cache to determine the application dir by using
the value of CMAKE_HOME_DIRECTORY.

With sysbuild, this leads to the wrong assumption that the sysbuild
itself is the application to build.

Instead, have west build look for APP_DIR which points to the correct
source dir when sysbuild is used. Use APPLICATION_SOURCE_DIR when
APP_DIR is not set, as this indicates a no-sysbuild build.

Keep CMAKE_HOME_DIRECTORY behavior as last fallback mechanism.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2024-06-17 12:09:45 -04:00