Commit graph

5595 commits

Author SHA1 Message Date
Carles Cufi
16f212d6f2 scripts: runners: nrf54H/92: Use RESET_HARD instead of RESET_SYSTEM
The nRF54H and the nRF92 series do not support the RESET_SYSTEM reset
kind. Instead one must use RESET_HARD, which this patch now implements.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2025-02-07 07:38:36 +01:00
Carles Cufi
7254a61530 scripts: runners: nrf: Fix flashing of FLPR/PPR on nRF54H20
After commit aaefaad, flashing of the FLPR and PPR cores in the nRF54H20
was broken due to those cores missing from the _get_core() function that
retrieves the core to program, triggering an exception. Fix it by
referencing those cores and mapping them to the application core.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2025-02-06 14:47:25 +01:00
Hake Huang
c268f15ae3 twister: testinstance: compare hwmap platform name with alias
in hwmap we may use short name, so need compare alias

Signed-off-by: Hake Huang <hake.huang@oss.nxp.com>
2025-02-06 07:44:55 +01:00
Benjamin Cabé
ead3a016cf scripts: footprint: Update hifive1 board name
Rev B of hifive1 is 'hifive1@B' since HWMv2 switch

Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
2025-02-05 21:03:08 +01:00
Carles Cufi
2effae5594 scripts: runners: nrfutil: Add a new external memory config param
This acts in a similar way to nrfjprog's --qspiini parameter introduced
in 7eb364b, but works slightly differently in nrfutil: it requires a
config file in JSON format instead of .ini, and it is passed along to
the `execute-batch` command instead of providing it together with the
`program` command.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2025-02-05 21:01:57 +01:00
Carles Cufi
f20168fe89 scripts: runners: nrf: Honor the --erase flag for external memory
Both backends supported as runners for nRF ICs, nrfjprog and nrfutil,
support erasing external memory as part of the programming operation.
Before this patch, and when the firmware was detected to be partially or
fully placed in external flash by inspecting the .hex address range, the
runner would instruct the backend tool to fully erase the external
flash (but the nrfjprog runner would ignore that, always erasing only
the sectors required). Instead, correctly default to erasing only the
sectors that are required to program the new firmware image in both tools,
and erase it completely only when the --erase flag is provided by the user.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2025-02-05 21:01:57 +01:00
Jamie McCrae
f8a5ebfd20 scripts: ci: check_compliance: Add check for disallowed Kconfigs
Adds a check for Kconfig settings that are disallowed from being
used in defconfig files (because it is the complete wrong place
for them), with the default entry being PINCTRL

Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
Co-authored-by: Benjamin Cabé <benjamin@zephyrproject.org>
2025-02-05 17:49:27 +01:00
Dmitrii Golovanov
e9cb2ef86b twister: runner: Log trace on general exception
Log trace on general exception at ProjectBuilder to provide
more details on its cause.

Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
2025-02-05 14:59:00 +01:00
Carles Cufi
17a671ca00 scripts: runners: nrf: Default to soft reset for the nRF52 series
The Nordic nRF52 series have a peculiarity that is not shared with any
other Nordic families of SoCs: the reset pin can be reconfigured as a
regular GPIO. This has an unintended consequence: if the user has
reconfigured the pin in Devicetree to be a GPIO, `west flash` will
override that and configure the IC to use it as a reset pin, and the
firmware at boot won't be able to switch it back to GPIO, because that
requires a UICR erase. This behavior is very confusing to users, because
the GPIO does not work at all, since it is now just a reset line.

With this patch, `west flash` defaults to using soft reset instead of
pin reset for the nRF52 family of devices, to avoid overwriting the
reset pin configuration that the user includes in the image.

In order to be able to continue to use pin reset for users that so
desire it, a new option `--pinreset` is added that forces the use of pin
reset. The existing `--softreset` option is left exactly as it was, but
it is now applicable only to families other than the nRF52.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2025-02-05 01:13:42 +01:00
Carles Cufi
706689b376 scripts: runners: nrf: Fix missing kind in erase operations
Commit dc7d8bb introduced the requirement to provide the erase kind when
executing the corresponding op, but it was forgotten in this particular
spot.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2025-02-04 15:01:36 +01:00
Erwan Gouriou
b859423051 scripts: twister: Board id not needed for stm32cubeprog in BOOT-SERIAL
Flashing in BOOT-SERIAL mode with stm32cubeprogrammer doesn't support
using serial number.

Signed-off-by: Erwan Gouriou <erwan.gouriou@st.com>
2025-02-03 19:56:49 +01:00
Erwan Gouriou
a37a9c323a scripts: stm32cubeprogrammer: Add 2 new options for start and download
Implement possibility to specify options for --start and --download
arguments.
These options are implemented as lists to better fit STM32CubeProgrammer
programming model.

This options are required to enable the programming of STM32N6 in USB-DFU
mode, which requires "-d my.bin 0x1 -s noack" arguments list.

Signed-off-by: Erwan Gouriou <erwan.gouriou@st.com>
2025-02-03 19:56:49 +01:00
Chris Friedt
7bd0e84d20 west: commands: blobs: fail on checksum error
If a blob checksum does not match what is recorded in the
associated module.yml file, `west blobs fetch` should
(eventually) fail.

Signed-off-by: Chris Friedt <cfriedt@tenstorrent.com>
2025-02-03 19:52:58 +01:00
Carles Cufi
dc7d8bbe16 scripts: runners: nrf: Implement the erase command
The `erase` command was being used but there was no "backend"
implementation for it in nrfutil nor nrfjprog. Simplify the syntax and
implement it for both tools.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2025-02-03 16:59:40 +01:00
Carles Cufi
4e8ec169fb scripts: runners: nrfjprog: Remove erasepage handling
This is no longer used at all.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2025-02-03 16:59:40 +01:00
Carles Cufi
fa66984c2b scripts: runners: nrf: Add UICR ranges for nRF54L
These are used to detect if the firmware image contains any values
inside the UICR for additional handling.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2025-02-03 16:59:40 +01:00
Carles Cufi
aaefaad898 scripts: runners: nrf: Remove the nRF5340 special handling
This is no longer necessary, because thanks to sysbuild we no longer
invoke a runner with a .hex file that is the result of merging builds
for more than one core.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2025-02-03 16:59:40 +01:00
Jacob Wienecke
8e778d86f7 scripts: ci: check_compliance.py: Fix perl script running on windows
Add an if clause to the CheckPatch Class that checks for windows os.
If windows os, check for perl installation.
If no perl installation present. Fail the check.
Without this change, CheckPatch can fail silently on windows,
as windows does not have a way to run perl scripts.

Signed-off-by: Jacob Wienecke <jacob.wienecke@nxp.com>
2025-02-01 11:27:28 +01:00
Daniel DeGrasse
b9fa7b95fb scripts: twisterlib: write full output of from pytest to twister harness
When pytest is executed, it only writes logging messages to the twister
harness, and does not include information about any test assertion
failures. This information is printed to stdout, so simply write
the contents stdout to the harness log.

Remove the log file arguments previously passed to pytest, as these are
no longer needed.

Signed-off-by: Daniel DeGrasse <ddegrasse@tenstorrent.com>
2025-01-31 14:02:59 +01:00
TOKITA Hiroshi
b739fbd20a scripts: west_commands: sdk: Display a download progress bar.
Improved the display using tqdm to show the download progress.

Signed-off-by: TOKITA Hiroshi <tokita.hiroshi@gmail.com>
2025-01-31 09:38:09 +01:00
Christophe Dufaza
ad22d34d16 edtlib: amend Node.props API documentation
Node properties are not created for all properties defined
by the node's binding (Binding.prop2specs),
only for those that actually have a value.

Signed-off-by: Christophe Dufaza <chris@openmarl.org>
2025-01-30 18:29:28 +01:00
Christophe Dufaza
c6e3de296d edtlib: amend PropertySpec.path API documentation
PropertySpec.path does NOT tell "the file where the property
was last modified", but instead the binding file specifying
the devicetree node of which this is a property.

See: #65135

Signed-off-by: Christophe Dufaza <chris@openmarl.org>
2025-01-30 18:29:28 +01:00
Dmitrii Golovanov
53f7f6bcd1 twister: coverage: Merge gcovr reports
In `--coverage-per-instance` mode for 'gcovr' merge individual
coverage.json reports into the aggregate code coverage report
instead of processing .gcda data files again.

Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
2025-01-30 18:29:08 +01:00
Dmitrii Golovanov
f93f82f160 twister: coverage: Data collection and reporting per-test instance
With this change, the coverage data (GCOV dump) is extracted from the
test log files after each of the test instance parallel execution,
instead of post processing all the logs at the end of the Twister run.

The new `--coverage-per-instance` mode extends Twister coverage operations
to report coverage statistics on each test instance execution individually
in addition to the default reporting mode which aggregates data to one
report with all the test instances in the current scope of the Twister run.
The split mode allows to identify precisely what amount of code coverage
each test suite provides and to analyze its contribution to the overall
test plan's coverage. Each test configuration's output directory will have
its own coverage report and data files, so the overall disk space and
the total execution time increase.

Another new `--disable-coverage-aggregation` option allows to execute
only the `--coverage-per-instance` mode when the aggregate coverage
report for the whole Twister run scope is not needed.

Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
2025-01-30 18:29:08 +01:00
Dmitrii Golovanov
220f251241 twister: coverage: Group command line arguments
Group Twister code coverage command line arguments for better usability.
Add more details in descriptions.

Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
2025-01-30 18:29:08 +01:00
Jamie McCrae
d9c5210dc8 scripts: requirements: Remove MCUboot imgtool requirement
MCUboot's imgtool requirement has been moved directly to MCUboot
which can be installed using west packages if needed, remove it
from Zephyr's requirements

Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
2025-01-30 14:16:50 +01:00
David Leach
21aefb60ea twister: add details on a depends_on filter hit
In situations where a test won't be run because there is a gap
in the hardware support, add the gap information to the "reason"
field to aid in debugging test cases.

With this change, the reason field for the error in test case
will show the gap:

e.g. reason field now updated with gaps

"No hardware support {'usbd'} but is one of the integration platforms"

Signed-off-by: David Leach <david.leach@nxp.com>
2025-01-30 07:20:38 +01:00
Carles Cufi
51661efae6 scripts: runners: nrf: Fix a missing indirection
In commit 6e9e839195, an indirection via
the new `options` dictionary was missed in the SUIT handling.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2025-01-29 15:13:46 +01:00
Benjamin Cabé
652e183037 Revert "scripts: support list of images in build info"
This reverts commit 034c36cfad.
as it's part of a series of commits causing issues on Windows

Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
2025-01-29 09:27:31 +01:00
Lukasz Mrugala
5834525649 scripts: twister: Fix too early error clearing
Actually using the error count before clearing it
would probably be more useful.
ExecutionCounter's error count was set to 0
before being used to lower the done count properly.
This should fix in-progress overcounting,
where Twister claimed to execute more tests than planned.

Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
2025-01-28 23:43:00 +01:00
Guillaume Gautier
5e40b65fe9 scripts: stm32cubeprogrammer: Add possibility to flash a bin file
Add the option to flash a bin file if one is available from the board's
cmake configuration.
Use of elf has prio over this, and bin has prio over hex which remains
the default option.
Also add a parameter to stm32cubeprogrammer runner to specify the download
address.

Signed-off-by: Guillaume Gautier <guillaume.gautier-ext@st.com>
2025-01-28 18:14:45 +01:00
Gerhard Jörges
4455cd7351 scripts: west_commands: patch: add src module argument
The src-module argument can be used to select the module in which west
patch searches for patch definitions. This commit also allows the use of
module names instead of the path.

Signed-off-by: Gerhard Jörges <joerges@metratec.com>
2025-01-28 12:00:45 +01:00
Irfan Ahmad
e30c108728 scripts: build: Optimize hex character list generation in file2hex.py
Read and convert binary data in chunks of 1024 bytes, instead of 8 bytes.
This significantly reduces the conversion time. The improvement increases
with increasing file sizes but saturates to around 60-61% as file size
approaches 64MiB, and beyond.

The existing generated output format of eight byte-values per line is still
preserved.

Signed-off-by: Irfan Ahmad <irfan.ahmad@siemens.com>
2025-01-28 09:49:01 +01:00
Carles Cufi
6e9e839195 scripts: runners: nrfutil: Switch to command-line args instead of JSON
Instead of pre-generating the JSON batch to then execute it, use
command-line arguments and --x-append-batch to generate the JSON file
tht will then be passed to x-execute-batch.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2025-01-27 21:04:06 +01:00
Torsten Rasmussen
034c36cfad scripts: support list of images in build info
Add 'images' as valid list in build info schema.
This allow sysbuild and regular Zephyr images to populate build_info.yml
with images included in the build.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2025-01-27 13:24:23 +01:00
Luca Burelli
7094eae111 llext: support multiple string and symbol tables
The ELF format allows for multiple string and symbol tables with
complex references between them. This is especially evident when
debugging information is included.

This patch fixes the issues that have been identified with multiple
string tables to allow LLEXT to properly parse those files:

* The symbol table used by LLEXT (LLEXT_MEM_SYMTAB) is now chosen
  depending on the loaded file type, and other tables are ignored.
  This change is also applied to the SLID injection script.

* The LLEXT string table (LLEXT_MEM_SYMTAB) is now correctly identified
  by the symbol table reference, instead of picking the first one.

* VMA range checks only make sense for allocated sections.

Signed-off-by: Luca Burelli <l.burelli@arduino.cc>
2025-01-27 08:54:44 +01:00
Arkadiusz Cholewinski
c5c061a5c0 Coverage: Update color to font_color in header_format
Updated the code to use the font_color property, which
is the correct attribute for setting the text color in
recent versions of xlsxwriter (3.2.1).

Signed-off-by: Arkadiusz Cholewinski <arkadiuszx.cholewinski@intel.com>
2025-01-24 08:39:16 +01:00
Dmitrii Golovanov
aa705089d3 twister: reporting: Add custom JSON encoder
Refactor Reporting to use custom ReportingJSONEncoder and encode
all pathlib.Path derived instances there which are possible
e.g. in 'environment.options' received from command line.

Fixes: #83823

Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
2025-01-23 19:22:40 +01:00
Yong Cong Sin
085df8bd1e arch: remove z_arch_esf_t
The exception stack frame type `z_arch_esf_t` had been deprecated
since #73593 for 2 releases, it is not used in the kernel since, and
applications/drivers should have been updated to use the
`struct arch_esf` now, remove it.

Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Signed-off-by: Yong Cong Sin <yongcong.sin@gmail.com>
2025-01-23 16:32:36 +01:00
Luca Burelli
16d71d0598 edtlib: add "hash" attribute to nodes
Add a new "hash" attribute to all Devicetree EDT nodes. The hash is
calculated on the full path of the node; this means that its value
remains stable across rebuilds.
The hash is checked for uniqueness among nodes in the same EDT.

This computed token is then added to `devicetree_generated.h` and made
accessible to Zephyr code via a new DT_NODE_HASH(node_id) macro.

Signed-off-by: Luca Burelli <l.burelli@arduino.cc>
2025-01-22 15:49:10 +01:00
Jordan Yates
ced3df6873 runners: openocd: support --erase
Add support for the `--erase` option on `west flash`, for boards that
specify at least one `--cmd-erase` option.

Signed-off-by: Jordan Yates <jordan@embeint.com>
2025-01-21 19:27:27 +01:00
Pieter De Gendt
18f02a76f4 scripts: west_commands: patch: Add gh-fetch subcommand
Add a gh-fetch subcommand to the west patch extension to download a patch
file from Github and generate the patch meta data.

The patch info is appended to the patches.yml file.

Signed-off-by: Pieter De Gendt <pieter.degendt@basalte.be>
2025-01-20 16:26:08 +01:00
Grzegorz Swiderski
15488be6af scripts: dts: Refactor gen_driver_kconfig_dts
The motivation for this patch was to improve the script's performance,
but some stylistic changes and cleanups are included as well.

The main optimization concerns the use of PyYAML, as it offers multiple
functions for interpreting YAML. The commonly used `load`/`safe_load`
converts a YAML stream to a dictionary. There are also `scan`, `parse`,
and `compose`, which return intermediate representations, the last one
being a graph. [1]

Since `gen_driver_kconfig_dts` scans DT bindings for compatible strings,
it only needs to look through top level keys in YAML. The intermediate
PyYAML graph is sufficient for this, and using it reduces the script's
execution time by about 30%, without making the code too complicated.

[1] - https://pyyaml.org/wiki/PyYAMLDocumentation

Signed-off-by: Grzegorz Swiderski <grzegorz.swiderski@nordicsemi.no>
2025-01-19 07:57:57 +01:00
Grzegorz Swiderski
7ce6ac225e scripts: ci: check_compliance: Add sysbuild Kconfig checks
Introduce sysbuild-specific variants of existing Kconfig checks:

  * SysbuildKconfig
  * SysbuildKconfigBasic
  * SysbuildKconfigBasicNoModules

This involves a few additions to the base `KconfigCheck` class:

  * Supporting a variable symbol prefix, to handle `SB_CONFIG_`.
  * Generating extra files, including `Kconfig.sysbuild.modules`.
    Although these are never sourced outside of sysbuild Kconfig,
    they're still generated for every regular Zephyr build, so it's
    natural to let all Kconfig checks follow this behavior.

Signed-off-by: Grzegorz Swiderski <grzegorz.swiderski@nordicsemi.no>
2025-01-17 16:35:53 +01:00
Grzegorz Swiderski
37bce60188 scripts: ci: check_compliance: Inline creation of HWMv2 temp dirs
The KCONFIG_BINARY_DIR/{boards,soc,arch}/ directories are only relevant
for `KconfigCheck.get_v2_model()`, so generate them within this method.

Signed-off-by: Grzegorz Swiderski <grzegorz.swiderski@nordicsemi.no>
2025-01-17 16:35:53 +01:00
Grzegorz Swiderski
c0d5e853a1 scripts: ci: check_compliance: Remove HWMv1 dead code
HWMv1-specific Kconfig checks were meant to be removed before HWMv2 was
merged into `main` (see `collab-hwm` history), but some traces remained.
Do another round of cleanup.

Signed-off-by: Grzegorz Swiderski <grzegorz.swiderski@nordicsemi.no>
2025-01-17 16:35:53 +01:00
Grzegorz Swiderski
50d9ed5ce1 scripts: ci: check_compliance: Update KconfigCheck subclassing
So far, the behavior of different Kconfig checks has been parametrized
using the `run()` method, and every new check has introduced with it a
new argument to that method.

It's possible to replace each `run()` argument by way of overriding
different class methods and making better use of inheritance:

    free=False          Stub check_no_undef_outside_kconfig()
    no_modules=True     Stub get_modules()
    filename            Introduce class member FILENAME
    hwm                 (unused)

This should establish a more scalable and straightforward pattern for
adding future Kconfig checks. It also favors composability, which will
come in handy when implementing checks for sysbuild Kconfig.

Additionally, avoid duplicating `doc` and `path_hint` in every subclass.

Signed-off-by: Grzegorz Swiderski <grzegorz.swiderski@nordicsemi.no>
2025-01-17 16:35:53 +01:00
Thomas Günther
c2d011f366 twister: fix shell prompt detection with VT100 colors disabled
Device adapter strips all whitespace from output lines causing test
failures when matching default shell prompt "uart:~$ " with trailing
space. Update _handle_device_output to only strip line endings (\r\n)
while preserving whitespace required for prompt detection.

A testcase sample.pytest.shell.no_vt100 was added to verify prompt
matching works with CONFIG_SHELL_VT100_COLORS disabled.

Signed-off-by: Thomas Günther <thomas.guenther@limatica.com>
2025-01-17 16:35:33 +01:00
Stephanos Ioannidis
0e6f6cfb12 twister: Remove 'xtools' toolchain variant references
This commit removes all references to the `xtools` toolchain variant in the
twister scripts.

Note that the `xtools` toolchain variant has been deprecated since Zephyr
v3.3.0 and now removed.

Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
2025-01-17 10:50:07 +01:00
Dmitrii Golovanov
b4d22420e9 twister: harness: recording: Allow multiple patterns
Extend Twister Harness 'recording' feature to allow multiple
regular expression patterns to extract different types of records
from test output.

Add 'merge' recording mode to collect all extracted data fields
into a single record object of the test instance.

Export to CSV file now takes all field names occurred in the collected
records, sort it alphabetically, and then use it for columns instead of
using only the first record's fields. This is done to address possible
situation when records have different set of fields.

Adjust Twister documentation and test suite to the above changes.

Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
2025-01-16 22:38:51 +01:00