Commit graph

5778 commits

Author SHA1 Message Date
Fin Maaß
9d0627a4bc scripts: west_commands: add --ignore-venv-check option
add --ignore-venv-check option to ignore the venv check.

Signed-off-by: Fin Maaß <f.maass@vogl-electronic.com>
2025-01-07 20:35:53 +01:00
Pieter De Gendt
4c5e74deae scripts: west_commands: patch: Use subprocess cwd
Instead of changing directories, pass the cwd argument to the different
subprocess calls.

Signed-off-by: Pieter De Gendt <pieter.degendt@basalte.be>
2025-01-07 14:12:57 +01:00
Pieter De Gendt
78f96a6aba scripts: west_commands: Add module filter argument to patch command
Allow to run the west patch extension for specific modules only.

Signed-off-by: Pieter De Gendt <pieter.degendt@basalte.be>
2025-01-07 14:12:57 +01:00
Benjamin Cabé
bc42004d1b scripts: ci: check_compliance: do not use git mailmap
When looking at git author in a commit range using git log, explicitly
indicate we don't want to use the mailmap file (default git client
behavior is to use it), so that the "actual" author info from the commit
is used to perform the compliance check, and not whatever identity the
author may have mapped to in the mailmap file (ex. their "main" email
address might be different from the one they used to commit).

Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
2025-01-07 11:53:20 +01:00
David Leach
90dfeaf61c scripts: west_commands: runners: Fix broken jlink runner
The cleanup pass (336c7da) to address long lines accidentally
removed a needed comma in the jlink runner.

Fixes #83605

Signed-off-by: David Leach <david.leach@nxp.com>
2025-01-07 00:27:28 +01:00
James Roy
802eac71f0 style: edtlib: Use a better line continuation operator
Replace backslashes('\') with PEP-8 recommended
parentheses('( )') as the line continuation operator.

Signed-off-by: James Roy <rruuaanng@outlook.com>
2025-01-04 14:15:37 +01:00
Gerhard Jörges
e4b2fe4b78 scripts: west_commands: fix type of path options
Using `west patch` with the `--patch-base`, `--patch-yaml` or
`--west-workspace` option results in an unhandled AttributeError in
filter_args() because the function expects these args to be of type
path. Adding the type to the argument definition forces argparse to
create variables of type path.

Signed-off-by: Gerhard Jörges <joerges@metratec.com>
2025-01-03 17:10:58 +00:00
Carles Cufi
3c420a4828 checkpatch: Adapt the braces check to Zephyr
scripts/checkpatch.pl was written originally for the Linux kernel, and
its code reflects the kernel's coding style. In particular, it has
checks for unneeded braces around single-statement if/else/for/while
conditions. In Zephyr however, braces are always required, and so the
checks needed modifying to verify the opposite condition.

In order to enable the now-compatible checks, we also remove the
--ignore BRACES statement in .checkpatch.conf.

Limitations: the current code works well if there are not conditional
statements (e.g. #if, #ifdef or #endif) next to the if/else/for/while
conditions. This is rarely the case, but triggers with the Bluetooth
controller in code like this:

```
 #if defined(CONFIG_BT_PERIPHERAL)
        if (!lll->is_hdcd)
 #endif /* CONFIG_BT_PERIPHERAL */
        {
```

```
        } else
 #endif /* CONFIG_BT_CTLR_PRIVACY */
        {

```

```
 #if defined(CONFIG_BT_CTLR_DF_ADV_CTE_TX)
        if (lll->cte_started) {
                radio_switch_complete(phy_s, 0, phy_s, 0);
        } else
 #endif /* CONFIG_BT_CTLR_DF_ADV_CTE_TX */
        {
```

```
 #ifdef DUAL_BANK
        while ((FLASH_STM32_REGS(dev)->SR1 & FLASH_SR_QW) ||
               (FLASH_STM32_REGS(dev)->SR2 & FLASH_SR_QW))
 #else
        while (FLASH_STM32_REGS(dev)->SR1 & FLASH_SR_QW)
 #endif
        {
```

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2024-12-30 21:46:47 +01:00
Guennadi Liakhovetski
6d0c23be31 soc: intel_adsp: add support for .cold* sections
Add support for .cold and .coldrodata sections, used to implement
a cold module. Assigning code and read-only data to those sections
makes them suitable for use in a cold module, stored and executed in
slow memory.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
2024-12-29 05:50:25 +01:00
Jamie McCrae
0711f42e3a sysbuild: Add support for snippets
Adds support for sysbuild loading snippets, these can be included
by using e.g.: cmake ... -DSB_SNIPPET=blah for sysbuild
directly or can be used with an application and sysbuild using
-DSNIPPET. Snippets for sysbuild can use SB_EXTRA_CONF_FILE in the
snippet file to specify an extra Kconfig fragment for sysbuild

Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
2024-12-27 07:18:13 +01:00
Chris Friedt
64442ae5cd scripts: west-commands: add an entry for the patch command
Add an entry to west-commands.yml for the west patch command.

Signed-off-by: Chris Friedt <cfriedt@tenstorrent.com>
2024-12-24 11:15:38 +01:00
Chris Friedt
ad611395ee scripts: west_commands: add the west patch command
In smaller projects and organizations, forking Zephyr is usually
a tenable solution for development continuity, in the case that
bug-fixes or enhancements need to be applied to Zephyr to
unblock development.

In larger organizations, perhaps in the absence of healthy patch
management, technical debt management, and open-source policies,
forking and in-tree changes can quickly get out of hand.

In other organizations, it may simply be preferable to have a
zero-forking / upstream-first policy.

Regardless of the reason, this change adds a `west patch`
command that enables users to manage patches locally in their
modules, under version control, with complete transparence.

The format of the YAML file (detailed in a previous comit)
includes fields for filename, checksum, author, email, dates,
along with pr and issue links. There are fields indicating
whether the patch is upstreamble or whether it has been merged
upstream already. There is a custom field that is not validated
and can be used for any purpose.

Workflows can be created to notify maintainers when a merged
patch may be discarded after a version or a commit bump.

In Zephyr modules, the file resides conventionally under
`zephyr/patches.yml`, and patch files reside under
`zephyr/patches/`.

Sample usage applying patches (the `-v` argument for additional
detail):
```shell
west -v patch apply
reading patch file zephyr/run-tests-with-rtt-console.patch
checking patch integrity... OK
patching zephyr... OK
reading patch file zephyr/twister-rtt-support.patch
checking patch integrity... OK
patching zephyr... OK
reading patch file zephyr/multiple_icntl.patch
checking patch integrity... OK
patching zephyr... OK
reading patch file zephyr/move-bss-to-end.patch
checking patch integrity... OK
patching zephyr... OK
4 patches applied successfully \o/
```

Cleaning previously applied patches
```shell
west patch clean
```

After manually corrupting a patch file (the `-r` option will
automatically roll-back all changes if one patch fails)
```shell
west -v patch apply -r
reading patch file zephyr/run-tests-with-rtt-console.patch
checking patch integrity... OK
patching zephyr... OK
reading patch file zephyr/twister-rtt-support.patch
checking patch integrity... OK
patching zephyr... OK
reading patch file zephyr/multiple_icntl.patch
checking patch integrity... OK
patching zephyr... OK
reading patch file zephyr/move-bss-to-end.patch
checking patch integrity... FAIL
ERROR: sha256 mismatch for zephyr/move-bss-to-end.patch:
expect: 00e42e5d89f68f8b07e355821cfcf492faa2f96b506bbe87a9b35a823fd719cb
actual: b9900e0c9472a0aaae975370b478bb26945c068497fa63ff409b21d677e5b89f
Cleaning zephyr
FATAL ERROR: failed to apply patch zephyr/move-bss-to-end.patch
```

Signed-off-by: Chris Friedt <cfriedt@tenstorrent.com>
2024-12-24 11:15:38 +01:00
Chris Friedt
39588e7ca2 scripts: schemas: add schema for patches.yml files
Add a pykwalify schema for patches.yml files, which are used by
the west patch command.

Signed-off-by: Chris Friedt <cfriedt@tenstorrent.com>
2024-12-24 11:15:38 +01:00
Ryan Grachek
47b4e4c2fb UF2: Enable support for Raspberry Pi RP2350
Add UF2 Family ID for Raspberry Pi 2350 and build
UF2 image by default for Pico 2 board

Signed-off-by: Ryan Grachek <grachek@gmail.com>
Signed-off-by: Andrew Featherstone <andrew.featherstone@gmail.com>
2024-12-23 23:57:57 +01:00
Øyvind Rønningstad
617df3b5c1 scripts: Remove zcbor requirement from requirements-extras.txt
lwm2m_README: Add instructions to install zcbor when needed

Signed-off-by: Øyvind Rønningstad <oyvind.ronningstad@nordicsemi.no>
2024-12-23 17:09:43 +01:00
Peter Johanson
27615a7cf1 west: runners: bossac: Honor --erase flag
Instead of unconditionally erasing the whole target, add
support for using the common --erase flag.

Signed-off-by: Peter Johanson <peter@peterjohanson.com>
2024-12-22 18:20:30 +00:00
Hake Huang
15edce26e4 twister: add dt_node_prop_enabled support
using "dt_node_prop_enabled" to filter out supported platforms
1. check node existing.
2. check prop in node.
3. check the prop is True.

Signed-off-by: Hake Huang <hake.huang@oss.nxp.com>
2024-12-20 18:30:43 +01:00
Benjamin Cabé
eacd5527aa Revert "cmake: remove enforcement of DWARF v4"
This reverts commit 9d849d92bf that seems to
be causing dozens of basic kernel test failures across various platforms.

Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
2024-12-20 16:05:23 +01:00
Benedikt Schmidt
9d849d92bf cmake: remove enforcement of DWARF v4
Remove the enforcement of DWARF v4 as pyelftools is able
to parse DWARF v5.

Signed-off-by: Benedikt Schmidt <benedikt.schmidt@embedded-solutions.at>
2024-12-20 12:36:46 +01:00
Benedikt Schmidt
895db44388 scripts: logging: dictionary: fix parsing of DWARF v5
Fix parsing of DWARF v5 in database_gen.py.

Signed-off-by: Benedikt Schmidt <benedikt.schmidt@embedded-solutions.at>
2024-12-20 12:36:46 +01:00
Pieter De Gendt
b65a97ef0c modules: nanopb: Move pip dependencies to optional module
The west packages extension can be used to install module dependencies.

Upstream nanopb has added the pip package dependencies to the
zephyr/module.yml file. Remove in-tree pip package dependencies.

Signed-off-by: Pieter De Gendt <pieter.degendt@basalte.be>
2024-12-19 15:20:11 +01:00
Josuah Demangeon
5847e04be8 west: runner: add support for ecpprog
The ecpprog command is an utility written by Greg Davill for flashing
FPGAs such as ECP5 or CrossLink-NX series. Devkits typically have an
FTDI interface chip to access the external flash. FPGA image is
typically at flash offset 0x00000000 flash offset, and the Zephyr
image offset can be set via CONFIG_FLASH_LOAD_OFFSET.

Signed-off-by: Josuah Demangeon <me@josuah.net>
2024-12-19 08:38:55 +01:00
James Roy
4553a21fe5 devicetree: Add DT_FOREACH_ANCESTOR macro
Add 'DT_FOREACH_ANCESTOR' macro to get a list of
ancestor node of a given node_id.

Signed-off-by: James Roy <rruuaanng@outlook.com>
2024-12-18 10:17:09 +01:00
James Roy
ab959d5a3b scripts: dts: Use better string concatenation
Replace string concatenation operator with f-string.

Signed-off-by: James Roy <rruuaanng@outlook.com>
2024-12-18 08:33:13 +01:00
Mathieu Choplain
4dc8acc6d1 scripts: west: runners: stm32cubeprogrammer: add start address argument
Add new '--start-address' argument to the STM32CubeProgrammer west runner.
This argument can be used to specify an address from which the MCU should
start executing, instead of relying on the default value in CubeProgrammer.

Also update STM32CubeProgrammer runner tests to support new argument.

Signed-off-by: Mathieu Choplain <mathieu.choplain@st.com>
2024-12-17 23:15:13 +01:00
Dmitrii Golovanov
26e0aa268f twister: handlers: Pass harness reason to instance
Pass a test failure reason text from harness to the instance
to convey more details on the failure.

Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
2024-12-17 11:37:40 +00:00
Dmitrii Golovanov
ea3f105f82 twister: binaryhandler: Fix None status on test fail
Set missing TestCase statuses when a test under the BinaryHandler failed
(crashed), so remaining 'STARTED' and 'NONE' are now 'BLOCK' instead.

Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
2024-12-17 11:37:40 +00:00
Guennadi Liakhovetski
733241d737 west: rimage: generate TOML with all LLEXTs included
"west sign" uses the C preprocessor to generate a TOML configuration
file, needed for rimage. When creating LLEXTs rimage also uses a
configuration file where all supporting components are configured as
LLEXT. This adds such a file generation.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
2024-12-17 11:35:32 +00:00
Declan Snyder
cc620a312c kconfig: Add dt_compat_any_not_has_prop function
Add preprocessor function that returns 'y' if any enabled node with
compat does NOT have a certain property. This is different from using
dt_compat_any_has_prop to check that they ALL don't have the property.

Signed-off-by: Declan Snyder <declan.snyder@nxp.com>
2024-12-16 13:09:14 +01:00
Benjamin Cabé
9d4c027993 twister: reports: use Path instead of PosixPath
Directly instantiating a PosixPath is not allowed on Windows, use
Path instead.

Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
2024-12-16 13:08:22 +01:00
Dmitrii Golovanov
2c5195c814 twister: reports: Pretty print XML reports
Pretty print XML reports with tab identation on levels for better
readability and ease text diff there as well.

Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
2024-12-16 08:35:34 +01:00
Dmitrii Golovanov
c5c4165f68 twister: reports: Long testcase.classname XML properties
The testcase.classname properties in `twister_report.xml` and
`twister_suite_report.xml` now contain the full testsuite name
(without its path prefix) instead of just two its dot-separated
leftmost components. This way `twister_report.xml` testsuite container
has no duplicate testcase elements with the same `classname` and `name`
properties executing with --no-detailed-test-id for a project with same
testcase names in its 'sibling' suites, for example`tests/kernel/sleep`.

Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
2024-12-16 08:35:34 +01:00
Dmitrii Golovanov
b69a8d1deb twister: ztest: short test case names on --no-detailed-test-id
Extend `--no-detailed-test-id` command line option: in addition to its
current behavior to exclude from a test Suite name its configuration path
prefix, also don't prefix each Ztest Case name with its Scenario name.

For example: 'kernel.common.timing' Scenario name, the same Suite name,
and 'sleep.usleep' test Case (where 'sleep' is its Ztest suite name
and 'usleep' is Ztest test name.

This way both TestSuite and TestCase names follow the same principle
having no parent object name prefix.

There is no information loss in Twister reports with this naming:
TestSuite is a container object for its TestCases, whereas TestSuite
has its configuration path as a property.

Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
2024-12-16 08:35:34 +01:00
Anas Nashif
82f7857a94 twister: debug: change skipped -> filtered
Change debug message to say filtered instead of skipped.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-12-13 09:12:59 +01:00
Anas Nashif
63caa67f52 twister: plan: unify messaging, testsuite vs testcase
Many places use test case instead of testsuite or testsuite scenario.
Unify to avoid confusion.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-12-13 09:12:59 +01:00
Marcin Niestroj
fc73447acb scripts: pytest: log msec with leading zeros
Milliseconds are logged as is, without leading zeros. This means that
following log content is produced, which is not aligned by columns:

  12:54:27.972:INFO:root: Test message
  12:54:27.992:INFO:root: Test message
  12:54:28.12:INFO:root: Test message
  12:54:28.32:INFO:root: Test message
  12:54:28.52:INFO:root: Test message
  12:54:28.73:INFO:root: Test message
  12:54:28.93:INFO:root: Test message
  12:54:28.113:INFO:root: Test message
  12:54:28.133:INFO:root: Test message

Change milliseconds logging to include leading zeroes. This allows to print
nicely aligned log messages:

  12:54:27.972:INFO:root: Test message
  12:54:27.992:INFO:root: Test message
  12:54:28.012:INFO:root: Test message
  12:54:28.032:INFO:root: Test message
  12:54:28.052:INFO:root: Test message
  12:54:28.073:INFO:root: Test message
  12:54:28.093:INFO:root: Test message
  12:54:28.113:INFO:root: Test message
  12:54:28.133:INFO:root: Test message

Additionally it is much easier to configure programs simplifying
timestamped log browsing (like in logview-mode for Emacs), as it is more
standard and does not require custom regex matching.

Signed-off-by: Marcin Niestroj <m.niestroj@emb.dev>
2024-12-13 03:03:07 +01:00
Pieter De Gendt
4d1d8a77a7 scripts: west_commands: runners: openocd: Fix debugging with TUI
Prevent server output when running gdb with --tui to prevent garbled
console output.

Signed-off-by: Pieter De Gendt <pieter.degendt@basalte.be>
2024-12-12 19:59:35 +01:00
Grzegorz Chwierut
c4d884cc65 twister: Pass extra test args to pytest
Pass additional args to test binary. Twister
passes it only for native_sim.
Fixes #82463

Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
2024-12-12 10:20:55 -05:00
Luca Burelli
78ae035f89 llext: fix link order for weak syscall symbols
The weak syscall symbols generated by gen_syscalls.py are currently
compiled in the LLEXT subsystem library, which is then linked among all
other Zephyr libraries in an unspecified order. This can cause the weak
symbols to override the actual syscall implementations, leading to
undefined behaviour.

To fix this, the currently generated file is split in two elements:

- syscall_exports_llext.c contains the EXPORT_SYMBOL directives for all
  syscalls. This part can be compiled with the LLEXT library and linked
  among all other Zephyr libraries, and ensures all syscalls symbols
  are preserved by the linker.

- syscall_weakdefs_llext.c contains the weak definitions for all syscalls.
  This file is compiled in a separate library that is linked last, so
  that the weak symbols are only used if no other implementation is
  available.

Signed-off-by: Luca Burelli <l.burelli@arduino.cc>
2024-12-10 20:38:48 +01:00
Félix Turgeon
9cdb07cb52 scripts: coredump: Fix loading coredumps when thread info is enabled.
When loading a coredump with an an elf built using
'CONFIG_DEBUG_THREAD_INFO=y', gdbstubs assumes that the thread info
memoryblock populated by 'CONFIG_DEBUG_COREDUMP_MEMORY_DUMP_THREADS=y'
will be present in the coredump. This is not always true and causes an
error, and a failure to load the coredump. Add a default value for the
threads_metadata variable in CoredumpLogFile which can be used to detect
when the memory block is not present. This allows the coredump to load
successfully.

Signed-off-by: Félix Turgeon <felixturgeon@meta.com>
2024-12-09 22:06:23 +00:00
Anas Nashif
072a430cea twister: handler: open files using with statement
Use with statement to open stdout/stderr and resolve current warning
when running twister:

	ResourceWarning: unclosed file....

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-12-09 15:10:19 +01:00
Anas Nashif
30ad1901f3 twister: testplan: do not verify_platforms_existence again
We already sanitize those lists very early on, no need to call this
check again on the same lists.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-12-09 09:49:46 +01:00
Anas Nashif
c0dbcdab10 twister: error on invalid platforms
In case of unknown platforms in filters such as plaform_allow,
integration_plaforms or platform_exclude, report an error and exit.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-12-09 09:49:46 +01:00
Artur Dobrynin
f84dadde59 scripts: twister: adding default cmake/config arguments for bsim
Bsim tests need some addtional kconfig arguments enabled by default.

Signed-off-by: Artur Dobrynin <artur.dobrynin@nordicsemi.no>
2024-12-07 02:03:31 +01:00
Thomas Günther
20409ca2cc doc: add get_filtered_output method documentation
Add documentation for the get_filtered_output method in both the Python
implementation and the Integration with pytest chapter. The method is used
to filter command outputs by removing prompts and log messages, making it
easier to process shell command results.

Signed-off-by: Thomas Günther <thomas.guenther@limatica.com>

removed traling whitespace
2024-12-05 20:00:47 -05:00
Hake Huang
6996436ed6 tests: twister: add quit-on-failure option
in CI, we may need to quit if there is any failure
to save time, so add this --quit-on-failure so that
any failure will quit the test.

Signed-off-by: Hake Huang <hake.huang@oss.nxp.com>
2024-12-05 22:10:28 +01:00
Andy Ross
82c5f548d9 scripts/west/sign: Auto-detect rimage ELF files
This code had a growing "enumerate all the platforms" wart.  In point
of fact only intel_adsp is special here.  Other rimage platforms don't
have a bootloader and use zephyr.elf directly.

Don't hard code platform names (especially since they aren't the same
as board/soc names!).  Just autodetect which scheme we have at
runtime.

Signed-off-by: Andy Ross <andyross@google.com>
2024-12-05 22:08:55 +01:00
Armin Kessler
443afd421a scripts: twister: add esp32 runner
Adds support for the `esp32` runner when using the hardware-map feature.

Signed-off-by: Armin Kessler <ake@espros.com>
2024-12-05 15:17:18 +01:00
Ilya Tagunov
ab8c035fbe scripts: build: gen_app_partitions: enquote archive name wildcards
Some linkers (e.g. the ARC MWDT one) don't recognize wildcards in file
names if they are not enclosed in quotes. Looks like the quotes have
no negative effect on the GNU ld linker, so just do it unconditionally.

Signed-off-by: Ilya Tagunov <Ilya.Tagunov@synopsys.com>
2024-12-05 09:41:28 +01:00
Dat Nguyen Duy
1069f57f5c twister: fix issue twister with --device-testing on Windows
There is an incorrect logic that causes twister with
--device-testing to skip all tests on Windows. Correct
the logical condition.

Signed-off-by: Dat Nguyen Duy <dat.nguyenduy@nxp.com>
2024-12-04 18:19:23 -05:00