Commit graph

4690 commits

Author SHA1 Message Date
Piotr Dymacz
fc72f696fe west_commands: sign: imgtool: fix slot size readout
This fixes wrong slot selection when reading partition size. According
to the comment in code, 'slot1_partition' size should be preferred:

  [...] slot1_partition size is used, when available, because in
  swap-move mode it can be one sector smaller. When not available,
  fallback to slot0_partition (single slot dfu). [...]

This fixes a typo in the if statement which currently always results in
use of 'slot0_partition' size.

Fixes: 86c4b4caa9 ("west/sign: Move from using partition label property")
Signed-off-by: Piotr Dymacz <pepe2k@gmail.com>
2024-04-25 07:25:04 -04:00
Lukasz Mrugala
ef792d1b9a scripts: twister: Fix duplicate scans
If the src_dir_path is a child of testsuite_path,
some c files might be scanned twice.

To prevent that, we check for parentage.

Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
2024-04-24 09:34:32 -04:00
Dmitrii Golovanov
50db868b58 twister: footprint: Improve error handling at gather_metrics
Add error handling for 'gather_metrics' stage and separate
build issues from 'scripts/footprint/size_report' issues.

Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
2024-04-24 09:52:57 +02:00
Lukasz Mrugala
f99dc66d8e scripts: twister: Fix duplicate statusless testcases
In some cases, Twister would report twice the amount
of testcases run than expected, with half of them
lacking any status.

This change fixes erroneous ELF testcase name extraction,
which deleted every instance of 'test_' in the name,
rather than just the first.

Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
2024-04-23 21:27:07 +00:00
Dmitrii Golovanov
978c0e4273 twister: testplan: Handle unknown test level
Fail a test run gracefully when an unknown test level
is requested to execute with the current test plan.

Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
2024-04-23 15:31:50 +02:00
Nick Goote
0b8714bcde scripts: Improve gen_kobject_list.py variable address determination
The gen_kobject_list.py script looks at DWARF debug information in the
elf file to determine the address of variables. Make sure that when
looking at DW_FORM_exprloc, it looks at both DW_OP_addr and
DW_OP_plus_uconst.

Signed-off-by: Nick Goote <ngoote@gmail.com>
2024-04-22 15:21:47 -04:00
Christophe Dufaza
33bb3b60d9 edtlib: test filters set by including bindings
Make sure filters set by property-allowlist and property-blocklist
in an including binding are recursively applied to included bindings.

Signed-off-by: Christophe Dufaza <chris@openmarl.org>
2024-04-22 06:50:55 -07:00
Christophe Dufaza
b3b5ad8156 edtlib: fix "last modified" semantic for included property specs
Although the PropertySpec.path attribute is documented as
"the file where the property was last modified",
all property specs in Binding.prop2specs will claim
they were last modified by the top-level binding itself.

Consider:
- I1 is a base binding that specifies properties x and y
- I2 is an "intermediate" binding that includes I1,
  modifying the specification for property x
- B is a top-level bindings that includes I2,
  and specifies an additional property p

When enumerating the properties of B,
we expect the values of PropertySpec.path to tell us:
- y was last modified by I1
- x was last modified by I2
- p was last modified by B

However, the Binding constructor:
- first merges all included bindings into the top-level one
- eventually initializes specifications for all the defined properties

As a consequence, all defined properties claim they were last modified
by the top-level binding file.

We should instead:
- first, take into account their own specifications for the
  included properties
- eventually update these specifications with the properties
  the top-level binding adds or modifies

Signed-off-by: Christophe Dufaza <chris@openmarl.org>
2024-04-22 06:50:55 -07:00
Christophe Dufaza
70eaa61cb0 edtlib: test "last modified" semantic for included property specs
Make sure the property specs answered by the Binding.prop2specs API
do not all claim (PropertySpec.path) they were last modified
by the top-level binding.

Signed-off-by: Christophe Dufaza <chris@openmarl.org>
2024-04-22 06:50:55 -07:00
Marcin Niestroj
8654daeebd twister: pytest: use runner_params from hardware map YAML file
Use 'runner_params' specified in hardware map YAML file. This allows to
configure custom params (like openocd's adapter configuration for
FT232-based probe) when used with pytest harness.

Signed-off-by: Marcin Niestroj <m.niestroj@emb.dev>
2024-04-20 13:45:44 -04:00
Bartlomiej Buczek
0b076a93ca twister: pytest: hardware_adapter: adapt to nrfutil.
add --dev-id to flashing command also when nrfutil runner is used.

Signed-off-by: Bartlomiej Buczek <bartlomiej.buczek@nordicsemi.no>
2024-04-20 13:44:24 -04:00
Martin Åberg
c4766f9572 scripts/checkpatch: Improve blank line check after declarations
This adds volatile type qualifier to the struct declaration matching
when looking for blank line after declarations.

Before this commit, this structure was not accepted by chheckpatch:
  struct cfg {
         struct gpio_driver_config common;
         volatile struct grgpio_regs *regs;
         int interrupt;
  };

checkpatch.pl generated the following warning:
  -:158: WARNING:LINE_SPACING: Missing a blank line after declarations
  #158: FILE: drivers/gpio/gpio_grgpio2.c:27:
  +	struct gpio_driver_config common;
  +	volatile struct grgpio_regs *regs;

With this commit, the warning is no longer generated.

Signed-off-by: Martin Åberg <martin.aberg@gaisler.com>
2024-04-19 17:03:49 +00:00
Benjamin Cabé
9ebf341977 west: spdx: introduce support for SPDX 2.3
Minor update to existing zspdx implementation to add support for
PrimaryPackagePurpose introduced in SPDX 2.3.

Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
2024-04-16 10:07:49 -07:00
Lukasz Mrugala
a0d1b778a4 scripts: twister: Fix overbroad Mock in environment unit tests
os.path.abspath was mocked too broadly, leading to errors for some users,
while being undetected in the CI.
This change narrows down the mock effect, fixing the problem.

Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
2024-04-15 16:25:51 +02:00
Théo Battrel
4a30aab2ae ci: compliance: Update Kconfig check
Take into account the Kconfig symbols in tests and samples that are
defined using the logging template.

This avoid using the `UNDEF_KCONFIG_WHITELIST` for those symbols.

Update the list to remove the symbols that were added for that reason.

Signed-off-by: Théo Battrel <theo.battrel@nordicsemi.no>
2024-04-13 07:17:45 -04:00
Aleksandar Cecaric
0144ed6b63 arch: riscv: update coredump for 64BIT RISCV
Add RISCV 64bit registers and parse them in coredump script.

Signed-off-by: Aleksandar Cecaric <aleksandar.cecaric@nextsilicon.com>
2024-04-13 07:03:23 -04:00
Alberto Escolar Piedras
1fcc245eda native_simulator: Get latest from upstream
Align with native_simulator's upstream main
6f6b359c4fe26ddeed8a65ad55f5bc402d3b7f91

Which includes:
* 6f6b359 Avoid issue with llvm address sanitizer

Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
2024-04-12 15:03:35 +02:00
Michal Smola
7bf2d8ff46 twister: fix NXP hw detection in hardware map
Some NXP boards are not detected by twister when creating hardware map,
because manufacture name is NXP instead of NXP Semiconductors expected
by twister.
Fix it by adding NXP to manufacturers list.

Signed-off-by: Michal Smola <michal.smola@nxp.com>
2024-04-12 08:44:41 -04:00
Miika Karanki
67de8d58ec scripts: logging: support native_posix dictionary logging
- Do not look CONFIG_* symbols from dynamic symbols which might
  be there before the symbol table where CONFIG_* symbols are found.

- Define posix architecture in ARCHS list.

- Check if architecture is not found instead of letting the code
  run into trying to access dictionary with None key.

Signed-off-by: Miika Karanki <miika.karanki@vaisala.com>
2024-04-12 11:54:18 +02:00
Jonathan Rico
55533bc607 scripts: Add APP_LOG_LEVEL_DBG
Someone should really fix this properly..

Signed-off-by: Jonathan Rico <jonathan.rico@nordicsemi.no>
2024-04-10 07:43:41 -04:00
Anas Nashif
66af7a6881 gitlint: do not allow treewide as an area in commit messages
Treewide changes touching a single area or topic should have the
area/subsystem at the start of the commit message. Treewide is very
ambigous.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2024-04-10 09:22:24 +02:00
Peter van der Perk
46cb2ad7c0 west: add teensy loader test
Adds test import for teensy_loader runner

Signed-off-by: Peter van der Perk <peter.vanderperk@nxp.com>
2024-04-09 11:07:21 +02:00
Peter van der Perk
197a0f39fb teensy: add west flash teensy_loader_cli integration
Adds loader, and flash target for teensy boards

Signed-off-by: Peter van der Perk <peter.vanderperk@nxp.com>
2024-04-09 11:07:21 +02:00
Sylvio Alves
e587249704 soc: espressif: esp32: update to hal_espressif v5.1
Modify and reorganize SoC to meet updated hal.

Signed-off-by: Sylvio Alves <sylvio.alves@espressif.com>
Signed-off-by: Lucas Tamborrino <lucas.tamborrino@espressif.com>
2024-04-05 13:39:53 +02:00
Dmitrii Golovanov
300507d43b twister: blackbox: pylint no-name-in-module warnings
Ignore no-name-in-module pylint warnings.

Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
2024-04-04 17:01:55 +02:00
Dmitrii Golovanov
a8eaa401a6 twister: blackbox: Fix test_output time-sensitive asserts
Exclude time-sensitive log entries at `test_output` to avoid
false negatives comparing build logs.

Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
2024-04-04 17:01:55 +02:00
Dmitrii Golovanov
2ef6b98b98 twister: footprint: Improve command-line arguments
Improve Twister command line arguments for memory footprint:

 * group and order footprint arguments meaningfully,
 * clearer help descriptions,
 * resolve logical inconsistences for combinations of arguments,
   in particular:
   `--last-metrics` now forces `--enable-size-report`,
   `--show-footprint` now controls only detailed log output of
       footprint deltas in comparison modes.
 * align twister tests accordingly.

Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
2024-04-04 17:01:55 +02:00
Fabio Baltieri
fec3cee0bb scripts: set_maintainer: add log messages for skipped collabs
Refactor the skip collaborator logic to add log messages when a
collaborator is skipped because they self removed of not part of the
org.

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2024-04-04 11:14:42 +01:00
Lukasz Mrugala
2c722f97b5 scripts: pylib: twister: Remove unused Twister flags
Deletes 4 Twister flags that were unusable:
* --enable-asserts
* --disable-asserts
* --no-skipped-report
* --error-on-deprecations

Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
2024-04-03 12:24:17 -05:00
Patryk Kuniecki
d8b368a160 scripts: tests: twister_blackbox: Add test_ignore_platform_key
Add new tests to test_filter.py in blackbox twister testing

Signed-off-by: Patryk Kuniecki <patryk.kuniecki@intel.com>
2024-04-03 10:34:35 +02:00
Daniel DeGrasse
1254092831 scripts: west_commands: linkserver: fix probe serial number support
With commit f419ea7990 (runner: linkerserver : remove the probe ID
hardcode), support was added to use serial numbers with the linkserver
--probe argument. However, one invocation of the argument was missed,
and still used the "probe index" syntax. Resolve this issue.

Signed-off-by: Daniel DeGrasse <daniel.degrasse@nxp.com>
2024-04-02 21:01:58 -04:00
Marcin Niestroj
8f9c28282b twister: pytest: escape expected command regex
When sending commands containing something like "AT+CFUN" the expected
regex was not escaped and as a result such regex never matched what has
been echoed over serial.

Escape all commands with re.escape() so that echoed shell commands are
always matched, even when they contain special characters (like '+' in the
example above).

Signed-off-by: Marcin Niestroj <m.niestroj@emb.dev>
2024-04-02 13:28:05 -04:00
Lukasz Mrugala
2302e5f766 scripts: tests: Move bbox platform tests to a single file
Blackbox tests related to platform choice
now moved to test_platform.py:
* test_emulation_only
* test_exclude_platform

Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
2024-03-31 17:37:09 -04:00
Lukasz Mrugala
2f97795360 scripts: tests: Blackbox test expansion - platform
Adds tests related to the Twister's platform selection:
* -A, --board-root
* -K, --force-platform
* -p, --platform

Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
2024-03-31 17:37:09 -04:00
Kamil Paszkiet
717fccaa09 scripts: tests: Blackbox test expansion - device
Adds tests related to device flags:
--seed

Signed-off-by: Kamil Paszkiet <kamilx.paszkiet@intel.com>
2024-03-29 15:07:38 +00:00
Artur Wilczak
6d8132c445 scripts: tests: Blackbox test expension - disable
Add test related to disable-suite-name-check flag

This flag disabling extended test suite name verification at
the beginning of Ztest test. This option could be useful for tests
or platforms, which from some reasons cannot print early logs.

Add test related disable-warnings-as-errors

Do not treat warning conditions as errors.

Signed-off-by: Artur Wilczak <arturx.wilczak@intel.com>
2024-03-29 15:07:19 +00:00
Maciej Dubiel
d076962f73 twister: ut for config_parser.py
initial commit

Signed-off-by: Maciej Dubiel <maciejx.dubiel@intel.com>
2024-03-29 15:06:44 +00:00
Javan lacerda
dbfc1aaec6 scripts: dts: update pyyaml version
The currently used PyYaml version has some vulnerabilies as
described on the pull request description. It updates to
version 6.0, removing these supply chain vulnerabily.
The OSSF Scorecard was the tool used for discovering
 these vulnerabilties.

Signed-off-by: Javan lacerda <javanlacerda@google.com>
2024-03-29 09:03:34 -04:00
Tristan Honscheid
36ad49bd65 twister: Remove ANSI sequences before writing to handler.log
ANSI color is great for viewing output in a terminal, but currently it
also gets written out to the `handler.log` file. Text editors usually
don't render these, resulting in a hard-to-read file with a lot of
gibberish interspered. This commit strips ANSI sequences from lines
before writing them to the handler log file. This change does not
affect what is printed in Twister's console output, so one would still
see the colors there.

Signed-off-by: Tristan Honscheid <honscheid@google.com>
2024-03-28 06:50:55 -04:00
Daniel Leung
f05111dea0 kernel: rename Z_KERNEL_STACK_LEN to K_KERNEL_STACK_LEN
Since there is a K_THREAD_STACK_LEN, its kernel counterpart
should also be prefixed with K_ for consistency.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2024-03-27 19:27:10 -04:00
Hake Huang
f419ea7990 runner: linkerserver : remove the probe ID hardcode
linkserver is hardcoded to probe ID, now with #70343 merged
the probe ID restriction can be removed.

Signed-off-by: Hake Huang <hake.huang@oss.nxp.com>
2024-03-27 15:19:15 +00:00
Jordan Yates
5c8f357f61 west_commands: sign: suppress imgtool prints
If `args.quiet` is set, suppress the useless `print` statements output
by `imgtool` (mcuboot script) by capturing `stdout`.

Old output:
```
[44/44] Linking C executable zephyr/zephyr.elf
Memory region         Used Size  Region Size  %age Used
           FLASH:      415192 B       824 KB     49.21%
             RAM:      163124 B       256 KB     62.23%
        IDT_LIST:          0 GB         2 KB      0.00%
image.py: sign the payload
image.py: sign the payload
```
New output:
```
[44/44] Linking C executable zephyr/zephyr.elf
Memory region         Used Size  Region Size  %age Used
           FLASH:      415192 B       824 KB     49.21%
             RAM:      163124 B       256 KB     62.23%
        IDT_LIST:          0 GB         2 KB      0.00%
```

Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
2024-03-27 15:18:23 +00:00
Arkadiusz Cholewinski
370dadbdab CI: Fix coverage analysis.
Fix for parsing testplan.

Signed-off-by: Arkadiusz Cholewinski <arkadiuszx.cholewinski@intel.com>
2024-03-27 14:32:53 +00:00
Kamil Paszkiet
fff833eda4 scripts: tests: Blackbox test expansion - filter
Adds tests related to filter flags:
--arch
--vendor

Signed-off-by: Kamil Paszkiet <kamilx.paszkiet@intel.com>
2024-03-27 14:31:43 +00:00
Ederson de Souza
62b19ef65c syscalls: llext: Export z_impl symbols so they are available to kernel
commit 67bb6db3f8 ("syscall: Export all emitted syscalls, enabling
them for extensions") exports all emitted syscalls, however, it does
that only for the `z_mrsh` symbols, effectively only available for
userspace. If an extension running at kernel level tries to use a
syscall, it will fail to load.

This patch fixes that by exposing the `z_impl` symbols instead. However,
this is not as straightforward as the `z_mrsh` ones. As, in their
signatures, they can basically contain any type, it's not just a matter
of emitting `EXPORT_SYMBOL(z_impl_<syscall>)`, as the compiler will
complain about the undefined types. Here, there are a few approaches.

One of them is to have the `EXPORT_SYMBOL` being generated on the same
files where the syscall is implemented - injecting it there would allow
it to access all known symbols. But changing a lot of files is
undesirable, and it was one of the nice points of first patch.

Another one would be to reconstruct - or simply use the absolute path -
for the includes where the syscalls are defined. Reconstruct the paths
seems fragile and I'm not sure using absolute paths is portable.

Finally, the approach used in this patch is to declare, on a different
generated file, all `z_impl_` symbols as `void *` - after all, only the
address (and the name) to the function is relevant to EXPORT_SYMBOL. By
living in an compilation unit that doesn't include any header which
would expose any of the syscalls, there shouldn't be any conflicts. And
to account for the possibility that a syscall is not compiled - due
being configured out via Kconfig - all those symbols are also weak
aliases to a pointer to NULL. This file is then included in
`llext_export.c` (which should naturally not include any conflicting
header).

Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
2024-03-26 19:31:56 -04:00
Lukasz Mrugala
3fb11e260a scripts: tests: Blackbox test expansion - addon
Adds tests related to the addon flags:
*     --enable-ubsan
*     --enable-lsan
*     --enable-asan
*     --enable-valgrind
*     --allow-installed-plugin
*     --pytest-args
* -x, --extra-args
* extra test args for the binary

Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
2024-03-26 18:08:31 -05:00
Kamil Paszkiet
ff5780fcc7 scripts: tests: Blackbox test expansion - coverage
Adds tests related to coverage flags:
--gcov-tool
--coverage-platform

Signed-off-by: Kamil Paszkiet <kamilx.paszkiet@intel.com>
2024-03-26 18:08:02 -05:00
Alberto Escolar Piedras
f9e3084daf scripts/checkpatch: Limit which feature test macros we prevent
Unfortunately this check as it is today is causing trouble, while
not checking too well for what it intended. Let's reduce its scope
until a better solution has been found.

Background:
This check intends to ensure coding guidelines Rules A.4 and A.5
are followed, but how it is implemented it does not work well enough.
1. These rules only apply to the kernel and some other parts of the
embedded codebase respectively, but this check is performed on the
whole tree.
2. This check works under the assumption that any attempt to set
these macros in source files is a violation of these rules, while
this is not necessary the case, as there are legitimate uses for these.
(Specially for _POSIX_C_SOURCE and _XOPEN_SOURCE)

This check also fails to detect these macros being set in cmake files,
so if users are faced with this failure they can trivially bypass it.

Having a CI check which produces too many false positives, while
at the same time being very easy to bypass is not a desirable situation
as that can result in lack of trust for this type of checks,
and an overall tendency to override these CI faults,
and overlooking actual violations of these rules by reviewers.

This check was originally added in
b021dece98

Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
2024-03-26 16:21:02 -04:00
Grzegorz Swiderski
9dabce43d2 scripts: Ignore duplicate roots in list_boards/hardware
When iterating over `--arch-root`, `--board-root`, and `--soc-root`,
treat them as collections of absolute paths with no repeats, to ensure
that no input root has to be handled more than once.

Signed-off-by: Grzegorz Swiderski <grzegorz.swiderski@nordicsemi.no>
2024-03-26 16:20:08 -04:00
Torsten Rasmussen
88d8b23836 scripts: remove board name from the qualifiers in list_boards.py
Fixes: #69329

The board name was printed as part of printing board qualifiers because
those was being concatenated in the `board_v2_qualifiers()` method.

Keep the qualifiers separated from the board name and let the caller
concatenate the strings when required.

Completion scripts are also updated to handle the corrected behaviour.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
(cherry picked from commit 66b475a3aa)
2024-03-26 16:20:08 -04:00