Stop Twister if there are too many backup copies of the output
directory already.
Before this fix, Twister silently kept artifacts from the last run,
unless `--clobber-output` was explicitly given.
Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
Extend the coverage tool to handle applications that generate multiple
gcov dumps in a single execution. This can happen when the application
calls `sys_reboot`.
Handling multiple dumps enables coverage testing of exception handlers.
Signed-off-by: Jordan Yates <jordan@embeint.com>
Since `writer.py` is the one writting the SPDX file, it should normalize
the name field and not `walker.py` which generates the SBOM components.
Signed-off-by: Thomas Gagneret <thomas.gagneret@hexploy.com>
Improve the SPDX with the current values:
- URL: extracted from `git remote`. If more than one remote, URL is not
set.
- Version: extracted from `git rev-parse` (commit id).
- PURL and CPE for Zephyr: generated from URL and version.
For zephyr, the tag is extracted, if present, and replace the commit id for
the version field.
Since official modules does not have tags, tags are not yet extracted for
modules.
To track vulnerabilities from modules dependencies, a new SBOM,
`modules-deps.spdx` was created. It contains the `external-references`
provided by the modules. It allows to easily track vulnerabilities from
these external dependencies.
Signed-off-by: Thomas Gagneret <thomas.gagneret@hexploy.com>
With many tests having 10s or scenarios and variants, anytime we make a
change to a test right now, we end up building all scenarios on all
platforms which ends up in multiple 10s of the thousands of instances
that need to run on 30 or 40 runners blocking CI for hours. We do not
really need that, a test needs to be smart about its coverage and not
rely on boiling the ocean to catch, mostly build errors that are
platform specific.
Change this to do the normal coverage we get on push events.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
ztest now provides functionality to compare strings.
These are simpler to use than the strcmp ways.
The semantic patch transforms many of the commonly used patterns.
It does not handle variable length macros.
Signed-off-by: Rubin Gerritsen <rubin.gerritsen@nordicsemi.no>
We want to able to use cocinelle on ztest functions as
well when transforming APIs.
Provide a simple macro so that test functions are recognized.
Signed-off-by: Rubin Gerritsen <rubin.gerritsen@nordicsemi.no>
Pass Twister pytest plugin's log output, as well as output from
a test image running with pytest, up to Twister log irregardless
of the current verbosity level set at Twister.
This allows to collect full test output and also ensures that
recording data embedded in the test log is passed to the Twister
pytest harness for export.
Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
Extend Twister Harness recording feature to allow selected data fields,
extracted from the log by a regular expression, to be parsed into JSON
objects and eventually reported in `twister.json` as `recording` list
property of the test suite.
With this extension, log records can convey layered data structures
passed from a test image as summary results, traces, statistics, etc.
This extension also allows flexible recording structure: a test image
can output different types of data records incapsulated into a fixed
set of fields, so `recording.csv` file columns are respected, whereas
some of the columns keep strings with json-encoded semi-structured data.
Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
The gdb-port defines the GDB port and the openocd runner passes
this value to openocd as well as gdb. However, the TI AM62x board
provides multiple ports for each subsystem. For example, systick
appears at 3333, A53 as 3334-3337, R5F as 3338 and M4F as 3339.
If we want to connect to the M4F, we need to add another port which
is different to the to the gdb-port value.
This patch adds an additional argument --gdb-client-port to define
the port which GDB should connect to. It defaults to 3333, identical
to gdb-port.
Signed-off-by: Daniel Schultz <d.schultz@phytec.de>
When brackets are used in macros, there may sometimes be a space in
front of them. The checkpatch script should allow this.
The change includes the example that triggered the need for this
change.
Signed-off-by: Rubin Gerritsen <rubin.gerritsen@nordicsemi.no>
More complex platforms require sysbuild to use always, even for
such "simple" samples like hello_world. Such platforms can have
`sysbuild: true` entry in their board_name.yaml used by twister.
Using such entry will tell twister, that sysbuild must always be used
on a given platform.
Twister is aligned to have information about need of sysbuild at
instance (platform + suite) level (was only at suite level before).
Instance.sysbuild is true whenever a test suite or a platform requires
sysbuild.
Twister pytest unit tests are aligned with changes.
Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
Fixes an issue whereby the domains file in sysbuild projects
would be loaded and used with outdated information if sysbuild
configuration was changed then west flash was ran directly after
it
Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
Currently we launch qemu (well, "ninja run" usually) using Popen and
request stdout and stderr to be redirected into a pipe. However we never
read that pipe so the information is not captured.
Instead log directly into files that can be inspected after a failed
to to find out why qemu run failed.
Note that this is really only useful in cases where qemu either fails to
launch or crashes.
Regular test data is still handled via the qemu fifo.
Signed-off-by: Juha Kuikka <juha.kuikka@gmail.com>
we use reason for a failure to indicate state and then set the status
later and reason for the failure, in case of the failure is taken from
the handler status. Clean this up by setting status and reason coming
from the handler very early, so we do not have to go through replacing
meaning later.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Instead of blindly using the module names provided by the user via
command-line arguments, check if those actually exist in the current
manifest, and error out if any of them does not.
Fixes#73901.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Add the following new macros:
- DT_FOREACH_NODELABEL
- DT_FOREACH_NODELABEL_VARGS
- DT_INST_FOREACH_NODELABEL
- DT_INST_FOREACH_NODELABEL_VARGS
These are for-each helpers for iterating over the node labels of a
devicetree node. Since node labels are unique in the entire
devicetree, their token representations can be useful as unique IDs in
code as well.
As a first user of these, add:
- DT_NODELABEL_STRING_ARRAY
- DT_INST_NODELABEL_STRING_ARRAY
The motivating use case for these macros is to allow looking up a
struct device by devicetree node label in Zephyr shell utilities.
The work on the shells themselves is deferred to other patches.
To make working with the string array helpers easier, add:
- DT_NUM_NODELABELS
- DT_INST_NUM_NODELABELS
Signed-off-by: Martí Bolívar <mbolivar@amperecomputing.com>
When dumping logs from the `--inline-logs` option, remove any coverage
information that may be contained in those logs. Coverage dumps are
unrelated to any test failures and make it harder to find the failing
test information. In extreme cases the relevant information is lost due
to terminal scrollback limits.
If the raw dump information is required, it is still present in the
original `handler.log` file.
Signed-off-by: Jordan Yates <jordan@embeint.com>
Handle MPI and root manifest for radio core separately from the main
build system logic.
Signed-off-by: Tomasz Chyrowicz <tomasz.chyrowicz@nordicsemi.no>
Add a kconfig preprocessor function to check if
any node of a certain compatible has a specific property in DT.
Signed-off-by: Declan Snyder <declan.snyder@nxp.com>
New Twister option `--footprint-report` is introduced to collect and
write detailed memory footprint results for symbols as an additional
JSON file. By default, the new option is disabled.
The new option implies and extends `--create-rom-ram-report`, so there
are three choices: 'ROM', 'RAM', and 'all' to select what memory area
symbols to report in `twister_footprint.json`.
In case of the custom report name, or per-platform report, it is always
composed with the rightmost '_footprint.json' suffix.
The memory footprint report has similar structure as `twister.json`
and compelements it having reduced set of test suite properties:
- instead of `testcases` it contains `footprint` object with
`rom.json` and `ram.json` artifacts embedded there;
- other properites are limited to represent only the essential test
suite context, thus to allow further data processing consistently
and independently from the `twister.json`.
- 'filtered' test instances are not included into the footprint report.
Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
When Twister composes resulting twister.json reports, add optional
filtering by a test instance resulting status and/or its individual
properties to be allowed/denied on output to the JSON file.
This internal feature is introduced to facilitate JSON reports with
a custom data schema.
Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
Previously, one-line changes were tagged as "Trivial".
The description of the "Trivial" label states:
"Changes that can be reviewed by anyone, i.e. doc changes, minor build
system tweaks, etc.".
Just because a change only affects a single line of code, it does not
mean that it is a trivial change. It may have difficult to understand
implications which require approval of the responsible maintainer.
For this reason, change the label to "size: XS" and let humans judge
if a PR is trivial or not.
Signed-off-by: Martin Jäger <martin@libre.solar>
Handling of board changes was broken and did not support v2 boards, fix
this to optimize CI execution on localized changes of board files.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Previously, dtlib would fail to parse the following:
/delete-node/ &{/};
This is accepted by dtc, so dtlib should be aligned.
The expected behavior is that the contents of the "deleted" root node
are emptied, but the node itself remains in the tree. This means that
it's possible to put that statement at the end of a DTS file and still
get a valid output. A small test case for this scenario is included.
Signed-off-by: Grzegorz Swiderski <grzegorz.swiderski@nordicsemi.no>
Make sure Twster DeviceHandler serial-pty process is terminated
with all its remaining children to avoid Twister hanging on it
infinitely.
The reolved issue occurs sometimes, for example when serial-pty
script is used for serial port tunneling over network.
Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
CI reports error:
kconfigfunctions.py:143:11: E0601: Using variable 'edtlib' before
assignment (used-before-assignment)
Initialize edtlib to none when there is no edt.
Signed-off-by: Ioannis Damigos <ioannis.damigos.uj@renesas.com>
Introduce dt_node_ph_prop_path function.
It takes a node 'path' and a phandle property name
and returns the path to the pointed-to node.
Signed-off-by: Ioannis Damigos <ioannis.damigos.uj@renesas.com>
Filtered testcases are removed by default from Twister tests.
Older functionality is preserved via a new Twister flag:
--report-filtered.
Old tests were adjusted and a new test for that flag added.
Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
Allow twister fixtures to contain extra information, which can be used for
test suite configuration. The extra information can be appended to existing
fixtures separated by a colon (i.e. <fixture>:<configuration>).
This is especially useful for the pytest harness, where a fixture of a
given type may need to refer to an instance of a particular piece of host
hardware needed by the pytest suite (e.g. a network interface, a UART, or a
CAN interface connected to the device under test).
Signed-off-by: Henrik Brix Andersen <hebad@vestas.com>
Pass the list of supported twister fixtures for a given platform to pytest
via DeviceConfig. This allows for the pytest suites to use knowledge of the
fixtures for test suite configuration.
Signed-off-by: Henrik Brix Andersen <hebad@vestas.com>
Add release notes for the deprecation of a number of POSIX
Kconfig options. These have been deprecated so that we can
use more normative Kconfig variable identifiers, based on
the Options and Option Groups of IEEE 1003.1-2017.
To simplify migration, use
python $ZEPHYR_BASE/scripts/utils/migrate_posix_kconfigs.py \
-r root_path
Additionally, document the removal of PTHREAD_BARRIER_DEFINE(),
EFD_IN_USE, EFD_FLAGS_SET, which were previously deprecated
>= 2 release cycles before.
Signed-off-by: Chris Friedt <cfriedt@tenstorrent.com>
When something goes wrong with execution, due to twister issues and
bugs, do not continue with execution and abort the process and report
and return an error code.
Fixes#72807
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
This commit introduces support for an alternate linking method in the
LLEXT subsystem, called "SLID" (short for Symbol Link Identifier),
enabled by the CONFIG_LLEXT_EXPORT_BUILTINS_BY_SLID Kconfig option.
SLID-based linking uses a unique identifier (integer) to identify
exported symbols, instead of using the symbol name as done currently.
This approach provides several benefits:
* linking is faster because the comparison operation to determine
whether we found the correct symbol in the export table is now an
integer compare, instead of a string compare
* binary size is reduced as symbol names can be dropped from the binary
* confidentiality is improved as a side-effect, as symbol names are no
longer present in the binary
Signed-off-by: Mathieu Choplain <mathieu.choplain@st.com>
Make the generated json file match the contents generated with:
* nrfutil device erase --uicr --core Application --x-operation-id 1
--x-family nrf54h --x-append-batch batch.json
* nrfutil device program --firmware uicr_merged.hex
--options chip_erase_mode=ERASE_NONE,verify=VERIFY_READ
--core Application --x-operation-id 2 --x-family nrf54h
--x-append-batch batch.json
* nrfutil device reset --reset-kind RESET_PIN --x-operation-id 3
--x-append-batch batch.json
The erase options are supposed to be placed under "option" subkey.
Without the "option" subkey, nrfutil defaulted to ERASE_ALL.
Remove the firmware file format field because it is auto detected by
nrfutil when executing the batch script.
Signed-off-by: Tomasz Moń <tomasz.mon@nordicsemi.no>
Read default domain from domains.yaml file and update
paths to proper build directory. It fixes native and qemu
pytest scenarios, when application is build with sysbuild.
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
Modify the BAP and PBP samples to start with the profile name
(BAP or PBP) and the role of the sample.
Signed-off-by: Emil Gydesen <emil.gydesen@nordicsemi.no>
Employs the same linkonce magic of sw_isr_table to fix the
multiple definition of the symtab variables issue that I
get in my application build that doesn't use `west`.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
`start_addr` is the address of the first symbol, rename it to
`first_addr` instead as it seems more intuitive and relatable
to the comments.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Append new entry to the symtab list only if it has unique
address.
Added a bit more comments and move the debug print to after
the list is sorted.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
