In order to reduce excessive false positives for Coding
Guidelines checks in CI, maintain a list of paths that are
marked "safe" for reserved names to be implemented or declared.
Signed-off-by: Chris Friedt <cfriedt@tenstorrent.com>
Fix confusing `TypeError: 'type' object is not subscriptable`
at _get_installed_packages() if twister runs on python version < 3.9
Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
Improve DUT selection at DeviceHandler: for each DUT it counts
how many test instances have been failed on it during the current
twister execution, so the next available DUT will be chosen
ordering the eligible DUTs by less failures occured so far.
The new selection mechanism should increase chances to retry failed
tests on different DUTs, for instance to resolve ploblems when some
DUTs have connectivity or HW issues slowing down test plan execution,
or even block the execution when only one test suite runs whereas
the same first DUT candidate in the list is not working and others
were not chosen.
Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
Change Twister PyTest plugin's test finalizing sequence to release
the DUT it is used as the very last operation, after the Test Instance
status becomes fully updated from the execution results.
This also fix a race condition possible when pytest plugin releases
the DUT and it becomes acquired by another test while the current test
is not yet finalized completely.
Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
Twister DeviceHandler - add test failure counter for how many
test instances have been failed on each DUT (Device Under Test)
when it executes the current test plan.
Output DUT falure counter summary at the end of Twister run.
Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
Fix Twister DeviceHandler exit on SerialException when
it connects to the serial device in 'flash before' mode.
Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
Several improvements at Twister DeviceHandler when it releases
current DUT (Device Under Test):
- release the exact DUT which is used for the test instance
instead of all configured DUTs which happened to have
the same serial device configured.
- Twister PyTtest harness plugin adjustment to the above.
- additional debug logging to track DUT waiting/retain/release.
Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
Updated the Pytest-Harness methods to capture and log error messages from
the subprocess, when pytest command fails.
Ensured that error messages are logged with `--inline-log` option
and placed in twister log reports.
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
While the option to inform pytest about which platform a test is for
(platform being the board, such as mps2/an385) is available, it wasn't
being used. This patch fixes that, by building a pytest command which
includes the platform.
Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
Add a new fixture, `unlaunched_dut`, which is basically the `dut`
one, but without launching the device - thus also not building the
application. It is useful for tests who need a finer control of
the building of the application, before the dut can be launched
to run it.
It will be used on a future patch, which will use it to enable LLEXT EDK
tests.
Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
Fix after #71401 merge at Twister Reporting:
* filter status value type mismatch;
* tesplan.json generation incorrect error logs.
Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
Gets experimental and deprecated symbols directly from Kconfig instead
of rely on hardened.csv.
This way we keep the tool consistent with Zephyr's code.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Samples, that were obtained by profiling perf tool, can be be translated
into flamegraph using stackcollapse.py script.
Originally-by: Yonatan Goldschmidt <yon.goldschmidt@gmail.com>
Signed-off-by: Mikhail Kushnerov <m.kushnerov@yadro.com>
Incorrect status assignments were previously just warnings in logs.
Now they are elevated to errors.
Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
Various different Statuses were joined into a single class,
TwisterStatus. This change anticipates further streamlining
of the Twister's approach to Status.
Code guarding Twister's Properties was shortened to a
value check only.
QEMUOutputStatus was left separate, as doubts were cast
whether it should remain a status. Leaving it separate makes
its removal easier.
Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
TestInstanceStatus of TIMEOUT and FLASH were never really used.
They were checked for, but never assigned.
Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
Now statuses are not just a str that can be easily mistyped
or assigned wrong. Now they are an Enum.
Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
This change is removing some deprecation warnings
which for some reason causing failing tests with
Python 3.12 on CI #76877.
Also, it fixes warnings from pytest like:
PytestCollectionWarning: cannot collect test class 'TestPlan'
because it has a __init__ constructor
(from: scripts/tests/twister/test_testplan.py)
Signed-off-by: Lukasz Fundakowski <lukasz.fundakowski@nordicsemi.no>
Add a new compliance check that reports any clang-format issues on
the git diff and prints a warning.
Signed-off-by: Pieter De Gendt <pieter.degendt@basalte.be>
Do not depend on platforms that need a HAL. This should speed things up
and should resolve issues where runner did not have enough space to deal
with all HALs.
t
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Driver for Nordic nRF70 Wi-Fi6 companion chipset, depends on
hal_nordic/nrf_wifi for OS agnostic part of the driver.
This supports (Q)SPI interface to communicate from host to chip.
Signed-off-by: Chaitanya Tata <Chaitanya.Tata@nordicsemi.no>
When presenting an example of how to run a failing test case with
`west build`, provide the source dir path relative to the current
working directory, not the zephyr root directory.
Signed-off-by: Jordan Yates <jordan@embeint.com>
It is safer to base the SUIT artifacts path on the path of the
configuration file than the HEX file that is being flashed.
The latter may be overriden by several scripts that merge/transform the
final firmware image.
Signed-off-by: Tomasz Chyrowicz <tomasz.chyrowicz@nordicsemi.no>
Read CONFIG_SHELL_PROMPT_UART from config file and use them
in shell fixture in pytest-harness package.
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
Moved helper methods from tests/boot/with_mcumgr to pytest-harness
package. It can be reused by other tests.
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
Add support in arm_cortex_m python script to read thread
registers off of a thread's stack when switching context.
When CONFIG_ARM_STORE_EXC_RETURN is enabled, check the exc_return value
in thread's arch struct to determine accurately where the stack pointer
is. Also, set r7 (the frame pointer register) in case the frame pointer
is not omitted.
Only 8 registers are read from the top of the stack
for other threads present in the dump. So update the
script to reflect that.
Signed-off-by: Mark Holden <mholden@meta.com>
Update zephyr gdb-server scripts to understand threads.
Parse the kernel_thread_info out of the elf file to be used
for finding offsets to data from _kernel structs or from
individual threads.
Update log_parser to understand latest format change, which
allows for the presence of a new section, threads metadata.
Update gdbstub to respond to various packets to describe
the threads present in a dump, and allow switching to
thread context of each thread.
Signed-off-by: Mark Holden <mholden@meta.com>
Twister scans C-files to find testcases that are implemented
using ZTest framework. Also runs scanning of Elf files
after building. Skip scanning files if it is not required.
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
probe-rs is a new programming and debugging tool written in Rust, supports
many probes and targets.
This commit introduces initial support for probe-rs to Zephyr.
Signed-off-by: Chen Xingyu <hi@xingrz.me>
Running 'west blobs fetch' does not verify the digest of downloaded files:
1. if the checksum of the previously downloaded file does match
that in the blob metadata (status BLOB_PRESENT), do nothing
2. if the checksum of the previously downloaded file does not match
that in the blob metadata (status BLOB_OUTDATED),
download the "up to date" file
3. if the blob has not yet been downloaded (status BLOB_NOT_PRESENT),
download it
None of the 2) and 3) code paths will verify that the checksum of the file
just downloaded actually matches the digest in the blob's metadata.
In the event that the metadata of a module is incorrect, then the user
will not notice anything, and may rely on an unexpected binary,
e.g. a static library for a different architecture.
According to the Binary Blobs documentation [1], the expected
behavior is to check the blob digest after downloading.
[1] Fetching blobs, Zephyr 3.6.0 (still applies to Zephyr 3.7.0rc3)
docs.zephyrproject.org/3.6.0/contribute/bin_blobs.html#fetching-blobs
Signed-off-by: Christophe Dufaza <chris@openmarl.org>
With the recent change to hwmv2, loading the
boards is extremely slow and can take several
seconds.
To solve that problem this commit add a cache
of the boards. The cache is updated based on:
- when the latest commit of the manifest
file directory is updated or;
- when the manifest file directory is not
a git directory, when the manifest file
content itself is updated.
At the same time:
- update how the board completion is
displayed by including the board vendor to it;
- add missing `--board` and `--board-dir`
options;
- remove `hwmv1` board completion code.
Signed-off-by: Théo Battrel <theo.util@protonmail.ch>
The current option used as an example, --recover, is actually a separate
option in the runner itself. Instead use --clockspeed as an example,
which is applicable to all nrfjprog commands.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Allow for users to provide a --qspiini parameter that is passed directly to
the nrfjprog executable but only in the --program operation. This is
required since e073210ec2 enabled the
-O/--tool-opt for all operations, but --qspiini is only allowed combined
with --program.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
When --erase was specified, esp32 runner was autodetecting serial port to
be used, regardless of --esp-device argument.
Append '--port SERIAL_DEVICE' parameter earlier, so that erase command
invocation uses explicitly specified serial device.
Signed-off-by: Marcin Niestroj <m.niestroj@emb.dev>
The default base timeout for pytest is statically set by
the TwisterHarnessConfig class to be 60 seconds. However,
sometimes it takes longer than 60s before the app starts
to run, especially on emulator/simulator where it takes
quite some time to start. So pass the test timeout as
the base timeout via pytest command line argument.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Current implementation will not work if comits were not provided.
ie. use case with list of changed files will fail as args.commits is None.
Signed-off-by: Piotr Kosycarz <piotr.kosycarz@nordicsemi.no>
Doing duplicates count of bugs, a PR fixing a bug is not a bug report.
Many PRs fixing an open bug are labeled with 'bug' and thuse are being
counted twice.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
