Display the total run duration to two decimal points instead of ~14,
similar to the other displayed timestamps.
Signed-off-by: Jordan Yates <jordan@embeint.com>
Display how long it takes to build the testplan in the logs. This can
be useful since the duration to bulid the plan can vary wildly depending
on the arguments given. Without this information, it is difficult to
tell (from CI logs) whether a long execution time is from actually
building and running the tests, or just generating the testplan.
For example, in a downstream repo:
```
./zephyr/scripts/twister --integration -iv -T . --dry-run
INFO - Building initial testsuite list...
INFO - Built testsuite list in 10.58 seconds
```
vs
```
./zephyr/scripts/twister --integration -iv -T . --dry-run \
--vendor nordic --vendor zephyr
INFO - Building initial testsuite list...
INFO - Built testsuite list in 201.01 seconds
```
Signed-off-by: Jordan Yates <jordan@embeint.com>
Now that the STM32CubeProgrammer runner supports the standard `--dev-id`
argument, replace usage of `--tool-opt` in the Twister device handler code.
Signed-off-by: Mathieu Choplain <mathieu.choplain-ext@st.com>
Now that the STM32CubeProgrammer runner supports the standard `--dev-id`
argument, replace usage of `--tool-opt` in the pytest harness hardware
adapter.
Signed-off-by: Mathieu Choplain <mathieu.choplain-ext@st.com>
Testing on linux, --flash-before argument did not work to wait for jlink
upload to complete. Instead Serial port opening throws file not found. The
fix ensures that the code waits for the serial port to appear before
attempting to connect. Tested on a proprietary platform with virtual UART
through usb.
Signed-off-by: Eemil Visakorpi <eemil@doublepoint.com>
Change direction for raw data file from $ZEPHYR_BASE
to <path_to>twister-out/../<test>/
Signed-off-by: Arkadiusz Cholewinski <arkadiuszx.cholewinski@intel.com>
The readlines_until method is part of the testing framework
(pytest harness) and should not print a full stack trace
when the expected line isn't found. It should also raise
an AssertionError instead of a TwisterHarnessTimeoutException,
allowing users to know that a test failed because the expected
condition did not happen.
Signed-off-by: Lukasz Fundakowski <lukasz.fundakowski@nordicsemi.no>
Extend twister's device serial handling to support multiple serial ports
for pytest-based tests. This enables testing scenarios that require
communication with multiple UART interfaces on the same device.
This enhancement enables comprehensive testing of multi-UART devices
while maintaining backward compatibility with single serial setups.
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
This commit makes two key improvements to the DeviceHandler:
1. Pass TWISTER_PLATFORM environment variable to serial PTY scripts
- Enables serial PTY scripts to access platform-specific configuration
- The platform name is now available via TWISTER_PLATFORM env variable
- Allows board-specific serial port and baudrate configuration in
external serial forwarding scripts
2. Reorder pre-script execution before serial PTY initialization
- Moved pre-script execution before serial PTY setup to avoid conflicts
- Prevents race conditions when pre-script performs board power cycling
or reset operations that might interfere with active serial
connections
These changes enable more flexible hardware testing scenarios where:
- Serial configuration can be automatically selected based on board type
- Board initialization (power cycling, FPGA programming) completes
before establishing serial connections
Signed-off-by: Appana Durga Kedareswara rao <appana.durga.kedareswara.rao@amd.com>
This requirements file can be confusing and will only get stale over
time. Drop it since these dependencies are actually captured in
scripts/requirements-run-test.txt
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
Rename dt_node_prop_enabled filtering expression to
dt_nodelabel_prop_enabled as it takes a node label as a 1st argument.
Enhance dt_node_has_prop inline doc and param name.
Signed-off-by: Abderrahmane JARMOUNI <git@jarmouni.me>
The display harness initial documentation was a separate, stale (had
references to a personal Github repo), README.rst buried in the
twister/scripts folder.
Move the docs to be a "first class" part of the Twister documentation
alongside other harnesses.
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
Collect filter commands into a single location and sort them in the
order: arch, vendor, platform, pattern, filter, tag. This enhances
the readability of help information and reduces the time spent
searching for filter command options.
Signed-off-by: WenBin Zhang <freey7955@gmail.com>
Collect device-testing commands into a single location to enhance
readability of help information and reduce the time spent
locating device-testing options.
Signed-off-by: WenBin Zhang <freey7955@gmail.com>
Add support for the Tigard multi-protocol hardware hacking tool from
SecuringHardware.com.
More information available at https://github.com/tigard-tools/tigard
Signed-off-by: Henrik Brix Andersen <henrik@brixandersen.dk>
multiprocessing no longer defaults to fork, so force it to maintain pre
3.14 behavior. We will need to revisit the implementation and at some
point move to forkserver
Fixes#95058
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Set the environment:os tag to the return value of platform.system(),
which is more useful than os.name() which always returns "posix".
Signed-off-by: Keith Short <keithshort@google.com>
Fix for collecting and presenting measured data.
During verify measured values, the test skipped all results
after the first incorrect one. Furthermore, it duplicated
already collected data on the output list. So in this case the
result list include wrong data.
Signed-off-by: Artur Wilczak <arturx.wilczak@intel.com>
Removed redundant try..except around the code
and fixed a test for that code which was false positive.
Signed-off-by: Łukasz Fundakowski <lukasz.fundakowski@nordicsemi.no>
Add 'dt_node_has_prop' expression to use the existence of a node property
as a filter in Twister test configuration yaml files.
First argument must be a node alias (in aliases node) or a node path,
not a nodelabel.
Signed-off-by: Abderrahmane JARMOUNI <git@jarmouni.me>
gcovr 8.0 introduces suspicious_hits detection, warn on such detection, but
do not stop coverage calculation at all.
Signed-off-by: Bartlomiej Buczek <bartlomiej.buczek@nordicsemi.no>
If twister exits by sys.exit function or by raising
the SystemExit exception, catch that exceptions and
return proper exit code.
Signed-off-by: Łukasz Fundakowski <lukasz.fundakowski@nordicsemi.no>
Use `collections.defaultdict` to simplify code, and use the `bytes`
type internally rather than hex strings.
Signed-off-by: Jordan Yates <jordan@embeint.com>
This feature enables sharing of built applications between test
scenarios, allowing tests to access build artifacts from other
applications.
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
for test machine without display, need support open-cv-headless mode.
1. support headless mode
2. add parser for environment path
Signed-off-by: Hake Huang <hake.huang@oss.nxp.com>
requests < 2.32.4 is subject to GHSA-9hjg-9r4m-mvj7
pyyaml < 5.4 is subject to GHSA-8q59-q68h-6hv4
pyyaml < 5.1 is subject to GHSA-rprw-h62v-c2w7
protobuf < 5.29.5 is subject to GHSA-8qvm-5x2c-j2w7
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
Use csv instead of a plain split to split extra arguments for extra west
flags, this allows correct passing of arguments that are internally
escaped in quotes, such as arugments containing commas.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
Refactored twister_main.py module to simplify the code and system tests.
Removed the need to patch `sys.argv` in blackbox tests.
Signed-off-by: Lukasz Fundakowski <lukasz.fundakowski@nordicsemi.no>
Modified the assertion statements to raise informative exceptions
that indicate which lines did not pass the assertions.
Added unit tests using pytest to validate the functionality of
the modified functions.
Signed-off-by: Lukasz Fundakowski <lukasz.fundakowski@nordicsemi.no>
When testing ESP32-based platforms in CI, early boot logs can be missed
if the serial connection isn't reset properly. This patch applies the
standard RTS/DTR toggle sequence to reset the ESP32 after flashing,
ensuring consistent log capture from the very beginning.
The logic is applied conditionally when the runner is "esp32".
Signed-off-by: Sylvio Alves <sylvio.alves@espressif.com>
Currently twister requires west to flash a board, either directly using
west-flash or indirectly through the cmake flash target.
Add an option to specify a custom flash script, this is passed a
build-dir and board-id flags so it can be used to implement custom
flashing scripts in a system that does not use west.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
STM32CubeProgrammer does not support flashing in BOOT-SERIAL mode
using the device's serial number.
Signed-off-by: Fabrice DJIATSA <fabrice.djiatsa-ext@st.com>
Multiple handlers were setting status/reason in slightly different ways,
but the code was mostly trying to do the same. Put this code in the
handler class and implement in one place.
This fixes an issue where in some cases qemu failures were not handled
correctly and status of cases was not updated, leading to multiple
failures.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Honor the --flash-before option when using a serial-pty script with
--device-serial-pty <script> and start the script only after
flashing the device.
Signed-off-by: Tim Pambor <tim.pambor@codewrights.de>
gcovr runs gcov to generate temporary coverage files into the gcov
working directory. The working directory defaults to gcovr's -r
argument.
When running twister with the --coverage-per-instance argument, multiple
gcovr instances may run in parallel. Since they all share the same gcov
working directory, they may interfere with each other, resulting in
errors like this:
AssertionError: Sanity check failed, output file
spinlock.h##4167b923a06cc9590c8eef372f016a6d.gcov doesn't exist
but no error from GCOV detected.
To fix this, specify a separate gcov working directory for every test
through gcovr's --gcov-object-directory option.
Signed-off-by: Marco Widmer <marco.widmer@bytesatwork.ch>
