requests < 2.32.4 is subject to GHSA-9hjg-9r4m-mvj7
pyyaml < 5.4 is subject to GHSA-8q59-q68h-6hv4
pyyaml < 5.1 is subject to GHSA-rprw-h62v-c2w7
protobuf < 5.29.5 is subject to GHSA-8qvm-5x2c-j2w7
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
Use csv instead of a plain split to split extra arguments for extra west
flags, this allows correct passing of arguments that are internally
escaped in quotes, such as arugments containing commas.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
Refactored twister_main.py module to simplify the code and system tests.
Removed the need to patch `sys.argv` in blackbox tests.
Signed-off-by: Lukasz Fundakowski <lukasz.fundakowski@nordicsemi.no>
Modified the assertion statements to raise informative exceptions
that indicate which lines did not pass the assertions.
Added unit tests using pytest to validate the functionality of
the modified functions.
Signed-off-by: Lukasz Fundakowski <lukasz.fundakowski@nordicsemi.no>
When testing ESP32-based platforms in CI, early boot logs can be missed
if the serial connection isn't reset properly. This patch applies the
standard RTS/DTR toggle sequence to reset the ESP32 after flashing,
ensuring consistent log capture from the very beginning.
The logic is applied conditionally when the runner is "esp32".
Signed-off-by: Sylvio Alves <sylvio.alves@espressif.com>
Currently twister requires west to flash a board, either directly using
west-flash or indirectly through the cmake flash target.
Add an option to specify a custom flash script, this is passed a
build-dir and board-id flags so it can be used to implement custom
flashing scripts in a system that does not use west.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
STM32CubeProgrammer does not support flashing in BOOT-SERIAL mode
using the device's serial number.
Signed-off-by: Fabrice DJIATSA <fabrice.djiatsa-ext@st.com>
Multiple handlers were setting status/reason in slightly different ways,
but the code was mostly trying to do the same. Put this code in the
handler class and implement in one place.
This fixes an issue where in some cases qemu failures were not handled
correctly and status of cases was not updated, leading to multiple
failures.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Honor the --flash-before option when using a serial-pty script with
--device-serial-pty <script> and start the script only after
flashing the device.
Signed-off-by: Tim Pambor <tim.pambor@codewrights.de>
gcovr runs gcov to generate temporary coverage files into the gcov
working directory. The working directory defaults to gcovr's -r
argument.
When running twister with the --coverage-per-instance argument, multiple
gcovr instances may run in parallel. Since they all share the same gcov
working directory, they may interfere with each other, resulting in
errors like this:
AssertionError: Sanity check failed, output file
spinlock.h##4167b923a06cc9590c8eef372f016a6d.gcov doesn't exist
but no error from GCOV detected.
To fix this, specify a separate gcov working directory for every test
through gcovr's --gcov-object-directory option.
Signed-off-by: Marco Widmer <marco.widmer@bytesatwork.ch>
Drop the logic to use the generator "flash" target, that is using west
anyway, always call flash directly anyway so that those targets can be
deprecated.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
now no-detailed-test-id is enabled by default, so switch the logic so we
do not require the extra parameter when reruning.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Replaces the check-then-create pattern for symlinks with
opportunistic creation. Instead of checking for existence
before creating the symlink, the code now attempts to
create it directly and gracefully handles the case
where it already exists.
Signed-off-by: Tim Pambor <tim.pambor@codewrights.de>
Use --test-pattern to filter scenarios using regular expressions, for
example:
./scripts/twister --test-pattern '^kernel.semaphore.*' -v
will run all those test using this identifier pattern, and nothing else.
Multiple patterns are possible.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Make this option default and reduce amount of details displayed by
twister.
This options has been enabled in CI for a while now with the intention
of making it default.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Change "Renaming output directory to" message to "Renaming previous
output directory to" to make it clearer that twister is renaming the
existing directory, not the current output directory.
This helps avoid confusion for new twister users who might think the
current output is going to the renamed directory.
Signed-off-by: Afonso Oliveira <afonsoo@synopsys.com>
Use --platform-pattern <regex> to match platforms, something like
twister -T samples/hello_world/ --platform-pattern ".*/stm32f4.*"
will build/run all those platform targets with stm32f4 in them.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
This fixes a typo in the formatting code that caused an exception
if the symlink creation failed.
Signed-off-by: Tim Pambor <tim.pambor@codewrights.de>
Split test configuration code into own class and do parsing in two steps
to get the right context of platforms and scenarios.
Fixeszephyrproject-rtos/zephyr#89774
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Fix amp to milliamp conversion where a 10^4 factor was applied instead
of 10^3.
Adjust existing test cases accordingly (including a small change to one
of the expected RMS values in pm.power_states that either changed
since the test was introduced or was set too aggressively back then).
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
If test configuration is not quarantined and `--quarantine-verify`
mode is enabled, these should be filtered and not attached
to reports.
Proposed change applies this approach.
Signed-off-by: Katarzyna Giądła <katarzyna.giadla@nordicsemi.no>
If test plan is used, memory usage metrics are overridden.
Result of this operation is missing memory footprint.
This change removes redundand metrics update and fixes
described inconvenience.
Signed-off-by: Katarzyna Giądła <katarzyna.giadla@nordicsemi.no>
With junitparser 4.0.2, the JunitXml.fromfile() now returns a JUnitXml
instance. In order to access the junit.TestSuite instance, we must use
TestSuite.fromelem() to convert it back.
Signed-off-by: Daniel DeGrasse <ddegrasse@tenstorrent.com>
When running pytest tests with coverage enabled, the test process starts
to write the coverage data when it receives the SIGTERM signal. On slow
machines, this can take longer than 0.5s. Increase the timeout and only
attempt to kill the process if it is still running after the timeout.
Signed-off-by: Marco Widmer <marco.widmer@bytesatwork.ch>
Enable `flash-before` mode for devices connected via serial-pty
the same way, as for physical serial.
This mode allows Twister to start reading from the DUT's serial
connection only after the flashing stage has been completed,
thus excluding an early part of the log stream not directly
related to the current test.
It can help in situations, when the device is connected via some
serial-over-network harness which is still transferring device
logs or keeping in its buffer a tail with different RunID etc.
from the previous test image run.
Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
Now that Python 3.12 is the minimum required version for Zephyr, ruff
should target that instead of 3.10.
Signed-off-by: Pieter De Gendt <pieter.degendt@basalte.be>
When we have skips in ctest, the overall result should not be marked as
a SKIP unless all tests have been skipped. If we have passing tests,
mark the test as PASS. Any failures/errors obviously will fail the
tests.
Also pass the skip reason from ctest to twister to give a hint about
what is going on in case of skips.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Statuses of quarantined test cases were not updated properly.
Quarantined tests shouldn't cause errors in integration mode.
Removed obsolate filter type.
Signed-off-by: Maciej Perkowski <maciej.perkowski@nordicsemi.no>
Some Build failures are actually cmake issues, so in that case, if
nothing is found as build failure, try to parse for cmake issues.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Skipped status fits quarantined items better than filtered.
Filtered tests are by default removed from reports, which
shouldn't be the case for quarantined tests.
Adjust tests unit and blackbox tests accordingly.
Signed-off-by: Maciej Perkowski <maciej.perkowski@nordicsemi.no>
Verbosity >1 should be enough to see filtered tests. Coupling
it with debug logging level makes the output messy.
Signed-off-by: Maciej Perkowski <maciej.perkowski@nordicsemi.no>
Just move a code with workaroud to convert platform names
to the full name with variants.
It fixes an issue, that not every platforms are printed
before running tests.
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
Some simulators - like simics - may end up adding extraneous suffixes to
logged lines. This may cause some regex that match too much fail. This
patch fixes two such cases:
- regex to find RunID changed to only match valid hexadecimal
characters;
- regex to match start of testsuite changed to only match valid word
characters (0-9A-Za-z_).
Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
use public queue methods to clear queue on failure
-s Felix Behnke felixbeiderarbeit@gmail.com
Signed-off-by: Felix Behnke <felixbeiderarbeit@gmail.com>
In twister coverage runs we create the gcov sym link to llvm-cov
executable. This code runs multiple times which causes errors with the
current implementation since creating the symlink will fail; then
copying will also fail because we're using the symlink (self copy the
file).
Add a helper function to `try_making_symlink` which will handle the
following cases:
1. If the symlink already exists and is correct, do nothing.
2. If the symlink exists but points to the wrong file, replace it.
3. If the symlink doesn't exist create it.
4. If creating a symlink fails, just copy the file.
Signed-off-by: Yuval Peress <peress@google.com>
This commit improves the alignment of columns in the verbose Twister
output by reclaiming extra space in the testsuite name column when the
platform name oveflows its allocated width.
Signed-off-by: Luca Burelli <l.burelli@arduino.cc>
