Temporarily disable qemu with robot. Some qemu jobs are not being
terminated correctly slowing CI execution.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Now we are able to run tests using the Robot framework with Qemu.
Assisted-by: GitHub Copilot:claude-opus-4.6
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
os.path.commonpath will raise ValueError for paths with different
drives on Windows
Added new is_subpath method which handles ValueError and returns
False since it can't be a sub path. Replaces all 3 occurrences of
os.path.commonpath in walker.py.
Fixes#109641
Signed-off-by: Sebastian Modersohn <im24@msohn.de>
QEMUHandler.handle() reads the qemu pid file at the end of a run and
then unlinks it:
if os.path.exists(self.pid_fn):
with open(self.pid_fn) as pid_file:
qemu_pid = int(pid_file.read())
os.unlink(self.pid_fn)
The existence check, the open(), and the unlink() are not atomic.
When qemu exits on its own (slow benchmark hitting a harness
timeout, hostshutdown issued via the monitor, qemu killed by an
external signal, ...) it removes its own pid file. If that removal
lands between the os.path.exists() check and the read+unlink,
either open() or os.unlink() raises FileNotFoundError. The
exception escapes handle(), is caught in runner.py pipeline_mgr,
and the worker process exits, aborting the whole twister run.
Other tests that had not yet been scheduled never get to run, and
tests still in flight on other DUTs are abandoned.
Keep the existence check as the happy-path guard (so the no-pid-file
case still logs qemu_pid=None as before), and wrap the read+unlink
in a try/except FileNotFoundError to close the race window. A
missing pid file at cleanup time is not an error -- by definition
the qemu process is gone, which is exactly the post-condition the
cleanup is trying to reach.
Reproduced with a long-running benchmark in qemu where the guest
hit its harness eof and exited a few ms before twister got around
to the unlink. With this fix the failing test is marked FAILED and
the run continues; without it the pipeline aborts on the
FileNotFoundError.
Signed-off-by: Parthiban Nallathambi <parthiban@linumiz.com>
QEMU_BIN_PATH is currently treated as a mandatory requirement for
QEMU-based tests on Windows.
Treat QEMU_BIN_PATH as an optional override and validate it only when
explicitly provided. This allows Twister to run QEMU tests using SDK or
other discovery mechanisms without requiring QEMU_BIN_PATH to be set.
Validation:
- QEMU_BIN_PATH unset -> qemu_x86 PASSED
- Invalid QEMU_BIN_PATH -> NOT RUN
- Valid QEMU_BIN_PATH -> qemu_x86 PASSED
Fixes#109332
Signed-off-by: Gobi M <gobi@aerlync.com>
Add a new testsuite YAML option `build` that allows a test
scenario to skip its own build step and instead use the build
artifacts from the first entry in `required_applications`.
Default is true; to skip building, set `build: false`.
This is useful for test scenarios that only run a harness (e.g.,
pytest, bsim) against an image built by another scenario, without
needing their own compiled binary.
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
This helps to diagnose issues when sys.exit is called with a string
as a parameter (as done e.g. in scripts/list_hardware.py).
Signed-off-by: Martin Jäger <martin@libre.solar>
Replace the bare 'pytest' string in the Pytest harness command
with sys.executable and '-m' so the subprocess always uses the
same Python interpreter that runs twister. This avoids a
FileNotFoundError when pytest is not on PATH, which happens
e.g. when launching twister from a VS Code debug configuration
that does not activate the virtual environment.
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
The --flash-timeout and --base-timeout options in plugin.py had
hardcoded defaults of 60. This breaks the 'or' fallback chain at
twister_harness_config.py:140-141 since 60 is truthy and always
wins over values from the YAML config file.
Result: Command line timeout options were completely ignored when
config file values were present.
Changes:
- Set plugin.py defaults to None so the fallback chain works correctly
- Add explicit 60 fallback at the end of the chain in
twister_harness_config.py to preserve default behavior
- Update test to use non-default value (120s) to verify config values
Signed-off-by: Jeremy Truttmann <jetstreeam.code@outlook.com>
Now use tests.yaml or tests.yml as the test definition file, sample.yaml
and testcase.yaml are deprecated but will still work for a while while
we transition.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Rework the sysbuild artifact relocation approach based on
reviewer feedback: instead of rewriting domains.yaml with
relative paths at package time, keep the original absolute
paths and rebase them transparently in Domains.from_file()
when the file has been moved to a different location.
This makes the fix generic - it works for any consumer of
Domains.from_file() (west flash, twister, pytest-twister-
harness), not just the --package-artifacts path.
Changes:
- domains.py: update from_file() to detect when build_dir
stored in domains.yaml differs from the file's actual
directory and rebase all domain paths accordingly.
Remove the now-unused _resolve_build_dir() helper and
get_domains_with_build_dirs_relative_to() method.
- runner.py: remove _sanitize_domains_file() and its call
site; path fixup is now handled by Domains.from_file().
- test_domains.py: replace the mocked __init__ approach
with end-to-end assertions; add a 'relocated artifacts'
test case that exercises the new rebasing logic.
- test_runner.py: remove test_projectbuilder_sanitize_
domains_file() and references to the deleted method.
Signed-off-by: Hake Huang <hake.huang@nxp.com>
Introduce a Script harness base class to enable running
shell-script-based tests through Twister, primarily
targeting BabbleSim test scenarios. The class provides
subprocess execution, output streaming, and log handling
that Pytest, Ctest, and Bsim now share through inheritance.
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
Add dt_label_compat_enabled() to filter targets that embed a node
label matching a target compatible string.
This helper function will be useful to consider zephyr,mapped-partition
compatible nodes that are now used instead of fixed-partition compatible
nodes for memory mapped flash partitions.
Signed-off-by: Etienne Carriere <etienne.carriere@st.com>
Fixes#105743
The build_dir path is not passed as a command-line argument but
through the YAML config file exchanged between Twister and the
pytest harness. Read build_dir from dut.device_config instead
of request.config.getoption.
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
Allow multi-DUT test scenarios to run on native_sim simulator.
For non-device handlers, create placeholder reserved_duts entries
using CompoundHardwareData so pytest harness can find DUT configs.
Multi-DUT on native_sim is only allowed when all required devices
use the same platform (no cross-platform multi-DUT in simulation).
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
Replace plain dict usage for required_application elements with
a typed RequiredApplication dataclass for improved type safety
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
Add support for flashing a different application to a required
DUT in multi-DUT testing. When a required_device entry specifies
an 'application' field, twister matches it against the
required_applications build dirs and assigns the correct
build_dir to the reserved DUT before flashing.
Move the required applications readiness check from
process_tasks() into ProjectBuilder.process() at the 'run'
stage. This ensures that building is not blocked by the
required_applications mechanism. Mutual
dependencies between instances is also allowed.
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
Main:
- Add HardwareReservationManager class and move DUT reservation
logic from HardwareMap to improve separation of concerns
- Add 'required_devices' structure to testsuite YAML schema to
define multiple device requirements for a test
- Add option to reserve multiple devices, filtered by platform
and fixtures. Fixtures can have an extra parameter (after ':')
that must match across the main DUT and required devices
Other:
- Add HarnessConfig dataclass to provide structured access to
harness configuration data instead of raw dict access
- Change 'fixture' parameter in harness_config from string to
list of strings for more flexibility when matching DUTs with
multiple fixture requirements
- Move conversion of short platform names to full target names
from hardware map into TestPlan module. This ensures platform
names are resolved before test discovery
- Add unit tests for HardwareReservationManager
- Align existing tests with new dataclass structures
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
Add option to execute command and then read the resposne for a
specified time. It can be useful if we want to validate additional
data that is output as the result of a command execution.
Signed-off-by: Krzysztof Chruściński <krzysztof.chruscinski@nordicsemi.no>
Twister would incorrectly fail, if you didn't have
the serial library installed. Instead moving it
to the try except clause causes it to be handled
correctly
Signed-off-by: Kasper Sloth <kasper.sloth99@gmail.com>
Parametrized tests in gtest are shown as the following example in a log:
[ FAILED ] FooTest/FooTypedTest/0.TypedTestFoo, where TypeParam = <type>
current regex of the gtest harness didn't detect such test so it marks
this as pass or that the result wasn't found. This commit extends the
regex to detect such behavior and adds unit test to verify that.
Signed-off-by: Martin Moya <moyamartin1@gmail.com>
that can never succeed because it is not in the com port filter list.
This would trigger every time that a combination of --flash-before and
--serial-pty is used.
The list of monitored serial ports - at least on linux - does not contain
/dev/pts but that list is constantly checked if the /dev/pts/X device might
magically appear some day.
This is the proper fix instead of adding /dev/pts/* to the monitor list
because the reason why we need to wait is because in the codepath without
serial_pty, we might be resetting the target device, causing the dev node
to disappear/reappear. This is not the case for the virtual ports created
for serial_pty.
Signed-off-by: Lucas Romero <lucas.romero@a-labs.io>
Moved all `sys.path.insert` from twister's modules
to `twisterlib.__init__.py` to have only one place where all paths
are added.
Signed-off-by: Łukasz Fundakowski <lukasz.fundakowski@nordicsemi.no>
Added unit tests for `twister_main.py` module to increase code coverage.
Did minor refactoring in `twister_main.py`.
Fixed type hints and added type ignores.
Removed unused variables.
Signed-off-by: Łukasz Fundakowski <lukasz.fundakowski@nordicsemi.no>
Signed-off-by: Lukasz Fundakowski <lukasz.fundakowski@nordicsemi.no>
When TOOLCHAIN_VARIANT_COMPILER is empty, we should not append
a trailing slash (/) to ZEPHYR_TOOLCHAIN_VARIANT. Or else any
matching of ZEPHYR_TOOLCHAIN_VARIANT would be incorrect.
For example, ZEPHYR_TOOLCHAIN_VARIANT == 'xt-clang' and west
build would have it correct, while twister would have it as
'xt-clang/'. This prevents kconfig from matching correctly
(possibly among other things).
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
- monitor_serial() opens handler.log in binary mode (\"wb\") and writes
re-encoded bytes. When the runner (e.g. linkserver for frdm-mcxn947)
produces serial output that is not cleanly decodable as UTF-8, the
decode('utf-8', 'ignore') silently drops bytes and the resulting
handler.log is unreadable.
- Fix by opening handler.log in text mode with explicit UTF-8 encoding,
consistent with BinaryHandler.handle_serial() which uses open(\"w\").
Change the decode error handler from 'ignore' to 'replace' so
corruption becomes visible as replacement characters rather than
silently dropped bytes.
Signed-off-by: Jjateen Gundesha <jjateen97@gmail.com>
Hardware reservation logic moved from Handler class to ProjectBuilder
runner using context manager approach. When NoDeviceAvailableException
is raised due to no free devices, the task is re-queued for later
processing instead of blocking the execution pipeline.
The reserve_hardware() context manager handles device reservation,
yields execution control only when hardware is available, and ensures
proper cleanup in the finally block.
All DUT management logic is now consolidated in HardwareMap class,
removing DUT references from Handler classes. This improves code
maintainability by centralizing responsibilities and eliminating
shared ownership of hardware resources between modules.
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
Increase the reset pulse from 10ms to 100ms to ensure reliable
boot across all Espressif SoCs. Shorter pulses may cause the UART
to miss early bootloader output, leading to false-positive test
failures.
Signed-off-by: Sylvio Alves <sylvio.alves@espressif.com>
support new syntax involving compiler and toolchains with multiple
compilers, i.e. zephyr/gnu
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
Update unit tests to verify YAML-based parameter serialization instead
of CLI arguments. Add tests for twister config file parsing and update
existing tests to check pytest_params object and YAML output.
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
Add comprehensive YAML configuration support for pytest-harness
integration, replacing CLI argument approach with structured
configuration files while maintaining backward compatibility.
Key changes:
- Add --twister-config option to load test parameters from YAML
- Refactor TwisterHarnessConfig to support CLI and YAML configuration
- Move pytest parameter generation from CLI args to config files
This enables more flexible test configuration through structured YAML
files and improves the overall architecture of pytest-twister
integration.
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
in some multi-core platform, `west flash` can not support all cores.
`west debug` is supported to test secondary core, add this to twister
such as with
`--west-flash-cmd=debug` in twister command
or with
`--west-flash-cmd: debug` in hwmap.yaml
and same with pytest
command line setting will overrid the one in hwmap
Signed-off-by: Hake Huang <hake.huang@nxp.com>
Co-authored-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
Simplify the design by removing the `detected` attribute from
HardwareMap class and DUT storage from ProjectBuilder class.
Instead of storing detected DUTs as instance attributes, pass
them as method parameters where needed.
This eliminates data duplication since ProjectBuilder can access DUTs
directly from the environment object, and HardwareMap doesn't need to
maintain its own copy of detected DUTs. The refactoring makes the code
cleaner and reduces the risk of inconsistent state between different
objects storing the same DUT information.
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
Update getting started guide, release notes, CI actions, ruff,
build system, and Python version compliance check to use Python
3.12 as minimum supported version.
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
Moving ZEPHYR_BASE from environment.py to constants.py
allows importing classes like HardwareMap in
pytest-twister-harness module without pulling in unnecessary
dependencies from environment.py.
This refactoring improves module independence and makes selective
imports cleaner across the twister Python package structure.
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
When user only wants to list tests or tags twister should
not clean previous results.
This change improves also performance for listing tests.
Signed-off-by: Lukasz Fundakowski <lukasz.fundakowski@nordicsemi.no>
Refactored main function for Twister execution.
- Rewrote the main function as a class and divided it into smaller,
more maintainable methods.
- Improved code structure, making it easier
to test with unit and module tests.
- Reduced the amount of code in a single function,
enhancing readability and understanding
Signed-off-by: Lukasz Fundakowski <lukasz.fundakowski@nordicsemi.no>
Move a check to run earlier in a for-loop to minimize doing unnecessary
work like creating TestInstance class instances.
This is a very minor optimization when the number of testsuites are low.
But starts to count when there are a lot of testsuites.
Signed-off-by: Guðni Már Gilbert <gudni.m.g@gmail.com>
Currently apply_filters is parsing & validating a snippet YAML
file multiple times, duplicating expensive work.
Do the work once per testsuite instead of for each testsuite
multiplied by number of toolchains.
Signed-off-by: Guðni Már Gilbert <gudni.m.g@gmail.com>
Exit from twister in case of user's error e.g. empty list
of tests provided by user instead of throwing unhandled
exception.
Signed-off-by: Lukasz Fundakowski <lukasz.fundakowski@nordicsemi.no>
