Rework the sysbuild artifact relocation approach based on
reviewer feedback: instead of rewriting domains.yaml with
relative paths at package time, keep the original absolute
paths and rebase them transparently in Domains.from_file()
when the file has been moved to a different location.
This makes the fix generic - it works for any consumer of
Domains.from_file() (west flash, twister, pytest-twister-
harness), not just the --package-artifacts path.
Changes:
- domains.py: update from_file() to detect when build_dir
stored in domains.yaml differs from the file's actual
directory and rebase all domain paths accordingly.
Remove the now-unused _resolve_build_dir() helper and
get_domains_with_build_dirs_relative_to() method.
- runner.py: remove _sanitize_domains_file() and its call
site; path fixup is now handled by Domains.from_file().
- test_domains.py: replace the mocked __init__ approach
with end-to-end assertions; add a 'relocated artifacts'
test case that exercises the new rebasing logic.
- test_runner.py: remove test_projectbuilder_sanitize_
domains_file() and references to the deleted method.
Signed-off-by: Hake Huang <hake.huang@nxp.com>
Introduce a Script harness base class to enable running
shell-script-based tests through Twister, primarily
targeting BabbleSim test scenarios. The class provides
subprocess execution, output streaming, and log handling
that Pytest, Ctest, and Bsim now share through inheritance.
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
Add dt_label_compat_enabled() to filter targets that embed a node
label matching a target compatible string.
This helper function will be useful to consider zephyr,mapped-partition
compatible nodes that are now used instead of fixed-partition compatible
nodes for memory mapped flash partitions.
Signed-off-by: Etienne Carriere <etienne.carriere@st.com>
Allow multi-DUT test scenarios to run on native_sim simulator.
For non-device handlers, create placeholder reserved_duts entries
using CompoundHardwareData so pytest harness can find DUT configs.
Multi-DUT on native_sim is only allowed when all required devices
use the same platform (no cross-platform multi-DUT in simulation).
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
Replace plain dict usage for required_application elements with
a typed RequiredApplication dataclass for improved type safety
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
Add support for flashing a different application to a required
DUT in multi-DUT testing. When a required_device entry specifies
an 'application' field, twister matches it against the
required_applications build dirs and assigns the correct
build_dir to the reserved DUT before flashing.
Move the required applications readiness check from
process_tasks() into ProjectBuilder.process() at the 'run'
stage. This ensures that building is not blocked by the
required_applications mechanism. Mutual
dependencies between instances is also allowed.
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
Main:
- Add HardwareReservationManager class and move DUT reservation
logic from HardwareMap to improve separation of concerns
- Add 'required_devices' structure to testsuite YAML schema to
define multiple device requirements for a test
- Add option to reserve multiple devices, filtered by platform
and fixtures. Fixtures can have an extra parameter (after ':')
that must match across the main DUT and required devices
Other:
- Add HarnessConfig dataclass to provide structured access to
harness configuration data instead of raw dict access
- Change 'fixture' parameter in harness_config from string to
list of strings for more flexibility when matching DUTs with
multiple fixture requirements
- Move conversion of short platform names to full target names
from hardware map into TestPlan module. This ensures platform
names are resolved before test discovery
- Add unit tests for HardwareReservationManager
- Align existing tests with new dataclass structures
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
Parametrized tests in gtest are shown as the following example in a log:
[ FAILED ] FooTest/FooTypedTest/0.TypedTestFoo, where TypeParam = <type>
current regex of the gtest harness didn't detect such test so it marks
this as pass or that the result wasn't found. This commit extends the
regex to detect such behavior and adds unit test to verify that.
Signed-off-by: Martin Moya <moyamartin1@gmail.com>
Moved all `sys.path.insert` from twister's modules
to `twisterlib.__init__.py` to have only one place where all paths
are added.
Signed-off-by: Łukasz Fundakowski <lukasz.fundakowski@nordicsemi.no>
Added unit tests for `twister_main.py` module to increase code coverage.
Did minor refactoring in `twister_main.py`.
Fixed type hints and added type ignores.
Removed unused variables.
Signed-off-by: Łukasz Fundakowski <lukasz.fundakowski@nordicsemi.no>
Signed-off-by: Lukasz Fundakowski <lukasz.fundakowski@nordicsemi.no>
Update unit tests after moving hardware reservation logic from
Handler class to HardwareMap. Relocate and rename tests to match
the new module structure and API changes.
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
This commit updates the Twister Black Box tests to expect the enhanced
toolchain variant names (i.e. `host/gnu` and `zephyr/gnu`).
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
Update unit tests to verify YAML-based parameter serialization instead
of CLI arguments. Add tests for twister config file parsing and update
existing tests to check pytest_params object and YAML output.
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
Assign a platform to the test so Twister doesn't
pick up multiple platforms.
This reduces the test time by ~2 minutes.
Before: 9 passed in 265.88s (0:04:25)
After: 9 passed in 140.24s (0:02:20)
Signed-off-by: Guðni Már Gilbert <gudni.m.g@gmail.com>
Exit from twister in case of user's error e.g. empty list
of tests provided by user instead of throwing unhandled
exception.
Signed-off-by: Lukasz Fundakowski <lukasz.fundakowski@nordicsemi.no>
jsonschema is more strict when it comes to typing and only allows
strings for 'serial' in hwmap-schema.yaml.
Since 'serial' is not a required key in the schema, this commit
removes the 'serial' key from generated hardware maps when the
'serial' value is empty /unknown.
Signed-off-by: Guðni Már Gilbert <gudni.m.g@gmail.com>
Convert the DUT class from a traditional class to a dataclass to enable
proper serialization and support for future multi-device testing with
pytest-harness.
Key changes:
- Migrated DUT class to use `dataclass` decorator with proper type hints
- Renamed `baud` property to `serial_baud` for consistency
- Updated hardware map schema to support both `baud` (legacy) and
`serial_baud` fields for backward compatibility
- Updated tests
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
TwisterEnv already parses west modules during initialization, but TestPlan
was parsing them again via zephyr_module.parse_modules(), duplicating work.
Store the parsed module objects on TwisterEnv (self.modules) and have
TestPlan derive its module name list from env.modules instead. This drops
the redundant parse in TestPlan by removing handle_modules() and its call
site.
Each call to parse_modules() takes around 250-300ms on my end.
Update twister unit tests
Signed-off-by: Guðni Már Gilbert <gudni.m.g@gmail.com>
This fixes the issue where --list-tests and --test-tree commands
showed duplicated testsuite IDs in the output,
simplifies testcase name handling in harness and runner modules.
Removed feature to extend behaviour of --no-detailed-test-id to shorten
test case names - feature was introduced in #82302.
This makes the twister output more predictable and the codebase easier
to maintain.
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
The extra_conf_files arg in testcase.yaml was being improperly
assigned to the CONF_FILE CMake arg, which caused
configuration_files.cmake to ignore soc and board Kconfig overlays.
Signed-off-by: Lauren Murphy <lauren.murphy@intel.com>
Make Domain's class constructor accept python object to avoid redundant
YAML serialize and unserialize which may introduce potential yaml type
errors.
Signed-off-by: Yves Wang <zhengjia.wang@nxp.com>
use the allow-list generated by --keep-artifacts whenever
cleanup_artifacts() is called. This resolves an issue where this allow-list
was not being honored when using the --prep-artifacts-for-testing flag.
Signed-off-by: Mike Szczys <mike@golioth.io>
Reduce the test time by ~300 seconds by mocking the listed serial port
Result from running:
pytest --durations=0 ./scripts/tests/twister/test_handlers.py
Before:
60.12s test_devicehandler_handle[valid pty]
60.12s test_devicehandler_handle[communicate timeout]
60.06s test_devicehandler_handle[popen called process error]
60.05s test_devicehandler_handle[valid dev]
60.03s test_devicehandler_handle[nonzero returncode]
0.01s test_devicehandler_handle[create serial failure]
After:
0.01s test_devicehandler_handle[valid dev]
0.01s test_devicehandler_handle[valid pty]
0.01s test_devicehandler_handle[nonzero returncode]
0.01s test_devicehandler_handle[create serial failure]
0.01s test_devicehandler_handle[communicate timeout]
0.01s test_devicehandler_handle[popen called process error]
Signed-off-by: Guðni Már Gilbert <gudni.m.g@gmail.com>
Now that the STM32CubeProgrammer runner supports the standard `--dev-id`
argument, replace usage of `--tool-opt` in the Twister device handler code.
Signed-off-by: Mathieu Choplain <mathieu.choplain-ext@st.com>
Silence CI warning caused by line "from . import ZEPHYR_BASE". This is a
legitimate pattern, but pylint seems unable to deduce so. Many other
scripts perform a similar silencing.
Signed-off-by: Mathieu Choplain <mathieu.choplain-ext@st.com>
Extend twister's device serial handling to support multiple serial ports
for pytest-based tests. This enables testing scenarios that require
communication with multiple UART interfaces on the same device.
This enhancement enables comprehensive testing of multi-UART devices
while maintaining backward compatibility with single serial setups.
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
Partially revert 1bcc0652241af70d0b28aee155f260cd9ca0fc74
This change impacts all other scripts that import those modules, while
those weren't updated.
It will be hard to maintain this list, whilst keeping the entire tree
compliant.
Signed-off-by: Pieter De Gendt <pieter.degendt@basalte.be>
Removed redundant try..except around the code
and fixed a test for that code which was false positive.
Signed-off-by: Łukasz Fundakowski <lukasz.fundakowski@nordicsemi.no>
This feature enables sharing of built applications between test
scenarios, allowing tests to access build artifacts from other
applications.
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
Use csv instead of a plain split to split extra arguments for extra west
flags, this allows correct passing of arguments that are internally
escaped in quotes, such as arugments containing commas.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
Refactored twister_main.py module to simplify the code and system tests.
Removed the need to patch `sys.argv` in blackbox tests.
Signed-off-by: Lukasz Fundakowski <lukasz.fundakowski@nordicsemi.no>
Currently twister requires west to flash a board, either directly using
west-flash or indirectly through the cmake flash target.
Add an option to specify a custom flash script, this is passed a
build-dir and board-id flags so it can be used to implement custom
flashing scripts in a system that does not use west.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
