twister: Account for board & SoC extensions

Problem
-------

Board & SoC extensions are used to define out-of-tree board variants or
SoC qualifiers. When a board is extended, it has multiple directories
associated with it (each with its own `board.yml`), where twister should
be able to find additional platform files to support these qualifiers.
Currently, this doesn't work, because twister only traverses the primary
BOARD_DIR and ignores the rest.

The fix would've been trivial in the case of "legacy" platform files,
i.e. those of the form `<normalized_board_target>.yaml`, but it's less
straightforward for the newly introduced `twister.yaml` format.

A `twister.yaml` file contains platform configuration that can be shared
by multiple board targets and tweaked for specific targets by using the
top-level `variants` key. Normally, there is at most one `twister.yaml`
per board, but the file isn't necessarily unique to one board. Instead,
it's unique to one directory, which may define multiple boards (as is
the case with e.g. `boards/qemu/x86/`).

With extensions in the picture, the goal is to initialize platforms when
given multiple `twister.yaml` per board. The OOT files are expected to
only provide information about OOT board targets, without being able to
override in-tree targets (same principle as in the Zephyr build system).

Solution
--------

The `twister.yaml` handling is broken up into multiple passes - first
loading all the files, then splitting the `variants` keys apart from the
shared configuration, before constructing the Platform instances.

The purpose of the split is to treat the variant information as global,
instead of making unnecessary or faulty assumptions about locality.
Remember that the build system can derive board target names not only
from `board.yml`, but from `soc.yml` too. Considering that any board may
end up using an OOT-extended SoC (and hence multiple `soc.yml` files),
not every board target can be said to belong to some board dir.

Unlike the variant data, the remaining top-level config is still rooted
to the primary BOARD_DIR and inherited by the extension dirs from there.
This is quite intuitive in most imagined cases, but there is a caveat:
if a `twister.yaml` resides in an extension dir, then it is allowed to
have a top-level config of its own, but it will be silently ignored.
This is to support corner cases where, much like how a single board dir
can define multiple boards, a single board dir can also extend multiple
boards, or even do both. In those cases, the primary BOARD_DIR rule
should make it unambiguous which config belongs to which board, even if
it may seem counter-intuitive at first.

For concrete examples of what this means, please see the newly added
platform unit tests.

As part of these functional changes, a good chunk of logic is moved out
of `TestPlan.add_configurations()` into a new function in `platform.py`.
This is because recombining the top-level and variant configs requires
direct manipulation of the loaded YAML contents, which would be improper
to do outside of the module responsible for encapsulating this data.

Signed-off-by: Grzegorz Swiderski <grzegorz.swiderski@nordicsemi.no>

scripts/pylib/twister/twisterlib/platform.py

@@ -9,7 +9,10 @@
 import logging
 import os
 import shutil
+from argparse import Namespace
+from itertools import groupby
 
+import list_boards
 import scl
 from twisterlib.constants import SUPPORTED_SIMS
 from twisterlib.environment import ZEPHYR_BASE
@@ -84,28 +87,18 @@ class Platform:
         self.filter_data = dict()
         self.uart = ""
         self.resc = ""
-        self.qualifier = None
 
-    def load(self, board, target, aliases, data):
+    def load(self, board, target, aliases, data, variant_data):
         """Load the platform data from the board data and target data
         board: the board object as per the zephyr build system
         target: the target name of the board as per the zephyr build system
         aliases: list of aliases for the target
-        data: the data from the twister.yaml file for the target
+        data: the default data from the twister.yaml file for the board
+        variant_data: the target-specific data to replace the default data
         """
         self.name = target
         self.aliases = aliases
 
-        # Get data for various targets and use the main board data as a
-        # defauly. Individual variant information will replace the default data
-        # provded in the main twister configuration for this board.
-        variants = data.get("variants", {})
-        variant_data = {}
-        for alias in aliases:
-            variant_data = variants.get(alias, {})
-            if variant_data:
-                break
-
         self.normalized_name = self.name.replace("/", "_")
         self.sysbuild = variant_data.get("sysbuild", data.get("sysbuild", self.sysbuild))
         self.twister = variant_data.get("twister", data.get("twister", self.twister))
@@ -194,3 +187,116 @@ class Platform:
 
     def __repr__(self):
         return f"<{self.name} on {self.arch}>"
+
+
+def generate_platforms(board_roots, soc_roots, arch_roots):
+    """Initialize and yield all Platform instances.
+
+    Using the provided board/soc/arch roots, determine the available
+    platform names and load the test platform description files.
+
+    An exception is raised if not all platform files are valid YAML,
+    or if not all platform names are unique.
+    """
+    alias2target = {}
+    target2board = {}
+    target2data = {}
+    dir2data = {}
+    legacy_files = []
+
+    lb_args = Namespace(board_roots=board_roots, soc_roots=soc_roots, arch_roots=arch_roots,
+                        board=None, board_dir=None)
+
+    for board in list_boards.find_v2_boards(lb_args).values():
+        for board_dir in board.directories:
+            if board_dir in dir2data:
+                # don't load the same board data twice
+                continue
+            file = board_dir / "twister.yaml"
+            if file.is_file():
+                data = scl.yaml_load_verify(file, Platform.platform_schema)
+            else:
+                data = None
+            dir2data[board_dir] = data
+
+            legacy_files.extend(
+                file for file in board_dir.glob("*.yaml") if file.name != "twister.yaml"
+            )
+
+        for qual in list_boards.board_v2_qualifiers(board):
+            if board.revisions:
+                for rev in board.revisions:
+                    if rev.name:
+                        target = f"{board.name}@{rev.name}/{qual}"
+                        alias2target[target] = target
+                        if rev.name == board.revision_default:
+                            alias2target[f"{board.name}/{qual}"] = target
+                        if '/' not in qual and len(board.socs) == 1:
+                            if rev.name == board.revision_default:
+                                alias2target[f"{board.name}"] = target
+                            alias2target[f"{board.name}@{rev.name}"] = target
+                    else:
+                        target = f"{board.name}/{qual}"
+                        alias2target[target] = target
+                        if '/' not in qual and len(board.socs) == 1 \
+                                and rev.name == board.revision_default:
+                            alias2target[f"{board.name}"] = target
+
+                    target2board[target] = board
+            else:
+                target = f"{board.name}/{qual}"
+                alias2target[target] = target
+                if '/' not in qual and len(board.socs) == 1:
+                    alias2target[board.name] = target
+                target2board[target] = board
+
+    for board_dir, data in dir2data.items():
+        if data is None:
+            continue
+        # Separate the default and variant information in the loaded board data.
+        # The default (top-level) data can be shared by multiple board targets;
+        # it will be overlaid by the variant data (if present) for each target.
+        variant_data = data.pop("variants", {})
+        for variant in variant_data:
+            target = alias2target.get(variant)
+            if target is None:
+                continue
+            if target in target2data:
+                logger.error(f"Duplicate platform {target} in {board_dir}")
+                raise Exception(f"Duplicate platform identifier {target} found")
+            target2data[target] = variant_data[variant]
+
+    # note: this inverse mapping will only be used for loading legacy files
+    target2aliases = {}
+
+    for target, aliases in groupby(alias2target, alias2target.get):
+        aliases = list(aliases)
+        board = target2board[target]
+
+        # Default board data always comes from the primary 'board.dir'.
+        # Other 'board.directories' can only supply variant data.
+        data = dir2data[board.dir]
+        if data is not None:
+            variant_data = target2data.get(target, {})
+
+            platform = Platform()
+            platform.load(board, target, aliases, data, variant_data)
+            yield platform
+
+        target2aliases[target] = aliases
+
+    for file in legacy_files:
+        data = scl.yaml_load_verify(file, Platform.platform_schema)
+        target = alias2target.get(data.get("identifier"))
+        if target is None:
+            continue
+
+        board = target2board[target]
+        if dir2data[board.dir] is not None:
+            # all targets are already loaded for this board
+            logger.error(f"Duplicate platform {target} in {file.parent}")
+            raise Exception(f"Duplicate platform identifier {target} found")
+
+        platform = Platform()
+        platform.load(board, target, target2aliases[target], data, variant_data={})
+        yield platform

scripts/pylib/twister/twisterlib/testplan.py

@@ -7,7 +7,6 @@
 # SPDX-License-Identifier: Apache-2.0
 import collections
 import copy
-import glob
 import itertools
 import json
 import logging
@@ -28,11 +27,10 @@ try:
 except ImportError:
     print("Install the anytree module to use the --test-tree option")
 
-import list_boards
 import scl
 from twisterlib.config_parser import TwisterConfigParser
 from twisterlib.error import TwisterRuntimeError
-from twisterlib.platform import Platform
+from twisterlib.platform import Platform, generate_platforms
 from twisterlib.quarantine import Quarantine
 from twisterlib.statuses import TwisterStatus
 from twisterlib.testinstance import TestInstance
@@ -437,99 +435,21 @@ class TestPlan:
         sys.stdout.write(what + "\n")
         sys.stdout.flush()
 
-    def find_twister_data(self, board_data_list, board_aliases):
-        """Find the twister data for a board in the list of board data based on the aliases"""
-        for board_data in board_data_list:
-            if board_data.get('identifier') in board_aliases:
-                return board_data
-
     def add_configurations(self):
         # Create a list of board roots as defined by the build system in general
         # Note, internally in twister a board root includes the `boards` folder
         # but in Zephyr build system, the board root is without the `boards` in folder path.
         board_roots = [Path(os.path.dirname(root)) for root in self.env.board_roots]
-        lb_args = Namespace(arch_roots=self.env.arch_roots, soc_roots=self.env.soc_roots,
+        soc_roots = self.env.soc_roots
-                            board_roots=board_roots, board=None, board_dir=None)
+        arch_roots = self.env.arch_roots
 
-        known_boards = list_boards.find_v2_boards(lb_args)
-        bdirs = {}
         platform_config = self.test_config.get('platforms', {})
 
-        # helper function to initialize and add platforms
+        for platform in generate_platforms(board_roots, soc_roots, arch_roots):
-        def init_and_add_platforms(data, board, target, qualifier, aliases):
-            platform = Platform()
-            if not new_config_found:
-                data = self.find_twister_data(bdirs[board.dir], aliases)
-                if not data:
-                    return
-            platform.load(board, target, aliases, data)
-            platform.qualifier = qualifier
-            if platform.name in [p.name for p in self.platforms]:
-                logger.error(f"Duplicate platform {platform.name} in {board.dir}")
-                raise Exception(f"Duplicate platform identifier {platform.name} found")
             if not platform.twister:
-                return
+                continue
             self.platforms.append(platform)
 
-        for board in known_boards.values():
-            new_config_found = False
-            # don't load the same board data twice
-            if not bdirs.get(board.dir):
-                datas = []
-                for file in glob.glob(os.path.join(board.dir, "*.yaml")):
-                    if os.path.basename(file) == "twister.yaml":
-                        continue
-                    try:
-                        scp = TwisterConfigParser(file, Platform.platform_schema)
-                        sdata = scp.load()
-                        datas.append(sdata)
-                    except Exception as e:
-                        logger.error(f"Error loading {file}: {e!r}")
-                        self.load_errors += 1
-                        continue
-                bdirs[board.dir] = datas
-            data = {}
-            if os.path.exists(board.dir / 'twister.yaml'):
-                try:
-                    scp = TwisterConfigParser(board.dir / 'twister.yaml', Platform.platform_schema)
-                    data = scp.load()
-                except Exception as e:
-                    logger.error(f"Error loading {board.dir / 'twister.yaml'}: {e!r}")
-                    self.load_errors += 1
-                    continue
-                new_config_found = True
-
-
-
-            for qual in list_boards.board_v2_qualifiers(board):
-
-                if board.revisions:
-                    for rev in board.revisions:
-                        if rev.name:
-                            target = f"{board.name}@{rev.name}/{qual}"
-                            aliases = [target]
-                            if rev.name == board.revision_default:
-                                aliases.append(f"{board.name}/{qual}")
-                            if '/' not in qual and len(board.socs) == 1:
-                                if rev.name == board.revision_default:
-                                    aliases.append(f"{board.name}")
-                                aliases.append(f"{board.name}@{rev.name}")
-                        else:
-                            target = f"{board.name}/{qual}"
-                            aliases = [target]
-                            if '/' not in qual and len(board.socs) == 1 \
-                                    and rev.name == board.revision_default:
-                                aliases.append(f"{board.name}")
-
-                        init_and_add_platforms(data, board, target, qual, aliases)
-                else:
-                    target = f"{board.name}/{qual}"
-                    aliases = [target]
-                    if '/' not in qual and len(board.socs) == 1:
-                        aliases.append(board.name)
-                    init_and_add_platforms(data, board, target, qual, aliases)
-
-        for platform in self.platforms:
             if not platform_config.get('override_default_platforms', False):
                 if platform.default:
                     self.default_platforms.append(platform.name)

scripts/tests/twister/test_platform.py

@@ -11,10 +11,13 @@ import os
 import mock
 import pytest
 
+from contextlib import nullcontext
+from pykwalify.errors import SchemaError
+
 ZEPHYR_BASE = os.getenv("ZEPHYR_BASE")
 sys.path.insert(0, os.path.join(ZEPHYR_BASE, "scripts/pylib/twister"))
 
-from twisterlib.platform import Platform, Simulator
+from twisterlib.platform import Platform, Simulator, generate_platforms
 
 
 TESTDATA_1 = [
@@ -129,3 +132,363 @@ def xtest_platform_load(platform_text, expected_data, expected_repr):
             assert att == v
 
     assert platform.__repr__() == expected_repr
+
+
+TESTDATA_2 = [
+    (
+        ['m0'],
+        None,
+        {
+            'p1e1/s1', 'p1e2/s1', 'p2/s1', 'p3@A/s2/c1', 'p3@B/s2/c1',
+        },
+    ),
+    (
+        ['m0', 'm1'],
+        None,
+        {
+            'p1e1/s1', 'p1e2/s1', 'p2/s1', 'p3@A/s2/c1', 'p3@B/s2/c1',
+            'p1e1/s1/v1', 'p1e1/s1/v2', 'p1e2/s1/v1', 'p2/s1/v1',
+        },
+    ),
+    (
+        ['m0', 'm1', 'm2'],
+        None,
+        {
+            'p1e1/s1', 'p1e2/s1', 'p2/s1', 'p3@A/s2/c1', 'p3@B/s2/c1',
+            'p1e1/s1/v1', 'p1e1/s1/v2', 'p1e2/s1/v1', 'p2/s1/v1',
+            'p3@A/s2/c2', 'p3@B/s2/c2', 'p4/s1',
+        },
+    ),
+    (
+        ['m0', 'm3'],
+        Exception("Duplicate platform identifier p1e1/s1 found"),
+        None,
+    ),
+    (
+        ['m0', 'm1', 'm4'],
+        Exception("Duplicate platform identifier p1e2/s1/v1 found"),
+        None,
+    ),
+    (
+        ['m0', 'm5'],
+        SchemaError(), # Unknown message as this is raised externally
+        None,
+    ),
+]
+
+@pytest.mark.parametrize(
+    'roots, expected_exception, expected_platform_names',
+    TESTDATA_2,
+    ids=[
+        'default board root',
+        '1 extra board root',
+        '2 extra board roots',
+        '1 extra board root, duplicate platform',
+        '2 extra board roots, duplicate platform',
+        '1 extra board root, malformed yaml',
+    ]
+)
+def test_generate_platforms(
+    tmp_path,
+    roots,
+    expected_exception,
+    expected_platform_names,
+):
+    tmp_files = {
+        'm0/boards/zephyr/p1/board.yml': """\
+boards:
+  - name: p1e1
+    vendor: zephyr
+    socs:
+      - name: s1
+  - name: p1e2
+    vendor: zephyr
+    socs:
+      - name: s1
+""",
+        'm0/boards/zephyr/p1/twister.yaml': """\
+type: native
+arch: x86
+variants:
+  p1e1:
+    twister: False
+  p1e2:
+    sysbuild: True
+""",
+        'm0/boards/zephyr/p2/board.yml': """\
+boards:
+  - name: p2
+    vendor: zephyr
+    socs:
+      - name: s1
+""",
+        'm0/boards/zephyr/p2/p2.yaml': """\
+identifier: p2/s1
+type: sim
+arch: x86
+vendor: vendor2
+testing:
+  default: True
+""",
+        'm0/boards/arm/p3/board.yml': """\
+board:
+  name: p3
+  vendor: arm
+  revision:
+    format: letter
+    default: "A"
+    revisions:
+      - name: "A"
+      - name: "B"
+  socs:
+    - name: s2
+""",
+        'm0/boards/arm/p3/twister.yaml': """\
+type: unit
+arch: arm
+vendor: vendor3
+sysbuild: True
+variants:
+  p3/s2/c1:
+    testing:
+      timeout_multiplier: 2.71828
+  p3@B/s2/c1:
+    testing:
+      timeout_multiplier: 3.14159
+""",
+        'm0/soc/zephyr/soc.yml': """\
+family:
+  - name: zephyr
+    series:
+      - name: zephyr_testing
+        socs:
+          - name: s1
+          - name: s2
+            cpuclusters:
+              - name: c1
+""",
+        'm1/boards/zephyr/p1e1/board.yml': """\
+board:
+  extend: p1e1
+  variants:
+    - name: v1
+      qualifier: s1
+    - name: v2
+      qualifier: s1
+""",
+        'm1/boards/zephyr/p1e1/twister.yaml': """\
+variants:
+  p1e1/s1/v1:
+    testing:
+      default: True
+""",
+        'm1/boards/zephyr/p1e2/board.yml': """\
+board:
+  extend: p1e2
+  variants:
+    - name: v1
+      qualifier: s1
+""",
+        'm1/boards/zephyr/p2/board.yml': """\
+board:
+  extend: p2
+  variants:
+    - name: v1
+      qualifier: s1
+""",
+        'm1/boards/zephyr/p2/p2_s1_v1.yaml': """\
+identifier: p2/s1/v1
+""",
+        'm2/boards/misc/board.yml': """\
+boards:
+  - extend: p3
+  - name: p4
+    vendor: misc
+    socs:
+      - name: s1
+""",
+        'm2/boards/misc/twister.yaml': """\
+type: qemu
+arch: riscv
+vendor: vendor4
+simulation:
+  - name: qemu
+variants:
+  p3@A/s2/c2:
+    sysbuild: False
+""",
+        'm2/soc/zephyr/soc.yml': """\
+socs:
+  - extend: s2
+    cpuclusters:
+      - name: c2
+""",
+        'm3/boards/zephyr/p1e1/board.yml': """\
+board:
+  extend: p1e1
+""",
+        'm3/boards/zephyr/p1e1/twister.yaml': """\
+variants:
+  p1e1/s1:
+    name: Duplicate Platform
+""",
+        'm4/boards/zephyr/p1e2/board.yml': """\
+board:
+  extend: p2
+""",
+        'm4/boards/zephyr/p1e2/p1e2_s1_v1.yaml': """\
+identifier: p1e2/s1/v1
+""",
+        'm5/boards/zephyr/p2/p2-2.yaml': """\
+testing:
+  ć#@%!#!#^#@%@:1.0
+identifier: p2_2
+type: sim
+arch: x86
+vendor: vendor2
+""",
+        'm5/boards/zephyr/p2/board.yml': """\
+board:
+  extend: p2
+""",
+    }
+
+    for filename, content in tmp_files.items():
+        (tmp_path / filename).parent.mkdir(parents=True, exist_ok=True)
+        (tmp_path / filename).write_text(content)
+
+    roots = list(map(tmp_path.joinpath, roots))
+    with pytest.raises(type(expected_exception)) if \
+          expected_exception else nullcontext() as exception:
+        platforms = list(generate_platforms(board_roots=roots, soc_roots=roots, arch_roots=roots))
+
+    if expected_exception:
+        if expected_exception.args:
+            assert str(expected_exception) == str(exception.value)
+        return
+
+    platform_names = {platform.name for platform in platforms}
+    assert len(platforms) == len(platform_names)
+    assert platform_names == expected_platform_names
+
+    expected_data = {
+        'p1e1/s1': {
+            'aliases': ['p1e1/s1', 'p1e1'],
+            # m0/boards/zephyr/p1/board.yml
+            'vendor': 'zephyr',
+            # m0/boards/zephyr/p1/twister.yaml (base + variant)
+            'twister': False,
+            'arch': 'x86',
+            'type': 'native',
+        },
+        'p1e2/s1': {
+            'aliases': ['p1e2/s1', 'p1e2'],
+            # m0/boards/zephyr/p1/board.yml
+            'vendor': 'zephyr',
+            # m0/boards/zephyr/p1/twister.yaml (base + variant)
+            'sysbuild': True,
+            'arch': 'x86',
+            'type': 'native',
+        },
+        'p1e1/s1/v1': {
+            'aliases': ['p1e1/s1/v1'],
+            # m0/boards/zephyr/p1/board.yml
+            'vendor': 'zephyr',
+            # m0/boards/zephyr/p1/twister.yaml (base)
+            # m1/boards/zephyr/p1e1/twister.yaml (variant)
+            'default': True,
+            'arch': 'x86',
+            'type': 'native',
+        },
+        'p1e1/s1/v2': {
+            'aliases': ['p1e1/s1/v2'],
+            # m0/boards/zephyr/p1/board.yml
+            'vendor': 'zephyr',
+            # m0/boards/zephyr/p1/twister.yaml (base)
+            'arch': 'x86',
+            'type': 'native',
+        },
+        'p1e2/s1/v1': {
+            'aliases': ['p1e2/s1/v1'],
+            # m0/boards/zephyr/p1/board.yml
+            'vendor': 'zephyr',
+            # m0/boards/zephyr/p1/twister.yaml (base)
+            'arch': 'x86',
+            'type': 'native',
+        },
+        'p2/s1': {
+            'aliases': ['p2/s1', 'p2'],
+            # m0/boards/zephyr/p2/board.yml
+            'vendor': 'zephyr',
+            # m0/boards/zephyr/p2/p2.yaml
+            'default': True,
+            'arch': 'x86',
+            'type': 'sim',
+        },
+        'p2/s1/v1': {
+            'aliases': ['p2/s1/v1'],
+            # m0/boards/zephyr/p2/board.yml
+            'vendor': 'zephyr',
+            # m1/boards/zephyr/p2/p2_s1_v1.yaml
+        },
+        'p3@A/s2/c1': {
+            'aliases': ['p3@A/s2/c1', 'p3/s2/c1'],
+            # m0/boards/arm/p3/board.yml
+            'vendor': 'arm',
+            # m0/boards/arm/p3/twister.yaml (base + variant)
+            'sysbuild': True,
+            'timeout_multiplier': 2.71828,
+            'arch': 'arm',
+            'type': 'unit',
+        },
+        'p3@B/s2/c1': {
+            'aliases': ['p3@B/s2/c1'],
+            # m0/boards/arm/p3/board.yml
+            'vendor': 'arm',
+            # m0/boards/arm/p3/twister.yaml (base + variant)
+            'sysbuild': True,
+            'timeout_multiplier': 3.14159,
+            'arch': 'arm',
+            'type': 'unit',
+        },
+        'p3@A/s2/c2': {
+            'aliases': ['p3@A/s2/c2', 'p3/s2/c2'],
+            # m0/boards/arm/p3/board.yml
+            'vendor': 'arm',
+            # m0/boards/arm/p3/twister.yaml (base)
+            # m2/boards/misc/twister.yaml (variant)
+            'sysbuild': False,
+            'arch': 'arm',
+            'type': 'unit',
+        },
+        'p3@B/s2/c2': {
+            'aliases': ['p3@B/s2/c2'],
+            # m0/boards/arm/p3/board.yml
+            'vendor': 'arm',
+            # m0/boards/arm/p3/twister.yaml (base)
+            'sysbuild': True,
+            'arch': 'arm',
+            'type': 'unit',
+        },
+        'p4/s1': {
+            'aliases': ['p4/s1', 'p4'],
+            # m2/boards/misc/board.yml
+            'vendor': 'misc',
+            # m2/boards/misc/twister.yaml (base)
+            'arch': 'riscv',
+            'type': 'qemu',
+            'simulators': [Simulator({'name': 'qemu'})],
+            'simulation': 'qemu',
+        },
+    }
+
+    init_platform = Platform()
+    for platform in platforms:
+        expected_platform_data = expected_data[platform.name]
+        for attr, default in vars(init_platform).items():
+            if attr in {'name', 'normalized_name', 'supported_toolchains'}:
+                continue
+            expected = expected_platform_data.get(attr, default)
+            actual = getattr(platform, attr, None)
+            assert expected == actual, \
+                f"expected '{platform}.{attr}' to be '{expected}', was '{actual}'"

scripts/tests/twister/test_testplan.py

@@ -1110,167 +1110,7 @@ def test_testplan_add_configurations(
     expected_platform_names,
     expected_defaults
 ):
-    # tmp_path
+    env = mock.Mock(board_roots=[tmp_path / 'boards'], soc_roots=[tmp_path], arch_roots=[tmp_path])
-    # └ boards  <- board root
-    #   ├ zephyr
-    #   │ ├ p1
-    #   │ | ├ p1e1.yaml
-    #   │ | └ p1e2.yaml
-    #   │ └ p2
-    #   │   ├ p2.yaml
-    #   │   └ p2-1.yaml <- duplicate
-    #   │   └ p2-2.yaml <- load error
-    #   └ arm
-    #     └ p3
-    #       ├ p3.yaml
-    #       └ p3_B.conf
-    tmp_soc_root_dir = tmp_path / 'soc'
-    tmp_soc_root_dir.mkdir()
-
-    tmp_vend1_dir = tmp_soc_root_dir / 'zephyr'
-    tmp_vend1_dir.mkdir()
-
-    tmp_soc1_dir = tmp_vend1_dir / 's1'
-    tmp_soc1_dir.mkdir()
-
-    soc1_yaml = """\
-family:
-  - name: zephyr
-    series:
-      - name: zephyr_testing
-        socs:
-          - name: unit_testing
-"""
-    soc1_yamlfile = tmp_soc1_dir / 'soc.yml'
-    soc1_yamlfile.write_text(soc1_yaml)
-
-    tmp_board_root_dir = tmp_path / 'boards'
-    tmp_board_root_dir.mkdir()
-
-    tmp_vend1_dir = tmp_board_root_dir / 'zephyr'
-    tmp_vend1_dir.mkdir()
-
-    tmp_p1_dir = tmp_vend1_dir / 'p1'
-    tmp_p1_dir.mkdir()
-
-    p1e1_bs_yaml = """\
-boards:
-
-  - name: p1e1
-    vendor: zephyr
-    socs:
-      - name: unit_testing
-  - name: p1e2
-    vendor: zephyr
-    socs:
-      - name: unit_testing
-"""
-    p1e1_yamlfile = tmp_p1_dir / 'board.yml'
-    p1e1_yamlfile.write_text(p1e1_bs_yaml)
-
-    p1e1_yaml = """\
-identifier: p1e1
-name: Platform 1 Edition 1
-type: native
-arch: x86
-vendor: zephyr
-toolchain:
-  - zephyr
-twister: False
-"""
-    p1e1_yamlfile = tmp_p1_dir / 'p1e1.yaml'
-    p1e1_yamlfile.write_text(p1e1_yaml)
-
-    p1e2_yaml = """\
-identifier: p1e2
-name: Platform 1 Edition 2
-type: native
-arch: x86
-vendor: zephyr
-toolchain:
-  - zephyr
-"""
-    p1e2_yamlfile = tmp_p1_dir / 'p1e2.yaml'
-    p1e2_yamlfile.write_text(p1e2_yaml)
-
-    tmp_p2_dir = tmp_vend1_dir / 'p2'
-    tmp_p2_dir.mkdir()
-
-    p2_bs_yaml = """\
-boards:
-
-  - name: p2
-    vendor: zephyr
-    socs:
-      - name: unit_testing
-  - name: p2_2
-    vendor: zephyr
-    socs:
-      - name: unit_testing
-"""
-    p2_yamlfile = tmp_p2_dir / 'board.yml'
-    p2_yamlfile.write_text(p2_bs_yaml)
-
-    p2_yaml = """\
-identifier: p2/unit_testing
-name: Platform 2
-type: sim
-arch: x86
-vendor: vendor2
-toolchain:
-  - zephyr
-testing:
-  default: True
-"""
-    p2_yamlfile = tmp_p2_dir / 'p2.yaml'
-    p2_yamlfile.write_text(p2_yaml)
-
-
-    p2_2_yaml = """\
-testing:
-  ć#@%!#!#^#@%@:1.0
-identifier: p2_2
-name: Platform 2 2
-type: sim
-arch: x86
-vendor: vendor2
-toolchain:
-  - zephyr
-"""
-    p2_2_yamlfile = tmp_p2_dir / 'p2-2.yaml'
-    p2_2_yamlfile.write_text(p2_2_yaml)
-
-    tmp_vend2_dir = tmp_board_root_dir / 'arm'
-    tmp_vend2_dir.mkdir()
-
-    tmp_p3_dir = tmp_vend2_dir / 'p3'
-    tmp_p3_dir.mkdir()
-
-    p3_bs_yaml = """\
-boards:
-  - name: p3
-    vendor: zephyr
-    socs:
-      - name: unit_testing
-"""
-    p3_yamlfile = tmp_p3_dir / 'board.yml'
-    p3_yamlfile.write_text(p3_bs_yaml)
-
-    p3_yaml = """\
-identifier: p3
-name: Platform 3
-type: unit
-arch: arm
-vendor: vendor3
-toolchain:
-  - zephyr
-testing:
-  default: True
-"""
-    p3_yamlfile = tmp_p3_dir / 'p3.yaml'
-    p3_yamlfile.write_text(p3_yaml)
-
-    env = mock.Mock(board_roots=[tmp_board_root_dir],soc_roots=[tmp_path], arch_roots=[tmp_path])
 
     testplan = TestPlan(env=env)
 
@@ -1281,8 +1121,23 @@ testing:
         }
     }
 
+    def mock_gen_plat(board_roots, soc_roots, arch_roots):
+        assert [tmp_path] == board_roots
+        assert [tmp_path] == soc_roots
+        assert [tmp_path] == arch_roots
 
-    testplan.add_configurations()
+        platforms = [
+            mock.Mock(aliases=['p1e1/unit_testing', 'p1e1'], twister=False, default=False),
+            mock.Mock(aliases=['p1e2/unit_testing', 'p1e2'], twister=True, default=False),
+            mock.Mock(aliases=['p2/unit_testing', 'p2'], twister=True, default=True),
+            mock.Mock(aliases=['p3/unit_testing', 'p3'], twister=True, default=True),
+        ]
+        for platform in platforms:
+            type(platform).name = mock.PropertyMock(return_value=platform.aliases[0])
+            yield platform
+
+    with mock.patch('twisterlib.testplan.generate_platforms', mock_gen_plat):
+        testplan.add_configurations()
 
     if expected_defaults is not None:
         print(expected_defaults)