twister: implement simulation_exclude filter

This gives us the flexibility to exclude certain platforms'
simulator that's known to fail from running in Twister, but
allows real hardware to be tested.

Signed-off-by: Yong Cong Sin <ycsin@meta.com>
This commit is contained in:
Yong Cong Sin 2023-12-12 15:40:49 +08:00 committed by Anas Nashif
commit 9d314ff86d
4 changed files with 42 additions and 1 deletions

View file

@ -70,6 +70,7 @@ class TwisterConfigParser:
"platform_exclude": {"type": "set"}, "platform_exclude": {"type": "set"},
"platform_allow": {"type": "set"}, "platform_allow": {"type": "set"},
"platform_key": {"type": "list", "default": []}, "platform_key": {"type": "list", "default": []},
"simulation_exclude": {"type": "list", "default": []},
"toolchain_exclude": {"type": "set"}, "toolchain_exclude": {"type": "set"},
"toolchain_allow": {"type": "set"}, "toolchain_allow": {"type": "set"},
"filter": {"type": "str"}, "filter": {"type": "str"},

View file

@ -219,7 +219,8 @@ class TestInstance:
target_ready = bool(self.testsuite.type == "unit" or \ target_ready = bool(self.testsuite.type == "unit" or \
self.platform.type == "native" or \ self.platform.type == "native" or \
self.platform.simulation in SUPPORTED_SIMS or \ (self.platform.simulation in SUPPORTED_SIMS and \
self.platform.simulation not in self.testsuite.simulation_exclude) or \
filter == 'runnable') filter == 'runnable')
# check if test is runnable in pytest # check if test is runnable in pytest

View file

@ -157,6 +157,24 @@ mapping:
required: false required: false
sequence: sequence:
- type: str - type: str
"simulation_exclude":
type: seq
required: false
sequence:
- type: str
enum:
[
"qemu",
"simics",
"xt-sim",
"renode",
"nsim",
"mdb-nsim",
"tsim",
"armfvp",
"native",
"custom",
]
"tags": "tags":
type: any type: any
required: false required: false
@ -356,6 +374,24 @@ mapping:
matching: "all" matching: "all"
sequence: sequence:
- type: str - type: str
"simulation_exclude":
type: seq
required: false
sequence:
- type: str
enum:
[
"qemu",
"simics",
"xt-sim",
"renode",
"nsim",
"mdb-nsim",
"tsim",
"armfvp",
"native",
"custom",
]
"tags": "tags":
type: any type: any
required: false required: false

View file

@ -90,6 +90,9 @@ pairs:
platform_exclude: <list of platforms> platform_exclude: <list of platforms>
Set of platforms that this test case should not run on. Set of platforms that this test case should not run on.
simulation_exclude: <list of simulators>
Set of simulators that this test case should not run on.
extra_sections: <list of extra binary sections> extra_sections: <list of extra binary sections>
When computing sizes, twister will report errors if it finds When computing sizes, twister will report errors if it finds
extra, unexpected sections in the Zephyr binary unless they are named extra, unexpected sections in the Zephyr binary unless they are named