michaelh/zephyr
1
0
Fork 0
Code Releases Activity
zephyr/scripts/schemas/twister/quarantine-schema.yaml

44 lines
1,001 B
YAML
Raw Permalink Normal View History

twister: Add quarantine feature Adds feature allowing to use yaml file with dictionaries defining tests to be quarantined (extra arg "--quarantine-list FILENAME"). The dictionaries are validated according to the proper schema and loaded. A flat list is created containing quarantined configurations (configuration = platform + scenario). Configurations under quarantine are skipped and get "Quarantine" as a reason in the results reports. A "comment" can be added to a quarantine entry in the quarantine yaml with more details (e.g. issue #) and it will be also added to the report. The status of tests under quarantine can be verify if `--quarantine-verify` is used in addition to "--quarantine-list FILENAME". Using these args will make twister skip all tests which are not on the quarantine list. Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2021-03-11 13:18:33 +01:00
#
# Schema to validate a YAML file providing the list of configurations
# under quarantine
#
# We load this with pykwalify
# (http://pykwalify.readthedocs.io/en/unstable/validation-rules.html),
# a YAML structure validator, to validate the YAML files that provide
# a list of configurations (scenarios + platforms) under quarantine
#
type: seq
matching: all
sequence:
- type: map
yamllint: fix all yamllint truthy errors Fix all thruthy errors detected by yamllint: yamllint -f parsable -c .yamllint $( find -regex '.*\.y[a]*ml' ) | \ grep '(truthy)' This only accepts true/false for boolean properties. Seems like python takes all sort of formats: https://github.com/yaml/pyyaml/blob/master/lib/yaml/constructor.py#L224-L235 But the current specs only mention "true" or "false" https://yaml.org/spec/1.2.2/#10212-boolean Which is the standard yamllint config. Excluding codeconv and workflow files, as some are using yes/no instead in the respective documentation. Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2023-01-03 12:23:56 +01:00
required: true
twister: Add quarantine feature Adds feature allowing to use yaml file with dictionaries defining tests to be quarantined (extra arg "--quarantine-list FILENAME"). The dictionaries are validated according to the proper schema and loaded. A flat list is created containing quarantined configurations (configuration = platform + scenario). Configurations under quarantine are skipped and get "Quarantine" as a reason in the results reports. A "comment" can be added to a quarantine entry in the quarantine yaml with more details (e.g. issue #) and it will be also added to the report. The status of tests under quarantine can be verify if `--quarantine-verify` is used in addition to "--quarantine-list FILENAME". Using these args will make twister skip all tests which are not on the quarantine list. Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2021-03-11 13:18:33 +01:00
matching: all
mapping:
"scenarios":
type: seq
twister: Refactor and extend quarantine implementation in twister Implementation ported from TwisterV2. - quarantine handled by separate module - multiple yaml allowed from args: --quarantine-list - scenarios, platforms, architectures keywords in quarantine yaml are optional, if not given - means take it all Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
2022-11-22 16:28:29 +01:00
required: false
twister: Add quarantine feature Adds feature allowing to use yaml file with dictionaries defining tests to be quarantined (extra arg "--quarantine-list FILENAME"). The dictionaries are validated according to the proper schema and loaded. A flat list is created containing quarantined configurations (configuration = platform + scenario). Configurations under quarantine are skipped and get "Quarantine" as a reason in the results reports. A "comment" can be added to a quarantine entry in the quarantine yaml with more details (e.g. issue #) and it will be also added to the report. The status of tests under quarantine can be verify if `--quarantine-verify` is used in addition to "--quarantine-list FILENAME". Using these args will make twister skip all tests which are not on the quarantine list. Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2021-03-11 13:18:33 +01:00
sequence:
- type: str
yamllint: fix all yamllint truthy errors Fix all thruthy errors detected by yamllint: yamllint -f parsable -c .yamllint $( find -regex '.*\.y[a]*ml' ) | \ grep '(truthy)' This only accepts true/false for boolean properties. Seems like python takes all sort of formats: https://github.com/yaml/pyyaml/blob/master/lib/yaml/constructor.py#L224-L235 But the current specs only mention "true" or "false" https://yaml.org/spec/1.2.2/#10212-boolean Which is the standard yamllint config. Excluding codeconv and workflow files, as some are using yes/no instead in the respective documentation. Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2023-01-03 12:23:56 +01:00
- unique: true
twister: Add quarantine feature Adds feature allowing to use yaml file with dictionaries defining tests to be quarantined (extra arg "--quarantine-list FILENAME"). The dictionaries are validated according to the proper schema and loaded. A flat list is created containing quarantined configurations (configuration = platform + scenario). Configurations under quarantine are skipped and get "Quarantine" as a reason in the results reports. A "comment" can be added to a quarantine entry in the quarantine yaml with more details (e.g. issue #) and it will be also added to the report. The status of tests under quarantine can be verify if `--quarantine-verify` is used in addition to "--quarantine-list FILENAME". Using these args will make twister skip all tests which are not on the quarantine list. Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2021-03-11 13:18:33 +01:00
"platforms":
twister: Refactor and extend quarantine implementation in twister Implementation ported from TwisterV2. - quarantine handled by separate module - multiple yaml allowed from args: --quarantine-list - scenarios, platforms, architectures keywords in quarantine yaml are optional, if not given - means take it all Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
2022-11-22 16:28:29 +01:00
required: false
type: seq
sequence:
- type: str
twister: docs: Update twister docs with quarantine Docs updated with changes proposed in #52179 Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
2023-01-13 13:28:31 +01:00
- unique: true
twister: Refactor and extend quarantine implementation in twister Implementation ported from TwisterV2. - quarantine handled by separate module - multiple yaml allowed from args: --quarantine-list - scenarios, platforms, architectures keywords in quarantine yaml are optional, if not given - means take it all Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
2022-11-22 16:28:29 +01:00
"architectures":
required: false
twister: Add quarantine feature Adds feature allowing to use yaml file with dictionaries defining tests to be quarantined (extra arg "--quarantine-list FILENAME"). The dictionaries are validated according to the proper schema and loaded. A flat list is created containing quarantined configurations (configuration = platform + scenario). Configurations under quarantine are skipped and get "Quarantine" as a reason in the results reports. A "comment" can be added to a quarantine entry in the quarantine yaml with more details (e.g. issue #) and it will be also added to the report. The status of tests under quarantine can be verify if `--quarantine-verify` is used in addition to "--quarantine-list FILENAME". Using these args will make twister skip all tests which are not on the quarantine list. Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2021-03-11 13:18:33 +01:00
type: seq
sequence:
- type: str
twister: docs: Update twister docs with quarantine Docs updated with changes proposed in #52179 Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
2023-01-13 13:28:31 +01:00
- unique: true
twister: quarantine: Added simulations field and regex to filter fields Regex are allowed on every field, not only scenarios. Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
2022-12-10 19:47:58 +01:00
"simulations":
required: false
type: seq
sequence:
- type: str
twister: docs: Update twister docs with quarantine Docs updated with changes proposed in #52179 Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
2023-01-13 13:28:31 +01:00
- unique: true
twister: Add quarantine feature Adds feature allowing to use yaml file with dictionaries defining tests to be quarantined (extra arg "--quarantine-list FILENAME"). The dictionaries are validated according to the proper schema and loaded. A flat list is created containing quarantined configurations (configuration = platform + scenario). Configurations under quarantine are skipped and get "Quarantine" as a reason in the results reports. A "comment" can be added to a quarantine entry in the quarantine yaml with more details (e.g. issue #) and it will be also added to the report. The status of tests under quarantine can be verify if `--quarantine-verify` is used in addition to "--quarantine-list FILENAME". Using these args will make twister skip all tests which are not on the quarantine list. Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2021-03-11 13:18:33 +01:00
"comment":
type: str
required: false
Copy permalink