Switching the intermediate file format from JSON to YAML has a very
significant benefit: the file is now loaded via yaml_load(), which
internally calls Python to parse the file into the JSON format that
CMake expects. This means that the file contents are now automatically
escaped properly for JSON; it is a huge improvement over the previous
implementation, which was escaping everything given as input to
to_yaml().
With the removal of the now-redundant call in to_yaml(), escaping is
applied exactly once per value or list, when it is passed to yaml_set().
This allows to convert the logic in zephyr_string(ESCAPE ...) to a more
robust "escape everything" approach.
These changes fix the handling of strings with backslashes and different
types of quotes passed either directly or via generator expression. The
existing tests are updated to cover these cases.
Two other small changes are made in this commit:
- a small check in internal_yaml_list_append() is removed, as the same
issue is already detected by the caller yaml_set() logic.
- the to_yaml() function is modified to initialize the YAML output
variable at the top level, which is the expected behavior. This
resulted in genex temp files sometimes having duplicate lines.
Signed-off-by: Luca Burelli <l.burelli@arduino.cc>
Store strings in YAML as single-quoted entries to avoid issues with
special characters. This also fixes a quirk with the current test
suite where the quotes in the expected value are filtered out by the
YAML import.
Signed-off-by: Luca Burelli <l.burelli@arduino.cc>
Migrates mps2 targets away from the pinmux driver to
the new pinctrl driver.
Signed-off-by: Samuel Chee <samche01@arm.com>
Signed-off-by: Sudan Landge <sudan.landge@arm.com>
Rename the driver from uart_native_posix to uart_native_pty.
Including renaming the DTS compatible, and kconfig options, deprecating
the old ones.
And refactor the driver, generalizing it, so we can have any number of
instances.
Note that, unfortunately generalizing to N instances cannot be done
without a degree of backwards compatibility breakage: This driver was
born with all its configuration and selection of the instances based on
kconfig.
When the driver was made to use DT, it was done in a way that required
both DT and kconfig needing to manually coherently enable the 2nd UART.
This has now been fixed, which it means only DT is used to decide how
many instances are avaliable, and UART_NATIVE_POSIX_PORT_1_ENABLE is
just ignored.
Including:
* Deprecate UART_NATIVE_WAIT_PTS_READY_ENABLE: the options is always on
now as it has no practical drawbacks.
* Deprecate UART_NATIVE_POSIX_PORT_1_ENABLE: DTS intanciation defines it
being available now.
* Rename a few functions and in general shorten pseudo-tty/pseudo-
terminal to PTY instead of PTTY.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
Including renaming the DTS binding and kconfig options
deprecating the old one.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
Rename the driver files, binding and kconfig options, while deprecating
the old binding and kconfig options.
Uses in tree are replaced.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
The current implementation requires SoCs/Boards to manualy instantiate
the preripherals and initilize them.
The change lets Zephyr rely on the device tree setup to instantiate &
initialize the relevant gpio peripheral.
Signed-off-by: Wilfried Chauveau <wilfried.chauveau@arm.com>
This commit introduce support for maps in a yaml list.
The yaml_set() function has been extended with the following signature:
> yaml_set(NAME <name> KEY <key>...
> [APPEND] LIST MAP <map1> MAP <map2> MAP ...
> )
where a `MAP <map>` has the form:
`MAP "<key1>: <value1>, <key2>: <value2>, ...`
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Add tests to the YAML suite for generator expressions. The test checks
that generator expressions are handled as expected at different stages:
- immediately after yaml_save(), the file must contain only non-genex
entries;
- at a later time, after the target has been built, the file must
contain all the expanded genex values.
Signed-off-by: Luca Burelli <l.burelli@arduino.cc>
To verify proper functionality, the YAML module test suite is now run
twice: once in the regular CMake project mode and once in the CMake
script mode.
Signed-off-by: Luca Burelli <l.burelli@arduino.cc>
This introductory commit removes the ignored 'actual' variable from the
'yaml_set()' calls in the tests to make the code cleaner. Also, a badly
indented block is fixed in the 'message()' macro.
No functional change is introduced by this commit.
Signed-off-by: Luca Burelli <l.burelli@arduino.cc>
This reverts commit f24f5288f3.
as it's part of a series of commits causing issues on Windows
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
This commit introduce support for maps in a yaml list.
The yaml_set() function has been extended with the following signature:
> yaml_set(NAME <name> KEY <key>...
> [APPEND] LIST MAP <map1> MAP <map2> MAP ...
> )
where a `MAP <map>` has the form:
`MAP "<key1>: <value1>, <key2>: <value2>, ...`
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Prevents sysbuild from being used as this adds a local snippet root
which should only be used (and tested) by application
Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
Re-run the zephyr_get() testsuite in script mode after the project
mode testsuite has been executed. This is to ensure that the
zephyr_get() function works correctly in script mode as well.
Signed-off-by: Luca Burelli <l.burelli@arduino.cc>
Targets are not available in script mode.
To support the Zephyr scoping feature used by snippets and yaml module
then this commit moves from using custom targets to use GLOBAL
properties for scopes.
A scope property is prefixed with `<scope>:<property>` to avoid naming
collisions.
A `scope:<scope-name>` global property is used to track created scopes.
Tracking valid scopes ensure that properties are only set on known
scopes and thus catches typos / naming errors.
Add zephyr_scope_exists() and zephyr_get_scoped() to abstract the
implementation details of the scoped property retrieval and refactor
current code to use them.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Signed-off-by: Luca Burelli <l.burelli@arduino.cc>
Convert them to native YAML lists. Support for space-separated
lists was deprecated in Twister a long time ago.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
This commit adds new tests for verifying the functionality of the board
and SoC extension feature.
It does so by defining:
- A new CPU cluster on an existing SoC
- Two new board variants on top of an existing board
The new board variants are defined on top of the existing `native_sim`
board, so that the following native_sim board targets are available for
the test.
Existing board targets:
- native_sim/native
- native_sim/native/64
Extended board targets:
- native_sim/native/one
- native_sim/native/64/two
The new CPU cluster is defined for the existing `an521` SoC.
Existing CPU Clusters on an521:
- cpu0
- cpu1
New CPU Cluster:
- cputest
For SoC tests the mps2 board is used.
This means that for testing, the following board targets using the an521
SoC are:
- mps2/an521/cpu0
- mps2/an521/cpu1
- mps2/an521/cputest
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Fixes: #79052
The yaml CMake tests overwrites `message()` in order to verify correct
error handling. This was original done in a function and then using
a return call when an expected error has been raised.
Using a function() means the `return()` only returns the `message()`
and thus wrongly continue processing rest of the yaml file.
Change `message()` to macro so the return call correctly returns from
the calling yaml functions.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
native_posix is now deprecated.
Building these tests in both native_sim and native_posix does not
improve coverage for the tests or subsystem but doubles CI time.
As anyhow native_posix will be removed all together in
2 releases, let's remove it already for these.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
`checkpatch.pl` requires that dts sources are indented with tabs,
fix all the spaces that slipped in while checkpatch wasn't watching.
Signed-off-by: Jordan Yates <jordan@embeint.com>
Clear the output variable with an empty string, before appending to it.
Unsetting the variable locally is insufficient, because its value from
the parent scope or cache can still creep into the final result.
Signed-off-by: Grzegorz Swiderski <grzegorz.swiderski@nordicsemi.no>
According to the doc comment:
If using MERGE then SYSBUILD GLOBAL will get both the local and global
sysbuild scope variables (in that order, if both exist).
This stopped working in commit 8460d91e32,
when support for `zephyr_get(... VAR <multiple-variables>)` was added.
Instead of returning both values, the local sysbuild scope value would
clobber the global one. Fix this by splitting the internal `sysbuild`
scope into `sysbuild_local` and `sysbuild_global`, in that order.
Signed-off-by: Grzegorz Swiderski <grzegorz.swiderski@nordicsemi.no>
As `zephyr_get()` grows in complexity and significance, it must continue
working as expected. Unfortunately, it has a few minor bugs already, but
they will be taken care of in the next few commits. Before that, though,
let's go ahead and add a test suite for this function.
For now, some tests will pass by using incorrect values, annotated with
the `# BUG` comments; they will be removed in later patches. The purpose
of having them now is to make it easier to see what's about to be fixed.
Signed-off-by: Grzegorz Swiderski <grzegorz.swiderski@nordicsemi.no>
This commit reworks the snippet test to more comprehensively test the
different processing modes of snippets.
The following testcases have been added:
1. Initial state with no snippet applied
2. Applying `foo` snippet from the default application snippet root
3. Applying `bar` snippet from an extra snippet root
4. Sequentially applying `foo` and `bar` snippets to test the snippet
processing order
5. Sequentially applying `bar` and `foo` snippets to test the snippet
processing order.
Signed-off-by: Stephanos Ioannidis <stephanos.ioannidis@nordicsemi.no>
This commit adds change snippets to use the new EXTRA_DTC_OVERLAY_FILE
and EXTRA_CONF_FILE Zephyr build system variables instead of
DTC_OVERLAY_FILE and OVERLAY_CONFIG.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Align Zephyr modules with other user facing variables where settings
can be defined or extended, meaning Zephyr modules now supports:
ZEPHYR_MODULES and EXTRA_ZEPHYR_MODULES.
Support for ZEPHYR_EXTRA_MODULES is kept foir backward compatibility.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Twister now supports using YAML lists for all fields that were written
as space-separated lists. Used twister_to_list.py script. Some artifacts
on string length are due to how ruamel dumps content.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
Add an initial test suite for the basic SNIPPET_ROOT, SNIPPET, and
'append' features in snippet.yml.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
This is an implementation of issue #48334 and adds support for
specifying additional config and device tree overlays through fields in
the testcase.yaml file, which is more readable than having to cram these
in to `extra_args`.
Consider this example which shows the original and new ways to add
config and DT overlays:
```
common:
extra_args: "CONF_FILE=a.conf;b.conf
DTC_OVERLAY_FILE=w.overlay;x.overlay OVERLAY_CONFIG=e.conf
UNRELATED=abc"
tests:
utilities.base64:
extra_conf_files:
- "c.conf"
- "d.conf"
extra_overlay_confs:
- "extra_overlay.conf"
extra_dtc_overlay_files:
- "y.overlay"
- "z.overlay"
extra_configs:
- CONFIG_SAMPLE=y
tags: base64
type: unit
```
The new fields are `extra_conf_files`, `extra_overlay_confs,
`extra_dtc_overlay_files`. Files specified in these sections are appended
to any `CONF_FILE`, `OVERLAY_CONFIG`, or `DTC_OVERLAY_FILE` fields in
`extra_args`, causing the following args being passed in to
`self.run_cmake` at `runner.py:850`:
```
['-DUNRELATED=abc',
'-DCONF_FILE=a.conf;b.conf;c.conf;d.conf',
'-DDTC_OVERLAY_FILE=w.overlay;x.overlay;y.overlay;z.overlay',
'-DOVERLAY_CONFIG=e.conf extra_overlay.conf '
'<build_dir>/twister/testsuite_extra.conf']
```
These fields can be used in the common or scenario-specific YAML
sections and will be merged in order of least to most specific:
1. config files extracted from common's extra_args
2. files listed in common's {extra_conf_files or extra_overlay_confs
or extra_dtc_overlay_files}
3. config files extracted from test scenario's extra_args
4. files listed in test scenario's {extra_conf_files or
extra_overlay_confs or extra_dtc_overlay_files}
Specifying these files in extra_args now triggers a deprecation warning,
as the direct YAML fields are preferred for readability. They will still
function for now but support will be dropped in the future. One
testcase.yaml
(`zephyr/tests/cmake/overlays/var_expansions/testcase.yaml`) is
converted to use the new fields. A follow-up PR will convert the
remaining files to the new format.
Signed-off-by: Tristan Honscheid <honscheid@google.com>
Fix all line-length errors detected by yamllint:
yamllint -f parsable -c .yamllint $( find -regex '.*\.y[a]*ml' ) | \
grep '(line-length)'
Using a limit is set to 100 columns, not touching the commandlines in
GitHub workflows (at least for now).
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
Support referencing module directories by name in CONF_FILE,
OVERLAY_CONFIG, and DTC_OVERLAY_FILE so that projects can reference
overlay files in arbitrary modules.
Verified by passing all the following tests:
./scripts/twister -T tests/cmake/overlays/
Fixes#41830
Signed-off-by: Gregory Shue <gregory.shue@legrand.us>
With this commit a dedicated APPLICATION_CONFIG_DIR is added to the
Zephyr build system.
Currently, the APPLICATION_SOURCE_DIR is identical also the base
location of configuration files.
This is very practical for simple samples, but also has it limitations
for more complex setups.
Introducing a dedicated APPLICATION_CONFIG_DIR allows users more
customization options in Zephyr build system.
Especially in terms of custom build configuration files unknown to
Zephyr itself.
For example, instead of all configuration files being located directly
in the application source folder, a project might prefer to organize
their configuration files on a per board basis, for example:
<app>/boards/custom_board_A/prj.conf
<app>/boards/custom_board_A/app.overlay
<app>/boards/custom_board_A/custom_file.mine
<app>/boards/custom_board_B/prj.conf
<app>/boards/custom_board_B/app.overlay
<app>/boards/custom_board_B/custom_file.mine
...
instead of n-files located in the root of the sample.
If the user / sample specifies APPLICATION_CONFIG_DIR, then this folder
will always be used instead of the default configuration folder.
As part of this extension the behaviour of
`-DCONF_FILE=<relative-path>/prj_<build>.conf` and additional Kconfig
fragments automatic sourcing has been aligned with the default behavior
of `prj.conf`.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
