The standard library copy module allows you to implement shallow and
deep copies of objects. See its documentation for more details on
these terms.
Implementing copy.deepcopy() support for DT objects will allow us to
"clone" devicetree objects in other classes. This in turn will enable
new features, such as native system devicetree support, within the
python-devicetree.
It is also a pure feature extension which can't harm anything and is
therefore safe to merge now, even if system devicetree is never
adopted in Zephyr.
Note that we are making use of the move from OrderedDict to regular
dict to make this implementation more convenient.
See https://github.com/devicetree-org/lopper/ for more information on
system devicetree. We want to add system devicetree support to dtlib
because it seems to be a useful way to model modern, heterogeneous
SoCs than traditional devicetree, which can really only model a single
CPU "cluster" within such an SoC.
In order to create 'regular' devicetrees from a system devicetree, we
will want a programming interface that does the following:
1. parse the system devicetree
2. receive the desired transformations on it
3. perform the desired transformations to make
a 'regular' devicetree
Step 3 can be done as a destructive modification on an object-oriented
representation of a system devicetree, and that's the approach we will
take in python-devicetree. It will therefore be convenient to have an
efficient deepcopy implementation to be able to preserve the original
system devicetree and the derived regular devicetree in memory in the
same python process.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Regular dicts are insertion-ordered since CPython 3.6 and Python 3.7.
Zephyr now requires Python 3.8, so it should be OK to replace
OrderedDict with regular dict now. This results in less typing and
more readable object representations.
A nitpicker could argue that this is a functional change, since if a
user is doing 'assert isinstance(node.props, OrderedDict)', that will
fail now, but:
1. nobody is doing something like that in the zephyr tree
2. that would be a silly thing to do
3. we don't currently make any API stability guarantees
for this module right now anyway
so it should be fine.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Refactor the file parsing methods for readability by moving the
_parse_header() and _parse_memreserves() calls from _parse_dt() to
_parse_file(). The header and memreserves are not part of the 'tree'
part of the devicetree; now that we have a dedicated _parse_file()
helper, it makes more sense to me to have them there.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Holy overloaded technical terms, Batman.
Here, 'property' and 'type' each mean two different things, which
we can distinguish like this:
- Property (capital P): dtlib.Property class, represents
a property in a devicetree node
- @property: a Python property
- type(): an "@property" in the Property class, that returns
a dtlib.Type value
- Type (capital T): dtlib.Type class, represents the devicetree
type of a Property value (dtlib.Type.BYTES, etc.)
The type() @property in the Property class currently has an 'int' as
its Python return type annotation. It really returns a dtlib.Type,
which is an int (since it's an IntEnum), but that's not the same thing
as an int.
Change this to Type to be clear that not just any int can be returned
by this @property.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Make attribute initialization order match the order that attributes
appear within the class docstring. Move the 'type' property definition
up by the constructor to make it more obvious that this 'attribute' is
a (Python) property. This is for readability.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Reorder attribute initialization to match the order that attributes
appear in the class level docstring. This is for readability.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Initialize all the public API interface related attributes within the
constructor instead of scattering them throughout the implementation
of the class, and make sure they all have type annotations.
Move all the parsing code away from the init routines and public API
down to the main parsing block.
This is for readability and paves the way for later changes that
affect the way initialization happens.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Version is not enough, we need the run date but also the commit date, to
ease reporting and use in dashboards.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
In some scenarios we might end up with a null version value in the json
file, if the version can't be determined, set it to Unkown instead.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Some linker scripts also associate memory regions with program headers
(PHDRS [1]). This patch adds support on gen_relocate_app.py to also
place a phdr alongside the memory region.
To keep things simple (and more natural), the format is basically the
same used in linker scripts - an space followed by ':<phdr_name>', like:
SRAM2\ :phdr0:COPY:/home/xyz/zephyr/samples/hello_world/src/main.c
(Note the escape char before the space.)
[1] https://sourceware.org/binutils/docs/ld/PHDRS.html
Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
On push, upload test results to opensearch for analysis and reporting.
Goal is to use this data to understand test coverage better and use this
services for all test reporting, also for results coming from testing on
hardware.
Opensearch is currenly being used for evaluation, we are considering
the switch to elasticsearch later.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
When twister is set to retry any failures, count the number of retries
and record the number in the json file. This will help us identify
unstable tests or tests requiring attention.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Modify description frame to wrap the text so the window
doesn't have to be scrolled horizontally.
Signed-off-by: Filip Zajdel <filip.zajdel@nordicsemi.no>
Both CONFIG_BIG_ENDIAN and CONFIG_LITTLE_ENDIAN can be used in the code,
so reflect that in the error message in checkpatch.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
in some builds there is need to pass different value for -i parameter
and current code will override it to 3
Also in rimage i parameter is by default set to 3
Signed-off-by: Adrian Bonislawski <adrian.bonislawski@intel.com>
Update twister to support running QEMU platforms with sysbuild, by parsing
domains.yaml and executing the "run" target of the default application.
This will allow twister to test QEMU targets with sysbuild. It is assumed
that QEMU targets will add any external images they need in the build
phase
Signed-off-by: Daniel DeGrasse <daniel.degrasse@nxp.com>
Since twister can make use of domains.py as well, refactor west's use
this file so that domains.py can be moved to a generic library folder.
Signed-off-by: Daniel DeGrasse <daniel.degrasse@nxp.com>
When running twister -vv many blank lines would be logged.
Fix handing of empty lines from the reader thread. Real blank likes will
be b"\n" not b"".
Change open call to use with, and logger to use %s to fix pylint
warnings.
Signed-off-by: Jeremy Bettis <jbettis@google.com>
Currently, it is not possible to use a normal flash command to flash two
hex files that contain data in the UICR region, even though these
data don't conflict with each other.
However, that is not a technical limitation of the device, but of the
nrfjprog runner as it fails as soon as the hex file contained UICR data
and there was no --force/--erase/--recover used.
That approach was used in the past as program operation was used
without --verify, but as this option was added a while ago it is an
unnecessary limitation.
If the UICR wasn't erased before programming, the verify of the program
operation will now fail, so giving the user a warning in this case that
the hex files contain data in the UICR will still give the user a hint
of what might have caused the failing verify.
Additionally, it enables partial programming of the UICR without the
use of the --force option.
Signed-off-by: Markus Swarowsky <markus.swarowsky@nordicsemi.no>
Modify the regex used to merge overlays specified for
test samples in twister tests.
Overlays without any prefixes will be merged and passed to cmake.
Overlays with prefixes won't be merged but passed to cmake as they are.
Signed-off-by: Piotr Węgliński <piotr.weglinski@nordicsemi.no>
When there are more than 15 reviewers, the `set_assignee` script was
adding the reviewers more or less on a random basis because the input
set was arbitrarily ordered (thanks to how Python "set" works), and the
attempt to add any reviewers beyond the count of 15 results in the
previoulsy added reviewer being removed.
This commit updates the `set_assignee` script such that:
1. The collaborator list (input for generating the reviewer list) is
ordered by the area match, such that the collaborators of the most
relevant area come first.
2. The reviewers of the relevant areas are added first, until the total
reviewer count is 15. The script does not attempt to add more than
15 reviewers because that can result in the previously added
reviewers being removed from the list.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
The logic to drain the leftover logs of previous test should be
put before the monitor_serial(). It was misplaced when re-adding
this logic back.
Signed-off-by: Ming Shao <ming.shao@intel.com>
This sample is intended to show users how they can create a sample which
has dedicated configuration files for extra images included in a build.
MCUboot is used as example on how its default can be adjusted and
MCUboot itself automatically be included in the build when using
sysbuild.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
This patch introduces a West runner for flashing and debugging with
Lauterbach TRACE32 debuggers. The runner consists of a wrapper around
TRACE32 software, and allows a Zephyr board to execute a custom start-up
script (Practice Script) for the different commands supported, including
the ability to pass extra arguments from CMake. Is up to the board using
this runner to define the actions performed on each command.
The `debug` command launches TRACE32 GUI to allow debug the Zephyr
application, while the `flash` command hides the GUI and executes the
start-up script in a background process.
Signed-off-by: Manuel Arguelles <manuel.arguelles@nxp.com>
Calling Binding.__repr__() when the attribute Binding.path is None
would raise TypeError: expected str, bytes or os.PathLike object,
not NoneType.
Known bindings that may not have a path (Binding.path is None)
include bindings for properties such as 'compatible', 'reg', 'status'.
Signed-off-by: Chris Duf <chris@openmarl.org>
Co-authored-by: Gerard Marull-Paretas <gerard@teslabs.com>
Attempting to access the property Property.description
when Property.spec.description is None would raise
AttributeError: 'NoneType' object has no attribute 'strip'.
Known properties that may not have a description
(Property.spec.description is None):
- 'compatible' for nodes such as / /soc /soc/timer@e000e010 /leds /pwmleds
- 'reg' for nodes such as /soc/timer@e000e010
- 'status' for nodes such as /soc/timer@e000e010
- 'gpios' for nodes such as /leds/led_0 /buttons/button_0
- 'pwms' for nodes such as /pwmleds/pwm_led_0
This patch checks the PropertySpec.description attribute before calling
strip(): will return None, and not raise AttributeError.
Signed-off-by: Chris Duf <chris@openmarl.org>
Co-authored-by: Gerard Marull-Paretas <gerard@teslabs.com>
Add a script to extract HTTP status values and format them
in a way that is both human readable and machine parseable.
Each line of output is of the form:
```
HTTP_{key}_{upper_val} = {key}, /**< val */
```
Signed-off-by: Christopher Friedt <cfriedt@meta.com>
Attempting to access the Binding.description property
when the description is unavailable would raise KeyError: 'description'.
Known bindings that won't define a 'description' key in the
Binding.raw dictionary include the 'compatible' property's binding
of nodes such as /, /soc, /leds or /pwmleds.
Note that this may also occur when a proper YAML
binding file is available (e.g. pwmleds.yaml).
This patch simply substitutes the Binding.raw dictionary indexing
with the get() function: will return None and not raise KeyError.
Signed-off-by: Chris Duf <chris@openmarl.org>
Instead of hardcoding alignment size for pass 2 device handles, use
Z_DECL_ALIGN. This makes sure gen_handles.py is always in sync with the
type defined in device.h. The build assert in device.h can be removed as
a result, since we do not hardcode handles size anywhere else.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
In BinaryHandler we process input from simulator with decode('utf-8')
to convert it to string. decode has strict error handling by default -
so it raises UnicodeDecodeError exception if it can't decode input
binary sequence.
So if test start to print some junk to uart console we get
UnicodeDecodeError exception which cause the whole twister crash.
To fix that switch decode to less strict error handling when it
just replace undecoded binary sequence with unicode replacement
character.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
We need to give gdb slash-separated paths to ELF files on Windows.
Just make sure we always have slash-separated paths. We use this
technique in various places around the tree.
Fixes: #50789
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Instead of 'reset halt', default the command to reset and halt the target
to 'reset init'.
OpenOCD strictly requires 'reset init' to halt and prepare the device
before flash programming.
See https://openocd.org/doc/html/Flash-Commands.html chapter
"12.2 Preparing a Target before Flash Programming."
For targets implementing the init hook, it will allows to perform some
target specific init improvements (such as speed flash improvements).
Targets not implementing this hook will end up executing the usual and
previous 'halt'.
Signed-off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
Follow up to 5b5aa6ebba adding model name
and existence macros for all compatibles of a node that match an entry
in vendor prefixes.
Signed-off-by: Maureen Helm <maureen.helm@intel.com>
Although 8644b333cd added support for some
missing commands an options, the bash completion file was still in bad
need for some cleanup, fixes and additions.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
On windows, some board manufacturers have name that is "Microsoft"
instead of real its name. Need add it to manufacturer list.
Product information is missing. we can assign it "unknown" and
then add missing informations by hand.
Signed-off-by: Cong Nguyen Huu <cong.nguyenhuu@nxp.com>
Currently, support for Twister on Windows is only build.
The only thing that needs to be ported is that select()
cannot wait on file descriptors on Windows. Therefore
the serial monitor function needs to be reworked to
support both OSes.
Signed-off-by: Cong Nguyen Huu <cong.nguyenhuu@nxp.com>
Add support for building with sysbuild using twister, via the "sysbuild"
yaml property in testsuites. This will currently disable Kconfig and
devicetree filtering.
Signed-off-by: Daniel DeGrasse <daniel.degrasse@nxp.com>
The log refers to Platform taking precedence but the loop handles
Documentation as well. Move it under the proper section, drop "Drivers"
as that is not handle in any special way.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
The script right now assumes that all the areas have a maintainer and
that the second area in particular has one in case of Documentation,
that results in a ValueError when it's not the case. Handle that by
checking the lists before using them.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
Add a new set of helpers for expanding property entries with a
separator. These macros complement DT(_INST)FOREACH_PROP_ELEM(_VARGS) by
adding the capability to expand with a custom separator between property
entries. This allows, in some cases, to re-use existing macros (e.g.
DT_PROP_BY_IDX) without creating an auxiliary macro that just appends a
separator. Example:
```dts
n: node {
...
my-gpios = <&gpioa 0 GPIO_ACTIVE_HIGH>,
<&gpiob 1 GPIO_ACTIVE_HIGH>;
};
```
Before:
```c
#define GPIO_DT_SPEC_BY_IDX_AND_COMMA(node_id, prop, idx) \
GPIO_DT_SPEC_BY_IDX(node_id, prop, idx),
struct gpio_dt_spec specs[] = {
DT_FOREACH_PROP_ELEM(DT_NODELABEL(n), my_gpios,
GPIO_DT_SPEC_BY_IDX_AND_COMMA)
};
```
After:
```c
struct gpio_dt_spec specs[] = {
DT_FOREACH_PROP_ELEM_SEP(DT_NODELABEL(n), my_gpios,
GPIO_DT_SPEC_BY_IDX, (,))
};
```
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
Due to the diverse coding styles and lack of preprocessing when
scanning for test cases, there were many unintended combinations
of yaml test scenarios and C test functions with the regex-based
test case discovery, which caused an inaccurate test plan and test
result stats.
As the new ztest fx is used, the test cases of a test instance can
be accurately determined via zephyr.symbols file.
Because the zephyr.symbols file is not available until after build,
test cases determination is placed right after the build stage of
the pipeline and before the runtime test collection.
For those test instances that don't go through the build stage,
such as those with "Skip filtered" (statically filtered) reason
before entering the pipeline, they are not affected.
This patch also adjust the stats logic because the zephyr.symbols
file is generated after build. So ExecutionCounter update is split
and some must be postponed until test execution pipeline is completed.
Some concepts:
A test instance = a yaml scenario + a platform
"Test instance" and "test config" are synonyms in twister, unfortunately
excessive IMHO...
A test instance can be filtered or skipped in 3 ways.
Let's define "runtime" as "after entering the execution pipeline".
1) statically filtered (before runtime)
Such test instance is filtered by analyzing the yaml and never
enters the execution pipeline.
2) cmake filtered (runtime)
Such test instance enters pipeline and is filtered at cmake stage.
3) build skipped (also runtime)
Such test instance enters pipeline and is skipped at build stage.
All the test instances that enter the execution pipeline will go
through the report stage, where ExecutionCounter is further updated.
The meaning of the fields of ExecutionCounter are:
.skipped_configs = static filtered + cmake filtered + build skipped
.skipped_runtime = cmake filtered + build skipped
.skipped_filter = static filtered
.done = instances that enter the execution pipeline
.passed = instances that actually executed and passed
Definition of the overall stats:
total_complete = .done + .skipped_filter
total = yaml test scenario * applicable platforms
complete percentage = total_complete / total
pass rate = passed / (total - skipped_configs)
Signed-off-by: Ming Shao <ming.shao@intel.com>
This commit removes the stale reference to `k_mem_pool_area` for the
memory pool API, which was removed in the v2.5.0 release.
Signed-off-by: Stephanos Ioannidis <stephanos.ioannidis@nordicsemi.no>
