Add some helper macros that work similarly to the
'DT_FOREACH_OKAY_INST_<compat>(fn)' macros, except they give 'fn'
node identifiers in their expansion instead of instance numbers.
This makes it possible to add for-each APIs to devicetree.h that work
on an arbitrary compatible, not just DT_DRV_COMPAT.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Create a "global" gen_defines.py option and edtlib.EDT constructor
kwarg that turns edtlib-specific warnings into errors. This applies to
edtlib-specific warnings only. Warnings that are just dupes of dtc
warnings are not affected.
Use it from twister to increase DT testing coverage in upstream zephyr.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
An unknown vendor prefix is now a warning. We augment the list of
vendor prefixes passed by the user with a grandfathered-in bunch from
Linux.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
To be able to get a tokenize DT string without the quotes. Deprecate
also DT_ENUM_TOKEN and DT_ENUM_UPPER_TOKEN.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
As a first step towards being more forgiving on invalid inputs, allow
string-valued aliases properties that do not point to valid nodes when
the user requests permissiveness.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Modeled after dtc's --force option, the idea is this will try harder
and harder over time to produce an object despite malformed input.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
`_FOREACH_` macros do not allow the caller to pass additional arguments
to the `fn`. A series of `_VARGS` variants have been added that allow
the caller to pass arbitrary number of arguments to the `fn`:
```
DT_FOREACH_CHILD_VARGS
DT_FOREACH_CHILD_STATUS_OKAY_VARGS
DT_FOREACH_PROP_ELEM_VARGS
DT_INST_FOREACH_CHILD_VARGS
DT_INST_FOREACH_STATUS_OKAY_VARGS
DT_INST_FOREACH_PROP_ELEM_VARGS
```
Signed-off-by: Arvin Farahmand <arvinf@ip-logix.com>
I made an alignment error in a dts binding, but the build was
successful. After some debugging I found the following warning
explaining the problem:
'/home/casper/src/zephyrproject/zephyr/dts/bindings/gpio/
gpio-keys.yaml' appears in binding directories but isn't valid
YAML: while parsing a block mapping
in "<unicode string>", line 11, column 8
did not find expected key
in "<unicode string>", line 18, column 9
I think this should be an error as there shouldn't be any invalid yaml.
Signed-off-by: Casper Meijn <casper@meijn.net>
Recent versions of mypy have learned that the yaml module has type
stubs and the tool is now erroring out when it discovers we import
yaml since the stubs are not involved.
This is breaking CI on unrelated patches; fix it following the
instructions here:
https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Error out on compatible properties with invalid values. The regular
expression used to validate them matches what's used in dt-schema.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Extend the steps taken in tox.ini by type checking the 'devicetree'
package. This will make it easier for callers to type-check code that
uses the low level DT module's public APIs, since they can rely on
the type checker a bit more.
It will also help avoid bugs by adding some type checking for future
changes to this module.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Mypy is complaining about this line for some reason I didn't have time
to figure out. Just shut it up for now; I'll look into this when I get
around to type annotating edtlib.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Now that all the other code it depends on is annotated, we can finish
up the type annotation of this module in the main DT class.
It's not worth it to try to annotate the private methods (the ones
that begin with '_'). Most of these are low level lexing helpers that
aren't particularly amenable to static type checking, because the type
of a token's value is often dependent on the token ID in ways that
static type annotations are not well equipped to capture.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
We'd like users of this API to know that DT.root is always a Node,
and not an Optional[Node].
However, although DT.__init__ throws an exception if the resulting DT
object would have no root node, static analysis can't tell that since
the root instance attribute starts out as None during initialization,
so checkers like mypy are convinced it's Optional[Node].
Since this is really OK, we'll quiet the type checker down by stashing
the instance attribute in self._root instead, and providing a root
property accessor that is annotated to return Node instead of
Optional[Node]. We can tell mypy to ignore what looks like a potential
None here to allow callers to treat the result as a Node.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
The documentation says DT.__init__ takes any iterable for the
include_path, but this leads to bad results when you pass it something
other than a 'real' sequence (list/tuple/etc), like a generator:
>>> dt = DT('/tmp/foo.dts', (x for x in ['a', 'b', 'c']))
>>> repr(dt)
"DT(filename='/tmp/foo.dts', include_path=<generator object ...>)"
Make a copy in list form just to avoid things like this.
Add a test for this and relax the regular expression in the existing
test case related to this.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Some of these are also tripping up a python 2 / python 3 warning
in mypy in the way that '{}'.format(b'foo') works, which we silence by
explicitly requesting the python 3 behavior.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
The way that _init_tokens() is manipulating globals() defeats static
analyses of the file that are trying to infer a type for the 'tok_id'
variable in assignment expressions like 'tok_id = _T_INCLUDE'.
To make it easier on the analyser, define the token types as an
enum.IntEnum named _T. This means we can write e.g. '_T.INCLUDE'
instead of '_T_INCLUDE', avoiding line length increases in the lexing
code.
While we're here, use '==' and '!=' instead of 'is' and 'is not'
when comparing a tok_id that is obtained from an re.Match.lastindex
with a _T.FOO value.
This is now necessary since an int object and a _T object definitely
don't point to the same memory. It worked previously because CPython
interns all integer instances from -5 to 256, but that's an
implementation detail and not a language feature. Since we're getting
the ints from an re.Match.lastindex instead of putting the exact
_T_FOO values into some list, this code probably should not strictly
speaking have been using 'is'.
Explicitly initialize the global _token_re also, to make it more
visible for static analysis.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Continue annotating the module. In some cases mypy will miss that
_err() calls means the function will not return, so we return an
unnecessary local variable to appease it.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Add a _MarkerType enum. A subsequent commit will use it for type
annotations in the Property class.
Fix an incorrect type in a comment while we're here.
We continue to use 'marker_type is _MarkerType.FOO' instead of
'marker_type == _MarkerType.FOO' because we are adding those actual
_MarkerType.FOO objects to each property, so 'is' comparison
is legitimate.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
A step along the way towards typing the whole module.
Fix an incorrect (or at best misleading) comment while we're here,
which was noticed by the type checker when I originally annotated
'props' as a Dict[str, bytes].
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Marking this NoReturn helps the type checker figure out that functions
which call it are only returning valid values or failing to
return. (It unfortunately doesn't always work as mypy's control flow
analysis seems to treat a direct 'raise DTError(...)' differently than
calling _err() in some situations, but it helps.)
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Move the DTError, Node, Type, and Property definitions to the top.
This way, class definitions occur before methods which use those
classes. This will be useful to avoid string literals in type
annotations that will be added later. Some can't be avoided due to
circular dependencies, but this will help.
Adjust whitespace.
No functional changes expected.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
It can be convenient to "iterate" over the elements of a property, in
the same way it is convenient to "iterate" over enabled instances.
Add a new macro for doing this, along with a DT_INST_FOREACH_PROP_ELEM
variant.
This is likely to be more convenient than UTIL_LISTIFY or FOR_EACH in
some situations because:
- it handles inputs of any length
- compiler error messages will be shorter and more self-contained
- it is easier to use with phandle-array type properties, which
require more complicated macro boilerplate when used with
util_macro.h APIs
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
We already have support for handling the Zephyr binding "path" type in
edtlib.Node._prop_val(), but the binding inference code isn't making
use of that. Handle this type as well, as it is just as convenient as
Type.PHANDLE and can be more idiomatic depending on the situation.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Instead of hard-coding constants, use an IntEnum.
These is still a subclass of 'int', but is both easier to import and
easier to read during debugging.
For example, compare:
>>> Type.BYTES
<Type.BYTES: 1>
with:
>>> TYPE_BYTES
1
However, 'Type.BYTES == 1' is still True, and the enum values
otherwise behave like you would expect.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Add the ability to filter which properties get imported when we do an
include. We add a new YAML form for this:
include:
- name: other.yaml
property-blocklist:
- prop-to-block
or
include:
- name: other.yaml
property-allowlist:
- prop-to-allow
These lists can intermix simple file names with maps, like:
include:
- foo.yaml
- name: bar.yaml
property-allowlist:
- prop-to-allow
And you can filter from child bindings like this:
include:
- name: bar.yaml
child-binding:
property-allowlist:
- child-prop-to-allow
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
We are now in the process of extracting edtlib and dtlib into a
standalone source code library that we intend to share with other
projects.
Links related to the work making this standalone:
https://pypi.org/project/devicetree/https://python-devicetree.readthedocs.io/en/latest/https://github.com/zephyrproject-rtos/python-devicetree
This standalone repo includes the same features as what we have in
Zephyr, but in its own 'devicetree' python package with PyPI
integration, etc.
To avoid making this a hard fork, move the code that's being made
standalone around in Zephyr into a new scripts/dts/python-devicetree
subdirectory, and handle the package and sys.path changes in the
various places in the tree that use it.
From now on, it will be possible to update the standalone repository
by just recursively copying scripts/dts/python-devicetree's contents
into it and committing the results.
This is an interim step; do NOT 'pip install devicetree' yet.
The code in the zephyr repository is still the canonical location.
(In the long term, people will get the devicetree package from PyPI
just like they do the 'yaml' package today, but that won't happen for
the foreseeable future.)
This commit is purely intended to avoid a hard fork for the standalone
code, and no functional changes besides the package structure and
location of the code itself are expected.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Whenever a child-binding: dict has a compatible, we ought to make
that available in EDT._compat2binding. If we don't, we can't look it
up later.
This has adverse effects like missing child bindings which describe
buses.
Fixes: #32071
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Commit 0d4dca10b2 ("scripts: edtlib:
child binding compatibles match parents") was a hack meant to keep the
edtlib.Binding class in place without modifying some twister behavior
that needed further changes to work properly with first-class binding
objects.
This is a hack and is no longer necessary, so back out of this change.
Child Binding objects now have None compatible properties unless the
binding YAML explicitly sets a compatible.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Allow users to set the new EDT constructor argument which errors out
on deprecated properties via a command line argument.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
We'd like to start eliminating deprecated properties from upstream
Zephyr devicetrees. To make that possible in the build system, add an
EDT kwarg that does just that.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Commit 6bf761fc0a ("dts: Remove support
for deprecated DTS binding syntax") removed most of the support for
the 'legacy' bindings syntax.
A few straggler keys are still around in the bindings check code,
though. This allows some legacy keys which should cause errors to pass
silently instead.
Fix the error handling and print good errors for cases which are
removed, just in case someone is still using them somewhere.
Clean up some other error messages in the same function while we're
here.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
This macro returns a node's name with unit-adddress, given its node
identifier.
The node name is useful information for the user to utilize for debug
information, similar to DT_NODE_PATH, or DT_LABEL.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Change the use of a f-string to a normal string as there is nothing
that needs formatting in the particular instance.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
While using the encoded path to a device tree node guarantees a unique
identifier for the corresponding device there is a limit on the number
of characters of that name that can be captured when looking up a
device by name from user mode, and the path can exceed that limit.
Synthesize a unique name from the node dependency ordinal instead, and
update the gen_defines script to record the name associated with the
full path in the extern declaration.
Add a build-time check that no device is created with a name that
violates the user mode requirement.
Also update the network device DTS helper functions to use the same
inference for dev_name and label that the real one does, since they
bypass the real one.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
Before we had a bindings index in the documentation, the generated
header file was (somewhat unfortunately) often our best reference for
what a particular binding or property within a binding ends up doing,
so it made good sense to put the description in the generated file.
Now that we have HTML documentation that's a bit more digestible than
the generated file, though, we can just point users at that. Do that
and remove the inline description from the generated file.
This makes it possible to put C-style multiline comments in the
descriptions themselves, which will be done in subsequent patches.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
This macro returns a node's full path, given its node identifier.
The entire path to a node is useful information for the user which can
be added to build-time error messages.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Generate a header (device_extern.h) that handles extern of possible
device structs that would come from devicetree. This removes the need
for DEVICE_DT_DECLARE and DEVICE_DT_INST_DECLARE which we can remove.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
There are some drivers in the tree that support devices on multiple
different buses, although so far this has not been represented in
device tree using the bus concept. In order to convert these drivers &
bindings to refer to a formal bus in device tree we need to be able to
match bindings which lack an explicit "on-bus: ..." value against any
parent bus. This will also be needed for any external bindings, since
those would not be aware of on-bus (as it's a Zephyhr-specific
extension).
The two drivers I'm particularly targeting is the ns16550 UART driver
(drivers/serial/uart_ns16550.c) and the DW I2C driver
(drivers/i2c/i2c_dw.c). They both support devices with a fixed MMIO
address as well as devices connected and discovered over PCIe. The
only issue is that instead of encoding the bus information the proper
DT way these bindings use a special "pcie" property in the DT node
entries to indicate whether the node is on the PCIe bus or not.
Being able to convert the above two drivers to use the DT bus concept
allow the removal of "hacks" like this:
if DT_INST_PROP(0, pcie) || \
DT_INST_PROP(1, pcie) || \
DT_INST_PROP(2, pcie) || \
DT_INST_PROP(3, pcie)
to the more intuitive:
if DT_ANY_INST_ON_BUS_STATUS_OKAY(pcie)
This also has the benefit that the driver doesn't need to make any
arbitrary assumptions of how many matching devices there may be but
works for any number of matches. This is already a problem now since
e.g. the ns16550 driver assumes a maximum of 4 nodes, whereas
dts/x86/elkhart_lake.dtsi defines up to 9 different ns16550 nodes.
Signed-off-by: Johan Hedberg <johan.hedberg@intel.com>
The DTS language permits zeroing out phandles in a phandle array to
say "there's nothing at this index", and dtlib manages that correctly,
but edtlib and gen_defines.py aren't equipped to do so.
Fix this by allowing None elements in the lists of ControllerAndData
values returned by edtlib for such properties.
Handle that in gen_defines.py by setting the generated
DT_N_<node>_P_<prop>_IDX_<i>_EXISTS macro to 0 in such cases.
The DT_N_<node>_P_<prop>_LEN macro still accounts for the entire
length of the phandle-array; it's just that some indexes may be
missing data.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
These are likely enough defined by mistake to emit a warning for.
Adjust tests to match, tweaking the test_warnings() setup: now that
we've got several test cases, it's a bit cleaner not to have to
copy/paste the ('edtlib', WARNING, ...) part of every expected log
record tuple.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
The edtlib strategy for emitting warnings is to print directly to
standard error. This in turn requires hacks to drop stored references
to stderr in various _warn_file attributes so the EDT objects can be
pickled.
In general, I think it's not really appropriate for library modules
like edtlib to be printing to stderr directly. The user should be able
to configure logging for general utility data munging modules like
this as they please, and not just deciding what file to print to.
Move this around so the standard logging module is used instead. We
can preserve backwards compatibility in gen_defines by customizing the
'edtlib' logging module behavior so it prints the exact same thing it
always has.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
