Commit graph

163 commits

Author SHA1 Message Date
Martí Bolívar
9d4ccf23ec dtlib: use IntEnum for token IDs
The way that _init_tokens() is manipulating globals() defeats static
analyses of the file that are trying to infer a type for the 'tok_id'
variable in assignment expressions like 'tok_id = _T_INCLUDE'.

To make it easier on the analyser, define the token types as an
enum.IntEnum named _T. This means we can write e.g. '_T.INCLUDE'
instead of '_T_INCLUDE', avoiding line length increases in the lexing
code.

While we're here, use '==' and '!=' instead of 'is' and 'is not'
when comparing a tok_id that is obtained from an re.Match.lastindex
with a _T.FOO value.

This is now necessary since an int object and a _T object definitely
don't point to the same memory. It worked previously because CPython
interns all integer instances from -5 to 256, but that's an
implementation detail and not a language feature. Since we're getting
the ints from an re.Match.lastindex instead of putting the exact
_T_FOO values into some list, this code probably should not strictly
speaking have been using 'is'.

Explicitly initialize the global _token_re also, to make it more
visible for static analysis.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-05-05 13:13:12 -05:00
Martí Bolívar
8427259ca2 dtlib: type annotate Property class
Continue annotating the module. In some cases mypy will miss that
_err() calls means the function will not return, so we return an
unnecessary local variable to appease it.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-05-05 13:13:12 -05:00
Martí Bolívar
a7e7964b14 dtlib: use IntEnum for marker types
Add a _MarkerType enum. A subsequent commit will use it for type
annotations in the Property class.

Fix an incorrect type in a comment while we're here.

We continue to use 'marker_type is _MarkerType.FOO' instead of
'marker_type == _MarkerType.FOO' because we are adding those actual
_MarkerType.FOO objects to each property, so 'is' comparison
is legitimate.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-05-05 13:13:12 -05:00
Martí Bolívar
f663521e4a dtlib: type annotate Node class
A step along the way towards typing the whole module.

Fix an incorrect (or at best misleading) comment while we're here,
which was noticed by the type checker when I originally annotated
'props' as a Dict[str, bytes].

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-05-05 13:13:12 -05:00
Martí Bolívar
a6ed7b262e dtlib: type annotate _err()
Marking this NoReturn helps the type checker figure out that functions
which call it are only returning valid values or failing to
return. (It unfortunately doesn't always work as mypy's control flow
analysis seems to treat a direct 'raise DTError(...)' differently than
calling _err() in some situations, but it helps.)

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-05-05 13:13:12 -05:00
Martí Bolívar
587b3248dc dtlib: code order and whitespace refactoring
Move the DTError, Node, Type, and Property definitions to the top.

This way, class definitions occur before methods which use those
classes. This will be useful to avoid string literals in type
annotations that will be added later. Some can't be avoided due to
circular dependencies, but this will help.

Adjust whitespace.

No functional changes expected.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-05-05 13:13:12 -05:00
Martí Bolívar
22fd259ce1 devicetree: fix stale comments
Update stale script comments from the package reorganization.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-05-05 13:13:12 -05:00
Martí Bolívar
62d8d1a4c6 edtlib: infer bindings for dtlib.Type.PATH
We already have support for handling the Zephyr binding "path" type in
edtlib.Node._prop_val(), but the binding inference code isn't making
use of that. Handle this type as well, as it is just as convenient as
Type.PHANDLE and can be more idiomatic depending on the situation.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-04-22 15:32:10 +02:00
Martí Bolívar
6c5254321c edtlib: improve inferred binding error message
Print the type which caused the error to make debugging easier.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-04-22 15:32:10 +02:00
Martí Bolívar
760d00913a edtlib: fix 'is not' -> '!='
We should not be comparing one int to another with 'is' or 'is not'.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-04-22 15:32:10 +02:00
Martí Bolívar
74f95688d9 dtlib: add Type enum
Instead of hard-coding constants, use an IntEnum.

These is still a subclass of 'int', but is both easier to import and
easier to read during debugging.

For example, compare:

>>> Type.BYTES
<Type.BYTES: 1>

with:

>>> TYPE_BYTES
1

However, 'Type.BYTES == 1' is still True, and the enum values
otherwise behave like you would expect.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-04-22 15:32:10 +02:00
Kumar Gala
6e01c6abb6 scripts: edt: Add support for include property filtering
Add the ability to filter which properties get imported when we do an
include.  We add a new YAML form for this:

include:
  - name: other.yaml
    property-blocklist:
      - prop-to-block

or

include:
  - name: other.yaml
    property-allowlist:
      - prop-to-allow

These lists can intermix simple file names with maps, like:

include:
  - foo.yaml
  - name: bar.yaml
    property-allowlist:
      - prop-to-allow

And you can filter from child bindings like this:

include:
  - name: bar.yaml
    child-binding:
      property-allowlist:
        - child-prop-to-allow

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-04-07 14:16:33 +02:00
Martí Bolívar
5332847644 dts: separate DT libraries from gen_defines.py
We are now in the process of extracting edtlib and dtlib into a
standalone source code library that we intend to share with other
projects.

Links related to the work making this standalone:

    https://pypi.org/project/devicetree/
    https://python-devicetree.readthedocs.io/en/latest/
    https://github.com/zephyrproject-rtos/python-devicetree

This standalone repo includes the same features as what we have in
Zephyr, but in its own 'devicetree' python package with PyPI
integration, etc.

To avoid making this a hard fork, move the code that's being made
standalone around in Zephyr into a new scripts/dts/python-devicetree
subdirectory, and handle the package and sys.path changes in the
various places in the tree that use it.

From now on, it will be possible to update the standalone repository
by just recursively copying scripts/dts/python-devicetree's contents
into it and committing the results.

This is an interim step; do NOT 'pip install devicetree' yet.
The code in the zephyr repository is still the canonical location.

(In the long term, people will get the devicetree package from PyPI
just like they do the 'yaml' package today, but that won't happen for
the foreseeable future.)

This commit is purely intended to avoid a hard fork for the standalone
code, and no functional changes besides the package structure and
location of the code itself are expected.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-04-02 08:28:12 -05:00