The documentation says DT.__init__ takes any iterable for the
include_path, but this leads to bad results when you pass it something
other than a 'real' sequence (list/tuple/etc), like a generator:
>>> dt = DT('/tmp/foo.dts', (x for x in ['a', 'b', 'c']))
>>> repr(dt)
"DT(filename='/tmp/foo.dts', include_path=<generator object ...>)"
Make a copy in list form just to avoid things like this.
Add a test for this and relax the regular expression in the existing
test case related to this.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Some of these are also tripping up a python 2 / python 3 warning
in mypy in the way that '{}'.format(b'foo') works, which we silence by
explicitly requesting the python 3 behavior.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
The way that _init_tokens() is manipulating globals() defeats static
analyses of the file that are trying to infer a type for the 'tok_id'
variable in assignment expressions like 'tok_id = _T_INCLUDE'.
To make it easier on the analyser, define the token types as an
enum.IntEnum named _T. This means we can write e.g. '_T.INCLUDE'
instead of '_T_INCLUDE', avoiding line length increases in the lexing
code.
While we're here, use '==' and '!=' instead of 'is' and 'is not'
when comparing a tok_id that is obtained from an re.Match.lastindex
with a _T.FOO value.
This is now necessary since an int object and a _T object definitely
don't point to the same memory. It worked previously because CPython
interns all integer instances from -5 to 256, but that's an
implementation detail and not a language feature. Since we're getting
the ints from an re.Match.lastindex instead of putting the exact
_T_FOO values into some list, this code probably should not strictly
speaking have been using 'is'.
Explicitly initialize the global _token_re also, to make it more
visible for static analysis.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Continue annotating the module. In some cases mypy will miss that
_err() calls means the function will not return, so we return an
unnecessary local variable to appease it.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Add a _MarkerType enum. A subsequent commit will use it for type
annotations in the Property class.
Fix an incorrect type in a comment while we're here.
We continue to use 'marker_type is _MarkerType.FOO' instead of
'marker_type == _MarkerType.FOO' because we are adding those actual
_MarkerType.FOO objects to each property, so 'is' comparison
is legitimate.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
A step along the way towards typing the whole module.
Fix an incorrect (or at best misleading) comment while we're here,
which was noticed by the type checker when I originally annotated
'props' as a Dict[str, bytes].
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Marking this NoReturn helps the type checker figure out that functions
which call it are only returning valid values or failing to
return. (It unfortunately doesn't always work as mypy's control flow
analysis seems to treat a direct 'raise DTError(...)' differently than
calling _err() in some situations, but it helps.)
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Move the DTError, Node, Type, and Property definitions to the top.
This way, class definitions occur before methods which use those
classes. This will be useful to avoid string literals in type
annotations that will be added later. Some can't be avoided due to
circular dependencies, but this will help.
Adjust whitespace.
No functional changes expected.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
This adds support to generate SPDX 2.2 tag-value documents via the
new west spdx command. The CMake file-based APIs are leveraged to
create relationships from source files to the corresponding
generated build files. SPDX-License-Identifier comments in source
files are scanned and filled into the SPDX documents.
Before `west build` is run, a specific file must be created in the
build directory so that the CMake API reply will run. This can be
done by running:
west spdx --init -d BUILD_DIR
After `west build` is run, SPDX generation is then activated by
calling `west spdx`; currently this requires passing the build
directory as a parameter again:
west spdx -d BUILD_DIR
This will generate three SPDX documents in `BUILD_DIR/spdx/`:
1) `app.spdx`: This contains the bill-of-materials for the
application source files used for the build.
2) `zephyr.spdx`: This contains the bill-of-materials for the
specific Zephyr source code files that are used for the build.
3) `build.spdx`: This contains the bill-of-materials for the built
output files.
Each file in the bill-of-materials is scanned, so that its hashes
(SHA256 and SHA1) can be recorded, along with any detected licenses
if an `SPDX-License-Identifier` appears in the file.
SPDX Relationships are created to indicate dependencies between
CMake build targets; build targets that are linked together; and
source files that are compiled to generate the built library files.
`west spdx` can be called with optional parameters for further
configuration:
* `-n PREFIX`: specifies a prefix for the Document Namespaces that
will be included in the generated SPDX documents. See SPDX spec 2.2
section 2.5 at
https://spdx.github.io/spdx-spec/2-document-creation-information/.
If -n is omitted, a default namespace will be generated according
to the default format described in section 2.5 using a random UUID.
* `-s SPDX_DIR`: specifies an alternate directory where the SPDX
documents should be written. If not specified, they will be saved
in `BUILD_DIR/spdx/`.
* `--analyze-includes`: in addition to recording the compiled
source code files (e.g. `.c`, `.S`) in the bills-of-materials, if
this flag is specified, `west spdx` will attempt to determine the
specific header files that are included for each `.c` file. This
will take longer, as it performs a dry run using the C compiler
for each `.c` file (using the same arguments that were passed to it
for the actual build).
* `--include-sdk`: if `--analyze-includes` is used, then adding
`--include-sdk` will create a fourth SPDX document, `sdk.spdx`,
which will list any header files included from the SDK.
Signed-off-by: Steve Winslow <steve@swinslow.net>
Since Pygments 2.7.0 devicetree syntax is supported, so there is no need
to use a custom lexer. Version 2.9.0 introduces a fix that is required
for some devicetree snippets.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
1. Add a class of Pytest(Harness) which can handle pytest script
in harness.py
2. Use running_dir to store current test directory which be
used in pytest scrpt.
3. Add usage of this feature into zephyr doc.
Signed-off-by: YouhuaX Zhu <youhuax.zhu@intel.com>
Support this when debugging also.
Tweak the style for brevity also while we're here by propertizing the
supports_nogui method, etc.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
This makes it easier for runners to check if the binary has thread
info support turned on, allowing automatic configuration of the
underlying tool to support threads, if possible.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Set options that are definitely true or false to True or False in the
options dict. Add a 'getboolean' method that also allows a fallback to
False in case the option is not mentioned in .config due to unmet
dependencies. This allows calling code to just ask about the option
they are interested in, even if the .config file doesn't mention the
option at all.
Propagate this to users within the runners package and 'west sign',
taking advantage of the new build_conf property.
Rename the 'bcfg' internal variable in sign.py to 'build_conf' to
match other source files that use BuildConfiguration instances, to
make it easier to grep for users.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Instead of mocking out the BuildConfiguration class, just create its
input file and let the real class do the work.
This in turn exposes a bug in the way the board name is being pulled
out of the BuildConfiguration, which we fix to keep the tests passing.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
This is really verbose, and I doubt anyone cares unless there is a
problem. Keep it around when run as 'west -v flash' to allow for
debugging, though.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Defer loading .config until we really need it, when we are flashing a
binary. Pre-emptively loading it is wasted effort if we're flashing a
.hex, which has been the default behavior when possible since
dcaabb860f ("west: runners: jlink: prefer .hex over .bin").
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Put it all in one log message rather than splitting it up.
This makes it look cleaner now that each log message is prefixed with
'runners.link:'.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
JLink versions like 'V7.0a' do not conform to PEP 440 version
conventions; the 'a' part is used by PEP 440 compliant versions for
alphas. It gets parsed to a legacy type by the packaging library,
which always is treated as a lower value when compared with a
conforming version string.
To fix, get the version from the shared library distributed with the
JLink tools. This has the side benefit of making the code work on
Windows. That's merely a nice to have for -nogui 1 detection for now,
but will be essential in the next commit.
Reported-by: Jake Mercer <jake.mercer@civica.co.uk>
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
This will be used to deal with the Segger shared library in a portable
way in the jlink runner.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Make it easier to get a BuildConfiguration from runner code.
Stash the result so it only has to be computed once.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
This option has existed since the beginning of the runners package,
which greatly predates the way DT is used in zephyr right now. It
never really worked the way I wanted it to but it's too much work to
fix it now. Try to improve the help a bit at least while I'm looking
at it again.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Copy a fix from test_nrfjprog.py to the other runner test suites. The
current code will enter an infinite recursion if you hit the path
where os.path.isfile is called, since it's been patched to
os_path_isfile_patch in the calling context. The fix is to cache the
'real' version in the parent scope and call it directly as a fallback.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Initialize `handle.device` to `None` so that the assert after the
matching has a chance to catch errors. Without this, a failed match will
raise an exception on the line above as it attempts to get a property
that doesn't exist.
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
This adds dictionary based logging support. Dictionary based
logging is binary based where one big difference is that
static strings are stored as pointers instead of the whole
string. This results in reduced space requirements for
storing log messages in certain scenairos.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Breathe project has released version 4.29.1, a fix release for the
issues found in 4.29.0. Relax version requirements by just skipping the
buggy version and staying to compatible releases.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
Due to the use of gperf to generate hash table for kobjects,
the addresses of these kobjects cannot change during the last
few phases of linking (especially between zephyr_prebuilt.elf
and zephyr.elf). Because of this, the gperf generated data
needs to be placed at the end of memory to avoid pushing symbols
around in memory. This prevents moving these generated blocks
to earlier sections, for example, pinned data section needed
for demand paging. So create placeholders for use in
intermediate linking to reserve space for these generated blocks.
Due to uncertainty on the size of these blocks, more space is
being reserved which could result in wasted space. Though, this
retains the use of hash table for faster lookup.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
There is no need to specify linker section for variables in
the output. The compiled object file from the produced
source file will be further processed to have them renamed
anyway.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
It's easier and safer to parse json file rather than extract
total size from stdout
Now the .json file has two keys:
{
"symbols": {
"children": [
...
],
"identifier": ":",
"name": "root",
"size": 2220
},
"total_size": 34272
}
Signed-off-by: Jingru Wang <jingru@synopsys.com>
Do calculation based on what is actually going to be run and evaluated
at runtime, ignore the cases we already know going to be skipped.
This fixes an issue where some sets would get majority of skips and
basically run nothing beside filtering.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
All skips for platforms listed in the .yaml's integration_platforms
will be treated as errors in the integration mode ('-G/--integration').
This feature serves to guarantee a proper testing scope in CI as no
integration platform will be skipped silently.
Fixes#33874
Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
It can be convenient to "iterate" over the elements of a property, in
the same way it is convenient to "iterate" over enabled instances.
Add a new macro for doing this, along with a DT_INST_FOREACH_PROP_ELEM
variant.
This is likely to be more convenient than UTIL_LISTIFY or FOR_EACH in
some situations because:
- it handles inputs of any length
- compiler error messages will be shorter and more self-contained
- it is easier to use with phandle-array type properties, which
require more complicated macro boilerplate when used with
util_macro.h APIs
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
We already have support for handling the Zephyr binding "path" type in
edtlib.Node._prop_val(), but the binding inference code isn't making
use of that. Handle this type as well, as it is just as convenient as
Type.PHANDLE and can be more idiomatic depending on the situation.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
