This makes the results easier to read, since node output appears in
the same order as the top level comment summarizing the tree.
Reorganize how flash partitions are emitted so they also appear in
dependency order, allowing us to extend the comment at the top of the
file to say that nodes are emitted in that order, along with other
useful information.
Print comments even for nodes that are disabled or have no binding,
to make it clearer that the script considered them and decided not to
output anything for them.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Add a script that sets up Docker networking and starts the net-tools
Docker container. If successful, run Zephyr with native_posix and
execute the appropriate net-tools container executable. The proper
net-tools executable and arguments is selected depending on the
basename of the sample directory. This script needs to be updated
if the net-tools Docker image is updated in some incompatible way.
The net-tools directory is assumed to exist at the same level as
the Zephyr base directory, but its location can be set using the
'-N' command line argument. Likewise, '-Z' sets the Zephyr top level
directory.
When stopping Zephyr, go through all child processes since running a
native posix or other variant of Zephyr may cause a hierarchy of
processes to be created.
Fixes: #19540
Signed-off-by: Patrik Flykt <patrik.flykt@intel.com>
Overlay files were concatenated into single string with no whitespace
between them. If '--extra-args OVERLAY_CONFIG="overlay-1.conf"
--extra-args OVERLAY_CONFIG="overlay-2.conf"' was used, then the result
was OVERLAY_CONFIG="overlay-1.confoverlay-2.conf".
Another thing was that overlay extra args were not properly removed from
the list of regular arguments. As a result we had incorrect list of
overlays and incorrect list of other arguments.
Rework code to extract overlays in loop in a safe manner. Use for that a
list of strings instead of string directly. Join those strings and form
a single OVERLAY_CONFIG argument just before running cmake.
Tested with following testcase.yaml line:
extra_args: ARG1 OVERLAY_CONFIG="overlay-1.conf"
ARG2 OVERLAY_CONFIG="overlay-2.conf"
Before this patch we got:
args = ['OVERLAY_CONFIG="overlay-1.conf"',
'OVERLAY_CONFIG="overlay-2.conf"']
After this patch we get:
args = ['ARG1', 'ARG2',
'OVERLAY_CONFIG="overlay-1.conf overlay-2.conf"']
While at it, fix also regex pattern by removing requirement of double
quotes around value. Match any option value instead and strip both
single and double quotes when match is positive. Tested with:
$ ./scripts/sanitycheck -T samples/hello_world/ -p qemu_x86 \
--extra-args OVERLAY_CONFIG=overlay1.conf '
--extra-args OVERLAY_CONFIG=\"overlay2.conf\" '
--extra-args OVERLAY_CONFIG=\'overlay3.conf\'
Signed-off-by: Marcin Niestroj <m.niestroj@grinn-global.com>
Change the output during CMake configure from
Devicetree configuration written to .../devicetree.conf
Parsing /home/ulf/z/z/Kconfig
Loaded configuration '.../frdm_kw41z_defconfig'
Merged configuration '.../prj.conf'
Configuration saved to '.../.config'
to
Devicetree header saved to '.../devicetree_unfixed.h'
Parsing /home/ulf/z/z/Kconfig
Loaded configuration '.../frdm_kw41z_defconfig'
Merged configuration '.../prj.conf'
Configuration saved to '.../.config'
Kconfig header saved to '.../autoconf.h'
devicetree_unfixed.h is more useful to be aware of than devicetree.conf
(still hoping we can get rid of it at some point), and seeing the
Kconfig header clarifies things to.
"Saved" instead of "written" for consistency. Maybe it could say
"Kconfig configuration" instead of "configuration" too, though it gets a
bit spammy in other contexts where the message shows up.
Updates Kconfiglib (and menuconfig/guiconfig, just to sync) to upstream
revision 061e71f7d7, to get this commit in, which is used to print the
header path:
Return a message from Kconfig.write_autoconf()
Like for Kconfig.write_config() and Kconfig.write_min_config(),
return a string from Kconfig.write_autoconf() with a message saying
that the header got saved, or that there were no changes to it. Can
be handy in tools.
Also make the "no change" message for the various files more
specific, by mentioning what type of file it is (configuration,
header, etc.)
Return True/False from Kconfig._write_if_changed() to indicate if
the file was updated. This also allows it to be reused in
Kconfig.write_min_config().
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Add error checking in zephyr_module.py so that if the user manually
specifies ZEPHYR_EXTRA_MODULES and the list contains something that
isn't in fact a valid module, we scream and die.
This should help with diagnosing module errors.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Add a new --check-missing-config-prefix check that looks for references
like
#if MACRO
#if(n)def MACRO
defined(MACRO)
IS_ENABLED(MACRO)
where MACRO is the name of a defined Kconfig symbol but doesn't have a
CONFIG_ prefix. Could be a typo.
Skip MACRO if it is #define'd somewhere, even if it looks like a Kconfig
symbol.
Found e.g. https://github.com/zephyrproject-rtos/zephyr/pull/22195.
Piggyback skipping binary files when grepping for Kconfig symbol
references.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
So that the EDTT tool could be used by this script,
set its path so other scripts could find it
Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
Only dump data when we are interested in the analysing coverage. By
default just collect the data.
CONFIG_COVERAGE_DUMP is used to control this behaviour.
This will help speed up sanitycheck and will avoid lots of noise in the
log when some tests with coverage enabled failed. Dumping data to
console is also suspected to be one of the reason why qemu hangs in CI.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
If 'imgtool' is not found in PATH look for 'imgtool.py' before bailing
out.
This allows adding the mcuboot/scripts directory to PATH and have
'west sign -t imgtool' automatically locate the 'imgtool.py' script.
Signed-off-by: Henrik Brix Andersen <hebad@vestas.com>
Add a '--dts-out <file>' flag to gen_defines.py that saves the final
devicetree to <file>, in DTS format, using the new EDT.dts_source
attribute.
Handy to have available as a debugging aid. Unused otherwise.
Also write a dummy <BOARD>.dts_compiled file that tells people to look
at zephyr.dts, for people that might be used to that file. It was
removed in commit 8d8b06978c ("dts: Remove generation of
<BOARD>.dts_compiled").
Fixes: #22272
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Include the "warning: "/"error: " part of the string when wrapping
lines, and consistenly start messages with a capital letter.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Turn the warning for selecting a symbol with unsatisfied dependencies
into an error. The last instance has been fixed (that triggers in CI at
least).
Also remove the warning whitelisting functionality, which was only used
to whitelist that warning. It hasn't been used for anything else in over
a year, so it probably wouldn't be useful to keep. Getting rid of it
makes the script easier to read.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Add an EDT.compat2enabled attribute that maps compatibles to enabled
devicetree nodes that implement them. For example,
EDT.compat2enabled["foo"] is a list of all enabled nodes with "foo" in
the 'compatible' property.
The old Node.instance_no functionality can be implemented in terms of
EDT.compat2enabled, so remove Node.instance_no. EDT.compat2enabled is
more flexible and easier to understand.
Simplify main() in gen_defines.py by using EDT.compat2enabled to
generate the DT_COMPAT_<compatible> existence macros. The behavior is
slightly different now, as DT_COMPAT_<compatible> is generated for
enabled nodes that don't have a binding as well, but that might be an
improvement overall. It probably doesn't hurt at least.
EDT.compat2enabled simplifies the implementation of the new
$(dt_compat_get_str) preprocessor function in
https://github.com/zephyrproject-rtos/zephyr/pull/21560. That was the
original motivation.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
This fixes a problem which appeared after bossac version was downgraded
to 1.7 which no longer accepts the -o/--offset parameter. Now the offset
is fed to bossac executable only if it's explicitly provided and not by
default.
Signed-off-by: Kuba Sanak <contact@kuba.fyi>
'type: path' was added to edtlib for completeness in commit 23a5b4963b
("dts: edtlib: Add 'type: path' for path references"). gen_defines.py
crashes if it's ever used though, because it gets confused for a
'type: phandle-array'.
Ignore 'type: path' properties in gen_defines.py, like how
'type: phandle' and 'type: phandles' are currently ignored too.
(Note that gen_defines.py is only one possible user of edtlib.)
Fixes: #22197
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
The terminal configuration should not be reset on exit if
sanitycheck's output is being directed to something that mediates the
terminal control, such as a pager. Only do the reset if stdout is
interactive.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
Use f-strings instead of .format() to makes all the macro identifier and
multiline comment building easier to read.
f-strings were added in Python 3.6, which is required by Zephyr now.
Convert some
... + "_" + ...
string building to f-strings as well.
f-strings are a bit faster too (because they avoid format()/str()
lookups), though it's not likely to be noticeable here.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Use f-strings instead of .format() to make things more readable. They
were added in Python 3.6, which Zephyr requires now.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Show which dependencies are unsatisfied when symbols don't get their
assigned value.
For example, assume that FOO below is assigned with CONFIG_FOO=y. Note
that BAR, BAZ, and STR = "hmm" are 'n'.
config FOO
bool "foo"
depends on BAR && BAZ && QAZ && STR = "hmm"
config BAR
def_bool n
config BAZ
def_bool n
config QAZ
def_bool y
config STR
def_string "zmh"
This now prints this warning:
warning: FOO (defined at /home/ulf/z/z/Kconfig:10) was assigned the
value 'y' but got the value 'n'. Check these unsatisfied
dependencies: BAR (=n), BAZ (=n), STR = "hmm" (=n). See ...
Fixes: #21888
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Sort by testcase name and path and not by platforms. This way we do not
have the same platform executing on CI system causing failures to
overload.
Right now we have almost all qemu targets executing on 1 or 2 nodes,
causing those systems to be overloaded. Instead of taking the default
sorting by platforms, we now sort by testcases, so the distribution is
more "random", but still reproducible.
Remove the ordering function that would put native_posix in the first
set, it is not being used anymore.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
In some cases the config is already provided for us e.g. when setting
OPENOCD_NRF5_SUBFAMILY. Also it looks like config was always supposed to
be optional (there are checks for it on do_run()).
Signed-off-by: Rihards Skuja <rihardssk@mikrotik.com>
Outer if is checking if the variable is in a list with two options
which means that we can just do an if / else and remove an assignment.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
There are some issues with the behavior when rerunning CMake in an
already initialized build directory:
1. The check for assignments to promptless symbols in configuration
fragments isn't run when reconfiguring, because it only runs if
zephyr/.config doesn't exist
2. As outlined in
https://github.com/zephyrproject-rtos/zephyr/issues/9573, you can
get into situations where zephyr/.config is invalid (e.g. due to
being outdated), but menuconfig/guiconfig can't be run to fix it
3. If kconfig.py fails while merging fragments during reconfiguration,
it will ignore the fragments during the next reconfiguration and use
the existing zephyr/.config instead, because the fragment checksum
is calculated and saved before running kconfig.py
(Footnote: The input configuration file(s) to kconfig.py can be either a
list of configuration fragments, when merging fragments, or just
zephyr/.config, if the merged configuration is up-to-date. The output
configuration file is always zephyr/.config.)
To fix the first two issues, explicitly tell kconfig.py when it's
dealing with handwritten configuration input (fragments), via a new
--handwritten-input-configs flag. This is more robust than checking
whether zephyr/.config exists, which was the old logic.
When dealing with handwritten input, there should be no assignments to
promptless symbols. Assignments to promptless symbols is expected in
zephyr/.config however, because it doubles as configuration output.
When running menuconfig/guiconfig, the input configuration is
zephyr/.config rather than configuration fragments, so this change also
makes sure that menuconfig can always be run as long as zephyr/.config
exists and is up-to-date.
To fix the last issue, only write the checksum for the configuration
fragments if kconfig.py succeeds (which means it wrote a
zephyr/.config).
Also improve naming a bit, add help texts for the command-line
parameters to kconfig.py, and simplify write_kconfig_filenames() by
moving logic into it.
Partial fix for
https://github.com/zephyrproject-rtos/zephyr/issues/9573, without the
part in #issuecomment-469701831. Can still run into issues when e.g.
when CMake files can't make sense of settings.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Timeouts were not reported correctly in the XML file as failures,
causing some confusion in the shippable results.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
It is often needed to run sanitycheck and exclude a specific platform or
multiple platforms. Add an option to provide this using
--exclude-platform. The option can be used multiple times to exclude
multiple platforms at the same time.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Assigning to promptless symbols has no effect.
Previously, the only check was for whether the value assigned to a
symbol matched its final value. This misses cases where a promptless
symbol is assigned to and just happens to get the assigned-to value as
its final value.
Instead, detect whether configuration files are being merged (by
checking if zephyr/.config already exists), and explicitly check for
assignments to promptless symbols in that case. We can't do it when
zephyr/.config already exists (and is being loaded), because it includes
values for promptless symbols as well.
With the no-prompt check moved out, also use a more specific message for
it, and remove stuff related to prompts elsewhere. Shorten messages a
bit at the same time, and add two warn() and err() helpers.
Fixes: #20697
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
generated_dts_board.h is pretty redundant and confusing as a name. Call
it devicetree.h instead.
dts.h would be another option, but DTS stands for "devicetree source"
and is the source code format, so it's a bit confusing too.
The replacement was done by grepping for 'generated_dts_board' and
'GENERATED_DTS_BOARD'.
Two build diagram and input-output SVG files were updated as well, along
with misc. documentation.
hal_ti, mcuboot, and ci-tools updates are included too, in the west.yml
update.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
With timestamps enabled, the log in shippable is very difficult to read,
so remove those and enable them when needed for debugging only.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Using --runtime-artifact-cleanup, the script will now remove all
artifacts of passing tests. This is useful for running sanitycheck on
systems with little disk space.
Fixes#21922
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
out_dev(), etc., as a name is a holdover from before commit 73ac1466fb
("scripts: edtlib: Call nodes "nodes" instead of "devices""). It's
confusing for the same reasons as mentioned in that commit.
Renamings:
- dev_ident() -> node_ident()
- out_dev() -> out_node()
- out_dev_s() -> out_node_s()
- dev_aliases() -> node_aliases()
- dev_path_aliases() -> node_path_aliases()
- dev_instance_aliases() -> node_instance_aliases()
Also clean up comments to talk about nodes instead of devices where it
makes sense.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
This guarantees muscle memory will work over multiple runs when the
same boards are connected. (The "nrfjprog --help" output for --ids
does not guarantee an order.)
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Right now, the nrfjprog runner will prompt the user for which board to
use if there are multiple possibilities and the --snr command line was
not given to specify one ahead of time.
Tweak this so it only happens if standard input is connected to a
terminal. This should avoid stalling the process on board farms when
this runner is used in automation.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
The create() classmethod should not be doing any I/O -- its only job
is to create the ZephyrBinaryRunner instance. It's currently trying to
figure out what the serial number of the board is, though. Let's defer
that work to do_run(), so it gets handled by run_common's exception
handler.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Catch RuntimeError when calling runner.run() and print a message
instead of dumping stack unless in verbose mode.
This improves the command line interface when runners raise exceptions.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Show how many tests passed out of those that were actually
performed, not including skipped tests (which we also report
a count of)
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
should_write() returns False for properties that are skipped in
write_properties(). Shortens write_properties() a bit.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Update menuconfig (and kconfiglib and guiconfig, just to sync) to
upstream revision 424d0d38e7, to get this commit in, which works around
a crash on some macOS Python installations.
menuconfig: Work around crash on resize on some macOS systems
get_wch() has started raising curses.error when resizing the
terminal on some macOS Python installations. Work around for now by
falling back on getch(), which still works.
See https://github.com/ulfalizer/Kconfiglib/issues/84. Needs more
investigation, but I don't have a Mac handy.
Based on https://github.com/ulfalizer/Kconfiglib/pull/85, but with
some more comments.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
The
"{" + ", ".join(...) + "}"
pattern is in a bunch of places now. Add an out_dev_init() helper for
it, with the same parameters as out_dev().
Also makes "{" vs. "{ " consistent, picking the first one.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Add a helper function verify_error() to testedtlib.py that takes a DTS
source and verifies that it generates a particular EDTError.
Use it to test the hints that the previous commit added to errors
generated by _slice().
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Show how the element size was calculated in the error message when a
'reg', 'ranges', or 'interrupts' property has the wrong size. This
should help with debugging. Also mention that #*-cells properties come
from the parent node, which can be subtle.
Came up in https://github.com/zephyrproject-rtos/zephyr/issues/21607
(though it seems the comment disappeared).
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
dt_chosen_reg() and dt_node_reg() take between 1 and 3 arguments, not
between 1 and 4 arguments. The implicit name argument isn't included in
the count.
These functions implement $(dt_chosen_reg_*) and $(dt_node_reg_*).
Giving the right max argument count makes Kconfiglib generate error
messages that give the location of the call, instead of getting a
cryptic generic Python error.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
pyocd 0.24.0 provides support for more user options.
This enables flashing of additional boards using pyocd.
Signed-off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
stderr from the binary handler (native_posix for example) was redirected
to the logger as errors, this is confusing the console and users, so
remove this.
Fixes#21784
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
The _thread_idx_map bitfield which has '1' set for free
thread indexes really needs to live in the
data section reserved for kernel object metadata, as this
is a part of memory that is allowed to shift addresses
between zephyr_prebuilt.elf and zephyr.elf.
However, if an application defines enough static threads
that there are no free indexes, the entire bitfield will
be zeroed and the bitfield will end up in the main BSS
section.
Force this data to always be in the .kobject_data.data
section regardless of its contents.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
If the working directory for a command was missing (usually due to
forgetting to run 'west update'), you'd get a FileNotFoundError
exception along with a cryptic error like
'git' not found
Only catch OSError instead (which is a base class of FileNotFoundError),
and always show the exception message. It makes it clear that it's the
working directory that's missing.
Add some other misc. improvements too:
- Turn stderr output from external commands into a warning instead of
an error
- Add err() and warn() helpers
- Include the command name in messages
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Add binding support for a 'path' property type, for properties that are
assigned node paths. Usually, paths are assigned with path references
like 'foo = &label' (common in /chosen), but plain strings are accepted
as well as long as they're valid paths.
The 'path' type is mostly for completeness at this point, but might be
useful for https://github.com/zephyrproject-rtos/zephyr/issues/21623.
The support is there already in dtlib.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
In some cases, we've seen the output files be truncated when the
python script has been rebuilt into a .pex before running it --
likely due to buffering.
Closing files explicitly is the right thing to do anyway, so let's
do it.
Signed-off-by: Olof Johansson <olof@lixom.net>
The SRAM address and size are currently available as both
DT_SRAM_{BASE_ADDRESS,SIZE} and as CONFIG_SRAM_{BASE_ADDRESS,SIZE} (via
the Kconfig preprocessor).
Use the CONFIG_SRAM_* versions everywhere, and remove generation of the
DT_SRAM_* versions from gen_defines.py.
The Kconfig symbols currently depend on 'ARC || ARM || NIOS2 || X86'.
Not sure why, so I removed it.
It looks like no configuration files set CONFIG_SRAM_* at the moment, so
another option might be to use the DT_* symbols everywhere instead. Some
Kconfig.defconfig.series files add defaults to them though.
Also improve the help texts for CONFIG_SRAM_* to say that they normally
come from devicetree rather than configuration files.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Reorganize write_regs() to reuse more code for *_BASE_ADDRESS and
*_SIZE, and get rid of reg_addr_ident() and reg_size_ident().
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Make sure we close the pipe file handles after we are done, otherwise we
will end up with too many open file descriptors and crash...
Fixes#20974Fixes#21637
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Makes it much easier to inspect them in a different terminal
from where the test was run. These paths tend to be long
anyway, even if relative.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Sanitycheck used to delete the output directory if -n
wasn't used, but now it renamed it. Add a new flag to
enable the old behavior.
Do not use logging for these early messages, logger
hasn't been set up yet.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
The code has a lot of sys.exit() calls, but internally these
just raise a special exception. Add a try...finally block
to ensure 'stty sane' is run before leaving the script.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Prior to this all hex files included in a merge would
be printed on every invokation of cmake.
Allow the user to explicitly require this information
by moving the print to a VERBOSE-only cmake message.
Signed-off-by: Håkon Øye Amundsen <haakon.amundsen@nordicsemi.no>
We have a collection of python scripts that are part of our build
system. This PR collects docstring comments added to these scripts into
a summary document. Previous references to just the script name in
other documentation are updated to point to this build tool
documentation.
Some of the scripts needed an update to be processed (via include
directives) consistently.
Signed-off-by: David B. Kinder <david.b.kinder@intel.com>
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Add shields_list_contains Kconfig function which return bool based on
check of shield presence in cmake SHIELD_AS_LIST.
Signed-off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
Update Kconfiglib to upstream revision 9c0b562c94 to get this commit in:
Add Kconfig.__init__() helper flag for suppressing tracebacks
Tools that don't use standard_kconfig() currently generate spammy
tracebacks for e.g. syntax errors.
Add a suppress_traceback flag to Kconfig.__init__() for catching
"expected" exceptions and printing them to stderr and exiting with
status 1. Use it to make all tools consistently hide tracebacks.
Use the new flag to hide tracebacks for expected exceptions in
kconfig.py, lint.py, and genrest.py.
Some menuconfig robustness tweaks for wonky terminals are included as
well, and a new feature for customizing .config and autoconf.h header
comments via environment variables.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
This is needed when a board needs to be reset using an external commands
or tools that are not part of the flash command. For example, power
reset or by poking a GPIO header on the board using external wiring.
add post_flash/pre_flash to the platform section in the hardware map.
For example:
- available: true
connected: true
id: OSHW000032254e4500128002ab98002784d1000097969900
platform: reel_board
post_script: /tmp/post_flash.sh
pre_script: /tmp/pre_flash.sh
product: DAPLink CMSIS-DAP
runner: pyocd
serial: /dev/ttyACM11
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
It is supported to have a zephyr module that does not have a
module.yml, but zephyr_module.py does not support it and will drop
such modules.
To fix this we add support in zephyr_module.py.
Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
We have an open request to make the help for the -t option a little
easier. Try to do that without adding too much length to the short
help.
Fixes: #16202
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
For the following devicetree, view 'nested' as being on the bus.
Previously, only 'node' was considered to be on the bus.
some-bus {
compatible = "foo,bus-controller";
node {
nested {
compatible = "foo,device-on-bus";
};
};
};
In practice, this means that a 'bus:' key in the binding for
'foo,bus-controller' will now get matched up to an 'on-bus:' key in the
binding for 'foo,device-on-bus'.
Change the meaning of Node.bus and add two new attributes Node.on_bus
and Node.bus_node, with these meanings:
Node.bus:
The bus type (as a string) if the node is a bus controller, and
None otherwise
Node.on_bus:
The bus type (as a string) if the node appears on a bus, and None
otherwise. The bus type is determined from the closest parent
that's a bus controller.
Node.bus_node:
The node for the bus controller if the node appears on a bus, and
None otherwise
It's a bit redundant to have both Node.bus_node and Node.on_bus, since
Node.on_bus is the same as Node.bus_node.bus, but Node.on_bus is pretty
handy to save some None checks.
Also update gen_defines.py to use Node.on_bus and Node.bus_node instead
of Node.parent wherever the code deals with buses.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
I keep mixing these up, so that's probably a sign that the names are
bad. The root of the problem is that "parent-bus" can be read as both
"this is the parent bus" and as "the parent bus is this".
Use 'bus:' for the bus "provider" and 'on-bus:' for nodes on the bus
instead, which is less confusing.
Support the old keys for backwards compatibility, along with a
deprecation warning.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Add a static method for dumping hardware map and reuse it across the
script reducing duplicated code.
Fixes#21475
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
wdt_install_timeout() was skipped as it installs an ISR-context
callback handler function. The rest are simple wrappers.
Added myself as the maintainer of the syscall handlers. WDT
subsystem appears to not currently have an owner.
Fixes: #21432
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Added support of the Microchip with FTDI serial devices to be able
create a hardware map for them. In the future that hardware map
for the Microchip board will let run automatic Sanitycheck tests on it.
Signed-off-by: Maksim Masalski <maksim.masalski@intel.com>
Most bindings look something like this:
title: Foo
description: This binding provides a base representation of Foo
That kind of description doesn't add any useful information, as it's
just the title along with some copy-pasted text. I'm not sure what "base
representation" was supposed to mean originally either.
Many bindings also put something that's closer to a description in the
title, because it's not clear what's expected or how the title is used.
In reality, the title isn't used anywhere. 'description:' on the other
hand shows up as a comment in the generated header.
Deprecate 'title:' and generate a long informative warning if it shows
up in a binding.
Next commits will clean up the 'description:' strings (bringing them
closer to 'title:' in most cases) and remove 'title:' from all bindings.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
timeouts were not reported correctly and we were getting the default
'N/A' in case of a timeout.
Fixes#21438
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
When testing with one device and without a hardware map, make sure we
load the temporary map into the suite.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
More changes moving away from using global options and instead using
class variables and parameters.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
This function is not used anywhere else now that we use logging module,
so push this into the class where it is being used.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
The use of a global options variable is making it very difficult to
create a testsuite for this script, so reduce and push options into
arguments instead.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Drop custom output functions and use logging mode for almost all
reporting. Also log everything into sanitycheck.log so detailed run
information can be inspected later, even if we did not run with
--verbose mode on the console.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Stop using global options and instead use propagated variable.
This is to make testing of sanitycheck easier.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
tabulate: Needed to show list of devices in a well formatted table.
anytree: needed to list testsuite in a tree form.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
When something goes bad during the flashing process set the reason
correctly and put the error messages from the flasher into device.log
and display the location of that file instead of an empty handler.log
right now.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Move all hardware map generation/usage to a seperate class. This will
make it easier to extend the supported hardware in the future.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
If modules announce they have tests, run sanitycheck on those modules.
This will run on first matrix node and as the last step after running
regular testsuite.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Generate options for sanitycheck to run tests and samples in modules.
Use the --sanitycheck-out <file> to generate a file that can be supplied
to sanitycheck on the commandline which will add additional testroots
and boards if the module does contain out of tree boards.
the module.yaml file now accepts the following:
samples:
- path/to/samples
tests:
- path/to/tests
boards:
- path/to/boards
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Now using only Path and PurePath in zephyr_modules.py to handle module
processing.
This make the code cleaner as well as remove an issue where a module
name would become an empty string when module path contained trailing
path separators.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
We need a format code for struct packing that fits in
a pointer value, "I" is fixed at 32-bit.
The conversion of string to pointer value now prints
8 bytes. This works for 32-bit since the leading
4 digits are always zero.
The replaced length check uses sizeof(void *) and not 4.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
It appears that ninja 1.6.0 or greater don't seem to send SIGTERM down
to the child processes and thus we don't terminate correctly. This
causes a hang with renode simulations.
Change terminate call to 'self.try_kill_process_by_pid()' when test
state is updated (i.e. done running with either passed or failed), in
order to explicitly send a SIGTERM to the simulator process before
sending a SIGTERM to ninja.
Refactor the terminate code so we encapsulate the behavior in one place
for a BinaryHandler.
Based on change from Stephanos Ioannidis <root@stephanos.io>
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
A property may be optional with a default in a base yaml, then
overridden to be required in a subordinate file. Don't prevent this
by complaining about having a default on a required property.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
Remove leading/trailing blank lines in .c, .h, .py, .rst, .yml, and
.yaml files.
Will avoid failures with the new CI test in
https://github.com/zephyrproject-rtos/ci-tools/pull/112, though it only
checks changed files.
Move the 'target-notes' target in boards/xtensa/odroid_go/doc/index.rst
to get rid of the trailing blank line there. It was probably misplaced.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
export compile commands when running with --cmake-only, this can be used
for analysis and coverage statistics.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Fetch HW models from a new west module.
And, remove all pre-west glue which was used to:
* Fetch them in CI
* Validate their vesion
* Modify the include path and link to them
Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
This option, if set, will add arguments to CMake whenever a new build
system is being generated.
It doesn't affect other invocations of CMake, such as when cmake(1) is
run in build tool mode to actually compile the application.
See the documentation changes for details.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
The ARM Generic Interrupt Controller (GIC) supports multiple interrupt
types whose linear IRQ numbers are offset by a type-specific base
number.
This commit adds a function to automatically fix up ARM GIC interrupts
in order to output a linear IRQ number that is offset by the interrupt
type-specific base number.
For more details, refer to the issue #19860.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
Make sure we do enable size tracking with options that depend on the
size of generated binaries to be available.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Shortens main() a bit and makes it easier to read.
Rename write_flash() to write_flash_node(), and let write_flash() be the
top-level function. Also move the looking-up of the /chosen properties
into the the various helper functions themselves, and shorten some names
that are clear in context.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Make the node/ordinal list a part of the header comment to make the
output prettier.
Before:
/*
* Generated by gen_defines.py
*
* DTS input file:
* rv32m1_vega_ri5cy.dts.pre.tmp
*
* Directories with bindings:
* $ZEPHYR_BASE/dts/bindings
*/
/* Nodes in dependency order (ordinal : path): */
/* 0 : / */
/* 1 : /aliases */
/* 2 : /chosen */
/* 3 : /connector */
/* 4 : /cpus */
/* 5 : /cpus/cpu@0 */
/* 6 : /gpio_keys */
/* 7 : /soc */
...
After:
/*
* Generated by gen_defines.py
*
* DTS input file:
* rv32m1_vega_ri5cy.dts.pre.tmp
*
* Directories with bindings:
* $ZEPHYR_BASE/dts/bindings
*
* Nodes in dependency order (ordinal and path):
* 0 /
* 1 /aliases
* 2 /chosen
* 3 /connector
* 4 /cpus
* 5 /cpus/cpu@0
* 6 /gpio_keys
* 7 /soc
* ...
*/
Also move the writing of the top comment and the node comments into
separate functions, to shorten main() and make it easier to follow.
Piggyback some minor comment-related simplifications.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
gcovr is already a dependency in scripts/requirements.txt. The
visualization is different, but the functionality should be the same.
Tested with gcovr 4.2.
Relates to #17626.
Signed-off-by: Christian Taedcke <hacking@taedcke.com>
We now use EDTS and gen_defines.py to generate DTS defines. We
deprecated the old script and defines several releases ago, so lets now
remove them.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Print all tests using a tree structure (depends on anytree module). This
now can be done using --test-tree option.
sanitycheck --test-tree -T tests/kernel/
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Basic tool to help checking Kconfig options against a list of
hardening preferences.
This tool is available as a kconfig target, so to run it:
make/ninja hardenconfig
[Flavio Ceolin: Simplify logic and fix python lint issues]
Signed-off-by: Lauren Murphy <lauren.murphy@intel.com>
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Erroring out for 'status = "ok"' broke backwards compatibility for a
downstream project. Accept it instead.
Maybe the error could be selectively re-enabled later.
The rest of the code only checks for 'status = "disabled"' (like the old
scripts), so no other updates are needed.
(It's a bit weird that we duplicate the property check in base.yaml.
Thinking of including base.yaml implicitly. Could clean things up then.)
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
There is no code to handle __syscall_inline so it is better removing
it from doxygen and checkpatch.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Use the folder the config file is in as an additional search directory
for openocd. This way additional files, like custom debug interface
files can be put in the support folder of a board and the openocd.cfg
can use them.
Signed-off-by: Christian Taedcke <christian.taedcke@lemonbeat.com>
We were parsing random FAIL messages from the output of test runs ad
testcases and capturing them in the xml output. Now we only parse the
name if it starts with test_.
Fixes#21162
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
gen_syscall_header.py is not longer necessary, it was just creating a
file including syscall.h. This header is now included directly by
gen_syscalls.py.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
The blackmagicprobe runner's Python process fails to ignore SIGINT
when it runs GDB from the debug and flash callbacks, which is wrong.
The other runners tend to use run_server_and_client() to properly
handle this, since they start a GDB server and connect to it with a
client. The BMP USB device presents itself as a serial device which
speaks the GDB serial protocol instead, so there's no server/client,
and thus no call to run_server_and_client().
The problem is that blackmagicprobe essentially uses
subprocess.check_call() to start GDB directly, without ignoring SIGINT
in the python process. Easy fix.
Fixes: #21139
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Fixed bug in parsing testnames from source files. We added 1cpu testing
but did not update the regex and we have been passing wrong path to the
glob as well, meaning tests were not parsed at all.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
When storing an updated hardware list remove the serial property value
from entries that are not connected, to reduce confusion about exactly
what ttyACM0 is connected to.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
Confirm to the user what devices were found and where they are before
starting a long run that might not have found everything it was supposed
to test.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
Boards like mimxrt1060_evk are recognized by the scanner through their
USB device which has an ID, but in some cases the board may be
programmed using an external J-Link probe. Support this by adding a
probe_id key that can be added to the yaml dictionary to override the
use of id for this purpose.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
Add manufacturer and product identifiers to produce board information
for SiLabs and NXP hardware when using --generate-hardware-map.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
This commit fixes the issue, that --enable-coverage alone did not create
coverage information. It also required to give --coverage-platform.
Now the fallback for coverage-platform to platform works as documented
also for enable-coverage, not only for coverage option.
Signed-off-by: Christian Taedcke <hacking@taedcke.com>
The discard report is now generated for every run as
sanity-out/sanitycheck_discard.csv, the option --discard-report was
dropped.
Fixes#20804
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
The interprocessor communication can be based on shared memory.
Allow to declare this memory as with a generic name derived from
chosen declaration.
Signed-off-by: Arnaud Pouliquen <arnaud.pouliquen@st.com>
This command needs access to DT configuration, but can no longer
access it through BuildConfiguration since
9da1d41a12.
Import edtlib and use that instead.
Fix up some other error handling and output issues while we're here to
make the script's behavior easier to inspect and debug.
Fixes: #20545
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Add support for boards that utilize the TI XDS110 like the CC3220SF,
CC3235SF, CC1352R1 and CC26x2R1 LaunchXL boards. The XDS110 can expose
multiple serial endpoints one for the uart device, but another for a
trace buffer. We assume that endpoint 0 will be the UART device.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Add setting runner type to jlink for 'J-Link' and openocd for
'STM32 STLink' when generating the hardware map.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
This makes the misc-flasher runner usable by passing the build
directory to the underlying tool.
Fixes: #20658
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
This commit relocates the QEMU-specific code that currently resides in
ProjectBuilder.run to QEMUHandler.handle, in order to align with what
other handlers are doing.
For more details, refer to the PR #20573.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
The current QEMUHandler implementation in sanitycheck does not check
for the process exit code and reports "PASS" even when either the QEMU
executable cannot be launched or exited immediately due to an error
(e.g. unsupported machine type, missing file, ...).
This commit adds QEMU process exit code check and error reporting when
the exit code is a non-zero value.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
Python 3.5 and earlier do not preserve dictionary insertion order when
iterating over dictionaries, and do not give the same order between
runs. This broke the dtlib and edtlib test suites and made the output
jump around randomly between runs. It also made device INST_<n> numbers
non-deterministic, which broke some code on Python 3.5 (though
hardcoding device instance numbers in the code might be a bit shaky).
Fix it by using collections.OrderedDict instead of plain dict wherever
order matters. This makes the output identical on all supported Python
versions. It also allows testdtlib.py and testedtlib.py to run in CI,
which uses Python 3.5.
Fixes: #20571
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
edtlib is a library, and modifying yaml.(C)Loader directly interferes
with any binding loading in edtlib clients. To avoid that, add a custom
loader for bindings.
Internally, PyYAML does this, which is why defining a separate class
works:
@classmethod
def add_constructor(cls, tag, constructor):
if not 'yaml_constructors' in cls.__dict__:
cls.yaml_constructors = cls.yaml_constructors.copy()
cls.yaml_constructors[tag] = constructor
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
commit 0df4a53107 changed the behavior of
how openocd commands are passed to openocd. We used to add -c to each
command, now the commands are being added without -c causing an error.
This adds "-c" to all commands instead of just passing a list.
Also fixes#20449.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
This commit extends the nrfjprog.py runner script so it adds
support for nRF53. In the wake of the changes done in the
runner script, we extend the testing done in test_nrfjprog.py,
adding the required coverage for the nRF53.
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
Promote the private z_arch_* namespace, which specifies
the interface between the core kernel and the
architecture code, to a new top-level namespace named
arch_*.
This allows our documentation generation to create
online documentation for this set of interfaces,
and this set of interfaces is worth treating in a
more formal way anyway.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Add API for accessing Electrically Erasable Programmable Read-Only
Memory (EEPROM) devices.
EEPROMs have an erase block size of 1 byte, a long lifetime, and allows
overwriting data on byte-by-byte access.
Signed-off-by: Henrik Brix Andersen <hebad@vestas.com>
Update Kconfiglib, menuconfig, and guiconfig to upstream revision
faa1d21998, mostly to get this commit in:
Add public helpers for generating "<name> (defined at ...)" strings
Have Symbol/Choice.name_and_loc return strings like
"MY_SYM (defined at foo:1, bar:2)"
"<choice> (defined at foo:4)"
I've added a function like that in at least four different scripts
now, so that's probably a sign that it's a worthwhile helper.
Clean up the tests/Klocation tests a bit while adding tests.
Use the new helper to simplify kconfig.py a bit. Also clean it up a bit
by removing some unused stuff.
Some other minor improvements are included as well, e.g. to make
menuconfig/guiconfig give more helpful errors on invalid arguments.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Add a cleaned-up version of a script I used to find a bunch of unused
symbols and some other Kconfig issues. It's set up to be run directly,
with few environment dependencies.
West is required, because the checks need to see Kconfig files and
source code from all modules.
Checks so far:
- Symbols that can never be anything but n/empty
- Symbols that look unused
- menuconfig symbols with empty menus
- Symbols only defined in Kconfig.defconfig files
See the help strings for the command-line flags for more information.
Some of these checks could probably be checked in CI later, though the
always-n and unused-symbol checks are a bit heuristic and might need a
lot of whitelisting.
Another reason I want to get this in is to have a clean standalone
reference for how to set up the environment for parsing the Kconfig
files. It's gotten trickier over time.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Turns
edt.required_by(node)
edt.depends_on(node)
into
node.required_by
node.depends_on
which might be a bit more readable.
One drawback is that @property hides that there's some slight overhead
in accessing them, but I suspect it won't be meaningful. Caching could
be added if it ever turns out to be.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Some boards require specific sequences of commands to run which aren't
generally useful for other boards. Add a catch-all runner to handle
these cases.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
This commit refactors kernel and arch headers to establish a boundary
between private and public interface headers.
The refactoring strategy used in this commit is detailed in the issue
This commit introduces the following major changes:
1. Establish a clear boundary between private and public headers by
removing "kernel/include" and "arch/*/include" from the global
include paths. Ideally, only kernel/ and arch/*/ source files should
reference the headers in these directories. If these headers must be
used by a component, these include paths shall be manually added to
the CMakeLists.txt file of the component. This is intended to
discourage applications from including private kernel and arch
headers either knowingly and unknowingly.
- kernel/include/ (PRIVATE)
This directory contains the private headers that provide private
kernel definitions which should not be visible outside the kernel
and arch source code. All public kernel definitions must be added
to an appropriate header located under include/.
- arch/*/include/ (PRIVATE)
This directory contains the private headers that provide private
architecture-specific definitions which should not be visible
outside the arch and kernel source code. All public architecture-
specific definitions must be added to an appropriate header located
under include/arch/*/.
- include/ AND include/sys/ (PUBLIC)
This directory contains the public headers that provide public
kernel definitions which can be referenced by both kernel and
application code.
- include/arch/*/ (PUBLIC)
This directory contains the public headers that provide public
architecture-specific definitions which can be referenced by both
kernel and application code.
2. Split arch_interface.h into "kernel-to-arch interface" and "public
arch interface" divisions.
- kernel/include/kernel_arch_interface.h
* provides private "kernel-to-arch interface" definition.
* includes arch/*/include/kernel_arch_func.h to ensure that the
interface function implementations are always available.
* includes sys/arch_interface.h so that public arch interface
definitions are automatically included when including this file.
- arch/*/include/kernel_arch_func.h
* provides architecture-specific "kernel-to-arch interface"
implementation.
* only the functions that will be used in kernel and arch source
files are defined here.
- include/sys/arch_interface.h
* provides "public arch interface" definition.
* includes include/arch/arch_inlines.h to ensure that the
architecture-specific public inline interface function
implementations are always available.
- include/arch/arch_inlines.h
* includes architecture-specific arch_inlines.h in
include/arch/*/arch_inline.h.
- include/arch/*/arch_inline.h
* provides architecture-specific "public arch interface" inline
function implementation.
* supersedes include/sys/arch_inline.h.
3. Refactor kernel and the existing architecture implementations.
- Remove circular dependency of kernel and arch headers. The
following general rules should be observed:
* Never include any private headers from public headers
* Never include kernel_internal.h in kernel_arch_data.h
* Always include kernel_arch_data.h from kernel_arch_func.h
* Never include kernel.h from kernel_struct.h either directly or
indirectly. Only add the kernel structures that must be referenced
from public arch headers in this file.
- Relocate syscall_handler.h to include/ so it can be used in the
public code. This is necessary because many user-mode public codes
reference the functions defined in this header.
- Relocate kernel_arch_thread.h to include/arch/*/thread.h. This is
necessary to provide architecture-specific thread definition for
'struct k_thread' in kernel.h.
- Remove any private header dependencies from public headers using
the following methods:
* If dependency is not required, simply omit
* If dependency is required,
- Relocate a portion of the required dependencies from the
private header to an appropriate public header OR
- Relocate the required private header to make it public.
This commit supersedes #20047, addresses #19666, and fixes#3056.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
(pinctrl-<index> is documented in
Documentation/devicetree/bindings/pinctrl/pinctrl-bindings.txt in
Linux.)
Add a new Node.pin_states property, derived from any pinctrl-<index>
properties on the node. Node.pin_states holds a list of PinState
objects, where each PinState represents a single pinctrl-<index>
property.
For example, Node.pin_states will have two elements for the 'device'
node below:
device {
pinctrl-0 = <&state_0>;
pinctrl-1 = <&state_1 &state_2>;
pinctrl-names = "default", "sleep";
};
pincontroller {
state_0: state_0 {
...
};
state_1: state_1 {
...
};
state_2: state_2 {
...
};
};
Each PinState holds the list of configurations nodes in
PinState.conf_nodes. For the node above, node.pin_states[1].conf_nodes
will contain the pincontroller/state_1 and pincontroller/state_2 nodes,
for example.
The new functionality isn't used by gen_defines.py yet, so this change
is a no-op in itself, except it adds some error checking for
pinctrl-<index> properties.
If needed, support for #pinctrl-cells and 'pinmux' (not the same thing
as the 'pinmux' properties in Zephyr I think) could be added separately
later. Not sure what belongs in edtlib.py there yet.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Let the user specify these arguments as many times as they want:
--cmd-pre-init
--cmd-pre-load
--cmd-post-verify
This makes it a bit easier to handle scripts that need to do a few
things in a row depending on conditions.
Handle --cmd-pre-init and the port arguments properly in the debug
related targets.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
I'd like to support nRF boards in a more generic way, so make it
optional. We can clean up the STM32 files later if there's a lot of
overlap in their .cfg files.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
None of the splitting logic is needed if 64-bit return
values or parameters fit inside a register.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
We need to pass system call args using a register-width
data type and not hard-code this to u32_t.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
This commit fixes an issue in windows where zephyr_modules.txt contains
a \ as path separator.
This causes issues later when using the path generated by
zephyr_module.txt are used as variables in CMake on windows.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
If we don't have a DTS (like nrf52_bsim) we shouldn't try and create an
EDT, but we still need to call expr_parser.parse to filter testcases.
So move the os.path.exists(dts_path) around the creation of the EDT.
Fixes: #20371
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
If we have something like dma-names we should ended up generating
something like:
DT_ST_STM32_I2S_40015000_RX_DMAS_CONTROLLER_1
This isn't quite what we expect. We should instead get:
DT_ST_STM32_I2S_40015000_RX_DMAS_CONTROLLER
DT_ST_STM32_I2S_40015000_DMAS_CONTROLLER_1
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
nrf52_bsim does not generate any DTS data to be processed by
sanitycheck, skip filtering if we have no dts data.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Device tree nodes have a dependency on their parents, as well as other
nodes. To ensure driver instances are initialized in the proper we
need to identify these dependencies and construct a total order on
nodes that ensures no node is initialized before something it depends
on.
Add a Graph class that calculates a partial order on nodes, taking
into account the potential for cycles in the dependency graph.
When generating devicetree value headers calculate the dependency
graph and document the order and dependencies in the derived files.
In the future these dependencies may be expressed in binding data.
Signed-off-by: Peter A. Bigot <pab@pabigot.com>
environment is passed to execution thread and should be available
regardless of SAN being enabled or not.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
sanitycheck now requires yaml.FullLoader to support generating
hardware maps. This feature was introduced in PyYAML 5.1.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
Add function that will return 'y' or 'n' if a node pointed to by a
chosen property exists and is enabled.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
All users of dt kconfig symbols for tests and samples are using dt
functions instead so we can remove the support for parsing the
generated file.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Add the following functions to allow filtering based on device tree
dt_compat_enabled(compat) - Returns true if a device tree node
compatible matches 'compat' and the node is enabled.
dt_alias_exists(alias) - Returns true if a device tree node exists with
'alias' and the node is enabled.
dt_compat_enabled_with_alias - Returns true if a device tree node
compatible matches 'compat' and the node has 'alias' and the node is
enabled.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Just calls through to concurrent.futures.Executor.shutdown() in the base
class, which has the same signature. Removing it means the base class
version will get used directly.
Fixes this pylint warning:
W0235: Useless super delegation in method 'shutdown'
(useless-super-delegation)
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Fixes this pylint warning:
R0201: Method could be a function (no-self-use)
Could also make it a class method.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
We try to demonstrate some concepts for user mode:
- Multiple logical applications, each with their own memory
domain
- Creation of a sys_mem_pool and assignment to a memory
partition
- Use of APIs like k_queue_alloc_append() which require
thread resource pools to be configured
- Management of permissions for kernel objects and drivers
- Show how application-specific system calls are defined
- Show IPC between ISR and application (using k_msgq) and
application-to-application IPC (using k_queue)
Fixes: #14683
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Multi-line comments were stuck as-is between /* and */ in the generated
header, which looks ugly and confusing e.g. for multi-line
binding/property descriptions.
Use this format for multi-line comments in the header instead:
/*
* First line
* Second line
*
* Line after blank line
*/
Also clean up the output a bit by turning some things that were separate
comments into a single multi-line comment. Add some air and reduce line
lengths too.
Before:
/* Generated by gen_defines.py */
/* DTS input file: hifive1.dts.pre.tmp */
/* Directories with bindings: $ZEPHYR_BASE/dts/bindings */
/* Devicetree node: /cpus/cpu@0/interrupt-controller */
/* Binding (compatible = riscv,cpu-intc): $ZEPHYR_BASE/... */
/* Binding description: This binding describes the RISC-V ...
Some extra lines
for testing */
#define DT_INST_0_RISCV_CPU_INTC 1
After:
/*
* Generated by gen_defines.py
*
* DTS input file:
* hifive1.dts.pre.tmp
*
* Directories with bindings:
* $ZEPHYR_BASE/dts/bindings
*/
/*
* Devicetree node:
* /cpus/cpu@0/interrupt-controller
*
* Binding (compatible = riscv,cpu-intc):
* $ZEPHYR_BASE/dts/bindings/interrupt-controller/...
*
* Description:
* This binding describes the RISC-V CPU Interrupt Controller
*
* Some extra lines
* for testing
*/
#define DT_INST_0_RISCV_CPU_INTC 1
Also tweak Node.description and Property.description in edtlib to be
strip()ed instead of rstrip()ed. There's probably no reason to
preserving leading whitespace in them either.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Use the LibYAML-based yaml.CLoader if available instead of yaml.Loader,
which is written in Python and slow. See
https://pyyaml.org/wiki/PyYAMLDocumentation.
This speeds up gen_defines.py from 0.2s to 0.07s on my system, for
-DBOARD=hifive1. It should also make scripts/kconfig/kconfig.py faster,
because it indirectly uses edtlib via
scripts/kconfig/kconfigfunctions.py.
yaml.CLoader seems to be available out of the box when installing with
pip on Ubuntu at least.
Helps with https://github.com/zephyrproject-rtos/zephyr/issues/20104.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Simplify use of property values that have multiple elements by
providing all of them in an initializer list, in addition to each one
as an enumerated value.
For example if a driver requires a sequence of operations with
instance-specific delays between stages the durations can be specified
with:
delays = <30000 20 45000>;
and the driver can use:
static u32_t delays[] = DT_INST_0_COMPAT_DELAYS;
rather than enumerating the instances. This is particularly useful
when the number of entries in the array varies per instance, in which
case such an initialization is not easily expressed in code.
Signed-off-by: Peter A. Bigot <pab@pabigot.com>
Replace:
dt_chosen_reg_addr
dt_chosen_reg_size
dt_node_reg_addr
dt_node_reg_size
with:
dt_chosen_reg_addr_int
dt_chosen_reg_size_int
dt_chosen_reg_addr_hex
dt_chosen_reg_size_hex
dt_node_reg_addr_int
dt_node_reg_size_int
dt_node_reg_addr_hex
dt_node_reg_size_hex
So that we get the proper formatted string for the type of symbol.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
save_reports should be one of the last tasks executed because it
closes the log file. Withouth it, other functions that use debug
functions like info and error will try to write into a close file.
This fixes the following problem:
sanitycheck -x=USE_CCACHE=0 -p native_posix -T samples/hello_world/ -b
-N --log-file sanity.log
JOBS: 8
Building initial testcase list...
1 test configurations selected, 0 configurations discarded due to filters.
Adding tasks to the queue...
total complete: 1/ 1 100% skipped: 0, failed: 0
1 of 1 tests passed (100.00%), 0 failed, 0 skipped with 0 warnings in
2.91 seconds
Traceback (most recent call last):
File "./zephyr/scripts/sanitycheck", line 3866, in <module>
main()
File "./zephyr/scripts/sanitycheck", line 3854, in main
suite.summary(options.disable_unrecognized_section_test)
File "./zephyr/scripts/sanitycheck", line 2306, in summary
self.duration))
File "./zephyr/scripts/sanitycheck", line 432, in info
log_file.write(what + "\n")
ValueError: I/O operation on closed file.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Using exact versions for our PyPI dependencies (i.e. requirements.txt
lines that look like "dependency==X.Y.Z") is rude since we're asking
users to install these dependencies outside of any virtual
environment. This causes conflicts with other packages which may
depend on the same things as us.
Use inclusive ordered comparison (>=) release specifiers instead:
https://www.python.org/dev/peps/pep-0440/#inclusive-ordered-comparison
If this causes problems for individual packages, we can refine them
using compatible release specifiers (~=):
https://www.python.org/dev/peps/pep-0440/#compatible-release
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Script to be used when creating a release. For regular releases this can
be called this way:
$ list_issues.py -f issues.md -s 2019-09-01
Will list all closed issues since september 1st, 2019 and will create a
markdown file with all issues that can be added as is to the release
notes.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
The lvgl library is configured in some tests to use
the libC malloc to allocate heap. In native_posix this
ends up in the underlying OS heap, which cannot be safely
freed in general when the program finnishes, and
ends up being reported by valgrind as leaked on exit. See
https://github.com/zephyrproject-rtos/zephyr/pull/18471
for a longer discussion.
Instead of trying to fix the leak, let's just accept it
but suppress the errors so it does not confuse other
developers
Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
Failed unit tests were setting wrong fail string (error instead of
failed) which made unit tests always pass.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Set threads spawned by BinaryHandler dameon, so when the main thread
exits for some unusual reason (e.g SIGINT) the child threads are
automatically killed.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
os.mkdir() is not suitable to create arbitrary directory path (can
create only a subdir of an existing dir, will error out if already
exists), os.makedirs() should be always used in such cases.
Signed-off-by: Paul Sokolovsky <paul.sokolovsky@linaro.org>
Previously, if two bindings had the same 'compatible:'/'parent-bus:'
values, the binding that happened to be loaded last would get used.
Turn it into an error instead. This avoids tricking people into thinking
that bindings get loaded in a defined order.
Maybe overriding bindings could be allowed later, if we need it.
Fixes: #19536
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Simplifies the code a bit.
Looks like the description wasn't rstrip()ed when it came from a
'child-binding:' either. This also indirectly fixes that.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Make sure we cleanup the manifest files at the right spot of the CI
process. When not building a PR, behavior is different.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
A complete overhaul of the sanitycheck script and how we build and run
tests. This new version of sanitycheck uses python for job distribution
and drop use of Make.
In addition to the move to python threading library, the following has
been changed:
- All handlers now run in parallel, meaning that any simulator will run
in parallel and when testing on multiple devices (using
--device-testing) the tests are run in parallel.
- Lexicial filtering (using the filter keyword in yaml files) is now
evaluated at runtime and is no long being pre-processed. This will allow
us to immediately start executing tests and skip the wait time that was
needed for filtering.
- Device testing now supports multiple devices connected at the same
time and is managed using a hardware map that needs to be generated and
maintained for every test environment. (using --generate-hardware-map
option).
- Reports are not long stored in the Zephyr tree and instead stored in
the output directory where all build artifacts are generated.
- Each tested target now has a junit report in the output directory.
- Recording option for performance data and other metrics is now
available. This will allow us to record the output from the console and
store the data for later processing. For example benchmark data can be
captured and uploaded to a tracking server.
- Test configurations (or instances) are no longer being sorted, this
will help with balancing the load when we run sanitycheck on multiple
hosts (as we do in CI).
And many other cleanups and improvements...
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
k_thread_create and K_THREAD_DEFINE both take a delay as the final
parameter. Most uses of K_THREAD_DEFINE pass either `K_NO_WAIT` or
`K_FOREVER`. Ensure that all uses of K_THREAD_DEFINE follow that
practice, and that the runtime k_thread_create calls do so as well.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
k_sleep uses the same underlying thread infrastructure as the other
functions that take timeouts, so the delay should be specified as a
timeout rather than milliseconds.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
Sort the functions within the regular expression so they can be
checked more easily.
Remove k_thread_deadline_set as it takes an argument in cycles. (The
one in-tree call to this function was not affected by this error.)
Add missed k_mbox_data_block_get.
Fix an overly ambitious multi-match disjunct that covered some
non-existent functions.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
Add support for the report and patch modes so this can be invoked by
coccicheck.
Use PCRE options to make the kernel timeout API identifier rule more
readable. Extend the pattern to new API.
Use rule extends and depends clauses, and pattern disjunction, to
avoid replicating metavariable content.
Hint that using --include-headers may be helpful (some patterns can be
found in static inline functions).
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
With multiple debug probes attached, attempting to launch multiple debug
servers resulted in "OSError: [Errno 98] Address already in use" despite
explicitly setting --gdb-port to unique values.
The issue was caused by the default telnet port: 4444. Adding
--telnet-port parameter allows to explicitly define the address to a
unique value and avoid the socket exception.
Signed-off-by: Rihards Skuja <rihardssk@mikrotik.com>
Node._prop_val() returned too early for non-existent booleans, letting
missing 'required: true' booleans slip through without an error.
Fix it by rearranging the code to always do the 'required' check before
returning.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Requested by Marc Herbert. Makes the output deterministic as long as all
binding directories are within $ZEPHYR_BASE (and a bit less spammy too).
Example output for header:
Before:
/* Directories with bindings: /home/ulf/z/z/dts/bindings */
...
/* Binding (...): /home/ulf/.../arm,v7m-nvic.yaml */
After:
/* Directories with bindings: $ZEPHYR_BASE/dts/bindings */
...
/* Binding (...): $ZEPHYR_BASE/dts/.../arm,v7m-nvic.yaml */
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Some legacy code still passes integer literals in milliseconds as the
value to functions that take a timeout. This usage interferes with
plans to replace the millisecond representation with a more generic
k_timeout_t value. Add a Coccinelle script to convert call sites to
use the proper constants and macros.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
verify_eq() can be used instead of verify_streq(), since
warnings.getvalue() already returns a string.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
With ST boards it is possible to specify the board ID when flashing using
openocd. This is very useful when having multiple devices connected.
This change allows us to address a device directly:
west flash -- --cmd-pre-init "hla_serial 066BFF535254887767174558"
This needs to be called before init, hence the new option.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Add a 'warn_file' parameter to EDT.__init__() that gives a 'file' object
to write warnings to. Use it to capture and verify warnings generated
for deprecated features in testedtlib.py. This indirectly gets rid of
possibly broken-looking output when running it.
Because any function that writes warnings now needs to use EDT._warn()
(as self._warn()), some functions were moved into the EDT class.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Implement a nice generalization suggested by Bobby Noelte.
Instead of having a generic #cells key in bindings, have source-specific
*-cells keys. Some examples:
interrupt-cells:
- irq
- priority
- flags
gpio-cells:
- pin
- flags
pwm-cells:
- channel
- period
This makes bindings a bit easier to read, and allows a node to be a
controller for many different 'phandle-array' properties.
The prefix before *-cells is derived from the property name, meaning
there's no fixed set of *-cells keys. This is possible because of the
earlier 'phandle-array' generalization.
The older #cells key is supported for backwards compatibility, but
generates a deprecation warning.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Fix a bug where DTS_BINDINGS_DIRS could only have one entry. When
there were more than one entry the command for invoking menuconfig
became corrupted.
This changes the separator of DTS_BINDINGS_DIR from a space to ? so
that the shell does not interpret the space as an argument separator.
Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
This commit addresses the following portability issues:
1. gen_syscalls incorrectly assumes that the compiler is always GCC.
2. pragma GCC diagnostic push and pop are not supported in GCC < 4.6.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
Prompted by an upstream bug report. Nothing in Zephyr triggers this at
the moment, but might as well fix it.
Update Kconfiglib to upstream revision 7d05084b7e, to get this commit
in:
Fix handling of parentheses in macro argument values
As an oversight, there was no check for nested parentheses in macro
arguments, making the preprocessor think the call ended after
'void)' in
def_bool $(success,echo 'void foo(void) { asm inline (""); }' \
| $(CC) -x c - -c -o /dev/null)
This broke the latest linux-next kernels (with a Kconfig error),
starting with commit eb111869301e1 ("compiler-types.h: add
asm_inline definition").
I remember seeing this when going through the C code, but somehow
forgot to put it in. Fix it, and clean up _expand_macro() a bit at
the same time.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Generating generic information for 'type: phandle-array' properties in
edtlib was difficult due to defining phandle-array as just a list of
phandles and numbers. To make sense of a phandle-array property like
'pwms', you have to know that #pwm-cells is expected to appear on
each referenced controller, and that the binding for the controller has
a #cells.
Because of this, handling of various 'type: phandle-array' properties
was previously hardcoded in edtlib and exposed through properties like
Node.pwms, instead of through the generic Node.props (though with a lot
of shared code).
In practice, it turns out that all 'type: phandle-array' properties in
Zephyr work exactly the same way: They all have names that end in -s,
the 's' is removed to derive the name of related properties, and they
all look up #cells in the binding for the controller, which gives names
to the data values.
Strengthen the definition of 'type: phandle-array' to mean a property
that works exactly like the existing phandle-array properties (which
also means requiring that the name ends in -s). This removes a ton of
hardcoding from edtlib and allows new 'type: phandle-array' properties
to be added without making any code changes.
If we ever need a property type that's a list of phandles and numbers
but that doesn't follow this scheme, then we could add a separate type
for it. We should check if the standard scheme is fine first though.
The only property type for which no information is generated is now
'compound'.
There's some inconsistency in how we generate identifiers for clocks
compared to other 'type: phandle-array' properties, so keep
special-casing them for now in gen_defines.py (see the comment in
write_clocks()).
This change also enabled a bunch of other simplifications, like reusing
the ControllerAndData class for interrupts.
Piggyback generalization of *-map properties so that they work for any
phandle-array properties. It's now possible to have things like
'io-channel-map', if you need to.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
'Specifier' is devicetree specalese for data associated with interrupts,
GPIOs, etc., e.g. <1 2> and <3 4> in
pwms = <&ctrl-1 1 2 &ctrl-2 3 4>;
It's probably unnecessarily confusing to call it that. Call it 'data'
instead, which is a bit more straightforward.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
edtlib.Device is just a devicetree node augmented with binding
information and some interpretation of properties. Rename it to
edtlib.Node to make that clearer. That also avoids calling things like
flash partition nodes "devices", which is a bit confusing.
I called it edtlib.Device instead of edtlib.Node originally to avoid
confusion with dtlib.Node, but in retrospect it probably makes it more
confusing on the whole. Something like edtlib.ENode might work too, but
it's probably overkill. Clients of edtlib.py only interact with
edtlib.Node, so the only potential for confusion is within edtlib.py
itself, and it doesn't get too bad there either.
Piggyback some documentation nits, and consistently write it
"devicetree" instead of "device tree", to match the spec.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
The ESP tool is being executed directly in the esp32 runner,
assuming the tool is executable by itself. However, it would
fail under Windows as subprocess.check_call() cannot execute
Python scripts directly. The fix is to execute the Python
interpreter and passing the script path as a command line
parameter.
Fixes#19098
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
As a slightly hairy but important optimization inherited from the old
scripts, the binding loading code only looks at binding files whose raw
text contains one of the compatible strings from the devicetree. For
such files, a second pass parses the file as YAML and tries to extract a
compatible string, and skips the file if it fails (e.g. due to spurious
text matches in 'include'd binding fragments).
Until now, the binding would always get fully loaded (have 'include'd
files merged in, checks run, etc.) if the second pass managed to extract
a compatible.
Do slightly better by only fully loading the binding if the extracted
compatible from the second pass appears in the devicetree. This gets rid
of unnecessary binding loading in rare cases.
Discovered by test-bindings/deprecated.yaml getting loaded even when
everything that referenced it in test.dts was commented out, because it
happened to mention 'child-binding' in a comment.
Also add a check for YAML errors in the second pass, to be slightly more
robust. Print a warning if a file that isn't valid YAML is found.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
API oversight. This was meant to be there all along together with
Device.parent, for navigating the devicetree, but since a need for it
never came up in gen_defines.py, it got overlooked.
Devices are just devicetree nodes augmented with binding information and
some interpretation of devicetree properties. I wonder if the name
should be changed to something like edtlib.Node to make that clearer.
Calling something like a flash partition a "device" is a bit weird, as
Galak pointed out.
I think I went with Device originally to avoid confusion with
dtlib.Node, but since edtlib users don't directly interact with dtlib,
it might not be that confusing in practice.
Piggyback some documentation clarifications.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
The devicetree check in check_compliance.py in ci-tools expects the
dtlib/edtlib test suites to exit with sys.exit() (which raises
SystemExit) on test failures, and interprets Exception as an internal
error in the test suite.
testedtlib.py accidentally raised Exception on test failures, making
check_compliance.py error out and skipping the rest of the tests when
there were failures. Fix it.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Python's Sets are not deterministic. This causes the following lines to
be emitted in random order in generated/syscall_dispatch.c
extern u32_t z_mrsh_k_object_release(u32_t arg1, ...
extern u32_t z_mrsh_k_object_access_grant(u32_t arg1, ...
extern u32_t z_mrsh_k_object_alloc(u32_t arg1, ...
Change noweak to a basic list.
Reproducibility regression introduced by commit 6564974bae
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Deprecate 'sub-node:' and add a more general 'child-binding:' mechanism
to bindings. Keep supporting 'sub-node:', but print a deprecation
warning when it's used.
Like 'sub-node:', 'child-binding:' gives a binding to child nodes, but
the binding is required to be a complete binding, and is treated (and
checked) like a normal binding.
'child-binding:' can in turn contain another 'child-binding:', up to any
number of levels. This is automatic from treating it like a normal
binding, and from the code initializing parent Devices before child
Devices.
This lets nodes give bindings to grandchildren.
For example, take this devicetree fragment:
parent {
compatible = "foo";
child-1 {
grandchild-1 {
...
};
grandchild-2 {
...
};
};
child-2 {
grandchild-3 {
...
};
};
};
The binding for 'foo' could provide bindings for grandchild-1/2/3 like
this:
compatible: "foo"
# Binding for children
child-binding:
title: ...
description: ...
...
# Binding for grandchildren
child-binding:
title: ...
description: ...
properties:
...
Due to implementation issues with the old devicetree scripts, only two
levels of 'child-binding:' is supported for now. This limitation will go
away in Zephyr 2.2.
Piggyback shortening 'description:' and 'title:' in some bindings that
provide child bindings. This makes the generated header a bit neater.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Currently, string values comming from "common" test definition section
and from test-specific section are just concatenated. Suppose, we want
to define some common filter condition, and also per-test additional
criteria. Currently, that leads to following syntax:
common:
filter: TOOLCHAIN_FOO == 1
tests:
sample.net.sockets.http_get.posix:
filter: and not CONFIG_BAR
That's arguable quite adhoc, and the only way to figure it out for
most people will be to add debug logging.
This patch proposes to use the expected syntax (i.e.
"filter: not CONFIG_BAR"), and combine conditions properly based on
their semantic meaning (which also includes parans for proper
evaluation order).
Signed-off-by: Paul Sokolovsky <paul.sokolovsky@linaro.org>
For cases like zephyr-test or for other reasons the repo dir might not
be called zephyr. Just use the basename of the dir instead of assuming
its called zephyr.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Instead of
child:
bus: foo
parent:
bus: bar
, have
child-bus: foo
parent-bus: bar
'bus' is the only key that ever appears under 'child' and 'parent'.
Support the old keys for backwards compatibility, with a deprecation
warning if they're used.
Also add 'child/parent-bus' tests to the edtlib test suite. It was
untested before.
I also considered putting more stuff under 'child' and 'parent', but
there's not much point when there's just a few keys I think. Top-level
stuff is cleaner and easier to read.
I'm planning to add a 'child-binding' key a bit later (like 'sub-node',
but more flexible), and child-* is consistent with that.
Also add an unrelated test-bindings/grandchild-3.yaml that was
accidentally left out earlier.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Some DWARF symbols for members of template classes members such as
numeric_limits<unsigned int> reference are static constexpr values
that do not have a data member location. Avoid attempting to
dereference the value for that attribute when it isn't present.
Signed-off-by: Peter A. Bigot <pab@pabigot.com>
Removed redundant linker flag --coverage from native posix
EXTRA_LDFLAGS.
The correct linker flags will be set by defining the Kconfig flag
CONFIG_COVERAGE, either by sanitcheck it self or via the test/sample
prj.conf.
Signed-off-by: Jan Van Winkel <jan.van_winkel@dxplore.eu>
Some confluence of recent changes resulted in builds with
application-specific bindings being unable to find bindings present in
the system directory. Add quotes and splits as necessary to propagate
multiple directories through the system.
Signed-off-by: Peter A. Bigot <pab@pabigot.com>
- Remove unused variables and an unused 'sys' import
- Simplify 'if len(foo) != 0' to 'if foo'. Non-empty lists/dicts/etc.
are truthy in Python.
- Use a raw string to fix this warning:
scripts/footprint/size_report:270:0: W1401: Anomalous backslash
in string: '\.'. String constant might be missing an r prefix.
(anomalous-backslash-in-string)
The issue is that '\.' just happens to work due to not being
recognized as an escape sequence.
Escape sequences are not interpreted in raw strings, so they're safer
for regexes.
- Replace 'is 0' with '== 0'. 'is' is for testing object identity, and
0 isn't guaranteed to be a unique object (but always is in practice).
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
As we work on removing the .conf file that is generated by the dt
scripts, mark dt_{int,hex,str}_val functions as deprecated.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Convert how we get the various chosen properties like "zephyr,console"
to use the new kconfig functions like dt_chosen_to_label.
Because of how kconfig parses things we define a set of variables of the
form DT_CHOSEN_Z_<PROP> since comma's are parsed as field seperators in
macros.
This conversion allows us to remove code in gen_defines.py for the
following chosen properties:
zephyr,console
zephyr,shell-uart
zephyr,bt-uart
zephyr,uart-pipe
zephyr,bt-mon-uart
zephyr,uart-mcumgr
zephyr,bt-c2h-uart
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Things that parse generated_dts_board.conf can't deal with entries
like:
DT_GPIO_KEYS_BUTTON_1_GPIOS={"GPIO_0", 14, 256}
so keep them from being added there.
Signed-off-by: Peter A. Bigot <pab@pabigot.com>
When a phandle-array (compound) has multiple members generate a define
that says how many there are; generate an initializer for each
individual member; and generate an initializer for the sequence of
members.
This allows drivers that expect multiple values in a compound to process
them without horrendous conditional compilation code attempting to
detect the number of elements in the compound.
It also eliminates the need to repeat the long prefix when initializing
a structure with the fields of a single compound.
Signed-off-by: Peter A. Bigot <pab@pabigot.com>
The lowest level output function is the one that determines the full
name of the macro, including DT_ prefix. Return the name of the macro
that provides the value for an identifier so that it can be used in
higher layers of the generator.
Signed-off-by: Peter A. Bigot <pab@pabigot.com>
Add str2str to make the conversion of a string into a C literal with all
necessary escapes and enclosing double quotes available outside a
function that emits a define.
Signed-off-by: Peter A. Bigot <pab@pabigot.com>
System call arguments, at the arch layer, are single words. So
passing wider values requires splitting them into two registers at
call time. This gets even more complicated for values (e.g
k_timeout_t) that may have different sizes depending on configuration.
This patch adds a feature to gen_syscalls.py to detect functions with
wide arguments and automatically generates code to split/unsplit them.
Unfortunately the current scheme of Z_SYSCALL_DECLARE_* macros won't
work with functions like this, because for N arguments (our current
maximum N is 10) there are 2^N possible configurations of argument
widths. So this generates the complete functions for each handler and
wrapper, effectively doing in python what was originally done in the
preprocessor.
Another complexity is that traditional the z_hdlr_*() function for a
system call has taken the raw list of word arguments, which does not
work when some of those arguments must be 64 bit types. So instead of
using a single Z_SYSCALL_HANDLER macro, this splits the job of
z_hdlr_*() into two steps: An automatically-generated unmarshalling
function, z_mrsh_*(), which then calls a user-supplied verification
function z_vrfy_*(). The verification function is typesafe, and is a
simple C function with exactly the same argument and return signature
as the syscall impl function. It is also not responsible for
validating the pointers to the extra parameter array or a wide return
value, that code gets automatically generated.
This commit includes new vrfy/msrh handling for all syscalls invoked
during CI runs. Future commits will port the less testable code.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
When foo.yaml set some property 'required: true' and bar.yaml set the
same property 'required: false', the check for changing
'required: false' to 'required: true' would raise an error for
include: [bar.yaml, foo.yaml]
(with that particular order due to implementation details).
The order files are included in shouldn't matter. To fix it, change the
logic so that 'required' values are ORed together between included files
(so that 'required: true' is always respected), and remove the
'required' true-to-false check when merging included files.
Keep the true-to-false check when merging the (merged) included files
into the main binding (the binding with the 'include:' in it). This
might give a good organization, and the old scripts do it too.
Piggyback two fixes/cleanups:
- 'compatible' should be allowed to appear in included files
- No need to allow an 'inherits' key in _check_binding(), because
it has been removed before then, when merging bindings
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
The 'fname' parameter to merge_included_bindings(), giving the path to
the top-level binding file, was accidentally shadowed in the
'for fname in fnames:' loop. This could lead to the wrong filename being
used in error messages.
Discovered via this pylint warning:
scripts/dts/extract_dts_includes.py:225:12: R1704: Redefining
argument with the local name 'fname' (redefined-argument-from-local)
Improve naming a bit to fix it.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
'a < b and b < c' can be simplified to 'a < b < c' in Python.
Fixes this pylint warning:
R1716: Simplify chained comparison between the operands
(chained-comparison)
Fixing pylint warnings for a CI check.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Fixes this pylint warning:
R0201: Method could be a function (no-self-use)
Another option would be to turn them into regular functions, but that'd
be a bigger change.
Piggyback a small ElfHelper.error() improvement: Passing a string to
sys.exit() prints it to stderr and exits with status 1.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Except for a few special properties like 'interrupts' and '#size-cells',
require all devicetree properties on nodes with bindings to be declared
in the binding.
This will help catch misspellings, and makes refactoring and cleanups
safer.
Suggested by Peter A. Bigot.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
When we have multi-level (ie chained interrupt controllers) Zephyr has a
schemee to encode the multi-level and IRQ values along the level's into
a single 32-bit value. This is the "IRQ" value expected by Zephyr APIs.
The encoding scheme is documented here:
doc/reference/kernel/other/interrupts.rst
Update the device tree generation to walk the interrupt levels and
generate the expected encoded value for the IRQ.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Put a type check in the offending method.
Now any scalar will be put in a list for
compatibility with following code.
Signed-off-by: Olle Norelius <norelius.olle@gmail.com>
Update Kconfiglib (and menuconfig/guiconfig, just to sync) to upstream
revision f2ce282eca, to get this commit in:
Allow preprocessor user functions to access the parsing location
Just requires making Kconfig.filename/linenr public.
'lineno' would be a more standard name, but be consistent with
MenuNode.linenr.
This can be used to give friendly errors in
scripts/kconfig/kconfigfunctions.py, e.g. for
https://github.com/zephyrproject-rtos/zephyr/pull/18752.
Some minor optimizations are included too.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
For missing optional properties, it can be handy to generate a default
value instead of no value, to cut down on #ifdefs.
Allow a default value to be specified in the binding, via a new
'default: <default value>' setting for properties in bindings.
Defaults are supported for both scalar and array types. YAML arrays are
used to specify the value for array types.
'default:' also appears in json-schema, with the same meaning.
Include misc. sanity checks, like the 'default' value matching 'type'.
The documentation changes in binding-template.yaml explain the syntax.
Suggested by Peter A. Bigot in
https://github.com/zephyrproject-rtos/zephyr/issues/17829.
Fixes: #17829
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
pylint does not like how this library works and generates spurious
warnings like
scripts/ci/get_modified_tests.py:55:13: E1121: Too many positional
arguments for function call (too-many-function-args)
scripts/ci/get_modified_tests.py:57:13: E1123: Unexpected keyword
argument '_tty_out' in function call (unexpected-keyword-arg)
These warnings are useful enough to want to have enabled in the upcoming
pylint CI check, so suppress them here.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Fixes this pylint warning:
Comparison to True should be just 'expr' (singleton-comparison)
Getting rid of pylint warnings for a CI check.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Non-empty sequences are truthy in Python, so len() can be skipped in
tests.
Fixes this pylint warning:
scripts/gitlint/zephyr_commit_rules.py:115:11: C1801: Do not use
`len(SEQUENCE)` to determine if a sequence is empty
(len-as-condition)
Fixing warnings for a CI check.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Promote a handy and often-overlooked sys.exit() feature: Passing it a
string (or any other non-int object) prints it to stderr and exits with
status 1.
See the documentation at
https://docs.python.org/3/library/sys.html#sys.exit.
This indirectly prints some errors to stderr that previously went to
stdout.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Instead of
properties:
compatible:
constraint: "foo"
, just have
compatible: "foo"
at the top level of the binding.
For backwards compatibility, the old 'properties: compatible: ...' form
is still accepted for now, and is treated the same as a single-element
'compatible:'.
The old syntax was inspired by dt-schema (though it isn't
dt-schema-compatible), which is in turn a thin wrapper around
json-schema (the idea is to transform .dts files into YAML and then
verify them).
Maybe the idea was to gradually switch the syntax over to dt-schema and
then be able to use unmodified dt-schema bindings, but dt-schema is
really a different kind of tool (a completely standalone linter), and
works very differently from our stuff (see schemas/dt-core.yaml in the
dt-schema repo to get an idea of just how differently).
Better to keep it simple.
This commit also piggybacks some clarifications to the binding template
re. '#cells:'.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
A bare 'except:' can do some annoying stuff like eat Ctrl-C (because it
catches KeyboardInterrupt). Better to use 'except Exception:' as a
catch-all.
Even better might be to catch some more specific exception, because even
'except Exception:' eats stuff like misspelled identifiers.
Fixes this pylint warning:
W0702: No exception type(s) specified (bare-except)
Fixing pylint warnings for a CI check.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Doesn't use 'self'. Fixes this pylint warning:
scripts/dts/edtlib.py:272:4: R0201: Method could be a function
(no-self-use)
Fixing pylint warnings for a CI check.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Having backwards compatibility for !include and 'constraint:' is silly
without also having backwards compatibility for 'category:', because
that forces a binding change anyway.
Add backwards compatibility for 'category:', and just print a
deprecation warning when it's used.
Also move tests for deprecated features into a dedicated
test-bindings/deprecated.yaml binding, instead of piggybacking on other
tests.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Suppress this pylint warning so that it can be enabled in the upcoming
CI check. The code is safe.
scripts/dts/dtlib.py:1904:13: W0631: Using possibly undefined loop
variable 'i' (undefined-loop-variable)
Also add some more comments to clarify _init_tokens().
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
"\w" gives a two-character string, but is iffy, because it relies on \w
not being defined as an escape sequence. r"\w" is better.
Fixes this pylint warning:
scripts/dts/devicetree.py:134:0: W1401: Anomalous backslash in
string: '\w'. String constant might be missing an r prefix.
(anomalous-backslash-in-string)
Wondering if I should exclude the old DTS scripts from the pylint CI
check, but doesn't hurt to fix it at least.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Move some property fetching and node deletion code from the DT class
over to the Node class. Reads pretty nicely, and indirectly gets rid of
two unused 'self' arguments.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
