This commit addresses the following portability issues:
1. gen_syscalls incorrectly assumes that the compiler is always GCC.
2. pragma GCC diagnostic push and pop are not supported in GCC < 4.6.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
Prompted by an upstream bug report. Nothing in Zephyr triggers this at
the moment, but might as well fix it.
Update Kconfiglib to upstream revision 7d05084b7e, to get this commit
in:
Fix handling of parentheses in macro argument values
As an oversight, there was no check for nested parentheses in macro
arguments, making the preprocessor think the call ended after
'void)' in
def_bool $(success,echo 'void foo(void) { asm inline (""); }' \
| $(CC) -x c - -c -o /dev/null)
This broke the latest linux-next kernels (with a Kconfig error),
starting with commit eb111869301e1 ("compiler-types.h: add
asm_inline definition").
I remember seeing this when going through the C code, but somehow
forgot to put it in. Fix it, and clean up _expand_macro() a bit at
the same time.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Generating generic information for 'type: phandle-array' properties in
edtlib was difficult due to defining phandle-array as just a list of
phandles and numbers. To make sense of a phandle-array property like
'pwms', you have to know that #pwm-cells is expected to appear on
each referenced controller, and that the binding for the controller has
a #cells.
Because of this, handling of various 'type: phandle-array' properties
was previously hardcoded in edtlib and exposed through properties like
Node.pwms, instead of through the generic Node.props (though with a lot
of shared code).
In practice, it turns out that all 'type: phandle-array' properties in
Zephyr work exactly the same way: They all have names that end in -s,
the 's' is removed to derive the name of related properties, and they
all look up #cells in the binding for the controller, which gives names
to the data values.
Strengthen the definition of 'type: phandle-array' to mean a property
that works exactly like the existing phandle-array properties (which
also means requiring that the name ends in -s). This removes a ton of
hardcoding from edtlib and allows new 'type: phandle-array' properties
to be added without making any code changes.
If we ever need a property type that's a list of phandles and numbers
but that doesn't follow this scheme, then we could add a separate type
for it. We should check if the standard scheme is fine first though.
The only property type for which no information is generated is now
'compound'.
There's some inconsistency in how we generate identifiers for clocks
compared to other 'type: phandle-array' properties, so keep
special-casing them for now in gen_defines.py (see the comment in
write_clocks()).
This change also enabled a bunch of other simplifications, like reusing
the ControllerAndData class for interrupts.
Piggyback generalization of *-map properties so that they work for any
phandle-array properties. It's now possible to have things like
'io-channel-map', if you need to.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
'Specifier' is devicetree specalese for data associated with interrupts,
GPIOs, etc., e.g. <1 2> and <3 4> in
pwms = <&ctrl-1 1 2 &ctrl-2 3 4>;
It's probably unnecessarily confusing to call it that. Call it 'data'
instead, which is a bit more straightforward.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
edtlib.Device is just a devicetree node augmented with binding
information and some interpretation of properties. Rename it to
edtlib.Node to make that clearer. That also avoids calling things like
flash partition nodes "devices", which is a bit confusing.
I called it edtlib.Device instead of edtlib.Node originally to avoid
confusion with dtlib.Node, but in retrospect it probably makes it more
confusing on the whole. Something like edtlib.ENode might work too, but
it's probably overkill. Clients of edtlib.py only interact with
edtlib.Node, so the only potential for confusion is within edtlib.py
itself, and it doesn't get too bad there either.
Piggyback some documentation nits, and consistently write it
"devicetree" instead of "device tree", to match the spec.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
The ESP tool is being executed directly in the esp32 runner,
assuming the tool is executable by itself. However, it would
fail under Windows as subprocess.check_call() cannot execute
Python scripts directly. The fix is to execute the Python
interpreter and passing the script path as a command line
parameter.
Fixes#19098
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
As a slightly hairy but important optimization inherited from the old
scripts, the binding loading code only looks at binding files whose raw
text contains one of the compatible strings from the devicetree. For
such files, a second pass parses the file as YAML and tries to extract a
compatible string, and skips the file if it fails (e.g. due to spurious
text matches in 'include'd binding fragments).
Until now, the binding would always get fully loaded (have 'include'd
files merged in, checks run, etc.) if the second pass managed to extract
a compatible.
Do slightly better by only fully loading the binding if the extracted
compatible from the second pass appears in the devicetree. This gets rid
of unnecessary binding loading in rare cases.
Discovered by test-bindings/deprecated.yaml getting loaded even when
everything that referenced it in test.dts was commented out, because it
happened to mention 'child-binding' in a comment.
Also add a check for YAML errors in the second pass, to be slightly more
robust. Print a warning if a file that isn't valid YAML is found.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
API oversight. This was meant to be there all along together with
Device.parent, for navigating the devicetree, but since a need for it
never came up in gen_defines.py, it got overlooked.
Devices are just devicetree nodes augmented with binding information and
some interpretation of devicetree properties. I wonder if the name
should be changed to something like edtlib.Node to make that clearer.
Calling something like a flash partition a "device" is a bit weird, as
Galak pointed out.
I think I went with Device originally to avoid confusion with
dtlib.Node, but since edtlib users don't directly interact with dtlib,
it might not be that confusing in practice.
Piggyback some documentation clarifications.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
The devicetree check in check_compliance.py in ci-tools expects the
dtlib/edtlib test suites to exit with sys.exit() (which raises
SystemExit) on test failures, and interprets Exception as an internal
error in the test suite.
testedtlib.py accidentally raised Exception on test failures, making
check_compliance.py error out and skipping the rest of the tests when
there were failures. Fix it.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Python's Sets are not deterministic. This causes the following lines to
be emitted in random order in generated/syscall_dispatch.c
extern u32_t z_mrsh_k_object_release(u32_t arg1, ...
extern u32_t z_mrsh_k_object_access_grant(u32_t arg1, ...
extern u32_t z_mrsh_k_object_alloc(u32_t arg1, ...
Change noweak to a basic list.
Reproducibility regression introduced by commit 6564974bae
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Deprecate 'sub-node:' and add a more general 'child-binding:' mechanism
to bindings. Keep supporting 'sub-node:', but print a deprecation
warning when it's used.
Like 'sub-node:', 'child-binding:' gives a binding to child nodes, but
the binding is required to be a complete binding, and is treated (and
checked) like a normal binding.
'child-binding:' can in turn contain another 'child-binding:', up to any
number of levels. This is automatic from treating it like a normal
binding, and from the code initializing parent Devices before child
Devices.
This lets nodes give bindings to grandchildren.
For example, take this devicetree fragment:
parent {
compatible = "foo";
child-1 {
grandchild-1 {
...
};
grandchild-2 {
...
};
};
child-2 {
grandchild-3 {
...
};
};
};
The binding for 'foo' could provide bindings for grandchild-1/2/3 like
this:
compatible: "foo"
# Binding for children
child-binding:
title: ...
description: ...
...
# Binding for grandchildren
child-binding:
title: ...
description: ...
properties:
...
Due to implementation issues with the old devicetree scripts, only two
levels of 'child-binding:' is supported for now. This limitation will go
away in Zephyr 2.2.
Piggyback shortening 'description:' and 'title:' in some bindings that
provide child bindings. This makes the generated header a bit neater.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Currently, string values comming from "common" test definition section
and from test-specific section are just concatenated. Suppose, we want
to define some common filter condition, and also per-test additional
criteria. Currently, that leads to following syntax:
common:
filter: TOOLCHAIN_FOO == 1
tests:
sample.net.sockets.http_get.posix:
filter: and not CONFIG_BAR
That's arguable quite adhoc, and the only way to figure it out for
most people will be to add debug logging.
This patch proposes to use the expected syntax (i.e.
"filter: not CONFIG_BAR"), and combine conditions properly based on
their semantic meaning (which also includes parans for proper
evaluation order).
Signed-off-by: Paul Sokolovsky <paul.sokolovsky@linaro.org>
For cases like zephyr-test or for other reasons the repo dir might not
be called zephyr. Just use the basename of the dir instead of assuming
its called zephyr.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Instead of
child:
bus: foo
parent:
bus: bar
, have
child-bus: foo
parent-bus: bar
'bus' is the only key that ever appears under 'child' and 'parent'.
Support the old keys for backwards compatibility, with a deprecation
warning if they're used.
Also add 'child/parent-bus' tests to the edtlib test suite. It was
untested before.
I also considered putting more stuff under 'child' and 'parent', but
there's not much point when there's just a few keys I think. Top-level
stuff is cleaner and easier to read.
I'm planning to add a 'child-binding' key a bit later (like 'sub-node',
but more flexible), and child-* is consistent with that.
Also add an unrelated test-bindings/grandchild-3.yaml that was
accidentally left out earlier.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Some DWARF symbols for members of template classes members such as
numeric_limits<unsigned int> reference are static constexpr values
that do not have a data member location. Avoid attempting to
dereference the value for that attribute when it isn't present.
Signed-off-by: Peter A. Bigot <pab@pabigot.com>
Removed redundant linker flag --coverage from native posix
EXTRA_LDFLAGS.
The correct linker flags will be set by defining the Kconfig flag
CONFIG_COVERAGE, either by sanitcheck it self or via the test/sample
prj.conf.
Signed-off-by: Jan Van Winkel <jan.van_winkel@dxplore.eu>
Some confluence of recent changes resulted in builds with
application-specific bindings being unable to find bindings present in
the system directory. Add quotes and splits as necessary to propagate
multiple directories through the system.
Signed-off-by: Peter A. Bigot <pab@pabigot.com>
- Remove unused variables and an unused 'sys' import
- Simplify 'if len(foo) != 0' to 'if foo'. Non-empty lists/dicts/etc.
are truthy in Python.
- Use a raw string to fix this warning:
scripts/footprint/size_report:270:0: W1401: Anomalous backslash
in string: '\.'. String constant might be missing an r prefix.
(anomalous-backslash-in-string)
The issue is that '\.' just happens to work due to not being
recognized as an escape sequence.
Escape sequences are not interpreted in raw strings, so they're safer
for regexes.
- Replace 'is 0' with '== 0'. 'is' is for testing object identity, and
0 isn't guaranteed to be a unique object (but always is in practice).
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
As we work on removing the .conf file that is generated by the dt
scripts, mark dt_{int,hex,str}_val functions as deprecated.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Convert how we get the various chosen properties like "zephyr,console"
to use the new kconfig functions like dt_chosen_to_label.
Because of how kconfig parses things we define a set of variables of the
form DT_CHOSEN_Z_<PROP> since comma's are parsed as field seperators in
macros.
This conversion allows us to remove code in gen_defines.py for the
following chosen properties:
zephyr,console
zephyr,shell-uart
zephyr,bt-uart
zephyr,uart-pipe
zephyr,bt-mon-uart
zephyr,uart-mcumgr
zephyr,bt-c2h-uart
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Things that parse generated_dts_board.conf can't deal with entries
like:
DT_GPIO_KEYS_BUTTON_1_GPIOS={"GPIO_0", 14, 256}
so keep them from being added there.
Signed-off-by: Peter A. Bigot <pab@pabigot.com>
When a phandle-array (compound) has multiple members generate a define
that says how many there are; generate an initializer for each
individual member; and generate an initializer for the sequence of
members.
This allows drivers that expect multiple values in a compound to process
them without horrendous conditional compilation code attempting to
detect the number of elements in the compound.
It also eliminates the need to repeat the long prefix when initializing
a structure with the fields of a single compound.
Signed-off-by: Peter A. Bigot <pab@pabigot.com>
The lowest level output function is the one that determines the full
name of the macro, including DT_ prefix. Return the name of the macro
that provides the value for an identifier so that it can be used in
higher layers of the generator.
Signed-off-by: Peter A. Bigot <pab@pabigot.com>
Add str2str to make the conversion of a string into a C literal with all
necessary escapes and enclosing double quotes available outside a
function that emits a define.
Signed-off-by: Peter A. Bigot <pab@pabigot.com>
System call arguments, at the arch layer, are single words. So
passing wider values requires splitting them into two registers at
call time. This gets even more complicated for values (e.g
k_timeout_t) that may have different sizes depending on configuration.
This patch adds a feature to gen_syscalls.py to detect functions with
wide arguments and automatically generates code to split/unsplit them.
Unfortunately the current scheme of Z_SYSCALL_DECLARE_* macros won't
work with functions like this, because for N arguments (our current
maximum N is 10) there are 2^N possible configurations of argument
widths. So this generates the complete functions for each handler and
wrapper, effectively doing in python what was originally done in the
preprocessor.
Another complexity is that traditional the z_hdlr_*() function for a
system call has taken the raw list of word arguments, which does not
work when some of those arguments must be 64 bit types. So instead of
using a single Z_SYSCALL_HANDLER macro, this splits the job of
z_hdlr_*() into two steps: An automatically-generated unmarshalling
function, z_mrsh_*(), which then calls a user-supplied verification
function z_vrfy_*(). The verification function is typesafe, and is a
simple C function with exactly the same argument and return signature
as the syscall impl function. It is also not responsible for
validating the pointers to the extra parameter array or a wide return
value, that code gets automatically generated.
This commit includes new vrfy/msrh handling for all syscalls invoked
during CI runs. Future commits will port the less testable code.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
When foo.yaml set some property 'required: true' and bar.yaml set the
same property 'required: false', the check for changing
'required: false' to 'required: true' would raise an error for
include: [bar.yaml, foo.yaml]
(with that particular order due to implementation details).
The order files are included in shouldn't matter. To fix it, change the
logic so that 'required' values are ORed together between included files
(so that 'required: true' is always respected), and remove the
'required' true-to-false check when merging included files.
Keep the true-to-false check when merging the (merged) included files
into the main binding (the binding with the 'include:' in it). This
might give a good organization, and the old scripts do it too.
Piggyback two fixes/cleanups:
- 'compatible' should be allowed to appear in included files
- No need to allow an 'inherits' key in _check_binding(), because
it has been removed before then, when merging bindings
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
The 'fname' parameter to merge_included_bindings(), giving the path to
the top-level binding file, was accidentally shadowed in the
'for fname in fnames:' loop. This could lead to the wrong filename being
used in error messages.
Discovered via this pylint warning:
scripts/dts/extract_dts_includes.py:225:12: R1704: Redefining
argument with the local name 'fname' (redefined-argument-from-local)
Improve naming a bit to fix it.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
'a < b and b < c' can be simplified to 'a < b < c' in Python.
Fixes this pylint warning:
R1716: Simplify chained comparison between the operands
(chained-comparison)
Fixing pylint warnings for a CI check.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Fixes this pylint warning:
R0201: Method could be a function (no-self-use)
Another option would be to turn them into regular functions, but that'd
be a bigger change.
Piggyback a small ElfHelper.error() improvement: Passing a string to
sys.exit() prints it to stderr and exits with status 1.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Except for a few special properties like 'interrupts' and '#size-cells',
require all devicetree properties on nodes with bindings to be declared
in the binding.
This will help catch misspellings, and makes refactoring and cleanups
safer.
Suggested by Peter A. Bigot.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
When we have multi-level (ie chained interrupt controllers) Zephyr has a
schemee to encode the multi-level and IRQ values along the level's into
a single 32-bit value. This is the "IRQ" value expected by Zephyr APIs.
The encoding scheme is documented here:
doc/reference/kernel/other/interrupts.rst
Update the device tree generation to walk the interrupt levels and
generate the expected encoded value for the IRQ.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Put a type check in the offending method.
Now any scalar will be put in a list for
compatibility with following code.
Signed-off-by: Olle Norelius <norelius.olle@gmail.com>
Update Kconfiglib (and menuconfig/guiconfig, just to sync) to upstream
revision f2ce282eca, to get this commit in:
Allow preprocessor user functions to access the parsing location
Just requires making Kconfig.filename/linenr public.
'lineno' would be a more standard name, but be consistent with
MenuNode.linenr.
This can be used to give friendly errors in
scripts/kconfig/kconfigfunctions.py, e.g. for
https://github.com/zephyrproject-rtos/zephyr/pull/18752.
Some minor optimizations are included too.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
For missing optional properties, it can be handy to generate a default
value instead of no value, to cut down on #ifdefs.
Allow a default value to be specified in the binding, via a new
'default: <default value>' setting for properties in bindings.
Defaults are supported for both scalar and array types. YAML arrays are
used to specify the value for array types.
'default:' also appears in json-schema, with the same meaning.
Include misc. sanity checks, like the 'default' value matching 'type'.
The documentation changes in binding-template.yaml explain the syntax.
Suggested by Peter A. Bigot in
https://github.com/zephyrproject-rtos/zephyr/issues/17829.
Fixes: #17829
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
pylint does not like how this library works and generates spurious
warnings like
scripts/ci/get_modified_tests.py:55:13: E1121: Too many positional
arguments for function call (too-many-function-args)
scripts/ci/get_modified_tests.py:57:13: E1123: Unexpected keyword
argument '_tty_out' in function call (unexpected-keyword-arg)
These warnings are useful enough to want to have enabled in the upcoming
pylint CI check, so suppress them here.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Fixes this pylint warning:
Comparison to True should be just 'expr' (singleton-comparison)
Getting rid of pylint warnings for a CI check.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Non-empty sequences are truthy in Python, so len() can be skipped in
tests.
Fixes this pylint warning:
scripts/gitlint/zephyr_commit_rules.py:115:11: C1801: Do not use
`len(SEQUENCE)` to determine if a sequence is empty
(len-as-condition)
Fixing warnings for a CI check.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Promote a handy and often-overlooked sys.exit() feature: Passing it a
string (or any other non-int object) prints it to stderr and exits with
status 1.
See the documentation at
https://docs.python.org/3/library/sys.html#sys.exit.
This indirectly prints some errors to stderr that previously went to
stdout.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Instead of
properties:
compatible:
constraint: "foo"
, just have
compatible: "foo"
at the top level of the binding.
For backwards compatibility, the old 'properties: compatible: ...' form
is still accepted for now, and is treated the same as a single-element
'compatible:'.
The old syntax was inspired by dt-schema (though it isn't
dt-schema-compatible), which is in turn a thin wrapper around
json-schema (the idea is to transform .dts files into YAML and then
verify them).
Maybe the idea was to gradually switch the syntax over to dt-schema and
then be able to use unmodified dt-schema bindings, but dt-schema is
really a different kind of tool (a completely standalone linter), and
works very differently from our stuff (see schemas/dt-core.yaml in the
dt-schema repo to get an idea of just how differently).
Better to keep it simple.
This commit also piggybacks some clarifications to the binding template
re. '#cells:'.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
