sanitycheck now requires yaml.FullLoader to support generating
hardware maps. This feature was introduced in PyYAML 5.1.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
Add function that will return 'y' or 'n' if a node pointed to by a
chosen property exists and is enabled.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
All users of dt kconfig symbols for tests and samples are using dt
functions instead so we can remove the support for parsing the
generated file.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Add the following functions to allow filtering based on device tree
dt_compat_enabled(compat) - Returns true if a device tree node
compatible matches 'compat' and the node is enabled.
dt_alias_exists(alias) - Returns true if a device tree node exists with
'alias' and the node is enabled.
dt_compat_enabled_with_alias - Returns true if a device tree node
compatible matches 'compat' and the node has 'alias' and the node is
enabled.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Just calls through to concurrent.futures.Executor.shutdown() in the base
class, which has the same signature. Removing it means the base class
version will get used directly.
Fixes this pylint warning:
W0235: Useless super delegation in method 'shutdown'
(useless-super-delegation)
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Fixes this pylint warning:
R0201: Method could be a function (no-self-use)
Could also make it a class method.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
We try to demonstrate some concepts for user mode:
- Multiple logical applications, each with their own memory
domain
- Creation of a sys_mem_pool and assignment to a memory
partition
- Use of APIs like k_queue_alloc_append() which require
thread resource pools to be configured
- Management of permissions for kernel objects and drivers
- Show how application-specific system calls are defined
- Show IPC between ISR and application (using k_msgq) and
application-to-application IPC (using k_queue)
Fixes: #14683
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Multi-line comments were stuck as-is between /* and */ in the generated
header, which looks ugly and confusing e.g. for multi-line
binding/property descriptions.
Use this format for multi-line comments in the header instead:
/*
* First line
* Second line
*
* Line after blank line
*/
Also clean up the output a bit by turning some things that were separate
comments into a single multi-line comment. Add some air and reduce line
lengths too.
Before:
/* Generated by gen_defines.py */
/* DTS input file: hifive1.dts.pre.tmp */
/* Directories with bindings: $ZEPHYR_BASE/dts/bindings */
/* Devicetree node: /cpus/cpu@0/interrupt-controller */
/* Binding (compatible = riscv,cpu-intc): $ZEPHYR_BASE/... */
/* Binding description: This binding describes the RISC-V ...
Some extra lines
for testing */
#define DT_INST_0_RISCV_CPU_INTC 1
After:
/*
* Generated by gen_defines.py
*
* DTS input file:
* hifive1.dts.pre.tmp
*
* Directories with bindings:
* $ZEPHYR_BASE/dts/bindings
*/
/*
* Devicetree node:
* /cpus/cpu@0/interrupt-controller
*
* Binding (compatible = riscv,cpu-intc):
* $ZEPHYR_BASE/dts/bindings/interrupt-controller/...
*
* Description:
* This binding describes the RISC-V CPU Interrupt Controller
*
* Some extra lines
* for testing
*/
#define DT_INST_0_RISCV_CPU_INTC 1
Also tweak Node.description and Property.description in edtlib to be
strip()ed instead of rstrip()ed. There's probably no reason to
preserving leading whitespace in them either.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Use the LibYAML-based yaml.CLoader if available instead of yaml.Loader,
which is written in Python and slow. See
https://pyyaml.org/wiki/PyYAMLDocumentation.
This speeds up gen_defines.py from 0.2s to 0.07s on my system, for
-DBOARD=hifive1. It should also make scripts/kconfig/kconfig.py faster,
because it indirectly uses edtlib via
scripts/kconfig/kconfigfunctions.py.
yaml.CLoader seems to be available out of the box when installing with
pip on Ubuntu at least.
Helps with https://github.com/zephyrproject-rtos/zephyr/issues/20104.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Simplify use of property values that have multiple elements by
providing all of them in an initializer list, in addition to each one
as an enumerated value.
For example if a driver requires a sequence of operations with
instance-specific delays between stages the durations can be specified
with:
delays = <30000 20 45000>;
and the driver can use:
static u32_t delays[] = DT_INST_0_COMPAT_DELAYS;
rather than enumerating the instances. This is particularly useful
when the number of entries in the array varies per instance, in which
case such an initialization is not easily expressed in code.
Signed-off-by: Peter A. Bigot <pab@pabigot.com>
Replace:
dt_chosen_reg_addr
dt_chosen_reg_size
dt_node_reg_addr
dt_node_reg_size
with:
dt_chosen_reg_addr_int
dt_chosen_reg_size_int
dt_chosen_reg_addr_hex
dt_chosen_reg_size_hex
dt_node_reg_addr_int
dt_node_reg_size_int
dt_node_reg_addr_hex
dt_node_reg_size_hex
So that we get the proper formatted string for the type of symbol.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
save_reports should be one of the last tasks executed because it
closes the log file. Withouth it, other functions that use debug
functions like info and error will try to write into a close file.
This fixes the following problem:
sanitycheck -x=USE_CCACHE=0 -p native_posix -T samples/hello_world/ -b
-N --log-file sanity.log
JOBS: 8
Building initial testcase list...
1 test configurations selected, 0 configurations discarded due to filters.
Adding tasks to the queue...
total complete: 1/ 1 100% skipped: 0, failed: 0
1 of 1 tests passed (100.00%), 0 failed, 0 skipped with 0 warnings in
2.91 seconds
Traceback (most recent call last):
File "./zephyr/scripts/sanitycheck", line 3866, in <module>
main()
File "./zephyr/scripts/sanitycheck", line 3854, in main
suite.summary(options.disable_unrecognized_section_test)
File "./zephyr/scripts/sanitycheck", line 2306, in summary
self.duration))
File "./zephyr/scripts/sanitycheck", line 432, in info
log_file.write(what + "\n")
ValueError: I/O operation on closed file.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
Using exact versions for our PyPI dependencies (i.e. requirements.txt
lines that look like "dependency==X.Y.Z") is rude since we're asking
users to install these dependencies outside of any virtual
environment. This causes conflicts with other packages which may
depend on the same things as us.
Use inclusive ordered comparison (>=) release specifiers instead:
https://www.python.org/dev/peps/pep-0440/#inclusive-ordered-comparison
If this causes problems for individual packages, we can refine them
using compatible release specifiers (~=):
https://www.python.org/dev/peps/pep-0440/#compatible-release
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Script to be used when creating a release. For regular releases this can
be called this way:
$ list_issues.py -f issues.md -s 2019-09-01
Will list all closed issues since september 1st, 2019 and will create a
markdown file with all issues that can be added as is to the release
notes.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
The lvgl library is configured in some tests to use
the libC malloc to allocate heap. In native_posix this
ends up in the underlying OS heap, which cannot be safely
freed in general when the program finnishes, and
ends up being reported by valgrind as leaked on exit. See
https://github.com/zephyrproject-rtos/zephyr/pull/18471
for a longer discussion.
Instead of trying to fix the leak, let's just accept it
but suppress the errors so it does not confuse other
developers
Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
Failed unit tests were setting wrong fail string (error instead of
failed) which made unit tests always pass.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Set threads spawned by BinaryHandler dameon, so when the main thread
exits for some unusual reason (e.g SIGINT) the child threads are
automatically killed.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
os.mkdir() is not suitable to create arbitrary directory path (can
create only a subdir of an existing dir, will error out if already
exists), os.makedirs() should be always used in such cases.
Signed-off-by: Paul Sokolovsky <paul.sokolovsky@linaro.org>
Previously, if two bindings had the same 'compatible:'/'parent-bus:'
values, the binding that happened to be loaded last would get used.
Turn it into an error instead. This avoids tricking people into thinking
that bindings get loaded in a defined order.
Maybe overriding bindings could be allowed later, if we need it.
Fixes: #19536
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Simplifies the code a bit.
Looks like the description wasn't rstrip()ed when it came from a
'child-binding:' either. This also indirectly fixes that.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Make sure we cleanup the manifest files at the right spot of the CI
process. When not building a PR, behavior is different.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
A complete overhaul of the sanitycheck script and how we build and run
tests. This new version of sanitycheck uses python for job distribution
and drop use of Make.
In addition to the move to python threading library, the following has
been changed:
- All handlers now run in parallel, meaning that any simulator will run
in parallel and when testing on multiple devices (using
--device-testing) the tests are run in parallel.
- Lexicial filtering (using the filter keyword in yaml files) is now
evaluated at runtime and is no long being pre-processed. This will allow
us to immediately start executing tests and skip the wait time that was
needed for filtering.
- Device testing now supports multiple devices connected at the same
time and is managed using a hardware map that needs to be generated and
maintained for every test environment. (using --generate-hardware-map
option).
- Reports are not long stored in the Zephyr tree and instead stored in
the output directory where all build artifacts are generated.
- Each tested target now has a junit report in the output directory.
- Recording option for performance data and other metrics is now
available. This will allow us to record the output from the console and
store the data for later processing. For example benchmark data can be
captured and uploaded to a tracking server.
- Test configurations (or instances) are no longer being sorted, this
will help with balancing the load when we run sanitycheck on multiple
hosts (as we do in CI).
And many other cleanups and improvements...
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
k_thread_create and K_THREAD_DEFINE both take a delay as the final
parameter. Most uses of K_THREAD_DEFINE pass either `K_NO_WAIT` or
`K_FOREVER`. Ensure that all uses of K_THREAD_DEFINE follow that
practice, and that the runtime k_thread_create calls do so as well.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
k_sleep uses the same underlying thread infrastructure as the other
functions that take timeouts, so the delay should be specified as a
timeout rather than milliseconds.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
Sort the functions within the regular expression so they can be
checked more easily.
Remove k_thread_deadline_set as it takes an argument in cycles. (The
one in-tree call to this function was not affected by this error.)
Add missed k_mbox_data_block_get.
Fix an overly ambitious multi-match disjunct that covered some
non-existent functions.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
Add support for the report and patch modes so this can be invoked by
coccicheck.
Use PCRE options to make the kernel timeout API identifier rule more
readable. Extend the pattern to new API.
Use rule extends and depends clauses, and pattern disjunction, to
avoid replicating metavariable content.
Hint that using --include-headers may be helpful (some patterns can be
found in static inline functions).
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
With multiple debug probes attached, attempting to launch multiple debug
servers resulted in "OSError: [Errno 98] Address already in use" despite
explicitly setting --gdb-port to unique values.
The issue was caused by the default telnet port: 4444. Adding
--telnet-port parameter allows to explicitly define the address to a
unique value and avoid the socket exception.
Signed-off-by: Rihards Skuja <rihardssk@mikrotik.com>
Node._prop_val() returned too early for non-existent booleans, letting
missing 'required: true' booleans slip through without an error.
Fix it by rearranging the code to always do the 'required' check before
returning.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Requested by Marc Herbert. Makes the output deterministic as long as all
binding directories are within $ZEPHYR_BASE (and a bit less spammy too).
Example output for header:
Before:
/* Directories with bindings: /home/ulf/z/z/dts/bindings */
...
/* Binding (...): /home/ulf/.../arm,v7m-nvic.yaml */
After:
/* Directories with bindings: $ZEPHYR_BASE/dts/bindings */
...
/* Binding (...): $ZEPHYR_BASE/dts/.../arm,v7m-nvic.yaml */
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Some legacy code still passes integer literals in milliseconds as the
value to functions that take a timeout. This usage interferes with
plans to replace the millisecond representation with a more generic
k_timeout_t value. Add a Coccinelle script to convert call sites to
use the proper constants and macros.
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
verify_eq() can be used instead of verify_streq(), since
warnings.getvalue() already returns a string.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
With ST boards it is possible to specify the board ID when flashing using
openocd. This is very useful when having multiple devices connected.
This change allows us to address a device directly:
west flash -- --cmd-pre-init "hla_serial 066BFF535254887767174558"
This needs to be called before init, hence the new option.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Add a 'warn_file' parameter to EDT.__init__() that gives a 'file' object
to write warnings to. Use it to capture and verify warnings generated
for deprecated features in testedtlib.py. This indirectly gets rid of
possibly broken-looking output when running it.
Because any function that writes warnings now needs to use EDT._warn()
(as self._warn()), some functions were moved into the EDT class.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Implement a nice generalization suggested by Bobby Noelte.
Instead of having a generic #cells key in bindings, have source-specific
*-cells keys. Some examples:
interrupt-cells:
- irq
- priority
- flags
gpio-cells:
- pin
- flags
pwm-cells:
- channel
- period
This makes bindings a bit easier to read, and allows a node to be a
controller for many different 'phandle-array' properties.
The prefix before *-cells is derived from the property name, meaning
there's no fixed set of *-cells keys. This is possible because of the
earlier 'phandle-array' generalization.
The older #cells key is supported for backwards compatibility, but
generates a deprecation warning.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Fix a bug where DTS_BINDINGS_DIRS could only have one entry. When
there were more than one entry the command for invoking menuconfig
became corrupted.
This changes the separator of DTS_BINDINGS_DIR from a space to ? so
that the shell does not interpret the space as an argument separator.
Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
This commit addresses the following portability issues:
1. gen_syscalls incorrectly assumes that the compiler is always GCC.
2. pragma GCC diagnostic push and pop are not supported in GCC < 4.6.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
Prompted by an upstream bug report. Nothing in Zephyr triggers this at
the moment, but might as well fix it.
Update Kconfiglib to upstream revision 7d05084b7e, to get this commit
in:
Fix handling of parentheses in macro argument values
As an oversight, there was no check for nested parentheses in macro
arguments, making the preprocessor think the call ended after
'void)' in
def_bool $(success,echo 'void foo(void) { asm inline (""); }' \
| $(CC) -x c - -c -o /dev/null)
This broke the latest linux-next kernels (with a Kconfig error),
starting with commit eb111869301e1 ("compiler-types.h: add
asm_inline definition").
I remember seeing this when going through the C code, but somehow
forgot to put it in. Fix it, and clean up _expand_macro() a bit at
the same time.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Generating generic information for 'type: phandle-array' properties in
edtlib was difficult due to defining phandle-array as just a list of
phandles and numbers. To make sense of a phandle-array property like
'pwms', you have to know that #pwm-cells is expected to appear on
each referenced controller, and that the binding for the controller has
a #cells.
Because of this, handling of various 'type: phandle-array' properties
was previously hardcoded in edtlib and exposed through properties like
Node.pwms, instead of through the generic Node.props (though with a lot
of shared code).
In practice, it turns out that all 'type: phandle-array' properties in
Zephyr work exactly the same way: They all have names that end in -s,
the 's' is removed to derive the name of related properties, and they
all look up #cells in the binding for the controller, which gives names
to the data values.
Strengthen the definition of 'type: phandle-array' to mean a property
that works exactly like the existing phandle-array properties (which
also means requiring that the name ends in -s). This removes a ton of
hardcoding from edtlib and allows new 'type: phandle-array' properties
to be added without making any code changes.
If we ever need a property type that's a list of phandles and numbers
but that doesn't follow this scheme, then we could add a separate type
for it. We should check if the standard scheme is fine first though.
The only property type for which no information is generated is now
'compound'.
There's some inconsistency in how we generate identifiers for clocks
compared to other 'type: phandle-array' properties, so keep
special-casing them for now in gen_defines.py (see the comment in
write_clocks()).
This change also enabled a bunch of other simplifications, like reusing
the ControllerAndData class for interrupts.
Piggyback generalization of *-map properties so that they work for any
phandle-array properties. It's now possible to have things like
'io-channel-map', if you need to.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
'Specifier' is devicetree specalese for data associated with interrupts,
GPIOs, etc., e.g. <1 2> and <3 4> in
pwms = <&ctrl-1 1 2 &ctrl-2 3 4>;
It's probably unnecessarily confusing to call it that. Call it 'data'
instead, which is a bit more straightforward.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
edtlib.Device is just a devicetree node augmented with binding
information and some interpretation of properties. Rename it to
edtlib.Node to make that clearer. That also avoids calling things like
flash partition nodes "devices", which is a bit confusing.
I called it edtlib.Device instead of edtlib.Node originally to avoid
confusion with dtlib.Node, but in retrospect it probably makes it more
confusing on the whole. Something like edtlib.ENode might work too, but
it's probably overkill. Clients of edtlib.py only interact with
edtlib.Node, so the only potential for confusion is within edtlib.py
itself, and it doesn't get too bad there either.
Piggyback some documentation nits, and consistently write it
"devicetree" instead of "device tree", to match the spec.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
The ESP tool is being executed directly in the esp32 runner,
assuming the tool is executable by itself. However, it would
fail under Windows as subprocess.check_call() cannot execute
Python scripts directly. The fix is to execute the Python
interpreter and passing the script path as a command line
parameter.
Fixes#19098
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
