Commit graph

5778 commits

Author SHA1 Message Date
Anas Nashif
cd1dccf9bc twister: fix json report
Fix JSON report generation and simplify it a bit.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-01-18 08:30:49 -05:00
Anas Nashif
211ef41125 twister: debug which coverage tool is being used
Add more debug messages about coverage tool being used.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-01-18 08:30:49 -05:00
Martí Bolívar
c19c6fb438 Revert "scripts: west build: default build.pristine to auto"
This reverts commit c505ca38cf.

Fixes: #31358

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-01-17 08:58:54 -06:00
Martí Bolívar
f7d33f2988 scripts: dts: gen_defines: don't include descriptions
Before we had a bindings index in the documentation, the generated
header file was (somewhat unfortunately) often our best reference for
what a particular binding or property within a binding ends up doing,
so it made good sense to put the description in the generated file.

Now that we have HTML documentation that's a bit more digestible than
the generated file, though, we can just point users at that. Do that
and remove the inline description from the generated file.

This makes it possible to put C-style multiline comments in the
descriptions themselves, which will be done in subsequent patches.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-01-16 07:59:27 -06:00
Martí Bolívar
00ffc7e7b7 devicetree: add DT_NODE_PATH()
This macro returns a node's full path, given its node identifier.

The entire path to a node is useful information for the user which can
be added to build-time error messages.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-01-16 07:59:27 -06:00
Martí Bolívar
0d5e6c13e9 boards/shields: re-work handling in cmake and west
Remove the boards and shields lists from the 'usage' target output.
That might have been readable at some point long ago in Zephyr's
history, when only a few boards were available, but right now it's
obscuring the high level targets we really want 'usage' to print.

Instead, add 'boards' and 'shields' targets which the user can run to
get those lists, and reference them from the 'usage' output. This
makes 'usage' squintable again. We use the new list_boards.py script
from the 'boards' target.

Reference the 'help' target from 'usage' as well, and drop the
recommendation that people run '--target help' from the 'west build
--help' output for the 'west build --target' option. The canonical
place to look is 'usage' now.

Use the new list_boards.py code from 'west boards' as well, which
allows us to add the board's directory as a format string key, in
addition to its name and architecture.

Keep west-completion.bash up to date. While doing that, I noticed that
a bunch of references to this file refer to a stale location, so fix
those too.

Finally, the 'usage' output is what we print for a failed board or
shield lookup, so that needs to be updated also. Handle that by
invoking boards.cmake and a new shields.cmake in CMake script mode to
print the relevant output.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-01-15 15:07:49 -05:00
Martí Bolívar
8bd6d08b0b scripts: add list_boards.py
This script is essentially a Python rewrite of the CMake code we're
using to print boards in cmake/boards.cmake, plus some extra features.

Having this in Python will simplify some later adjustments to our
'usage' build system target in ways that will make its output easier
to read, while simultaneously making 'west boards' more useful.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-01-15 15:07:49 -05:00
Erwan Gouriou
b711028539 scripts/Kconfig: Add dt_node_has_prop Kconfig function
Add dt_node_has_prop function to query the presence of 'prop'
for given node label.

Signed-off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
2021-01-15 10:29:30 -05:00
Kumar Gala
98b6e4f834 devicetree: generate extern's for devicetree struct devices
Generate a header (device_extern.h) that handles extern of possible
device structs that would come from devicetree.  This removes the need
for DEVICE_DT_DECLARE and DEVICE_DT_INST_DECLARE which we can remove.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2021-01-15 07:16:21 -06:00
Anas Nashif
5d50797dad ci: handle checpatch warnings as errors
Fail CI if we have both errors and warnings.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-01-14 17:29:37 -05:00
Anas Nashif
827ecb7bd0 twister: fix build_on_all configuration
Logic for scope of platforms to be checked was changed and this options
now only applies to limited scope, fix this and reset scope when a
testcase declares it needs to build on available platforms.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-01-14 13:59:42 -05:00
Kumar Gala
3befd7d45f cleanup: rename sanitycheck references to twister
Cleanup references to sanitycheck that should now be twister.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2021-01-14 07:50:52 -06:00
Martí Bolívar
7745f3edba west boards: add --name filter
This is an optional regular expression that can be used to filter the
output.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-01-14 12:39:21 +01:00
Marc Herbert
6d36386e13 requirements-compliance: restrict junitparser version < 2
junitparser version 2 is incompatible with check_compliance.py, it fails
like this:

File "./scripts/ci/check_compliance.py", line 295, in parse_kconfig
   self.skip("Not a Zephyr tree (ZEPHYR_BASE unset)")
File "./scripts/ci/check_compliance.py", line 141, in skip
   self.case.result = Skipped(msg, "skipped")
File "/usr/local/lib/python3.9/site-packages/junitparser/junitparser.py"
   line 682, in result
   for entry in value:

TypeError: 'Skipped' object is not iterable

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2021-01-13 07:58:37 -05:00
Marc Herbert
205dcdf636 check_compliance: fix broken format() in exception handler
Likely an accident when commit 288ae28c13 moved the script to the main
repo. Stayed unnoticed because this handler rarely ever runs.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2021-01-13 07:58:37 -05:00
Guennadi Liakhovetski
3de40b48c3 rimage: update rimage: add configuration and extended manifest
rimage dropped its "-m" parameter and switched over to using "-c"
for a configuration file, including a target name.

Add support for extended manifest for all cAVS versions.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
2021-01-11 16:10:23 -05:00
Marc Herbert
c6067e7f1c requirements-doc: add missing pkwalify
doc/CMakeLists.txt uses zephyr_module

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2021-01-11 14:51:45 -05:00
Hake Huang
d5c60ff2f1 test_version: update tset version with 12 digits
update the version with 12 digits in aligned with uniq request from

Signed-off-by: Hake Huang <hake.huang@oss.nxp.com>
2021-01-10 17:00:17 -05:00
Eugeniy Paltsev
79b3a50772 twister: BinaryHandler: call try_kill_process_by_pid in a 'with' scope
The commit
531fe89e80 (sanitycheck: use multiprcoessing instead of threads)
introduce regression for ARC nsim_hs_smp platform.

The regression is that verification on nsim_hs_smp hangs. That
happens because now we don't call try_kill_process_by_pid()
in the 'with' scope of the 'subprocess.Popen' when we spawning
BinaryHandler thread. Previously it was called via `terminate`
method of `BinaryHandler` but it was changed in 531fe89e80.
So if we can't terminate the simulator in a normal way
(which is expected for 'mdb' which is used for nsim_hs_smp
simulation) we will hang forever - we will never return from
'with' scope of the 'subprocess.Popen' as we are waiting for
process termination but the try_kill_process_by_pid() is located
latter and we never reach it.

Fix that.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
2021-01-10 15:42:48 -05:00
Anas Nashif
b2a5782570 ci: remove west extension call
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-01-08 07:49:19 -05:00
Johan Hedberg
4ba387829b edtlib: Match any parent bus when binding lacks an explicit on-bus
There are some drivers in the tree that support devices on multiple
different buses, although so far this has not been represented in
device tree using the bus concept. In order to convert these drivers &
bindings to refer to a formal bus in device tree we need to be able to
match bindings which lack an explicit "on-bus: ..." value against any
parent bus. This will also be needed for any external bindings, since
those would not be aware of on-bus (as it's a Zephyhr-specific
extension).

The two drivers I'm particularly targeting is the ns16550 UART driver
(drivers/serial/uart_ns16550.c) and the DW I2C driver
(drivers/i2c/i2c_dw.c). They both support devices with a fixed MMIO
address as well as devices connected and discovered over PCIe. The
only issue is that instead of encoding the bus information the proper
DT way these bindings use a special "pcie" property in the DT node
entries to indicate whether the node is on the PCIe bus or not.

Being able to convert the above two drivers to use the DT bus concept
allow the removal of "hacks" like this:

 if DT_INST_PROP(0, pcie) || \
       DT_INST_PROP(1, pcie) || \
       DT_INST_PROP(2, pcie) || \
       DT_INST_PROP(3, pcie)

to the more intuitive:

 if DT_ANY_INST_ON_BUS_STATUS_OKAY(pcie)

This also has the benefit that the driver doesn't need to make any
arbitrary assumptions of how many matching devices there may be but
works for any number of matches. This is already a problem now since
e.g. the ns16550 driver assumes a maximum of 4 nodes, whereas
dts/x86/elkhart_lake.dtsi defines up to 9 different ns16550 nodes.

Signed-off-by: Johan Hedberg <johan.hedberg@intel.com>
2021-01-07 20:07:12 +02:00
Anas Nashif
f1dea627cc ci: fetch sof submodules
Fetch submodules needed to build with SOF.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-01-06 07:53:46 -06:00
Martí Bolívar
a8612eb308 runners: fix nrfjprog UICR check
It's fine to flash an image with UICR data as long as the user has
given any of the --erase, --recover, or --force options. Silence the
error correctly.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-01-05 08:44:05 -06:00
Jingru Wang
b9c953c126 twister: add 12 hexadecimal digits to version
With the earlier version of Git, git describe command will
use the default 7 hexadecimal digits as the abbreviated
object name. But currently the Linux kernel project needs
11 to 12 hexdigits to stay unique.So it's necessary to add
--abbrev=12 to ensure twister always generate the same tags.

In https://testing.zephyrproject.org/daily_tests/versions.json,
all tags have 10 hexadecimal digits.

Signed-off-by: Jingru Wang <jingru@synopsys.com>
2021-01-04 08:42:40 -05:00
Fabio Utzig
7cc84af393 doc: fix broken text search
Require Sphinx<3.4.0 to avoid the issue described in:

https://github.com/sphinx-doc/sphinx/issues/8603

This requirement can be relaxed once a new sphinx-rtd-theme is
released.

Signed-off-by: Fabio Utzig <fabio.utzig@nordicsemi.no>
2021-01-04 07:57:07 -05:00
Eugeniy Paltsev
3134bc1ea0 runners: nsim: allow to use runner if gdb is missing
As of today we check for gdb presence in 'do_create' method and
raise an exception in case of gdb is missing. That makes nsim
runner unusable without gdb even for the commands which don't use
it (like 'flash' command).

Fix that.

Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
2020-12-27 18:19:10 +01:00
Martí Bolívar
38ede5aded dts: handle unspecified phandle-array elements
The DTS language permits zeroing out phandles in a phandle array to
say "there's nothing at this index", and dtlib manages that correctly,
but edtlib and gen_defines.py aren't equipped to do so.

Fix this by allowing None elements in the lists of ControllerAndData
values returned by edtlib for such properties.

Handle that in gen_defines.py by setting the generated
DT_N_<node>_P_<prop>_IDX_<i>_EXISTS macro to 0 in such cases.

The DT_N_<node>_P_<prop>_LEN macro still accounts for the entire
length of the phandle-array; it's just that some indexes may be
missing data.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-12-22 07:34:29 -06:00
Martí Bolívar
42504e8c2c dts: tests: delete extraneous whitespace
This line is required by the commit linter.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-12-22 07:34:29 -06:00
Anas Nashif
4fa82c44af twister: fix hardware map load/save
Do not store shared memory variables into yaml and check for validity of
imported hardware map before attempting to modify it.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-19 19:20:09 -05:00
Anas Nashif
5be8afcf73 ci: update versions file with weekly label
Sync with version in test_results.
Set the version to be a weekly on Mondays.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-18 07:46:47 -05:00
Jingru Wang
f20c15caaf sanitycheck: Add exception check
Add exception check for psutil when running CI with nSIM

Sometimes nSIM random failures may turn nsimdrv process into zombie
process, parent process can't terminate it by calling proc.wait(0.1)
and kill command doesn't apply to zombie process

Signed-off-by: Jingru Wang <jingru@synopsys.com>
2020-12-17 09:18:00 -05:00
Jacob Trueb
278aca926b scripts: west_commands: runners: Extend jlink timeout
With Rosetta, jlink command runs longer than 0.1 seconds.

Signed-off-by: Jacob Trueb <jtrueb@northwestern.edu>
2020-12-16 14:26:13 +01:00
Anas Nashif
4594370b46 twister: remove some leftover sanitycheck mentions
Few leftover mentions of Sanitycheck.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-14 14:13:16 -05:00
Martí Bolívar
f673e2060c edtlib: warn on non-tokenizable enums
These are likely enough defined by mistake to emit a warning for.

Adjust tests to match, tweaking the test_warnings() setup: now that
we've got several test cases, it's a bit cleaner not to have to
copy/paste the ('edtlib', WARNING, ...) part of every expected log
record tuple.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-12-13 19:25:24 -05:00
Martí Bolívar
0985849aa2 edtlib: use standard logging module
The edtlib strategy for emitting warnings is to print directly to
standard error. This in turn requires hacks to drop stored references
to stderr in various _warn_file attributes so the EDT objects can be
pickled.

In general, I think it's not really appropriate for library modules
like edtlib to be printing to stderr directly. The user should be able
to configure logging for general utility data munging modules like
this as they please, and not just deciding what file to print to.

Move this around so the standard logging module is used instead. We
can preserve backwards compatibility in gen_defines by customizing the
'edtlib' logging module behavior so it prints the exact same thing it
always has.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-12-13 19:25:24 -05:00
Peter Bigot
345da78b5b devicetree: basic support for enum tokens
Whenever a devicetree binding defines a string property whose
enumerated values are all tokenizable, generate C macros for each
property value that are the corresponding tokens.

Note that "token" is distinct from "identifier": both 'foo' and '123'
are valid tokens, but only 'foo' is a valid identifier. We permit some
strings which are not valid identifiers in anticipation that the
generalization may be useful, e.g. when defining macros that paste the
token onto a prefix that makes the whole thing an identifier.

Fixes: #21273
Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-12-13 19:25:24 -05:00
Martí Bolívar
b6dc0a24f7 edtlib: improve enum support
Add enum_tokenizable and enum_upper_tokenizable to PropertySpec. These
allow a PropertySpec to declare that it both has an enumeration of
values and all of them are strings which are "tokenizable". Don't
bother extending Property with these; the user can access the
information through Property.spec now, so the extra delegation is
unnecessary.

See the docstrings for details on what "tokenizable" means. The basic
idea is that we should be able to use the DT binding's enum values as
C 'enum' enumerators in a "reasonable way".

Add val_as_token to Property. This produces a canonical token for the
property value.

Add tests for this feature in particular and property enumerations in
general.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-12-13 19:25:24 -05:00
Martí Bolívar
88094ecfb1 edtlib: tweak Property initialization
In the 'normal' case of a property whose definition is taken from a
binding YAML file, a fair number (three at present) of the attributes
available on Property objects are directly taken from the
corresponding PropertySpec object.

Refactor the internals of how a Property gets initialized so that it
has a direct reference to its PropertySpec, and make those attributes
properties which just delegate to the PropertySpec (which in turn just
delegate to the binding). Additionally, expose the PropertySpec
directly.

This will make it easier to extend the Property class with additional
attributes that normally come from the PropertySpec without having to
touch all the locations where Property.__init__ is called.

In the case of the 'default' properties, we handle this by dummying
out some PropertySpec objects. These dummy PropertySpecs in turn
require a dummy Binding.

This change has the advantage that it improves the degree to which
these defaults are checked, e.g. it makes sure that 'status' is one of
the permitted values.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-12-13 19:25:24 -05:00
Anas Nashif
b517b1fc60 sanitycheck: add more logging
This will let us know what is going on, especially on operations that
take some time to complete.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-11 17:41:40 -05:00
Anas Nashif
2d4891725c sanitycheck: handle keyboard interrupts
Handle keyboard interrupts (Ctrl-C) and terminate gracefully.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-11 17:41:40 -05:00
Anas Nashif
a5d16ab027 sanitycheck: add option to store platform reports
Now we create plaform reports for each enabled platform, this is time
consuming, especially if you are not using those reports for anything.

Make this an option and enable only if the generated reports are going
to be used.

Individual reports can also be generated from the JSON file if needed.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-11 17:41:40 -05:00
Anas Nashif
7822e7b50d sanitycheck: retry-failed comment
Add a comment about how deal with retries when a test fails to build.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-11 17:41:40 -05:00
Martí Bolívar
8fce3004fe runners: mdb: fix imports
Runner implementations are only allowed to unconditionally import
modules in the python standard library. They are not allowed to import
anything that comes from pip or other third party sources unless they
catch the ImportError and gracefully do nothing.

Fix the imports in the mdb runner accordingly, sorting the imports
into sections to make it clearer what's stdlib, what's runners, and
what's third party while we're here.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-12-11 16:49:59 -05:00
Anas Nashif
487af47f58 twister: add a wrapper script with old name
Add a simpel wrapper script called 'sanitycheck' to be removed after
Zephyr 2.5.

This script has a warning and a 5 sec delay to get the attention of the
caller.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-11 14:13:02 -05:00
Anas Nashif
5471398ff0 modules: rename options for twister support
Zephyr modules with test support should also be using twister instead of
sanitycheck.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-11 14:13:02 -05:00
Anas Nashif
5725ed8dd1 twister: fixed pylint issues
Fixed import and unused variable warning.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-11 14:13:02 -05:00
Anas Nashif
9318e6c8f7 twister: tests: rename sanitycheck -> twister
Change all tests to use twister instead of sanitycheck.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-11 14:13:02 -05:00
Anas Nashif
a05d056943 ci: rename sanitycheck -> twister
Change all CI scripts to use the new script name: twister.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-11 14:13:02 -05:00
Anas Nashif
2e4b0fad98 twister: adapt platform schema
Adapt schema to use twister instead of sanitycheck.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-11 14:13:02 -05:00
Anas Nashif
92c9d4ba14 twister: move unit tests
Move all unittests to a new folder with the twister name.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-12-11 14:13:02 -05:00