Commit graph

2587 commits

Author SHA1 Message Date
Andrzej Puzdrowski ad6215dcf7 manifest: upgrade mcuboot
Include build assert macros rework.
Update pip requirements by cbor package required by MCUBoot since
https://github.com/JuulLabs-OSS/mcuboot/pull/674 was merge.

Signed-off-by: Andrzej Puzdrowski <andrzej.puzdrowski@nordicsemi.no>
2020-03-31 07:18:06 +02:00
Anas Nashif f6462a3a8c sanitycheck: get rid of global VERBOSE
Do not declare VERBOSE as global, instead pass verbosity as argument.
Also get rid of options as global and fix coverage class to not use
global option variable.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-03-30 09:59:42 -04:00
Anas Nashif 13d0c03761 sanitycheck: remove unused arg files
We do not use those arg files anymore, so remove them to avoid
confusion and for the sake of cleanup.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-03-30 09:59:42 -04:00
Anas Nashif 017ce9fa2e sanitycheck: fix testsuite
Import sanitycheck classes from new module.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-03-30 09:59:42 -04:00
Anas Nashif b366949498 sanitycheck: fix issue with generator setting
Do not set generator on None handler.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-03-30 09:59:42 -04:00
Anas Nashif ce2b418027 sanitycheck: split main script into an exe and classes
Move most of the classes to sanitycheck.py under sanity_check directory.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-03-30 09:59:42 -04:00
Anas Nashif 1f6cdbe70a sanitycheck: fix scope and cleanup
Reorder classes to be on top of loose functions. Move a few globals into
the related classes where they are being used.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-03-30 09:59:42 -04:00
Anas Nashif 507e75b9b5 sanitycheck: remove more global usage of arg options
This will make it easier for us to test classes and functions without
having arg options in the middle.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-03-30 09:59:42 -04:00
Anas Nashif 64a6c9e3df sanitycheck: use cmake_only from class
Do not use option from args, use class settings instead.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-03-30 09:59:42 -04:00
Anas Nashif 6667dc35e7 sanitycheck: catch build failures
CMake is not returning any error codes on build failures so those go
undetected in some cases. Handle the case where we get no output at all
from cmake and deal with that as a failure.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-03-30 10:33:21 +02:00
Torsten Rasmussen e819fa46cd scripts: west-commands: Updates needed when removing ZEPHYR_BASE env
This commit includes the following fixes in order to remove environment
setting of ZEPHYR_BASE is west extension commands.

- Build command
  west build --pristine will now use the ZEPHYR_BASE variable found
  in CMakeCache.txt in the build folder.

  This ensures that the pristine command is executed from the same
  Zephyr that was used for compilation.

- Board command
  The west boards command no longer sets Zephyr base before invoking
  cmake -P cmake/boards.cmake

  Instead boards.cmake uses find_package(Zephyr) to ensure consistent
  behavior with Zephyr samples, so that the detection of Zephyr base is
  uniform across CMake commands.

  It also changes BOARD_ROOT_SPACE_SEPARATED to BOARD_ROOT in order to
  be consistent with existing user documentation.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-03-27 16:23:46 +01:00
Torsten Rasmussen e85cfe4554 cmake: West extension command for Zephyr config package registration
This command make it possible to use west for first time registration
of a Zephyr config package in CMake.

To register Zephyr as a CMake config package, invoke:
west zephyr-export

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2020-03-27 16:23:46 +01:00
Torsten Rasmussen 39cd4c8f52 cmake: propagating zephyr base to west
West utilizes Zephyr base when invoked out-of-tree in order to determine
west topdir.

This commit ensures that zephyr base when invoking west from CMake is
set to current zephyr base.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2020-03-27 16:23:46 +01:00
Torsten Rasmussen d7862cf776 cmake: using ${ZEPHYR_BASE} instead of $ENV{ZEPHYR_BASE}
With the introduction of ZephyrConfig.cmake all parts of CMake code
should rely on the CMake ZEPHYR_BASE variable instead of the environment
setting.

This ensures that after the first CMake invocation, then all subsequent
invocation in same build folder will use same zephyr base.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2020-03-27 16:23:46 +01:00
Kumar Gala 299bfd0553 scripts: dts: gen_defines: Fix issue if there are no compats
We get the following error:

  ValueError: max() arg is an empty sequence

if the compatiable section of the device tree is empty or doesn't exist.
Fix this by havingin max_len get a default value of 0.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-03-26 03:22:49 -05:00
Martí Bolívar a5268773a6 scripts: run_common: don't dump stack on unavailable runner
Originally reported in #23539 (though that seems to have been another
problem), west flash and friends are dumping stack when used with an
unconfigured runner.

Let's just promote the warning about this to an error. The idea that
this ever could have worked without explicit support has not worked
out in practice, to my knowledge.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-03-25 19:43:30 +01:00
Patrik Flykt c99392f2d6 scripts: net: Add TLS test for mqtt_publisher
In addition to the existing plain TCP MQTT test, add the same test
with TLS applied. Robert Lubos provided the updated test_cert.h, which
contains the same certificate as in the echo samples.

Signed-off-by: Patrik Flykt <patrik.flykt@intel.com>
2020-03-25 08:39:39 +02:00
Aastha Grover e27cf15763 scripts: tests: sanitycheck: Add basic foundation for sanitycheck testsuite
This commit adds basic testcases for sanitycheck tool using pytest.
Coverage for the sanitycheck tool is obtained using coverage tool.
Instructions are included in the README.md in
scripts/tests/sanitycheck directory.

Signed-off-by: Aastha Grover <aastha.grover@intel.com>
2020-03-24 22:32:26 -04:00
Anas Nashif 56993fb9ab sanitycheck: remove confusing message when loading files
This message was intended for retries only, not every time we loaded a
file with a test manifest.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-03-24 12:15:36 -04:00
Anas Nashif c7092a1c4d ci: when merging files, keep one header row
When merging files, make sure to remove header rows from each file and
just keep one header. This will stop us from parsing header rows as
tests.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-03-24 12:15:36 -04:00
Anas Nashif 923b3bf0ae sanitycheck: Add named exceptions
Add named exceptions instead of a generic one. Should help point to the
right issue when something goes wrong.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-03-24 12:15:36 -04:00
Martí Bolívar dc85edd2e9 scripts: dts: write new gen_defines.py
This is joint work with Kumar Gala (see signed-off-by).

This supports a new devicetree macro syntax coming. It's not really
worth mixing up the old and the new generation scripts into one file,
because:

- we aim to remove support for the old macros at some point, so it
  will be cleaner to start fresh with a new script based on the old one
  that only generates the new syntax

- it will avoid regressions to leave the existing code alone while
  we're moving users to the new names

Keep the existing script by moving it to gen_legacy_defines.py and
changing a few comments and strings around. It's responsible for
generating:

- devicetree.conf: only needed by deprecated kconfigfunctions
- devicetree_legacy_unfixed.h: "old" devicetree_unfixed.h macros

Put a new gen_defines.py in its place. It generates:

- zephyr.dts
- devicetree_unfixed.h in the new syntax

Include devicetree_legacy_unfixed.h from devicetree.h so no DT users
are affected by this change.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-03-24 10:11:20 -05:00
Kumar Gala 8af311ab3b edtlib: allow register addr / size to be None
If the #address-cells property for a register is 0 than we set the addr
value of the reg to None.  Similar, if #size-cells is 0 than we set the
size value to None for the reg.

Fixup kconfigfunctions.py to handle reg.size and reg.addr being None.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-03-24 10:11:20 -05:00
Kumar Gala c7f2c89783 gen_defines.py: rename gen_defines.py to gen_legacy_defines.py
This rename is mostly to easy git managment and review so any changes or
the addition of the new gen_defines.py doesn't look like a diff against
the old code if you look at just that commit.

We keep changes to a minimum to just keep things building with the
renamed gen_legacy_defines.py.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-03-24 10:11:20 -05:00
Martí Bolívar 0de9a08e94 scripts: dts: edtlib: add EDT.chosen_nodes property
This returns the entire logical {name: Node} dictionary which is
currently being accessed element by element via chosen_node(name).

It will be used in a new gen_defines.py for moving the handling of
chosen nodes into C from Python.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-03-24 10:11:20 -05:00
Anas Nashif 2f7c187d77 sanitycheck: cleanup loading of previous results
Merge two functions loading previous results into one and support
filtering based on status.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-03-24 08:11:07 -04:00
Anas Nashif a45c1da6d7 sanitycheck: use status column for status
We used the column 'passed' as a boolean to signify pass or fail,
however we do have other states that need to be tracked.
Remove the boolean and use a text field instead that has the status as a
string.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-03-24 08:11:07 -04:00
Anas Nashif 49f6d5a6c6 sanitycheck: fix counting when using --test option
When running 1 specific test, counting was off. Combine functions adding
tests into one and optimize filtering.

Fixes #22270

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-03-24 08:10:44 -04:00
Ioannis Glaropoulos f3807f19dc boards: arm: nrf51_pca10028: rename board to nrf51dk_nrf51422
We rename the nRF51 Dev Kit board target (nrf51_pca10028)
to nrf51dk_nrf51422. We update all associated references
in the supportive documentation and all nRF51-related
cofigurations and overlay files in the samples and tests
in the tree.

Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
2020-03-24 11:45:27 +01:00
Emil Hammarstrom 9ec8da2363 scripts: Bump west version
Default to west >= 0.7.2

Signed-off-by: Emil Hammarstrom <emil.a.hammarstrom@gmail.com>
2020-03-23 17:55:14 +01:00
Kumar Gala a457681b1c scripts/requirements: Split & document requirement.txt
Split up requirements.txt into several files so that CI tools can
utilize/reference the specific requirements-<FOO>.txt they may need
while keep things in sync with the development.  This is to reduce
both time and amount of work CI actions due to python package install.

Create the following groupings:

1. BASE - needed to build or create zephyr images
2. BUILD-TEST - need to run compile/build tests
3. DOC - need to build the docs
4. RUN-TEST - need for runtime testing
5. EXTRAS - optional or useful for development/developers workflow

Also tried to add a comment about what or why a given package is being
pulled in for.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-03-19 15:41:08 +01:00
Andrew Boie b42fe9c764 scripts: gen_kobject_list.py cleanup
Un-do the changes made to the code when this was separated out
to elf_helper.py.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-03-17 20:11:27 +02:00
Andrew Boie fc2f7c3a55 scripts: merge elf_helper into gen_kobject_list
No need for this to be separated out any more.
Minimal changes made to get it to still work.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-03-17 20:11:27 +02:00
Andrew Boie 28be793cb6 kernel: delete separate logic for priv stacks
This never needed to be put in a separate gperf table.
Privilege mode stacks can be generated by the main
gen_kobject_list.py logic, which we do here.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-03-17 20:11:27 +02:00
Andrew Boie ae8acffaa6 doc: fix some missed renames
These were renamed to z_ prefix some time ago, but not updated
in these places.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-03-17 20:11:27 +02:00
Andrew Boie 2dc2ecfb60 kernel: rename struct _k_object
Private type, internal to the kernel, not directly associated
with any k_object_* APIs. Is the return value of z_object_find().
Rename to struct z_object.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-03-17 20:11:27 +02:00
Andrew Boie 4bad34e749 kernel: rename _k_thread_stack_element
Private data type, prefix with z_.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-03-17 20:11:27 +02:00
Andrew Boie f2734ab022 kernel: use a union for kobject data values
Rather than stuffing various values in a uintptr_t based on
type using casts, use a union for this instead.

No functional difference, but the semantics of the data member
are now much clearer to the casual observer since it is now
formally defined by this union.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-03-17 20:11:27 +02:00
peng1 chen 921226493b sanitycheck: delete the '\r' char in front of serial output
add lstrip function to delete the '\r' char in front of the
serial output from serail readline, because this '\r'
will influence the result match function in harness.py.

Signed-off-by: peng1 chen <peng1.chen@intel.com>
2020-03-17 12:34:21 -04:00
Patrik Flykt c9ad901fb7 scripts: net: Clarify and add documentation to README
The README file was in need of clarification how to set up the
net-tools Docker image and how the IP addresses were assigned.
mqtt_publisher is also mentioned.

Signed-off-by: Patrik Flykt <patrik.flykt@intel.com>
2020-03-17 13:14:41 +02:00
Patrik Flykt 2b42e1274e scripts: net: Support testing of mqtt_publisher
With net-tools and the mqtt_publisher sample updated, add support
for running the sample with the network sample test script.

Signed-off-by: Patrik Flykt <patrik.flykt@intel.com>
2020-03-17 13:14:41 +02:00
Patrik Flykt 26685b727b scripts: net: Properly return Zephyr exit code to script
Save the exit code from 'wait' so that it is not lost when PID
is set to zero.

Signed-off-by: Patrik Flykt <patrik.flykt@intel.com>
2020-03-17 13:14:41 +02:00
Kumar Gala 9b9436dfbe Revert "scripts: gen_defines: augment edtlib nodes"
This reverts commit dd1d58cad6.

Reverted as the changes to gen_defines.py break things.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-03-14 10:38:52 -06:00
Kumar Gala 6b97a220b4 Revert "scripts: gen_defines: re-work write_regs() with augmented nodes"
This reverts commit ba1c8b523c.

Reverted as the changes to gen_defines.py break things.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-03-14 10:38:52 -06:00
Kumar Gala c6a7ecda93 Revert "scripts: gen_defines: re-work write_irqs() with augmented nodes"
This reverts commit f68da76d36.

Reverted as the changes to gen_defines.py break things.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-03-14 10:38:52 -06:00
Kumar Gala 6499addb84 Revert "scripts: gen_defines: re-work write_clocks() with augmented nodes"
This reverts commit c048f872d7.

Reverted as the changes to gen_defines.py break things.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-03-14 10:38:52 -06:00
Kumar Gala 1ebe6f2bdd Revert "scripts: gen_defines: re-work write_props() with augmented nodes"
This reverts commit 86645aafdf.

Reverted as the changes to gen_defines.py break things.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-03-14 10:38:52 -06:00
Kumar Gala a3049ad801 Revert "scripts: gen_defines: re-work write_spi_dev() with augmented nodes"
This reverts commit 0471199e63.

Reverted as the changes to gen_defines.py break things.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-03-14 10:38:52 -06:00
Kumar Gala 98ae26f689 Revert "scripts: gen_defines: re-work write_bus() with augmented nodes"
This reverts commit 3939556fc3.

Reverted as the changes to gen_defines.py break things.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-03-14 10:38:52 -06:00
Kumar Gala 22342bec9d Revert "scripts: gen_defines: remove out()'s aliases kwarg"
This reverts commit 47073fd11c.

Reverted as the changes to gen_defines.py break things.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-03-14 10:38:52 -06:00
Kumar Gala b379e4547f Revert "scripts: gen_defines: tweak out_define() whitespace"
This reverts commit 3b7ddc83d1.

Reverted as the changes to gen_defines.py break things.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-03-14 10:38:52 -06:00
Corey Wharton 86bfc489f4 scripts: Replace hard-coded subsystem list with __subsystem annotations.
This change removes the hardcoded subsystem list in gen_kobject_list.py
favor of marking the relevant driver API structs with the _subsystem
sentinel.

Signed-off-by: Corey Wharton <coreyw7@fb.com>
2020-03-11 17:13:39 +02:00
Corey Wharton ccd15df510 scripts: Dynamically add driver subsystems to subsystems list
This change extends the parse_syscalls.py script to scan for a
__subsystem sentinal added to driver api declarations. It thens
generates a list that is passed into gen_kobject_list.py to extend
the subsystems list. This allows subsystems to be declared in the
code instead of a separate python list and provides a mechanism for
defining out-of-tree subsystems.

Signed-off-by: Corey Wharton <coreyw7@fb.com>
2020-03-11 17:13:39 +02:00
Kumar Gala 0353d22202 kconfig: Add dt_nodelabel_enabled function
Add a function takes a 'label' and returns "y" if we find an
"enabled" node that has a 'nodelabel' of 'label' in the EDT
otherwise we return "n"

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-03-11 07:59:38 -06:00
Martí Bolívar 7846fd103d edtlib: add 'labels' attribute to Node class
This corresponds to the attribute by the same name in dtlib.Node.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-03-11 07:59:38 -06:00
Martí Bolívar 3b7ddc83d1 scripts: gen_defines: tweak out_define() whitespace
Pad node identifiers to 60 characters. This results in better
alignment in practice than the current value of 40, which is a bit
low.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-03-11 07:59:38 -06:00
Martí Bolívar 47073fd11c scripts: gen_defines: remove out()'s aliases kwarg
Write the same results explicitly in its only remaining caller.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-03-11 07:59:38 -06:00
Martí Bolívar 3939556fc3 scripts: gen_defines: re-work write_bus() with augmented nodes
This too is an attempt to reimplement the previous behavior exactly,
modulo the order in which things are defined.

This is the last function which is calling into the previous
implementation's out_node and node_*_alias() functions, so these can
be removed now.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-03-11 07:59:38 -06:00
Martí Bolívar 0471199e63 scripts: gen_defines: re-work write_spi_dev() with augmented nodes
This uses augmented nodes in the same way as done previously.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-03-11 07:59:38 -06:00
Martí Bolívar 86645aafdf scripts: gen_defines: re-work write_props() with augmented nodes
This is similar to the work already done for regs.

Other than the order in which they appear and comments, the output
before and after this patch should be exactly the same.

We're intentionally leaving some of the helpers in module scope here
to use some of the subroutines elsewhere later on when reworking
write_spi_dev().

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-03-11 07:59:38 -06:00
Martí Bolívar c048f872d7 scripts: gen_defines: re-work write_clocks() with augmented nodes
Mirror the change already done to write_regs().

Other than the order in which they appear and comments, the output
before and after this patch should be exactly the same.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-03-11 07:59:38 -06:00
Martí Bolívar f68da76d36 scripts: gen_defines: re-work write_irqs() with augmented nodes
Use augmented nodes to print macros grouped by namespace.

Other than the order in which they appear and comments, the output
before and after this patch should be exactly the same.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-03-11 07:59:38 -06:00
Martí Bolívar ba1c8b523c scripts: gen_defines: re-work write_regs() with augmented nodes
Group the macros together by namespace rather than putting all the
BASE_ADDRESS macros together and all the SIZE macros together. E.g.,
all the DT_INST_<x> namespace macros for each node now appear
consecutively.

Add a comment making it clear that this output comes from "regs",
since "BASE_ADDRESS" and "SIZE" are not property names.

Other than the order in which they appear and comments, the output
before and after this patch should be exactly the same.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-03-11 07:59:38 -06:00
Martí Bolívar dd1d58cad6 scripts: gen_defines: augment edtlib nodes
Add additional attributes to each edtlib.Node we process, before
calling into the write_foo() routines.

This includes the identifier returned by node_ident(), which is used
as the primary identifier for the node, as well as lists for instance
and aliases nodes, and a catchall list that contains all the other
identifiers in addition to the primary one.

Use this information in a new for_each_ident(node, ident) def that
will be put to use in the various write_foo() routines.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-03-11 07:59:38 -06:00
Ulf Magnusson 8cacb9fd8c dts: edtlib: Turn global spi_dev_cs_gpio() into Node.spi_cs_gpio
spi_dev_cs_gpio() takes a Node and returns the chip select GPIO for it.
Having that information available directly from Node is neater, so turn
it into a Node.spi_cs_gpio property instead.

That gets rid of the only public global function in edtlib, which might
make the API design clearer too.

Tested with the sensortile_box board, which uses SPI chip select.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2020-03-11 07:59:38 -06:00
Ulf Magnusson 35166c45d9 dts: edtlib/gen_defines: Fix API design re. dtc flags
It's better to allow per-instance EDT configuration than to set a global
variable on the edtlib module. Enable/disable the warning for reg/unit
address mismatches via a flag to EDT.__init__(), instead of via a global
variable. That makes it consistent too.

Another option would be to pass the 'dtc' flags to EDT.__init__(), but
it makes the interface a bit ugly. Maybe if it needs to emulate lots of
other flags later.

Clarify that edtlib itself isn't meant to have any state in the comment
at the top of the module.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2020-03-11 07:59:38 -06:00
Patrik Flykt cbef6760c1 scripts: net: Enhance error reporting
Enhance error reporting by returning immediately when the calling
function reports an error.

Signed-off-by: Andrei Laperie <andrei.laperie@intel.com>
Signed-off-by: Patrik Flykt <patrik.flykt@intel.com>
2020-03-10 16:43:10 +02:00
Patrik Flykt 3401192e7f scripts: net: Add option to keep container after test
Add '--keep' option to leave Docker container and network running
after the test.

Signed-off-by: Patrik Flykt <patrik.flykt@intel.com>
2020-03-10 16:43:10 +02:00
Patrik Flykt ad1b38bebe scripts: net: Return final return value from Docker test
Return the final return value from Docker script when running echo
client in the container, first for UDP and then TCP, for both IPv4
and IPv6.

Signed-off-by: Patrik Flykt <patrik.flykt@intel.com>
2020-03-10 16:43:10 +02:00
Patrik Flykt 8f451b7338 scripts: net: Add overlay file to sample script
Add an '--overlay' command line option in order to use an overlay
file when compiling Zephyr.

Signed-off-by: Patrik Flykt <patrik.flykt@intel.com>
2020-03-10 16:43:10 +02:00
Martí Bolívar 34a59168d5 build: clean up west build
The header printed at the beginning of every west build is kind of
annoying. Let's remove it and just print the application source
directory at cmake time instead. The build directory and board are
already printed there, anyway, and that's all IDE users will see.

Let's clean up the BOARD to make it say "board" instead. That matches
the west build --board option name a bit more closely and is still
legible.

Likewise, let's not print any west build messages if we're just
incrementally recompiling. That's noisy.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-03-10 14:53:28 +02:00
Mikkel Jakobsen 98eb316bdf scripts: build.py: make _sanity_check() case insensitive on windows
When checking build directory against cache on windows, certain
corner cases can end up failing the comparison because of case
difference on an otherwise identical path. This can be avoided
by ignoring case on windows.

Signed-off-by: Mikkel Jakobsen <mikkel.aunsbjerg@prevas.dk>
2020-03-10 12:19:35 +02:00
Anas Nashif 497fe4267a release: update sanitycheck footprint file
Update for latest release.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-03-09 23:40:45 +02:00
Martí Bolívar 67d8f755e2 runners: misc-flasher: hotfix
The 'command' command line argument for this flasher is now being
overridden by common code, which attaches the west subcommand name to
this.

Let's just hotfix this by renaming the argument in misc-flasher.
We can revisit the boundary between run_common.py arguments and
runners package arguments after the release.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-03-04 21:48:09 +02:00
Martí Bolívar 24ce05c841 doc: improve west flash help for dfu-util
Try to make it clearer what's going on here.

Suggested-by: Lucian Copeland <hierophect@gmail.com>
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-03-03 09:30:32 +02:00
Martí Bolívar dc0948424e scripts: run_common: fix command line --hex-file and friends
This is yet another bug introduced by the move to runners.yaml.
Sigh. I should have tested this better.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-02-29 19:38:49 +02:00
Martí Bolívar f8cb3d4d53 scripts: run_common: rename 'args' to 'user_args'
This will also make the reason for a following bug fix easier to see.

Update a comment block to include all the work that needs doing.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-02-29 19:38:49 +02:00
Martí Bolívar 97dbda2088 scripts: run_common: rename unknown_args to user_runner_args
This makes its true value clearer, and will make a later bug fix patch
easier to understand.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-02-29 19:38:49 +02:00
Martí Bolívar 6dab163cad scripts: fix west sign when ZEPHYR_BASE is unset
Due to cleanups in west targeted at getting rid of zephyr-specific
code, extension commands can no longer rely on ZEPHYR_BASE being set
in the calling environment at import time (it's still set at run()
time for now, though, to keep west build working).

Add a new helper to make dealing with this easier from west sign.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-02-20 09:06:09 +02:00
Martí Bolívar eb95bed552 scripts: west_commands: fix argument handling with runners.yaml
Refactor the code to support the new runners.yaml file created by the
build system.

Compared to fishing around in the CMake cache, this makes it trivial
to put all the command line arguments to a runner-based command on
equal footing, regardless of if they're defined in the runners package
proper or defined in run_common.py.

This allows board.cmake files to do things like this:

   board_set_runner_args(foo
     --bin-file=${PROJECT_BINARY_DIR}/my-signed.bin)

While at it, make some other cleanups:

- Stop using the obsolete and deprecated west.cmake module while we're
  here in favor of the zcmake.py module which was added to Zephyr a long
  time ago. Yikes. I had forgotten this was still here.

- Stop using west.util's wrap function in favor of raw use of
  textwrap. The west function splits on hyphens, which is breaking
  runner names like "em-starterkit".

- Clean up the --context output a bit

Fixes: #22563
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-02-20 09:06:09 +02:00
Stephanos Ioannidis 354f2bbf1e sanitycheck: Update section names for memory size calculation
This commit updates the section names for memory size calculation in
the sanitycheck script as follows:

1. Remove `_TEXT_SECTION_NAME_2` section, which no longer exists, from
  the `rw_sections`.

2. Add `rom_start` section, which mostly contains read-only data such
  as the exception vector table, to the `ro_sections`.

Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
2020-02-18 12:59:25 -05:00
Alexey Brodkin 5bd2857c73 board: arc: Switch to generic OpenOCD runner
We used to use "em-starterkit" runner for ARC which is
basically heavy-modified "openocd" runner tweaked to
use GDB for loading and starting Elf files.

Now when loading and running Elf files is possible with generic
"openocd" runner we may us it. So we switch and get rid of
"em-starterkit" as well since we no longer need it.

Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
2020-02-13 10:45:44 +02:00
Alexey Brodkin 83b346edef runners: opeocd: Allow loading Elf-files
In some cases especially for on-going development & debugging of real
application it might be useful to load and run not from flash but
from RAM in that case there's one catch: we cannot reset the board
after loading memory with our app.

That's because:
 a) RAM we use might be either cleared on reset or might enter
    unpredictable state with portion of previously loaded data
    being corrupted.
 b) Reset vector most probably still point to ROM/flash and so
    our current application won't be executed on reset.

So instead of "run reset" command of OpenOCD we'll use
"resume 0x12345678". Where 0x12345678 is our application's
entry-point (which BTW may very well not match beginning of
the .text section or link base).

Now to extract the entry-point we need our application's zephyr.elf
and since we already have a requirement for Elf we may use it for
loading because OpenOCD does it perfectly fine moreover automatically
detecting loaded image type (binary, hex, Elf etc).

And to use that nice feature just add "--use-elf" to west's
command-line for boards that use "openocd" runner. Like that:
----------->8--------------
west flash --use-elf
----------->8--------------

Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
2020-02-13 10:45:44 +02:00
Alexey Brodkin 5a4237ed3b west: openocd: Allow to specify FTDI instance by its serial number
To be used in setups with multiple boards attached to the same one
host we need to have an ability to specify precisely which JTAG probe
to use for a particular board.

This is done by passing "ftdi_serial XXX" command to OpenOCD.
And the serial ("XXX") is supposed to be passed from higher level,
typically via west's options. And exactly for that we add another
"openocd" runner option "--serial=XXX" which sets
a Tcl's "_ZEPHYR_BOARD_SERIAL" variable that later gets passed
to OpenOCD's "ftdi_serial" command.

See more discussions on the matter here:
https://github.com/zephyrproject-rtos/zephyr/issues/22543

Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
2020-02-13 10:45:44 +02:00
Kumar Gala 0ce53610f8 scripts/requirements: Remove unused packages
Remove 'git-spindle' as there are not users in any scripts or build
system and not references in any of our docs.

Remove 'wheel' as nothing seems to use or need it directly.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-02-12 10:30:56 -06:00
Martí Bolívar dc1edb946f scripts: zephyr_module: hack to match west 0.7 output
The west command line output is not stable and has changed for 0.7.
Match it in zephyr_module.py's check for whether we are in a
workspace.

The real fix is to start using 'west topdir' whenever the west version
is at least 0.7.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-02-12 11:37:52 +01:00
Andy Ross 7353c7f95d kernel/userspace: Move syscall_frame field to thread struct
The syscall exception frame was stored on the CPU struct during
syscall execution, but that's not right.  System calls might "feel
like" exceptions, but they're actually perfectly normal kernel mode
code and can be preempted and migrated between CPUs at any time.

Put the field on the thread struct.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2020-02-08 08:51:04 -05:00
Kumar Gala b81eec951b sanitycheck: make pyserial optional
We only need pyserial python module if we are doing device testing.
Treat it similar to how we treat tabulate module.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-02-07 19:48:24 -06:00
Anas Nashif a9f2fcb64a requirements: require gcovr >= 4.2
This is the version needed to generate coverage using sanitycheck.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-02-07 15:14:24 -05:00
Anas Nashif 323161dfcd sanitycheck: point to correct gcov path
The SDK has a new path for x86 gcov, point to the new binary. Also, do
not clean artifacts when running coverage.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-02-07 15:14:24 -05:00
Martí Bolívar cea6e3f17a scripts: west_commands: fix help text for flash
The 'command' variable points at a python command object, not a
string. Take its name so the help text for west flash -h correctly
says 'flash' instead of 'debug'.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-02-06 23:06:04 -05:00
Ulf Magnusson fad697aed9 scripts: gen_defines.py: Add out_dev() flag for deprecated macros
Add a 'deprecation_msg' string/flag to out_dev(). When 'deprecation_msg'
is passed, all generated macros include

    __WARN(<deprecation_msg>)

which prints a custom warning if the macro is used.

Meant to be used when improving the output format.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2020-02-06 17:27:49 -06:00
Kumar Gala 2fd7876aa7 ci: github: west: Add a GH workflow to run the west command tests
Replace running west command tests in run_ci.sh with a github workflow.
This provides some benefits in that we can run the west command tests on
multiple python versions and host OSes (linux, mac and windows).

Also have the benefit that the tests are only run on modifications to
files in scripts/west_commands/ or scripts/west-commands.yml.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-02-06 08:34:43 -06:00
Martí Bolívar 1c80d79fdd scripts: west_commands: skip bossac tests unless on linux
That's the only platform the tool is available.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-02-06 08:34:43 -06:00
Wentong Wu c35fa6caa6 scripts: add script for usb backend to receive tracing data
Add script for usb backend to receive tracing stream data. This
script, trace_capture_usb.py, is based on pyusb, so install it
correctly before using the script.

Signed-off-by: Wentong Wu <wentong.wu@intel.com>
2020-02-05 23:54:26 -05:00
Wentong Wu b4449a0c2c scripts: add script to capture tracing data with UART backend
Add script to capture tracing stream data with UART backend. This
script is developed based on pyserial, so install it correctly
before using the script.

Signed-off-by: Wentong Wu <wentong.wu@intel.com>
2020-02-05 23:54:26 -05:00
Kumar Gala de70e9c5ba scripts/sanitycheck: be more fair with dividing subsets
Since we do interger division to determine how many items are in each
set, its possible we can have bad rounding error and the last set having
a much larger amount of items compared to all other sets.

For example, total = 3740, sets = 300:
	per_set = 3740 / 300 = 12.466 = 12
	last_set = 3740 - 299 * 12 = 152

So instead of doing simple division, we add on extra item to the early
sets and we've exhausted the rounding error:

	num_extra_sets = total - per_set * sets
		   140 = 3740 - 12 * 300

So the first 140 subsets will have 13 per_set, and the last 160 will
have 12 per_set.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-02-05 18:57:49 -05:00
Andrei Gansari e87fec1554 scripts: simple-bus warning suppression
Single-bus warning in python parsing of device tree is suppressed if
this is also suppresed in the device tree compiler.

Signed-off-by: Andrei Gansari <andrei.gansari@nxp.com>
2020-02-05 13:04:44 -06:00
Andrei Emeltchenko d44d986a03 west: Add signing support using the rimage tool from SOF
This way "west sign" can be used for signing images.

Signed-off-by: Andrei Emeltchenko <andrei.emeltchenko@intel.com>
2020-02-05 10:43:25 -05:00
Martí Bolívar 54ddd514ae scripts: grutils: tweak sort keys to group siblings
Change the key function used to sort nodes so that unit addresses, if
present, break ties between sibling nodes. This orders siblings in
increasing order by unit-address in any gen_defines output that is
sorted by ordinal.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-02-04 07:28:45 -06:00
Martí Bolívar d406439570 gen_defines: don't emit comments with "# " lines
If the line is empty, just emit a single "#\n" instead of "# \n".

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-02-04 07:28:45 -06:00
Martí Bolívar c63069489c gen_defines: emit output for all nodes in dependency order
This makes the results easier to read, since node output appears in
the same order as the top level comment summarizing the tree.

Reorganize how flash partitions are emitted so they also appear in
dependency order, allowing us to extend the comment at the top of the
file to say that nodes are emitted in that order, along with other
useful information.

Print comments even for nodes that are disabled or have no binding,
to make it clearer that the script considered them and decided not to
output anything for them.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-02-04 07:28:45 -06:00
Frank Li 59208ac13b runners: pyocd: add --tool-opt parameter
add --tool-opt to support more pyocd parameter

Signed-off-by: Frank Li <lgl88911@163.com>
2020-02-03 13:32:38 -05:00
Patrik Flykt 65ffc71466 scripts: net: Add script for running sample tests
Add a script that sets up Docker networking and starts the net-tools
Docker container. If successful, run Zephyr with native_posix and
execute the appropriate net-tools container executable. The proper
net-tools executable and arguments is selected depending on the
basename of the sample directory. This script needs to be updated
if the net-tools Docker image is updated in some incompatible way.

The net-tools directory is assumed to exist at the same level as
the Zephyr base directory, but its location can be set using the
'-N' command line argument. Likewise, '-Z' sets the Zephyr top level
directory.

When stopping Zephyr, go through all child processes since running a
native posix or other variant of Zephyr may cause a hierarchy of
processes to be created.

Fixes: #19540

Signed-off-by: Patrik Flykt <patrik.flykt@intel.com>
2020-02-03 09:04:12 -05:00
Marcin Niestroj be0f5fe0b0 scripts/sanitycheck: fix merging OVERLAY_CONFIG extra args
Overlay files were concatenated into single string with no whitespace
between them. If '--extra-args OVERLAY_CONFIG="overlay-1.conf"
--extra-args OVERLAY_CONFIG="overlay-2.conf"' was used, then the result
was OVERLAY_CONFIG="overlay-1.confoverlay-2.conf".

Another thing was that overlay extra args were not properly removed from
the list of regular arguments. As a result we had incorrect list of
overlays and incorrect list of other arguments.

Rework code to extract overlays in loop in a safe manner. Use for that a
list of strings instead of string directly. Join those strings and form
a single OVERLAY_CONFIG argument just before running cmake.

Tested with following testcase.yaml line:

  extra_args: ARG1 OVERLAY_CONFIG="overlay-1.conf"
              ARG2 OVERLAY_CONFIG="overlay-2.conf"

Before this patch we got:

  args = ['OVERLAY_CONFIG="overlay-1.conf"',
    'OVERLAY_CONFIG="overlay-2.conf"']

After this patch we get:

  args = ['ARG1', 'ARG2',
    'OVERLAY_CONFIG="overlay-1.conf overlay-2.conf"']

While at it, fix also regex pattern by removing requirement of double
quotes around value. Match any option value instead and strip both
single and double quotes when match is positive. Tested with:

  $ ./scripts/sanitycheck -T samples/hello_world/ -p qemu_x86 \
      --extra-args OVERLAY_CONFIG=overlay1.conf '
      --extra-args OVERLAY_CONFIG=\"overlay2.conf\" '
      --extra-args OVERLAY_CONFIG=\'overlay3.conf\'

Signed-off-by: Marcin Niestroj <m.niestroj@grinn-global.com>
2020-01-31 15:55:36 -05:00
Ulf Magnusson d0f082dd3b kconfig/devicetree: Print path to headers when configuring
Change the output during CMake configure from

    Devicetree configuration written to .../devicetree.conf
    Parsing /home/ulf/z/z/Kconfig
    Loaded configuration '.../frdm_kw41z_defconfig'
    Merged configuration '.../prj.conf'
    Configuration saved to '.../.config'

to

    Devicetree header saved to '.../devicetree_unfixed.h'
    Parsing /home/ulf/z/z/Kconfig
    Loaded configuration '.../frdm_kw41z_defconfig'
    Merged configuration '.../prj.conf'
    Configuration saved to '.../.config'
    Kconfig header saved to '.../autoconf.h'

devicetree_unfixed.h is more useful to be aware of than devicetree.conf
(still hoping we can get rid of it at some point), and seeing the
Kconfig header clarifies things to.

"Saved" instead of "written" for consistency. Maybe it could say
"Kconfig configuration" instead of "configuration" too, though it gets a
bit spammy in other contexts where the message shows up.

Updates Kconfiglib (and menuconfig/guiconfig, just to sync) to upstream
revision 061e71f7d7, to get this commit in, which is used to print the
header path:

    Return a message from Kconfig.write_autoconf()

    Like for Kconfig.write_config() and Kconfig.write_min_config(),
    return a string from Kconfig.write_autoconf() with a message saying
    that the header got saved, or that there were no changes to it. Can
    be handy in tools.

    Also make the "no change" message for the various files more
    specific, by mentioning what type of file it is (configuration,
    header, etc.)

    Return True/False from Kconfig._write_if_changed() to indicate if
    the file was updated. This also allows it to be reused in
    Kconfig.write_min_config().

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2020-01-31 19:45:43 +01:00
Martí Bolívar 34346c41ac modules: fail on invalid ZEPHYR_EXTRA_MODULES
Add error checking in zephyr_module.py so that if the user manually
specifies ZEPHYR_EXTRA_MODULES and the list contains something that
isn't in fact a valid module, we scream and die.

This should help with diagnosing module errors.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-01-31 19:35:43 +01:00
Ulf Magnusson e2f647b08b scripts: kconfig: lint.py: Add check for missing CONFIG_ prefix
Add a new --check-missing-config-prefix check that looks for references
like

    #if MACRO
    #if(n)def MACRO
    defined(MACRO)
    IS_ENABLED(MACRO)

where MACRO is the name of a defined Kconfig symbol but doesn't have a
CONFIG_ prefix. Could be a typo.

Skip MACRO if it is #define'd somewhere, even if it looks like a Kconfig
symbol.

Found e.g. https://github.com/zephyrproject-rtos/zephyr/pull/22195.

Piggyback skipping binary files when grepping for Kconfig symbol
references.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2020-01-31 19:29:49 +01:00
Alberto Escolar Piedras 1a01a10a87 ci: Run also BT edtt_ble_test_app tests
Include in the CI run the EDTT tests

Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
2020-01-31 18:36:40 +01:00
Alberto Escolar Piedras 9338e9215b run_ci.sh: Set EDTT_PATH
So that the EDTT tool could be used by this script,
set its path so other scripts could find it

Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
2020-01-31 18:36:40 +01:00
Anas Nashif 471ffbe77d coverage: do not dump coverage data by default
Only dump data when we are interested in the analysing coverage. By
default just collect the data.

CONFIG_COVERAGE_DUMP is used to control this behaviour.

This will help speed up sanitycheck and will avoid lots of noise in the
log when some tests with coverage enabled failed. Dumping data to
console is also suspected to be one of the reason why qemu hangs in CI.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-01-30 16:04:03 -05:00
Henrik Brix Andersen deccb71f1d west: commands: sign: look for imgtool.py in path
If 'imgtool' is not found in PATH look for 'imgtool.py' before bailing
out.

This allows adding the mcuboot/scripts directory to PATH and have
'west sign -t imgtool' automatically locate the 'imgtool.py' script.

Signed-off-by: Henrik Brix Andersen <hebad@vestas.com>
2020-01-30 14:17:07 -05:00
Anas Nashif 923ca14b7d sanitycheck: show which threads are stuck
Print a warning if some threads are taking more time and seem to be
stuck.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-01-30 16:49:03 +01:00
Anas Nashif 2c1c399872 sanitycheck: minor comment and style cleanup
Remove an unrelated comment and a minor style change.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-01-30 16:49:03 +01:00
Ulf Magnusson edfe5a653e dts: gen_defines.py: Save final devicetree to zephyr/zephyr.dts
Add a '--dts-out <file>' flag to gen_defines.py that saves the final
devicetree to <file>, in DTS format, using the new EDT.dts_source
attribute.

Handy to have available as a debugging aid. Unused otherwise.

Also write a dummy <BOARD>.dts_compiled file that tells people to look
at zephyr.dts, for people that might be used to that file. It was
removed in commit 8d8b06978c ("dts: Remove generation of
<BOARD>.dts_compiled").

Fixes: #22272

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2020-01-30 04:27:39 -06:00
Ulf Magnusson a758af4988 dts: edtlib: Add an EDT.dts_source attribute that holds the source code
EDT.dts_source is the source code of the loaded .dts file, after merging
nodes.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2020-01-30 04:27:39 -06:00
Ulf Magnusson 8dfbd9f080 kconfig.py: Error/warning formatting nits
Include the "warning: "/"error: " part of the string when wrapping
lines, and consistenly start messages with a capital letter.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2020-01-30 03:51:04 -06:00
Ulf Magnusson ac78e2d37a kconfig.py: Add a description at the top of the file
Give a short overview of what the script is about.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2020-01-30 03:51:04 -06:00
Ulf Magnusson bf4133d262 kconfig.py: Flag selects with unsatisfied deps and remove whitelisting
Turn the warning for selecting a symbol with unsatisfied dependencies
into an error. The last instance has been fixed (that triggers in CI at
least).

Also remove the warning whitelisting functionality, which was only used
to whitelist that warning. It hasn't been used for anything else in over
a year, so it probably wouldn't be useful to keep. Getting rid of it
makes the script easier to read.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2020-01-30 03:51:04 -06:00
Ulf Magnusson 88db84b89b dts: edtlib: Turn Node.instance_no into EDT.compat2enabled
Add an EDT.compat2enabled attribute that maps compatibles to enabled
devicetree nodes that implement them. For example,
EDT.compat2enabled["foo"] is a list of all enabled nodes with "foo" in
the 'compatible' property.

The old Node.instance_no functionality can be implemented in terms of
EDT.compat2enabled, so remove Node.instance_no. EDT.compat2enabled is
more flexible and easier to understand.

Simplify main() in gen_defines.py by using EDT.compat2enabled to
generate the DT_COMPAT_<compatible> existence macros. The behavior is
slightly different now, as DT_COMPAT_<compatible> is generated for
enabled nodes that don't have a binding as well, but that might be an
improvement overall. It probably doesn't hurt at least.

EDT.compat2enabled simplifies the implementation of the new
$(dt_compat_get_str) preprocessor function in
https://github.com/zephyrproject-rtos/zephyr/pull/21560. That was the
original motivation.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2020-01-29 09:46:38 -06:00
Kuba Sanak c0e31e7d71 flashing: don't give bossac offset parameter unless explicitly provided
This fixes a problem which appeared after bossac version was downgraded
to 1.7 which no longer accepts the -o/--offset parameter. Now the offset
is fed to bossac executable only if it's explicitly provided and not by
default.

Signed-off-by: Kuba Sanak <contact@kuba.fyi>
2020-01-28 12:53:13 -06:00
Ulf Magnusson 3e37447802 dts: gen_defines.py: Do not crash on 'type: path' properties
'type: path' was added to edtlib for completeness in commit 23a5b4963b
("dts: edtlib: Add 'type: path' for path references"). gen_defines.py
crashes if it's ever used though, because it gets confused for a
'type: phandle-array'.

Ignore 'type: path' properties in gen_defines.py, like how
'type: phandle' and 'type: phandles' are currently ignored too.

(Note that gen_defines.py is only one possible user of edtlib.)

Fixes: #22197

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2020-01-28 12:51:35 -06:00
Ulf Magnusson cf89ba33ea global: Fix up leading/trailing blank lines in files
To make the updated test in
https://github.com/zephyrproject-rtos/ci-tools/pull/121 clean, though it
only checks modified files.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2020-01-27 17:41:55 -06:00
Peter Bigot ab7773dae8 sanitycheck: leave non-tty terminal unmodified on exit
The terminal configuration should not be reset on exit if
sanitycheck's output is being directed to something that mediates the
terminal control, such as a pager.  Only do the reset if stdout is
interactive.

Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
2020-01-26 17:54:41 +01:00
Ulf Magnusson 08484d0ac4 dts: gen_defines.py: Convert to use f-strings
Use f-strings instead of .format() to makes all the macro identifier and
multiline comment building easier to read.

f-strings were added in Python 3.6, which is required by Zephyr now.

Convert some

    ... + "_" + ...

string building to f-strings as well.

f-strings are a bit faster too (because they avoid format()/str()
lookups), though it's not likely to be noticeable here.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2020-01-24 21:33:28 -05:00
Ulf Magnusson a5f6233428 kconfig.py: Convert to use f-strings
Use f-strings instead of .format() to make things more readable. They
were added in Python 3.6, which Zephyr requires now.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2020-01-24 15:57:23 +01:00
Ulf Magnusson ba2f95ab39 kconfig: Show unsatisfied deps. when assignments don't take
Show which dependencies are unsatisfied when symbols don't get their
assigned value.

For example, assume that FOO below is assigned with CONFIG_FOO=y. Note
that BAR, BAZ, and STR = "hmm" are 'n'.

    config FOO
           bool "foo"
           depends on BAR && BAZ && QAZ && STR = "hmm"

    config BAR
           def_bool n

    config BAZ
           def_bool n

    config QAZ
           def_bool y

    config STR
           def_string "zmh"

This now prints this warning:

    warning: FOO (defined at /home/ulf/z/z/Kconfig:10) was assigned the
    value 'y' but got the value 'n'. Check these unsatisfied
    dependencies: BAR (=n), BAZ (=n), STR = "hmm" (=n). See ...

Fixes: #21888

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2020-01-23 23:04:09 +01:00
Anas Nashif dd65f7c38a sanitycheck: change distribution of tests per set
Sort by testcase name and path and not by platforms. This way we do not
have the same platform executing on CI system causing failures to
overload.

Right now we have almost all qemu targets executing on 1 or 2 nodes,
causing those systems to be overloaded. Instead of taking the default
sorting by platforms, we now sort by testcases, so the distribution is
more "random", but still reproducible.

Remove the ordering function that would put native_posix in the first
set, it is not being used anymore.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-01-23 14:02:59 -05:00
Rihards Skuja f59b92b3ee runners: openocd: don't throw when no config is provided
In some cases the config is already provided for us e.g. when setting
OPENOCD_NRF5_SUBFAMILY. Also it looks like config was always supposed to
be optional (there are checks for it on do_run()).

Signed-off-by: Rihards Skuja <rihardssk@mikrotik.com>
2020-01-23 10:38:57 -05:00
Flavio Ceolin 661035a6b8 west: runners: stm32: Remove unecessary assignment
msg_text is unconditionally assigned later and this inital value is
never used.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2020-01-22 17:37:41 -05:00
Flavio Ceolin 9dbb50bac8 scripts: sanitycheck: Simplify if statement
Outer if is checking if the variable is in a list with two options
which means that we can just do an if / else and remove an assignment.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2020-01-22 17:37:41 -05:00
Ulf Magnusson 45050dda48 kconfig/cmake: Improve reconfiguration behavior
There are some issues with the behavior when rerunning CMake in an
already initialized build directory:

 1. The check for assignments to promptless symbols in configuration
    fragments isn't run when reconfiguring, because it only runs if
    zephyr/.config doesn't exist

 2. As outlined in
    https://github.com/zephyrproject-rtos/zephyr/issues/9573, you can
    get into situations where zephyr/.config is invalid (e.g. due to
    being outdated), but menuconfig/guiconfig can't be run to fix it

 3. If kconfig.py fails while merging fragments during reconfiguration,
    it will ignore the fragments during the next reconfiguration and use
    the existing zephyr/.config instead, because the fragment checksum
    is calculated and saved before running kconfig.py

(Footnote: The input configuration file(s) to kconfig.py can be either a
list of configuration fragments, when merging fragments, or just
zephyr/.config, if the merged configuration is up-to-date. The output
configuration file is always zephyr/.config.)

To fix the first two issues, explicitly tell kconfig.py when it's
dealing with handwritten configuration input (fragments), via a new
--handwritten-input-configs flag. This is more robust than checking
whether zephyr/.config exists, which was the old logic.

When dealing with handwritten input, there should be no assignments to
promptless symbols. Assignments to promptless symbols is expected in
zephyr/.config however, because it doubles as configuration output.

When running menuconfig/guiconfig, the input configuration is
zephyr/.config rather than configuration fragments, so this change also
makes sure that menuconfig can always be run as long as zephyr/.config
exists and is up-to-date.

To fix the last issue, only write the checksum for the configuration
fragments if kconfig.py succeeds (which means it wrote a
zephyr/.config).

Also improve naming a bit, add help texts for the command-line
parameters to kconfig.py, and simplify write_kconfig_filenames() by
moving logic into it.

Partial fix for
https://github.com/zephyrproject-rtos/zephyr/issues/9573, without the
part in #issuecomment-469701831. Can still run into issues when e.g.
when CMake files can't make sense of settings.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2020-01-22 18:28:07 +01:00
Anas Nashif e337882189 sanitycheck: fix reporting of timeouts
Timeouts were not reported correctly in the XML file as failures,
causing some confusion in the shippable results.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-01-22 01:39:28 -05:00
Anas Nashif 9fe1e2a663 sanitycheck: support excluding platforms
It is often needed to run sanitycheck and exclude a specific platform or
multiple platforms. Add an option to provide this using
--exclude-platform. The option can be used multiple times to exclude
multiple platforms at the same time.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-01-21 21:10:49 -05:00
Ulf Magnusson 7419f4f46f scripts: kconfig.py: Simplify write_kconfig_filenames()
The assert has never hit and probably won't be useful. Shorten the code
a bit.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2020-01-17 19:39:14 +01:00
Ulf Magnusson 7a531ea22b scripts: kconfig.py: Use err() helper for warnings-turned-errors
De-spams the code a bit.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2020-01-17 19:39:14 +01:00
Ulf Magnusson 7a148ef9da scripts: kconfig.py: Rename verify_*() functions to check_*()
Bit shorter, matches the devicetree code.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2020-01-17 19:39:14 +01:00
Ulf Magnusson bb4a4e79dc scripts: kconfig.py: Detect all assignments to promptless symbols
Assigning to promptless symbols has no effect.

Previously, the only check was for whether the value assigned to a
symbol matched its final value. This misses cases where a promptless
symbol is assigned to and just happens to get the assigned-to value as
its final value.

Instead, detect whether configuration files are being merged (by
checking if zephyr/.config already exists), and explicitly check for
assignments to promptless symbols in that case. We can't do it when
zephyr/.config already exists (and is being loaded), because it includes
values for promptless symbols as well.

With the no-prompt check moved out, also use a more specific message for
it, and remove stuff related to prompts elsewhere. Shorten messages a
bit at the same time, and add two warn() and err() helpers.

Fixes: #20697

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2020-01-17 19:39:14 +01:00
Ulf Magnusson 4e85006ba4 dts: Rename generated_dts_board*.{h,conf} to devicetree*.{h,conf}
generated_dts_board.h is pretty redundant and confusing as a name. Call
it devicetree.h instead.

dts.h would be another option, but DTS stands for "devicetree source"
and is the source code format, so it's a bit confusing too.

The replacement was done by grepping for 'generated_dts_board' and
'GENERATED_DTS_BOARD'.

Two build diagram and input-output SVG files were updated as well, along
with misc. documentation.

hal_ti, mcuboot, and ci-tools updates are included too, in the west.yml
update.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2020-01-17 17:57:59 +01:00
Anas Nashif d6d0b6454b ci: do not display timestamps
With timestamps enabled, the log in shippable is very difficult to read,
so remove those and enable them when needed for debugging only.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-01-17 08:30:23 -06:00
Anas Nashif 1342d21fc0 sanitycheck: support runtime artifact cleanup
Using --runtime-artifact-cleanup, the script will now remove all
artifacts of passing tests. This is useful for running sanitycheck on
systems with little disk space.

Fixes #21922

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-01-16 19:19:07 -05:00
Ulf Magnusson d0f7652858 dts: gen_defines.py: Rename *_dev() to *_node()
out_dev(), etc., as a name is a holdover from before commit 73ac1466fb
("scripts: edtlib: Call nodes "nodes" instead of "devices""). It's
confusing for the same reasons as mentioned in that commit.

Renamings:

 - dev_ident()            -> node_ident()
 - out_dev()              -> out_node()
 - out_dev_s()            -> out_node_s()
 - dev_aliases()          -> node_aliases()
 - dev_path_aliases()     -> node_path_aliases()
 - dev_instance_aliases() -> node_instance_aliases()

Also clean up comments to talk about nodes instead of devices where it
makes sense.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2020-01-16 18:32:58 +01:00
Martí Bolívar 60db6a6e2f scripts: runners: nrfjprog: sort SNR list before prompting
This guarantees muscle memory will work over multiple runs when the
same boards are connected. (The "nrfjprog --help" output for --ids
does not guarantee an order.)

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-01-16 10:20:39 -05:00
Martí Bolívar e4768ad7e7 scripts: runners: nrfjprog: restrict when we guess snr
Right now, the nrfjprog runner will prompt the user for which board to
use if there are multiple possibilities and the --snr command line was
not given to specify one ahead of time.

Tweak this so it only happens if standard input is connected to a
terminal. This should avoid stalling the process on board farms when
this runner is used in automation.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-01-16 10:20:39 -05:00
Martí Bolívar ae2f27cf16 scripts: runners: defer ensure_snr() to run in nrfjprog
The create() classmethod should not be doing any I/O -- its only job
is to create the ZephyrBinaryRunner instance. It's currently trying to
figure out what the serial number of the board is, though. Let's defer
that work to do_run(), so it gets handled by run_common's exception
handler.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-01-16 10:20:39 -05:00
Martí Bolívar 8837469492 scripts: runners: improve error handling CLI
Catch RuntimeError when calling runner.run() and print a message
instead of dumping stack unless in verbose mode.

This improves the command line interface when runners raise exceptions.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-01-16 10:20:39 -05:00
Andrew Boie 75bd9b6ce0 sanitycheck: improve summary output
Show how many tests passed out of those that were actually
performed, not including skipped tests (which we also report
a count of)

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-01-14 23:26:06 -05:00
Anas Nashif cd1f017468 sanitycheck: set extra_args from command line
Missed setting extra_args coming from command line in the testsuite.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-01-14 13:39:46 -05:00
Ulf Magnusson 75621c3b45 dts: gen_defines.py: Add should_write() helper for properties
should_write() returns False for properties that are skipped in
write_properties(). Shortens write_properties() a bit.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2020-01-14 17:31:59 +01:00
Ulf Magnusson 776de86dc7 menuconfig: Update to work around resizing crash on macOS
Update menuconfig (and kconfiglib and guiconfig, just to sync) to
upstream revision 424d0d38e7, to get this commit in, which works around
a crash on some macOS Python installations.

    menuconfig: Work around crash on resize on some macOS systems

    get_wch() has started raising curses.error when resizing the
    terminal on some macOS Python installations. Work around for now by
    falling back on getch(), which still works.

    See https://github.com/ulfalizer/Kconfiglib/issues/84. Needs more
    investigation, but I don't have a Mac handy.

    Based on https://github.com/ulfalizer/Kconfiglib/pull/85, but with
    some more comments.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2020-01-13 14:03:12 +01:00
Ulf Magnusson 02dab5a5a8 dts: gen_defines.py: Add helper for outputting {} initializers
The

    "{" + ", ".join(...) + "}"

pattern is in a bunch of places now. Add an out_dev_init() helper for
it, with the same parameters as out_dev().

Also makes "{" vs. "{ " consistent, picking the first one.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2020-01-13 13:50:15 +01:00
Ulf Magnusson 405daa0081 dts: edtlib: Add helper for testing error messages and test _slice()
Add a helper function verify_error() to testedtlib.py that takes a DTS
source and verifies that it generates a particular EDTError.

Use it to test the hints that the previous commit added to errors
generated by _slice().

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2020-01-13 13:49:33 +01:00
Ulf Magnusson 57b2d2749c dts: edtlib: Improve error for wrong reg/ranges/interrupts size
Show how the element size was calculated in the error message when a
'reg', 'ranges', or 'interrupts' property has the wrong size. This
should help with debugging. Also mention that #*-cells properties come
from the parent node, which can be subtle.

Came up in https://github.com/zephyrproject-rtos/zephyr/issues/21607
(though it seems the comment disappeared).

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2020-01-13 13:49:33 +01:00
Ulf Magnusson b4e18072a8 kconfig: Fix wrong max argument count for some preprocessor functions
dt_chosen_reg() and dt_node_reg() take between 1 and 3 arguments, not
between 1 and 4 arguments. The implicit name argument isn't included in
the count.

These functions implement $(dt_chosen_reg_*) and $(dt_node_reg_*).

Giving the right max argument count makes Kconfiglib generate error
messages that give the location of the call, instead of getting a
cryptic generic Python error.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2020-01-13 13:48:17 +01:00
Erwan Gouriou a11405df52 scripts: requirements: Request version 0.24.0 for pyocd
pyocd 0.24.0 provides support for more user options.
This enables flashing of additional boards using pyocd.

Signed-off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
2020-01-13 11:49:57 +01:00
Anas Nashif e7656ea831 sanitycheck: do not log stderr from BinaryHandler
stderr from the binary handler (native_posix for example) was redirected
to the logger as errors, this is confusing the console and users, so
remove this.

Fixes #21784

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-01-09 14:28:32 -05:00
Andrew Boie 9b34ecd0c8 userspace: fix incorrect linker routing
The _thread_idx_map bitfield which has '1' set for free
thread indexes really needs to live in the
data section reserved for kernel object metadata, as this
is a part of memory that is allowed to shift addresses
between zephyr_prebuilt.elf and zephyr.elf.

However, if an application defines enough static threads
that there are no free indexes, the entire bitfield will
be zeroed and the bitfield will end up in the main BSS
section.

Force this data to always be in the .kobject_data.data
section regardless of its contents.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-01-09 09:13:23 -08:00
Ulf Magnusson 5372dedf2f scripts: kconfig: lint.py: Improve error reporting
If the working directory for a command was missing (usually due to
forgetting to run 'west update'), you'd get a FileNotFoundError
exception along with a cryptic error like

    'git' not found

Only catch OSError instead (which is a base class of FileNotFoundError),
and always show the exception message. It makes it clear that it's the
working directory that's missing.

Add some other misc. improvements too:

 - Turn stderr output from external commands into a warning instead of
   an error

 - Add err() and warn() helpers

 - Include the command name in messages

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2020-01-09 11:19:57 -05:00
Ulf Magnusson 23a5b4963b dts: edtlib: Add 'type: path' for path references
Add binding support for a 'path' property type, for properties that are
assigned node paths. Usually, paths are assigned with path references
like 'foo = &label' (common in /chosen), but plain strings are accepted
as well as long as they're valid paths.

The 'path' type is mostly for completeness at this point, but might be
useful for https://github.com/zephyrproject-rtos/zephyr/issues/21623.
The support is there already in dtlib.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2020-01-08 08:02:00 -06:00
Olof Johansson 8067aca7e9 gen_defines: close files before exit
In some cases, we've seen the output files be truncated when the
python script has been rebuilt into a .pex before running it --
likely due to buffering.

Closing files explicitly is the right thing to do anyway, so let's
do it.

Signed-off-by: Olof Johansson <olof@lixom.net>
2020-01-07 21:06:58 +01:00
Ulf Magnusson def1f0e2d5 devicetree: Remove DT_SRAM_{BASE_ADDRESS,SIZE}, use CONFIG_* versions
The SRAM address and size are currently available as both
DT_SRAM_{BASE_ADDRESS,SIZE} and as CONFIG_SRAM_{BASE_ADDRESS,SIZE} (via
the Kconfig preprocessor).

Use the CONFIG_SRAM_* versions everywhere, and remove generation of the
DT_SRAM_* versions from gen_defines.py.

The Kconfig symbols currently depend on 'ARC || ARM || NIOS2 || X86'.
Not sure why, so I removed it.

It looks like no configuration files set CONFIG_SRAM_* at the moment, so
another option might be to use the DT_* symbols everywhere instead. Some
Kconfig.defconfig.series files add defaults to them though.

Also improve the help texts for CONFIG_SRAM_* to say that they normally
come from devicetree rather than configuration files.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2020-01-07 17:19:36 +01:00
Ulf Magnusson 95c18f1e6b dts: gen_defines.py: Clean up 'reg' code
Reorganize write_regs() to reuse more code for *_BASE_ADDRESS and
*_SIZE, and get rid of reg_addr_ident() and reg_size_ident().

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2020-01-07 16:42:33 +01:00
Anas Nashif e9450065de sanitycheck: additional scripting hook after flashing
Support custom hooks after flashing is completed.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-01-06 13:58:10 -05:00
Anas Nashif 4522e10d65 sanitycheck: report overflow on SRAM overflow
Instead of failing, report overflow when the SRAM overflows.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-01-06 13:58:10 -05:00
Anas Nashif bdffb88950 sanitycheck: close pipe file handles
Make sure we close the pipe file handles after we are done, otherwise we
will end up with too many open file descriptors and crash...

Fixes #20974
Fixes #21637

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-01-06 13:58:10 -05:00
Anas Nashif 64c84b15ce sanitycheck: more debug messages
More debug messages to help locate issues and debug bugs.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-01-06 13:58:10 -05:00
Anas Nashif 1b1a7e242b sanitycheck: run pre/post script using dedicated function
Reduce duplicated code by introducing a dedicated function for running
custom scripts.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-01-06 13:58:10 -05:00
Anas Nashif 32e924459d sanitycheck: log errors when executing binaries
Add some verbosity into the log file to help with debugging.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-01-06 13:58:10 -05:00
Anas Nashif 52b66c6ad1 sanitycheck: when timing out in handler, show correct log file
Fix issue where we timeout in handler and show the device.log file
instead.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-01-06 13:58:10 -05:00
Anas Nashif 813deb7967 sanitycheck: use Unknown instead of N/A for fail reason
Instead of N/A, use Unknown which is more appropriate, N/A does not
really apply here.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-01-06 13:58:10 -05:00
Anas Nashif 5c331f95ad sanitycheck: add Digilent to device discovery
Some platforms use this USB device for serial, detect it when generating
hardware map.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-01-06 13:58:10 -05:00
Andrew Boie bd13710524 sanitycheck: use abs path for logs
Makes it much easier to inspect them in a different terminal
from where the test was run. These paths tend to be long
anyway, even if relative.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-01-03 08:40:34 -05:00
Andrew Boie 114c01b3b9 sanitycheck: allow outdir to be overwritten
Sanitycheck used to delete the output directory if -n
wasn't used, but now it renamed it. Add a new flag to
enable the old behavior.

Do not use logging for these early messages, logger
hasn't been set up yet.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-01-03 08:40:34 -05:00
Andrew Boie 2d8d4c5562 sanitycheck: fix early messages
When managing build artifacts the logger hasn't been
initialized. Use print() here instead.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-01-03 08:40:34 -05:00
Andrew Boie 1fe1f3a23d sanitycheck: clean up TTY on exit
The code has a lot of sys.exit() calls, but internally these
just raise a special exception. Add a try...finally block
to ensure 'stty sane' is run before leaving the script.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-01-03 08:40:34 -05:00
Håkon Øye Amundsen c9a2a5e7fb scripts: cmake: move print from python to cmake
Prior to this all hex files included in a merge would
be printed on every invokation of cmake.

Allow the user to explicitly require this information
by moving the print to a VERBOSE-only cmake message.

Signed-off-by: Håkon Øye Amundsen <haakon.amundsen@nordicsemi.no>
2020-01-03 08:20:08 -05:00
David B. Kinder 17299f0734 doc: document python build scripts
We have a collection of python scripts that are part of our build
system.  This PR collects docstring comments added to these scripts into
a summary document.  Previous references to just the script name in
other documentation are updated to point to this build tool
documentation.

Some of the scripts needed an update to be processed (via include
directives) consistently.

Signed-off-by: David B. Kinder <david.b.kinder@intel.com>
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-01-02 17:09:29 -05:00
Erwan Gouriou 17537c77c8 script/Kconfig: kconfig functions: Add shields_list_contains function
Add shields_list_contains Kconfig function which return bool based on
check of shield presence in cmake SHIELD_AS_LIST.

Signed-off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
2020-01-02 17:02:41 -05:00
Ulf Magnusson e181e1b773 kconfiglib: Update to hide tracebacks for expected errors
Update Kconfiglib to upstream revision 9c0b562c94 to get this commit in:

    Add Kconfig.__init__() helper flag for suppressing tracebacks

    Tools that don't use standard_kconfig() currently generate spammy
    tracebacks for e.g. syntax errors.

    Add a suppress_traceback flag to Kconfig.__init__() for catching
    "expected" exceptions and printing them to stderr and exiting with
    status 1. Use it to make all tools consistently hide tracebacks.

Use the new flag to hide tracebacks for expected exceptions in
kconfig.py, lint.py, and genrest.py.

Some menuconfig robustness tweaks for wonky terminals are included as
well, and a new feature for customizing .config and autoconf.h header
comments via environment variables.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-12-20 19:48:09 -05:00
Anas Nashif 2a5d61d3ea sanitycheck: define schema for hardware map
Add schema for hardware map and verify it.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-12-19 22:55:16 -05:00
Anas Nashif fc85ff052f sanitycheck: support pre/post flash scripts
This is needed when a board needs to be reset using an external commands
or tools that are not part of the flash command. For example, power
reset or by poking a GPIO header on the board using external wiring.

add post_flash/pre_flash to the platform section in the hardware map.
For example:

- available: true
  connected: true
  id: OSHW000032254e4500128002ab98002784d1000097969900
  platform: reel_board
  post_script: /tmp/post_flash.sh
  pre_script: /tmp/pre_flash.sh
  product: DAPLink CMSIS-DAP
  runner: pyocd
  serial: /dev/ttyACM11

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-12-19 22:55:16 -05:00
Sebastian Bøe bb95dce98c west: Fix defining modules without a module.yml
It is supported to have a zephyr module that does not have a
module.yml, but zephyr_module.py does not support it and will drop
such modules.

To fix this we add support in zephyr_module.py.

Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
2019-12-19 15:52:44 -05:00
Martí Bolívar 150eb6575c scripts: west build: slight --help tweak
We have an open request to make the help for the -t option a little
easier. Try to do that without adding too much length to the short
help.

Fixes: #16202
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2019-12-19 11:29:39 +01:00
Ulf Magnusson 5e55eda30e scripts: edtlib: Support nested nodes on buses
For the following devicetree, view 'nested' as being on the bus.
Previously, only 'node' was considered to be on the bus.

    some-bus {
    	compatible = "foo,bus-controller";
    	node {
    		nested {
    			compatible = "foo,device-on-bus";
    		};
    	};
    };

In practice, this means that a 'bus:' key in the binding for
'foo,bus-controller' will now get matched up to an 'on-bus:' key in the
binding for 'foo,device-on-bus'.

Change the meaning of Node.bus and add two new attributes Node.on_bus
and Node.bus_node, with these meanings:

    Node.bus:
      The bus type (as a string) if the node is a bus controller, and
      None otherwise

    Node.on_bus:
      The bus type (as a string) if the node appears on a bus, and None
      otherwise. The bus type is determined from the closest parent
      that's a bus controller.

    Node.bus_node:
      The node for the bus controller if the node appears on a bus, and
      None otherwise

It's a bit redundant to have both Node.bus_node and Node.on_bus, since
Node.on_bus is the same as Node.bus_node.bus, but Node.on_bus is pretty
handy to save some None checks.

Also update gen_defines.py to use Node.on_bus and Node.bus_node instead
of Node.parent wherever the code deals with buses.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-12-19 11:02:28 +01:00
Ulf Magnusson 379145ffce scripts: edtlib: Rename 'child-bus:'/'parent-bus:' to 'bus:'/'on-bus:'
I keep mixing these up, so that's probably a sign that the names are
bad. The root of the problem is that "parent-bus" can be read as both
"this is the parent bus" and as "the parent bus is this".

Use 'bus:' for the bus "provider" and 'on-bus:' for nodes on the bus
instead, which is less confusing.

Support the old keys for backwards compatibility, along with a
deprecation warning.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-12-19 11:02:28 +01:00
Anas Nashif 0c69c285ec sanitycheck: cleanup hardware map display
Add a static method for dumping hardware map and reuse it across the
script reducing duplicated code.

Fixes #21475

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-12-18 21:52:02 +01:00
Anas Nashif 81b2f37a9d ci: run sanitycheck in verbose mode
Call sanitycheck in verbose mode

Fixes #21483

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-12-18 12:07:49 -05:00
Andrew Boie 13b8b41676 drivers: watchdog: add system calls
wdt_install_timeout() was skipped as it installs an ISR-context
callback handler function. The rest are simple wrappers.

Added myself as the maintainer of the syscall handlers. WDT
subsystem appears to not currently have an owner.

Fixes: #21432

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-12-18 09:28:52 -05:00
Maksim Masalski 1b4b9ef535 sanitycheck: hardware map added support of the microchip board
Added support of the Microchip with FTDI serial devices to be able
create a hardware map for them. In the future that hardware map
for the Microchip board will let run automatic Sanitycheck tests on it.

Signed-off-by: Maksim Masalski <maksim.masalski@intel.com>
2019-12-18 08:05:05 -05:00
Ulf Magnusson 36b7ca44b8 scripts: edtlib.py: Deprecate 'title:'
Most bindings look something like this:

    title: Foo

    description: This binding provides a base representation of Foo

That kind of description doesn't add any useful information, as it's
just the title along with some copy-pasted text. I'm not sure what "base
representation" was supposed to mean originally either.

Many bindings also put something that's closer to a description in the
title, because it's not clear what's expected or how the title is used.
In reality, the title isn't used anywhere. 'description:' on the other
hand shows up as a comment in the generated header.

Deprecate 'title:' and generate a long informative warning if it shows
up in a binding.

Next commits will clean up the 'description:' strings (bringing them
closer to 'title:' in most cases) and remove 'title:' from all bindings.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-12-18 11:52:45 +01:00
Anas Nashif 17d066b9dc sanitycheck: fix reporting of timeouts
timeouts were not reported correctly and we were getting the default
'N/A' in case of a timeout.

Fixes #21438

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-12-17 17:21:56 -05:00
Anas Nashif 3cfc608934 sanitycheck: remove commented out code
Remove commented out debug code.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-12-17 17:21:56 -05:00
Anas Nashif 9744568350 sanitycheck: use colorama module for colors
Use colorama module for colors instead of custom codes.

Fixes #21409

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-12-17 17:21:56 -05:00
Anas Nashif ebf8dae241 sanitycheck: fix legacy device testing support
When testing with one device and without a hardware map, make sure we
load the temporary map into the suite.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-12-17 17:21:56 -05:00
Anas Nashif d988238970 sanitycheck: pylint issues
run through pylint and fix.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-12-17 17:21:56 -05:00
Anas Nashif 7148021627 sanitycheck: remove unused globals
A few globals that were never used anymore.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-12-17 17:21:56 -05:00
Anas Nashif c5ee3951fd sanitycheck: get rid of more global options
More global option cleanup.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-12-17 17:21:56 -05:00
Anas Nashif e9eb00945d sanitycheck: move log_info_* to ProjectBuilder class
Move this to where it is being used and called.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-12-17 17:21:56 -05:00
Anas Nashif 56656848f5 sanitycheck: do not use gloabl options
More changes moving away from using global options and instead using
class variables and parameters.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-12-17 17:21:56 -05:00
Anas Nashif fa8085eb7b sanitycheck: all output from flasher goes to logger.debug
Redirect all output from flasher to logger.debug().

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-12-17 17:21:56 -05:00