Commit graph

2407 commits

Author SHA1 Message Date
Kumar Gala
a457681b1c scripts/requirements: Split & document requirement.txt
Split up requirements.txt into several files so that CI tools can
utilize/reference the specific requirements-<FOO>.txt they may need
while keep things in sync with the development.  This is to reduce
both time and amount of work CI actions due to python package install.

Create the following groupings:

1. BASE - needed to build or create zephyr images
2. BUILD-TEST - need to run compile/build tests
3. DOC - need to build the docs
4. RUN-TEST - need for runtime testing
5. EXTRAS - optional or useful for development/developers workflow

Also tried to add a comment about what or why a given package is being
pulled in for.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-03-19 15:41:08 +01:00
Andrew Boie
b42fe9c764 scripts: gen_kobject_list.py cleanup
Un-do the changes made to the code when this was separated out
to elf_helper.py.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-03-17 20:11:27 +02:00
Andrew Boie
fc2f7c3a55 scripts: merge elf_helper into gen_kobject_list
No need for this to be separated out any more.
Minimal changes made to get it to still work.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-03-17 20:11:27 +02:00
Andrew Boie
28be793cb6 kernel: delete separate logic for priv stacks
This never needed to be put in a separate gperf table.
Privilege mode stacks can be generated by the main
gen_kobject_list.py logic, which we do here.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-03-17 20:11:27 +02:00
Andrew Boie
ae8acffaa6 doc: fix some missed renames
These were renamed to z_ prefix some time ago, but not updated
in these places.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-03-17 20:11:27 +02:00
Andrew Boie
2dc2ecfb60 kernel: rename struct _k_object
Private type, internal to the kernel, not directly associated
with any k_object_* APIs. Is the return value of z_object_find().
Rename to struct z_object.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-03-17 20:11:27 +02:00
Andrew Boie
4bad34e749 kernel: rename _k_thread_stack_element
Private data type, prefix with z_.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-03-17 20:11:27 +02:00
Andrew Boie
f2734ab022 kernel: use a union for kobject data values
Rather than stuffing various values in a uintptr_t based on
type using casts, use a union for this instead.

No functional difference, but the semantics of the data member
are now much clearer to the casual observer since it is now
formally defined by this union.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-03-17 20:11:27 +02:00
peng1 chen
921226493b sanitycheck: delete the '\r' char in front of serial output
add lstrip function to delete the '\r' char in front of the
serial output from serail readline, because this '\r'
will influence the result match function in harness.py.

Signed-off-by: peng1 chen <peng1.chen@intel.com>
2020-03-17 12:34:21 -04:00
Patrik Flykt
c9ad901fb7 scripts: net: Clarify and add documentation to README
The README file was in need of clarification how to set up the
net-tools Docker image and how the IP addresses were assigned.
mqtt_publisher is also mentioned.

Signed-off-by: Patrik Flykt <patrik.flykt@intel.com>
2020-03-17 13:14:41 +02:00
Patrik Flykt
2b42e1274e scripts: net: Support testing of mqtt_publisher
With net-tools and the mqtt_publisher sample updated, add support
for running the sample with the network sample test script.

Signed-off-by: Patrik Flykt <patrik.flykt@intel.com>
2020-03-17 13:14:41 +02:00
Patrik Flykt
26685b727b scripts: net: Properly return Zephyr exit code to script
Save the exit code from 'wait' so that it is not lost when PID
is set to zero.

Signed-off-by: Patrik Flykt <patrik.flykt@intel.com>
2020-03-17 13:14:41 +02:00
Kumar Gala
9b9436dfbe Revert "scripts: gen_defines: augment edtlib nodes"
This reverts commit dd1d58cad6.

Reverted as the changes to gen_defines.py break things.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-03-14 10:38:52 -06:00
Kumar Gala
6b97a220b4 Revert "scripts: gen_defines: re-work write_regs() with augmented nodes"
This reverts commit ba1c8b523c.

Reverted as the changes to gen_defines.py break things.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-03-14 10:38:52 -06:00
Kumar Gala
c6a7ecda93 Revert "scripts: gen_defines: re-work write_irqs() with augmented nodes"
This reverts commit f68da76d36.

Reverted as the changes to gen_defines.py break things.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-03-14 10:38:52 -06:00
Kumar Gala
6499addb84 Revert "scripts: gen_defines: re-work write_clocks() with augmented nodes"
This reverts commit c048f872d7.

Reverted as the changes to gen_defines.py break things.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-03-14 10:38:52 -06:00
Kumar Gala
1ebe6f2bdd Revert "scripts: gen_defines: re-work write_props() with augmented nodes"
This reverts commit 86645aafdf.

Reverted as the changes to gen_defines.py break things.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-03-14 10:38:52 -06:00
Kumar Gala
a3049ad801 Revert "scripts: gen_defines: re-work write_spi_dev() with augmented nodes"
This reverts commit 0471199e63.

Reverted as the changes to gen_defines.py break things.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-03-14 10:38:52 -06:00
Kumar Gala
98ae26f689 Revert "scripts: gen_defines: re-work write_bus() with augmented nodes"
This reverts commit 3939556fc3.

Reverted as the changes to gen_defines.py break things.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-03-14 10:38:52 -06:00
Kumar Gala
22342bec9d Revert "scripts: gen_defines: remove out()'s aliases kwarg"
This reverts commit 47073fd11c.

Reverted as the changes to gen_defines.py break things.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-03-14 10:38:52 -06:00
Kumar Gala
b379e4547f Revert "scripts: gen_defines: tweak out_define() whitespace"
This reverts commit 3b7ddc83d1.

Reverted as the changes to gen_defines.py break things.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-03-14 10:38:52 -06:00
Corey Wharton
86bfc489f4 scripts: Replace hard-coded subsystem list with __subsystem annotations.
This change removes the hardcoded subsystem list in gen_kobject_list.py
favor of marking the relevant driver API structs with the _subsystem
sentinel.

Signed-off-by: Corey Wharton <coreyw7@fb.com>
2020-03-11 17:13:39 +02:00
Corey Wharton
ccd15df510 scripts: Dynamically add driver subsystems to subsystems list
This change extends the parse_syscalls.py script to scan for a
__subsystem sentinal added to driver api declarations. It thens
generates a list that is passed into gen_kobject_list.py to extend
the subsystems list. This allows subsystems to be declared in the
code instead of a separate python list and provides a mechanism for
defining out-of-tree subsystems.

Signed-off-by: Corey Wharton <coreyw7@fb.com>
2020-03-11 17:13:39 +02:00
Kumar Gala
0353d22202 kconfig: Add dt_nodelabel_enabled function
Add a function takes a 'label' and returns "y" if we find an
"enabled" node that has a 'nodelabel' of 'label' in the EDT
otherwise we return "n"

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-03-11 07:59:38 -06:00
Martí Bolívar
7846fd103d edtlib: add 'labels' attribute to Node class
This corresponds to the attribute by the same name in dtlib.Node.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-03-11 07:59:38 -06:00
Martí Bolívar
3b7ddc83d1 scripts: gen_defines: tweak out_define() whitespace
Pad node identifiers to 60 characters. This results in better
alignment in practice than the current value of 40, which is a bit
low.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-03-11 07:59:38 -06:00
Martí Bolívar
47073fd11c scripts: gen_defines: remove out()'s aliases kwarg
Write the same results explicitly in its only remaining caller.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-03-11 07:59:38 -06:00
Martí Bolívar
3939556fc3 scripts: gen_defines: re-work write_bus() with augmented nodes
This too is an attempt to reimplement the previous behavior exactly,
modulo the order in which things are defined.

This is the last function which is calling into the previous
implementation's out_node and node_*_alias() functions, so these can
be removed now.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-03-11 07:59:38 -06:00
Martí Bolívar
0471199e63 scripts: gen_defines: re-work write_spi_dev() with augmented nodes
This uses augmented nodes in the same way as done previously.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-03-11 07:59:38 -06:00
Martí Bolívar
86645aafdf scripts: gen_defines: re-work write_props() with augmented nodes
This is similar to the work already done for regs.

Other than the order in which they appear and comments, the output
before and after this patch should be exactly the same.

We're intentionally leaving some of the helpers in module scope here
to use some of the subroutines elsewhere later on when reworking
write_spi_dev().

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-03-11 07:59:38 -06:00
Martí Bolívar
c048f872d7 scripts: gen_defines: re-work write_clocks() with augmented nodes
Mirror the change already done to write_regs().

Other than the order in which they appear and comments, the output
before and after this patch should be exactly the same.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-03-11 07:59:38 -06:00
Martí Bolívar
f68da76d36 scripts: gen_defines: re-work write_irqs() with augmented nodes
Use augmented nodes to print macros grouped by namespace.

Other than the order in which they appear and comments, the output
before and after this patch should be exactly the same.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-03-11 07:59:38 -06:00
Martí Bolívar
ba1c8b523c scripts: gen_defines: re-work write_regs() with augmented nodes
Group the macros together by namespace rather than putting all the
BASE_ADDRESS macros together and all the SIZE macros together. E.g.,
all the DT_INST_<x> namespace macros for each node now appear
consecutively.

Add a comment making it clear that this output comes from "regs",
since "BASE_ADDRESS" and "SIZE" are not property names.

Other than the order in which they appear and comments, the output
before and after this patch should be exactly the same.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-03-11 07:59:38 -06:00
Martí Bolívar
dd1d58cad6 scripts: gen_defines: augment edtlib nodes
Add additional attributes to each edtlib.Node we process, before
calling into the write_foo() routines.

This includes the identifier returned by node_ident(), which is used
as the primary identifier for the node, as well as lists for instance
and aliases nodes, and a catchall list that contains all the other
identifiers in addition to the primary one.

Use this information in a new for_each_ident(node, ident) def that
will be put to use in the various write_foo() routines.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-03-11 07:59:38 -06:00
Ulf Magnusson
8cacb9fd8c dts: edtlib: Turn global spi_dev_cs_gpio() into Node.spi_cs_gpio
spi_dev_cs_gpio() takes a Node and returns the chip select GPIO for it.
Having that information available directly from Node is neater, so turn
it into a Node.spi_cs_gpio property instead.

That gets rid of the only public global function in edtlib, which might
make the API design clearer too.

Tested with the sensortile_box board, which uses SPI chip select.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2020-03-11 07:59:38 -06:00
Ulf Magnusson
35166c45d9 dts: edtlib/gen_defines: Fix API design re. dtc flags
It's better to allow per-instance EDT configuration than to set a global
variable on the edtlib module. Enable/disable the warning for reg/unit
address mismatches via a flag to EDT.__init__(), instead of via a global
variable. That makes it consistent too.

Another option would be to pass the 'dtc' flags to EDT.__init__(), but
it makes the interface a bit ugly. Maybe if it needs to emulate lots of
other flags later.

Clarify that edtlib itself isn't meant to have any state in the comment
at the top of the module.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2020-03-11 07:59:38 -06:00
Patrik Flykt
cbef6760c1 scripts: net: Enhance error reporting
Enhance error reporting by returning immediately when the calling
function reports an error.

Signed-off-by: Andrei Laperie <andrei.laperie@intel.com>
Signed-off-by: Patrik Flykt <patrik.flykt@intel.com>
2020-03-10 16:43:10 +02:00
Patrik Flykt
3401192e7f scripts: net: Add option to keep container after test
Add '--keep' option to leave Docker container and network running
after the test.

Signed-off-by: Patrik Flykt <patrik.flykt@intel.com>
2020-03-10 16:43:10 +02:00
Patrik Flykt
ad1b38bebe scripts: net: Return final return value from Docker test
Return the final return value from Docker script when running echo
client in the container, first for UDP and then TCP, for both IPv4
and IPv6.

Signed-off-by: Patrik Flykt <patrik.flykt@intel.com>
2020-03-10 16:43:10 +02:00
Patrik Flykt
8f451b7338 scripts: net: Add overlay file to sample script
Add an '--overlay' command line option in order to use an overlay
file when compiling Zephyr.

Signed-off-by: Patrik Flykt <patrik.flykt@intel.com>
2020-03-10 16:43:10 +02:00
Martí Bolívar
34a59168d5 build: clean up west build
The header printed at the beginning of every west build is kind of
annoying. Let's remove it and just print the application source
directory at cmake time instead. The build directory and board are
already printed there, anyway, and that's all IDE users will see.

Let's clean up the BOARD to make it say "board" instead. That matches
the west build --board option name a bit more closely and is still
legible.

Likewise, let's not print any west build messages if we're just
incrementally recompiling. That's noisy.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-03-10 14:53:28 +02:00
Mikkel Jakobsen
98eb316bdf scripts: build.py: make _sanity_check() case insensitive on windows
When checking build directory against cache on windows, certain
corner cases can end up failing the comparison because of case
difference on an otherwise identical path. This can be avoided
by ignoring case on windows.

Signed-off-by: Mikkel Jakobsen <mikkel.aunsbjerg@prevas.dk>
2020-03-10 12:19:35 +02:00
Anas Nashif
497fe4267a release: update sanitycheck footprint file
Update for latest release.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-03-09 23:40:45 +02:00
Martí Bolívar
67d8f755e2 runners: misc-flasher: hotfix
The 'command' command line argument for this flasher is now being
overridden by common code, which attaches the west subcommand name to
this.

Let's just hotfix this by renaming the argument in misc-flasher.
We can revisit the boundary between run_common.py arguments and
runners package arguments after the release.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-03-04 21:48:09 +02:00
Martí Bolívar
24ce05c841 doc: improve west flash help for dfu-util
Try to make it clearer what's going on here.

Suggested-by: Lucian Copeland <hierophect@gmail.com>
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-03-03 09:30:32 +02:00
Martí Bolívar
dc0948424e scripts: run_common: fix command line --hex-file and friends
This is yet another bug introduced by the move to runners.yaml.
Sigh. I should have tested this better.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-02-29 19:38:49 +02:00
Martí Bolívar
f8cb3d4d53 scripts: run_common: rename 'args' to 'user_args'
This will also make the reason for a following bug fix easier to see.

Update a comment block to include all the work that needs doing.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-02-29 19:38:49 +02:00
Martí Bolívar
97dbda2088 scripts: run_common: rename unknown_args to user_runner_args
This makes its true value clearer, and will make a later bug fix patch
easier to understand.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-02-29 19:38:49 +02:00
Martí Bolívar
6dab163cad scripts: fix west sign when ZEPHYR_BASE is unset
Due to cleanups in west targeted at getting rid of zephyr-specific
code, extension commands can no longer rely on ZEPHYR_BASE being set
in the calling environment at import time (it's still set at run()
time for now, though, to keep west build working).

Add a new helper to make dealing with this easier from west sign.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-02-20 09:06:09 +02:00
Martí Bolívar
eb95bed552 scripts: west_commands: fix argument handling with runners.yaml
Refactor the code to support the new runners.yaml file created by the
build system.

Compared to fishing around in the CMake cache, this makes it trivial
to put all the command line arguments to a runner-based command on
equal footing, regardless of if they're defined in the runners package
proper or defined in run_common.py.

This allows board.cmake files to do things like this:

   board_set_runner_args(foo
     --bin-file=${PROJECT_BINARY_DIR}/my-signed.bin)

While at it, make some other cleanups:

- Stop using the obsolete and deprecated west.cmake module while we're
  here in favor of the zcmake.py module which was added to Zephyr a long
  time ago. Yikes. I had forgotten this was still here.

- Stop using west.util's wrap function in favor of raw use of
  textwrap. The west function splits on hyphens, which is breaking
  runner names like "em-starterkit".

- Clean up the --context output a bit

Fixes: #22563
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-02-20 09:06:09 +02:00