Commit graph

5,899 commits

Author SHA1 Message Date
Gerard Marull-Paretas
b0d4aea769 scripts: migrate_includes: migrate zephyr.h to kernel.h
The script will now automatically migrate all <zephyr.h> or
<zephyr/zephyr.h> (for already migrated projects) to <zephyr/kernel.h>.

Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
2022-09-05 16:31:47 +02:00
Gerard Marull-Paretas
79e6b0e0f6 includes: prefer <zephyr/kernel.h> over <zephyr/zephyr.h>
As of today <zephyr/zephyr.h> is 100% equivalent to <zephyr/kernel.h>.
This patch proposes to then include <zephyr/kernel.h> instead of
<zephyr/zephyr.h> since it is more clear that you are including the
Kernel APIs and (probably) nothing else. <zephyr/zephyr.h> sounds like a
catch-all header that may be confusing. Most applications need to
include a bunch of other things to compile, e.g. driver headers or
subsystem headers like BT, logging, etc.

The idea of a catch-all header in Zephyr is probably not feasible
anyway. Reason is that Zephyr is not a library, like it could be for
example `libpython`. Zephyr provides many utilities nowadays: a kernel,
drivers, subsystems, etc and things will likely grow. A catch-all header
would be massive, difficult to keep up-to-date. It is also likely that
an application will only build a small subset. Note that subsystem-level
headers may use a catch-all approach to make things easier, though.

NOTE: This patch is **NOT** removing the header, just removing its usage
in-tree. I'd advocate for its deprecation (add a #warning on it), but I
understand many people will have concerns.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2022-09-05 16:31:47 +02:00
Emil Gydesen
174c74206e footprint: Add bt_hci_rpmsg with ISO for the nRF5340 netcore
Add footprint tracking of the samples/bluetooth/hci_rpmsg
sample for the nRF5340 with ISO broadcast and ISO receive respectively.

Signed-off-by: Emil Gydesen <emil.gydesen@nordicsemi.no>
2022-09-02 11:08:31 +00:00
Maureen Helm
5b5aa6ebba scripts: gen_defines: Add vendor name macros based on vendor prefixes
Adds vendor name and existence macros for all compatibles of a node that
match an entry in the vendor prefixes file.

Signed-off-by: Maureen Helm <maureen.helm@intel.com>
2022-09-01 14:53:55 -07:00
Maureen Helm
a1b0c39c37 scripts: edtlib: Add compat2vendor LUT constructed from vendor prefixes
Adds a new compat2vendor lookup table that maps compatibles to vendor
names, constructed from the vendor prefixes file. This approach is a
more scalable alternative to adding a vendor name property to devicetree
bindings, as was previously proposed.

Signed-off-by: Maureen Helm <maureen.helm@intel.com>
2022-09-01 14:53:55 -07:00
Torsten Rasmussen
8d2998d4f9 cmake: rework of host tools and generic toolchain handling
Follow-up: #41301

This commit is a rework and cleanup of the tools handling in Zephyr
CMake build system.

Instead of directly loading code a CMake modules for tool lookup, the
host tools now follows the CMake `find_package()` pattern for finding
programs / tools in module mode.

This makes it more clear which modules are responsible for finding tools
and which modules provides build integration / features.

The following tools can now be found using `find_package()`:
- Zephyr-sdk        : find_package(Zephyr-sdk <version>)
- Generic host tools: find_package(HostTools)

This further allows us to decouple the `verify-toolchain` CMake script
part required by `twister` into a tool lookup module and a dedicated
CMake script which utilizes the lookup module.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2022-09-01 15:59:43 +02:00
Tristan Honscheid
52db2d8279 twister: Remove import exception handler for psutil
`handlers.py` treats the `psutil` package as optional and quietly prints
a message to stdout if it is not present, but it is actually a hard
requirement for the base `Handler` class. This try/except block is bad
as it hides the requirement until a timeout occurs and the handler
attempts to use `psutil` in the terminate method, which will crash if
the module happens to be missing.

This PR removes the try/except guard so missing `psutil` will cause
twister to immediately fail. `psutil` is already specified in the
`requirements.txt` file, so this should only affect users who
unwittingly have an outdated/incorrect environment.

Also update the Github Action for twister tests to install from
requirements files.

Signed-off-by: Tristan Honscheid <honscheid@google.com>
2022-09-01 06:22:04 -04:00
Erwan Gouriou
7cd727b003 scripts: requirements-extras: Request imgtool 1.9 for write-block-size > 8
On some targets (such as STM32U5) using write-block-size >8, imgtool
should be called using option '--align 16/32' which is only available
starting version 1.9.

Update requirement for imgtool to make it available to such platforms.

Signed-off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
2022-09-01 10:28:33 +02:00
Andrzej Kaczmarek
868acfd505 scripts: Add ezFlashCLI runner
This adds support for ezFlashCLI tool to allow flashing DA1469x MCU
family from Renesas.

The tool is available at https://github.com/ezflash/ezFlashCLI or can
be installed via pip.

Signed-off-by: Andrzej Kaczmarek <andrzej.kaczmarek@codecoup.pl>
Signed-off-by: Ben Lauret <ben.lauret.wm@renesas.com>
2022-08-31 20:32:03 +02:00
Emil Gydesen
6cc8b57bc5 footprint: Add bt_hci_rpmsg for the nRF5340 netcore
Add footprint tracking of the samples/bluetooth/hci_rpmsg
sample for the nRF5340. This is a BT controller-only build
for the nRF5340 with minimum features.

Signed-off-by: Emil Gydesen <emil.gydesen@nordicsemi.no>
2022-08-31 10:41:32 +02:00
Carles Cufi
8644b333cd west: Update bash autocomplete file
Update the existing bash autocomplete with the latest commands and
command-line switches.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-08-31 10:38:01 +02:00
Gerard Marull-Paretas
fff9ecbc7f devicetree: add DT_(INST_)FOREACH_CHILD(_STATUS_OKAY)_SEP(_VARGS)
It is frequent to see in Devicetree code constructs like:

```c
 #define NAME_AND_COMMA(node_id) DT_NODE_FULL_NAME(node_id),

const char *child_names[] = {
	DT_FOREACH_CHILD(DT_NODELABEL(n), NAME_AND_COMMA)
};
```

That is, an auxiliary macro to append a separator character in
DT_FOREACH* macros. Non-DT API, e.g. FOR_EACH(), takes a separator
argument to avoid such intermediate macros.

This patch adds DT_FOREACH_CHILD_SEP (and instance/status okay/vargs
versions of it). They all take an extra argument: a separator. With this
change, the example above can be simplified to:

```c
const char *child_labels[] = {
	DT_FOREACH_CHILD(DT_NODELABEL(n), DT_NODE_FULL_NAME, (,))
};
```

Notes:
- Other DT_FOREACH* macros could/should be extended as well

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2022-08-30 16:19:57 +02:00
Gerard Marull-Paretas
d77f4e61de scripts: dts: gen_defines: add DT_DEBRACKET_INTERNAL helper
Add a utility macro used to remove brackets from around a single
argument. While __DEBRACKET exists in util_internal.h, this change makes
DT independent (otherwise we should include util_internal.h, which is
another option).

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2022-08-30 16:19:57 +02:00
Thomas Stranger
f123094482 scripts: west: runners: stm32cubeprogrammer: enable exec from PATH
Allows to use the stm32cubeprogrammer runner on Linux to be executed
from any installation directory as long as the executable is found
in PATH.

Until now on Linux the programmer could only be used if it either has
been installed in the default location or if the executable path was
passed via the cli parameter.

Signed-off-by: Thomas Stranger <thomas.stranger@outlook.com>
2022-08-29 17:31:56 +02:00
Carles Cufi
035dffcfcf scripts: modules: Mark the build as tainted if blobs present
Whenever binary blobs are present in the filesystem (i.e. fetched,
regardless of whether they are up-to-date or not) we consider the Zephyr
build as tainted. In order to ensure that it is marked as so, add a
Kconfig `select` statement whenever this is the case.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-08-29 15:57:59 +02:00
Carles Cufi
b662bc9093 scripts: modules: Move blob processing to zephyr_module.py
In preparation for the handling of taint flags in zephyr_module.py, move
blob-processing code from the west command to it.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-08-29 15:57:59 +02:00
Yuval Peress
7187d8b7fc ztest: add framework tests for failed states
Add specialized tests that execute another test indirectly and ensure
that the results report errors correctly. See the README.rst file
provided with this commit for details.

Signed-off-by: Yuval Peress <peress@google.com>
2022-08-29 10:27:00 +02:00
Jeremy Bettis
562b291704 twister: Set stdin explicitly on stty calls
stty operates on the file descriptor from stdin, but we want to operate
on stdout. If stdin and stdout are not the same tty then stty prints an
error.

Redirect stdout to stdin for stty calls.

Tested by running twister -T tests/subsys/shell </dev/null

Signed-off-by: Jeremy Bettis <jbettis@google.com>
2022-08-26 12:51:28 +02:00
Martí Bolívar
88aa873fbc twister: improve logging of cmake commands
With verbose output enabled, twister logs the cmake arguments as a
python array whenever it invokes cmake to generate a build system.

This is clunky output for manual reproduction, because it forces the
developer to manually join the arguments together for consumption by
the shell.

As an enhancement for this use case, use the standard library
shlex.join() function to produce a string that can be copy/pasted
directly into the shell on POSIX platforms.

We should use this function instead of str.join, because it correctly
handles things like shell quoting for options with spaces in them.

This function is not available on Windows, so we fall back on behavior
that is similar to the old one there.

The main difference on Windows is that the output now includes the
path to cmake as well, and not just its arguments. (This is the same
on POSIX, and is intended to help debug if multiple cmakes are
installed on the same machine.)

We are running cmake from a couple of different places, so doing this
cleanly requires adding a shared module with the platform abstraction.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2022-08-26 08:53:14 +02:00
Kumar Gala
f1660f4d51 edtlib: allow const arrays
Allow for having array types (array, uint8-array, string-array) be const.
This would allow for something like:

properties:
    reg-names:
        const: ["foo", "bar"]

To be supported.

Renamed function _check_prop_type_and_default to _check_prop_by_type
as part of this change and Moved the check for 'const' types into
_check_prop_by_type as its similar to the prop_type check and it was
easier to implement in _check_prop_by_type as we already extract
prop_type from the option in that function.

Signed-off-by: Kumar Gala <galak@kernel.org>
2022-08-25 18:11:27 -07:00
Kumar Gala
8a5d568a79 dts: Standardize string token names for arrays
Use the same postfixes for string arrays as we use for strings.  Keep
just the _STRING_*_TOKEN) variants.

Signed-off-by: Kumar Gala <galak@kernel.org>
2022-08-25 15:00:24 -07:00
Carles Cufi
25db534a07 west: blobs: Finally fix the --format option handling
Turns out that `Namespace.format` was not a good fix to the issue
described in f033040812, it was returning
the default format even when the user specified -f/--format.

Replace "getattr" with a simple logical operation.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-08-24 17:31:24 +02:00
Carles Cufi
f033040812 west: blobs: Fix --format handling
There were two issues with the original commit
e04487cad2:

- getattr() doesn't seem to work properly with args
- Comparison between bound methods with "is" seems not to work, so
  replace it with a simpler string comparison

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-08-24 07:38:42 -07:00
Carles Cufi
8f0af41fd8 scripts: compliance: Add commit range info output
Log at the info level the commit range that the script is acting upon.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-08-24 07:40:27 -04:00
Carles Cufi
0082dbfbd5 scripts: compliance: Remove unused command-line args
There were a bunch of old command-line arguments that haven't been used
in a while, remove them.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-08-24 07:40:27 -04:00
Carles Cufi
73598db4db scripts: compliance: Tiny cleanup for -c
Use a common source for default and doc.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-08-24 07:40:27 -04:00
Carles Cufi
c10a0f28b4 scripts: compliance: Clean up logging
- Add a choice in the cmd-line param
- Use the logger object everywhere
- Remove stale todo

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-08-24 07:40:27 -04:00
Carles Cufi
d160d0c20c scripts: compliance: Complete transition to junitparser v2
Follow-up to b187477fe7.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-08-24 07:40:27 -04:00
Carles Cufi
5ee76c97ab scripts: compliance: Remove unused variable
EDIT_TIP has not been used for a while now, remove it.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-08-24 07:40:27 -04:00
Peter Marheine
d400b8135c arch/riscv: support CONFIG_CODE_DATA_RELOCATION
This implements support for relocating code to chosen memory regions via
the `zephyr_code_relocate` CMake function for RISC-V SoCs. ARM-specific
assumptions that were made by gen_relocate_app.py need to be corrected,
in particular not assuming any particular name for the default RAM
section (which is 'SRAM' for most ARM pltaforms) and not assuming 32-bit
pointers (so the test works on RV64).

Signed-off-by: Peter Marheine <pmarheine@chromium.org>
2022-08-24 10:08:06 +02:00
Peter Marheine
9bd46852a3 build: don't assume FLASH is the ROM region
Most ARM platforms name their ROM region `FLASH`, but this assumption is
not portable. Have gen_relocate_app refer to ROMABLE_REGION instead of
hard-coding the platform-specific memory region name.

Signed-off-by: Peter Marheine <pmarheine@chromium.org>
2022-08-24 10:08:06 +02:00
Peter Marheine
3a8c20d849 build: don't assume .symtab section type
Only the elftools SymbolTableSection section type provides an
iter_symbols() method, and compilers are free to emit sections that
include the substring '.symtab' in their name which causes errors if
gen_relocate_app examines only the section name. Instead check whether a
section is a symbol table to skip attempting to inspect sections that
are not actually symbol tables.

Signed-off-by: Peter Marheine <pmarheine@chromium.org>
2022-08-24 10:08:06 +02:00
Kumar Gala
4f84bc8f40 scripts/ci/check_compliance.py: Allow to run w/ZEPHYR_BASE
Tweak check_compliance.py to allow user to run it w/o having
ZEPHYR_BASE set.  This is similar to what we do for twister.

Signed-off-by: Kumar Gala <galak@kernel.org>
2022-08-23 13:26:27 +02:00
Radosław Koppel
7614110a20 dts: Add _STRING_TOKEN and _STRING_UPPER_TOKEN to string-array
This commit adds string token versions of the values also
in items inside string-array.

Signed-off-by: Radosław Koppel <r.koppel@k-el.com>
Co-authored-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
Co-authored-by: Kumar Gala <galak@kernel.org>
2022-08-22 13:58:30 -05:00
Martí Bolívar
21214cc83f west blobs: decrease verbosity
Make some no-op log messages require 'west -v blobs ...'.

The current implementation won't scale as the number of blobs goes up.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2022-08-22 16:37:07 +00:00
Martí Bolívar
e04487cad2 west blobs: fix --format handling
This option does not apply to 'west blobs clean' or 'west blobs
fetch'. Error out if it is given to either of those.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2022-08-22 16:37:07 +00:00
Martí Bolívar
b4c4736fb0 west blobs: tweak --format help
Make it consistent with the usual argparse style by using lower case
and no period at the end.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2022-08-22 16:37:07 +00:00
Martí Bolívar
a3f4edb127 west blobs: fix modules help
The help text is mixing up 'blobs' with 'modules', and doesn't
document the new behavior that uses all modules when nothing is
mentioned. Fix both issues.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2022-08-22 16:37:07 +00:00
Martí Bolívar
3b84dff0a5 west blobs: fix subcmd help
Without this patch, "west blobs -h" says:

  {list,fetch,clean} Select the sub-command to execute. Sub-commands
  available: - list: list binary blobs - fetch: fetch and store binary
  blobs - clean: remove fetched binary blobs

With this patch:

  {list,fetch,clean}    sub-command to execute

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2022-08-22 16:37:07 +00:00
Kumar Gala
f16e2a9d80 twister: Add esp32 section name to overflow check
esp32 platforms can overflow their dram0_1_seg so add that section name
to the overflow check in twister.  We were seeing this with the
samples/arch/smp/pi/sample.smp.pi test and would get:

 zephyr_pre0.elf section `noinit' will not fit in region `dram0_1_seg'
 region `dram0_1_seg' overflowed by 17456 bytes

Fixes #49164

Signed-off-by: Kumar Gala <galak@kernel.org>
2022-08-22 12:04:24 -04:00
Martí Bolívar
b3151bcea3 scripts: fix "kconfig: tweak dt_chosen_label"
Commit 57538262e1
(" scripts: kconfig: tweak dt_chosen_label") forgot to update the
docstring for a function whose behavior was changed. Fix it.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2022-08-19 07:03:31 -05:00
Gerard Marull-Paretas
2d712c6c55 scripts: west_commands: runners: jlink: support pylink >= 0.14.2
It looks like the latest release, 0.14.2, changed the contents of
JLINK_SDK_NAME as it was before 0.14.0 release. That means that the
previous fix is only applicable to a couple of releases: 0.14.0/0.14.1.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2022-08-19 14:00:26 +02:00
Torsten Rasmussen
f249f278bf west: warn that using runner args on multiple domains is experimental
The introduction of sysbuild and west build / flash / debug / etc.
support raises questions regarding how to handle user specified runner
arguments.

Some arguments may be acceptable for all domains, whereas others are
only valid for a single or few domains in a multi domain build.

Therefore, consider the usages of runner specific options experimental
when applied to multiple domains.

The following warning is printed to the user when using runner specific
arguments on multiple domains:
> WARNING: Specifying runner options for multiple domains is
> experimental.
> If problems are experienced, please specify a single domain using
> '--domain <domain>'

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2022-08-19 09:21:38 +02:00
Carles Cufi
0424509d6e west: blobs: Ensure that folder exists before fetching
The `<module>/zephyr/blobs/` folder, as well as any additional
sub-folders that might be required under `blobs/` may not exist when
fetching a blob. Ensure that the folder exists, create it if it doesn't.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-08-18 15:25:43 +00:00
Carles Cufi
5524f7a950 west: blobs: Add a clean sub-command
The new "clean" blobs sub-command deletes any blobs present on the
filesystem.

Also improve the help text for sub-commands.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-08-18 15:25:43 +00:00
Carles Cufi
02c95c64fe west: blobs: Rework the -m option
To align with other commands, such as "west update", convert the
accumulative -m option into an nargs="*"-type one that takes the list of
modules directly without a flag.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-08-18 15:25:43 +00:00
Carles Cufi
389af0885c west: blobs: Default to listing and fetching all blobs
Instead of both listing and fetching only the blobs in the
modules listed by the user in the command-line, default to
listing/fetching them all for ease-of-use and consistency with other
commands. Remove the --all options since it's meaningless now.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-08-18 15:25:43 +00:00
Daniel Leung
835680f0e7 twister: enhance support for board@revision as platform name
This adds an extra layer to match the revision part where this
needs to be of permitted patterns as described in.
cmake/modules/extensions.cmake. Without this, the matching
may produce undesirable results. For example, if there is
a file named qemu_x86_tiny_1.conf, the testplan will create
a board name qemu_x86@tiny.1 (which is definitely not correct).

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2022-08-18 17:23:18 +02:00
Torsten Rasmussen
c3620c8a8e scripts: remove ZEPHYR_BASE as default root for board listing
Remove ZEPHYR_BASE as default root in list_boards.py.

This allows list_boards.py to be used to only print boards at given
root(s) without printing Zephyr default boards.

Secondly it remove the need in list_boards.py to have any knowledge of
ZEPHYR_BASE.

The `west boards` command already has ZEPHYR_BASE knowledge and can
easily add ZEPHYR_BASE to the list of roots it is already constructing,
thus removing the need for knowing that ZEPHYR_BASE is always added.

Update boards.cmake to pass ZEPHYR_BASE as an additional arch root, as
arch.cmake has not yet been processed,which means ZEPHYR_BASE is missing
in ARCH_ROOT list at this point in time.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2022-08-18 14:29:14 +02:00
Eivind Jølsgard
3865e08c96 scripts/Kconfig: add dt_node_array_prop Kconfig functions
The motivation is to allow accessing array elements from device tree
properties in Kconfig.
* Add dt_node_array_prop and _node_array_prop functions to extract the
array elements from properties.
* Add 'dt_node_array_prop_int' and 'dt_node_array_prop_hex' keys to use
in Kconfig.

Signed-off-by: Eivind Jølsgard <eivind.jolsgard@nordicsemi.no>
2022-08-18 12:21:33 +02:00