Commit graph

5615 commits

Author SHA1 Message Date
Fabio Baltieri
d43e4e9ca4 scripts: dump_bugs_pickle: strip bug titles
Strip trailing and leading whitespaces from but titles so that they can
be used directly in the release notes.

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2022-09-27 16:51:45 -05:00
Fabio Baltieri
47ae66d828 scripts: release: strip title names
Make sure listed titles have no trailing/leading space (they often do
and right now they have to be cleaned manually before going into the
release notes file).

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2022-09-27 16:51:45 -05:00
Fabio Baltieri
daff79b957 scripts: release: set per_page=100 for issue requests
GitHub issue API allows a max of 100 entries per page (up to a default
of 30). Use it for list_issues, makes the script runs a lot faster.

Api documentation:
  https://docs.github.com/en/rest/issues/issues
  https://docs.github.com/en/rest/pulls/pulls

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2022-09-27 16:51:45 -05:00
Stephanos Ioannidis
f503847d14 doc: Do not use Sphinx 5.2.0.post0
Sphinx 5.2.0.post0 release has a known compatibility issue with the
`sphinx_rtd_theme` (see the issue readthedocs/sphinx_rtd_theme#1343).

Revert this commit once this compatibility issue is resolved.

Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
2022-09-25 10:37:32 +02:00
Ming Shao
2cea5850e2 twisterlib: re-add the logic to drain the serial leftover
This logic was added before but seems to be missed during the
twister refactoring.

Some tests can cause serial leftover logs buffered somewhere.
Such leftover can interfere with the next test case because the
serial log monitoring thread is started before a board is flashed.
And the monitoring thread can be fooled by such leftover logs and
make incorrect judgement of the test result.

A simple ser.flush() is not enough to eliminate such leftovers.
So add explicit readline() to drain such logs which ensures a clean
serial context for the case that follows.

An example from reel board captured with this patch:

leftover log of previous test: b'.287 seconds\r\n'
leftover log of previous test: b' - SKIP - [...test_coredump_backend]...
leftover log of previous test: b'\r\n'
leftover log of previous test: b'------ TESTSUITE SUMMARY END ------\r\n'
leftover log of previous test: b'\r\n'
leftover log of previous test: b'=====================================...
leftover log of previous test: b'RunID: 4e93757ad...53dcab9f0f5c6\r\n'
leftover log of previous test: b'PROJECT EXECUTION SUCCESSFUL\r\n

Signed-off-by: Ming Shao <ming.shao@intel.com>
2022-09-22 15:15:55 -05:00
Fabio Baltieri
96fc137654 twister: treat imgtool image exceeded as flash overflow
Twister detects FLASH overflow and skips tests that trigger that
condition by default, but sometimes images are just on the flash size
limit and then overflows at a later stage when imgtool adds a trailer,
which gets detected as a build fail and fails the run.

This detects the imgtool flash overflow together with the normal build
ones, causing the build to be skipped on imgtool flash overflow as well.

Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
2022-09-22 12:41:25 +00:00
Anas Nashif
b3ead37efa twister: temporarily disable mec15xx boards
A workaround to exclude boards from twister. Other solutions do not work,
setting twister fields to false in the board yaml file is not enough,
given that this board has many associated samples and tests, so it can't
be disabled on its own.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2022-09-21 18:05:51 +00:00
Martí Bolívar
22012894c0 dtlib: error out on duplicate node names
Attempts to define two nodes with the same name within a single set of
curly brackets should fail.

For example, this is invalid DTS according to dtc:

    / { foo {}; foo {}; };

By contrast, this is valid since the node named 'foo' appears twice in
two different sets of curly brackets:

    / { foo {}; };
    / { foo {}; };

Zephyr's dtlib currently does not error out on the invalid condition.

Now that Zephyr itself has been updated to not include such nodes (to
the best of my ability), we can fix this divergence from current dtc
behavior and add a regression test in dtlib.

Fixes: #49590
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2022-09-21 07:55:40 -07:00
Yves Vandervennet
9a4272853e west: jlink: adding optional flash loader flag required for some boards
With revision 7.70d, Segger changed the command line interface of the
jlink client, requiring some flash specific information, such as
the base address and the type of memory, e.g. QSPI.

This commit adds an optional argument to the West jlink runner,
'--loader' which passes the information to the jlink client.
This change is backward compatible with versions of jlink older
than v7.70d which introduces the CLI change.

Fixes: #50327

Signed-off-by: Yves Vandervennet <yves.vandervennet@nxp.com>
2022-09-21 09:33:42 -05:00
Daniel DeGrasse
6064da8690 scripts: kconfig: update size functions to convert bits to bytes
Update size functions within kconfig helpers to support converting
values stored in kilobits, mebibits, or gibibits to kilobytes,
mebibytes, and gibibytes via use of different size arguments.

Signed-off-by: Daniel DeGrasse <daniel.degrasse@nxp.com>
2022-09-21 15:22:36 +02:00
Torsten Rasmussen
65f45507a5 west: fix warning on user args in run_common
When using west runners with multi domains we want to warn users when
they user runner args on multi domain builds.

Therefore fix the `>1` to the correct `>0` which ensures the warning
is printed in all expected cases.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2022-09-20 16:58:09 +02:00
Daniel Leung
418c915f1b dts: add primitive support for multi-bus in bindings
For a single bus that supports multiple protocols, e.g. I3C and I2C,
the single value "bus:" setting is no longer sufficient, as a I3C bus
cannot be matched to a device having "on-bus: I2C". This commit
extends the "bus:" setting so that it can accept a list of values.
This change allows corresponding devicetree macros to be generated
so that DT_ON_BUS() can work properly in this scenario.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2022-09-09 17:42:33 -04:00
Stephanos Ioannidis
3e706376e6 scripts: ci: test_plan: Specify ZEPHYR_BASE for list_boards
The commit c3620c8a8e changed the
`list_boards` script such that it no longer searches the boards from
the `ZEPHYR_BASE` path by default.

This commit updates the `test_plan` script to invoke the `list_boards`
script with the `ZEPHYR_BASE` as a search path.

Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
2022-09-08 18:17:10 +09:00
Martí Bolívar
7ef9c4b20e edtlib: remove python 3.5 workaround
Remove a yaml monkeypatch. It is no longer needed since we support 3.6
or later on Zephyr v2.7 LTS and 3.8 or later on what will become v3.2.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2022-09-07 15:38:26 +02:00
Gerard Marull-Paretas
e6fca0ef89 scripts: ci: check_compliance: ignore missing APP_LOG_LEVEL
Application log level is not detected correctly as the option is defined
using a template, so it can't be grepped.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2022-09-07 11:18:27 +00:00
Carles Cufi
13ea65b6a9 west: Add completion support for zsh
As part of the overall work on improving west integration with Zephyr,
add support for zsh completion in the `west completion` command.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-09-06 08:43:15 -07:00
Gerard Marull-Paretas
b0d4aea769 scripts: migrate_includes: migrate zephyr.h to kernel.h
The script will now automatically migrate all <zephyr.h> or
<zephyr/zephyr.h> (for already migrated projects) to <zephyr/kernel.h>.

Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
2022-09-05 16:31:47 +02:00
Gerard Marull-Paretas
79e6b0e0f6 includes: prefer <zephyr/kernel.h> over <zephyr/zephyr.h>
As of today <zephyr/zephyr.h> is 100% equivalent to <zephyr/kernel.h>.
This patch proposes to then include <zephyr/kernel.h> instead of
<zephyr/zephyr.h> since it is more clear that you are including the
Kernel APIs and (probably) nothing else. <zephyr/zephyr.h> sounds like a
catch-all header that may be confusing. Most applications need to
include a bunch of other things to compile, e.g. driver headers or
subsystem headers like BT, logging, etc.

The idea of a catch-all header in Zephyr is probably not feasible
anyway. Reason is that Zephyr is not a library, like it could be for
example `libpython`. Zephyr provides many utilities nowadays: a kernel,
drivers, subsystems, etc and things will likely grow. A catch-all header
would be massive, difficult to keep up-to-date. It is also likely that
an application will only build a small subset. Note that subsystem-level
headers may use a catch-all approach to make things easier, though.

NOTE: This patch is **NOT** removing the header, just removing its usage
in-tree. I'd advocate for its deprecation (add a #warning on it), but I
understand many people will have concerns.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2022-09-05 16:31:47 +02:00
Emil Gydesen
174c74206e footprint: Add bt_hci_rpmsg with ISO for the nRF5340 netcore
Add footprint tracking of the samples/bluetooth/hci_rpmsg
sample for the nRF5340 with ISO broadcast and ISO receive respectively.

Signed-off-by: Emil Gydesen <emil.gydesen@nordicsemi.no>
2022-09-02 11:08:31 +00:00
Maureen Helm
5b5aa6ebba scripts: gen_defines: Add vendor name macros based on vendor prefixes
Adds vendor name and existence macros for all compatibles of a node that
match an entry in the vendor prefixes file.

Signed-off-by: Maureen Helm <maureen.helm@intel.com>
2022-09-01 14:53:55 -07:00
Maureen Helm
a1b0c39c37 scripts: edtlib: Add compat2vendor LUT constructed from vendor prefixes
Adds a new compat2vendor lookup table that maps compatibles to vendor
names, constructed from the vendor prefixes file. This approach is a
more scalable alternative to adding a vendor name property to devicetree
bindings, as was previously proposed.

Signed-off-by: Maureen Helm <maureen.helm@intel.com>
2022-09-01 14:53:55 -07:00
Torsten Rasmussen
8d2998d4f9 cmake: rework of host tools and generic toolchain handling
Follow-up: #41301

This commit is a rework and cleanup of the tools handling in Zephyr
CMake build system.

Instead of directly loading code a CMake modules for tool lookup, the
host tools now follows the CMake `find_package()` pattern for finding
programs / tools in module mode.

This makes it more clear which modules are responsible for finding tools
and which modules provides build integration / features.

The following tools can now be found using `find_package()`:
- Zephyr-sdk        : find_package(Zephyr-sdk <version>)
- Generic host tools: find_package(HostTools)

This further allows us to decouple the `verify-toolchain` CMake script
part required by `twister` into a tool lookup module and a dedicated
CMake script which utilizes the lookup module.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2022-09-01 15:59:43 +02:00
Tristan Honscheid
52db2d8279 twister: Remove import exception handler for psutil
`handlers.py` treats the `psutil` package as optional and quietly prints
a message to stdout if it is not present, but it is actually a hard
requirement for the base `Handler` class. This try/except block is bad
as it hides the requirement until a timeout occurs and the handler
attempts to use `psutil` in the terminate method, which will crash if
the module happens to be missing.

This PR removes the try/except guard so missing `psutil` will cause
twister to immediately fail. `psutil` is already specified in the
`requirements.txt` file, so this should only affect users who
unwittingly have an outdated/incorrect environment.

Also update the Github Action for twister tests to install from
requirements files.

Signed-off-by: Tristan Honscheid <honscheid@google.com>
2022-09-01 06:22:04 -04:00
Erwan Gouriou
7cd727b003 scripts: requirements-extras: Request imgtool 1.9 for write-block-size > 8
On some targets (such as STM32U5) using write-block-size >8, imgtool
should be called using option '--align 16/32' which is only available
starting version 1.9.

Update requirement for imgtool to make it available to such platforms.

Signed-off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
2022-09-01 10:28:33 +02:00
Andrzej Kaczmarek
868acfd505 scripts: Add ezFlashCLI runner
This adds support for ezFlashCLI tool to allow flashing DA1469x MCU
family from Renesas.

The tool is available at https://github.com/ezflash/ezFlashCLI or can
be installed via pip.

Signed-off-by: Andrzej Kaczmarek <andrzej.kaczmarek@codecoup.pl>
Signed-off-by: Ben Lauret <ben.lauret.wm@renesas.com>
2022-08-31 20:32:03 +02:00
Emil Gydesen
6cc8b57bc5 footprint: Add bt_hci_rpmsg for the nRF5340 netcore
Add footprint tracking of the samples/bluetooth/hci_rpmsg
sample for the nRF5340. This is a BT controller-only build
for the nRF5340 with minimum features.

Signed-off-by: Emil Gydesen <emil.gydesen@nordicsemi.no>
2022-08-31 10:41:32 +02:00
Carles Cufi
8644b333cd west: Update bash autocomplete file
Update the existing bash autocomplete with the latest commands and
command-line switches.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-08-31 10:38:01 +02:00
Gerard Marull-Paretas
fff9ecbc7f devicetree: add DT_(INST_)FOREACH_CHILD(_STATUS_OKAY)_SEP(_VARGS)
It is frequent to see in Devicetree code constructs like:

```c
 #define NAME_AND_COMMA(node_id) DT_NODE_FULL_NAME(node_id),

const char *child_names[] = {
	DT_FOREACH_CHILD(DT_NODELABEL(n), NAME_AND_COMMA)
};
```

That is, an auxiliary macro to append a separator character in
DT_FOREACH* macros. Non-DT API, e.g. FOR_EACH(), takes a separator
argument to avoid such intermediate macros.

This patch adds DT_FOREACH_CHILD_SEP (and instance/status okay/vargs
versions of it). They all take an extra argument: a separator. With this
change, the example above can be simplified to:

```c
const char *child_labels[] = {
	DT_FOREACH_CHILD(DT_NODELABEL(n), DT_NODE_FULL_NAME, (,))
};
```

Notes:
- Other DT_FOREACH* macros could/should be extended as well

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2022-08-30 16:19:57 +02:00
Gerard Marull-Paretas
d77f4e61de scripts: dts: gen_defines: add DT_DEBRACKET_INTERNAL helper
Add a utility macro used to remove brackets from around a single
argument. While __DEBRACKET exists in util_internal.h, this change makes
DT independent (otherwise we should include util_internal.h, which is
another option).

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2022-08-30 16:19:57 +02:00
Thomas Stranger
f123094482 scripts: west: runners: stm32cubeprogrammer: enable exec from PATH
Allows to use the stm32cubeprogrammer runner on Linux to be executed
from any installation directory as long as the executable is found
in PATH.

Until now on Linux the programmer could only be used if it either has
been installed in the default location or if the executable path was
passed via the cli parameter.

Signed-off-by: Thomas Stranger <thomas.stranger@outlook.com>
2022-08-29 17:31:56 +02:00
Carles Cufi
035dffcfcf scripts: modules: Mark the build as tainted if blobs present
Whenever binary blobs are present in the filesystem (i.e. fetched,
regardless of whether they are up-to-date or not) we consider the Zephyr
build as tainted. In order to ensure that it is marked as so, add a
Kconfig `select` statement whenever this is the case.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-08-29 15:57:59 +02:00
Carles Cufi
b662bc9093 scripts: modules: Move blob processing to zephyr_module.py
In preparation for the handling of taint flags in zephyr_module.py, move
blob-processing code from the west command to it.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-08-29 15:57:59 +02:00
Yuval Peress
7187d8b7fc ztest: add framework tests for failed states
Add specialized tests that execute another test indirectly and ensure
that the results report errors correctly. See the README.rst file
provided with this commit for details.

Signed-off-by: Yuval Peress <peress@google.com>
2022-08-29 10:27:00 +02:00
Jeremy Bettis
562b291704 twister: Set stdin explicitly on stty calls
stty operates on the file descriptor from stdin, but we want to operate
on stdout. If stdin and stdout are not the same tty then stty prints an
error.

Redirect stdout to stdin for stty calls.

Tested by running twister -T tests/subsys/shell </dev/null

Signed-off-by: Jeremy Bettis <jbettis@google.com>
2022-08-26 12:51:28 +02:00
Martí Bolívar
88aa873fbc twister: improve logging of cmake commands
With verbose output enabled, twister logs the cmake arguments as a
python array whenever it invokes cmake to generate a build system.

This is clunky output for manual reproduction, because it forces the
developer to manually join the arguments together for consumption by
the shell.

As an enhancement for this use case, use the standard library
shlex.join() function to produce a string that can be copy/pasted
directly into the shell on POSIX platforms.

We should use this function instead of str.join, because it correctly
handles things like shell quoting for options with spaces in them.

This function is not available on Windows, so we fall back on behavior
that is similar to the old one there.

The main difference on Windows is that the output now includes the
path to cmake as well, and not just its arguments. (This is the same
on POSIX, and is intended to help debug if multiple cmakes are
installed on the same machine.)

We are running cmake from a couple of different places, so doing this
cleanly requires adding a shared module with the platform abstraction.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2022-08-26 08:53:14 +02:00
Kumar Gala
f1660f4d51 edtlib: allow const arrays
Allow for having array types (array, uint8-array, string-array) be const.
This would allow for something like:

properties:
    reg-names:
        const: ["foo", "bar"]

To be supported.

Renamed function _check_prop_type_and_default to _check_prop_by_type
as part of this change and Moved the check for 'const' types into
_check_prop_by_type as its similar to the prop_type check and it was
easier to implement in _check_prop_by_type as we already extract
prop_type from the option in that function.

Signed-off-by: Kumar Gala <galak@kernel.org>
2022-08-25 18:11:27 -07:00
Kumar Gala
8a5d568a79 dts: Standardize string token names for arrays
Use the same postfixes for string arrays as we use for strings.  Keep
just the _STRING_*_TOKEN) variants.

Signed-off-by: Kumar Gala <galak@kernel.org>
2022-08-25 15:00:24 -07:00
Carles Cufi
25db534a07 west: blobs: Finally fix the --format option handling
Turns out that `Namespace.format` was not a good fix to the issue
described in f033040812, it was returning
the default format even when the user specified -f/--format.

Replace "getattr" with a simple logical operation.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-08-24 17:31:24 +02:00
Carles Cufi
f033040812 west: blobs: Fix --format handling
There were two issues with the original commit
e04487cad2:

- getattr() doesn't seem to work properly with args
- Comparison between bound methods with "is" seems not to work, so
  replace it with a simpler string comparison

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-08-24 07:38:42 -07:00
Carles Cufi
8f0af41fd8 scripts: compliance: Add commit range info output
Log at the info level the commit range that the script is acting upon.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-08-24 07:40:27 -04:00
Carles Cufi
0082dbfbd5 scripts: compliance: Remove unused command-line args
There were a bunch of old command-line arguments that haven't been used
in a while, remove them.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-08-24 07:40:27 -04:00
Carles Cufi
73598db4db scripts: compliance: Tiny cleanup for -c
Use a common source for default and doc.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-08-24 07:40:27 -04:00
Carles Cufi
c10a0f28b4 scripts: compliance: Clean up logging
- Add a choice in the cmd-line param
- Use the logger object everywhere
- Remove stale todo

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-08-24 07:40:27 -04:00
Carles Cufi
d160d0c20c scripts: compliance: Complete transition to junitparser v2
Follow-up to b187477fe7.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-08-24 07:40:27 -04:00
Carles Cufi
5ee76c97ab scripts: compliance: Remove unused variable
EDIT_TIP has not been used for a while now, remove it.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2022-08-24 07:40:27 -04:00
Peter Marheine
d400b8135c arch/riscv: support CONFIG_CODE_DATA_RELOCATION
This implements support for relocating code to chosen memory regions via
the `zephyr_code_relocate` CMake function for RISC-V SoCs. ARM-specific
assumptions that were made by gen_relocate_app.py need to be corrected,
in particular not assuming any particular name for the default RAM
section (which is 'SRAM' for most ARM pltaforms) and not assuming 32-bit
pointers (so the test works on RV64).

Signed-off-by: Peter Marheine <pmarheine@chromium.org>
2022-08-24 10:08:06 +02:00
Peter Marheine
9bd46852a3 build: don't assume FLASH is the ROM region
Most ARM platforms name their ROM region `FLASH`, but this assumption is
not portable. Have gen_relocate_app refer to ROMABLE_REGION instead of
hard-coding the platform-specific memory region name.

Signed-off-by: Peter Marheine <pmarheine@chromium.org>
2022-08-24 10:08:06 +02:00
Peter Marheine
3a8c20d849 build: don't assume .symtab section type
Only the elftools SymbolTableSection section type provides an
iter_symbols() method, and compilers are free to emit sections that
include the substring '.symtab' in their name which causes errors if
gen_relocate_app examines only the section name. Instead check whether a
section is a symbol table to skip attempting to inspect sections that
are not actually symbol tables.

Signed-off-by: Peter Marheine <pmarheine@chromium.org>
2022-08-24 10:08:06 +02:00
Kumar Gala
4f84bc8f40 scripts/ci/check_compliance.py: Allow to run w/ZEPHYR_BASE
Tweak check_compliance.py to allow user to run it w/o having
ZEPHYR_BASE set.  This is similar to what we do for twister.

Signed-off-by: Kumar Gala <galak@kernel.org>
2022-08-23 13:26:27 +02:00
Radosław Koppel
7614110a20 dts: Add _STRING_TOKEN and _STRING_UPPER_TOKEN to string-array
This commit adds string token versions of the values also
in items inside string-array.

Signed-off-by: Radosław Koppel <r.koppel@k-el.com>
Co-authored-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
Co-authored-by: Kumar Gala <galak@kernel.org>
2022-08-22 13:58:30 -05:00