Commit graph

5615 commits

Author SHA1 Message Date
Martí Bolívar
ff8c63c03b edtlib: move PropertySpec
This is just moving the class definition higher in the file
to make it easier to type annotate the module.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2023-04-17 08:58:14 -07:00
Martí Bolívar
bef3970573 edtlib: move Binding
This is just moving the class definition higher in the file. I am
reordering the classes to make it possible to type annotate the module
in a more readable way.

Git might make the diff look bigger than it really is.
To verify this is just moving code, use 'git diff --minimal'.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2023-04-17 08:58:14 -07:00
Martí Bolívar
cf9cfc31bd edtlib: implement copy.deepcopy() for EDT
Just like we did for dtlib in 15e3e317f7
("dtlib: implement copy.deepcopy() for DT"), except this time it's for
EDT. This also can do no harm and will be useful for implementing
system devicetree support.

No functional changes expected under the assumption that no users are
relying on us having stashed the exact bindings_dirs list passed to
the constructor. This patch switches to making a defensive copy, which
is safer and makes implementing this a little cleaner.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2023-04-17 08:58:14 -07:00
Martí Bolívar
f4b487aea2 edtlib: refactor some internals
Move all the initial settings of instance attributes to the
constructor, so we can keep track of them all more easily.

No functional changes expected.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2023-04-17 08:58:14 -07:00
Martí Bolívar
867dd1c84c devicetree: use c89 comments in test file
This is still the preferred style in a zephyr DTS.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2023-04-17 08:58:14 -07:00
Martí Bolívar
dcf1fc0592 dtlib: fix docstring
The reference is to an incorrect method.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2023-04-17 08:58:14 -07:00
Piotr Golyzniak
fec7881117 scripts: tests: tests for choosing binaries
Tests dedicated for choosing binaries basing on platform.binaries and
runners.yaml file. Additional tests for paths sanitizing.

Signed-off-by: Piotr Golyzniak <piotr.golyzniak@nordicsemi.no>
2023-04-14 08:01:31 -04:00
Tristan Honscheid
bfd6177631 twister: Include unit test testbinary in ELF search space
`TestInstance.get_elf_file()` does not include the binary name used for
unit testing (`testbinary`) in its search, causing unit tests to not
complete successfully. This does not cause tests to fail and thus goes
unnoticed.

Update the glob expressions used in `get_elf_file()` to also look for
files called `testbinary`

Signed-off-by: Tristan Honscheid <honscheid@google.com>
2023-04-13 19:56:26 -04:00
Andrei Emeltchenko
b9f880a141 twister: trivial: Correct license year
Use correct year in license.

Signed-off-by: Andrei Emeltchenko <andrei.emeltchenko@intel.com>
2023-04-13 11:05:47 -04:00
Gerard Marull-Paretas
4355b7f179 scripts: utils: add sys_init migration script
Utility script to migrate SYS_INIT() calls from the old signature:

```c
int (*init_fn)(const struct device *dev)
```
to

```c
int (*init_fn)(void)
```

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2023-04-12 14:28:07 +00:00
Magdalena Kasenberg
661f07ba0f bluetooth: tester: Add CONFIG_BTTESTER_LOG_LEVEL
to set a different log level than the default.

Signed-off-by: Magdalena Kasenberg <magdalena.kasenberg@codecoup.pl>
2023-04-12 13:05:32 +02:00
Kumar Gala
ae23da81f2 twister: make get_elf_file sysbuild aware
If we are doing a sysbuild get_elf_file needs to return the default
domain's build_dir.  The standard build_dir will not any ELF builds
in it so will error out.

Signed-off-by: Kumar Gala <kumar.gala@intel.com>
2023-04-11 20:33:29 -04:00
Anas Nashif
d8126be557 twister: ignore remapped ELF file when getting ELF name
This is usually used only when doing size calculation, but now it is
being used for extracting symbols from the ELF for testing purposes and
some issues arise with multiple ELF files found.

Simplify the code a bit to make it easy to maintain a lists of things to
exclude.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-04-11 14:01:39 +02:00
Dmitrii Golovanov
7efec71a85 twister: Alleviate DeviceHandler serial port TOCTOU
The DeviceHandler class method monitor_serial() has time-of-check
to time-of-use (TOCTOU) race condition possible on its serial port
which might be closed and its file descriptor cleared in between
of isOpen() and is_waiting status checks.
The issue is alleviated with controlled exception handling.

Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
2023-04-11 09:34:57 +02:00
Dmitrii Golovanov
0d2ba9c793 twister: Fix DeviceHandler hangs without suitable DUT
Fixes an issue when DeviceHandler awaited for a suitable DUT infinitely.
It was possible if the device map has no devices with required platform,
or no devices with fixture needed for the test case, or devices without
serial connection. A trivial case is a typo in platform name.

Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
2023-04-11 09:34:41 +02:00
Marc Herbert
fad2da39aa intel_adsp: move west sign from west flash to earlier west build
Invoking `west sign` in `west build` accelerates twister because `west
build` is run in parallel, see rationale in superseded and very
different (CMake-based) PR #52942.

To maximize backwards compatibility:
- `west sign` is optional in `west build`
- `west flash` will sign (again) if any rimage --option is passed

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2023-04-10 22:04:47 -04:00
Marc Herbert
2c80c4daa4 west: sign: add new rimage option --if-tool-available
Moving `west sign` from `west flash` to `west build` for rimage has
multiple advantages (including a bit more consistency with
imgtool). However it makes `west build` fail when rimage is missing.

To avoid forcing every CI and developer who never used it before to
install rimage, make signing optional when passing new `west sign`
option --if-tool-available. A clear warning is printed.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2023-04-10 22:04:47 -04:00
Marc Herbert
5c4319d54f west: sign: clean any stale rimage output
We never want to leave stale outputs behind after failing.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2023-04-10 22:04:47 -04:00
Marc Herbert
5bdb1467cf west: sign: define filename constants earlier in rimage signer
Zero functional change, pure preparation for the next commit.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2023-04-10 22:04:47 -04:00
Marc Herbert
2fbcdb5ff2 west: sign: add new west config [rimage].extra-args and a default key
Make RIMAGE_SIGN_KEY a CMake CACHE variable so `west sign` can find it
and use it as a default value.

Fixes https://github.com/thesofproject/sof/issues/6917

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2023-04-10 22:04:47 -04:00
Marc Herbert
66ac6252ed west: sign: add west config [rimage].path = /path/to/rimage
Add a 3rd option besides --tool-path and $PATH

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2023-04-10 22:04:47 -04:00
Marc Herbert
dedb002214 west: sign: rimage: simplify rimage --tool-data code
Reduce duplication, no functional change except for a shorter log
statement with slightly less information.

This is required by the next commits.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2023-04-10 22:04:47 -04:00
Marc Herbert
5831164e7a west: sign: add west config [sign].tool = rimage/imgtool
The --tool parameter is not required anymore thanks a [sign] entry in
west config like this one:

```
[sign]
tool = imgtool
```

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2023-04-10 22:04:47 -04:00
Marc Herbert
030b740bd1 west: sign: add ability to read parameters from west config
No functional change yet, will be used in the next commits.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2023-04-10 22:04:47 -04:00
Marc Herbert
efb855134e west: sign: rimage: add default value '-c sof/rimage/config'
Don't force 99% of the users to pass the same value.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2023-04-10 22:04:47 -04:00
Kumar Gala
c09e8fae23 twister: Use pyelf to extract symbol information
For ztest twister would parse the symbol information that was
generated as part of the build (zephyr.symbols).  However the format
of the zephyr.symbols files is highly dependant on the toolchain.

Move to using pyelf to parse the symbol information directly from
zephyr.elf instead so that this works regardless of toolchain.

Signed-off-by: Kumar Gala <kumar.gala@intel.com>
2023-04-08 18:37:52 +02:00
Kumar Gala
53bae96793 twister: allow get_elf_file() to be called on native platforms
On native platforms we typically have CONFIG_BUILD_OUTPUT_EXE set so
we end up with both zephyr.elf and zephyr.exe.  This would cause a
call to get_elf_file() to fail.  There isn't any reason to error out
since zephyr.elf and zephyr.exe should be identical.  So allow it
if the platform type is native.

Signed-off-by: Kumar Gala <kumar.gala@intel.com>
2023-04-08 18:37:52 +02:00
Dmitrii Golovanov
f31aab58d5 twister: Add configurable flashing timeout
Use explicit flash command execution timeout at DeviceHandler
instead of the hardcoded value of 60 sec.

When a HW platform executes test cases right after the flash command,
the test case remaining timeout is affected by how much time the flash
operation consumes. Some simulation platforms need to spend significant
amount of time on each 'flash' cycle, usually adding the same delay on
each test case.

This improvement adds two new command line options and device map fields:

  --device-flash-timeout - for the flash operation timeout
  ('flash-timeout' device map field).

  --device-flash-with-test - to indicate that the platform flash
  command also runs a test case, so the overall timeout should be
  calculated as a sum of the flash timeout and the current test case
  timeout to receive all console output from the platform
  ('flash-with-test' device map field).

The device map field values override command line values for the
particular platform where configured.

Default behavior is backward compatible: flash operation fixed timeout
is 60 sec. not including the test case timeout.

Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
2023-04-08 18:37:33 +02:00
Al Semjonovs
fb8d8c5773 scripts: Create a script to generate an NTC thermistor R/T table
Script to generate an R/T table in device tree format given
R25 and Beta values.

Signed-off-by: Al Semjonovs <asemjonovs@google.com>
2023-04-07 16:23:17 -05:00
Keith Short
3406d40ee0 scripts: runners: bossac: cleanup edt load
An explicit import of edtlib is not required when using
pickle.load() and creates a duplicate module object.

Local testing shows that pickle.load() throws the ModuleNotFoundError if
the sys.path cannot find the edtlib or associated modules.

Signed-off-by: Keith Short <keithshort@google.com>
2023-04-07 13:37:51 +02:00
Keith Short
df0942f343 scripts: kconfigfunctions: cleanup edt loading
An explicit import of edtlib is not required when using
pickle.load() and creates a duplicate module object.

Retrieve the correct module object directly from the pickle file
using inspect.getmodule().

Signed-off-by: Keith Short <keithshort@google.com>
2023-04-07 13:37:51 +02:00
Keith Short
52e6b4e1fe scripts: generate_vif: cleanup edt loading
An explicit import of edtlib is not required when using
pickle.load() and creates a duplicate module object.

Also eliminate the check for the EDTError, as this exception cannot be
raised by pickle.load().

Signed-off-by: Keith Short <keithshort@google.com>
2023-04-07 13:37:51 +02:00
Keith Short
23433d89e5 scripts: gen_handles: cleanup edt loading
An explicit import of edtlib is not required when using
pickle.load() and creates a duplicate module object.

Signed-off-by: Keith Short <keithshort@google.com>
2023-04-07 13:37:51 +02:00
Peter Marheine
5773862844 gen_relocate_app: avoid hard-coding section name fragments
This refactors the script to introduce a SectionKind enum that
represents each of the kinds of sections handled by this system (text,
data, rodata or bss), using that to improve the code structure by
reducing the use of strings in favor of indicating the use of values by
their types.

Signed-off-by: Peter Marheine <pmarheine@chromium.org>
2023-04-07 13:12:13 +02:00
Peter Marheine
12c6dda9e2 gen_relocate_app: restrict relocations to listed files
When using code_data_relocation the script identifies sections that
should be relocated, but cannot guarantee that section names are unique
across all linked files. If a section name is not unique then matching
all sections with that name across all input files will relocate more
data than intended.

As a simple example, if both file1 and file2 contain .rodata.strings
sections, then if gen_relocate_app inspects only file1 it will generate
a linker script fragment `*(.rodata.strings)` that matches both object
files.

This commit changes gen_relocate_app to make the linker match on object
files as well (`*file1.o(.rodata.strings)`) to ensure unwanted sections
don't get relocated.

Signed-off-by: Peter Marheine <pmarheine@chromium.org>
2023-04-07 13:12:13 +02:00
Kumar Gala
17f0a64bd4 twister: Enable armclang on all arm targets
Rather than having to add 'armclang' to every <BOARD>.yaml, just list
it as a generally supported toolchain if the architecture is 'arm'.

Signed-off-by: Kumar Gala <kumar.gala@intel.com>
2023-04-07 11:29:43 +02:00
Grzegorz Chwierut
482c94819f tests: boot: test_mcuboot: Add mcuboot log level
Added logging of MCUboot messages to fix twister test scenario.
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
2023-04-06 18:51:23 +02:00
Martí Bolívar
5422a317ad runners: bossac: add --delay option
Add an escape hatch so users can insert a customizable delay
between asking the board to reboot into bootloader mode and
trying to program the new image. This lets people account for
OS-specific enumeration time done after the board resets.

Fixes: #56540
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2023-04-05 19:42:49 +02:00
Anas Nashif
dbb1b87dbc twister: instance: fix pylint warning
C0325:Unnecessary parens after '=' keyword (superfluous-parens)
File:scripts/pylib/twister/twisterlib/testinstance.py
Line:134
Column:0

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-04-05 10:27:28 +02:00
Anas Nashif
81a1e9d686 twister: marked tests causing timeout as fail
When a test start, mark it as such, if we do not get any results due to
a timeout, it will be finally marked as failed. Previously such tests
causing a freeze were marked as blocked (or not run), which is not
exactly right.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-04-05 10:27:28 +02:00
Anas Nashif
65b483bd7b twister: harness: rename match variable
Avoid confusion with python match builtin.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2023-04-05 10:27:28 +02:00
Maciej Perkowski
9f207ef650 twister: Treat overflows as errors when integration mode is on
Skips due to overflows on integration_platforms during integretion
mode should be treated as errors.

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2023-04-05 08:30:35 +02:00
Maciej Perkowski
b934142177 twister: Refactor "skips to errors if integration"
Export the logic to a separate method for reusability

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2023-04-05 08:30:35 +02:00
Maciej Perkowski
3b5758c4f3 twister: Remove redundant line.
`self.instances[filtered_instance.name]` and `filtered_instance`
are already referancing the same object.

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2023-04-05 08:30:35 +02:00
Piotr Golyzniak
2fcfd48db2 scripts: twister: fix zephyr base sanitizing
If canonical zephyr base is already ended by trailing slash, then it
should not be added redundantly.

Signed-off-by: Piotr Golyzniak <piotr.golyzniak@nordicsemi.no>
2023-04-04 08:43:15 +02:00
Piotr Golyzniak
d98afb1cf1 scripts: twister: fix files sanitizing
In some cases built binary files have different names than "zephyr.hex"
files, and for those cases, information about names of file to keep
should be taken from runners.yaml file.

Moreover if in runners.yaml file exist absolute files paths to binary
files, then they should be changed to relative ones.

Signed-off-by: Piotr Golyzniak <piotr.golyzniak@nordicsemi.no>
2023-03-30 13:48:13 +02:00
Daniel Leung
4d914f4f89 toolchain: move CONFIG_LLVM_USE_LD into cmake/toolchain/llvm
This moves CONFIG_LLVM_USE_LD into cmake/toolchain/llvm as this
is a toolchain kconfig. Also make it a choice to allow the use
of LLVM's lld as linker.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2023-03-29 19:48:47 -04:00
Jamie McCrae
ef146c3dbd scripts: compliance: Add MCUboot Kconfig to allow list
Adds a Kconfig for MCUboot which is used in the documentation to
the CI Kconfig check allow list.

Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
2023-03-29 11:30:36 +02:00
Jamie McCrae
c5fbcc468e snippets: Fix cmake path output on Windows
Uses linux style path output instead of windows style to prevent
issues with cmake assuming the slashes are for escape code
sequences.

Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
2023-03-27 14:44:39 +00:00
Carles Cufi
e0c90b9393 scripts: requirements: Move requests and pyserial to base
Both the requests and pyserial Python packages are used by west commands
(west fetch and west build -b esp* respectively) so move them to the
requirements-base.txt file.

Fixes #56215.
Fixes #56224.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2023-03-27 07:12:27 -07:00