Create helper class Shell which can be used in tests which require
interaction with shell application.
Signed-off-by: Piotr Golyzniak <piotr.golyzniak@nordicsemi.no>
Move helper.py file content to utils.py file in device directory,
where those helper functions are used.
Signed-off-by: Piotr Golyzniak <piotr.golyzniak@nordicsemi.no>
Divide dut fixture into three smaller fixtures to make it easier
to user to change dut fixture scope or to write her/his own dut
fixture. Add initialize_log_files method to make it possible to
initialize log files per each test.
Signed-off-by: Piotr Golyzniak <piotr.golyzniak@nordicsemi.no>
Pass pytest test name to handler.log header to make it easier to
analyze handler.log content.
Signed-off-by: Piotr Golyzniak <piotr.golyzniak@nordicsemi.no>
Add readlines_until method to DeviceAdapter class to simplify using
adapters in tests by give user possibility to read output from device
until following conditions:
- provided regex was found
- provided number of lines was already read
- timeout was occurred
Signed-off-by: Piotr Golyzniak <piotr.golyzniak@nordicsemi.no>
Introduce changes into FifoHandler class to make it more safe
(especially to avoid blocking all program when FIFO queue is not opened
by other application).
Signed-off-by: Piotr Golyzniak <piotr.golyzniak@nordicsemi.no>
Select and place common code of three adapters (HardwareAdapter,
BinaryAdapterBase and QemuAdapter) into basic DeviceAdapter class.
Introduce new way of reading device output by run separate thread which
try to read device output and place it into internal python queue.
Thanks to this, now it is possible to create readline method for all
adapters, which can be unblock when timeout occur.
Collect all common steps which have to be done before setup device in
launch method. The same was done for teardown operations which were
placed into close method.
Additionally some protection mechanisms were introduced to prevent for
undesirable side-effects when user could try to launch to already
launched device or try to send some data to disconnected device.
iter_stdout method was replaced by two new methods: readline and
readlines. To make it possible to remove all read output from internal
buffer (queue), clear_buffer method was introduced.
Also unit tests were rewritten to work properly with current version
of adapters.
Signed-off-by: Piotr Golyzniak <piotr.golyzniak@nordicsemi.no>
Change name of SimulatorAdapterBase class to more meaningful like
BinaryAdapterBase. This changes is also made to be more align with
Twister terminology.
Signed-off-by: Piotr Golyzniak <piotr.golyzniak@nordicsemi.no>
Add possibility of passing timeouts from Twister to pytest and unify
various timeouts used in adapters to point to one main timeout.
Signed-off-by: Piotr Golyzniak <piotr.golyzniak@nordicsemi.no>
PytestAssertRewriteWarning is raised when pytest-twister-harness
plugin is called from twister. Call register method to get rid
of that warning
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
When parsing test names from symbol names, use a non-greedy match to
avoid parsing test names with __ in them.
The symbol looks like z_ztest_unit_test__SUITE__TEST. It is more likely
that a test name will contain __ than a suite name, so now something
z_ztest_unit_test__a_b__c_d__e_f will be parsed as suite:a_b
test:c_d__e_f, whereas it used to be parsed as suite:a_b__c_d test:e_f
Signed-off-by: Jeremy Bettis <jbettis@google.com>
If a test doesn't start with test_, twister currently emits an error:
suite_name.test_name: Unknown status 'started'
Following the suggestion by Maksim Masalski on PR#21191, change the
regex to exclude spaces in the test name, and remove the check that the
test_ prefix exists.
See also #21162
Signed-off-by: Jeremy Bettis <jbettis@google.com>
Now, if the rimage target (meaning rimage configuration and platform name)
is not defined in board.cmake the sign script returns fatal error.
Change this to a warning since there are configurations that are
not using 'west sign' or is used just to glue the headers of the
final image.
Also, update the documentation accordingly.
Signed-off-by: Iuliana Prodan <iuliana.prodan@nxp.com>
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
In addition to the white-box testing and unit tests,
we would like to have some black-box testing for the
console commands of twister.
This serves as a Proof of Concept for future expansion.
Special dummy tests are made for blackbox testing.
Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
Following PR #61471, ehl_crb board is now intel_ehl_crb.
Update the footprint test plan accordingly.
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
Fix an "if" that should have been an "elif". This is currently causing
the script to fallback into the "do all unassigned PRs of the day" path
when a PR is specified.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
Add a -I option to process an issue and, if it's not assigned already,
add assignees based on the labels, matching the ones set in the
MAINTAINERS file areas.
Suggested-by: Henrik Brix Andersen <henrik@brixandersen.dk>
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
Add a -m option to set_assignees to have the script find all unassigned
module PRs for areas that have an assignee and set the assignee on those
PRs.
This should help avoiding forgotten PRs on modules. It's implemented as
a search and meant to run periodically, so we don't have to add a
workflow for this on every individual module.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
The file kconfig.py was originally introduced by myself in:
https://github.com/zephyrproject-rtos/zephyr/pull/5569
The file was a modified version of a Kconfiglib one, so it makes sense
to keep the original license (ISC).
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Force default stdout encoding to utf-8 as otherwise ram_report/rom_report
may fail to render (ex. in CLion IDE built-in terminal).
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
Add platform filtering when generating tests for manifest modules.
In the clang workflow, we do select to run only on one platform, so this
needs to be applied as well or we end up building unwanted tests using
the wrong toolchain.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
strtok_r is part of GNU99, but default Zephyr is built with C99, so,
the GNU headers are not pulled in, even if we add "--std=gnu99" it
won't override the C99 standard.
As a workaround for build we redefine the prototype, but this causes a
coding guideline warning, so, for now disable this warning
for strtok_r.
Signed-off-by: Sachin D Kulkarni <sachin.kulkarni@nordicsemi.no>
The generated stubs use retval for function return value.
However, this is too generic and is going to collide with
other code. So rename those to be more specific. Note
that the double underscore is intentional to minimize
chance of being shadow variables.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
While the code already includes some provisions to allow running outside
of a git checkout, attempting that right now causes a failure to generate
the XML report, as the JSON report creates a null `zephyr_version` field.
The reason for that is that the original code doesn't set `self.version`
when the subprocess returns a non-zero status (and also doesn't log.)
Instead of having to set `self.version` for all failure cases, initialize
it to the failure state, and log if it hasn't changed from that.
Signed-off-by: Diego Elio Pettenò <flameeyes@meta.com>
When using the hardware map and testing on device, print out the name of
the dut (from the hardware map) on the script and report it in the json
output as well.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
PCI devices are have some differences to regular nodes:
* node name specifies device/function e.g. "pcie@1,0"
* register address has a different meaning
* zero-sized register is allowed
This improves alignment with Linux DT for PCI devices
Signed-off-by: Grant Ramsay <gramsay@enphaseenergy.com>
The anytree package has introduced a breaking change in version 2.9.0 by
adding a 'size' property to the NodeMixin class. Since the TreeNode class
in size_report derives from NodeMixin and defines an attribute with the
same name, an AttributeError is raised when generating reports.
With this change, the attributes of the TreeNode class are prefixed with
an underscore to resolve the name collision and to prevent future name
collisions.
Fixes#60213.
Signed-off-by: Christian Marx <c.marx@vega.com>
Align with the latest upstream native simulator
4c595794588f9d7f67fcf0fe05c3db02892a00f9
including:
* 4c59579 Makefile: Add option to build native part
* 910f934 Makefile: NSI_EXTRA_INCLUDES option and lots of commentary
* d9bf489 cmd line parsin: Minor header refactoring
* 02f3555 cmd line cleanup: Run as NSI_TASK instead of calling expl.
* 2c88173 Split exit call in two
* 2b989b4 CPU IF change: nsif_cpu0_cleanup() to return int
* e696228 HW scheduler: Add API to get next event time
* ae0e9e8 native irq ctrl: Miscellaneous fixes and improvements
* 3fd84cd NSI_TASK: Add compile check of valid priority
* 7e09fb8 HW events: Change internal storage
And two minor updates to the native_sim board,
to align with this updated version:
* nsif_cpu0_cleanup(void) now must return an int
* We need to explicitly tell the native simulator build we want
the native components built
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
In Linux, checkpatch.pl relies on the vendor-prefixes.yaml file
to validate manufacturers in compatible strings.
In addition to the vendors defined in vendor-prefixes.txt,
the YAML file includes expressions for "prefixes which are not vendors":
these expressions do NOT define special manufacturers that may appear
in compatible strings, and are never involved as such in DTS files.
We can rather see them as bulk-definitions of JSON/YAML properties
suitable for the dt-schema tools.
OTHO, in Zephyr, checkpatch.pl relies on the vendor-prefixes.txt file,
which does not include these additional prefixes, but edtlib.EDT adds
them as hard-coded special values.
This is confusing, if not incorrect:
- the fact that edtlib.EDT (and thus its client code in the
zephyr/scripts directory) actually allows these vendors
in compatible strings is buried in the source code
- checkpatch.pl (with vendor-prefixes.txt) in Zephyr behaves neither like
checkpatch.pl (with vendor-prefixes.yaml) in Linux, nor like edtlib.EDT
(with _VENDOR_PREFIX_ALLOWED)
- Zephyr should not treat these "prefixes which are not vendors" as
valid manufacturers in compatible strings to begin with
Signed-off-by: Christophe Dufaza <chris@openmarl.org>
Currently, this script will blow up when given a symlink for a path (e.g.,
to a kconfig file).
This is annoying, as when wrapping the build system in a hermetic build
system like Bazel, Bazel likes to limit the input file set by creating
symlinks.
Resolve the path, if it is a file once resolved, then it's OK.
Signed-off-by: Jack Rosenthal <jrosenth@chromium.org>
Some architectures such as RISC-v support more than 255 interrupts
per aggrigator. This diff adds the ability to forgo the aggrigator
pattern and use a configurable number of bits for multilevel
interruts.
Signed-off-by: Joshua Lilly <jgl@meta.com>
DTC_OVERLAY_FILE was replaced with EXTRA_DTC_OVERLAY_FILE. Adjust example
usage in schema file.
Fixes: 1561a0705f ("snippets: support for EXTRA_DTC_OVERLAY_FILE and
EXTRA_CONF_FILE")
Signed-off-by: Marcin Niestroj <m.niestroj@emb.dev>
Commit 9b30667c77
("build: drop LEGACY_INCLUDE_PATH support")
removed scripts/utils/migrate_includes.py.
This was premature. This script is still useful to users of older
versions of zephyr that are attempting to migrate to current ones.
For example, users of v2.7 LTS who are migrating to v3.4 should
still be able to run the script to migrate their code bases.
Restore it. Maybe after next LTS we can remove it. It costs
basically nothing to keep it in the tree.
Signed-off-by: Martí Bolívar <mbolivar@amperecomputing.com>
If size is given then twister prnits the size information for the ELF
but returns one which if used in automation fails the command.
Return zero as the command is successful. Also, fix the unit test case.
Signed-off-by: Chaitanya Tata <Chaitanya.Tata@nordicsemi.no>
Let's not provide default toolchains for 'arc' intentionally: some
targets can't be built with GNU toolchain ("zephyr", "cross-compile",
"xtools" options) and for some targets we haven't provided MWDT
compiler / linker options in corresponding SoC file in Zephyr, so
these targets can't be built with ARC MWDT toolchain
("arcmwdt" option) by Zephyr build system.
Instead for 'arc' we rely on 'toolchain' option in board yaml
configuration.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
This patch adds Kconfig options to select either GNU libgcc or LLVM
compiler-rt. The 'rtlib' flag is provided in a config file, so this
patch introduces 'clang_libgcc.cfg' and 'clang_compiler_rt.cfg' which
enable appropriate library. The file is selected by concatenating
the 'clang_' prefix with library name.
Signed-off-by: Patryk Duda <pdk@semihalf.com>
This is a one-line fix for edtlib, which lets gen_defines.py indicate
whether the `ranges` property exists within a given node.
Although address translation through ranges is typically automatic,
users can choose to manually inspect ranges using DT_FOREACH_RANGE(),
DT_NUM_RANGES(), and other DT_RANGES_* macros. These can be used to
implement manual translation at runtime, which is currently done for
PCIe controllers.
The only thing missing is being able to check if a node contains an
empty `ranges;`, which signifies a 1:1 translation to the parent bus.
Checking DT_NUM_RANGES() is insufficient, because it returns zero
whether or not `ranges;` is present.
It should be possible to use DT_NODE_HAS_PROP(), but it was not working,
because edtlib ignores properties which are undeclared in bindings and
don't have a default type. Add a missing PropertySpec for `ranges` with
"compound" type; it can't be "array" because it can be empty-valued.
Signed-off-by: Grzegorz Swiderski <grzegorz.swiderski@nordicsemi.no>
Enable the possibility for boards to implement a custom `run` target in
its board.cmake to run any arbitrary commands. This is helpful for devs
who would like to add support for proprietary simulator to their boards
that can't be upstreamed.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Add an additional option to the `append` schema for appending to the
`DTS_EXTRA_CPPFLAGS` cmake cache variable, enabling finer control over
the content of devicetree files.
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
For applications relocating big parts of the code with many sections,
builds were failing on Windows due to hitting the max command-line
length on that platform.
Fix this by using a file to store the dictionary passed to the python
script.
Fixes https://github.com/zephyrproject-rtos/zephyr/issues/60994.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
We have been dealing with missing and multiple binaries the same way and
both would result in a build error, which is not accurate. multiple
binaries in the build directory are fine, we just need to pick the right
one for parsing.
If we get no binaries, raise an exception and report failure, however,
if we have multiple binaries, filter intermediate artifacts out and
parse what remains.
qemu binaries generated after a run are also being filtered here. Those
are not build artificats and appear only after running in qemu. However
they have been causing issues on retries.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
When no/too many elf files are detected after a build a BuildError
exception is raised. However, it was not being counted as an issue
with a test. With the patch satuses of tests' with such exception
are reported as errors. Whithout it, twister finished without
reported errors and was getting green CI checks.
Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
The check currently only runs if the maintainers file itself is changed,
but that means that the check is going to miss every PR that moves
directory or delete files that can potentially trigger an error.
This check is cheap to run, just run it unconditionally.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
Update the script to parse the new section naming. The ordering type
is converted from an integer to a tuple, which still compares correctly
due to the elementwise behaviour of tuple comparison.
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
Generate a zero padded variant of `_ORD` that is suitable for use in
linker scripts with the `SORT` property, so that `6` is correctly placed
before `24`, and so on.
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
Generate a zero padded variant of `_ORD` that is suitable for use in
linker scripts with the `SORT` property, so that `6` is correctly placed
before `24`, and so on.
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
This is meant as a substitute for sys_clock_timeout_end_calc()
Current sys_clock_timeout_end_calc() usage opens up many bug
possibilities due to the actual timeout evaluation's open-coded nature.
Issue ##50611 is one example.
- Some users store the returned value in a signed variable, others in
an unsigned one, making the comparison with UINT64_MAX (corresponding
to K_FOREVER) wrong in the signed case.
- Some users compute the difference and store that in a signed variable
to compare against 0 which still doesn't work with K_FOREVER. And when
this difference is used as a timeout argument then the K_FOREVER
nature of the timeout is lost.
- Some users complexify their code by special-casing K_NO_WAIT and
K_FOREVER inline which is bad for both code readability and binary
size.
Let's introduce a better abstraction to deal with absolute timepoints
with an opaque type to be used with a well-defined API.
The word "timeout" was avoided in the naming on purpose as the timeout
namespace is quite crowded already and it is preferable to make a
distinction between relative time periods (timeouts) and absolute time
values (timepoints).
A few stacks are also adjusted as they were too tight on X86.
Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
The new stack zephyr,cdc-acm-uart driver has two separate init path, one
of which kicks in before the USB stack to start buffering log messages.
This generates a false positive in the build time priority checking,
adding a check to ignore that compatible entirely.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
Parse common section and append to cmake args if available when using
--test-item option of west build.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Linkserver is a utility for launching and managing GDB servers for NXP
debug probes, which also provides a command-line target flash programming
capabilities. Linkserver can be used with NXP MCUXpresso for Visual Studio
Code.
For more information about LinkServer, please visit the LinkServer web
page (link [1] below).
This commit adds a runner to west, supporting debug and flash commands.
Documentation is also added.
[1] - LinkServer web page:
https://www.nxp.com/design/software/development-software/mcuxpresso-software-and-tools-/linkserver-for-microcontrollers:LINKERSERVER
Signed-off-by: Yves Vandervennet <yves.vandervennet@nxp.com>
Add the BT Telephony and Media Audio Profile broadcast samples.
These reflect devices that control telephone and media audio
of LE Audio.
Signed-off-by: Emil Gydesen <emil.gydesen@nordicsemi.no>
Adds a guide on how to make out-of-tree MCUmgr function handlers
and groups, with an example showing a test implementation.
Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
When using twister to generate coverage with the coverage tool lcov,
allow using --coverage-formats to pick if you want lcov or html,
defaulting to both. Picking html will also use lcov, since that is
required for geninfo. This will allow callers to avoid the potentially
slow and disk intensive html reports if they only wanted the lcov info
file.
Signed-off-by: Jeremy Bettis <jbettis@google.com>
Fixes Issue #60522 by guarding member access against None parameters.
Shortens the init by using the if...else assignment.
Fixes two typos.
Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
Re-ordering the API names moved a trailing '|' causing the pattern to match
*every* #define in the input.
Signed-off-by: Keith Packard <keithp@keithp.com>
An arg --test-item makes west loading twister's test configurations
form sample/testcase.yaml. It has to mirror twister's behavior.
It was not the case with "extra_args" section. Quotation marks were
not removed in west as they were in twister. The quotation marks
have to be removed from the extra_args section but left in
extra_configs. The commit adds differentiation for those.
fixes: #60297
Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
The `NEW_TYPEDEFS` rule consistently generates false
positives when implementing POSIX standard APIs.
It makes sense to disable this check for the POSIX haders
rather than requiring merge superpowers constantly. That
way, we can merge as per usual after sufficient approvals
rather than waiting for someone with merge superpowers to
override / manually merge.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
All code in the Zephyr core must use only the Zephyr C library API
according to rules A.4 and A.5. Such code is not permitted to request API
extensions from the C library via any of the API request mechanisms.
This addition to checkpatch.pl verifies that patches don't #define
any of these:
__STRICT_ANSI__
_POSIX_SOURCE
_POSIX_C_SOURCE
_XOPEN_SOURCE
_ISOC99_SOURCE
_ISOC11_SOURCE
_ATFILE_SOURCE
_GNU_SOURCE
_BSD_SOURCE
_SVID_SOURCE
_DEFAULT_SOURCE
Reference: #49922
Signed-off-by: Keith Packard <keithp@keithp.com>
As we aim to enhance our test coverage, environment module
is a target-rich environment. Some errors in the original
module were discovered and tests regarding these cases were
commented out until they are fixed.
Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
Fix `--device-flash-with-test` command line options check logic
to be effective with `--device-testing`.
Remove `--device-flash-timeout` check for `--device-testing`
presence as marginal.
Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
Add the BT Telephony and Media Audio Profile samples.
These reflect devices that control telephone and media audio
of LE Audio.
The central is typically a more resourceful device, and the
peripheral a resource-constrained device.
Signed-off-by: Emil Gydesen <emil.gydesen@nordicsemi.no>
Here we achieve 100% coverage for the quarantine module,
thanks to unit tests for QuarantineElement and QuarantineData dataclasses.
Implemented PR suggestions of gchwier.
Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
Zephyr scripts do not require documentation dependencies, so let's
move them from scripts/ to doc/.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
hardware was referenced as return value, without being declared. When the
program goes into the 'except' path, hardware is unbound, but returned,
which throws an error:
`UnboundLocalError`: local variable 'hardware' referenced before assignment
Signed-off-by: Manoel Brunnen <manoel.brunnen@ithinx.io>
This fixes the following build error:
```
zephyr/zephyr.elf', needed by 'zephyr/zephyr.ri', missing and no known
rule to make it
```
This appears when CONFIG_KERNEL_BIN_NAME is used.
Therefore, do not use zephyr.elf since some samples might be called
based on CONFIG_KERNEL_BIN_NAME.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Signed-off-by: Iuliana Prodan <iuliana.prodan@nxp.com>
De-duplicate the `if target in ('imx8', 'imx8m'):` copy/paste/diverge
before someone updates one and not the other. Also better to define
bootloader always.
Zero functional change.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Add the _VARGS variant of DT_FOREACH_NODE and
DT_FOREACH_STATUS_OKAY_NODE for when we want to do some kind of
operation on all the nodes in the tree.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
If build is used with -T option, sysbuild flag was ignored
neverthelesss being present in sample.yaml/testcase.yaml
file.
Signed-off-by: Emil Obalski <Emil.Obalski@nordicsemi.no>
Add support for disabling loading of the flasher stub and instead only talk
to the ROM bootloader.
Signed-off-by: Henrik Brix Andersen <henrik@brixandersen.dk>
The `USE_NEGATIVE_ERRNO` rule consistently generates false
positives when working with certain areas of POSIX.
It makes sense to disable this check for the POSIX API
rather than requiring merge superpowers constantly. That
way, we can merge as per usual after sufficient approvals
rather than waiting for someone with merge superpowers to
override / manually merge.
Signed-off-by: Christopher Friedt <cfriedt@meta.com>
Things have moved around so we need to update the
suppression file so the callstack and types of
leaks valgrind reports still match.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
Add the first version of the native simulator.
The simultaor is taken as is from
https://github.com/BabbleSim/native_simulator/
sha: 74986abfe088a1780e604dae65f87470b4c2a0eb
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
Add a variation of the basic Kconfig check that runs with no modules,
catches symbols that are used in the main repository but are defined
only in modules, which are potentially problematic if a downstream
project is not using the specific module.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
Some node items in Kconfig can be kconfiglib.MENU or kconfiglib.COMMENT.
Those are integers and thus do not contain a node.item.name field.
Handle those separately to avoid hitting the followig exception:
Traceback (most recent call last):
File "/home/runner/work/zephyr/zephyr/./scripts/ci/check_compliance.py",\
line 1307, in main
n_fails = _main(args)
File "/home/runner/work/zephyr/zephyr/./scripts/ci/check_compliance.py",\
line 1242, in _main
test.run()
File "/home/runner/work/zephyr/zephyr/./scripts/ci/check_compliance.py",\
line 277, in run
self.check_no_redefined_in_defconfig(kconf)
File "/home/runner/work/zephyr/zephyr/./scripts/ci/check_compliance.py",\
line 445, in check_no_redefined_in_defconfig
Kconfig node '{node.item.name}' found with prompt or help in\
{node.filename}.
AttributeError: 'int' object has no attribute 'name'
Seen in #58454.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Instead of breathe.
I'm not sure of the best version comparison to use. There is no
intention to break the API for docleaf but it is also still young
in some ways and there might be adjustments to the configuration values
that would need to be reflected in the version number as it is still
pre-1.0 that would only be a minor version change.
Signed-off-by: Michael Jones <m.pricejones@gmail.com>
* Add support for coredump on ARM64 architectures.
* Add the script used for post-processing coredump output.
Signed-off-by: Marcelo Ruaro <marcelo.ruaro@huawei.com>
Signed-off-by: Rodrigo Cataldo <rodrigo.cataldo@huawei.com>
Signed-off-by: Roberto Medina <roberto.medina@huawei.com>
Extended parsing of report.xml file - produced by pytest-twister-harness
plugin. Now testcases are properly extracted and added to twister
report. Added unit tests.
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
We have many tests that are marked as slow that do not run in CI and any
known workflow, to isolate them and be able to run them on their own,
add a new option --enable-slow-only which causes only those tests to build
and execute.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
The current toolchain version for espressif SoCs does not
have the same limitation on registers indexing as the previous
one, enabling sending the correct A0-A15 register values directly
Signed-off-by: Lucas Tamborrino <lucas.tamborrino@espressif.com>
Rename the device dependencies array variable to use the "deps" name, in
line with latest renamings in device.h
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
Rename multiple internal device macros to use the DEVICE_DEPS naming, so
that it is clear they belong to the device dependencies APIs.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
These macros are used internally by the device dependencies functions.
There's no need to expose them publicly, so prefix them with Z_ and add
them under INTERNAL_HIDDEN docs section.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
Use the "device_deps" naming scheme to emphasize we are storing device
dependencies. The fact we are using device handles to store them is an
implementation detail.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
Rename the gen_handles script and all of its references/associated files
to gen_device_deps. The new new makes things more clear, because the
script just take care of generating, for each device, an array of device
dependencies. While device handles are used internally to store this
information, it is in reality an implementation detail.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
Add DeviceAbstract class to default imports from pytest-twister-harness
package to simplify importing DUT package, when creating tests.
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
The code_relocation feature creates generic section names that sometimes
conflict with already existing names.
This patch adds a '_reloc_' word to the created names to reduce the risk
of conflict.
This solves #54785.
Signed-off-by: Björn Stenberg <bjorn@haxx.se>
I noticed that not all strings were put into database. That is because
algorithm was searching for any null terminated byte arrays in the
section (e.g. rodata) and then attempting to decode it to string.
However, section may contain other static const variable and if a
string is preceded by a variable that has non-printable bytes and no
zero at the end then algorithm was picking a candidate that started
with non-printable characters followed by the string. Such candidate
was discarded because it was not a valid string.
Algorithm is changed to treat as string candidate any sequences that
contains printable bytes followed by 0.
Signed-off-by: Krzysztof Chruściński <krzysztof.chruscinski@nordicsemi.no>
Fixes an issue on windows where there would be many permutations
of paths with different folder separators by converting to
posix-style path separators.
Issue was introduced by
https://github.com/zephyrproject-rtos/zephyr/pull/58351
Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
rimage always requires a key argument even when it does not use it.
Some configurations (IMX) use rimage only for "stitching" the image but
not for signing. These were failing to build directly from west with
pretty cryptic stack trace, see below. This has never been a problem for
the SOF project because it has always worked around this rimage
limitation by providing a bogus -k argument - very much like the one
added by this commit.
```
File "zephyrproject/zephyr/scripts/west_commands/sign.py", in sign
extra_ri_args += [ '-k', str(sof_src_dir / 'keys' / cmake_default_key) ]
TypeError: unsupported operand type(s) for /: 'PosixPath' and 'NoneType'
ninja: build stopped: subcommand failed.
```
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
The syscall generation phase parses all header files to look
for potential syscalls, and emits all the relevant files to
enable syscalls. However, this results in all the syscall
marshalling functions being included in the final binary.
This is due to these functions being referred to inside
the dispatch list, resulting in ineffective garbage
collection during linking. Previous commits allows each
drivers and subsystems to specify which header files
containing syscalls are relevant. So this commit changes
the syscall generation to only include the syscalls needed
for the build in the syscall dispatch list and removing
various bits related to that. This allows the linker to
garbage collect unused syscall related function, and thus
reducing final binary size.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Use the natural sort of list when generating a hardware map. The
list is sorted with a serial port as a key. When more than 10 ports
are active and some of devices use more than one port, the ports
of one device may be listed in wrong sequence, which may cause
futher problems with selecting the right port for listening to
the device.
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
When CONFIG_HAS_DYNAMIC_DEVICE_HANDLES is selected, devices handles are
placed in RAM region so that they can be modified at runtime. However,
the gen_handles.py script added the `const` attribute to the device
handle arrays regardless of this setting, leading to faults when running
the application. This may be an indicator that this feature is not
actively used.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
We have been launching 10 runners up a certain number of tests, although
we only neeed half of that for some scenarios.
Too many runners started that have to execute just a small number of
tests wastes times on setup and blocks the queue. Just start the number
of nodes needed based on initial calculation.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Add protection timeout for pytest subprocess, to avoid situation of
suspending whole Twister in case of internal pytest test problem.
Co-authored-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
Signed-off-by: Piotr Golyzniak <piotr.golyzniak@nordicsemi.no>
Added waiting for prompt instead of calling time.sleep,
to fix issue with longer setting up platforms with TF-M
enabled. Wrapped iter_stdout generator, to avoid issues
when generator is created and called multiple times
in test scenarios.
Fixes: #58747
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
Print a clean error message when provided a relocatable file, instead
of generating a source file which will cause compilation errors further
away from the source.
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
Based on investigations with relocatable `.elf` files, the symbol table
data in relocatable files is not shifted by the section address. Remove
the shift from these files to ensure that data can be pulled from the
symbol table.
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
Retrieve the section holding the symbol data directly from the symbol
information, instead of trying to find a section that contains data at
the correct address. The later approach does not work for relocatable
files, which contain multiple sections all containing data in
overlapping temporary memory addresses (usually starting at 0).
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
The linker generates an .elf file wth encoded strings containing
filenames. If those filenames contain non-ASCII characters we need to
know which encoding was used by the linker to store those filenames. Use
the system's preferred encoding for that purpose.
Fixes#55148.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
UTF-8 is not always the default encoding on some platforms, but it's the
only reliable way to store encoded strings for filenames using
non-ASCII characters.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Add support for more verbose output enabling debugging output and add a
first debug print to output the file to device path mapping. Found this
to be extremely useful to find what file is instantiating a specific
device for non obvious cases.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
os.path.commonpath() throws a ValueError exception when paths in the
list provided share nothing in common (like for example two Windows
paths on different drive letters). Handle that special case properly
instead of letting the exception propagate up.
Fixes#51549.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
When calling twister with --coverage and no platform specified on the
command line, set the the value of coverage platforms to an empty list.
Fixes#57534
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
testinstance.py contains a string literal comparison to the
word 'platform', which has been misspelled as 'plaform'.
The change is verified by a test expansion.
Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
Removed few VIF properties which are being hardcoded
Updated the script to parse source VIF XML and add information to
the output
Added optional Kconfig option to configure custom source VIF XML path
Cleaned up the code
Signed-off-by: Madhurima Paruchuri <mparuchuri@google.com>
Making the necessary changes to enable the new pytest plugin.
By default Twister should work without the pytest-twister-harness
plugin installed. To achieve this, each time Twister calls pytest,
the PYTHONPATH environment variable is expanded and the
`-p twister_harness.plugin` option is added to the pytest command.
Co-authored-by: Piotr Golyzniak <piotr.golyzniak@nordicsemi.no>
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
Adding pytest plugin dedicated to running pytest tests in Zephyr
project. This plugin provides a dut fixture which allows to handle
bidirectional communication with the device under test. This version
of plugin can be used for tests dedicated to real hardware, QEMU and
native_posix simulator.
Co-authored-by: Lukasz Fundakowski <lukasz.fundakowski@nordicsemi.no>
Co-authored-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
Co-authored-by: Katarzyna Giadla <katarzyna.giadla@nordicsemi.no>
Signed-off-by: Piotr Golyzniak <piotr.golyzniak@nordicsemi.no>
Update the sensor shell logic to use the new sensor_read() APIs and
make triggers an option of the sensor_shell sample (this avoids the
trigger stealing the interrupt status from one-shot reads).
Signed-off-by: Yuval Peress <peress@google.com>
Add a new async API based on the RTIO subsystem. This new API allows:
1. Users to create sampling configs (telling the sensor which channels
they want to sample together).
2. Sample data in an asynchronous manner which provides greater control
over the data processing priority.
3. Fully backwards compatible API with no driver changes needed for
functionality (they are needed to improve performance).
4. Helper functions for processing loop.
Signed-off-by: Yuval Peress <peress@google.com>
Add support for calling the `renode-test` command from west and twister.
Enable running Robot Framework tests suites in Renode.
Signed-off-by: Michał Szprejda <mszprejda@antmicro.com>
Signed-off-by: Mateusz Hołenko <mholenko@antmicro.com>
Internal bootloader will only run application image if valid product
header is present on flash. This means product header is required for
application that are not linked to code partition. Other applications
that are linked to code partition are meant to be run by MCUboot and
should not touch product header, with the exception of MCUboot itself
which is also run by internal bootloader so requires product header.
Default flash load offset for applications not linked to code partition
is set to 0x2400 as this is where internal bootloader looks for an
application image to run based on product header written by flasher.
Flash load offset for MCUboot is set from boot partition.
Valid product header is added by ezFlashCLI when using "flash_image"
command.
Co-authored-by: Andrzej Kaczmarek <andrzej.kaczmarek@codecoup.pl>
Signed-off-by: Andrzej Kaczmarek <andrzej.kaczmarek@codecoup.pl>
Signed-off-by: Niek Ilmer <niek.ilmer.aj@renesas.com>
Current implementation of ConfigurationError causes
an exception to be thrown.
It is because a PosixPath is being concatenated with an str with a '+'
operator, which is not permitted.
The path in question is now cast to an str.
Additional test has been added to test_twister.py to track regressions.
Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
For the arch's that have standard toolchains thare are supported
across the arch set them in twister rather than in <BOARD>.yaml.
Signed-off-by: Kumar Gala <kumar.gala@intel.com>
This commit updates the `snippets.py` script to process the snippets
in the order that the `--snippet` arguments are specified, which
corresponds to the order of snippets listed in the `SNIPPET` CMake
variable.
Signed-off-by: Stephanos Ioannidis <stephanos.ioannidis@nordicsemi.no>
Add a check_init_priorities.py scripts. This goes through all the object
files looking for known initialization sections and building a map of
device ordinals and their effective initialization priority. Then
compares that with the list of dependencies generated based on the
device tree and reports any warning (dependent devices, same priority)
or error (incorrect priority).
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
Fixes: #57139
Snippets provides the possibility of defining EXTRA_DTC_OVERLAY_FILE and
EXTRA_CONF_FILE in snippets.
Snippets must co-exist with existing infrastructure of
EXTRA_DTC_OVERLAY_FILE and EXTRA_CONF_FILE, and a user specifying a
snippet must be able to specify extra files for adjusting the snippet.
This means that if the following is specified:
`-DSNIPPET=some_snippet -DEXTRA_CONF_FILE=extra.conf`
then `extra.conf` may contain adjustments to the snippet.
Similar to sysbuild. Imagine a sysbuild controlled image uses a default
snippet for building, by ensuring that any extra
`-D<image>_EXTRA_CONF_FILE=extra.conf` arguments takes precedence over
the snippet we allow users to make adjustments if they need.
This commit introduces a snippets scope where snippet scoped variables
can be set with `zephyr_set()` and then `zephyr_get()` will take the
snippet scoped variables into consideration before returning.
Adjust calls to `zephyr_get(EXTRA_DTC_OVERLAY_FILE)` and
`zephyr_get(EXTRA_CONF_FILE)` to use `MERGE` to ensure all scopes are
considered.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
This commit adds change snippets to use the new EXTRA_DTC_OVERLAY_FILE
and EXTRA_CONF_FILE Zephyr build system variables instead of
DTC_OVERLAY_FILE and OVERLAY_CONFIG.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Sysbuild now generates a .config.sysbuild config file which specifies
settings controlled by sysbuild.
Any setting specified in this .config will overrule user provided
setting, and a warning will be raised if the sysbuild controlled value
is different from the value specified by the user.
This has the following benefits:
- Allow sysbuild to control any build setting without adjustments to
the existing Kconfig tree
- Allow sysbuild to adjust settings based on knowledge regarding enabled
images / bootloaders.
- Cleanup CMake code, as settings in sysbuild no longer needs to be
propagated using CMake cache variables.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
This commit introduces an internal FORCE_CONF_FILE CMake setting which
allows higher order build systems to generate a configuration file
which will always take precedence.
This means that in case a user tries to change any setting to be
different than the defined value in the FORCE_CONF_FILE provide file(s),
then a warning will be printed and the setting will be reset to the
value given in the FORCE_CONF_FILE file.
Example of such warning:
<path>/.config.sysbuild:1: warning: BOOTLOADER_MCUBOOT
(defined at Kconfig.zephyr:766) set more than once.
Old value "n", new value "y".
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Some 3P logic (Pigweed for example) is already heavily invested in
gTest as a testing framework. Adding the `gtest` harness enables
running the existing 3P tests on various Zephyr platforms and configs.
Signed-off-by: Yuval Peress <peress@google.com>
Simics converts newline characters when writing console output to
stdout, so we need to remove them as a suffix string. Otherwise the
console harness fails to match expected PASS/FAIL/RunID string patterns
and twister tests timeout.
Signed-off-by: Maureen Helm <maureen.helm@intel.com>
Minimal requirements for nanopb were not accurate
to allow its usage (and causing related tests to build fail).
This change updates requirements for protobuf (>=3.20.3) and
grpcio-tools (>=1.47.0) to use protocol buffer.
Fixes#56103
Signed-off-by: Katarzyna Giądła <katarzyna.giadla@nordicsemi.no>
After generating a hardware-map a platform field is marked
as 'unknown'. If it is not changed, do not process them
when running --device-testing.
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
In some cases CMake doesn't create build directory and in those cases
Twister should create this directory by itself.
Signed-off-by: Piotr Golyzniak <piotr.golyzniak@nordicsemi.no>
Also add a comment in each file reminding to keep them the same.
Fixes 251f269e23 ("west: v0.14.0 is required now (and soon, v1.1")
Confusing error message before this commit:
```
-- Found west (found suitable version 0.13.1, minimum required is 0.7.1)
CMake Error at SOF/zephyr/cmake/modules/zephyr_module.cmake:77 (message):
Traceback (most recent call last):
File "SOF/zephyr/scripts/zephyr_module.py", line 733, in <module>
main()
File "SOF/zephyr/scripts/zephyr_module.py", line 678, in main
west_projs = west_projects()
^^^^^^^^^^^^^^^
File "SOF/zephyr/scripts/zephyr_module.py", line 536, in west_projects
from west.configuration import MalformedConfig
ImportError: cannot import name 'MalformedConfig'
from 'west.configuration'
(west/src/west/configuration.py)
```
Clearer error message after this commit:
```
CMake Error at SOF/zephyr/cmake/modules/west.cmake:68 (message):
The detected west version, 0.13.1, is unsupported.
The minimum supported version is 0.14.0.
Please upgrade with:
/usr/bin/python3.11 -m pip install --upgrade west
```
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
This significantly improve user experience as 32 bit mdb binary
require to install multiple libraries before it can be used on
modern linux distros.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
Commit ce2a7d9a1a
("scripts: zephyr_module: handle expected west errors")
introduced some better error handling that requires
west version v0.14.0 or later to work.
Bump the west version in requirements-base.txt accordingly.
Due to the way zephyr_module.py is handling imports, this API change
resulted in zephyr_module.py running on older versions of west
reacting as if west was not installed, instead of erroring out.
Fix that so users who are on older west will get a hard error.
(We're about to force everyone to move to west v1.1 as soon as I can
get that release done, but this hotfix should still be helpful in the
interim as well.)
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
This commit adds a new `GitDiffCheck` compliance check that checks the
newly added commits with `git diff --check` and reports them back if an
error is found.
This check is needed for some files (e.g. Kconfig) as they are not checked
by `Kconfig` and `KconfigBasic` checks on every commit in a pull request.
Signed-off-by: Filip Kokosinski <fkokosinski@antmicro.com>
Errors can occur if the user's workspace is not set up properly.
Output a better error message instead of dumping stack in these cases.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
sign.py has an internal and indirect way to compute a default -c signing
schema option and pass it to rimage. It is built by appending
`$platform.toml` to whatever `rimage/config/` location was
found. Defaults are very convenient but in this case this computed -c
option can conflict with an explicit -c option spelled out by the user.
Scan for any explicit -c coming directly from the user. If any found,
ignore the default we computed. This is what is already being done for -k.
The precedence across rimage parameters coming from different places is
too complicated. Not passing multiple -c options simplifies the logic a
little bit.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
We have been clearning the list of testcases in every iteration over the
symbols ending up with only one testcase in the instance.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Adds a Kconfig option which controls the version of the application
to use when the image is signed using imgtool.
When an application VERSION file is present, the default value will be
identical to the application version, else it will be 0.0.0+0, but a
project may still decide another value, if it so prefers.
Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Support use of these macros with properties of type phandle and
string by allowing iterating over:
- a phandle as if it were a phandles of length 1, for convenience and
consistency with our ability to take its length (and getting 1)
- the non-null characters in a string: we exclude the null for
consistency with the return value of DT_PROP_LEN() on string
properties, which, like strlen(), does not include the null
With this and a previous patch expanding the usage of DT_PROP_LEN(),
there is now a relationship between being able to take a property's
logical length with DT_PROP_LEN() and being able to iterate over its
logical elements with DT_FOREACH_PROP_ELEM(). Explain this in the
documentation.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
It will be convenient to treat these respectively as degenerate cases
of 'phandles' and 'string-array'. Add support for this and regression
tests. (There's nothing to do in the case of 'phandle' beyond
documenting the guarantee.)
For the record, the other DT_PROP_LEN() tests for each type are in:
type test case property
------------ -------------------- ------------
array test_arrays a
string-array test_path_props compatible
uint8-array test_arrays b
phandles test_phandles phs
phandle-array test_phandles pha-gpios
phandle test_phandles ph
Update docstrings and fix some issues in them.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
We generally try to have comments in this file that show the form of
each generated macro. This is particularly important in the
write_vanilla_props() function, since that is called on every node in
the tree and handles generic macros that are widely applicable.
Various generated macros have been added over time that don't have
the corresponding comments; add these now.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Add a new option to Twister that allows to load alternative
configuration files, `--alt-config-root`. This new option takes an
arbitrary number of root folders where alternative configuration files
can be stored. Twister will check if a test configuration file exists in
any of the alternative test configuration folders. For example, given
`$test_root/tests/foo/testcase.yaml`, Twister will use
`$alt_config_root/tests/foo/testcase.yaml` if it exists.
This feature can be useful if an out-of-tree project needs to run
upstream tests in different configurations, potentially not available
upstream (e.g. an out-of-tree board, or Kconfig setting).
Note that overlaying has been discarded because we can't easily delete
fields from the original configuration file, something that in certain
cases could be required. Current approach may lead to some dupplication,
but guarantees full control of the test configuration.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
Make twister to raise error by default whenever duplicated test ids
are found. Remove redundant option to list duplicates.
Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
Add a define of the form
`DT_N_<node-id>_P_<prop-id>_ENUM_VAL_<val>_EXISTS` for enumerated
devicetree properties. This enables the devicetree API to check whether
an enum is a given value directly, without resorting to error-prone
checks against the enum index.
Example generated defines (int and string):
`#define DT_N_S_test_S_enum_4_P_val_ENUM_VAL_5_EXISTS 1`
`#define DT_N_S_test_S_enum_6_P_val_ENUM_VAL_zero_EXISTS 1`
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
Adjust WEST_PYTHON to posix path to be consistent with other Python
scripts which passes paths to the Zephyr CMake build system.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Don't override a test case name check failed result by the following
RunID check failed result to report clearly about test case mismatch
situation as well as more specifically when the same test case appears
with unexpected RunID.
Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
Fix suite-name-check bypassed at DeviceHandler on flash errors.
If the flash timeout is smaller than needed, but test images are
actually flashed successfully and appropriate tests executed,
then results from the previous test case were taken because
the flash error status makes the suite-name-check bypassed.
Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
We can now have situations like this when it comes to lists in Twister
config:
```yaml
common:
platform_exclude: foo
tests:
my.test:
platform_exclude: bar
```
```yaml
common:
platform_exclude: foo
tests:
my.test:
platform_exclude:
- bar
- baz
```
```yaml
common:
platform_exclude:
- foo
- bar
tests:
my.test:
platform_exclude: baz
```
```yaml
common:
platform_exclude:
- foo
- bar
tests:
my.test:
platform_exclude:
- baz
- bazz
```
This patch fixes handling of cases (2), (3) and (4).
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
Updates the pyOCD requirement to 0.35.0 to support a new flasher
option of flashing and preventing reset of the device.
Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
Add a new tool that convers string-based lists on Twister config files
to native YAML lists.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
This allows to write both, lists or strings in the YAML file.
Unfortunately pykwalify schema language does not support multi-type
fields. JSON schema or Marshmallow could be used instead.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
This patch handles mixed types between common/tests sections in the
configuration, e.g. str+list, list+str, str+str.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
Since pykwalify does not support multi-type fields, we are forced to use
the 'any' type. This is used for fields that allow sets/lists written as
either YAML lists or space-separated strings. This patch checks for
str/list Python type, and raises a configuration error if not true so
that user obtains better error messages in case the configuration file
contains invalid format.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
Twister configuration files have a few fields defined with the "set"
type. The set needs to be specified as a space-separated list of
entries, for example:
```yaml
plaform_allow: boardx boardy
```
This patch allows users to write these fields as YAML lists, for
example:
```yaml
platform_allow:
- boardx
- boardy
```
The list gets converted to a Python set, therfore the same exact
behavior is kept in Twister.
In some cases like in the example above, using lists allows to have more
readable configuration files, specially if the list is long. It also
helps on diffs, provided we have one item per line.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
A change to the scope selection rules was introduced by #52715
but it doesn't comply with the description in the code. What's more,
the change introduces major issues i.e. non deterministic scope
selection, especially in a CI environment. More context at #57595
Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
Clean up log_const to utilize macros for handling sections.
Update database_gen.py to match naming convention change.
Signed-off-by: Kumar Gala <kumar.gala@intel.com>
Twister does not handle out-of-tree boards automatically. Modules like
e.g. example-application that define their own board root folder, are
required to inform twister via `--board-root`. In contrast, west is able
to to this automatically. This patch makes twister a bit smarter by
automatically filling the board root list with module paths as well.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
Add option to shuffle tests (randomly) before they are split into subsets.
This allow to fairly distribute tests across subsets,
so that long tests are not crowded in a single group.
This also allows to detect unwanted dependencies in test execution order.
Also, added option to provide custom seed to random generator
used to shuffle tests.
This will allow to reproduce certain test order if needed.
Used seed is printed at console.
Signed-off-by: Piotr Kosycarz <piotr.kosycarz@nordicsemi.no>
The now legacy --nrf-family parameter takes the "raw" family name as
nrfjprog accepts it. But now we use an intermediate representation with
{FAMILY}_FAMILY as the format, so covert to it as required.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
pylint reports the following issues on the
scripts/build/gen_isr_tables.py:
C0325:Unnecessary parens after '=' keyword (superfluous-parens)
File:scripts/build/gen_isr_tables.py
Line:16
Column:0
C0325:Unnecessary parens after '=' keyword (superfluous-parens)
File:scripts/build/gen_isr_tables.py
Line:316
Column:0
Fix that so that unrelated PR touching the same file have a chance to
pass the compliance checks.
Signed-off-by: Aurelien Jarno <aurelien@aurel32.net>
- Runs Silabs Commanders which is useful for SiLabs boards that
can not use J-Link for various reasons.
- Flash support only, no debug.
Signed-off-by: Roman Dobrodii <rdobrodii@antmicro.com>
The `get_maintainer` script currently generates an error when a non-
slash-suffixed glob pattern matches any directories -- this is
incorrect because the glob pattern may be used to match files.
This commit updates the script to only generate an error when a
non-slash suffixed _only_ matches directories.
Signed-off-by: Stephanos Ioannidis <stephanos.ioannidis@nordicsemi.no>
Some compilers and static analyzers warn when non-static (globals) are
not declared in some .h file. Tweak the generated code and add an
`extern` declaration before each device handle definition to make it
look it was declared in a (non-existent) .h file.
This makes the code a bit longer but it is already generated and very
repetitive anyway.
In the typical SOF configuration this silences about 150 sparse
warnings; as many as device handles.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
The '%ProgramFiles%' is not guaranteed to be 'C:\Program Files' on
Windows x64.
The value will be 'C:\Program Files (x86)' if the west is executed by
32-bit Python.
To correct the issue '%ProgramW6432%' must be used.
Signed-off-by: Georgij Cernysiov <geo.cgv@gmail.com>
Adds mcuboot Kconfig options to the ignore list as these are
defined in mcuboot but used in documentation and samples which
utilise sysbuild to build mcuboot.
Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
Removes converting the dummpy IMG MGMT group Kconfig as this has
been removed from the tree and users would not have used it
anyway.
Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
Sometimes we want to force the inlining of a __syscall. Introduce a new
__syscall_always_inline symbol to do that.
Signed-off-by: Carlo Caione <ccaione@baylibre.com>
The initshell sections in the linker scripts where associated with theo
old shell code. The old shell code has been removed for some time so
remove references to initshell in the linker scripts and size_calc.py
Signed-off-by: Kumar Gala <kumar.gala@intel.com>
Convert following construction:
elif self.options.device_serial or self.options.device_serial_pty:
if self.options.device_serial:
...
else
...
to the construction:
elif self.options.device_serial:
...
else self.options.device_serial_pty:
...
Signed-off-by: Andrei Emeltchenko <andrei.emeltchenko@intel.com>
Use flash_timeout and flash_with_test options not only for
serial console but also for device_serial_pty.
Signed-off-by: Andrei Emeltchenko <andrei.emeltchenko@intel.com>
compliance script runs pylink on size_calc.py and reports:
C0325:Unnecessary parens after '=' keyword (superfluous-parens)
Fix warning by remove unnecessary parens.
Signed-off-by: Kumar Gala <kumar.gala@intel.com>
Refactor the existing nrfjprog tests to make them compatible with the
new nrfutil runner implementation, and extend them so that they are
exectued for both tools.
There is missing verification of the JSON generated by the nrfutil
runner, whcih will be added in a follow-up commit.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
- Rename the test file to test_nrf.py in preparation for future tests
covering nrfutil as well
- Rename the test folder from nrfjprog/ to nrf/
- Change the id_fn returned string formatting so that it's compatible
with executing a single test case via -k as a parameter (no spaces or
commas)
You can now execute:
python -m pytest tests/test_nrf.py -k \
'test_nrfjprog_init[NRF51-pin_reset-override_snr-sector[anduicr]\
_erase-no_recover-no_tool_opt]'
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Add support for Nordic's new nRF Util tool, aka nrfutil. Via its
"device" command, nrfutil now supports most of the functionality offered
by nrfjprog.
The tool itself can be found here:
https://www.nordicsemi.com/Products/Development-tools/nrf-util
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
In order to better support the ability of the upcoming nrfutil runner to
defer all operations and execute them all at once, defer flushing right
until the end, when everything has been queued up (recover, program and
reset) instead of forcing execution at the program stage.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Standardize to nrfutil error codes instead of nrfjprog ones. This is
because nrfutil has a wider range of functionality and will eventually
be the default runner for all nRF devices. It is also consistent with
the internal operation representation, which is now using the nrfutil
format already.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
This concludes the type annotations for the public API for the module,
along with the relevant internal state. It's not worth type annotating
the internal backwards compatibility shim for !include.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
This requires adding a private constructor so that mypy
can tell what all the final instance state is going to be.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Converting this to a dataclass will make it easier to type annotate.
Adding type annotations is incremental progress towards type checking
the entire module.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Converting this to a dataclass will make it easier to type annotate.
Adding type annotations is incremental progress towards type checking
the entire module.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Converting this to a dataclass will make it easier to type annotate.
Adding type annotations is incremental progress towards type checking
the entire module.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Converting this to a dataclass will make it easier to type annotate.
Adding type annotations is incremental progress towards type checking
the entire module.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Converting this to a dataclass will make it easier to type annotate.
Adding type annotations is incremental progress towards type checking
the entire module.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Incremental progress towards type annotating the whole module.
Annotate helper procedures used by the class as well.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
This is just moving the class definition higher in the file. I am
reordering the classes to make it possible to type annotate the module
in a more readable way.
Git might make the diff look bigger than it really is.
To verify this is just moving code, use 'git diff --minimal'.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
This is just moving the class definition higher in the file
to make it easier to type annotate the module.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
This is just moving the class definition higher in the file
to make it easier to type annotate the module.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
This is just moving the class definition higher in the file
to make it easier to type annotate the module.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
This is just moving the class definition higher in the file
to make it easier to type annotate the module.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
This is just moving the class definition higher in the file
to make it easier to type annotate the module.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
This is just moving the class definition higher in the file
to make it easier to type annotate the module.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
This is just moving the class definition higher in the file. I am
reordering the classes to make it possible to type annotate the module
in a more readable way.
Git might make the diff look bigger than it really is.
To verify this is just moving code, use 'git diff --minimal'.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Just like we did for dtlib in 15e3e317f7
("dtlib: implement copy.deepcopy() for DT"), except this time it's for
EDT. This also can do no harm and will be useful for implementing
system devicetree support.
No functional changes expected under the assumption that no users are
relying on us having stashed the exact bindings_dirs list passed to
the constructor. This patch switches to making a defensive copy, which
is safer and makes implementing this a little cleaner.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Move all the initial settings of instance attributes to the
constructor, so we can keep track of them all more easily.
No functional changes expected.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Tests dedicated for choosing binaries basing on platform.binaries and
runners.yaml file. Additional tests for paths sanitizing.
Signed-off-by: Piotr Golyzniak <piotr.golyzniak@nordicsemi.no>
`TestInstance.get_elf_file()` does not include the binary name used for
unit testing (`testbinary`) in its search, causing unit tests to not
complete successfully. This does not cause tests to fail and thus goes
unnoticed.
Update the glob expressions used in `get_elf_file()` to also look for
files called `testbinary`
Signed-off-by: Tristan Honscheid <honscheid@google.com>
Utility script to migrate SYS_INIT() calls from the old signature:
```c
int (*init_fn)(const struct device *dev)
```
to
```c
int (*init_fn)(void)
```
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
If we are doing a sysbuild get_elf_file needs to return the default
domain's build_dir. The standard build_dir will not any ELF builds
in it so will error out.
Signed-off-by: Kumar Gala <kumar.gala@intel.com>
This is usually used only when doing size calculation, but now it is
being used for extracting symbols from the ELF for testing purposes and
some issues arise with multiple ELF files found.
Simplify the code a bit to make it easy to maintain a lists of things to
exclude.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
The DeviceHandler class method monitor_serial() has time-of-check
to time-of-use (TOCTOU) race condition possible on its serial port
which might be closed and its file descriptor cleared in between
of isOpen() and is_waiting status checks.
The issue is alleviated with controlled exception handling.
Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
Fixes an issue when DeviceHandler awaited for a suitable DUT infinitely.
It was possible if the device map has no devices with required platform,
or no devices with fixture needed for the test case, or devices without
serial connection. A trivial case is a typo in platform name.
Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
Invoking `west sign` in `west build` accelerates twister because `west
build` is run in parallel, see rationale in superseded and very
different (CMake-based) PR #52942.
To maximize backwards compatibility:
- `west sign` is optional in `west build`
- `west flash` will sign (again) if any rimage --option is passed
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Moving `west sign` from `west flash` to `west build` for rimage has
multiple advantages (including a bit more consistency with
imgtool). However it makes `west build` fail when rimage is missing.
To avoid forcing every CI and developer who never used it before to
install rimage, make signing optional when passing new `west sign`
option --if-tool-available. A clear warning is printed.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Make RIMAGE_SIGN_KEY a CMake CACHE variable so `west sign` can find it
and use it as a default value.
Fixes https://github.com/thesofproject/sof/issues/6917
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Reduce duplication, no functional change except for a shorter log
statement with slightly less information.
This is required by the next commits.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
The --tool parameter is not required anymore thanks a [sign] entry in
west config like this one:
```
[sign]
tool = imgtool
```
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
For ztest twister would parse the symbol information that was
generated as part of the build (zephyr.symbols). However the format
of the zephyr.symbols files is highly dependant on the toolchain.
Move to using pyelf to parse the symbol information directly from
zephyr.elf instead so that this works regardless of toolchain.
Signed-off-by: Kumar Gala <kumar.gala@intel.com>
On native platforms we typically have CONFIG_BUILD_OUTPUT_EXE set so
we end up with both zephyr.elf and zephyr.exe. This would cause a
call to get_elf_file() to fail. There isn't any reason to error out
since zephyr.elf and zephyr.exe should be identical. So allow it
if the platform type is native.
Signed-off-by: Kumar Gala <kumar.gala@intel.com>
Use explicit flash command execution timeout at DeviceHandler
instead of the hardcoded value of 60 sec.
When a HW platform executes test cases right after the flash command,
the test case remaining timeout is affected by how much time the flash
operation consumes. Some simulation platforms need to spend significant
amount of time on each 'flash' cycle, usually adding the same delay on
each test case.
This improvement adds two new command line options and device map fields:
--device-flash-timeout - for the flash operation timeout
('flash-timeout' device map field).
--device-flash-with-test - to indicate that the platform flash
command also runs a test case, so the overall timeout should be
calculated as a sum of the flash timeout and the current test case
timeout to receive all console output from the platform
('flash-with-test' device map field).
The device map field values override command line values for the
particular platform where configured.
Default behavior is backward compatible: flash operation fixed timeout
is 60 sec. not including the test case timeout.
Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
An explicit import of edtlib is not required when using
pickle.load() and creates a duplicate module object.
Local testing shows that pickle.load() throws the ModuleNotFoundError if
the sys.path cannot find the edtlib or associated modules.
Signed-off-by: Keith Short <keithshort@google.com>
An explicit import of edtlib is not required when using
pickle.load() and creates a duplicate module object.
Retrieve the correct module object directly from the pickle file
using inspect.getmodule().
Signed-off-by: Keith Short <keithshort@google.com>
An explicit import of edtlib is not required when using
pickle.load() and creates a duplicate module object.
Also eliminate the check for the EDTError, as this exception cannot be
raised by pickle.load().
Signed-off-by: Keith Short <keithshort@google.com>
An explicit import of edtlib is not required when using
pickle.load() and creates a duplicate module object.
Signed-off-by: Keith Short <keithshort@google.com>
This refactors the script to introduce a SectionKind enum that
represents each of the kinds of sections handled by this system (text,
data, rodata or bss), using that to improve the code structure by
reducing the use of strings in favor of indicating the use of values by
their types.
Signed-off-by: Peter Marheine <pmarheine@chromium.org>
When using code_data_relocation the script identifies sections that
should be relocated, but cannot guarantee that section names are unique
across all linked files. If a section name is not unique then matching
all sections with that name across all input files will relocate more
data than intended.
As a simple example, if both file1 and file2 contain .rodata.strings
sections, then if gen_relocate_app inspects only file1 it will generate
a linker script fragment `*(.rodata.strings)` that matches both object
files.
This commit changes gen_relocate_app to make the linker match on object
files as well (`*file1.o(.rodata.strings)`) to ensure unwanted sections
don't get relocated.
Signed-off-by: Peter Marheine <pmarheine@chromium.org>
Rather than having to add 'armclang' to every <BOARD>.yaml, just list
it as a generally supported toolchain if the architecture is 'arm'.
Signed-off-by: Kumar Gala <kumar.gala@intel.com>
Add an escape hatch so users can insert a customizable delay
between asking the board to reboot into bootloader mode and
trying to program the new image. This lets people account for
OS-specific enumeration time done after the board resets.
Fixes: #56540
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
When a test start, mark it as such, if we do not get any results due to
a timeout, it will be finally marked as failed. Previously such tests
causing a freeze were marked as blocked (or not run), which is not
exactly right.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Skips due to overflows on integration_platforms during integretion
mode should be treated as errors.
Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
`self.instances[filtered_instance.name]` and `filtered_instance`
are already referancing the same object.
Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
If canonical zephyr base is already ended by trailing slash, then it
should not be added redundantly.
Signed-off-by: Piotr Golyzniak <piotr.golyzniak@nordicsemi.no>
In some cases built binary files have different names than "zephyr.hex"
files, and for those cases, information about names of file to keep
should be taken from runners.yaml file.
Moreover if in runners.yaml file exist absolute files paths to binary
files, then they should be changed to relative ones.
Signed-off-by: Piotr Golyzniak <piotr.golyzniak@nordicsemi.no>
This moves CONFIG_LLVM_USE_LD into cmake/toolchain/llvm as this
is a toolchain kconfig. Also make it a choice to allow the use
of LLVM's lld as linker.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Adds a Kconfig for MCUboot which is used in the documentation to
the CI Kconfig check allow list.
Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
Uses linux style path output instead of windows style to prevent
issues with cmake assuming the slashes are for escape code
sequences.
Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
Both the requests and pyserial Python packages are used by west commands
(west fetch and west build -b esp* respectively) so move them to the
requirements-base.txt file.
Fixes#56215.
Fixes#56224.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Writing '-S foo' is a convenience shorthand for adding snippets at
CMake time without having to add '-- -DSNIPPET=foo' to the west build
command line. It is worth adding a new one-letter command line option
because snippets are anticipated to be a very commonly used feature.
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
This key may appear in a module.yml's 'settings' map, and works
analogously to the already existing 'board_root', 'dts_root', etc.
keys: if set, its value is interpreted as a directory that will be
added to SNIPPET ROOT.
For example, if a module.yml contains:
settings:
snippet_root: foo
Then 'foo', relative to the module's base directory, will be added to
SNIPPET_ROOT. This then means that any snippets in foo/snippets will
be discovered by the build system.
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
Add a new script, snippets.py, which is responsible for searching
SNIPPET_ROOT for snippet definitions, validating them, and informing
the build system about what needs doing as a result.
Use this script in snippets.cmake to:
- validate any discovered snippet.yml files
- error out on undefined snippets
- add a 'snippets' build system target that prints all snippet
names (analogous to 'boards' and 'shields' targets)
- handle any specific build system settings properly,
by include()-ing a file it generates
With this patch, you can define or extend a snippet in a snippet.yml
file anywhere underneath a directory in SNIPPET_ROOT. The snippet.yml
file format has a schema whose initial definition is in a new file,
snippet-schema.yml.
This initial snippet.yml file format supports adding .overlay and
.conf files, like this:
name: foo
append:
DTC_OVERLAY_FILE: foo.overlay
OVERLAY_CONFIG: foo.conf
boards:
myboard:
append:
DTC_OVERLAY_FILE: myboard.overlay
OVERLAY_CONFIG: myboard.conf
/my-regular-expression-over-board-names/:
append:
DTC_OVERLAY_FILE: myregexp.overlay
OVERLAY_CONFIG: myregexp.conf
(Note that since the snippet feature is intended to be extensible, the
same snippet name may appear in multiple files throughout any
directory in SNIPPET_ROOT, with each addition augmenting prior ones.)
This initial syntax aligns with the following snippet design goals:
- extensible: you can add board-specific support for an existing
snippet in another module
- able to combine multiple types of configuration: we can now apply a
.overlay and .conf at the same time
- specializable: this allows you to define settings that only apply
to a selectable set of boards (including with regular expression
support for matching against multiple similar boards that follow
a naming convention)
- DRY: you can use regular expressions to apply the same snippet
settings to multiple boards like this: /(board1|board2|...)/
This patch is not trying to design and implement everything up front.
Additional features can and will be added to the snippet.yml format
over time; using YAML as a format allows us to make
backwards-compatible extensions as needed.
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
This automates much of the drudgery of enumerating changes to
devicetree bindings at release time. Some customizations and
release-specific tweaks to the script will probably always be needed,
but it's a good starting point.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Currently in binary handler and qemu handler we have status `failed`
in case of test timeout, but in device handler we have status `error`
This not only adds inconsistency between handlers, but also prevents
us from usage test retry functionality for the runs on HW.
Fix timeout status by changing it to `failed` instead of `error`.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
WebP images are smaller for similar resolutions and quality while being
compatible with web browsers and other image processing tools.
For higher resolution images, be sure to limit the display size to make
the rendered pages look reasonable.
This was approved by Benjamin Cabé (@kartben) in this discussion thread:
https://github.com/zephyrproject-rtos/zephyr/pull/55488#issuecomment-1461792751
Signed-off-by: Jason Kridner <jkridner@beagleboard.org>
the options --verify and --verify-only where only used when flashing elf
files and were ignored for flashing hex files for example when using
openocd runner.
The command to verify the image is now dependant on the presence of
--verify or --verify-only, and the load command is not executed if
--verify-only is provided.
Signed-off-by: Nicolas VINCENT <nicolas.vincent@vossloh.com>
Add hostlink-awareness to mdb-hw runner. The mdb-nsim and arc-nsim
runners (as well as cmake scripting for nSIM boards) doesn't require
any changes.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
I was using a source file provided by TDK for a sensor driver
and the file was (originally) not encoded as UTF-8. When the read()
function was called, the build would fail but I had no idea why.
This change wrapps the 'read()' call and prints the error with
the file name as context.
Signed-off-by: Yuval Peress <peress@google.com>
The QSPI erase switch is not part of the "firmware" dict, but rather
placed in the overall operation dict.
Fixes#55625.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
In many of test specifications yaml files (testcase.yaml, sample.yaml)
section `platform_allow` is added. This change allows to test some
scenarios on platforms, that are not added yet to platform allow
list (or are not going to be added for some reasons).
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
Fix how DeviceHandler class does 'serial leftover' cleanup
with loss of test output observed on fast console connections,
e.g. over telnet: either 1 sec. of output might be missed,
or up to 1000 bytes/chars, whatever happens first.
Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
There is no need to generate all the *_mrsh.c files for
marshalling syscall arguments when userspace is not enabled.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
We are wrongly claiming no console output, yet console output is there
and the failure is just a regular timeout, i.e. the test did not
complete within allocated time.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
This commit extends the Zephyr module yaml scheme with additional
entries for sysbuild in the build section.
This allows for Zephyr modules to extend the sysbuild infrastructure
by providing additional CMake and Kconfig files to be included in
sysbuild.
The new settings are:
build:
sysbuild-cmake: <path>
sysbuild-kconfig: <path>/<file>
sysbuild-ext: <true>|<false>
sysbuild-kconfig-ext: <true>|<false>
those settings follow the same pattern as the equivalent Zephyr build
settings but are processed by sysbuild.
Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Add support test levels and the ability to assign a specific test to one
or more levels. Using command line options of twister it is then possible
to select a level and just execute the tests included in this level.
Additionally, a test configuration allows definiing level
dependencies and additional inclusion of tests into a specific level if
the test itself does not have this information already.
In the configuration file you can include complete components using
regular expressions and you can specify which test level to import from
the same file, making management of levels easier.
To help with testing outside of upstream CI infrastructure, additional
options are available in the configuration file, which can be hosted
locally. As of now, those options are available:
- Ability to ignore default platforms as defined in board definitions
(Those are mostly emulation platforms used to run tests in upstream
CI)
- Option to specify your own list of default platforms overriding what
upstream defines.
- Ability to override build_onl_all options used in some testscases.
This will treat tests or sample as any other just build for default
platforms you specify in the configuation file or on the command line.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Fixes#37538 by correctly detecting WSL and blocking bossac from running
on that platform, but allowing it to run on Windows native.
Signed-off-by: João Dullius <joaodullius@bpmrep.com.br>
Pass on the tool-specific options to nrfjprog during all operations, and
not only when programming. This is useful when combined with options
like --ip, that allows west flash to be used over the network.
Fixes#55340.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Add tests for the --tool-opt command-line switch, to ensure that if the
user includes additional tool options those get passed on to nrfjprog at
the end of the command-line to override any preceding ones.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Generalize the logic that was previously in nrfjprog.py into a new
nrf_common.py, which can then use specific tool subclasses, one of which
is nrfjprog.py, to implement the actual tool invocation.
This commit lays down the groundwork for the addition of a new backend
for nRF boards, the new nrfutil tool from Nordic. Both nrfjprog and
nrfutil will coexist in the immediate future, but nrfutil is expected to
be the sole tool for everything nRF in due time. This is why the internal
representation of the operations is based in JSON and is using the exact
format that nrfutil expects when using the 'execute-batch' option, that
takes a JSON file with an array of operations to be executed.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Allow for multiple levels of inheritance in the runners in order to make
it possible to share common infrastructure in similar runners.
Also check if the class can be instantiated (i.e. it is not abstract) to
avoid returning abstract base classes.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Twister allows filtering based on kconfigs and dts, however the
filtration is a part of the cmake stage, i.e. the stage has to pass
first and then twister checks if required properties are available.
This causes problems, when the full cmake stage is unable to pass.
If so, other filtration methods had to be used, e.g. platform_allow.
The commit modifies the twister workflow:
if a test defines filters based on kconfig/dts first a cmake package
helper script is used to extract dt and/or kconfigs and if the
conditions are fulfilled it proceeds to a regular cmake stage.
If not, test is skipped.
Co-authored-by: Daniel DeGrasse <daniel.degrasse@nxp.com>
Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
This is an implementation of issue #48334 and adds support for
specifying additional config and device tree overlays through fields in
the testcase.yaml file, which is more readable than having to cram these
in to `extra_args`.
Consider this example which shows the original and new ways to add
config and DT overlays:
```
common:
extra_args: "CONF_FILE=a.conf;b.conf
DTC_OVERLAY_FILE=w.overlay;x.overlay OVERLAY_CONFIG=e.conf
UNRELATED=abc"
tests:
utilities.base64:
extra_conf_files:
- "c.conf"
- "d.conf"
extra_overlay_confs:
- "extra_overlay.conf"
extra_dtc_overlay_files:
- "y.overlay"
- "z.overlay"
extra_configs:
- CONFIG_SAMPLE=y
tags: base64
type: unit
```
The new fields are `extra_conf_files`, `extra_overlay_confs,
`extra_dtc_overlay_files`. Files specified in these sections are appended
to any `CONF_FILE`, `OVERLAY_CONFIG`, or `DTC_OVERLAY_FILE` fields in
`extra_args`, causing the following args being passed in to
`self.run_cmake` at `runner.py:850`:
```
['-DUNRELATED=abc',
'-DCONF_FILE=a.conf;b.conf;c.conf;d.conf',
'-DDTC_OVERLAY_FILE=w.overlay;x.overlay;y.overlay;z.overlay',
'-DOVERLAY_CONFIG=e.conf extra_overlay.conf '
'<build_dir>/twister/testsuite_extra.conf']
```
These fields can be used in the common or scenario-specific YAML
sections and will be merged in order of least to most specific:
1. config files extracted from common's extra_args
2. files listed in common's {extra_conf_files or extra_overlay_confs
or extra_dtc_overlay_files}
3. config files extracted from test scenario's extra_args
4. files listed in test scenario's {extra_conf_files or
extra_overlay_confs or extra_dtc_overlay_files}
Specifying these files in extra_args now triggers a deprecation warning,
as the direct YAML fields are preferred for readability. They will still
function for now but support will be dropped in the future. One
testcase.yaml
(`zephyr/tests/cmake/overlays/var_expansions/testcase.yaml`) is
converted to use the new fields. A follow-up PR will convert the
remaining files to the new format.
Signed-off-by: Tristan Honscheid <honscheid@google.com>
This will make it more convenient to use it from multiple different
places, which we will have a need for in the future.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
We need to have an _include_path attribute to pretty-print
this object from within pdb, for some reason. Add it.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
There's no need for _parse_node() to return the Node instance that is
its sole argument. The only user of the return value is a dead store.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
This helper lets you place a node (really the entire subtree rooted at
that node) elsewhere in the devicetree. This will be useful when
adding system devicetree support, when we'll want to be able to, for
example, move the CPU cluster node selected by the current execution
domain to /cpus.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Introduce a context manager that will save some typing
when dealing with expected exceptions.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Percentage calculation was off due to a recent fix in counting and lines
were not breaking correctly on errors.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
MCUmgr no longer uses DTS node property label to identify
slots for image upload and running application. It now uses DTS node
labels for that purpose.
This commit moves `west sign` from using label properties
to DTS node labels, same way the MCUmgr does.
Signed-off-by: Dominik Ermel <dominik.ermel@nordicsemi.no>
Fixed runtime cleanup option. A typo in a conditional prevented the
cleanup from happened when a test has passed.
Fixes#54240
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
To get consistent and 1 single rundate per CI run, set run_date when
we upload the data. This is optional mostly for CI and to avoid
duplication when we re-run jobs.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
LEGACY_INCLUDE_PATH has been defaulting to "n" and marked as deprecated
in both v3.2 and v3.3. Drop the option entirely for v3.4.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
Initialize 'use_system_gcov', so it is always set before being referenced
in "elif" statement.
Fixes: 1440b19bde ("twister: restore logic to determine which gcov we
should use")
Signed-off-by: Marcin Niestroj <m.niestroj@emb.dev>
Add a new UF2 runner, supporting only the flash capability.
Searches for FAT partitions containing `INFO_UF2.TXT` files,
and can optionally filter on a matching `Board-ID` value in
that file.
Signed-off-by: Peter Johanson <peter@peterjohanson.com>
Stop counting errors as failures and report them as is. This fixes
issues with counting when retrying failures and errors.
We have been adding errors to failures and then double count them when
displaying information on the screen instead of keep an accurate count
in the counter class.
This is now fixed and we do report errors seperately, ie. build errors.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
This commit adds access to the string values without a quotes.
Signed-off-by: Radosław Koppel <r.koppel@k-el.com>
Co-authored-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
We do not want to use gcov from the SDK for native posix and unit tests,
instead we want the system gcov. Restore logic that determines that.
Fixes#54226
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
The YAML parsing will currently return a None-type if no input data
is empty, and the subsequent check for a substring will cause an
exception.
Signed-off-by: Allan Norgaard <allannk@gmail.com>
Move extra warning option from generic twister script into
compiler-dependent config files.
ARCMWDT compiler doesn't support extra warning options ex.
"-Wl,--fatal-warnings". To avoid build fails flag
"disable_warnings_as_errors" should be passed to twister.
This allows all warning messages and make atomatic test useles.
Signed-off-by: Nikolay Agishev <agishev@synopsys.com>
Make the error messages which appear in CI for invalid commit messages
match the text in the documentation which describes the rules for
commit messages exactly. This hopefully makes it easier for people to
read failing CI results and map them to documentation links describing
what to do instead.
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
This commit fixes#54492 where return code 1 is returned
when twister tests fail when twister is called directly and
return code 0 in similar scenario but twister is invoked using
west.
Signed-off-by: Andrey Borisovich <andrey.borisovich@intel.com>
Fix the spelling of Babeltrace library in the warning message displayed
when such a library can not be imported.
Signed-off-by: Gustavo Romero <gustavo.romero@linaro.org>
Tests under quarantine are not treated as errors in the integration
mode. However --quarantine-verify argument allowing to execute just
test under quarantine and skipping others was not considered in
relation to integration mode. Those skips are wrongly threated as
errors. This commit fix this relation and makes thoses skips not
turned to errors.
fixes: #54516
Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
Take an instance that has failed and not whatever was last processed in
the for loop.
Also, change instructions to use `west twister` rather than
`./scripts/twister` to be consistent.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
This should fix issue like #54337 where architecture code changes were
filtering out kernel tests, where those changes are being tested. Kernel
and arch code is tightly coupled and we need to run kernel tests.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
This is a partial revert of commit
2cee5ff519
("scripts: west_commands: runners: remove deprecated options").
I remarked at the time that the removal of the older way of writing
things from the runners themselves seemed gratuitous since they are
easy to continue to support indefinitely and people may have been used
to the old way of doing things. It didn't seem worth the fight to push
for a revert at the time, though.
Since then I've run into real problems that their removal has caused
in the wild and I am convinced that this part of that patch was wrong.
Restore the original, undeprecated forms of these options, but make it
clear in the command line help that they're just obsolete alternative
spellings at this point.
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
Currently we just pass in case of the unsupported jtag adapter
is selected. Fix that by rising ValueError as we do in case of
other incorrect arguments.
While I'm at it add few missing test-cases for mdb west runners.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
Currently, in case of multi-core target or in case jtag adapter
auto-selectio, we pass unexpected empty argument (empty quotes `''`)
to MDB binary due to argument generation error.
Fix that.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
Remove code which looking for `cld` process pid - this functionality
isn't use anymore by twister. And as manual MDB termination is also
fixed now this functionality isn't required for anything now.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
Currently we try to start MDB (MetaWare Debugger) in background
with ignored sigint no matter if we are planing to do debug
session (via `west debug`) or just run something on HW or on
nSIM (via `west flash`)
That cause really bad user experience as after we do
`west debug` or `west flash` we can't terminate the debugger
from the console. Moreover even if we terminate the debugger
process from separate terminal the console stil would be
broken so we need to do `stty sane` in it to make it usable
again.
Fix all that by proper starting the debugger process.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
Change the check_compliance main invocation to pass an explicit argv
list from main to parse_args(). This does not change anything for direct
invocation, but allows wrapping the compliance script and using it as a
library while controlling its behavior by passing the intended flags to
main().
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
If errors are found and tests are failing, print a brief summary showing
the top 10 items failing with instructions on how to reproduce either
with twister or west.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
This will allow something like this:
west build -b qemu_x86 -T samples/hello_world/sample.basic.helloworld
instead of:
cd samples/hello_world
west build -b qemu_x86 -T sample.basic.helloworld
Using this option will allow to take twister output and just pass it to
west to reproduce failures.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Existence of enabled GPIO hog nodes cannot be determined using any of the
existing kconfig functions. Add custom kconfig helper function to determine
whether any GPIO hogs nodes are enabled.
Signed-off-by: Henrik Brix Andersen <hebad@vestas.com>
GPIO hog nodes contain a "gpios" property, but unlike other "*-gpios"
properties, these are not phandle-arrays as they only carry the data part
(e.g. pin, flags) but lack the phandles to the (parent) GPIO controller.
Add special devicetree tooling to handle the "gpios" property of GPIO hog
nodes and generate special devicetree helper macros as if they were phandle
arrays.
Signed-off-by: Henrik Brix Andersen <hebad@vestas.com>
Move away from opensearch and switch to elasticsearch for reporting.
The results will be available at https://kibana.zephyrproject.io/
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Static code analyser (SCA) tools are important in software development.
CMake offers built-in support for some tools, such as cppcheck and
clang-tidy.
Other tools, such as sparse, are not directly supported.
This commit provides a uniform way for users to specify a supported
SCA using `ZEPHYR_SCA_VARIANT=<tool>` which is consistent with how
toolchains are specified.
ZEPHYR_SCA_VARIANT can be set using `-D` or in environment.
Support for an SCA tool is done in `cmake/sca/<tool>/sca.cmake`.
SCA_ROOT can be used to specify additional search paths when looking up
implementation for a tool. SCA_ROOT can also be specified in
`zephyr/module.yml` as setting. This makes it possible to provide SCA
tool implementation as part of a Zephyr module.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
The set assignee script currently exits if there's too many labels
associated with a PR. Change that to just skip the label assignment, but
continue on to assign a maintainer based on the most relevant area.
Tested with:
./scripts/set_assignees.py -v --dry-run -P 52716
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
Enables running CI checks which will now search for initialisation
of argument parser where abbreviations are not disabled.
Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
Adds a pylint checker for ensuring that the argument parser
library is setup with abbreviations disabled.
Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
Disables allowing the python argparse library from automatically
shortening command line arguments, this prevents issues whereby
a new command is added and code that wrongly uses the shortened
command of an existing argument which is the same as the new
command being added will silently change script behaviour.
Signed-off-by: Jamie McCrae <jamie.mccrae@nordicsemi.no>
Support running the compliance check script without writing the output
to any file.
This adds a check for the --output flag to not write to the output file
if the value is empty, and add a second flag for inhibiting the
individual case files.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
When twister is started from make, use the information in MAKEFLAGS to
participate in the job server started by make.
When twister is running standalone, start a jobserver to share with
child processes. If using the kitware version of ninja, then this will
prevent ninja+twister from starting more processes than XX when twister
is run with -jXX.
There is no implementation for windows yet.
Signed-off-by: Jeremy Bettis <jbettis@google.com>
This reverts commit 21eb27c5c0.
This change has been a source of much trouble and ends up preventing us
from tracking test results across tree. It uses the repo name as the
namespace, and that is not always the same and does not have to be
called 'zephyr', depending on where you are running and in which
environment.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Improves the message for tests being excluded using the platform key,
describes the key already found and the platform being used to run the test
in its place.
Signed-off-by: Tom Burdick <thomas.burdick@intel.com>
Implementation ported from TwisterV2.
- quarantine handled by separate module
- multiple yaml allowed from args: --quarantine-list
- scenarios, platforms, architectures keywords in
quarantine yaml are optional, if not given - means take it all
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
Add some logic in blackmagicprobe.py to guess the GDB serial port based
on the OS instead of just trying the flag or /dev/ttyACM0.
The new logic explicitly looks for the well known path for BMP on Linux
(/dev/ttyBmpGdb), so that we can remove it from the board files that
have it set explicitly right now. At the same time it still allows
setting the port from the flag and also checks an environment variable,
so that it can be used for systems with multiple probes.
As a fallback, it looks for a valid serial port, picking the first one
(the probe creates two, the GDB one is the first one), and raise an
exception if nothing is found.
Signed-off-by: Fabio Baltieri <fabio.baltieri@gmail.com>
Recently the blackmagicprobe command for "connect under reset" has been
updated from "connect_srst" to "connect_rst". Update the flag name in
the west blackmagicprobe binding, keep the old one as well for
compatibility with out of tree boards, and send both the old and new
commands to GDB so that it works with both the old and new firmware. GDB
is going to get a "command not supported by this target" for one of the
two and proceed with the other one.
Signed-off-by: Fabio Baltieri <fabio.baltieri@gmail.com>
Some hardware boards are supported in qemu as simulation platforms. If
we are testing with the real hardware, make sure we treat this as a
device and not as a simulator. First priority when initializing a
handler is to check if device testing is requested, then we can handle
the rest without --device-testing.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Update gen_relocate_app.py to use "|" to separate code relocation
directives, and ";" to separate multiple files in a relocation directive.
This will enable multiple files to be passed to zephyr_code_relocate,
as well as multiple files to be passed from a CMake generator expression.
The script will then seperate these files and relocate each according to
the arguments given to zephyr_code_relocate.
Note! This commit will break support for zephyr_code_relocate until
the CMake function is updated
Signed-off-by: Daniel DeGrasse <daniel.degrasse@nxp.com>
Inconsistency between python-devicetree version numbers
may be confusing:
- the last version at PyPI is 0.0.2
- zephyr-rtos/python-devicetree/setup.py sets the version
to 0.0.2 (this is probably the setup file used when uploading
to PyPI)
- zephyr-rtos/zephyr/scripts/dts/python-devicetree/setup.py sets
the version to 0.0.1
This may suggest that the mirror repository, and PyPI, are more
up-to-date than zephyr-rtos/zephyr/scripts/dts/python-devicetree.
Repositories being otherwise mostly identical (1), also bumping
the python-devicetree's version here seems a sane option.
(1) Ignoring the doc directory (only at zephyr-rtos/python-devicetree).
Signed-off-by: Chris Duf <chris@openmarl.org>
This is essentially a revert of PR #46311 "python-devicetree: CI hotfix",
assuming the original issue has been resolved, either upstream in
types-PyYAML or in Zephyr itself.
Tested with types-PyYAML 6.0.12.2 (current version at PyPI):
$ python -m mypy --config-file=tox.ini --package=devicetree
dtlib.py:962: note: By default the bodies of untyped functions [...]
dtlib.py:964: note: By default the bodies of untyped functions [...]
dtlib.py:965: note: By default the bodies of untyped functions [...]
dtlib.py:967: note: By default the bodies of untyped functions [...]
Success: no issues found in 4 source files
The "notes" above are harmless (use of type hinting to define local
variables while mypy won't "check the bodies of untyped functions").
References:
- python-devicetree tox run fails (issue #46286)
- python-devicetree: CI hotfix (PR #46311)
- python-devicetree: CI hotfix (commit f6a6843)
Signed-off-by: Chris Duf <chris@openmarl.org>
Adds an option to inform twister a testsuite should only be built and
run for platforms with unique sets of attributes. This enables
for example keying on unique (arch, simulation) platforms to run the test
suite on.
The most common usage may be test suites configured to run once per
(arch, simulation) pair as being enough. Additional information about
platforms may enable running a test once per hardware IP block or once
per soc family or soc avoiding duplicated effort in building and running
tests when once suffices.
Signed-off-by: Tom Burdick <thomas.burdick@intel.com>
We want to be able to have platform or architecture extra configs
without having to duplicate a whole section of the test specification.
This adds support for namespacing of extra configs, for example:
arch:nios2:CONFIG_SYS_CLOCK_TICKS_PER_SEC=1000
or
platform:qemu_x86:CONFIG_FOO=y
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Previous commit dealing with cleanup (c304db56f1) was not taking
--prep-artifacts-for-testing into consideration. Fix this and make sure
we do cleanup when this option is enabled.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Have boards define what binaries should be kept for device testing. The
purpose of this is to reduce the amount of artifacts and just keep the
binaries needed for flashing a device, reducing the size of the final
payload.
A board can have twister keep multiple binaries needed for testing when
--prep-artifacts-for-testing option is used.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Add SDO block transfer support for the CANopen program download west
runner. SDO block transfers are experimental in the underlying CANopen for
Python.
Reduce the program download chunk size to half the program download buffer
size to avoid blocking IO.
Signed-off-by: Henrik Brix Andersen <hebad@vestas.com>
When re-generating hardware map for DKs with more than 1 serial port (more
enries in the hardware map), then we keep enfo only about the last entry.
For boards like nrf5340 it works, because the last serial port is used,
but for nrf52840 with two ports, the first one is important, so after
re-generating the hardware map it does not work.
The solution is to keep all entries after re-generating the hardware map.
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
Expands kconfigfunctions to include checks for value existence in
an array property by nodelabel, or a chosen's boolean prop value.
Signed-off-by: Stephen Stauts <stephen.stauts@nordicsemi.no>
Add a YAMLLint compliance check that uses the yamllint package to report
linting error on YAML files.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
When scanning for available elfs, twister discards ones with `_pre`
in their name, to not include prebuilds like 'zephyr_pre0.elf`.
However, the scan looks at the whole test name. If _pre is in the name
(e.g. _precision) it would be wrongly discarded ending with twister
reporting error that no elfs were found. This commit makes the discard
to look only at the last part of the name (i.e. not including test name
but only a lowest level, where zephyr.elf etc are located.)
Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
Fix all comments-indentation errors detected by yamllint:
yamllint -f parsable -c .yamllint $( find -regex '.*\.y[a]*ml' ) | \
grep '(comments-indentation)'
This checks that the comment is aligned with the content.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
Fix all thruthy errors detected by yamllint:
yamllint -f parsable -c .yamllint $( find -regex '.*\.y[a]*ml' ) | \
grep '(truthy)'
This only accepts true/false for boolean properties. Seems like python
takes all sort of formats:
https://github.com/yaml/pyyaml/blob/master/lib/yaml/constructor.py#L224-L235
But the current specs only mention "true" or "false"
https://yaml.org/spec/1.2.2/#10212-boolean
Which is the standard yamllint config.
Excluding codeconv and workflow files, as some are using yes/no instead
in the respective documentation.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
When using llvm we need to set the gcov-tool to "llvm-cov gcov" but
the lcov tool is incapable of passing arguments to the gcov-tool. i.e.
the following cannot work:
$ lcov --gcov-tool "llvm-cov gcov" ...
Instead, create a symlink to llvm-cov prefixed as `gcov` which by the
documentation of llvm-cov will alias to `llvm-cov gcov` subcommand.
Signed-off-by: Yuval Peress <peress@google.com>
Add a compliance check that tries to load MAINTAINERS.yml with
get_maintainer.Maintainers() if it's been modified by the CL, and fail
compliance if it fails to be parsed.
Example output:
```
ERROR : Test MaintainersFormat failed:
Error parsing MAINTAINERS.yml: MAINTAINERS.yml: YAML error: while
scanning a simple key
in "MAINTAINERS.yml", line 976, column 1
could not find expected ':'
in "MAINTAINERS.yml", line 977, column 3
```
```
ERROR : Test MaintainersFormat failed:
Error parsing MAINTAINERS.yml: MAINTAINERS.yml: glob pattern
'drivers/regulator' in 'files' in area 'Drivers: Regulators' matches a
directory, but has no trailing '/'
```
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
Make sure that this runner can handle multiple --tool-opt arguments,
and that the options specified for pyocd are passed to that tool in
order.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Change the bindings file filter to check for "dts/bindings/" as a
substring rather than prefix. This makes the check catch files in
subdirectories as well (for sample tests and samples), and makes the
check usable on external projects where the bindings may not live in the
project root.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
Move the file iteration loops of the bindings compliance check in the
callers, so that if we add more checks we don't have iterate on each one
of them and a check works on a single file.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
Add a check to validate and limit the size of images in the repository,
setting stricter limit for files in boards/.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
Git normally only works on files, but can include directories if there's
any submodule in the repository. This is currently checked in
filter_py(), move the check into get_files() so it does not have to be
copied around.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
DevicetreeBindingsCheck is missing the splitlines() call to process the
get_files() output, hence cycling through each character of the output
rather than each file, which causes the check to never run on anything.
Fix it by moving the splitline call into get_files() itself, since every
caller is going to use it anyway.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
This commit expands Twister's CLI so that arguments passed after
a double dash (`--`) get forwarded to the ztest test executable when
run. When built for native_posix, ztest provides a useful CLI that
allows filtering individual suites and tests, adjusting timing settings,
and controlling the shuffling seed. Currently there is no easy way to
use these (with the exception of `--seed`) without needing to dig out
the resulting `zephyr.exe` binary from the build dir and invoke it
manually.
### Examples
Run a specific ztest suite only (useful when writing tests and you don't
want to run a long testcase in its entirety)
```
$ scripts/twister \
-p native_posix \
-s
zephyr/tests/lib/cmsis_dsp/complexmath/libraries.cmsis_dsp.complexmath \
-- -test=complexmath_q31::*
```
Unrecognized arguments that precede the double dash will result in an
error message:
```
$ scripts/twister \
-p native_posix \
--foobar 123 \
-- -test=basic_math_q7::*
...
Unrecognized arguments found: '--foobar 123'. Use -- to delineate extra
arguments for test binary or pass -h for help.
```
Signed-off-by: Tristan Honscheid <honscheid@google.com>
Zephyr's coding guidelines require braces on every code block body.
Enable InsertBraces option so clang-format automatically adds these
braces.
Signed-off-by: Keith Short <keithshort@google.com>
This commit adds the requirement for the `TIMER_RANDOM_GENERATOR`
Kconfig to be set to `n` because it enables timer-based pseudo-random
number generator, which is only intended to be used for testing
purposes.
Signed-off-by: Stephanos Ioannidis <stephanos.ioannidis@nordicsemi.no>
This commit updates the KconfigCheck to add the Kconfig symbols from
both "config" and "choice" symbol lists to the defined symbol list, as
opposed to the "config" list only, in order to prevent any references
to the choice symbols outside the Kconfig files (e.g. in documentation)
from being reported as undefined.
Signed-off-by: Stephanos Ioannidis <stephanos.ioannidis@nordicsemi.no>
This PR clarifies the violation messages emitted by gitlint when
checking the commit message. For example:
* Before: `43: UC4 Line exceeds max length (N>75): "..."`
* After: `43: UC4 Commit line exceeds max length (N>75): "..."`
This makes it easier to identify the source of the error since there is
currently no additional context besides the error code UC*. I recently
pushed a commit that had some sample code as part of the commit body
that exceeded the lenght limit, and thought the error was referring to
one of my source files based on the line it showed. (feel free to laugh
at me, but let's make it better for the next person)
Signed-off-by: Tristan Honscheid <honscheid@google.com>
Generates XML file containing VIF policies by reading the device tree
using EDT.pickle generated during build
This script writes a subset of general and sink-pdo VIF policies in
output file
This script gets invoked during build if enabled through kconfig
The generated XML containing USB-C VIF policies could be used by
USB PD/Type-C analysers/testers to understand USB-C properties and
perform tests accordingly
Signed-off-by: Madhurima Paruchuri <mparuchuri@google.com>
path for posix was not being matched correctly due to a missing /. So
some posix tests were excluded and not excercised.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
The current docs are incorrect as comma separated paths
are not working. Providing a separate --exclude argement
per path does however work.
Signed-off-by: Håkon Øye Amundsen <haakon.amundsen@nordicsemi.no>
Ignore unit tests and EFI binaries when calculating footprints with
--show-footprint, or we will get a build error.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
The --snr (nrfjprog) --id (jlink) and --board-id (pyocd) options were
deprecated a long time ago in favor of --dev-id. It is time to remove
them.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
This commit calculate memory footprint from build.log and
proposes an alternative approach to #2465.
Signed-off-by: Katarzyna Giądła <katarzyna.giadla@nordicsemi.no>
non-simulation platforms set key to 'na', we need to check for that
instead of just checking for existance of key.
Fixes#52595
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Both get_defined_syms and UNDEF_KCONFIG_WHITELIST are used exclusively
by the Kconfig check, so move them to the class.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Instead of globally resolving ZEPHYR_BASE, move it to _main() so we can
catch potential exceptions when doing so and for consistency with the
rest of globals, like GIT_TOP. This also includes documenting a new
"<zephyr-base>" path hint for classes.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
The global variable ZEPHYR_BASE is always set in the script, so no need
to check for its existence.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Some errors are attributable to a specific file, but not a specific line
within that file. Allow for formatted errors that refer to a file but
not a line.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Many tests invoke `git diff` independently with very similar parameters
to obtain a filtered list of files.
Consolidate those into a new `get_files()` function.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
The workaround for old pytest bug
https://github.com/PyCQA/pylint/issues/2906
is not required anymore after more than 3 years, so remove it.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
If a default platform is not able to run for whatever reason, treat it
as any other platform. Default platforms shall ALWAYS be runnable in CI,
or else we are just re-building for no good reason.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Provide the information as part of the board yaml file, this way we will
know what binary to check for.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
the ser.in_wait sometime meet issue like serial handler error.
This is because the serial port is in reset status.
add final protection to retry later
error log:
Traceback (most recent call last):
File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
self.run()
File "/usr/lib/python3.10/threading.py", line 953, in run
self._target(*self._args, **self._kwargs)
File "zephyr/scripts/pylib/twister/twisterlib/handlers.py", line 345,
in monitor_serial
if not ser.in_waiting:
File "~/python3.10/site-packages/serial/serialposix.py", line 549,
in in_waiting
s = fcntl.ioctl(self.fd, TIOCINQ, TIOCM_zero_str)
OSError: [Errno 9] Bad file descriptor
Signed-off-by: Hake Huang <hake.huang@oss.nxp.com>
There were many instances in the code where f-strings were more
appropriate or fitting, change those to use them.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
As per the current Python documentation, use subprocess.run() since the
other, older methods are not recommended anymore.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
The checkpatch regex matched too greedily, so that descriptions with ":"
would match too much, example:
-:20: ERROR:SPACING: spaces required around that '=' (ctx:VxW)
Would match "SPACING: spaces required around that '=' (ctx" instead of
just "SPACING".
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
This is a follow-up to commit b1a3ce4016.
`parse_input_string()` needs to use `:NOCOPY:`, not `:NOCOPY`, when
partitioning input lines, otherwise, when a line contains the NOCOPY
flag, the file name returned by the function starts with `:` and the
file cannot be then found by the script.
Such problem can be observed in the code_relocation_nocopy sample,
which without this fix does not actually relocate code from ext_code.c.
Signed-off-by: Andrzej Głąbek <andrzej.glabek@nordicsemi.no>
Add a compliance check that fails if the diff is adding any binary file
to prevent committing them by mistake.
Only check for added files and explicitly allow image files in doc/ and
boards/.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
Check names must not contain spaces, since they are passed via `-m` in
the command-line.
Also update .gitignore and compliance.yml.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Add ZCBOR to the requirements-extras.txt and require at least version
0.6.0 which is the current version of the ZCBOR C library.
Signed-off-by: Veijo Pesonen <veijo.pesonen@nordicsemi.no>
add 2 choices for '--runtime-artifact-cleanup' option,
'passing' to delete artifacts of passing tests,
'all' to delete artifacts of both passing tests
and failing tests
keep testcase_extra.config, it contains some extra Kconfig
needed for the tests and they is only generated first time
when run retry failed
Signed-off-by: Cong Nguyen Huu <cong.nguyenhuu@nxp.com>
Add dt_node_has_compat kconfig function, to check if a node path has a
given compatible string while parsing kconfig files
Signed-off-by: Daniel DeGrasse <daniel.degrasse@nxp.com>
Add dt_node_parent kconfig function, to get parent path for a given
devicetree path when parsing kconfig files
Signed-off-by: Daniel DeGrasse <daniel.degrasse@nxp.com>
Add `DevicetreeBindingsCheck` and `required_false_check()`, which
checks to make sure that no new Devicetree bindings add the
redundant `required: false` property.
Signed-off-by: Chris Friedt <cfriedt@meta.com>
Add test coverage for the child-binding include feature. It includes
verification of included properties as well as usage of allow/blocklist.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
checkpatch identifies a label only when a terminating colon
immediately follows an identifier.
Bitfield definitions can appear to be labels so ignore any
spaces between the identifier terminating colon and any digit
that may be used to define a bitfield length.
Miscellanea:
o Improve the initial checkpatch comment
o Use the more typical '&&' instead of 'and'
o Require the initial label character to be a non-digit
(Can't use $Ident here because $Ident allows ## concatenation)
o Use $sline instead of $line to ignore comments
o Use '$sline !~ /.../' instead of '!($line =~ /.../)'
Link: https://lkml.kernel.org/r/b54d673e7cde7de5de0c9ba4dd57dd0858580ca4.camel@perches.com
Signed-off-by: Joe Perches <joe@perches.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Manikishan Ghantasala <manikishanghantasala@gmail.com>
Cc: Alex Elder <elder@ieee.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Stephanos Ioannidis <stephanos.ioannidis@nordicsemi.no>
gen_image_info.py was generating invalid include guard. Properly strip
all special characters from filename to generate include guard #ifdef
Signed-off-by: Daniel DeGrasse <daniel.degrasse@nxp.com>
Convert domains.py to use basic python logging module implementation to
remove west logging dependency. This removes hard dependency on west from
twister.
Fixes#51731
Signed-off-by: Daniel DeGrasse <daniel.degrasse@nxp.com>
Add test cases verifing that test id for external and module tests is
not --testsuite-root dependant.
Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
With the patch a relative path between a test and west's project top
directory is always calculated instead of being relative to zephyr
base. The test names and results structur change by addition of one
level. This way the names/structure conventon for tests from zephyr and
modules is alligned.
For external tests ".." parts are replaced with "external".
This allows to maintain unified folder structure also for results of
external tests.
Without this patch the folder/name allignment is lost for
external and module test becaming depandant on a way how
twister was called.
.
Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
From docs "If a component is an absolute path, all previous
components are thrown away and joining continues from the
absolute path component." This commit fixes the args and
corrects the expected value.
https://docs.python.org/3/library/os.path.html#os.path.join
Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
junitparser has a bug that prevents it from working correctly when lxml
is not installed.
See this GH issue for more information:
https://github.com/weiwei/junitparser/issues/99
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Parse the output generated by pylint and store it as FmtdFailure
instances, so that they can be annotated later.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
The current recommended API call for subprocess instantiation is run(),
so replace the low-level Popen and communicate() with it.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Format the JUnit failure according to the standard:
https://www.ibm.com/docs/en/developer-for-zos/14.2?topic=formats-junit-xml-format
This also includes a new FmtdFailure class, which will be used in
subsequent commits to implement GitHub-compatible annotations. A list of
FmtdFailure instances is kept in the ComplianceTest instance because
the TestCase instance cannot restore classes derived from Result that
the junitparser module doesn't know about.
Use the new function to format checkpatch errors.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Update the configuration files for the footprint
tracking for the nrf5340dk_nrf5340_cpunet iso-broadcast
and iso-receive, after the files were recently renamed.
Signed-off-by: Emil Gydesen <emil.gydesen@nordicsemi.no>
Fixes for the following issues that a new version of mypy started
reporting recently:
runners/trace32.py:34: error: Incompatible default for argument
"startup_args" (default has type "None", argument has type
"List[str]") [assignment]
runners/trace32.py:137: error: Incompatible default for argument
"cfg" (default has type "None", argument has type "Path") [assignment]
runners/trace32.py:34: note: PEP 484 prohibits implicit Optional.
Accordingly, mypy has changed its default to no_implicit_optional=True
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
KconfigBasic should not be inheriting from both KconfigCheck and
ComplianceTest. This was a workaround to find all inheritors of
ComplianceTest, so fix the inheritors search instead.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Instead of arbitrarily storing the string provided by the caller in
either the message attribute or the element text, let the reporter
choose instead.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Instead of accumulating failure information in a single Result instance,
located at index 0, generate one Result sub-class instance per
failure/error/skip as it is intended in the original library.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
The junitparser Python module now comes with a setter and getter for
Result.text, use them instead of peeking into the underlying XML.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Add support for userspace with RTIO by making rtio and rtio_iodev
k_objects. As well as adding three syscalls for copying in submissions,
copying out completions, and starting tasks with submit.
For the small devices Zephyr typically runs on one of the most important
attributes tends to be low memory usage. To maintain the low footprint of
RTIO and its current executor implementations the rings are not shared with
userspace. Sharing the rings it turns out would require copying submissions
before working with them to avoid TOCTOU issues.
The API could still support shared rings in the future so that a
kernel thread could directly poll, copy, verify, and start the submitted
work. This would require a third executor implementation that maintains its
own copy of submissions similiar to how io_uring in Linux works.
Signed-off-by: Tom Burdick <thomas.burdick@intel.com>
At some point in the past, we had to suppress a couple of false
positive pylint warnings to pass CI. But now the linter seems to have
figured out its original mistake and is complaining about a useless
supression. Sigh. Play along.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
The standard library copy module allows you to implement shallow and
deep copies of objects. See its documentation for more details on
these terms.
Implementing copy.deepcopy() support for DT objects will allow us to
"clone" devicetree objects in other classes. This in turn will enable
new features, such as native system devicetree support, within the
python-devicetree.
It is also a pure feature extension which can't harm anything and is
therefore safe to merge now, even if system devicetree is never
adopted in Zephyr.
Note that we are making use of the move from OrderedDict to regular
dict to make this implementation more convenient.
See https://github.com/devicetree-org/lopper/ for more information on
system devicetree. We want to add system devicetree support to dtlib
because it seems to be a useful way to model modern, heterogeneous
SoCs than traditional devicetree, which can really only model a single
CPU "cluster" within such an SoC.
In order to create 'regular' devicetrees from a system devicetree, we
will want a programming interface that does the following:
1. parse the system devicetree
2. receive the desired transformations on it
3. perform the desired transformations to make
a 'regular' devicetree
Step 3 can be done as a destructive modification on an object-oriented
representation of a system devicetree, and that's the approach we will
take in python-devicetree. It will therefore be convenient to have an
efficient deepcopy implementation to be able to preserve the original
system devicetree and the derived regular devicetree in memory in the
same python process.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Regular dicts are insertion-ordered since CPython 3.6 and Python 3.7.
Zephyr now requires Python 3.8, so it should be OK to replace
OrderedDict with regular dict now. This results in less typing and
more readable object representations.
A nitpicker could argue that this is a functional change, since if a
user is doing 'assert isinstance(node.props, OrderedDict)', that will
fail now, but:
1. nobody is doing something like that in the zephyr tree
2. that would be a silly thing to do
3. we don't currently make any API stability guarantees
for this module right now anyway
so it should be fine.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Refactor the file parsing methods for readability by moving the
_parse_header() and _parse_memreserves() calls from _parse_dt() to
_parse_file(). The header and memreserves are not part of the 'tree'
part of the devicetree; now that we have a dedicated _parse_file()
helper, it makes more sense to me to have them there.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Holy overloaded technical terms, Batman.
Here, 'property' and 'type' each mean two different things, which
we can distinguish like this:
- Property (capital P): dtlib.Property class, represents
a property in a devicetree node
- @property: a Python property
- type(): an "@property" in the Property class, that returns
a dtlib.Type value
- Type (capital T): dtlib.Type class, represents the devicetree
type of a Property value (dtlib.Type.BYTES, etc.)
The type() @property in the Property class currently has an 'int' as
its Python return type annotation. It really returns a dtlib.Type,
which is an int (since it's an IntEnum), but that's not the same thing
as an int.
Change this to Type to be clear that not just any int can be returned
by this @property.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Make attribute initialization order match the order that attributes
appear within the class docstring. Move the 'type' property definition
up by the constructor to make it more obvious that this 'attribute' is
a (Python) property. This is for readability.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Reorder attribute initialization to match the order that attributes
appear in the class level docstring. This is for readability.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Initialize all the public API interface related attributes within the
constructor instead of scattering them throughout the implementation
of the class, and make sure they all have type annotations.
Move all the parsing code away from the init routines and public API
down to the main parsing block.
This is for readability and paves the way for later changes that
affect the way initialization happens.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Version is not enough, we need the run date but also the commit date, to
ease reporting and use in dashboards.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
In some scenarios we might end up with a null version value in the json
file, if the version can't be determined, set it to Unkown instead.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Some linker scripts also associate memory regions with program headers
(PHDRS [1]). This patch adds support on gen_relocate_app.py to also
place a phdr alongside the memory region.
To keep things simple (and more natural), the format is basically the
same used in linker scripts - an space followed by ':<phdr_name>', like:
SRAM2\ :phdr0:COPY:/home/xyz/zephyr/samples/hello_world/src/main.c
(Note the escape char before the space.)
[1] https://sourceware.org/binutils/docs/ld/PHDRS.html
Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
On push, upload test results to opensearch for analysis and reporting.
Goal is to use this data to understand test coverage better and use this
services for all test reporting, also for results coming from testing on
hardware.
Opensearch is currenly being used for evaluation, we are considering
the switch to elasticsearch later.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
When twister is set to retry any failures, count the number of retries
and record the number in the json file. This will help us identify
unstable tests or tests requiring attention.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Modify description frame to wrap the text so the window
doesn't have to be scrolled horizontally.
Signed-off-by: Filip Zajdel <filip.zajdel@nordicsemi.no>
Both CONFIG_BIG_ENDIAN and CONFIG_LITTLE_ENDIAN can be used in the code,
so reflect that in the error message in checkpatch.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
in some builds there is need to pass different value for -i parameter
and current code will override it to 3
Also in rimage i parameter is by default set to 3
Signed-off-by: Adrian Bonislawski <adrian.bonislawski@intel.com>
Update twister to support running QEMU platforms with sysbuild, by parsing
domains.yaml and executing the "run" target of the default application.
This will allow twister to test QEMU targets with sysbuild. It is assumed
that QEMU targets will add any external images they need in the build
phase
Signed-off-by: Daniel DeGrasse <daniel.degrasse@nxp.com>
Since twister can make use of domains.py as well, refactor west's use
this file so that domains.py can be moved to a generic library folder.
Signed-off-by: Daniel DeGrasse <daniel.degrasse@nxp.com>
When running twister -vv many blank lines would be logged.
Fix handing of empty lines from the reader thread. Real blank likes will
be b"\n" not b"".
Change open call to use with, and logger to use %s to fix pylint
warnings.
Signed-off-by: Jeremy Bettis <jbettis@google.com>
Currently, it is not possible to use a normal flash command to flash two
hex files that contain data in the UICR region, even though these
data don't conflict with each other.
However, that is not a technical limitation of the device, but of the
nrfjprog runner as it fails as soon as the hex file contained UICR data
and there was no --force/--erase/--recover used.
That approach was used in the past as program operation was used
without --verify, but as this option was added a while ago it is an
unnecessary limitation.
If the UICR wasn't erased before programming, the verify of the program
operation will now fail, so giving the user a warning in this case that
the hex files contain data in the UICR will still give the user a hint
of what might have caused the failing verify.
Additionally, it enables partial programming of the UICR without the
use of the --force option.
Signed-off-by: Markus Swarowsky <markus.swarowsky@nordicsemi.no>
Modify the regex used to merge overlays specified for
test samples in twister tests.
Overlays without any prefixes will be merged and passed to cmake.
Overlays with prefixes won't be merged but passed to cmake as they are.
Signed-off-by: Piotr Węgliński <piotr.weglinski@nordicsemi.no>
When there are more than 15 reviewers, the `set_assignee` script was
adding the reviewers more or less on a random basis because the input
set was arbitrarily ordered (thanks to how Python "set" works), and the
attempt to add any reviewers beyond the count of 15 results in the
previoulsy added reviewer being removed.
This commit updates the `set_assignee` script such that:
1. The collaborator list (input for generating the reviewer list) is
ordered by the area match, such that the collaborators of the most
relevant area come first.
2. The reviewers of the relevant areas are added first, until the total
reviewer count is 15. The script does not attempt to add more than
15 reviewers because that can result in the previously added
reviewers being removed from the list.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
The logic to drain the leftover logs of previous test should be
put before the monitor_serial(). It was misplaced when re-adding
this logic back.
Signed-off-by: Ming Shao <ming.shao@intel.com>
This sample is intended to show users how they can create a sample which
has dedicated configuration files for extra images included in a build.
MCUboot is used as example on how its default can be adjusted and
MCUboot itself automatically be included in the build when using
sysbuild.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
This patch introduces a West runner for flashing and debugging with
Lauterbach TRACE32 debuggers. The runner consists of a wrapper around
TRACE32 software, and allows a Zephyr board to execute a custom start-up
script (Practice Script) for the different commands supported, including
the ability to pass extra arguments from CMake. Is up to the board using
this runner to define the actions performed on each command.
The `debug` command launches TRACE32 GUI to allow debug the Zephyr
application, while the `flash` command hides the GUI and executes the
start-up script in a background process.
Signed-off-by: Manuel Arguelles <manuel.arguelles@nxp.com>
Calling Binding.__repr__() when the attribute Binding.path is None
would raise TypeError: expected str, bytes or os.PathLike object,
not NoneType.
Known bindings that may not have a path (Binding.path is None)
include bindings for properties such as 'compatible', 'reg', 'status'.
Signed-off-by: Chris Duf <chris@openmarl.org>
Co-authored-by: Gerard Marull-Paretas <gerard@teslabs.com>
Attempting to access the property Property.description
when Property.spec.description is None would raise
AttributeError: 'NoneType' object has no attribute 'strip'.
Known properties that may not have a description
(Property.spec.description is None):
- 'compatible' for nodes such as / /soc /soc/timer@e000e010 /leds /pwmleds
- 'reg' for nodes such as /soc/timer@e000e010
- 'status' for nodes such as /soc/timer@e000e010
- 'gpios' for nodes such as /leds/led_0 /buttons/button_0
- 'pwms' for nodes such as /pwmleds/pwm_led_0
This patch checks the PropertySpec.description attribute before calling
strip(): will return None, and not raise AttributeError.
Signed-off-by: Chris Duf <chris@openmarl.org>
Co-authored-by: Gerard Marull-Paretas <gerard@teslabs.com>
Add a script to extract HTTP status values and format them
in a way that is both human readable and machine parseable.
Each line of output is of the form:
```
HTTP_{key}_{upper_val} = {key}, /**< val */
```
Signed-off-by: Christopher Friedt <cfriedt@meta.com>
Attempting to access the Binding.description property
when the description is unavailable would raise KeyError: 'description'.
Known bindings that won't define a 'description' key in the
Binding.raw dictionary include the 'compatible' property's binding
of nodes such as /, /soc, /leds or /pwmleds.
Note that this may also occur when a proper YAML
binding file is available (e.g. pwmleds.yaml).
This patch simply substitutes the Binding.raw dictionary indexing
with the get() function: will return None and not raise KeyError.
Signed-off-by: Chris Duf <chris@openmarl.org>
Instead of hardcoding alignment size for pass 2 device handles, use
Z_DECL_ALIGN. This makes sure gen_handles.py is always in sync with the
type defined in device.h. The build assert in device.h can be removed as
a result, since we do not hardcode handles size anywhere else.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
In BinaryHandler we process input from simulator with decode('utf-8')
to convert it to string. decode has strict error handling by default -
so it raises UnicodeDecodeError exception if it can't decode input
binary sequence.
So if test start to print some junk to uart console we get
UnicodeDecodeError exception which cause the whole twister crash.
To fix that switch decode to less strict error handling when it
just replace undecoded binary sequence with unicode replacement
character.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: Evgeniy Paltsev <PaltsevEvgeniy@gmail.com>
We need to give gdb slash-separated paths to ELF files on Windows.
Just make sure we always have slash-separated paths. We use this
technique in various places around the tree.
Fixes: #50789
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Instead of 'reset halt', default the command to reset and halt the target
to 'reset init'.
OpenOCD strictly requires 'reset init' to halt and prepare the device
before flash programming.
See https://openocd.org/doc/html/Flash-Commands.html chapter
"12.2 Preparing a Target before Flash Programming."
For targets implementing the init hook, it will allows to perform some
target specific init improvements (such as speed flash improvements).
Targets not implementing this hook will end up executing the usual and
previous 'halt'.
Signed-off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
Follow up to 5b5aa6ebba adding model name
and existence macros for all compatibles of a node that match an entry
in vendor prefixes.
Signed-off-by: Maureen Helm <maureen.helm@intel.com>
Although 8644b333cd added support for some
missing commands an options, the bash completion file was still in bad
need for some cleanup, fixes and additions.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
On windows, some board manufacturers have name that is "Microsoft"
instead of real its name. Need add it to manufacturer list.
Product information is missing. we can assign it "unknown" and
then add missing informations by hand.
Signed-off-by: Cong Nguyen Huu <cong.nguyenhuu@nxp.com>
Currently, support for Twister on Windows is only build.
The only thing that needs to be ported is that select()
cannot wait on file descriptors on Windows. Therefore
the serial monitor function needs to be reworked to
support both OSes.
Signed-off-by: Cong Nguyen Huu <cong.nguyenhuu@nxp.com>
Add support for building with sysbuild using twister, via the "sysbuild"
yaml property in testsuites. This will currently disable Kconfig and
devicetree filtering.
Signed-off-by: Daniel DeGrasse <daniel.degrasse@nxp.com>
The log refers to Platform taking precedence but the loop handles
Documentation as well. Move it under the proper section, drop "Drivers"
as that is not handle in any special way.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
The script right now assumes that all the areas have a maintainer and
that the second area in particular has one in case of Documentation,
that results in a ValueError when it's not the case. Handle that by
checking the lists before using them.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
Add a new set of helpers for expanding property entries with a
separator. These macros complement DT(_INST)FOREACH_PROP_ELEM(_VARGS) by
adding the capability to expand with a custom separator between property
entries. This allows, in some cases, to re-use existing macros (e.g.
DT_PROP_BY_IDX) without creating an auxiliary macro that just appends a
separator. Example:
```dts
n: node {
...
my-gpios = <&gpioa 0 GPIO_ACTIVE_HIGH>,
<&gpiob 1 GPIO_ACTIVE_HIGH>;
};
```
Before:
```c
#define GPIO_DT_SPEC_BY_IDX_AND_COMMA(node_id, prop, idx) \
GPIO_DT_SPEC_BY_IDX(node_id, prop, idx),
struct gpio_dt_spec specs[] = {
DT_FOREACH_PROP_ELEM(DT_NODELABEL(n), my_gpios,
GPIO_DT_SPEC_BY_IDX_AND_COMMA)
};
```
After:
```c
struct gpio_dt_spec specs[] = {
DT_FOREACH_PROP_ELEM_SEP(DT_NODELABEL(n), my_gpios,
GPIO_DT_SPEC_BY_IDX, (,))
};
```
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
Due to the diverse coding styles and lack of preprocessing when
scanning for test cases, there were many unintended combinations
of yaml test scenarios and C test functions with the regex-based
test case discovery, which caused an inaccurate test plan and test
result stats.
As the new ztest fx is used, the test cases of a test instance can
be accurately determined via zephyr.symbols file.
Because the zephyr.symbols file is not available until after build,
test cases determination is placed right after the build stage of
the pipeline and before the runtime test collection.
For those test instances that don't go through the build stage,
such as those with "Skip filtered" (statically filtered) reason
before entering the pipeline, they are not affected.
This patch also adjust the stats logic because the zephyr.symbols
file is generated after build. So ExecutionCounter update is split
and some must be postponed until test execution pipeline is completed.
Some concepts:
A test instance = a yaml scenario + a platform
"Test instance" and "test config" are synonyms in twister, unfortunately
excessive IMHO...
A test instance can be filtered or skipped in 3 ways.
Let's define "runtime" as "after entering the execution pipeline".
1) statically filtered (before runtime)
Such test instance is filtered by analyzing the yaml and never
enters the execution pipeline.
2) cmake filtered (runtime)
Such test instance enters pipeline and is filtered at cmake stage.
3) build skipped (also runtime)
Such test instance enters pipeline and is skipped at build stage.
All the test instances that enter the execution pipeline will go
through the report stage, where ExecutionCounter is further updated.
The meaning of the fields of ExecutionCounter are:
.skipped_configs = static filtered + cmake filtered + build skipped
.skipped_runtime = cmake filtered + build skipped
.skipped_filter = static filtered
.done = instances that enter the execution pipeline
.passed = instances that actually executed and passed
Definition of the overall stats:
total_complete = .done + .skipped_filter
total = yaml test scenario * applicable platforms
complete percentage = total_complete / total
pass rate = passed / (total - skipped_configs)
Signed-off-by: Ming Shao <ming.shao@intel.com>
This commit removes the stale reference to `k_mem_pool_area` for the
memory pool API, which was removed in the v2.5.0 release.
Signed-off-by: Stephanos Ioannidis <stephanos.ioannidis@nordicsemi.no>
Strip trailing and leading whitespaces from but titles so that they can
be used directly in the release notes.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
Make sure listed titles have no trailing/leading space (they often do
and right now they have to be cleaned manually before going into the
release notes file).
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
Sphinx 5.2.0.post0 release has a known compatibility issue with the
`sphinx_rtd_theme` (see the issue readthedocs/sphinx_rtd_theme#1343).
Revert this commit once this compatibility issue is resolved.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
This logic was added before but seems to be missed during the
twister refactoring.
Some tests can cause serial leftover logs buffered somewhere.
Such leftover can interfere with the next test case because the
serial log monitoring thread is started before a board is flashed.
And the monitoring thread can be fooled by such leftover logs and
make incorrect judgement of the test result.
A simple ser.flush() is not enough to eliminate such leftovers.
So add explicit readline() to drain such logs which ensures a clean
serial context for the case that follows.
An example from reel board captured with this patch:
leftover log of previous test: b'.287 seconds\r\n'
leftover log of previous test: b' - SKIP - [...test_coredump_backend]...
leftover log of previous test: b'\r\n'
leftover log of previous test: b'------ TESTSUITE SUMMARY END ------\r\n'
leftover log of previous test: b'\r\n'
leftover log of previous test: b'=====================================...
leftover log of previous test: b'RunID: 4e93757ad...53dcab9f0f5c6\r\n'
leftover log of previous test: b'PROJECT EXECUTION SUCCESSFUL\r\n
Signed-off-by: Ming Shao <ming.shao@intel.com>
Twister detects FLASH overflow and skips tests that trigger that
condition by default, but sometimes images are just on the flash size
limit and then overflows at a later stage when imgtool adds a trailer,
which gets detected as a build fail and fails the run.
This detects the imgtool flash overflow together with the normal build
ones, causing the build to be skipped on imgtool flash overflow as well.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
A workaround to exclude boards from twister. Other solutions do not work,
setting twister fields to false in the board yaml file is not enough,
given that this board has many associated samples and tests, so it can't
be disabled on its own.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Attempts to define two nodes with the same name within a single set of
curly brackets should fail.
For example, this is invalid DTS according to dtc:
/ { foo {}; foo {}; };
By contrast, this is valid since the node named 'foo' appears twice in
two different sets of curly brackets:
/ { foo {}; };
/ { foo {}; };
Zephyr's dtlib currently does not error out on the invalid condition.
Now that Zephyr itself has been updated to not include such nodes (to
the best of my ability), we can fix this divergence from current dtc
behavior and add a regression test in dtlib.
Fixes: #49590
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
With revision 7.70d, Segger changed the command line interface of the
jlink client, requiring some flash specific information, such as
the base address and the type of memory, e.g. QSPI.
This commit adds an optional argument to the West jlink runner,
'--loader' which passes the information to the jlink client.
This change is backward compatible with versions of jlink older
than v7.70d which introduces the CLI change.
Fixes: #50327
Signed-off-by: Yves Vandervennet <yves.vandervennet@nxp.com>
Update size functions within kconfig helpers to support converting
values stored in kilobits, mebibits, or gibibits to kilobytes,
mebibytes, and gibibytes via use of different size arguments.
Signed-off-by: Daniel DeGrasse <daniel.degrasse@nxp.com>
When using west runners with multi domains we want to warn users when
they user runner args on multi domain builds.
Therefore fix the `>1` to the correct `>0` which ensures the warning
is printed in all expected cases.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
For a single bus that supports multiple protocols, e.g. I3C and I2C,
the single value "bus:" setting is no longer sufficient, as a I3C bus
cannot be matched to a device having "on-bus: I2C". This commit
extends the "bus:" setting so that it can accept a list of values.
This change allows corresponding devicetree macros to be generated
so that DT_ON_BUS() can work properly in this scenario.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
The commit c3620c8a8e changed the
`list_boards` script such that it no longer searches the boards from
the `ZEPHYR_BASE` path by default.
This commit updates the `test_plan` script to invoke the `list_boards`
script with the `ZEPHYR_BASE` as a search path.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
Remove a yaml monkeypatch. It is no longer needed since we support 3.6
or later on Zephyr v2.7 LTS and 3.8 or later on what will become v3.2.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Application log level is not detected correctly as the option is defined
using a template, so it can't be grepped.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
As part of the overall work on improving west integration with Zephyr,
add support for zsh completion in the `west completion` command.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
The script will now automatically migrate all <zephyr.h> or
<zephyr/zephyr.h> (for already migrated projects) to <zephyr/kernel.h>.
Signed-off-by: Gerard Marull-Paretas <gerard@teslabs.com>
As of today <zephyr/zephyr.h> is 100% equivalent to <zephyr/kernel.h>.
This patch proposes to then include <zephyr/kernel.h> instead of
<zephyr/zephyr.h> since it is more clear that you are including the
Kernel APIs and (probably) nothing else. <zephyr/zephyr.h> sounds like a
catch-all header that may be confusing. Most applications need to
include a bunch of other things to compile, e.g. driver headers or
subsystem headers like BT, logging, etc.
The idea of a catch-all header in Zephyr is probably not feasible
anyway. Reason is that Zephyr is not a library, like it could be for
example `libpython`. Zephyr provides many utilities nowadays: a kernel,
drivers, subsystems, etc and things will likely grow. A catch-all header
would be massive, difficult to keep up-to-date. It is also likely that
an application will only build a small subset. Note that subsystem-level
headers may use a catch-all approach to make things easier, though.
NOTE: This patch is **NOT** removing the header, just removing its usage
in-tree. I'd advocate for its deprecation (add a #warning on it), but I
understand many people will have concerns.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
Add footprint tracking of the samples/bluetooth/hci_rpmsg
sample for the nRF5340 with ISO broadcast and ISO receive respectively.
Signed-off-by: Emil Gydesen <emil.gydesen@nordicsemi.no>
Adds vendor name and existence macros for all compatibles of a node that
match an entry in the vendor prefixes file.
Signed-off-by: Maureen Helm <maureen.helm@intel.com>
Adds a new compat2vendor lookup table that maps compatibles to vendor
names, constructed from the vendor prefixes file. This approach is a
more scalable alternative to adding a vendor name property to devicetree
bindings, as was previously proposed.
Signed-off-by: Maureen Helm <maureen.helm@intel.com>
Follow-up: #41301
This commit is a rework and cleanup of the tools handling in Zephyr
CMake build system.
Instead of directly loading code a CMake modules for tool lookup, the
host tools now follows the CMake `find_package()` pattern for finding
programs / tools in module mode.
This makes it more clear which modules are responsible for finding tools
and which modules provides build integration / features.
The following tools can now be found using `find_package()`:
- Zephyr-sdk : find_package(Zephyr-sdk <version>)
- Generic host tools: find_package(HostTools)
This further allows us to decouple the `verify-toolchain` CMake script
part required by `twister` into a tool lookup module and a dedicated
CMake script which utilizes the lookup module.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
`handlers.py` treats the `psutil` package as optional and quietly prints
a message to stdout if it is not present, but it is actually a hard
requirement for the base `Handler` class. This try/except block is bad
as it hides the requirement until a timeout occurs and the handler
attempts to use `psutil` in the terminate method, which will crash if
the module happens to be missing.
This PR removes the try/except guard so missing `psutil` will cause
twister to immediately fail. `psutil` is already specified in the
`requirements.txt` file, so this should only affect users who
unwittingly have an outdated/incorrect environment.
Also update the Github Action for twister tests to install from
requirements files.
Signed-off-by: Tristan Honscheid <honscheid@google.com>
On some targets (such as STM32U5) using write-block-size >8, imgtool
should be called using option '--align 16/32' which is only available
starting version 1.9.
Update requirement for imgtool to make it available to such platforms.
Signed-off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
This adds support for ezFlashCLI tool to allow flashing DA1469x MCU
family from Renesas.
The tool is available at https://github.com/ezflash/ezFlashCLI or can
be installed via pip.
Signed-off-by: Andrzej Kaczmarek <andrzej.kaczmarek@codecoup.pl>
Signed-off-by: Ben Lauret <ben.lauret.wm@renesas.com>
Add footprint tracking of the samples/bluetooth/hci_rpmsg
sample for the nRF5340. This is a BT controller-only build
for the nRF5340 with minimum features.
Signed-off-by: Emil Gydesen <emil.gydesen@nordicsemi.no>
It is frequent to see in Devicetree code constructs like:
```c
#define NAME_AND_COMMA(node_id) DT_NODE_FULL_NAME(node_id),
const char *child_names[] = {
DT_FOREACH_CHILD(DT_NODELABEL(n), NAME_AND_COMMA)
};
```
That is, an auxiliary macro to append a separator character in
DT_FOREACH* macros. Non-DT API, e.g. FOR_EACH(), takes a separator
argument to avoid such intermediate macros.
This patch adds DT_FOREACH_CHILD_SEP (and instance/status okay/vargs
versions of it). They all take an extra argument: a separator. With this
change, the example above can be simplified to:
```c
const char *child_labels[] = {
DT_FOREACH_CHILD(DT_NODELABEL(n), DT_NODE_FULL_NAME, (,))
};
```
Notes:
- Other DT_FOREACH* macros could/should be extended as well
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
Add a utility macro used to remove brackets from around a single
argument. While __DEBRACKET exists in util_internal.h, this change makes
DT independent (otherwise we should include util_internal.h, which is
another option).
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
Allows to use the stm32cubeprogrammer runner on Linux to be executed
from any installation directory as long as the executable is found
in PATH.
Until now on Linux the programmer could only be used if it either has
been installed in the default location or if the executable path was
passed via the cli parameter.
Signed-off-by: Thomas Stranger <thomas.stranger@outlook.com>
Whenever binary blobs are present in the filesystem (i.e. fetched,
regardless of whether they are up-to-date or not) we consider the Zephyr
build as tainted. In order to ensure that it is marked as so, add a
Kconfig `select` statement whenever this is the case.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
In preparation for the handling of taint flags in zephyr_module.py, move
blob-processing code from the west command to it.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Add specialized tests that execute another test indirectly and ensure
that the results report errors correctly. See the README.rst file
provided with this commit for details.
Signed-off-by: Yuval Peress <peress@google.com>
stty operates on the file descriptor from stdin, but we want to operate
on stdout. If stdin and stdout are not the same tty then stty prints an
error.
Redirect stdout to stdin for stty calls.
Tested by running twister -T tests/subsys/shell </dev/null
Signed-off-by: Jeremy Bettis <jbettis@google.com>
With verbose output enabled, twister logs the cmake arguments as a
python array whenever it invokes cmake to generate a build system.
This is clunky output for manual reproduction, because it forces the
developer to manually join the arguments together for consumption by
the shell.
As an enhancement for this use case, use the standard library
shlex.join() function to produce a string that can be copy/pasted
directly into the shell on POSIX platforms.
We should use this function instead of str.join, because it correctly
handles things like shell quoting for options with spaces in them.
This function is not available on Windows, so we fall back on behavior
that is similar to the old one there.
The main difference on Windows is that the output now includes the
path to cmake as well, and not just its arguments. (This is the same
on POSIX, and is intended to help debug if multiple cmakes are
installed on the same machine.)
We are running cmake from a couple of different places, so doing this
cleanly requires adding a shared module with the platform abstraction.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Allow for having array types (array, uint8-array, string-array) be const.
This would allow for something like:
properties:
reg-names:
const: ["foo", "bar"]
To be supported.
Renamed function _check_prop_type_and_default to _check_prop_by_type
as part of this change and Moved the check for 'const' types into
_check_prop_by_type as its similar to the prop_type check and it was
easier to implement in _check_prop_by_type as we already extract
prop_type from the option in that function.
Signed-off-by: Kumar Gala <galak@kernel.org>
Turns out that `Namespace.format` was not a good fix to the issue
described in f033040812, it was returning
the default format even when the user specified -f/--format.
Replace "getattr" with a simple logical operation.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
There were two issues with the original commit
e04487cad2:
- getattr() doesn't seem to work properly with args
- Comparison between bound methods with "is" seems not to work, so
replace it with a simpler string comparison
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
There were a bunch of old command-line arguments that haven't been used
in a while, remove them.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
This implements support for relocating code to chosen memory regions via
the `zephyr_code_relocate` CMake function for RISC-V SoCs. ARM-specific
assumptions that were made by gen_relocate_app.py need to be corrected,
in particular not assuming any particular name for the default RAM
section (which is 'SRAM' for most ARM pltaforms) and not assuming 32-bit
pointers (so the test works on RV64).
Signed-off-by: Peter Marheine <pmarheine@chromium.org>
Most ARM platforms name their ROM region `FLASH`, but this assumption is
not portable. Have gen_relocate_app refer to ROMABLE_REGION instead of
hard-coding the platform-specific memory region name.
Signed-off-by: Peter Marheine <pmarheine@chromium.org>
Only the elftools SymbolTableSection section type provides an
iter_symbols() method, and compilers are free to emit sections that
include the substring '.symtab' in their name which causes errors if
gen_relocate_app examines only the section name. Instead check whether a
section is a symbol table to skip attempting to inspect sections that
are not actually symbol tables.
Signed-off-by: Peter Marheine <pmarheine@chromium.org>
Tweak check_compliance.py to allow user to run it w/o having
ZEPHYR_BASE set. This is similar to what we do for twister.
Signed-off-by: Kumar Gala <galak@kernel.org>
This commit adds string token versions of the values also
in items inside string-array.
Signed-off-by: Radosław Koppel <r.koppel@k-el.com>
Co-authored-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
Co-authored-by: Kumar Gala <galak@kernel.org>
Make some no-op log messages require 'west -v blobs ...'.
The current implementation won't scale as the number of blobs goes up.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
This option does not apply to 'west blobs clean' or 'west blobs
fetch'. Error out if it is given to either of those.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Make it consistent with the usual argparse style by using lower case
and no period at the end.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
The help text is mixing up 'blobs' with 'modules', and doesn't
document the new behavior that uses all modules when nothing is
mentioned. Fix both issues.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Without this patch, "west blobs -h" says:
{list,fetch,clean} Select the sub-command to execute. Sub-commands
available: - list: list binary blobs - fetch: fetch and store binary
blobs - clean: remove fetched binary blobs
With this patch:
{list,fetch,clean} sub-command to execute
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
esp32 platforms can overflow their dram0_1_seg so add that section name
to the overflow check in twister. We were seeing this with the
samples/arch/smp/pi/sample.smp.pi test and would get:
zephyr_pre0.elf section `noinit' will not fit in region `dram0_1_seg'
region `dram0_1_seg' overflowed by 17456 bytes
Fixes#49164
Signed-off-by: Kumar Gala <galak@kernel.org>
Commit 57538262e1
(" scripts: kconfig: tweak dt_chosen_label") forgot to update the
docstring for a function whose behavior was changed. Fix it.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
It looks like the latest release, 0.14.2, changed the contents of
JLINK_SDK_NAME as it was before 0.14.0 release. That means that the
previous fix is only applicable to a couple of releases: 0.14.0/0.14.1.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
The introduction of sysbuild and west build / flash / debug / etc.
support raises questions regarding how to handle user specified runner
arguments.
Some arguments may be acceptable for all domains, whereas others are
only valid for a single or few domains in a multi domain build.
Therefore, consider the usages of runner specific options experimental
when applied to multiple domains.
The following warning is printed to the user when using runner specific
arguments on multiple domains:
> WARNING: Specifying runner options for multiple domains is
> experimental.
> If problems are experienced, please specify a single domain using
> '--domain <domain>'
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
The `<module>/zephyr/blobs/` folder, as well as any additional
sub-folders that might be required under `blobs/` may not exist when
fetching a blob. Ensure that the folder exists, create it if it doesn't.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
The new "clean" blobs sub-command deletes any blobs present on the
filesystem.
Also improve the help text for sub-commands.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
To align with other commands, such as "west update", convert the
accumulative -m option into an nargs="*"-type one that takes the list of
modules directly without a flag.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Instead of both listing and fetching only the blobs in the
modules listed by the user in the command-line, default to
listing/fetching them all for ease-of-use and consistency with other
commands. Remove the --all options since it's meaningless now.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
This adds an extra layer to match the revision part where this
needs to be of permitted patterns as described in.
cmake/modules/extensions.cmake. Without this, the matching
may produce undesirable results. For example, if there is
a file named qemu_x86_tiny_1.conf, the testplan will create
a board name qemu_x86@tiny.1 (which is definitely not correct).
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Remove ZEPHYR_BASE as default root in list_boards.py.
This allows list_boards.py to be used to only print boards at given
root(s) without printing Zephyr default boards.
Secondly it remove the need in list_boards.py to have any knowledge of
ZEPHYR_BASE.
The `west boards` command already has ZEPHYR_BASE knowledge and can
easily add ZEPHYR_BASE to the list of roots it is already constructing,
thus removing the need for knowing that ZEPHYR_BASE is always added.
Update boards.cmake to pass ZEPHYR_BASE as an additional arch root, as
arch.cmake has not yet been processed,which means ZEPHYR_BASE is missing
in ARCH_ROOT list at this point in time.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
The motivation is to allow accessing array elements from device tree
properties in Kconfig.
* Add dt_node_array_prop and _node_array_prop functions to extract the
array elements from properties.
* Add 'dt_node_array_prop_int' and 'dt_node_array_prop_hex' keys to use
in Kconfig.
Signed-off-by: Eivind Jølsgard <eivind.jolsgard@nordicsemi.no>
On qemu_cortex_a53 with icount enabled, the qemu process will spin
hard at the end of its run (common with icount), but will also ignore
the SIGTERM twister is sending to terminate the (successful!) test
run. The resulting zombie spinning processes end up accumulating and
poisoning the test run.
Just use a hard KILL instead of trying to be a good citizen (and in
lieu of fixing the upstream qemu bug). I can't see why this should be
a problem.
Signed-off-by: Andy Ross <andyross@google.com>
Fix the missing logic of setting the flash_error flag when
caught the exceptions. And we should let the serial thread
can be run first if flash error. Otherwise the twister will
hang up while an exception happens on closing serial, after
the #47820 got merged. This mostly happens on frdm_k64f.
Fixes#49086
Signed-off-by: Enjia Mai <enjia.mai@intel.com>
Add parameter for possible extending in the future, --tool-opt
passes arguments to execution of the run service.
Ex.
--tool-opt=--arg='white space' --tool-opt=-r --tool-opt=12345
will be parsed as ['--arg=white space', '-r', '12345']
Signed-off-by: Enjia Mai <enjia.mai@intel.com>
Because we use INADDR_ANY by default now, specifying a net interface
is no longer needed. Remove it and remove the dependency of the
netifaces python package.
Fixes. #48584
Signed-off-by: Enjia Mai <enjia.mai@intel.com>
Add specifying the port for the remote cavstool remote service.
Ex.
west flash --remote-host {host}:{port} \
--pty {host}:{port}
Specify the port is optional when running west.
And another major change is now we have to specify the remote log
service by --pty to make it output the log message immediately.
Previously it will output directly. Why we make this change is
because this is more close to the behavior we use west flash.
Fixes. #46865
Signed-off-by: Enjia Mai <enjia.mai@intel.com>
This patch introduces the basic infrastructure to list and fetch binary
blobs. This includes:
- The new 'blobs' extension command
- An implementation of the `west blobs list` command
with custom formatting
- A very simple mechanism for loading fetchers
- A basic implementation of an HTTP fetcher
In order to ensure consistency among the west extension commands in the
main zephyr tree, we reuse a similar class factory pattern that is present
for ZephyrBinaryRunner instances in the ZephyrBlobFetcher case. This
could be achieved with a simpler mechanism, but opted for consistency
before simplicity.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Instead of using strings, use pathlib's functionality to handle the
fact that it can have two different extensions.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
prevent `Wundef` warnings from occurring due to
missing CONFIG_ symbols and __cplusplus.
Signed-off-by: Christoph A Schnetzler <Christoph.Schnetzler@husqvarnagroup.com>
Currently, twister keeps on monitoring serial output until timeout
even if we have a flash error. And the error reason will be reported
as 'No Console Output(Timeout)', which is not accurate.
So add a flash_error flag to control if we need to monitor the output.
Signed-off-by: Guo Lixin <lixinx.guo@intel.com>
This is this initial sysbuild guide on what sysbuild is and how to build
a sample using sysbuild.
It provides an architectural overview of sysbuild.
Descriptions on how to use `west build` or `cmake` + `ninja` to build
projects with the sysbuild infrastructure.
Flashing is described through the use of `west flash`.
Extending sysbuild with additional Zephyr based applications are
described, and reference to CMake documentation for including non-Zephyr
based applications are provided.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Update to Sphinx 5.x, also update breathe since version 4.34 is the
first to officially support Sphinx 5.x.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
This reverts commit 87c6789355, restoring
commit 3b341085a2.
Restore the original change now that the underlying issue has been fixed
in 6cfb18686e.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
Add the hearing aid sample to the footprint tracking for the
nrf52840dk_nrf52840 and nrf5340dk_nrf5340_cpuapp.
The sample should represent a simple hearing aid implementation,
and is thus a very good candidate for tracking the footprint
of LE Audio in Zephyr.
Signed-off-by: Emil Gydesen <emil.gydesen@nordicsemi.no>
The boards command was not properly using the zephyr_module
functionality to obtain the board roots of all modules. Fix that by
moving the functionality required to the core zephyr_module file and
reuse it from external scripts.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
support to use board@revision as platform filter when running
twister, like "twister -p nucleo_f030r8@1 ...".
Signed-off-by: Chen Peng1 <peng1.chen@intel.com>
This file is no longer used by device.h, so let's avoid spending time
generating the content formerly in it.
In order to leave a pointer in place for users who are expecting to
see it or are pulling it into their own builds, however, replace its
contents with an #error directive that tells them what happened. This
can be removed later on when we expect people are used to the
transition.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Add two helper macros:
- DT_FOREACH_HELPER
- DT_FOREACH_OKAY_HELPER
These are internal generated macros which will be used to define
devicetree.h APIs for iterating over every node in the tree, and every
node in the tree with status "okay", respectively.
We can use these new APIs to implement the functionality in
device_extern.h natively from device.h without requiring a generated
header. It will also be useful for other purposes later on.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Now that we've removed the label property from most devicetree nodes
the dt_chosen_label will end up returning "" in most cases. For the
small handful of cases that the function is used, return the node.name
instead as this matches what DEVICE_DT_NAME will do.
Signed-off-by: Kumar Gala <galak@kernel.org>
Replicate the devicetree dependencies into a sorted list. This ensures
that the structures added to the .__device_handles_pass2 section are
reproducible from build to build.
Tested with: west build -b native_posix tests/drivers/build_all/sensor
Without this change, two consecutive builds do not compare.
Signed-off-by: Keith Short <keithshort@google.com>
pylink 0.14.0 changed the class variable where JLink DLL library name
(libjlinkarm) is stored. This patch adds support for new pylink
libraries while keeping backwards compatibility.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
The elf_parser library now generates a dot file with device dependencies
that can be later rendered using Graphviz. Each node in the diagram
contains the device label (taken from DT node). In some cases the label
property can be None, leading to build failures like:
```
line 273, in device_dependency_graph
text = '{:s}\\nOrdinal: {:d} | Handle: {:d}\\n{:s}'.format(
TypeError: unsupported format string passed to NoneType.__format__
```
This patch switches to node name instead, which will always be set to
some value. This value is actually what devices get now as a name if
they do not have a label set.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
There is a breaking change in Junitparser 'TestCase.result' between
v1.x and v2.x.
Update check_compliance.py minimally to take into account this change.
This avoids relying on an outdated package.
Signed-off-by: Henri Xavier <datacomos@huawei.com>
This commit extends the west commands build, flash, and debug to support
--domain when having multiple domains (images) defined in a domains.yaml
build file.
The domains.yaml uses the following yaml format to specify the
build directory of each domain in the multi image build:
> default: <domain-n>
> domains:
> <domain-1>:
> build_dir: <build_dir-domain-1>
> <domain-2>:
> build_dir: <build_dir-domain-2>
> ...
`west <build|flash|debug>` has been extended to support
`--domain <domain>`.
`west build` calls CMake to create the build system, and if `--domain`
is given, then the build tool will be invoked afterwards for the
specified domain.
`west flash` will default flash all domains, but `--domain <domain>`
argument can be used to select a specific domain to flash, for example:
> west flash --domain mcuboot
`west debug` only a single domain can be debugged at any given time.
If `--domain` is not specified, then the default domain specified in the
domains.yml file will be used.
Users can still select a different domain, for example with:
> west debug --domain mcuboot
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
west build now support the sysbuild CMake project which allows users
to easily enable and build a bootloader together with any sample.
This will allow for cleaning up samples and boards which already does
custom multi image in a non-generic way, for example:
- mps2_an521 which includes a build for remote board
- samples/ipc/ which includes a samples for remote board
- esp32 which includes custom bootloaders
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
The --tool-opt runner option is the recommended practice for allowing
runners to take additional arguments that are passed on to the
underlying tool. It exists because we don't want to add one runner
option for every single tool option that users might want to tweak --
that would be a nightmare.
Enough runners are using this option that it's time to promote it to a
common runner capability with consistent behavior, the same way we did
for the --dev-id option in the past. This removes boilerplate from
individual runner files and ensures consistent argument handling for
this option when it is supported.
Since --tool-opt is a bit long to type, and we've had some complaints
about that, take this as an opportunity to standardize on -O as a
short option equivalent for it.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Even when BreakBeforeBraces is set to Linux, implying BraceWrapping for
AfterEnum to be true, clang-format version 12 puts the opening brace of
an enum declaration on a new line. [1]
One possible workaround would be to change AllowShortEnumsOnASingleLine
from false to true, but that would yield a different kind of unwanted
formatting.
clang-format version 13 and newer have this issue fixed, therefore
bumping the minimal version.
[1] https://github.com/llvm/llvm-project/issues/48983
Signed-off-by: Reto Schneider <reto.schneider@husqvarnagroup.com>
If a device manually specifies that it depends on a second DT device,
add the first device to the second devices list of supported devices.
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
Output the final dependency graph as a `.dot` file, which when rendered
by graphviz can be easier to comprehend than the text descriptions.
This output is optional in that it will not be generated if `graphviz`
is not installed.
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
Use the new elf_parser module to simplify the process of generating the
final device handle arrays.
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
Adds a python module intended to simplify other scripts that need to
work with device information compiled into the first pass `.elf` file.
Scripts utilizing this module can focus on iterating over dependency
graphs to generate the desired output, instead of also needing to
extract and build the graphs in the first place.
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
Rename the placeholder variable generated for PM slots so that the
prefix doesn't colide with the PM structs declared by devices. This
simplifies the process of searching for symbols in `.elf` files.
Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
Escape quotes provided in extra arguments on the command line.
example: ... -x "CONFIG_COMPILER_OPT=\"-fanalyzer\""
Fixes#46382
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
When doing a split build and test with coverage enabled, (i.e.
`twister --coverage --build-only` followed by `twister --coverage
--test-only`), Twister attempts to run coverage reports immediately
after building, which fails because tests haven't actually run yet
and causes several error messages to print in to the log.
This change causes twister to skip the call to `run_coverage()` if
`--build-only` is set and print an info message. This suppresses the
error messages from the coverage report tool complaining about missing
coverage files, but still instruments the built code for coverage data
collection. When twister is invoked again, but with `--test-only`
instead, the coverage files will be emitted and the reports
successfully generated.
#### Testing
I ran...
```
$ zephyr/scripts/twister -T zephyr/tests/lib/cmsis_dsp/basicmath/
--coverage -p native_posix --build-only
$ zephyr/scripts/twister -T zephyr/tests/lib/cmsis_dsp/basicmath/
--coverage -p native_posix --test-only
```
... followed by a combined build and test ...
```
$ zephyr/scripts/twister -T zephyr/tests/lib/cmsis_dsp/basicmath/
--coverage -p native_posix
```
... as a control and diffed the HTML coverage reports, which were
identical other than a timestamp.
Signed-off-by: Tristan Honscheid <honscheid@google.com>
Fix bug in verify_platforms_existence method - make it possible to
verify all platform names from list - not only first.
Fixes: #48321
Signed-off-by: Piotr Golyzniak <piotr.golyzniak@nordicsemi.no>
This commit makes the `--ninja` argument default to `true` when neither
`-k` nor `--make` is specified (i.e. when Make generator is not
selected).
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
When `--short-build-path` argument is specified to the twister, the
following error message is displayed because the Python argparse module
does not allow specifying a mutually exclusive argument as required:
ValueError: mutually exclusive arguments must be optional
This commit removes the `required` parameter when adding the `--ninja`
argument, which is a mutually exclusive argument, and adds a manual
check to validate this condition instead.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
In cases of global changes where 100s of nodes are launched, i.e. on
samples and tests (more than 20 tests/samples changed), do a full
covrage run.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
