Allow using the C pre-processor to generate a
`rimage/config/platform.toml` file from a "source"
`rimage/config/platform.toml.h` file.
This is optional and fully backwards compatible.
To use, do not use `-c` and point west sign at a configuration directory
instead or let it use the default `rimage/config/` directory and change
the files there.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
rimage is very verbose by default and has no -q(uiet) option, so saving
one line out of more than 100 lines is pointless.
RimageSigner.sign() was already very complex and suffering from
combinatorial explosion of parameters. With .toml
pre-processing (#65411) it's getting worse, so we really need all build
logs to show the complete rimage command.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
In the following command, the first argument `for_rimage` is passed to
`rimage` whereas `--for west` goes to west.
```
west sign for_rimage --for west
```
This is somehow valid but we really don't want anyone to do that.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Zero functional change, preparation for the .toml modularization.
RimageSigner.sign() is also way too long and too complex and this helps.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
`west sign` has been invoked by `west build` (through CMake) since
commit fad2da39aa, almost one year ago. During that time, this new
workflow has been refined and successfully used by at least two vendors,
multiple CIs across both SOF and Zephyr and many developers.
At the time, the ability to sign from `west flash` was preserved for
backwards compatibility. This means rimage parameters can come from many
different places at once and that rimage can be invoked twice during a
single `west flash` invocation!
Now that Zephyr 3.5 has been released, we need to reduce the number of
rimage use cases and the corresponding validation complexity and
maintenance workload to simplify and accelerate new features like
splitting rimage configuration files (#65411)
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
This removes escape from generate XML_CATALOG_EACH string, because
the string between CDATA tag need to be treated as normal text.
Signed-off-by: Fang Huang <fang.huang@intel.com>
Fixes#65477
Platform key checking seemed to be erroneous;
now the variable names, comments and code seem in line with each other.
Signed-off-by: Lukasz Mrugala <lukaszx.mrugala@intel.com>
Support lcov > 2.0 tool which has strict error checking and some new
configuration options deprecating syntax used in 1.4 versions.
Fixes#62202
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Improve documentation about pytest integration with Twister. Add
examples of usage, improve description of available options and
introduce automatic doc generation of two plugin classes (DeviceAdapter
and Shell) basing on their docstrings from source code.
Signed-off-by: Piotr Golyzniak <piotr.golyzniak@nordicsemi.no>
Introduce Kconfig option in zephyr build system that reflects the TF-M
cmake config variable with the same default value for dummy provisioning
and have it satisfy the IAK present requirement.
This configuration is not suitable for production, and by having this
in zephyr configuration we can have this as part of the hardened
configuration check.
Signed-off-by: Joakim Andersson <joakim.andersson@nordicsemi.no>
as in windows platfrom the format by default is not utf-8,
and we will see below error
Traceback (most recent call last):
File "...zephyr\scripts\twister", line 211, in <module>
ret = main(options)
^^^^^^^^^^^^^
File "...scripts/pylib/twister\twisterlib\twister_main.py",
tplan.load()
File "...scripts/pylib/twister\twisterlib\testplan.py",
self.load_from_file(last_run, filter_platform=connected_list)
File "...scripts/pylib/twister\twisterlib\testplan.py",
instance.create_overlay(platform, self.options.enable_asan,
self.options.enable_ubsan, self.options.enable_coverage,
self.options.coverage_platform)
File "...scripts/pylib/twister\twisterlib\testinstance.py"
f.write(content)
UnicodeEncodeError: 'gbk' codec can't encode character '\xf8'
in position 64: illegal multibyte sequence
Signed-off-by: Hake Huang <hake.huang@oss.nxp.com>
It's not good to see struct_tags.json change from one build to the next
when nothing changes.
Python's sets are not deterministic, see long story for older commit
f896fc2306 ("scripts: gen_handles: Sort the device handles")
Simply convert multiple_directories to a sorted list before using it.
Fixes commit 80e78208e6 ("kernel: syscalls: no need to include all
syscalls in binary")
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
The Console Harness is able to parse its log with patterns to compose
extracted fields into records in 'recording.csv' file in the test's build
directory. This feature allows to extract custom test results like
performance counters.
With this change the extracted records are also written into 'twister.json'
as a part of each test suite object. This makes easier to store
all the data collected by the test for its further processing.
Other improvements:
- compile parsing pattern only once instead of at each input line;
- quote fields in '.csv' to avoid unexpected field separators;
- make 'regex' a required schema field of 'harness_config';
- Twister documentation update.
Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
This changes the logic in __init__() so that if a path to
MAINTAINERS.yml is passed in, it uses the passed-in value
instead of blindly running git to find the top level of
Zephyr tree.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Change the keep-sorted check to handle blocks of code uniformly
indented.
Suggested-by: Benjamin Cabé <benjamin@zephyrproject.org>
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
A race condition was possible at QEMUHandler which didn't wait for
its harness to complete causing false negatives and test suite
retries, for example when the console harness has got rather
long output from a test application to check for patterns.
Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
Python's sets are not deterministic.
`devices` were already sorted but `dev_supports` is still a
non-deterministic set. Sort dev_supports to make the graph output
deterministic.
Fixes commit 29942475c5 ("scripts: gen_handles: output dependency graph")
It is quite ironic that this initial and non-deterministic graph commit
was concurrent with and slightly delayed other commit
f896fc2306 ("scripts: gen_handles: Sort the device handles") which
fixed another, similar non-determinism issue in the same area. A true
"whack-a-mole"!
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
This allows sorting objects consistently without having to specific a
key.
No functional change yet.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
Using --platform-reports will also generate json reports
with results from a single platform. Needed for on-target
results publishing process.
Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
As native_sim is replacing native_posix overall and
becoming the default test platform.
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
One can not even replace sucessfully pytest basic sample `pytest-args`
with command line "--pytest-args", as all it does is to append a single
string to current list of commands, making it impossible to send several
arguments.
This patch fixes that by allowing several instances of `--pytest-args`
to compose the whole list of arguments to be passed to pytest.
Signed-off-by: Ederson de Souza <ederson.desouza@intel.com>
If a platform is fitlered, do not add it to keyed test map, otherwise we
will end up skipping all platforms from the same class without any
coverage.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Add `possible` to match-leak-kinds to prevent false positives caused by
POSIX soc no cpu cleanup.
The leak can be reproduced by adding CONFIG_NETWORKING=y to
tests/lib/cpp/libcxx/prj.conf and run twister:
twister -p native_sim -s tests/lib/cpp/libcxx/cpp.libcxx.host
--enable-valgrind
Signed-off-by: Jeppe Odgaard <jeppe.odgaard@prevas.dk>
Useful option during development and when applying smaller change to the
code and verifying changes using tests. This works fine as long as no
dependencies or major changes are done, i.e. when editing few files and
in a subsystem. Use with caution and use on your own risk :-)
When building multiple tests, this provide significant boost, up to 300%
faster than when rebuilding everything from scratch or when re-running
cmake every single time.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Align with native_simulator's upstream main
ae241af736d06874ec02deb9aacb79918d745aba
Which includes:
* ae241af native timer_model: Allow calling into get_host_us_time
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
Measure both cmake and make/ninja build times and log them in debug mode
and additionally put the result in the json report for tracking build
times.
Cleanup what build/cmake function return and remove unused keys in the
result map.
Remove some excessive logging of launched jobs or default platforms that
gets in the way when in debug mode.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Add `__unused` to the `$Attribute` family along with its
`__maybe_unused`, `__always_unused` & `__used` brothers, so
that:
```c
__unused int ret;
```
is recognized as variable declaration, and doesn't raise
`LINE_SPACING` warning in CI.
Signed-off-by: Yong Cong Sin <ycsin@meta.com>
Extend Twister command line with --pytest-args. This parameter
is passed to pytest subprocess. It allows to select a specific
testcase from a test suite.
Signed-off-by: Grzegorz Chwierut <grzegorz.chwierut@nordicsemi.no>
Align with native_simulator's upstream main
7d652dbfb313260cf07d595ccf26638f2b3c2959
Which includes:
* 7d652db Provide macros for noreturn and unreachable & annotate
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
In the docs replace references to native_posix with native_sim
Switch the default test platform to native_sim from native_posix
Signed-off-by: Alberto Escolar Piedras <alberto.escolar.piedras@nordicsemi.no>
Sync the spelling.txt file with a recent version of the one found in
Linux kernel.
List sorting was preserved (however inconsistent it might be) to
simplify future syncs.
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
Somehow this is seems to be a very common typo. Adding it to spelling.txt
to catch it as part of compliance check.
Signed-off-by: Benjamin Cabé <benjamin@zephyrproject.org>
On MacOS, ps utils raises a `NoSuchProcess` error rather than a
`ProcessLookupError` when a pid no longer exists.
Signed-off-by: Wilfried Chauveau <wilfried.chauveau@arm.com>
The sorted check code crashes on binary files. Add a check on file type
and only process text ones.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
Implement a workaround for Console harness to compose TestCase
identifier correctly when a Ztest suite with a single testcase
uses this harness type. Normally, a Ztest suite should use the
Ztest Twister harness.
Without this workaround each Ztest TestCase result on Console is
duplicated (and written into twister.json) with its 'identifier'
attribute set to TestSuite id only, no TestCase suffix added;
the resulting entry with the full TestCase id is also stored,
but its values are empty with defaults.
Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
If Console Harness 'harness_config' properties have no 'regex'
patterns or no correct 'type' set, then ConfigurationError exception
is raised, handled, and the test instance error is accounted
in the summary results.
Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
Fix the Twister Console harness unordered pattern matching
to treat the ztest as failed when not all of the
expected patterns were found in the console output, but the ztest
application itself reports 'PROJECT EXECUTION SUCCESSFUL'.
Unify debug logging on pattern match for ordered and unordered patterns.
Signed-off-by: Dmitrii Golovanov <dmitrii.golovanov@intel.com>
