Commit graph

5615 commits

Author SHA1 Message Date
Maciej Perkowski
e37d12e631 Twister: Refactor Test(Harness).handle()
The function `handle()` from class `Test(Harness)` had most of its
code being the direct copy of the code from
`Harness.process_test(line)`. This patch replaces the copied code
with the call for `process_test(line)` and removes repetition of
this function call.

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2021-03-17 15:18:06 -04:00
Martí Bolívar
847ba43df9 scripts: run_common: improve error handling for missing runner
Catch the ValueError returned by get_runner_cls() and turn it in to a
log.die().

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-03-17 11:30:45 +01:00
Martí Bolívar
d61545b513 scripts: runners: handle failure to import individual runners
We need all available runners to be defined as subclasses of
runners.core.ZephyrBinaryRunner in order to be able to look them up by
name at runtime. We do this by importing them from runners.__init__.

This process periodically fails when some runner or other cannot be
imported, usually because it is trying to import something outside of
stdlib and not handling ImportError.

Rather than letting this bring down the entire Python process, catch
and log the error. Sort the list again while we're here.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-03-17 11:30:45 +01:00
Martí Bolívar
e6435fda94 scripts: runners: jlink: handle packaging ImportError
This too is violating the 'stdlib or handle ImportError' rule.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-03-17 11:30:45 +01:00
Martí Bolívar
a56e1fa52d scripts: runners: bossac: handle edtlib ImportError
Runners may not import anything outside of stdlib unless they handle
the possible ImportError.

Apply this rule to the bossac runner. The general pattern is to have a
MISSING_REQUIREMENTS global that we handle at the beginning of
do_run(), but in this case this isn't a missing pip package, but a
misconfiguration. Just log a warning; it won't happen in practice.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-03-17 11:30:45 +01:00
Martí Bolívar
b3c6470bb3 scripts: run_common: make edtlib importable to all runners
Runners can (and many should) depend on the devicetree to decide what
to do, especially as it relates to the flash layout. Make it so that
they do not have to manipulate sys.path to get a hold of the edtlib
directory by doing it in one place, before importing any of them.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-03-17 11:30:45 +01:00
Anas Nashif
a983a99318 twister: set verdict on tests after evaluating results
Do not set results before we finished parsing output.

Fixes #32514

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-03-17 11:28:22 +01:00
Ioannis Glaropoulos
488e7ef39a west commands: runners: support for including config file in pyocd
Add support for including a board-specific config.yaml file
in the pyocd flash command. Similary to openOCD, the config
file is placed in the board directory under support/.

Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
2021-03-16 20:01:53 -04:00
Johann Fischer
44a02c5ec5 twister: use edt.label2node in dt_compat_enabled_with_label DT filter
Use edt.label2node in dt_compat_enabled_with_label DT filter.

Signed-off-by: Johann Fischer <johann.fischer@nordicsemi.no>
Co-authored-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-03-16 17:01:34 -05:00
Maciej Perkowski
f050a999f2 twister: Improve handling of errors while flashing device
The patch improves handling runners errors from flashing. It
add thread termination in case of process returning non 0 exit
code. Before, even though the error was returned,
a thread was not terminated and test ended
only after timeout termination. The patch also adds information about
the failure reason to the reports

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2021-03-16 09:05:28 -04:00
Jose Alberto Meza
49c626a33d scripts: west_commands: tests: Update dediprog expected command
Update test to account for new default parameters that allow
use image smaller than flash device capacity.

Signed-off-by: Jose Alberto Meza <jose.a.meza.arellano@intel.com>
2021-03-16 10:02:57 +01:00
Jose Alberto Meza
4cbfc324e3 scripts: west_commands: runners: Remove need to match device capacity
Use fill option to be able to flash file images small than
flash device capacity.

Signed-off-by: Jose Alberto Meza <jose.a.meza.arellano@intel.com>
2021-03-16 10:02:57 +01:00
Johann Fischer
bb4749709e twister: add dt_compat_enabled_with_label DT filter function
The function returns true if the node with the referenced
node label is compatible and enabled.

Signed-off-by: Johann Fischer <johann.fischer@nordicsemi.no>
2021-03-15 10:15:44 -04:00
Torsten Rasmussen
b92b580835 scripts: updated gen_handles.py to take Zephyr base as argument
The script gen_handles.py was introduce in #32127 but relies on
ZEPHYR_BASE being set in environment.

However, it is not a requirement to set Zephyr base in the users
environment, therefore this is changed to an argument `-z` or
`--zephyr-base` which will be passed from the build system to the
script.

If `-z` or `--zephyr-base` is not provided, the environment will be
checked for a ZEPHYR_BASE setting there.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2021-03-15 09:30:44 -04:00
Steven Huang
1413c55fbc twister: Raise exception if cmake is not found in path
Otherwise the real failure is swallowed and a TypeError exception is
raised instead.

Signed-off-by: Steven Huang <stevenhuang1995@gmail.com>
2021-03-15 08:58:34 -04:00
Anas Nashif
4bccc1db10 twister: fix integration selection
--integration without integration_platforms in the test definition is a
noop.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-03-12 19:50:54 -05:00
Carles Cufi
e9e28c0562 ci: Re-enable bsim in CI
BabbleSim execution was (AFAICT mistakenly) disabled in
dda6a5ee90d0f5b31e7d269761dc2ab1a2f8510c. Re-enable it so we run them in
CI again.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2021-03-11 08:52:18 -05:00
Enjia Mai
2c5e2c2394 tests: ztest: provide test execution times per ztest testcase
Add execution time for testing result of each ztest testcase as:

START - test_sem_multi_take_timeout_diff_sem
PASS - test_sem_multi_take_timeout_diff_sem in 2.54 seconds

Fix #32137.

Signed-off-by: Enjia Mai <enjiax.mai@intel.com>
2021-03-11 07:52:17 -05:00
Anas Nashif
e618a5964b twister: fix integration_platforms selection
Logic for selecting non-default platforms listed in
integration_platforms did not work. Functionality is restored in this
change.

Fixes #32835

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-03-11 05:48:40 -05:00
Martí Bolívar
f5409dec01 scripts: edtlib: do not inherit parent compatibles
Commit 0d4dca10b2 ("scripts: edtlib:
child binding compatibles match parents") was a hack meant to keep the
edtlib.Binding class in place without modifying some twister behavior
that needed further changes to work properly with first-class binding
objects.

This is a hack and is no longer necessary, so back out of this change.

Child Binding objects now have None compatible properties unless the
binding YAML explicitly sets a compatible.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-03-10 15:09:09 -05:00
Martí Bolívar
51f55b437f twister: replace dt_compat_enabled_with_alias filter
Originally added in 7733b94224.

This filter is not well-formed. It's meant to match nodes like
/leds/led_0 in this DTS:

/ {
	aliases {
		led0 = &led0;
	};

	leds {
		compatible = "gpio-leds";
		led0: led_0 {
			gpios = <...>;
			label = "LED 0";
		};
	};
};

Uses look like this:

    filter: dt_compat_enabled_with_alias("gpio-leds", "led0")

But notice how the led_0 node doesn't have compatible "gpio-leds";
it's actually the *parent* node that has that compatible.

Replace this with a new filter, dt_enabled_alias_with_parent_compat(),
which is used like this:

    filter: dt_enabled_alias_with_parent_compat("led0", "gpio-leds")

This has a name and argument order that makes the meaning of the
filter clearer.

Replace in-tree users with the new filter.

Deprecate the old filter and warn about its use using the standard
logging module.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-03-10 15:09:09 -05:00
Martí Bolívar
d802fc3bf0 scripts: twister: sort expr_parser.py imports
Just tidying up.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-03-10 15:09:09 -05:00
Kumar Gala
eea17e5441 ci: scripts/ci/run_ci: Cleanup file names
Rename test_file_[1..4] to something a bit more meaningful.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2021-03-08 08:35:56 -06:00
James Harris
074dbb9982 scripts: gen_syscalls: declare syscalls with void instead of zero args
In C, `void foo(void);` and `void foo();` mean subtly different things.
The former means "foo takes zero arguments"; the latter means
"foo takes an unspecified number of arguments". This can result in
calling convention mismatches in exceptional cases.

Change to emitting `(void)` instead of `()` for a syscall with
zero arguments.

Signed-off-by: James Harris <james.harris@intel.com>
2021-03-05 12:42:36 -05:00
Andy Ross
098fce351f scripts/twister: Fix race with device-testing
CPython is sometimes described as "single threaded" due to the GIL,
but the interpreter will still "preemptively" switch between threads
(the details seem poorly documented).

So the time between checking whether acquire is 1 and decrementing the
count could result in more than one thread seeing an "available"
device, and more than one test being run (simultaneously, on the same
physical device!).  We have a big herd of threads all polling for
this, so in a large test run this would happen maybe one time out of
20-30 tries.

Use a lock.  Also remove the very similar looking
DUT.get_available_device() method, which had the same bug but appears
to be dead code.

Fixes #32679

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2021-03-04 12:39:54 -05:00
Anas Nashif
82daea910b ci: limit CI to only changed architecture
If files are changed for architecture code, then only run CI on those
architectures.

Combine all scripts into one and make it simpler.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-03-04 07:14:11 -06:00
Anas Nashif
28bc79b49f ci: do not run twister CI when changing checkpatch
changes to checkpatch have no impact on code and twister results.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-03-04 07:14:11 -06:00
Anas Nashif
3572b518f6 ci: remove shippable references
We now use buildkite.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-03-04 07:14:11 -06:00
Joakim Andersson
200f4687e0 Bluetooth: host: Log security keys needed by sniffer
Log the security keys that the sniffer needs in order to sucessfully
decrypt the connection.
This option allows the sniffer to work in the cases where enabling
using the SMP debug keys is not wanted, either because it changes the
way the peer behaves or is denied by the peer.
It also enables the sniffer to decrypt a connection where the bond
already exists.

Signed-off-by: Joakim Andersson <joakim.andersson@nordicsemi.no>
2021-03-04 14:20:25 +02:00
Martí Bolívar
fbd34dca1e twisterlib: add deprecated DT props to warnings_as_errors
Use EXTRA_GEN_DEFINES_ARGS to error out on deprecated devicetree
properties when warnings are treated as errors.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-03-03 10:22:36 -06:00
Martí Bolívar
df5a55c638 dts: gen_defines: add CLI option to error out on deprecated props
Allow users to set the new EDT constructor argument which errors out
on deprecated properties via a command line argument.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-03-03 10:22:36 -06:00
Martí Bolívar
40c4f0048e dts: edtlib: add EDT kwarg to error out on deprecated props
We'd like to start eliminating deprecated properties from upstream
Zephyr devicetrees. To make that possible in the build system, add an
EDT kwarg that does just that.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-03-03 10:22:36 -06:00
Martí Bolívar
563a46c7d4 dts: clean up binding syntax checks
Commit 6bf761fc0a ("dts: Remove support
for deprecated DTS binding syntax") removed most of the support for
the 'legacy' bindings syntax.

A few straggler keys are still around in the bindings check code,
though. This allows some legacy keys which should cause errors to pass
silently instead.

Fix the error handling and print good errors for cases which are
removed, just in case someone is still using them somewhere.

Clean up some other error messages in the same function while we're
here.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-03-03 08:18:10 -06:00
Carlo Caione
6ff37886dd aarch64: mem_domain: Introduce k_mem_partition_attr_t
Introduce the kernel macros for memory attribution also for AArch64.

Signed-off-by: Carlo Caione <ccaione@baylibre.com>
2021-03-01 09:25:44 -05:00
Kumar Gala
4aac908973 devicetree: add DT_NODE_FULL_NAME()
This macro returns a node's name with unit-adddress, given its node
identifier.

The node name is useful information for the user to utilize for debug
information, similar to DT_NODE_PATH, or DT_LABEL.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2021-02-25 19:24:08 -06:00
Kumar Gala
270a05f697 dts: gen_defines: Change f-string to normal string
Change the use of a f-string to a normal string as there is nothing
that needs formatting in the particular instance.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2021-02-25 19:24:08 -06:00
Jordan Yates
dcaabb860f west: runners: jlink: prefer .hex over .bin
Update the jlink runner to prefer flashing .hex files instead of .bin.
This can increase programming speed in cases where there are large
amounts of padding in an application.

Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
2021-02-25 10:59:13 -06:00
peng1 chen
6478735493 twister: fix the inconsistent total tests number
sometimes a test instance hasn't been boot up to execute successfully,
so the results is an empty dictionary, it will not be added into final
twister_report.xml file.
in order to include it, fill the results as BLOCK.

Signed-off-by: peng1 chen <peng1.chen@intel.com>
2021-02-24 16:00:35 -05:00
Pete Johanson
310a464acf cmake: Add UF2 as an additional build output option.
Add ability to build a UF2 (https://github.com/Microsoft/uf2)
image as an additional output type. This leverages the code
partition offset for the UF2 base address, and a configurable
UF2 family ID.

Includes an unmodified (except for headers for licensing, pylit
disabling) version of the uf2conv.py script copied the UF2
format specification repository, used to convert the bin to UF2.

Origin: UF2 file format specification reference utilies
License: MIT
URL: https://github.com/microsoft/uf2/blob/master/utils/uf2conv.py
commit: 587abb8b909266e9b468d6284f2fbd425235d1b5
Signed-off-by: Pete Johanson <peter@peterjohanson.com>
2021-02-23 21:11:04 +01:00
Sebastian Wezel
148dbdc9fe twisterlib: checking for classname and testname
Currently duplicated testcases are only determined by the classname.
This results in testcases not beeing added to the twister.xml if the
testcases are from the same class (e.g. sample.yaml), but with different
names.

The changed check if the combination of classname and testname
already exists.

Signed-off-by: Sebastian Wezel <sebastian.wezel@nordicsemi.no>
2021-02-23 10:57:14 -05:00
Peter Bigot
d554d34137 device: add post-process of elf file to manage device handles
Following the idiom used for system calls, add script support to read
the initial application binary to identify which devices are defined,
and to use their offset in the device array as their unique handle
rather than the externally-defined ordinal from devicetree.  The
device dependency arrays are updated to use these handles.

Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
2021-02-19 15:46:16 -05:00
Flavio Ceolin
155a06a1c9 scripts: Update paths on tags.sh
Update source paths.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2021-02-19 05:28:09 -06:00
Andrei Emeltchenko
10fa48dd3e twister: Include coverage support for device testing
Include coverage support for device testing. It works by switching
capture_coverage to True (as it would meet coverage data start
pattern). Then we continue tot read coverage data until we get
coverage data end pattern. Otherwise, after receiving test result
pattern, we close serial console and do not get time to find start
coverage data pattern.

Signed-off-by: Andrei Emeltchenko <andrei.emeltchenko@intel.com>
2021-02-15 08:13:48 -05:00
Jian Kang
8c9b06ad82 board: cavs15: Add a option to control signing ways
Zephyr testcases(not SOF case) not use kernel DSP driver to load image
on ADSP board, thus do not need signing with xman. So add a input
'--no-manifest' to specify signing without xman in image. If use DSP
driver load image, we should not specify this.

Signed-off-by: Jian Kang <jianx.kang@intel.com>
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-02-11 13:52:30 -05:00
Andrzej Głąbek
f9c1f89109 scripts: kconfig: Fix uses of edt.compat2okay in helper functions
Check if a given 'compat' is a key in edt.compat2okay before accessing
its entry in dt_compat_on_bus() and dt_nodelabel_has_compat(), as for
a non-existing key a new entry with an empty list would be created,
and dt_compat_enabled() would then incorrectly return "y" when called
for the same 'compat'.

Signed-off-by: Andrzej Głąbek <andrzej.glabek@nordicsemi.no>
2021-02-09 13:33:10 -05:00
Martí Bolívar
16a7465d93 scripts: zcmake: fix for prerelease cmake versions
Handle a semantic versioning case used by Kitware that is not
supported by the packaging module we're using for version comparisons.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-02-09 09:14:37 -05:00
Maciej Perkowski
b6a69aa820 twister: bugfix: Fix twister output
Fix twister output. "Test suite" used previously did not correspond
to what was counted (test scenarios). A test scenario is an entry from
testcase/sample.yaml under "tests:", which specify configuration used to
build a test. Changes in docs will follow later on. Also replace the
amount of preselectd platforms with the amount of platforms that
were not filtered out and actually tested.

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2021-02-09 06:55:41 -05:00
Martí Bolívar
7adc2a00ea guiconfig: fix search function
The _update_jump_to_matches() function used when searching from the
jump dialog is using the re module without importing it. Fix that.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-02-08 16:24:05 -05:00
Torsten Rasmussen
a088970bc3 scripts: using pathlib for in get_toolchain() method
Followup commit on comment in #32003.

Using pathlib to avoid mixing of `/` and `\` if twister someday will be
able to run on Windows.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2021-02-05 06:48:18 -05:00
Anas Nashif
ce954d4769 twister: fix HW map fixtures parsing
We were not parsing fixtures from hardware map, so some tests are being
skipped.

Fixes #31948

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-02-05 06:36:14 -05:00