Commit graph

5,899 commits

Author SHA1 Message Date
Anas Nashif
8ebc67ee0d action: timeout on scripts taking too long
Some scripts might take very long to execute, kill those after 10s and
let the action continue.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-05-03 17:11:12 -04:00
Martí Bolívar
55e8807b08 runners: jlink: support thread_info_enabled
Automatically enable per-thread info in GDB if the binary and J-Link
support it.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-05-03 09:55:33 -05:00
Martí Bolívar
61996a35ac runners: jlink: extend 'nogui' support
Support this when debugging also.

Tweak the style for brevity also while we're here by propertizing the
supports_nogui method, etc.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-05-03 09:55:33 -05:00
Martí Bolívar
b733ddb48d runners: core: add thread_info_enabled property
This makes it easier for runners to check if the binary has thread
info support turned on, allowing automatic configuration of the
underlying tool to support threads, if possible.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-05-03 09:55:33 -05:00
Martí Bolívar
250b2133ee scripts: re-work BuildConfiguration boolean handling
Set options that are definitely true or false to True or False in the
options dict. Add a 'getboolean' method that also allows a fallback to
False in case the option is not mentioned in .config due to unmet
dependencies. This allows calling code to just ask about the option
they are interested in, even if the .config file doesn't mention the
option at all.

Propagate this to users within the runners package and 'west sign',
taking advantage of the new build_conf property.

Rename the 'bcfg' internal variable in sign.py to 'build_conf' to
match other source files that use BuildConfiguration instances, to
make it easier to grep for users.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-05-03 09:55:33 -05:00
Martí Bolívar
778a18cb1a runners: tests: don't patch BuildConfiguration
Instead of mocking out the BuildConfiguration class, just create its
input file and let the real class do the work.

This in turn exposes a bug in the way the board name is being pulled
out of the BuildConfiguration, which we fix to keep the tests passing.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-05-03 09:55:33 -05:00
Martí Bolívar
f31a8f67a9 runners: jlink: silence commander output by default
This is really verbose, and I doubt anyone cares unless there is a
problem. Keep it around when run as 'west -v flash' to allow for
debugging, though.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-05-03 09:55:33 -05:00
Martí Bolívar
6357770eef runners: jlink: rework --dt-flash handling
Defer loading .config until we really need it, when we are flashing a
binary. Pre-emptively loading it is wasted effort if we're flashing a
.hex, which has been the default behavior when possible since
dcaabb860f ("west: runners: jlink: prefer .hex over .bin").

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-05-03 09:55:33 -05:00
Martí Bolívar
7640c39505 runners: jlink: clean up commander script logging
Put it all in one log message rather than splitting it up.

This makes it look cleaner now that each log message is prefixed with
'runners.link:'.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-05-03 09:55:33 -05:00
Martí Bolívar
2d43f660d9 runners: jlink: fix --commander help
The default commander path is platform specific.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-05-03 09:55:33 -05:00
Martí Bolívar
069c18d3ad runners: jlink: replace version parsing code
JLink versions like 'V7.0a' do not conform to PEP 440 version
conventions; the 'a' part is used by PEP 440 compliant versions for
alphas. It gets parsed to a legacy type by the packaging library,
which always is treated as a lower value when compared with a
conforming version string.

To fix, get the version from the shared library distributed with the
JLink tools. This has the side benefit of making the code work on
Windows. That's merely a nice to have for -nogui 1 detection for now,
but will be essential in the next commit.

Reported-by: Jake Mercer <jake.mercer@civica.co.uk>
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-05-03 09:55:33 -05:00
Martí Bolívar
2fb7e9dc63 requirements-base.txt: add pylink
This will be used to deal with the Segger shared library in a portable
way in the jlink runner.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-05-03 09:55:33 -05:00
Martí Bolívar
aded635dfa runners: core: add build_conf property
Make it easier to get a BuildConfiguration from runner code.
Stash the result so it only has to be computed once.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-05-03 09:55:33 -05:00
Martí Bolívar
e1e1bdc20f runners: core: add **kwargs to other subprocess wrappers
Just to make these match check_output().

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-05-03 09:55:33 -05:00
Martí Bolívar
bbb17df9a2 runners: core: refactor helper for getting flash address
Separate the logic that gets the right address from .config
into its own helper.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-05-03 09:55:33 -05:00
Martí Bolívar
c2c8e8824e runners: core: try to improve --dt-flash option help
This option has existed since the beginning of the runners package,
which greatly predates the way DT is used in zephyr right now. It
never really worked the way I wanted it to but it's too much work to
fix it now. Try to improve the help a bit at least while I'm looking
at it again.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-05-03 09:55:33 -05:00
Martí Bolívar
47007338af runners: core: return shutil.which() value from require()
Make it easy to get the full path to a required binary.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-05-03 09:55:33 -05:00
Martí Bolívar
09f55767b1 runners: tests: fix os.path.isfile patch
Copy a fix from test_nrfjprog.py to the other runner test suites. The
current code will enter an infinite recursion if you hit the path
where os.path.isfile is called, since it's been patched to
os_path_isfile_patch in the calling context. The fix is to cache the
'real' version in the parent scope and call it directly as a fallback.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-05-03 09:55:33 -05:00
Anas Nashif
b669f0c017 ci: fix regex in guideline_check.py
Also match _ in file names.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-05-03 08:08:25 -05:00
Anas Nashif
184b786fe2 coccinelle: check reserved symbols based on a file
Use a file with reserved symbols instead of checking against a fixed
list of strings.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-04-30 22:09:43 -04:00
Jordan Yates
371a83b9d9 scripts: gen_handles.py: initialize handle.device
Initialize `handle.device` to `None` so that the assert after the
matching has a chance to catch errors. Without this, a failed match will
raise an exception on the line above as it attempts to get a property
that doesn't exist.

Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
2021-04-30 15:16:02 +02:00
Daniel Leung
a5ab1a7518 logging: add support for dictionary based logging
This adds dictionary based logging support. Dictionary based
logging is binary based where one big difference is that
static strings are stored as pointers instead of the whole
string. This results in reduced space requirements for
storing log messages in certain scenairos.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-04-28 22:25:42 +02:00
Gerard Marull-Paretas
dfcc5fd7cc doc: make Sphinx version specifier more flexible
Allow all compatible versions starting from 3.3, that is >=3.3.0,<4.0.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2021-04-27 21:38:39 -04:00
Gerard Marull-Paretas
57dc1eb9c4 doc: make breathe version specifier more flexible
Breathe project has released version 4.29.1, a fix release for the
issues found in 4.29.0. Relax version requirements by just skipping the
buggy version and staying to compatible releases.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2021-04-27 21:38:39 -04:00
Hou Zhiqiang
0700a24276 devicetree: Add DT_FOREACH_CHILD_STATUS_OKAY macro
The macro only iterates the enabled child nodes and invokes
provided macro for each node.

Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
2021-04-27 13:32:55 -04:00
Daniel Leung
1117169980 kernel: generate placeholders for kobj tables before final build
Due to the use of gperf to generate hash table for kobjects,
the addresses of these kobjects cannot change during the last
few phases of linking (especially between zephyr_prebuilt.elf
and zephyr.elf). Because of this, the gperf generated data
needs to be placed at the end of memory to avoid pushing symbols
around in memory. This prevents moving these generated blocks
to earlier sections, for example, pinned data section needed
for demand paging. So create placeholders for use in
intermediate linking to reserve space for these generated blocks.
Due to uncertainty on the size of these blocks, more space is
being reserved which could result in wasted space. Though, this
retains the use of hash table for faster lookup.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-04-27 13:32:00 -04:00
Daniel Leung
317dcd908f scripts: gen_kobject_list: no need to specify linker section
There is no need to specify linker section for variables in
the output. The compiled object file from the produced
source file will be further processed to have them renamed
anyway.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-04-27 13:32:00 -04:00
Daniel Leung
c3711b3446 kernel: change metadata for priv_stacks to const
The metadata array for priv_stacks doesn't change and
can be marked const so this can put in ROM.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-04-27 13:32:00 -04:00
Jingru Wang
a3469a0497 script: add ram and rom total size to json file
It's easier and safer to parse json file rather than extract
total size from stdout

Now the .json file has two keys:
{
    "symbols": {
       "children": [
           ...
	],
       "identifier": ":",
       "name": "root",
       "size": 2220
    },
    "total_size": 34272
}

Signed-off-by: Jingru Wang <jingru@synopsys.com>
2021-04-26 14:23:13 -04:00
Anas Nashif
e366d7948a ci: remove bsim from buildkit execution
We now have a dedicated action for this.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-04-23 21:40:10 -04:00
Anas Nashif
555145b17c twister: improve set balancing
Do calculation based on what is actually going to be run and evaluated
at runtime, ignore the cases we already know going to be skipped.

This fixes an issue where some sets would get majority of skips and
basically run nothing beside filtering.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-04-22 21:15:07 -04:00
Maciej Perkowski
d08ee92d26 twister: No skips for integration platforms in integration mode
All skips for platforms listed in the .yaml's integration_platforms
will be treated as errors in the integration mode ('-G/--integration').
This feature serves to guarantee a proper testing scope in CI as no
integration platform will be skipped silently.
Fixes #33874

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2021-04-22 20:10:26 +02:00
Martí Bolívar
9c229a417c devicetree: add DT_FOREACH_PROP_ELEM(node_id, prop, fn)
It can be convenient to "iterate" over the elements of a property, in
the same way it is convenient to "iterate" over enabled instances.

Add a new macro for doing this, along with a DT_INST_FOREACH_PROP_ELEM
variant.

This is likely to be more convenient than UTIL_LISTIFY or FOR_EACH in
some situations because:

- it handles inputs of any length
- compiler error messages will be shorter and more self-contained
- it is easier to use with phandle-array type properties, which
  require more complicated macro boilerplate when used with
  util_macro.h APIs

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-04-22 15:32:10 +02:00
Martí Bolívar
62d8d1a4c6 edtlib: infer bindings for dtlib.Type.PATH
We already have support for handling the Zephyr binding "path" type in
edtlib.Node._prop_val(), but the binding inference code isn't making
use of that. Handle this type as well, as it is just as convenient as
Type.PHANDLE and can be more idiomatic depending on the situation.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-04-22 15:32:10 +02:00
Martí Bolívar
6c5254321c edtlib: improve inferred binding error message
Print the type which caused the error to make debugging easier.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-04-22 15:32:10 +02:00
Martí Bolívar
760d00913a edtlib: fix 'is not' -> '!='
We should not be comparing one int to another with 'is' or 'is not'.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-04-22 15:32:10 +02:00
Martí Bolívar
74f95688d9 dtlib: add Type enum
Instead of hard-coding constants, use an IntEnum.

These is still a subclass of 'int', but is both easier to import and
easier to read during debugging.

For example, compare:

>>> Type.BYTES
<Type.BYTES: 1>

with:

>>> TYPE_BYTES
1

However, 'Type.BYTES == 1' is still True, and the enum values
otherwise behave like you would expect.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-04-22 15:32:10 +02:00
Nicolas Pitre
f97d12936e kernel: add an architecture specific structs header
Add the ability to define architecture specific structures, notably
the ability to extend struct _cpu with per-CPU arch-specific stuff that
can be accessed with _current_cpu->arch.* similarly to _current->arch.*
for per-thead architecture data.

This is opt-in for architectures that want to benefit from this,
otherwise empty defaults are provided. A placeholder for ARM64 is
included to show the pattern.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2021-04-21 09:03:47 -04:00
Torsten Rasmussen
3dd65a7663 runners: remove dependencies from runners
Today, there is a build target is added for each runner: flash, debug,
debugserver, attach.

And those runners will have a dependency to Zephyr logical target that
is built before invoking `west <runner>`.

This design has some flaws, mainly that additional dependencies directly
on the target will not be built when running `west <runner>` directly.
That generator expressions cannot be used for the DEPENDS argument.

Instead, the build target `<runner>` will not have any dependencies, and
will raise a build error if a dependency is added to the target.
Due to how `add_dependencies()` work, this must be done as a build time
check, and not configure time check.

`west <runner>` will invoke a build before executing the runner, and
this way ensure the build target is up-to-date, which again removes the
need for a dedicated `west_<runner>_target`.

It also minimizes the risk of developer errors, as developers no longer
need to consider the need for adding additional dependencies.
If a custom target is part of the default `all` build, then it's ensured
to be up-to-date.

Fixes: Issue reported on slack.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2021-04-19 15:35:30 +02:00
Maciej Perkowski
484ef6710d twister: Fix memory footprint reporting logic
With the previous logic memory footprint can only be saved in a report
if a given test/sample was fully executed and passed (built and run),
hence build-only tests were not providing these metrics. This commit
modifies the logic so that it is enough to have the build successful
to be able to get the memory footprint.

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2021-04-18 21:49:21 -04:00
Maciej Perkowski
73c5594573 twister: Fix ram/rom reporting
The info of rom/ram usage by an application was lost along the
way of data processing in twister. The commit add a line which
pass further these metrices as well.

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2021-04-18 21:49:21 -04:00
Jordan Yates
c6cc927506 scripts: gen_relocate_app.py: fix undefined _ADDR
With the addition of #34185, it is not longer gauranteed that for a
memory region `NAME` there exists a symbol `NAME_ADDR`. Use the linker
builtin function `ORIGIN` to instead directly get the start address of
the selected memory region.

Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
2021-04-16 14:44:10 -05:00
Anas Nashif
bc747e7167 tracing: remove cpu_stats in favor of thread runtime stats
Removing CONFIG_TRACING_CPU_STATS in favor of
CONFIG_THREAD_RUNTIME_STATS which provides per thread stats. The same
functionality is also available when Thread analyzer is enabled with the
runtime stats enabled.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-04-16 15:31:24 -04:00
Jorgen Kvalvaag
36d96d2cd9 twister: Allow deep folder scan
Upon discovery of a sample/testcase.yaml file, twister will disregard
further subfolders. Given the folder structure below, the testcase.yaml
will currently not be discovered:

samples/sample1/
samples/sample1/sample.yaml
samples/sample1/tests/
samples/sample1/tests/testcase.yaml

This is desirable to allow placing test code closer to sample code.

Signed-off-by: Jorgen Kvalvaag <jorgen.kvalvaag@nordicsemi.no>
2021-04-16 15:46:09 +02:00
Yasushi SHOJI
2b32e47a9a scripts: coccinelle: Fix parse error
The commit coccinelle/coccinelle@47bd4cae52 changed a behavior of
parsing right before coccinelle v1.1.0.  With the commit, the current
scripts under scripts/coccinelle/ errors out with:

   minus: parse error:
     File ".../zephyr/scripts/coccinelle/deref_null.cocci",
     line 25, column 42, charpos = 666
     around = '...',
     whole content =  (E != NULL && ...) ? <+...E->f@p1...+> : ...

I've already raised an issue upstream coccinelle/coccinelle#257.  But
Debian is already shipping v1.1.0 and we need a fix.

The proposed fix doesn't change the semantics, it just explicitly
states that the rule is an expression.

Signed-off-by: Yasushi SHOJI <yashi@spacecubics.com>
2021-04-14 14:18:02 -04:00
Detlev Zundel
e5f33bd740 scripts: size_report: Add handling of depth argument
The script already accepts a depth parameter to configure the output,
but that parameter was not used at all.  This commit adds the correct
handling by passing it to anytrees RenderTree iterator.

Signed-off-by: Detlev Zundel <dzu@member.fsf.org>
2021-04-14 07:05:58 -04:00
Anas Nashif
70dc6b40ff ci: pylint: relax similar lines rule
We are getting hits on similar imports in different files, relax that.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-04-13 16:33:18 -05:00
Jaxson Han
8af11d40d0 cmake: emu: Add arm fvp emulator
Add arm fvp emulator in order to use ninja run or west build -t run.
Add armfvp in order to run twister.

Set env ARMFVP_BIN_PATH before using it,
e.g. export ARMFVP_BIN_PATH=<path/to/fvp/dir>
NOTE: ARMFVP_BIN_PATH is the dir path.

Signed-off-by: Jaxson Han <jaxson.han@arm.com>
2021-04-13 07:47:44 -04:00
Håkon Øye Amundsen
c5c55ee47e scripts: size_report: fix bug where key is used as a dict
When printing the unassigned values the 'sym' variable is
used as a dict from which we try to get the 'name' value.
However, 'symbols['unassigned']' gives a list of keys, so
we get an 'TypeError' when trying to access ['name'] of
a string (the key).

Fix this issue by iterating over the values from
the 'symbols['unassigned']' dict instead.

Signed-off-by: Håkon Øye Amundsen <haakon.amundsen@nordicsemi.no>
2021-04-13 07:16:15 -04:00
Maciej Perkowski
9c6dfce0c0 twister: Add quarantine feature
Adds feature allowing to use yaml file with dictionaries defining
tests to be quarantined (extra arg "--quarantine-list FILENAME").
The dictionaries are validated according to the proper schema
and loaded.

A flat list is created containing quarantined configurations
(configuration = platform + scenario). Configurations under quarantine
are skipped and get "Quarantine" as a reason in the results reports.

A "comment" can be added to a quarantine entry in the quarantine yaml
with more details (e.g. issue #) and it will be also added to
the report.

The status of tests under quarantine can be verify if
`--quarantine-verify` is used in addition to
"--quarantine-list FILENAME". Using these args will make twister skip
all tests which are not on the quarantine list.

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2021-04-12 08:42:29 -04:00