Commit graph

3029 commits

Author SHA1 Message Date
Jordan Yates
371a83b9d9 scripts: gen_handles.py: initialize handle.device
Initialize `handle.device` to `None` so that the assert after the
matching has a chance to catch errors. Without this, a failed match will
raise an exception on the line above as it attempts to get a property
that doesn't exist.

Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
2021-04-30 15:16:02 +02:00
Daniel Leung
a5ab1a7518 logging: add support for dictionary based logging
This adds dictionary based logging support. Dictionary based
logging is binary based where one big difference is that
static strings are stored as pointers instead of the whole
string. This results in reduced space requirements for
storing log messages in certain scenairos.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-04-28 22:25:42 +02:00
Gerard Marull-Paretas
dfcc5fd7cc doc: make Sphinx version specifier more flexible
Allow all compatible versions starting from 3.3, that is >=3.3.0,<4.0.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2021-04-27 21:38:39 -04:00
Gerard Marull-Paretas
57dc1eb9c4 doc: make breathe version specifier more flexible
Breathe project has released version 4.29.1, a fix release for the
issues found in 4.29.0. Relax version requirements by just skipping the
buggy version and staying to compatible releases.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2021-04-27 21:38:39 -04:00
Hou Zhiqiang
0700a24276 devicetree: Add DT_FOREACH_CHILD_STATUS_OKAY macro
The macro only iterates the enabled child nodes and invokes
provided macro for each node.

Signed-off-by: Hou Zhiqiang <Zhiqiang.Hou@nxp.com>
2021-04-27 13:32:55 -04:00
Daniel Leung
1117169980 kernel: generate placeholders for kobj tables before final build
Due to the use of gperf to generate hash table for kobjects,
the addresses of these kobjects cannot change during the last
few phases of linking (especially between zephyr_prebuilt.elf
and zephyr.elf). Because of this, the gperf generated data
needs to be placed at the end of memory to avoid pushing symbols
around in memory. This prevents moving these generated blocks
to earlier sections, for example, pinned data section needed
for demand paging. So create placeholders for use in
intermediate linking to reserve space for these generated blocks.
Due to uncertainty on the size of these blocks, more space is
being reserved which could result in wasted space. Though, this
retains the use of hash table for faster lookup.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-04-27 13:32:00 -04:00
Daniel Leung
317dcd908f scripts: gen_kobject_list: no need to specify linker section
There is no need to specify linker section for variables in
the output. The compiled object file from the produced
source file will be further processed to have them renamed
anyway.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-04-27 13:32:00 -04:00
Daniel Leung
c3711b3446 kernel: change metadata for priv_stacks to const
The metadata array for priv_stacks doesn't change and
can be marked const so this can put in ROM.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2021-04-27 13:32:00 -04:00
Jingru Wang
a3469a0497 script: add ram and rom total size to json file
It's easier and safer to parse json file rather than extract
total size from stdout

Now the .json file has two keys:
{
    "symbols": {
       "children": [
           ...
	],
       "identifier": ":",
       "name": "root",
       "size": 2220
    },
    "total_size": 34272
}

Signed-off-by: Jingru Wang <jingru@synopsys.com>
2021-04-26 14:23:13 -04:00
Anas Nashif
e366d7948a ci: remove bsim from buildkit execution
We now have a dedicated action for this.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-04-23 21:40:10 -04:00
Anas Nashif
555145b17c twister: improve set balancing
Do calculation based on what is actually going to be run and evaluated
at runtime, ignore the cases we already know going to be skipped.

This fixes an issue where some sets would get majority of skips and
basically run nothing beside filtering.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-04-22 21:15:07 -04:00
Maciej Perkowski
d08ee92d26 twister: No skips for integration platforms in integration mode
All skips for platforms listed in the .yaml's integration_platforms
will be treated as errors in the integration mode ('-G/--integration').
This feature serves to guarantee a proper testing scope in CI as no
integration platform will be skipped silently.
Fixes #33874

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2021-04-22 20:10:26 +02:00
Martí Bolívar
9c229a417c devicetree: add DT_FOREACH_PROP_ELEM(node_id, prop, fn)
It can be convenient to "iterate" over the elements of a property, in
the same way it is convenient to "iterate" over enabled instances.

Add a new macro for doing this, along with a DT_INST_FOREACH_PROP_ELEM
variant.

This is likely to be more convenient than UTIL_LISTIFY or FOR_EACH in
some situations because:

- it handles inputs of any length
- compiler error messages will be shorter and more self-contained
- it is easier to use with phandle-array type properties, which
  require more complicated macro boilerplate when used with
  util_macro.h APIs

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-04-22 15:32:10 +02:00
Martí Bolívar
62d8d1a4c6 edtlib: infer bindings for dtlib.Type.PATH
We already have support for handling the Zephyr binding "path" type in
edtlib.Node._prop_val(), but the binding inference code isn't making
use of that. Handle this type as well, as it is just as convenient as
Type.PHANDLE and can be more idiomatic depending on the situation.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-04-22 15:32:10 +02:00
Martí Bolívar
6c5254321c edtlib: improve inferred binding error message
Print the type which caused the error to make debugging easier.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-04-22 15:32:10 +02:00
Martí Bolívar
760d00913a edtlib: fix 'is not' -> '!='
We should not be comparing one int to another with 'is' or 'is not'.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-04-22 15:32:10 +02:00
Martí Bolívar
74f95688d9 dtlib: add Type enum
Instead of hard-coding constants, use an IntEnum.

These is still a subclass of 'int', but is both easier to import and
easier to read during debugging.

For example, compare:

>>> Type.BYTES
<Type.BYTES: 1>

with:

>>> TYPE_BYTES
1

However, 'Type.BYTES == 1' is still True, and the enum values
otherwise behave like you would expect.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2021-04-22 15:32:10 +02:00
Nicolas Pitre
f97d12936e kernel: add an architecture specific structs header
Add the ability to define architecture specific structures, notably
the ability to extend struct _cpu with per-CPU arch-specific stuff that
can be accessed with _current_cpu->arch.* similarly to _current->arch.*
for per-thead architecture data.

This is opt-in for architectures that want to benefit from this,
otherwise empty defaults are provided. A placeholder for ARM64 is
included to show the pattern.

Signed-off-by: Nicolas Pitre <npitre@baylibre.com>
2021-04-21 09:03:47 -04:00
Torsten Rasmussen
3dd65a7663 runners: remove dependencies from runners
Today, there is a build target is added for each runner: flash, debug,
debugserver, attach.

And those runners will have a dependency to Zephyr logical target that
is built before invoking `west <runner>`.

This design has some flaws, mainly that additional dependencies directly
on the target will not be built when running `west <runner>` directly.
That generator expressions cannot be used for the DEPENDS argument.

Instead, the build target `<runner>` will not have any dependencies, and
will raise a build error if a dependency is added to the target.
Due to how `add_dependencies()` work, this must be done as a build time
check, and not configure time check.

`west <runner>` will invoke a build before executing the runner, and
this way ensure the build target is up-to-date, which again removes the
need for a dedicated `west_<runner>_target`.

It also minimizes the risk of developer errors, as developers no longer
need to consider the need for adding additional dependencies.
If a custom target is part of the default `all` build, then it's ensured
to be up-to-date.

Fixes: Issue reported on slack.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2021-04-19 15:35:30 +02:00
Maciej Perkowski
484ef6710d twister: Fix memory footprint reporting logic
With the previous logic memory footprint can only be saved in a report
if a given test/sample was fully executed and passed (built and run),
hence build-only tests were not providing these metrics. This commit
modifies the logic so that it is enough to have the build successful
to be able to get the memory footprint.

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2021-04-18 21:49:21 -04:00
Maciej Perkowski
73c5594573 twister: Fix ram/rom reporting
The info of rom/ram usage by an application was lost along the
way of data processing in twister. The commit add a line which
pass further these metrices as well.

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2021-04-18 21:49:21 -04:00
Jordan Yates
c6cc927506 scripts: gen_relocate_app.py: fix undefined _ADDR
With the addition of #34185, it is not longer gauranteed that for a
memory region `NAME` there exists a symbol `NAME_ADDR`. Use the linker
builtin function `ORIGIN` to instead directly get the start address of
the selected memory region.

Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
2021-04-16 14:44:10 -05:00
Anas Nashif
bc747e7167 tracing: remove cpu_stats in favor of thread runtime stats
Removing CONFIG_TRACING_CPU_STATS in favor of
CONFIG_THREAD_RUNTIME_STATS which provides per thread stats. The same
functionality is also available when Thread analyzer is enabled with the
runtime stats enabled.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-04-16 15:31:24 -04:00
Jorgen Kvalvaag
36d96d2cd9 twister: Allow deep folder scan
Upon discovery of a sample/testcase.yaml file, twister will disregard
further subfolders. Given the folder structure below, the testcase.yaml
will currently not be discovered:

samples/sample1/
samples/sample1/sample.yaml
samples/sample1/tests/
samples/sample1/tests/testcase.yaml

This is desirable to allow placing test code closer to sample code.

Signed-off-by: Jorgen Kvalvaag <jorgen.kvalvaag@nordicsemi.no>
2021-04-16 15:46:09 +02:00
Yasushi SHOJI
2b32e47a9a scripts: coccinelle: Fix parse error
The commit coccinelle/coccinelle@47bd4cae52 changed a behavior of
parsing right before coccinelle v1.1.0.  With the commit, the current
scripts under scripts/coccinelle/ errors out with:

   minus: parse error:
     File ".../zephyr/scripts/coccinelle/deref_null.cocci",
     line 25, column 42, charpos = 666
     around = '...',
     whole content =  (E != NULL && ...) ? <+...E->f@p1...+> : ...

I've already raised an issue upstream coccinelle/coccinelle#257.  But
Debian is already shipping v1.1.0 and we need a fix.

The proposed fix doesn't change the semantics, it just explicitly
states that the rule is an expression.

Signed-off-by: Yasushi SHOJI <yashi@spacecubics.com>
2021-04-14 14:18:02 -04:00
Detlev Zundel
e5f33bd740 scripts: size_report: Add handling of depth argument
The script already accepts a depth parameter to configure the output,
but that parameter was not used at all.  This commit adds the correct
handling by passing it to anytrees RenderTree iterator.

Signed-off-by: Detlev Zundel <dzu@member.fsf.org>
2021-04-14 07:05:58 -04:00
Anas Nashif
70dc6b40ff ci: pylint: relax similar lines rule
We are getting hits on similar imports in different files, relax that.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2021-04-13 16:33:18 -05:00
Jaxson Han
8af11d40d0 cmake: emu: Add arm fvp emulator
Add arm fvp emulator in order to use ninja run or west build -t run.
Add armfvp in order to run twister.

Set env ARMFVP_BIN_PATH before using it,
e.g. export ARMFVP_BIN_PATH=<path/to/fvp/dir>
NOTE: ARMFVP_BIN_PATH is the dir path.

Signed-off-by: Jaxson Han <jaxson.han@arm.com>
2021-04-13 07:47:44 -04:00
Håkon Øye Amundsen
c5c55ee47e scripts: size_report: fix bug where key is used as a dict
When printing the unassigned values the 'sym' variable is
used as a dict from which we try to get the 'name' value.
However, 'symbols['unassigned']' gives a list of keys, so
we get an 'TypeError' when trying to access ['name'] of
a string (the key).

Fix this issue by iterating over the values from
the 'symbols['unassigned']' dict instead.

Signed-off-by: Håkon Øye Amundsen <haakon.amundsen@nordicsemi.no>
2021-04-13 07:16:15 -04:00
Maciej Perkowski
9c6dfce0c0 twister: Add quarantine feature
Adds feature allowing to use yaml file with dictionaries defining
tests to be quarantined (extra arg "--quarantine-list FILENAME").
The dictionaries are validated according to the proper schema
and loaded.

A flat list is created containing quarantined configurations
(configuration = platform + scenario). Configurations under quarantine
are skipped and get "Quarantine" as a reason in the results reports.

A "comment" can be added to a quarantine entry in the quarantine yaml
with more details (e.g. issue #) and it will be also added to
the report.

The status of tests under quarantine can be verify if
`--quarantine-verify` is used in addition to
"--quarantine-list FILENAME". Using these args will make twister skip
all tests which are not on the quarantine list.

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2021-04-12 08:42:29 -04:00
Gerard Marull-Paretas
61b85b60d6 doc: allow up to breathe 4.28.x
It looks like latest release (4.29.0) breaks docs build.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2021-04-10 07:27:58 -04:00
Kumar Gala
d590e0d18d twister: Adjust platform selection logic for platform_allow
In CI we typically build for "default" platforms.  However some
testcases that set "platform_allow" may have no overlap with the default
platforms and thus thoses testcases will not get built.

Change the platform selection logic in these cases to set the platforms
to the list of allowed platforms ("platform_allow") in the testcase.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2021-04-09 14:54:34 -04:00
Dominik Ermel
96bfaffc02 runners: nrfjprog: Fix nrfjprog after changes to BuildConfiguration
The commit fixes problem with nrfjprog runner for west flash,
that has been introduced with latest changes to BuildConfiguration
class.

Signed-off-by: Dominik Ermel <dominik.ermel@nordicsemi.no>
2021-04-09 16:34:01 +02:00
Dominik Ermel
e12e377e72 scripts: runners: Modify BuildConfiguration._parse to iterate over file
The method has been changed to iterate directly over file instead of
preloading it.

Signed-off-by: Dominik Ermel <dominik.ermel@nordicsemi.no>
2021-04-08 16:38:28 +02:00
Dominik Ermel
0dfd191e04 scripts: west_commands: Remove no longer needed load_dot_config(path)
The functionality of the function is now covered by BuildConfiguration
class.

Signed-off-by: Dominik Ermel <dominik.ermel@nordicsemi.no>
2021-04-08 16:38:28 +02:00
Dominik Ermel
09980d6a50 scripts/west sign: Use BuildConfiguration instead of load_dot_config
The ImgtoolSigner.sign() is passed a BuildConfiguration object instance
that holds all of Kconfig options that have been defined for a build;
yet it has been reading the .config file once again, with the
load_dot_config, to check some of these options.
The commit replaces the code that has been using the load_dot_config
obtained data with the code that uses the BuildConfiguration object.

Signed-off-by: Dominik Ermel <dominik.ermel@nordicsemi.no>
2021-04-08 16:38:28 +02:00
Dominik Ermel
cabb7d69ee scripts: runners: Copy load_dot_config to BuildConfiguration.get()
The body of load_dot_config method has been reimplemented in
BuildConfiguration.get(), replacing the previous code.

Signed-off-by: Dominik Ermel <dominik.ermel@nordicsemi.no>
2021-04-08 16:38:28 +02:00
Dominik Ermel
185d695986 scripts: runners: Add BuildConfiguration.path attribute to class
The BuildConfiguration.path attribute has been added that stores path
to .config file which has been used as source for options collected to
BuildCOnfiguration object.

Signed-off-by: Dominik Ermel <dominik.ermel@nordicsemi.no>
2021-04-08 16:38:28 +02:00
Gerard Marull-Paretas
274f44d272 doc: remove docutils dependency
docutils is a core dependency of Sphinx. It is only used by some of our
custom Sphinx extensions.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2021-04-08 15:21:40 +02:00
Pieter De Gendt
b6979736ca modules: nanopb: introduce new module
Add the nanopb library and generator tools as a module.
Nanopb is a small code-size Protocol Buffers implementation in ansi C.
It is especially suitable for use in microcontrollers,
but fits any memory restricted system.

Nanopb home: https://jpa.kapsi.fi/nanopb/

Signed-off-by: Pieter De Gendt <pieter.degendt@basalte.be>
2021-04-08 14:57:55 +02:00
Flavio Ceolin
a632c52f50 ci: guideline: Run script to check rule 14.4
Run script to check violations to rule 14.4

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2021-04-08 07:56:28 -04:00
Flavio Ceolin
a025fb2747 coccinelle: Add script to check rule 14.4
Controlling expression shall have essentially boolean
type.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2021-04-08 07:56:28 -04:00
Flavio Ceolin
0bbe5e4fda ci: coccinelle: Suppress pylint false positive
Suppress pylint false positive on 'sh' library.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2021-04-08 07:56:28 -04:00
Flavio Ceolin
58fdc01ca0 ci: coccinelle: Add support to write to a file
Add parameter to write the script output to a file.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2021-04-08 07:56:28 -04:00
Flavio Ceolin
a869da55c6 scripts: coccinelle: Add guideline number on a script output
Add information about the guideline number in the script checking a
identifier length.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2021-04-08 07:56:28 -04:00
Flavio Ceolin
8552542f6e ci: Apply coccinelle scripts in git diffs
This scripts receives the same parameter of what_changed.py. And run
coccinelle scripts for code guideline compliance in the given git
commits. e.g: ./guideline_check.py --commits origin/master..HEAD

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2021-04-08 07:56:28 -04:00
Flavio Ceolin
e8fdac3be0 scripts: coccinelle: Check rule 21.2
This coccinelle script can check some violations for rule 21.2.
Currently it is checking the follow reserved names:

"remove", "rewind", "malloc", "free", "exp", "signal"

It can easily be extended in the future though.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>

Rebnase
2021-04-08 07:56:28 -04:00
Flavio Ceolin
0058c02252 scripts: coccinelle: Check rule 5.7
This coccinelle script can check some violations for rule 5.7.
It can identify things like:

struct device *device

But it is not capable to identify:

struct test { ... }

...

int test;

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2021-04-08 07:56:28 -04:00
Gerard Marull-Paretas
e14b44d720 scripts: ci: twister_ignore: remove old reference to .known-issues
Remove old reference to the .known-issues folder.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2021-04-08 09:55:16 +02:00
Gerard Marull-Paretas
5f42a3d40b doc: flexibilize sphinx-rtd-theme allowed versions
Allow versions starting from 0.5.2 up to < 1.0, be the next major
releases. Major releases tend to include breaking changes, so better
restrict such releases until they are manually tested for potential
regressions.

Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
2021-04-07 17:39:43 -04:00