Commit graph

5,899 commits

Author SHA1 Message Date
Kumar Gala
9d9b108dab scripts/dts: Sort output of DT_COMPAT_ defines
Sort the DT_COMPAT_ defines so comparing them from different builds is
easier.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-07-31 05:33:41 -04:00
Kumar Gala
4e7863dc41 dts: Make instance defines consistent
We generated a define for each instance to convey its existance of the
form:
	#define DT_<COMPAT>_<INSTANCE> 1

However we renamed all other instance defines to be of the form
DT_INST_<INSTANCE>_<FOO>.  To make things consistent we now generate a
define of the form:

	#define DT_INST_<INSTANCE>_<COMPAT> 1

We also now deprecate the DT_<COMPAT>_<INSTANCE> form and fixup all uses
to use the new form.

Fixes: #17650

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-07-30 17:10:31 -05:00
Anas Nashif
578ae40761 boards: remove quarl_se_c1000
This board and SoC was discontinued some time ago and is currently not
maintained in the zephyr tree.
Remove all associated configurations and variants from the tree.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-07-29 21:30:25 -07:00
Ulf Magnusson
7de2f4da14 scripts/dts: Add deprecation comments to old scripts
Makes it easier to understand what's going on.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-07-29 16:22:17 -04:00
Ulf Magnusson
62d5741476 dts: Add new DTS/binding parser
Add a new DTS/binding parser to scripts/dts/ for generating
generated_dts_board.conf and generated_dts_board_unfixed.h.

The old code is kept to generate some deprecated defines, using the
--deprecated-only flag. It will be removed later.

The new parser is implemented in three files in scripts/dts/:

dtlib.py:
  A low-level .dts parsing library. This is similar to devicetree.py in
  the old code, but is a general robust DTS parser that doesn't rely on
  preprocessing.

edtlib.py (e for extended):
  A library built on top of dtlib.py that brings together data from DTS
  files and bindings and creates Device instances with all the data for
  a device.

gen_defines.py:
  A script that uses edtlib.py to generate generated_dts_board.conf and
  generated_dts_board_unfixed.h. Corresponds to extract_dts_includes.py
  and the files in extract/ in the old code.

testdtlib.py:
  Test suite for dtlib.py. Can be run directly as a script.

testedtlib.py (uses test.dts and test-bindings/):
  Test suite for edtlib.py. Can be run directly as a script.

The test suites will be run automatically in CI.

The new code turns some things that were warnings (or not checked) in
the old code into errors, like missing properties that are specified
with 'category: required' in the binding for the node.

The code includes lots of documentation and tries to give helpful error
messages instead of Python errors.

Co-authored-by: Kumar Gala <kumar.gala@linaro.org>
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-07-29 16:22:17 -04:00
Ulf Magnusson
a72e451624 scripts/dts: Rename --deprecate-only to --deprecated-only
--deprecate-only sounded like a command to "only deprecate (something)"
to me at first. --deprecated-only might make it clearer that it's about
only generating deprecated stuff.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-07-29 16:22:17 -04:00
Jukka Rissanen
f95938da0f net: ppp: Initial support for point-to-point protocol
This implements ppp L2 component, LCP and IPCP modules.

Fixes #14034

Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
2019-07-29 10:24:46 +03:00
Andrew Boie
81ef42d2bc sanitycheck: simplify fault detection
Any fatal error will print "ZEPHYR FATAL ERROR" now, so
we don't have to maintain a set of strings in the
sanitycheck harness.py

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-07-25 15:06:58 -07:00
Andrew Boie
96571a8c40 kernel: rename NANO_ESF
This is now called z_arch_esf_t, conforming to our naming
convention.

This needs to remain a typedef due to how our offset generation
header mechanism works.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-07-25 15:06:58 -07:00
Andrew Boie
7875707a98 userspace: add kobject flag for drivers
This new flag will indicate that the kernel object represents
an instance of a device driver object.

Fixes: #14037

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-07-24 12:33:34 +02:00
Ulf Magnusson
0ec0c84808 dts: bindings: Remove unused 'version' field
No binding has anything but 'version: 0.1', and the code in scripts/dts/
never does anything with it except print a warning if it isn't there.
It's undocumented what it means.

I suspect it's overkill if it's meant to be the binding format version.
If we'd need to tell different versions from each other, we could change
some other minor thing in the format, and it probably won't be needed.

Remove the 'version' fields from the bindings and the warning from the
scripts/dts/ scripts.

The new device tree script will give an error when unknown fields appear
in bindings.

The deletion was done with

    git ls-files 'dts/bindings/*.yaml' | xargs sed -i '/^\s*version: /d'

Some blank lines at the beginning of bindings were removed as well.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-07-22 09:28:07 -04:00
Alexander Wachter
b4c5f4b32b linker: Add dtcm section for Cortex M7 MCUs
This commit adds a DTCM (Device Tightly Coupled Memory) section for
Cortex F7 MCUs. The Address and length is defined in the corresponding
device tree file.

Signed-off-by: Alexander Wachter <alexander.wachter@student.tugraz.at>
2019-07-19 10:05:46 +02:00
Wentong Wu
fa7ce3ff70 scripts: make mpu align in sections for code reloation
make mpu align inside sections instead of outside to avoid
overlap for code relocation feature.

Signed-off-by: Wentong Wu <wentong.wu@intel.com>
2019-07-18 11:09:48 -04:00
Carles Cufi
9754201e04 west: Load CMakeCache.txt as a UTF-8 encoded file
CMake has been using UTF-8 as an encoding for CMakeCache.txt since 3.2.

Fixes #17635

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-07-18 16:50:08 +02:00
Carles Cufi
3ad1f27efc cmake: Encode and decode in UTF-8 in build system
In order to make sure that the build works in folders that require a UTF
encoding, make sure that both CMake and the various Python scripts that
interact with each other on files use the same encoding, in this case
UTF-8.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-07-18 16:50:08 +02:00
Kumar Gala
1c6952c4f6 dts/extract: Mark DT_FLASH_AREA_*_LABEL defines as deprecated
We don't use the DT_FLASH_AREA_*_LABEL defines today so lets mark them
deprecated until we actually need something.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-07-15 13:40:33 -04:00
Kumar Gala
4d2625cd79 scripts/dts: Add option to generate only deprecated defines
Add a --deprecate-only flag to only generate the deprecated defines.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-07-15 13:40:07 -04:00
Andy Ross
50b78f08ea cmake: Remove check_link_map.py
This script is looking for a hyperspecific error (mismatched padding
when linking into two simultaneous output sections) that bit us once,
in an era where the linker scripts were less unified.  We haven't seen
it crop up since, and multiple platforms have changed the way they do
this anyway.

It's needless complexity.  Junk it.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2019-07-14 04:54:29 -07:00
Alberto Escolar Piedras
f8908cd0d8 sanitycheck: Add missing section to whitelist
In a3bea8872b (PR #16352)
_bt_br_channels_area was added to the code but not the
sanitycheck sections whitelist => Add it

Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
2019-07-12 17:08:25 -07:00
Alberto Escolar Piedras
010c609bba sanitycheck: update sections whitelist
In c20ff1150f
and 5f19c8160a
(PR #16897)
_bt_settings_area was replaced with _settings_handlers_area

Update sanitycheck sections whitelist accordingly

Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
2019-07-12 13:36:13 +02:00
Peter A. Bigot
98a8e293cb scripts: gen_cfb_font_header: correct command sanitization
The code to sanitize the generated font path by removing bindir
improperly stripped the first character of every argument when --bindir
was not provided, corrupting the command documentation when fonts are
generated manually.  Only sanitize if --bindir was provided with
non-empty content.

Signed-off-by: Peter A. Bigot <pab@pabigot.com>
2019-07-11 20:20:59 -07:00
Peter A. Bigot
8b4d529706 scripts: gen_cfb_font_header: correct last character
DELETE is not a printable character.

Signed-off-by: Peter A. Bigot <pab@pabigot.com>
2019-07-11 20:20:59 -07:00
Kumar Gala
31052837ed scripts/dts: cleanup 'generation' usage
* Remove dead code that referenced 'generation' but didn't do anything
  with it
* Replace looking at 'generation' with a simple check for property
  starting with # (for things like #address-cells, etc)  or ending in
  -map (for things like gpio-map) to skip

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-07-11 06:51:17 -04:00
Peter A. Bigot
79765b8ad7 scripts/dts: fix support for uint8-array property values
uint8-array is the name for what the devicetree specification calls a
bytestring.

The original parsing code treated square brackets as equivalent to
quotes or angle brackets, failing to interpret elements as hex-encoded.
This was a bug, corrected in this patch by processing content as a
sequence of hex-encoded bytes with arbitrary whitespace.

The original generating code emitted the property as individual
elements.  Replace that with generation of a single structure
initializer, which is more useful in cases where the length of a
property value may vary between nodes.

Signed-off-by: Peter A. Bigot <pab@pabigot.com>
2019-07-11 06:50:27 -04:00
Manivannan Sadhasivam
7ac767b880 scripts: dts: Match alpha numeric property values
For using alpha numeric property values in a devicetree node, we
need to match the values starts with a number. Current scenario will
return the value as a numeric literal if it starts with a number. This
will not work for a compatible like, "96b-ls-con" which is proposed in
issue #15598.

Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
2019-07-11 06:15:22 -04:00
Michael Scott
4ca54390f4 sanitycheck: handle west-flash dependency for device-testing
sanitycheck --help mentions that west-flash requires device-testing
to be enabled (and indeed it does because DeviceHandler will never
be called where west_flash option is used.  Let's generate an error
if west-flash is used w/o specifying device-testing.

Also cleanup help text which looks odd in both sanitycheck --help
and in the file itself.

Signed-off-by: Michael Scott <mike@foundries.io>
2019-07-10 08:13:57 -07:00
Michael Scott
421ce46f4b sanitycheck: add west-runner parameter
Several boards have multiple runners setup.  We need a way to specify
which runner to use with sanitycheck.  Introduce --west-runner option.

Signed-off-by: Michael Scott <mike@foundries.io>
2019-07-10 08:13:57 -07:00
Kumar Gala
8083f8a6b0 scripts/dts: deprecate DT_<COMPAT>_<INSTANCE ID>_BUS_<BUS>
We don't have any uses of this form of define so deprecate it for now.
If needed this should be DT_INST_<INSTANCE ID>_<COMPAT>_BUS_<BUS>.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-07-10 06:24:39 -05:00
Kumar Gala
81072b53a6 dts/scripts: mark all non DT_ prefix deprecated
Now that we've converted LED and SW to use DT_ prefix we can mark the
non-DT_ prefixed versions as deprecated.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-07-10 05:23:24 -05:00
Kumar Gala
5c78b93c6b scripts/dts: Mark 'non-S' forms of cell and controllers as deprecated
Now that we've converted all _GPIO_ to _GPIOS_ we can mark the _GPIO_
form as deprecated (same for _PWM_ / _PWMS_).

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-07-10 05:23:24 -05:00
Kumar Gala
e5d838081c dts: Fix handling of fixed-clock
A number of minor issues with the 'fixed-clock' support:
* Fix the #clock-cells to be 0
* Fix nxp_ke1xf.dtsi to set #clock-cells 0 and the clock reference to
  only be a phandle.
* Fix the generation script to only generate what it should for a
  'fixed-clock'

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-07-10 05:23:07 -05:00
Marc Herbert
c7633de038 sanitycheck: fix priority of --extra-args=CONFIG_ over testcase.yaml
sanitycheck takes any "extra_config" list found in the testcase.yaml
file and generates an "overlay" file from it. This file is placed in
the per-test build directory and passed to cmake/kconfig.cmake through a
-DOVERLAY_CONF= option set in the (also) generated sanity-out/Makefile.

This commit moves this generated config overlay to a subdirectory one
level down from the build directory, otherwise kconfig.cmake picks it
up *twice*: once from the -DOVERLAY_CONF= option already mentioned above
and a second time because kconfig.cmake scans the build directory and
blindly picks up ALL files ending with .conf[*]. The second pickup is
problematic because kconfig.cmake currently gives it the top precedence,
even higher than anything the user espressed with --extra-args=CONFIG_*

Here's a quick and simple demonstration of the issue fixed by this
commit:

  cd $ZEPHYR_BASE/samples/net/sockets/net_mgmt/
  sanitycheck -T. -p qemu_x86 -b -v # --extra-args=CONFIG_USERSPACE=y|n
  grep CONFIG_USERSPACE $(find sanity-out/ -name .config)

  .net_mgmt.kernelmode/zephyr/.config: # CONFIG_USERSPACE is not set
    .net_mgmt.usermode/zephyr/.config: CONFIG_USERSPACE=y

  grep 'Merged configuration' $(find sanity-out/ -name build.log)

Without this commit, attemps to override anything with
--extra-args=CONFIG_ are silently dropped on the floor.

For more background this issue was found while using the recipe in
commit message 4afcc0f8af

[*] picking up all .conf files is debatable but a much bigger debate
with backward compatibility implications. This small fix makes
absolutely zero difference to anyone or anything not using sanitycheck.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-07-09 19:16:30 -07:00
Andrew Boie
49cf4867c7 sanitycheck: select gcov tool based on target
Use PATH gcov if no SDK in use, or native-posix derived
targets are in use.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-07-09 10:20:31 +02:00
Wentong Wu
859ca42006 scripts: elf_helper: correct kernel object address check range
kernel object should not located in app_smem area.

Fixes: #17338.

Signed-off-by: Wentong Wu <wentong.wu@intel.com>
2019-07-08 13:28:13 -07:00
Marc Herbert
e5cedcae43 sanitycheck help: how to spot tests that were not run and just built
There are a few reasons why sanitycheck will only build a test and not
run it: list them in the developer guide. Also lists the --options that
provide that information and update their --help message.

A couple other --help fixes.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-07-08 08:07:57 -04:00
Marc Herbert
d7ca903b72 sanitycheck: replace cmake -H "not meant for public use" with -S
https://www.mail-archive.com/cmake-developers@cmake.org/msg16769.html

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-07-08 08:04:39 -04:00
Sebastian Bøe
b8d4a77ecd size_report: Don't crash when observing stripped symbols
When stripped symbols are present in the build size_report has crashed
due to it not being aware of this case.

This patch causes stripped symbols to be ignored instead, allowing
analysis of the non-stripped symbols.

It is left as future work to somehow display how much space stripped
symbols are taking up.

Stripped symbols can be present when third-party binaries are linked
into the build.

Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
2019-07-05 14:18:49 -04:00
Marc Herbert
9e57338efd sanitycheck: fix --jobs help message + other cosmetic fixes
The --jobs default was recently changed in commit 9f4f57eed3, update
its help message.

Add the hopefully last missing verbose("Spawning...") statement.

Fix comment updated in commit 095b82a301.

Replace two tags with whitespace.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-07-03 23:04:16 -07:00
Andrew Boie
1578ef71b1 sanitycheck: workaround file limits
sanitycheck is opening an insane number of file descriptors
simultaneously as it opens up communication pipes with
every test that supports emulation, on every emulated
board target.

Increase the resource limit on open files until this code
can be properly refactored.

Workaround for: #17239

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-07-03 16:49:21 -04:00
Andrew Boie
8047a6f4d0 sanitycheck: make coverage reports easier to do
- The --gcov-tool argument now has a reasonable default
  if the Zephyr SDK is in use.
- --coverage-platform, if unspecified, defaults to what
  was passed to --platform
- --coverage implies --enable-slow, so that tests with
  the 'slow' tag are built and run.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-07-02 22:56:14 -04:00
Andrew Boie
095b82a301 sanitycheck: correct timeout extension comment
This wasn't discussing why there is extra time for when
code coverage is enabled.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-07-02 22:54:24 -04:00
Andrew Boie
9f4f57eed3 sanitycheck: don't multiply CPU count
We have a number of timing sensitive tests which run
correctly on a much more frequent basis if the system
is not so heavily loaded. Instead of squeezing a few
more crumbs of performance by doubling the CPU count,
just use the number of CPUs reported by the system.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-07-02 22:51:24 -04:00
Andrew Boie
d62e229680 sanitycheck: extend coverage timeout
If GCOV coverage is enabled, the coverage dump happens after
"PROJECT EXECUTION SUCCESSFUL" is printed. In some cases,
the additional time added was not enough to capture all the
GCOV output on a heavily loaded system before the emulator
gets killed.

Ideally, the decision to kill the emulator needs to be smarter
and less race-prone, but that can wait for a future
enhancement.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-07-02 20:30:41 -04:00
Alberto Escolar Piedras
b91f37494b sanitycheck: Minor optimization in BinaryHandler
Do not search for the valgrind binary (which) if valgrind
was not selected to be used.

Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
2019-07-02 18:49:44 -04:00
Alberto Escolar Piedras
649368c018 sanitycheck: Record time used in BinaryHandler
So the time used to run boards which use the BinaryHandler can be
reported, record the time used from spawning the process until
it finnishes or is killed.
The BinaryHandler is used by the "native" boards, unit tests,
nsim and Renode.

Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
2019-07-02 18:49:44 -04:00
Andrzej Głąbek
bb8dddfa99 scripts: dts: Add extraction of clock frequency for fixed clock
This patch adds generation of `*_FIXED_CLOCK_FREQUENCY` macros for
clock consuming nodes that are provided with a fixed rate clock.

Signed-off-by: Andrzej Głąbek <andrzej.glabek@nordicsemi.no>
2019-06-28 09:45:35 -05:00
Andrzej Głąbek
c782ba82ae scripts: dts: Add special handling of '+' in 'str_to_label' function
This patch is needed in order to get compilable macros for compatibles
like "arm,cortex-m0+".

Signed-off-by: Andrzej Głąbek <andrzej.glabek@nordicsemi.no>
2019-06-28 07:32:01 -05:00
Marc Herbert
eefea9da9c gen_app_partitions: deterministic _and_ decreasing app_smem_*.ld
Commit 212ec9a29a / feature #14121 already ordered partitions by
decreasing size, however it was common in samples/userspace/shared_mem/
/sample.kernel.memory_protection.shared_mem for two partitions to have
the same size and be randomly ordered between them. This adds the
partition name as a second sort key.

Unlike previous attempt in commit 725abdf430 this doesn't use the
partition name as the first (and only) key and doesn't break the
decreasing size order.  Huge thanks to Sigvart Hovland for spotting this
in a post-merge but prompt code review.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-06-27 18:06:30 -07:00
Marc Herbert
85bc0d2fe5 Revert "gen_app_partitions.py: make generated/app_smem_*.ld files deterministic"
This reverts commit 725abdf430 which did get rid of randomness in the
order of the partition _names_ as claimed but regressed commit
212ec9a29a / feature #14121 and broke the previous size order which I
missed. Huge thanks to Sigvart Hovland for spotting this in a post-merge
but prompt code review. Proper fix in the next commit.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-06-27 18:06:30 -07:00
Marc Herbert
725abdf430 gen_app_partitions.py: make generated/app_smem_*.ld files deterministic
Dictionaries are not ordered in Python 3.5 and before, so building twice
in a row could lead to a different partition order, different
build/zephyr/include/generated/app_smem_*.ld files and different
binaries.

Fix with a minor change to the "for" loop in the output function:
make it iterate on sorted(partitions.items()) instead of the raw and
randomly ordered partitions dictionary.

It is easy to reproduce the issue even without downgrading to an
obsolete Python version; pick a test like samples/userspace/shared_mem/
and simply change the code to this:

--- a/scripts/gen_app_partitions.py
+++ b/scripts/gen_app_partitions.py
@@ -159,10 +159,12 @@ def parse_elf_file(partitions):
                     partitions[partition_name][SZ] += size

+import random
 def generate_final_linker(linker_file, partitions):
     string = linker_start_seq
     size_string = ''
-    for partition, item in sorted(partitions.items()):
+    for partition, item in sorted(partitions.items(),
+                                  key=lambda x: random.random()):
         string += data_template.format(partition)
         if LIB in item:
             for lib in item[LIB]:

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-06-27 07:18:09 -04:00