Commit graph

5615 commits

Author SHA1 Message Date
Ulf Magnusson
e8a077b6a3 scripts/dts: Remove redundant parentheses around 'if' conditions
Not needed in Python.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-07 10:39:37 +01:00
Ulf Magnusson
b278500d94 scripts/dts: Remove dead and confusing pwm/gpio-related code
- Maybe the code is trying to turn integers into single-element list,
   but it won't work (needs to be [value] rather than list(value)).
   Don't think you could ever end up with an integer either.

 - extract_controller() assumes that prop_values is a list, so
   reduced[...].get(prop) should be reduced[...][prop]. get() means it's
   okay for the key to be missing, which makes the code confusing.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-07 10:39:37 +01:00
Ulf Magnusson
d2985b31d8 scripts/dts: Remove dead and confusing code in get_phandles()
- The 'props' key is known to exist (and 'if enabled == ...:' would
    crash if it ever was missing)

  - 'd' is always a dictionary

  - There's no need to test if d['children'] is non-empty before looping
    through it

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-07 10:39:37 +01:00
Ulf Magnusson
23afd94704 scripts/dts: Remove dead and confusing code in get_all_compatibles()
- The 'props' key is known to exist (and 'if enabled == ...:' would
   crash if it ever was missing)

 - 'd' is always a dictionary

 - There's no need to test if d['children'] is non-empty before looping
   through it

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-07 10:39:37 +01:00
Ulf Magnusson
d26cb1fb55 scripts/dts: Remove redundant and confusing dict.get() calls
dict.get(key) signals to people reading the code that you're not sure
whether the key exists. It returns None if it doesn't.

When the key is known to exist, dict[key] should be used. This also
helps catch bugs by raising an exception if the key is missing.

Similarly, whether a key in a dict should be tested with

    if key in dict:

instead of with

    if dict.get(key):

The second version signals that you both want to make sure that the
exists and that it's truthy (e.g., non-empty). That's confusing if a
simple existence check was meant.

There seems to be a bug in output_keyvalue_lines() where

    fd.write("%s=%s\n" % (entry, defs[node].get(a)))

can end up writing '...=None'. Removing the .get() makes it throw an
exception instead. Keep the .get() for now and don't attempt to fix the
bug.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-07 10:39:37 +01:00
Ulf Magnusson
01db6469ca scripts/dts: Remove redundant dict.keys() calls
'x in dict' and 'for x in dict' in Python always uses the keys.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-07 10:39:37 +01:00
Andrew Boie
f753ae92fb gen_mmu_x86: remove unused function
Fixes #13096

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-02-06 14:34:20 -08:00
Andrew Boie
2d9bbdf5f3 x86: remove support for non-PAE page tables
PAE tables introduce the NX bit which is very desirable
from a security perspetive, back in 1995.

PAE tables are larger, but we are not targeting x86 memory
protection for RAM constrained devices.

Remove the old style 32-bit tables to make the x86 port
easier to maintain.

Renamed some verbosely named data structures, and fixed
incorrect number of entries for the page directory
pointer table.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-02-05 20:51:21 -08:00
Timon Baetz
4cc355d423 boards: hifive1: Added flash runner
HiFive1 flash runner using OpenOCD and GDB

Signed-off-by: Timon Baetz <timon.baetz@gmail.com>
2019-02-05 14:29:16 -06:00
Anas Nashif
c96c90acf7 sanitycheck: return build.log when we think we crashed
In some cases sanitycheck handles a build error that started with 'make
run' as a handler crash when it actually failed during building. Right
now we try to attach the handler.log to the XML output even if it did
not exist. Check for the file and if it was not found, go back to the
build.log

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-02-05 13:40:49 -05:00
Alberto Escolar Piedras
834b86f8d7 sanitycheck: coverage: Tolerate missing source files
When calling genhtml to generate a coverage report directly from
sanitycheck (-C option), call it with the "--ignore-errors source"
option, so it does not exit when it meets a missing file, but
instead it warns the user and continues.

Fixes #13014

Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
2019-02-03 15:28:53 -05:00
Maureen Helm
987313e102 scripts: west: Update pyocd runner to use unified tool subcommands
pyocd 0.14.0 merged its command-line tools into a unified pyocd tool
with subcommands. The separate command-line tools still remain, but are
deprecated. Update the pyocd runner to use the new unified pyocd tool
with subcommands.

Signed-off-by: Maureen Helm <maureen.helm@nxp.com>
2019-02-02 10:32:30 -05:00
Anas Nashif
12f737163f ci: run_ci.sh: make script run locally
Allow running script locally. For example, on a local tree with local
changes:

./scripts/ci/run_ci.sh -b master -r upstream  -l

This will be have the same way as in CI and creates a test manifest
based on changes.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-02-01 19:04:55 -05:00
Anas Nashif
8a51a4bf26 ci: do not abort if west is already initialised
We should not abort if west was initiaized, someone must have
initialized, i.e. locally and everything is already setup for us and
ready to go.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-02-01 19:04:55 -05:00
Kumar Gala
ce7ed18989 dts: Move CONFIG_SRAM_* & CONFIG_FLASH_* back to Kconfig
dts will now generate DT_SRAM_BASE_ADDRESS, DT_SRAM_SIZE,
DT_FLASH_BASE_ADDRESS, and DT_FLASH_SIZE defines.  Kconfig can utilize
these defines to set defaults for the CONFIG_ variants.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-02-01 10:16:06 -06:00
Carles Cufi
fa26ef02d2 doc: Add KCONFIG_DOC_MODE env var handling
Export a new KCONFIG_DOC_MODE environment variable when building the doc
and invoking Kconfig, so that the functions that expect a build folder
can accordingly return a hardcoded value.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-02-01 10:16:06 -06:00
Kumar Gala
87915cd36f dts: stop generating CONFIG_FLASH_LOAD_{OFFSET,SIZE}
As a step to completing removing use of Kconfig defines in dts files we
need to change how we configure/set CONFIG_FLASH_LOAD_{OFFSET,SIZE}.

Previously we would set them based on how the chosen property
'zephyr,code-partition' was set to.  If 'zephyr,code-partition' wasn't
set we would default the values to 0.  We had some generic overlay logic
which would define 'zephyr,code-partition' based on
CONFIG_BOOTLOADER_MCUBOOT.

Going forward if the DTS has 'zephyr,code-partition' set we will
generate DT_CODE_PARTITION_OFFSET & DT_CODE_PARTITION_SIZE defines.  We
will leave it to Kconfig to set CONFIG_FLASH_LOAD_OFFSET &
CONFIG_FLASH_LOAD_SIZE.

We introduce a python script that allows Kconfig to extract data from
the DTS and thus we can utilize DT_CODE_PARTITION_OFFSET and
DT_CODE_PARTITION_SIZE values to set defaults for
CONFIG_FLASH_LOAD_OFFSET & CONFIG_FLASH_LOAD_SIZE.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-02-01 10:16:06 -06:00
Andrew Boie
c253a686bf app_shmem: auto-initialize partitions
There are no longer per-partition initialization functions.
Instead, we iterate over all of them at boot to set up the
derived k_mem_partitions properly.

Some ARC-specific hacks that should never have been applied
have been removed from the userspace test.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-01-30 23:15:51 -05:00
Carles Cufi
109f80f142 west: runners: nrfjprog: Enable testing of nRF91
Enable the nRF91 suite of tests.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-01-30 20:16:27 +01:00
Carles Cufi
13ce9d94b9 west: runners: nrfjprog: Correct nRF91 unit test reference data
nRF91 only does `--sectorerase` (no UICR) and does not need to
enable pin reset (`--pinresetenable`).

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-01-30 20:16:27 +01:00
Ioannis Glaropoulos
a97dbec8c1 scripts: west: explicitly ask for --sectorerase for nRF9160_pca10090
When flashing nRF9160_pca10090 board with nrfjprog, explicitly
request for --sectorerase, instead of --sectoranduicrerase (used
when flashing boards with nRF52 SOC).

Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
2019-01-30 20:16:27 +01:00
Kumar Gala
7d52cb7d4a scripts/dts/extract: Rework to extract_{controller,cells}
Rework extract_{controller,cells} so they aren't recursive.  As part of
this change we take the flat cell array property and build a proper list
based on what the number of cells each element would represent.

Fixes #12724

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-01-30 09:41:10 -06:00
Kumar Gala
d5b58e1655 scripts/dts/extract: Move flattening of list to one place
Because of differents in dtc versions the output of a list property may
vary.  We would flatten the list of lists to a single list to match how
older dtc versions did things.  Rather than doing that flattening in
multiple places, move it to get_reduced() so we do it just once.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-01-30 09:41:10 -06:00
Anas Nashif
f29087ee1b sanitycheck: Handle data over UART from gcov
Make sure we capture data from gcov and do not timeout before all the
data has been captured. Also report on incomplete data capture.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-01-29 15:03:38 -05:00
Anas Nashif
a34f371e36 scripts: gen_gcov_files: report if data capture is not complete
If we get incomplete data, report on the screen.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-01-29 15:03:38 -05:00
Anas Nashif
47cf4bf544 sanitycheck: add option to point to gcov-tool
When building with older versions of the toolchain we need to use gcov
from the same tool set, otherwise data formar might not be compatible.

Added an option to point to the gcov tool from the same toolchain used
to build the code. By default the host gcov is used.

Relates to #12571

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-01-29 15:03:38 -05:00
Anas Nashif
a4b5a739d9 sanitycheck: link with --coverage only for native_posix
Link with --coverage only when building with the host toolchain.

Relates to #12571

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-01-29 15:03:38 -05:00
Sebastian Bøe
6d8d4442b0 mergehex: Improve the error feedback when merged hex files overlap
When merging hex files, and there is an overlap between the hex files,
display the hex file that failed to merge.

Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
2019-01-29 17:04:42 +01:00
Marti Bolivar
808028b46f scripts: west commands: add text for "west --help"
Although each of these command implementations has good output for
"west <command> --help", the "west --help" output is missing the
one-line descriptions.

Add them by modifying west-commands.yml to use a new 'help' key in
each command name. These need to be kept in sync with the Python
sources. For now just do that by copy/pasting. We could add fancy
logic to load the help from west-commands.yml later if we want.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2019-01-29 10:15:01 +01:00
Marti Bolivar
ecae1e9b17 west_commands: add information on running tests
Add information to the README on how to run the tests, and note that
they are part of Zephyr's CI.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2019-01-29 10:15:01 +01:00
Carles Cufi
89ee604836 scripts: west: Add support for nRF91 in nrfjprog
Extend the list with the new nRF91 SoC.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
2019-01-29 10:15:01 +01:00
Carles Cufi
6b00bc9357 west: Multiple CI fixes for the topic-west branch
In order to be able to merge the topic branch, we require a few
fixes to CI.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-01-29 10:15:01 +01:00
Marti Bolivar
63841df184 scripts: run_ci.sh: add pytest-based testing for west commands
Incorporate these into run_ci.sh the same way that btsim results were
done. This adds a dependency on pytest.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2019-01-29 10:15:01 +01:00
Marti Bolivar
9cecb5df4a scripts: run_ci.sh: remove dead code
Commit 88ece494 ("ci: do not build docs in main CI job") removed the
call to build_docs but not the function itself. That makes it
confusing to try to find where it's called; remove it.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2019-01-29 10:15:01 +01:00
Marti Bolivar
ab82264ace scripts: add west build, flash, and debug commands
West now supports a mechanism for extension commands. Use it to move
the command implementations that are tightly coupled with boards and
the zephyr build system back into the Zephyr repository.

This patch doesn't include test cases. Those will be moved over in a
subsequent patch.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2019-01-29 10:15:01 +01:00
Carles Cufi
6aa87b6f5b ci: Update to the new west model
The new west is able to initialize itself from an existing zephyr clone
without modifying it, adapt the CI script accordingly.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-01-29 10:15:01 +01:00
Anas Nashif
ca51ac916c ci: Use west to clone zephyr
West right now modifies the zephyr tree and points to master. In CI we
need to point to the pull-request HEAD instead, so so whatever shippable
does after cloning with west.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-01-29 10:15:01 +01:00
Carles Cufi
9b4eb37f38 scripts: remove west from scripts/
west will now be installed via pip in order for the bootstrapper to be
decoupled from the west runners.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-01-29 10:15:01 +01:00
Sebastian Bøe
361daf8ebd DeviceTree: Fix generated header comment
The header comment in 'generated_dts_board_unfixed.h' was including
'compatible' as a helpful text.

But something has been broken with how we extract 'compatible' because
it was being evaluated to 'q'. Giving a confusing header text:

Generated include file q

Since the mechanism is broken, and does not appear to be important, we
remove it.

Also, modernize how we generate multi-line strings.

Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
2019-01-25 03:02:33 -06:00
Sebastian Bøe
c23cc26c73 DeviceTree: Convert 'generated_dts_board.h' into a source file
extract_dts_includes.py has been generating DT output and then
concatenating it with fixup header files to create
'generated_dts_board.h'.

In this patch we instead introduce a source file named
'generated_dts_board.h' and have it \#include the appropriate DT
output and fixup files.

This results in a simpler system because users can now read
'generated_dts_board.h' as C source code to see how fixup files and
generated DT output relate to each other. Whereas before they would
have to either read documentation or python code to gain the same
understanding.

Also, it reduces the scope and complexity of one of our most bloated
python scripts, extract_dts_includes.py.

Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
2019-01-25 03:02:33 -06:00
Kumar Gala
f592c528f4 scripts/dts/extract: Add support for generating SPI GPIO CS defines
Add support to generation script to generate defines for GPIO CS
[WIP], need to clarify some issues on how CS's look when we mix GPIO-CS
and HW CS.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-01-23 18:12:24 -06:00
Kumar Gala
4bd4185a2c scripts/dts/extract: Add single extract to extract_{controller,cells}
Add the ability to extract just a single instance from extract_cells and
extract_controller.  This lets us pass in a prop_values like:

cs-gpios =  < 0x02 0x11 0x00 >, < 0x03 0x05 0x00 >;

and extract just index = 1 if we want < 0x03 0x05 0x00 >.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-01-23 18:12:24 -06:00
Kumar Gala
44fd0c5624 scripts/dts/extract: Move extract_{controller,cells} into globals
Move extract_cells and extract_controller functions into globals so they
can be utilized outside of just the main extract_dts_includes script.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-01-23 18:12:24 -06:00
Anas Nashif
939f15195d requirements: update to pyocd 0.15.0
Previous release had some issues with unicode and was failing to install
in our CI. Those issues were fixed in this release.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-01-23 15:42:40 -05:00
Kumar Gala
9948eb036c scripts/dts/extract: Don't warn for 'category' merge of 'required'
We get warnings like:
	foo.yaml('BAH') merge of property 'category': 'required'
	overwrites 'optional'

This warning isn't meaningful as its reasonable to change the 'category'
of a property from 'optional' to 'required'.  So don't warn in the case
anymore.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-01-23 06:17:06 -06:00
Sebastian Bøe
56d7471b0c sanitycheck: Error out when running sanitycheck from Windows
Running sanitycheck from Windows is not supported yet. Unfortunately,
the error message is obscure when a user is unaware of this and runs
it from Windows anyway.

This patch gives an easy to understand error message explaining this
issue.

Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
2019-01-22 07:44:06 -05:00
Anas Nashif
eecd90f1d8 ci: remove check-compliance script
This is now maintained outside of the tree in ci-scripts repo.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-01-18 09:49:56 -05:00
Kumar Gala
32cbc42ce6 scripts/dts/extract: fix handling of non-MMIO SPI flash
The Intel S1000 board has a SPI flash that isn't directly MMIO
addressable.  For this case we shouldn't generate CONFIG_FLASH_SIZE or
CONFIG_FLASH_BASE_ADDRESS.

We use the heuristic of assuming if the flash node #size-cells is 0, its
a SPI flash.  Than if the parent node (SPI controller) has only a single
reg element we assume its a non-MMIO addressable SPI flash.  If there is
more than one reg element we assume the last reg element is the MMIO
address region for access to the flash.

Fixes #12530

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-01-17 21:29:07 -05:00
Anas Nashif
74dbe338d4 sanitycheck: do not keep extending timeout
While trying to catch crashes after test completes, do not keep
extending the timeout which might lead to and endless loop and not
getting out based on set timeout.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-01-17 19:46:45 -05:00
Peter A. Bigot
bfad9721d2 kernel: remove k_alert API
This API was used in only one place in non-test code.  See whether we
can remove it.

Closes #12232

Signed-off-by: Peter A. Bigot <pab@pabigot.com>
2019-01-16 21:34:07 -05:00