Commit graph

2587 commits

Author SHA1 Message Date
Anas Nashif
dd3776cfa5 requirements: sort alphabetically
Sort packages alphabetically.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-08-23 13:50:16 -04:00
Kumar Gala
a0a576d384 scripts/dts/gen_defines.py: Fix generation for multiple IO Channels
If there is more than one IO Channel than generate a define with a
trailing index for the IO Channel.  This matches what we do for GPIOs
and PWMs.

So something like:
  DT_FOOBAR_IO_CHANNELS_CONTROLLER_0
  DT_FOOBAR_IO_CHANNELS_CONTROLLER_1
  ...
  DT_FOOBAR_IO_CHANNELS_CONTROLLER_<N>

Fixes #18352

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-08-19 14:48:44 -05:00
Håkon Øye Amundsen
208c4ecacb scripts: kconfig: normalize paths to avoid duplicate
There is a bug where non-normalized paths can introduce multiple entries
for the same file. E.g. '/1/2/../2.file' and '/1/2/3/../../2.file' would
both be listed, and end in a ninja error because multiple targets
generate the same file.

This commit fixes this issue by normalizing the paths.

Signed-off-by: Håkon Øye Amundsen <haakon.amundsen@nordicsemi.no>
2019-08-19 16:26:31 +02:00
Marti Bolivar
f7a4275328 scripts: west flash: fix --board-id for pyocd
Newer pyocd versions (specifically the 0.21.0 we have in our
requirements.txt) no longer support -b and have moved the same option
to -u. Keep up.

Fixes: #17554
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-08-19 11:03:03 +02:00
Marti Bolivar
08e0d1ff46 scripts: runners: improve hex file detection and output
This is a band-aid to make it more obvious to potential users of 'west
sign' and 'west flash' which hex file they are flashing, when they are
falling back on a binary file, and erroring out when a hex file does
not exist.

Fixes: #18201
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-08-16 18:42:41 +02:00
Kumar Gala
c484935fe5 scripts: openocd: Add error if hex file is missing when flashing
Add a check to make sure the hex file exists as that is what we utilize
in openocd to flash.  If its missing we report that its likely due to
not having CONFIG_BUILD_OUTPUT_HEX set.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-08-14 07:55:10 -05:00
Ulf Magnusson
c2d702b961 dts: dtlib: Fix parsing of character literals
This was allowed due to a misunderstanding:

    foo = 'x';

In reality, 'x' works like an integer literal, and is used like this:

    foo = < 'x' >;

Fix character literal parsing to match the C tools.

Also fix backslash escape parsing to match the C tools exactly
(get_escape_char() in util.c): \<char> should be turned into <char> if
<char> isn't recognized as a special escape character, instead of being
left alone. This fixes parsing of e.g. '\'' (a character literal with a
single quote in it).

Piggyback some more tests for weird property/node names.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-08-13 07:41:45 -05:00
Ulf Magnusson
7168118290 dts: edtlib: Fix outdated doc comment for _init_devices()
No longer takes the DT instance as a parameter.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-08-13 07:41:45 -05:00
Ulf Magnusson
06b746cc58 dts: dtlib/edtlib: Add a syntax-based type-checking system
Property type-checking has been pretty rudimentary until now, only
checking things like the length being divisible by 4 for 'type: array',
and strings being null-terminated. In particular, no checking was done
for 'type: uint8-array', letting

  jedec-id = < 0xc8 0x28 0x17 >;

slip through when

  jedec-id = [ 0xc8 0x28 0x17 ];

was intended.

Fix it by adding a syntax-based type checker:

  1. Add Property.type, which gives a high-level type for the property,
     derived from the markers added in the previous commit.

     This includes types like TYPE_EMPTY ('foo;'),
     TYPE_NUM ('foo = < 3 >;'), TYPE_BYTES ('foo = [ 01 02 ];'),
     TYPE_STRINGS ('foo = "bar", "baz"'),
     TYPE_PHANDLE ('foo = < &bar >;'), and TYPE_COMPOUND (everything not
     recognized).

     See the Property.type docstring in dtlib for more info.

  2. Use the high-level type in
     Property.to_num()/to_string()/to_node()/etc. to verify that the
     property was assigned in an expected way for the type.

     If the assignment looks bad, give a helpful error:

       expected property 'nums' on /foo/bar in some.dts to be assigned
       with 'nums = < (number) (number) ... >', not 'nums = "oops";'

Some other related changes are included as well:

  - There's a new Property.to_bytes() function that works like accessing
    Property.bytes, except with an added check for the value being
    assigned like 'foo = [ ... ]'.

    This function solves problems like the jedec-id one.

  - There's a new Property.to_path() function for fetching the
    referenced node for assignments like 'foo = &node;', with type
    checking. (Strings are accepted too, as long as they give the path
    to an existing node.)

    This function is used for /chosen and /aliases.

  - A new 'type: phandle' type can now be given in bindings, for
    properties that are assigned like 'foo = < &node >;'.

  - Property.__str__() now displays phandles and path references as they
    were written (e.g. '< &foo >' instead of '< 0x1 >', if the
    allocated phandle happened to be 1).

  - Property.to_num() and Property.to_nums() no longer take a 'length'
    parameter, because it makes no sense with the type checking.

  - The global dtlib.to_string() and dtlib.to_strings() functions were
    removed, because they're not that useful.

  - More tests were added, along with misc. minor cleanup in various
    places.

  - Probably other stuff I forgot.

The more strict type checking in dtlib indirectly makes some parts of
edtlib more strict as well (wherever Property.to_*() is used).

Fixes: #18131

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-08-13 07:41:45 -05:00
Ulf Magnusson
c9ac5e60e3 dts: dtlib: Remember the format of assignments
Previously, dtlib just stored the raw 'bytes' value for each property,
along with some markers in Property._markers for phandle and path
references.

Extend Property._markers to also remember where different data blocks
start, so that e.g.

    foo = <1 2 3>, "bar", [00 01];

can be reproduced as written.

Use the new information to reproduce properties as written in
Property.__str__(). This gives good test coverage as well, since the
test suite checks literal __str__() output.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-08-13 07:41:45 -05:00
Kumar Gala
b2ba44c1d7 scripts/dts/gen_defines.py: Fix generation for multiple PWMs
If there is more than one PWM than generate a define with a trailing
index for the PWM.  This matches what we do for GPIOs.

So something like:
  DT_PWM_LEDS_RED_PWM_LED_PWMS_CONTROLLER_0
  DT_PWM_LEDS_RED_PWM_LED_PWMS_CONTROLLER_1
  ...
  DT_PWM_LEDS_RED_PWM_LED_PWMS_CONTROLLER_<N>

Fixes #18171

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-08-12 17:42:15 +02:00
Marcin Sloniewski
b78bf8d6c1 requirements.txt: change package name "hub" to "git-spindle"
Package "hub" has been renamed some time ago to "git-spindle".
Recently package with the same name "hub" has been released on PyPi
page, which causes installing pip requirements.txt to fail since
it is trying to install different package.

Signed-off-by: Marcin Sloniewski <marcin.sloniewski@gmail.com>
2019-08-12 11:53:10 +02:00
Kumar Gala
480dcc1634 scripts/dts/gen_defines.py: Ignore 'gpio-controller' prop
We don't want any defines generated for the boolean
'gpio-controller'.  So skip it in write_props if we see it.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-08-09 14:10:40 -05:00
Kumar Gala
b0a6b3c0be scripts/dts/gen_defines.py: Ignore 'interrupt-controller' prop
We don't want any defines generated for the boolean
'interrupt-controller'.  So skip it in write_props if we see it.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-08-09 14:10:40 -05:00
Jim Paris
998d7a01dd scripts/dts: add basic test for IO channel property
Test the new io-channel support

Signed-off-by: Jim Paris <jim@jtan.com>
2019-08-09 13:16:09 -05:00
Jim Paris
67f53ba18f scripts/dts: Support 'io-channels' property just like 'pwms'
This is a direct search-and-replace copy of the PWM code.
The name is chosen to match Linux's iio-bindings.txt.

Signed-off-by: Jim Paris <jim@jtan.com>
2019-08-09 13:16:09 -05:00
Kumar Gala
5dd715b9c1 scripts/dts/edtlib.py: error check # and -map properties
Move when we early out for properties that start with # like
"#address-cells" or end with -map like "interrupt-map" to after we do
some error checking.  This allows us to check those properties at least
exist if they are required.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-08-09 11:49:16 -05:00
Ulf Magnusson
75391c4f2c dts: edtlib: Shorten some license headers
One line instead of three.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-08-09 00:03:19 -05:00
Ulf Magnusson
4985c214ef dts: edtlib: Shorten _init_{interrupts,gpios}() a bit
Also makes them consistent with _simple_phandle_val_list().

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-08-09 00:03:19 -05:00
Ulf Magnusson
f7f0e0adf9 dts: edtlib: Simplify 'category: optional' code in Device._prop_val()
Reuse Device.enabled().

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-08-09 00:03:19 -05:00
Ulf Magnusson
c0c8dd1b85 dts: edtlib: Improve naming in _check_binding()
'keys' is really a dictionary of options (like {"type": "int", ...}) for
a property. Calling it 'options' makes it clearer.

Also s/prop/prop_name/.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-08-09 00:03:19 -05:00
Kumar Gala
e328e56b30 dts/gen_defines.py: Use err() instead of _err()
_err comes from edtlib and we should be using err() inside of
gen_defines.py.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-08-08 23:53:22 -05:00
Marcin Niestroj
b08adcdd7e scripts: openocd: use hex file for flash command
Use hex file for flash command, instead of elf file. This allows to
flash signed firmware, which is not available in elf format, by
specifying --hex-file command line argument.

Signed-off-by: Marcin Niestroj <m.niestroj@grinn-global.com>
2019-08-08 22:16:50 +02:00
Marcin Niestroj
6778468c73 scripts: openocd: allow to overwrite elf file used to flash device
So far zephyr.elf file was hardcoded in cmake files. Remove it from
there and use cfg.elf_file from python, which can be overwritten by
specifying --elf-file command line option.

Signed-off-by: Marcin Niestroj <m.niestroj@grinn-global.com>
2019-08-08 22:16:50 +02:00
Ulf Magnusson
acf276f1df edtlib: Check that 'status' has one of the values from the DT spec.
dtlib is meant to be general and anything-goes re. property values, but
edtlib can be pickier. Check that all 'status' properties have one of
the values listed in the devicetree specification
(https://www.devicetree.org/specifications/), and error out otherwise.

This will get rid of the 'status = "ok"'s (should be "okay") that keep
cropping up.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-08-08 14:03:25 -05:00
Kumar Gala
8622c3415d scripts/dts: Fix detection of a GPIO specifier
We should only assume a GPIO specifier is either named <FOO>-gpios or
gpios.  Any other form like ngpios should not be considered a GPIO
specifier.  especially since 'ngpios' is the standard property name for
the number of gpio's that are implemented.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-08-07 13:15:51 -04:00
Kumar Gala
99769990ed scripts/dts/gen_defines: Cleanup write_prop for which props we process
We don't create props for compound types so we dont need to check for
'pwms', 'gpios', 'clocks'.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-08-07 13:15:51 -04:00
Ulf Magnusson
95deec1d24 scripts: edtlib: Reduce code duplication in phandle/value list parsing
Most of the logic for initializing 'clocks' and 'pwms' is the same and
can be shared. Add an EDT._simple_phandle_val_list() helper function for
initializing 'clocks', 'pwms', and any other properties on the form

    <foo>s = <phandle value phandle value ...>

where the nodes pointed at by the phandles are expected to have a
'#<foo>-cells' property.

This should make it easier to add similar properties.

There's still some code duplication in the classes (PWM, Clock, etc.),
but also some differences, so I'm wondering if requiring a class for
each might be okay. Maybe some more class-related stuff could be
factored out later.

Piggyback some related cleanup:

 - Have _phandle_val_list() take the name of the #foo-cells property
   instead of a function for fetching the value. The pattern is always
   the same.

 - Have _add_names() just take the "foo" part of "foo-names". Same
   pattern everywhere.

 - Clean up some redundant comments for stuff that's already documented
   in docstrings

 - Fix error messages printed by _named_cells() ("GPIOS controller" ->
   "GPIO controller", etc.)

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-08-07 11:48:53 -04:00
Piotr Zierhoffer
a3ddc09abf build: west: Update the error message when not in west dir
Zephyr verifies if it is built from a west-initialized directory.
The message changed in west@c08061cef1c7b0e19a58a82db731098e2f4bba4a.

Closes #18034.

Signed-off-by: Piotr Zierhoffer <pzierhoffer@antmicro.com>
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-08-06 19:13:24 +02:00
Paul Sokolovsky
04da7eaf07 scripts: dts: Consistently open text files with utf-8.
Zephyr codebase standardizes in UTF-8 as file encoding. To
accommodate this, we explicitly pass encoding="utf-8" to Python's
open() function, to be independent of any locale setting of a
particular system (e.g., CI/build systems oftentimes have "C",
i.e. ASCII-only, locale). In a few places, we lacked this parameter,
so add it consistently.

Signed-off-by: Paul Sokolovsky <paul.sokolovsky@linaro.org>
2019-08-05 17:47:57 -04:00
Marti Bolivar
7118d084c9 scripts: un-break test_nrfjprog.py
The pytest.raises context manager is now returning an ExceptionInfo
whose str() doesn't contain the str() of the underlying exception
object. Take str(e.value) directly to make sure we're looking at the
exception string.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-08-05 15:17:45 +02:00
Marti Bolivar
2b3d9df1d7 west: require v0.6.0 or higher
The main change is the elimination of the bootstrapper, a design flaw
/ misfeature.

Update the documentation to be compatible with the 0.6.x releases as
well. This has to be done atomically, as there were incompatible
changes.  Make use of the versionchanged and versionadded directives
to begin keeping track of how these APIs are evolving.

(Note that west 0.6.0 will remain compatible with the extension
commands in Zephyr v1.14 LTS as long as that is still alive. This
change is targeted towards Zephyr 2.0 users.)

This requires a bump in the shippable container and allows us to
simplify the west_commands test procedure.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-08-05 15:17:45 +02:00
Ulf Magnusson
fe2d858b5f scripts: dts: Add test for multiple binding directories
Add two bindings

    test-bindings/multidir.yaml
    test-bindings-2/multidir.yaml

and a new test-multidir.dts with two nodes that use them.

Verify that the two bindings were found by checking the
Device.binding_path attribute for the two device nodes.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-08-03 14:17:49 -04:00
Michael Scott
b8909439bf scripts: dts: support multiple binding dirs in new scripts
gen_defines.py and edtlib.py were recently added in
commit 62d5741476 ("dts: Add new DTS/binding parser").

The old extract_dts_includes.py script allowed for multiple
dts bindings dirs.  Let's add that functionality to the new
scripts as well.

Signed-off-by: Michael Scott <mike@foundries.io>
2019-08-03 14:17:49 -04:00
Ulf Magnusson
5d0db517b9 dts: riscv: Add sifive,plic-1.0.0 binding and fix riscv,ndev values
Add a new sifive,plic-1.0.0 binding that inherits from the riscv,plic0
binding. The new binding adds a required riscv,ndev property, which
gives the number of external interrupts supported.

Use the new binding for microsemi-miv.dtsi (with a value of 31 for
riscv,ndev, from http://www.actel.com/ipdocs/MiV_RV32IMAF_L1_AHB_HB.pdf)
and riscv32-fe310.dtsi (which already assigns riscv,ndev).

Also remove a spurious riscv,ndev assignment from
riscv32-litex-vexriscv.dtsi.

Also make edtlib and the old scripts/dts/ scripts replace '.' in
compatible strings with '_' when generating identifiers.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-08-02 11:44:09 +02:00
Kumar Gala
11e91a83c1 scripts/sanitycheck: optimize disk usage for --save-tests
If we are doing a nightly build we can utilize a large (over 50G) of
disk space just to generate the list of tests to build.  We need to
optimize this so as we finish building the initial pass we clean up
as we go and only keep around the files we need (like .config,
generated_dts_board.conf, CMakeCache.txt, etc).

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-07-31 18:46:32 -04:00
Peter A. Bigot
9a0d9e3af0 scripts: gen_cfb_font_header: modify to replicate cfb fonts
The content of subsys/fb/cfb_fonts cannot be replicated by the existing
script due to lack of positioning options and use of a full-color frame
buffer, which affects the generated bitmap.  Switch to the solution used
in the original script, add the required options, and document the
process of regenerating the fonts.

This commit also determines the required bounding box for the glyphs to
be sure that the user-provided value is sufficient to avoid partial
characters.  Ideally the calculated width and height would be used for
font characters, but this would require significant restructuring of the
script to make calculated values available at the point where the
arguments are used to produce output.

Signed-off-by: Peter A. Bigot <pab@pabigot.com>
2019-07-31 05:43:50 -04:00
Peter A. Bigot
f14b19b00a scripts: gen_cfb_font_header: remove guard and make font data private
The output of this script is intended to be put into an implementation
file, where the font data is accessed by index to an array maintained by
the linker script.  There is no need for protection against multiple
includes, and the font data array should not be a global symbol.

Signed-off-by: Peter A. Bigot <pab@pabigot.com>
2019-07-31 05:43:50 -04:00
Kumar Gala
9d9b108dab scripts/dts: Sort output of DT_COMPAT_ defines
Sort the DT_COMPAT_ defines so comparing them from different builds is
easier.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-07-31 05:33:41 -04:00
Kumar Gala
4e7863dc41 dts: Make instance defines consistent
We generated a define for each instance to convey its existance of the
form:
	#define DT_<COMPAT>_<INSTANCE> 1

However we renamed all other instance defines to be of the form
DT_INST_<INSTANCE>_<FOO>.  To make things consistent we now generate a
define of the form:

	#define DT_INST_<INSTANCE>_<COMPAT> 1

We also now deprecate the DT_<COMPAT>_<INSTANCE> form and fixup all uses
to use the new form.

Fixes: #17650

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-07-30 17:10:31 -05:00
Anas Nashif
578ae40761 boards: remove quarl_se_c1000
This board and SoC was discontinued some time ago and is currently not
maintained in the zephyr tree.
Remove all associated configurations and variants from the tree.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-07-29 21:30:25 -07:00
Ulf Magnusson
7de2f4da14 scripts/dts: Add deprecation comments to old scripts
Makes it easier to understand what's going on.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-07-29 16:22:17 -04:00
Ulf Magnusson
62d5741476 dts: Add new DTS/binding parser
Add a new DTS/binding parser to scripts/dts/ for generating
generated_dts_board.conf and generated_dts_board_unfixed.h.

The old code is kept to generate some deprecated defines, using the
--deprecated-only flag. It will be removed later.

The new parser is implemented in three files in scripts/dts/:

dtlib.py:
  A low-level .dts parsing library. This is similar to devicetree.py in
  the old code, but is a general robust DTS parser that doesn't rely on
  preprocessing.

edtlib.py (e for extended):
  A library built on top of dtlib.py that brings together data from DTS
  files and bindings and creates Device instances with all the data for
  a device.

gen_defines.py:
  A script that uses edtlib.py to generate generated_dts_board.conf and
  generated_dts_board_unfixed.h. Corresponds to extract_dts_includes.py
  and the files in extract/ in the old code.

testdtlib.py:
  Test suite for dtlib.py. Can be run directly as a script.

testedtlib.py (uses test.dts and test-bindings/):
  Test suite for edtlib.py. Can be run directly as a script.

The test suites will be run automatically in CI.

The new code turns some things that were warnings (or not checked) in
the old code into errors, like missing properties that are specified
with 'category: required' in the binding for the node.

The code includes lots of documentation and tries to give helpful error
messages instead of Python errors.

Co-authored-by: Kumar Gala <kumar.gala@linaro.org>
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-07-29 16:22:17 -04:00
Ulf Magnusson
a72e451624 scripts/dts: Rename --deprecate-only to --deprecated-only
--deprecate-only sounded like a command to "only deprecate (something)"
to me at first. --deprecated-only might make it clearer that it's about
only generating deprecated stuff.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-07-29 16:22:17 -04:00
Jukka Rissanen
f95938da0f net: ppp: Initial support for point-to-point protocol
This implements ppp L2 component, LCP and IPCP modules.

Fixes #14034

Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
2019-07-29 10:24:46 +03:00
Andrew Boie
81ef42d2bc sanitycheck: simplify fault detection
Any fatal error will print "ZEPHYR FATAL ERROR" now, so
we don't have to maintain a set of strings in the
sanitycheck harness.py

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-07-25 15:06:58 -07:00
Andrew Boie
96571a8c40 kernel: rename NANO_ESF
This is now called z_arch_esf_t, conforming to our naming
convention.

This needs to remain a typedef due to how our offset generation
header mechanism works.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-07-25 15:06:58 -07:00
Andrew Boie
7875707a98 userspace: add kobject flag for drivers
This new flag will indicate that the kernel object represents
an instance of a device driver object.

Fixes: #14037

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-07-24 12:33:34 +02:00
Ulf Magnusson
0ec0c84808 dts: bindings: Remove unused 'version' field
No binding has anything but 'version: 0.1', and the code in scripts/dts/
never does anything with it except print a warning if it isn't there.
It's undocumented what it means.

I suspect it's overkill if it's meant to be the binding format version.
If we'd need to tell different versions from each other, we could change
some other minor thing in the format, and it probably won't be needed.

Remove the 'version' fields from the bindings and the warning from the
scripts/dts/ scripts.

The new device tree script will give an error when unknown fields appear
in bindings.

The deletion was done with

    git ls-files 'dts/bindings/*.yaml' | xargs sed -i '/^\s*version: /d'

Some blank lines at the beginning of bindings were removed as well.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-07-22 09:28:07 -04:00
Alexander Wachter
b4c5f4b32b linker: Add dtcm section for Cortex M7 MCUs
This commit adds a DTCM (Device Tightly Coupled Memory) section for
Cortex F7 MCUs. The Address and length is defined in the corresponding
device tree file.

Signed-off-by: Alexander Wachter <alexander.wachter@student.tugraz.at>
2019-07-19 10:05:46 +02:00
Wentong Wu
fa7ce3ff70 scripts: make mpu align in sections for code reloation
make mpu align inside sections instead of outside to avoid
overlap for code relocation feature.

Signed-off-by: Wentong Wu <wentong.wu@intel.com>
2019-07-18 11:09:48 -04:00
Carles Cufi
9754201e04 west: Load CMakeCache.txt as a UTF-8 encoded file
CMake has been using UTF-8 as an encoding for CMakeCache.txt since 3.2.

Fixes #17635

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-07-18 16:50:08 +02:00
Carles Cufi
3ad1f27efc cmake: Encode and decode in UTF-8 in build system
In order to make sure that the build works in folders that require a UTF
encoding, make sure that both CMake and the various Python scripts that
interact with each other on files use the same encoding, in this case
UTF-8.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-07-18 16:50:08 +02:00
Kumar Gala
1c6952c4f6 dts/extract: Mark DT_FLASH_AREA_*_LABEL defines as deprecated
We don't use the DT_FLASH_AREA_*_LABEL defines today so lets mark them
deprecated until we actually need something.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-07-15 13:40:33 -04:00
Kumar Gala
4d2625cd79 scripts/dts: Add option to generate only deprecated defines
Add a --deprecate-only flag to only generate the deprecated defines.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-07-15 13:40:07 -04:00
Andy Ross
50b78f08ea cmake: Remove check_link_map.py
This script is looking for a hyperspecific error (mismatched padding
when linking into two simultaneous output sections) that bit us once,
in an era where the linker scripts were less unified.  We haven't seen
it crop up since, and multiple platforms have changed the way they do
this anyway.

It's needless complexity.  Junk it.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2019-07-14 04:54:29 -07:00
Alberto Escolar Piedras
f8908cd0d8 sanitycheck: Add missing section to whitelist
In a3bea8872b (PR #16352)
_bt_br_channels_area was added to the code but not the
sanitycheck sections whitelist => Add it

Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
2019-07-12 17:08:25 -07:00
Alberto Escolar Piedras
010c609bba sanitycheck: update sections whitelist
In c20ff1150f
and 5f19c8160a
(PR #16897)
_bt_settings_area was replaced with _settings_handlers_area

Update sanitycheck sections whitelist accordingly

Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
2019-07-12 13:36:13 +02:00
Peter A. Bigot
98a8e293cb scripts: gen_cfb_font_header: correct command sanitization
The code to sanitize the generated font path by removing bindir
improperly stripped the first character of every argument when --bindir
was not provided, corrupting the command documentation when fonts are
generated manually.  Only sanitize if --bindir was provided with
non-empty content.

Signed-off-by: Peter A. Bigot <pab@pabigot.com>
2019-07-11 20:20:59 -07:00
Peter A. Bigot
8b4d529706 scripts: gen_cfb_font_header: correct last character
DELETE is not a printable character.

Signed-off-by: Peter A. Bigot <pab@pabigot.com>
2019-07-11 20:20:59 -07:00
Kumar Gala
31052837ed scripts/dts: cleanup 'generation' usage
* Remove dead code that referenced 'generation' but didn't do anything
  with it
* Replace looking at 'generation' with a simple check for property
  starting with # (for things like #address-cells, etc)  or ending in
  -map (for things like gpio-map) to skip

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-07-11 06:51:17 -04:00
Peter A. Bigot
79765b8ad7 scripts/dts: fix support for uint8-array property values
uint8-array is the name for what the devicetree specification calls a
bytestring.

The original parsing code treated square brackets as equivalent to
quotes or angle brackets, failing to interpret elements as hex-encoded.
This was a bug, corrected in this patch by processing content as a
sequence of hex-encoded bytes with arbitrary whitespace.

The original generating code emitted the property as individual
elements.  Replace that with generation of a single structure
initializer, which is more useful in cases where the length of a
property value may vary between nodes.

Signed-off-by: Peter A. Bigot <pab@pabigot.com>
2019-07-11 06:50:27 -04:00
Manivannan Sadhasivam
7ac767b880 scripts: dts: Match alpha numeric property values
For using alpha numeric property values in a devicetree node, we
need to match the values starts with a number. Current scenario will
return the value as a numeric literal if it starts with a number. This
will not work for a compatible like, "96b-ls-con" which is proposed in
issue #15598.

Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
2019-07-11 06:15:22 -04:00
Michael Scott
4ca54390f4 sanitycheck: handle west-flash dependency for device-testing
sanitycheck --help mentions that west-flash requires device-testing
to be enabled (and indeed it does because DeviceHandler will never
be called where west_flash option is used.  Let's generate an error
if west-flash is used w/o specifying device-testing.

Also cleanup help text which looks odd in both sanitycheck --help
and in the file itself.

Signed-off-by: Michael Scott <mike@foundries.io>
2019-07-10 08:13:57 -07:00
Michael Scott
421ce46f4b sanitycheck: add west-runner parameter
Several boards have multiple runners setup.  We need a way to specify
which runner to use with sanitycheck.  Introduce --west-runner option.

Signed-off-by: Michael Scott <mike@foundries.io>
2019-07-10 08:13:57 -07:00
Kumar Gala
8083f8a6b0 scripts/dts: deprecate DT_<COMPAT>_<INSTANCE ID>_BUS_<BUS>
We don't have any uses of this form of define so deprecate it for now.
If needed this should be DT_INST_<INSTANCE ID>_<COMPAT>_BUS_<BUS>.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-07-10 06:24:39 -05:00
Kumar Gala
81072b53a6 dts/scripts: mark all non DT_ prefix deprecated
Now that we've converted LED and SW to use DT_ prefix we can mark the
non-DT_ prefixed versions as deprecated.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-07-10 05:23:24 -05:00
Kumar Gala
5c78b93c6b scripts/dts: Mark 'non-S' forms of cell and controllers as deprecated
Now that we've converted all _GPIO_ to _GPIOS_ we can mark the _GPIO_
form as deprecated (same for _PWM_ / _PWMS_).

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-07-10 05:23:24 -05:00
Kumar Gala
e5d838081c dts: Fix handling of fixed-clock
A number of minor issues with the 'fixed-clock' support:
* Fix the #clock-cells to be 0
* Fix nxp_ke1xf.dtsi to set #clock-cells 0 and the clock reference to
  only be a phandle.
* Fix the generation script to only generate what it should for a
  'fixed-clock'

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-07-10 05:23:07 -05:00
Marc Herbert
c7633de038 sanitycheck: fix priority of --extra-args=CONFIG_ over testcase.yaml
sanitycheck takes any "extra_config" list found in the testcase.yaml
file and generates an "overlay" file from it. This file is placed in
the per-test build directory and passed to cmake/kconfig.cmake through a
-DOVERLAY_CONF= option set in the (also) generated sanity-out/Makefile.

This commit moves this generated config overlay to a subdirectory one
level down from the build directory, otherwise kconfig.cmake picks it
up *twice*: once from the -DOVERLAY_CONF= option already mentioned above
and a second time because kconfig.cmake scans the build directory and
blindly picks up ALL files ending with .conf[*]. The second pickup is
problematic because kconfig.cmake currently gives it the top precedence,
even higher than anything the user espressed with --extra-args=CONFIG_*

Here's a quick and simple demonstration of the issue fixed by this
commit:

  cd $ZEPHYR_BASE/samples/net/sockets/net_mgmt/
  sanitycheck -T. -p qemu_x86 -b -v # --extra-args=CONFIG_USERSPACE=y|n
  grep CONFIG_USERSPACE $(find sanity-out/ -name .config)

  .net_mgmt.kernelmode/zephyr/.config: # CONFIG_USERSPACE is not set
    .net_mgmt.usermode/zephyr/.config: CONFIG_USERSPACE=y

  grep 'Merged configuration' $(find sanity-out/ -name build.log)

Without this commit, attemps to override anything with
--extra-args=CONFIG_ are silently dropped on the floor.

For more background this issue was found while using the recipe in
commit message 4afcc0f8af

[*] picking up all .conf files is debatable but a much bigger debate
with backward compatibility implications. This small fix makes
absolutely zero difference to anyone or anything not using sanitycheck.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-07-09 19:16:30 -07:00
Andrew Boie
49cf4867c7 sanitycheck: select gcov tool based on target
Use PATH gcov if no SDK in use, or native-posix derived
targets are in use.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-07-09 10:20:31 +02:00
Wentong Wu
859ca42006 scripts: elf_helper: correct kernel object address check range
kernel object should not located in app_smem area.

Fixes: #17338.

Signed-off-by: Wentong Wu <wentong.wu@intel.com>
2019-07-08 13:28:13 -07:00
Marc Herbert
e5cedcae43 sanitycheck help: how to spot tests that were not run and just built
There are a few reasons why sanitycheck will only build a test and not
run it: list them in the developer guide. Also lists the --options that
provide that information and update their --help message.

A couple other --help fixes.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-07-08 08:07:57 -04:00
Marc Herbert
d7ca903b72 sanitycheck: replace cmake -H "not meant for public use" with -S
https://www.mail-archive.com/cmake-developers@cmake.org/msg16769.html

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-07-08 08:04:39 -04:00
Sebastian Bøe
b8d4a77ecd size_report: Don't crash when observing stripped symbols
When stripped symbols are present in the build size_report has crashed
due to it not being aware of this case.

This patch causes stripped symbols to be ignored instead, allowing
analysis of the non-stripped symbols.

It is left as future work to somehow display how much space stripped
symbols are taking up.

Stripped symbols can be present when third-party binaries are linked
into the build.

Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
2019-07-05 14:18:49 -04:00
Marc Herbert
9e57338efd sanitycheck: fix --jobs help message + other cosmetic fixes
The --jobs default was recently changed in commit 9f4f57eed3, update
its help message.

Add the hopefully last missing verbose("Spawning...") statement.

Fix comment updated in commit 095b82a301.

Replace two tags with whitespace.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-07-03 23:04:16 -07:00
Andrew Boie
1578ef71b1 sanitycheck: workaround file limits
sanitycheck is opening an insane number of file descriptors
simultaneously as it opens up communication pipes with
every test that supports emulation, on every emulated
board target.

Increase the resource limit on open files until this code
can be properly refactored.

Workaround for: #17239

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-07-03 16:49:21 -04:00
Andrew Boie
8047a6f4d0 sanitycheck: make coverage reports easier to do
- The --gcov-tool argument now has a reasonable default
  if the Zephyr SDK is in use.
- --coverage-platform, if unspecified, defaults to what
  was passed to --platform
- --coverage implies --enable-slow, so that tests with
  the 'slow' tag are built and run.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-07-02 22:56:14 -04:00
Andrew Boie
095b82a301 sanitycheck: correct timeout extension comment
This wasn't discussing why there is extra time for when
code coverage is enabled.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-07-02 22:54:24 -04:00
Andrew Boie
9f4f57eed3 sanitycheck: don't multiply CPU count
We have a number of timing sensitive tests which run
correctly on a much more frequent basis if the system
is not so heavily loaded. Instead of squeezing a few
more crumbs of performance by doubling the CPU count,
just use the number of CPUs reported by the system.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-07-02 22:51:24 -04:00
Andrew Boie
d62e229680 sanitycheck: extend coverage timeout
If GCOV coverage is enabled, the coverage dump happens after
"PROJECT EXECUTION SUCCESSFUL" is printed. In some cases,
the additional time added was not enough to capture all the
GCOV output on a heavily loaded system before the emulator
gets killed.

Ideally, the decision to kill the emulator needs to be smarter
and less race-prone, but that can wait for a future
enhancement.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-07-02 20:30:41 -04:00
Alberto Escolar Piedras
b91f37494b sanitycheck: Minor optimization in BinaryHandler
Do not search for the valgrind binary (which) if valgrind
was not selected to be used.

Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
2019-07-02 18:49:44 -04:00
Alberto Escolar Piedras
649368c018 sanitycheck: Record time used in BinaryHandler
So the time used to run boards which use the BinaryHandler can be
reported, record the time used from spawning the process until
it finnishes or is killed.
The BinaryHandler is used by the "native" boards, unit tests,
nsim and Renode.

Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
2019-07-02 18:49:44 -04:00
Andrzej Głąbek
bb8dddfa99 scripts: dts: Add extraction of clock frequency for fixed clock
This patch adds generation of `*_FIXED_CLOCK_FREQUENCY` macros for
clock consuming nodes that are provided with a fixed rate clock.

Signed-off-by: Andrzej Głąbek <andrzej.glabek@nordicsemi.no>
2019-06-28 09:45:35 -05:00
Andrzej Głąbek
c782ba82ae scripts: dts: Add special handling of '+' in 'str_to_label' function
This patch is needed in order to get compilable macros for compatibles
like "arm,cortex-m0+".

Signed-off-by: Andrzej Głąbek <andrzej.glabek@nordicsemi.no>
2019-06-28 07:32:01 -05:00
Marc Herbert
eefea9da9c gen_app_partitions: deterministic _and_ decreasing app_smem_*.ld
Commit 212ec9a29a / feature #14121 already ordered partitions by
decreasing size, however it was common in samples/userspace/shared_mem/
/sample.kernel.memory_protection.shared_mem for two partitions to have
the same size and be randomly ordered between them. This adds the
partition name as a second sort key.

Unlike previous attempt in commit 725abdf430 this doesn't use the
partition name as the first (and only) key and doesn't break the
decreasing size order.  Huge thanks to Sigvart Hovland for spotting this
in a post-merge but prompt code review.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-06-27 18:06:30 -07:00
Marc Herbert
85bc0d2fe5 Revert "gen_app_partitions.py: make generated/app_smem_*.ld files deterministic"
This reverts commit 725abdf430 which did get rid of randomness in the
order of the partition _names_ as claimed but regressed commit
212ec9a29a / feature #14121 and broke the previous size order which I
missed. Huge thanks to Sigvart Hovland for spotting this in a post-merge
but prompt code review. Proper fix in the next commit.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-06-27 18:06:30 -07:00
Marc Herbert
725abdf430 gen_app_partitions.py: make generated/app_smem_*.ld files deterministic
Dictionaries are not ordered in Python 3.5 and before, so building twice
in a row could lead to a different partition order, different
build/zephyr/include/generated/app_smem_*.ld files and different
binaries.

Fix with a minor change to the "for" loop in the output function:
make it iterate on sorted(partitions.items()) instead of the raw and
randomly ordered partitions dictionary.

It is easy to reproduce the issue even without downgrading to an
obsolete Python version; pick a test like samples/userspace/shared_mem/
and simply change the code to this:

--- a/scripts/gen_app_partitions.py
+++ b/scripts/gen_app_partitions.py
@@ -159,10 +159,12 @@ def parse_elf_file(partitions):
                     partitions[partition_name][SZ] += size

+import random
 def generate_final_linker(linker_file, partitions):
     string = linker_start_seq
     size_string = ''
-    for partition, item in sorted(partitions.items()):
+    for partition, item in sorted(partitions.items(),
+                                  key=lambda x: random.random()):
         string += data_template.format(partition)
         if LIB in item:
             for lib in item[LIB]:

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-06-27 07:18:09 -04:00
Marc Herbert
6ccd026f7c gen_relocate_app.py: make generated/linker_relocate.ld deterministic
Dictionaries are not ordered in Python 3.5 and before, so building twice
${ZEPHYR_BASE}/samples/application_development/code_relocation/
in a row could lead to a different sections order, different
build/zephyr/include/generated/linker_relocate.d and code_relocation.c
and different binaries.

Fix with a minor change to three "for" loops in the output functions:
make them iterate on sorted(list of sections) instead of the raw and
randomly ordered dictionaries of sections.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-06-27 07:18:09 -04:00
Carles Cufi
41f1f648f6 west: runners: Guess build folder
When using a build folder format with build.dir-fmt that includes any
parameters that need resolving, the west runners cannot find the folder
since the required information (board, source dir or app) is not
available.
Add a very simple heuristic to support the case where a build folder
starts with a hardcoded prefix (for example 'build/') and a single build
is present under that prefix.
The heuristic is gated behind a new configuration option:
build.guess-dir

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-06-26 11:50:26 -04:00
Carles Cufi
3b3bbc1ee0 Bluetooth: hci_uart: Use DT for the hci_uart UART
Use Device Tree,and in particular a new 'bt-c2h-uart' to select which
UART is being used to communicate with an external BLE Host when acting
as a Controller.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-06-26 12:37:04 +02:00
Marti Bolivar
814bc7ba8e scripts: remove runner related cruft
There's some old cache variables that don't matter anymore in here.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-06-26 01:25:54 +02:00
Marti Bolivar
f08935fa73 scripts: west_commands: improve run_common error handling
Print a friendlier error message on ValueError, but don't throw away
the stack trace.

Move another call to log.die().

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-06-26 01:25:54 +02:00
Marti Bolivar
e6873b8d7e scripts: tweak west build output
Follow along with changes made in west flash/debug/etc to make it
easier to see the output steps visually.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-06-26 01:25:54 +02:00
Marti Bolivar
ec8dbf34dc scripts: tweak runner-related command output
Having common log handlers now lets us improve our logging output so
that info messages are prefixed with the runner they came from, and
doing something similar with the high level steps as we go, like this:

   -- west <command>: using runners
   -- runners.RUNNER_NAME: doing something
   <output from RUNNER_NAME subprocesses go here>
   -- runners.RUNNER_NAME: all done, bye

We can also colorize the west output to make it stand out better from
subprocesses, using the same output formatting style that west
commands like west list do.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-06-26 01:25:54 +02:00
Marti Bolivar
ddce583ca2 scripts: west_commands: decouple runners pkg from west
I've had some requests to be able to use code in the runners package
without having west installed.

It turns out to be pretty easy to make this happen, as west is
currently only used for west.log and some trivial helper methods:

- To replace west log, use the standard logging module
- Add an appropriate handler for each runner's logger in
  run_common.py which delegates to west.log, to keep
  output working as expected.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-06-26 01:25:54 +02:00
Marti Bolivar
d176cc36ad scripts: runners: update ZephyrBinaryRunner docstring
Fix some things that are incorrect and try to make it more clear.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-06-26 01:25:54 +02:00
Marti Bolivar
6440514bec scripts: west_commands: fix help string
The option name is wrong.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-06-26 01:25:54 +02:00
Marti Bolivar
725e98e300 scripts: fix pyocd runner --flash-opt help
This doesn't work. Fix the example.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-06-25 22:33:02 +02:00
Kumar Gala
95f28eb764 scripts/dts: Remove DT_CHOSEN generation
We never used DT_CHOSEN so lets remove this and add it back if/when
there's an actual need for it.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-06-25 08:43:58 -05:00
Kumar Gala
08a5f9f8eb scripts/dts: Add ability to mark cell/ctrl defines as deprecated
Add support so that we can flag any "defines" associated with a call to
either extract_cells or extract_controller as deprecated.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-06-25 08:43:35 -05:00
Kumar Gala
79b59e8205 scripts/dts: Remove 'use-prop-name' from clock generation
We never set 'use-prop-name' on clock bindings so lets just always
use CLOCK_CONTROLLER as the define name we generate.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-06-25 08:42:31 -05:00
Kumar Gala
593d628e32 dts/extract: Remove 'use-prop-name' from cell and controller handling
We have 'use-prop-name' flag in the bindings which is specifically used
for GPIO properties to control if we get "GPIO" or "GPIOS" as the
generated define name.

Lets remove the inconsistancy and use "GPIOS" as the preferred name as
this matches the DTS property name.  Towards that we will generate both
forms and remove support for 'use-prop-name'.

This also impacts "PWM" generation.  So we'll have "PWM" and "PWMS"

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-06-25 08:40:33 -05:00
Jukka Rissanen
c9aaab74db net: ptp: clock: Add usermode support to ptp_clock_get()
It is useful that the ptp_clock_get() function can be called from
the userspace. Create also unit test for calling that function
from userspace.

Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
2019-06-25 15:22:51 +03:00
Wentong Wu
5611e92347 kernel: add futex support
A k_futex is a lightweight mutual exclusion primitive designed
to minimize kernel involvement. Uncontended operation relies
only on atomic access to shared memory. k_futex structure lives
in application memory. And when using futexes, the majority of
the synchronization operations are performed in user mode. A
user-mode thread employs the futex wait system call only when
it is likely that the program has to block for a longer time
until the condition becomes true. When the condition comes true,
futex wake operation will be used to wake up one or more threads
waiting on that futex.

This patch implements two futex operations: k_futex_wait and
k_futex_wake. For k_futex_wait, the comparison with the expected
value, and starting to sleep are performed atomically to prevent
lost wake-ups. If different context changed futex's value after
the calling use-mode thread decided to block himself based on
the old value, the comparison will help observing the value
change and will not start to sleep. And for k_futex_wake, it
will wake at most num_waiters of the waiters that are sleeping
on that futex. But no guarantees are made on which threads are
woken, that means scheduling priority is not taken into
consideration.

Fixes: #14493.

Signed-off-by: Wentong Wu <wentong.wu@intel.com>
2019-06-24 15:38:21 -07:00
Ulf Magnusson
a570b40252 genrest: De-spam docs by skipping direct deps. in more places
Similar deal to commit cc14c40a2d ("kconfiglib: Unclutter symbol
strings, avoid redundant writes, misc.").

Hide the direct dependencies in the defaults, selects, and implies
sections. Do the same in menuconfig/guiconfig as well.

This uses a new Kconfiglib API, so update Kconfiglib to upstream
revision 164ef007a8. This also includes some minor optimizations and
cleanups.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-06-24 12:14:06 -07:00
Ioannis Glaropoulos
88959e75dd scripts: gen_priv_stacks.py: use the right alignment for priv stacks
The privilege stacks are not sandboxed inside an MPU region,
so they do not have to be aligned with the stack buffer size.
We fix this by using the PRIVILEGE_STACK_ALIGN macro, which
is defined in arch.h and reflects the minimum alignment
requirement for privilege stack buffers.

Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
2019-06-24 10:16:57 -07:00
Kumar Gala
74818bcb49 scripts/dts: Treat array always as a list
If the type of property is a 'array' we should generate defines as
if its a list even if theres only a single element in the list.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-06-24 07:16:59 -04:00
Kumar Gala
344737d759 scripts/dts: Treat string-list always as a list
If the type of property is a 'string-list' we should generate defines as
if its a list even if theres only a single element in the list.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-06-22 10:03:17 -05:00
Marc Herbert
35dc96399c sanitycheck: show handler_time in -v output (qemu 2.032s)
We already have the info so let's show it. This helps spots intermittent
issues[*], gives an indication of the time --build-only saves, can help
spot an overloaded test system, highlights the most time-consuming tests
which may need a longer timeout in their config, shows the effective
timeout value when one occurs... all this for a dirt cheap screen estate
price and two extra lines of code.

Sample -v output:

32/81 board123  tests/testme              PASSED (qemu 2.049s)
33/81 board456  samples/hello             PASSED (build)
34/81 qemu_x3   tests/kernel.stack.usage  FAILED: timeout (qemu 60.029s)
     see: sanity-out/qemu_x3/tests/kernel.stack.usage/handler.log
35/81 board456  tests/testme              PASSED (build)
36/81 qemu_x5   tests/kernel.queue        FAILED: failed (qemu 2.191s)
     see: sanity-out/qemu_x5/tests/kernel.queue/handler.log

[*] running qemu in heavily packed cloud virtual machines comes to mind,
    also see #12553, #14173 etc.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-06-21 13:47:32 -04:00
Kumar Gala
bf0f6d911d dts: Add concept of sub-nodes to YAML and generator
Several bindings have an expectation of sub-nodes that describe the
actual infomation.  The sub-nodes don't have any compatiable so we can't
key on that.

So we can add the concept of a sub-node to the YAML to handle cases like
'gpio-keys', 'gpio-leds', 'pwm-leds', etc..

The sub-node in the YAML is effective the "binding" params that describe
what properties should exist in the sub-node.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-06-21 15:34:28 +02:00
Kumar Gala
fe2062b38b dts/extract: deprecate old IRQ alias define generation
Make the old IRQ aliases that where missing the _0, _1, etc. part as
deprecated.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-06-21 07:53:05 -05:00
Kumar Gala
7b638bf3f6 dts/extract: Fix IRQ alias define generation
The alias generation wasn't doing the right thing with regards to
keeping the names consistent.  We would drop the index from the define
name for aliases.

So we'd get
	DT_NXP_KINETIS_GPIO_GPIO_D_IRQ
which should be
	DT_NXP_KINETIS_GPIO_GPIO_D_IRQ_0

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-06-21 07:53:05 -05:00
Kumar Gala
6a79bf2571 dts/extract: Add ability to mark aliases as deprecated
Add a deprecate flag to add_prop_aliases so we can make the aliases it
generates as deprecated if needed.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-06-21 07:53:05 -05:00
Anas Nashif
cbfba62182 ci: increase sleep time between sanitycheck runs
Give the system more time to settle before we try to re-run the tests
again.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-06-20 22:15:27 +02:00
Ulf Magnusson
e1f0b61d23 scripts/dts: Clarify extract/flash.py some more
Think I understand it now, and that was the goal.

 - _extract_partition() adds index-based entries. extract_partition()
   adds label-based entries.

   Rename them to _add_partition_index_entries() and
   _add_partition_label_entries(), and call them from a top-level
   extract_partition() function.

   This makes the logic clearer. It took me a long time to spot it.

 - Generate indicies with a simple counter and remove the _flash_area
   logic. This would break if partitions were extracted more than once,
   but they aren't, and now you know that they can't be.

 - Rename _create_legacy_label() to add_legacy_alias() and make it
   global. It doesn't use 'self'.

 - Break the logic for finding the flash controller into a separate
   helper function

 - Add doc-comments for the new functions

 - Misc. other small clean-ups

generated_dts_board.conf and generated_dts_board_unfixed.h were verified
to be identical for disco_l475_iot1 and frdm_kw41z before and after the
cleanup.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-06-20 11:35:05 +02:00
Anas Nashif
f2cb20c772 docs: fix misspelling across the tree
Found a few annoying typos and figured I better run script and
fix anything it can find, here are the results...

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-06-19 15:34:13 -05:00
Andy Ross
5efdd6a525 sanitycheck harness: Correct ordered regex handling
The way sanitycheck did its ordered regexes is that it would test
every regex against every line, and store the matching lines and their
regexes in an OrderedDict and check that they happened in the right
order.

That's wrong, because it disallows matching against a line that
previously appeared (and should have been ignored) in the input
stream.  The watchdog sample is the best illustration: the first boot
will (by definition) contain all the output already, but the regex has
to match against a line from the SECOND boot and not the same one it
saw earlier.

Do this the simple way: keep a counter of which regex we're trying to
apply next and increment it on a match.  This is faster too as we only
need to check one pattern per line.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2019-06-19 14:37:20 -04:00
Tyler Hall
bb359257bf scripts/sanitycheck: fix ZEPHYR_BASE subdir check edge case
There is a case where using startswith to determine if a path is a
subdirectory of another path can erroneously match. When using a
testcase root outside of ZEPHYR_BASE, an erroneous match will cause the
relative path containing ".." to get prepended to the test output
directory.

Example:

$HOME/zephyr/zephyr # ZEPHYR_BASE
$HOME/zephyr/zephyr-rust/tests # testcase root

The relative path prepended to the testcase name is ../zephyr-rust/tests
and an example test output dir is
./sanity-out/qemu_x86/../zephyr-rust/tests/rust/rust.main

In this case, the build directory escapes the board directory and is no
longer unique. Parallel tests then clobber each other.

Use pathlib instead of string matching to cover this case.

Signed-off-by: Tyler Hall <tylerwhall@gmail.com>
2019-06-19 13:56:26 -04:00
Marc Herbert
6f98db61f1 generated/cfb_font_dice.h: don't leak absolute paths in comment
The start of generated/cfb_font_dice.h looked like this:

/*
 * This file was automatically generated using the following command:
 * /home/john/zephyrproject/zephyr/scripts/gen_cfb_font_header.py
 * --input fonts/dice.png --output
 * /home/john/tmp/build/zephyr/include/generated//cfb_font_dice.h
 * --width 32 --height 32 --first 49 --last 54
 */

For build reproduction and "privacy" reasons, change it to this:

/*
 * This file was automatically generated using the following command:
 * ${ZEPHYR_BASE}/scripts/gen_cfb_font_header.py
 * --input fonts/dice.png --output
 * zephyr/include/generated//cfb_font_dice.h
 * --width 32 --height 32 --first 49 --last 54
 */

Test with:
  sanitycheck  -p reel_board \
  -T $ZEPHYR_BASE/samples/display/cfb_custom_font/

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-06-19 08:40:59 -04:00
Erwan Gouriou
8a9ca0d864 scripts: Upgrade pyocd for pack support
pyocd 0.21.0 provides pack support 'pack support' functionality,
as opposed to current 'buitlin support'.
This new feature enables the possibility to add pyocd support
for any chip that is documented in Keil database. Then one doesn't
need anymore to wait pyocd is updated with a new target to use
pyocd with his target, as long as it is populated in Keil database.

Signed-off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
2019-06-18 23:03:35 -04:00
Kumar Gala
84cf9dc671 scripts/sanitycheck: Add --cmake-only option
Add an option that only invokes the cmake phase of sanitycheck.  This
can be useful for any testing that only needs to initial generation
phase of cmake, for example device tree.  Also useful if we want to
just generate compile_commands.json files from cmake via:

./sanitycheck -xCMAKE_EXPORT_COMPILE_COMMANDS=1 --cmake-only

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-06-18 12:02:03 -04:00
Ioannis Glaropoulos
a4afe8c0fa scripts: gen_priv_stacks: remove unused variable
Variable priv_stack_decl_size is not used by the script,
so we can remove it.

Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
2019-06-17 12:33:58 -04:00
Marti Bolivar
c24e044c6e scripts: runners: check for required programs
Add self.require() checks before running commands. Increase test
coverage, including for this feature, while we are here.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-06-17 15:06:21 +02:00
Marti Bolivar
c07267a26a scripts: runners: abstract jlink's missing program support
The runners/jlink.py script has a mechanism for erroring out if a host
tool is not installed. Abstract it into runners/core.py and handle it
from run_common.py. This will let it be used in more places.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-06-17 15:06:21 +02:00
Anas Nashif
49b22d4fc3 sanitycheck: report tests that never build/run
Add new option --report-excluded to list all those tests with bad
filtering that never build or run. This option produces accurate results
with --all but can be used with default sanitycheck options to see what
does not run/build in CI for example. (limited coverage).

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-06-16 14:13:25 -04:00
Marti Bolivar
d371c54f51 scripts: west sign: test for input file existence
Check the CONFIG_BUILD_OUTPUT_HEX and CONFIG_BUILD_OUTPUT_BIN options
are enabled before attempting to build signed versions of these formats.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-06-16 10:16:15 -04:00
Marti Bolivar
2f839da163 scripts: west sign: improve user friendliness
Add more error handling and warnings. Doing this nicely requires a bit
of re-work to the control flow.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-06-16 10:16:15 -04:00
Marti Bolivar
33cbba3457 scripts: west sign: move hex and bin default locations
When signing binaries from multiple build directories, it is
inconvenient to have to specify the output file locations by hand each
time. It's also a little weird that they're not next to zephyr.bin and
zephyr.hex.

Move them to the build directory, next to their unsigned variants.

Suggested by Piotr Mienkowski.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-06-16 10:16:15 -04:00
Marti Bolivar
06c9f8e9e6 scripts: west sign: code style refactor
Let's leave self.args as the actual parsed argument namespace.
Pass the final computed build directory separately.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-06-16 10:16:15 -04:00
Marti Bolivar
3bd07a244a scripts: west sign: make it work again
This got broken in the patches which added the build.dir-fmt config
option when BUiLD_DIR_DESCRIPTION was renamed.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-06-16 10:16:15 -04:00
Marti Bolivar
fe91e48418 scripts: runners: add __contains__ to BuildConfiguration
Implement the in operator for this class.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-06-16 10:16:15 -04:00
Ulf Magnusson
045c809561 scripts/dts: Make extract/flash.py easier to read
Clean up the code a bit:

 - Simplify the loops over the flash 'reg' properties by using range()

 - Build identifier names with a plain .format() where possible. This
   makes them stand out better in the code.

 - Remove redundant variables

 - Move variables close to where they're used

 - Misc. other minor improvements

generated_dts_board.conf and generated_dts_board_unfixed.h were verified
to be identical for disco_l475_iot1 and frdm_kw41z before and after the
cleanup.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-06-16 10:10:06 -04:00
Piotr Mienkowski
9ac4dc6631 scripts/dts: generate DT_ALIAS_<ALIAS>_<PROP> defines
DT_ALIAS_<ALIAS>_<PROP> defines are a convenient and portable way to get
the device instance name despite different naming conventions used by
the device drivers.

Signed-off-by: Piotr Mienkowski <piotr.mienkowski@gmail.com>
2019-06-15 11:15:26 -05:00
Anas Nashif
b58aa20e13 ci: skip handling coverage data
We not analysing coverage data at all in CI right now. Disable this
while we figure out a better solution for reporting data.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-06-14 09:26:40 -04:00
Kumar Gala
01e54a5472 scripts/dts: Change DT_<COMPAT>_<INSTANCE>_<PROP> define
Its possible that the <INSTANCE> number could conflict with the register
number.  This is shown to happen for a device like soc-nv-flash at
address 0.

So change naming convention to DT_INST_<INSTANCE>_<COMPAT>_<PROP> and
make DT_<COMPAT>_<INSTANCE>_<PROP> as deprecated.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-06-14 08:02:15 -05:00
Kumar Gala
062bd0656a scripts/dts: Sort instance IDs by reg addr
Because of how generate defines for instances its possible that we have
a name conflict if the instance ID and reg addr space clash.

For example on qemu_x86 there are current two 'soc-nv-flash' nodes and
one is at reg addr 0, but instance id 1, the other is reg addr 0x1000
and instance id 0.  We'd possibly get this conflict:

For the 'soc-nv-flash' at 0x1000 (instance 0):
	(instance define)
	#define DT_SOC_NV_FLASH_0_BASE_ADDRESS          0x1000

For the 'soc-nv-flash' at 0x0 (instance 1):
	(address define)
	#define DT_SOC_NV_FLASH_0_BASE_ADDRESS          0x0

To deal with this we make sure that the lower reg address is instance 0,
than things work out ok.  To handle this case, if we sort the instance
IDs based on reg addr than if we have something at reg addr 0, it will
also than be an instand ID 0.

The longer term solution will be to deprecated the old defines and
remove the conflict between instance ID defines and normal DT defines.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-06-13 14:11:13 -05:00
Marti Bolivar
8718d5c84b scripts: west: improve error message on missing cache
In the west flash/debug commands, if the user gives an invalid build
directory, they'll get a stack trace instead of a helpful error
message when the cache can't be built.

Fix that.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-06-10 17:46:27 -05:00
Carles Cufi
98980c6b7c west: build: Configurable build folder format
Add the possibility of configuring the build folder format in west's
configuration system.
The build.dir-fmt configuration option controls how west will create
build folders when not specified, the following parameters are currently
accepted:

- board: The board name
- source_dir: The relative path from CWD to the source directory
- app: The name of the source directory

If CWD is below source_dir in the directory hierarchy then source_dir is
set to an empty string.

This means that if one sets:

[build]
dir-fmt = build/{board}/{source_dir}

Then when building samples/hello_world from zephyr's root for the
reel_board the build folder will be:

./build/reel_board/samples/hello_world

but when building it from inside the samples/hello_world folder it will
instead be:

./build/reel_board

Fixes https://github.com/zephyrproject-rtos/west/issues/124

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-06-11 00:00:34 +02:00
Carles Cufi
49c4b1c303 west: Use find_build_dir in run_common
In preparation for upcoming changes to the way the default build folder
is defined, switch to using the common find_build_dir() function in the
runners.
This actually changes the behavior for the west build command slightly,
since the current working directory (cwd) will now be checked after the
default build folder ('build'). This brings it in line with what is
used for the runners.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-06-11 00:00:34 +02:00
Andrew Boie
a42313d488 requirements.txt: add gcovr
This tool is cited in our documentation for producing coverage
reports, add it to the list of packages pulled in by pip during
workstation setup.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-06-07 14:15:16 -07:00
Kumar Gala
1c55882ae3 dts/extract: Remove bogus pinctrl generation
The pinctrl generation has never done anything useful so lets remove it
as its just dead code.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-06-07 10:35:58 -06:00
Ulf Magnusson
f46ebc3c97 kconfig.py: Use messages returned by load_config() and write_config()
load_config() and write_config() now return a message to print. This
message also says whether the configuration was loaded (replace=True) or
merged (replace=False), and whether the new .config is different from
the old (for write_config()).

Print the returned messages and remove some old print()s.

Also switch to an improved warning control API (the old one is still
supported, but might as well).

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-06-07 18:20:22 +02:00
Ulf Magnusson
cc14c40a2d kconfiglib: Unclutter symbol strings, avoid redundant writes, misc.
Update kconfiglib, menuconfig, and guiconfig to upstream revision
5c904f4549 to get various improvements and fixes in:

 - Marc Herbert found an issue involving symlinks, absolute paths,
   and rsource that could lead to files not being found. The root cause
   was relpath() assuming that symlink/../bar is the same as bar/, which
   isn't guaranteed.

   Fix it by handling paths in a simpler, more textual way.

 - Propagated dependencies from 'depends on' are now stripped from
   properties when symbols are printed (e.g. in information dialogs and
   generated documentation).

   The printed representation now also uses shorthands.

   Before:

     config A
             bool
             prompt "foo" if C && D
             default A if B && C && D
             depends on C && D

   After:

     config A
             bool "foo"
             default A if B
             depends on C && D

 - Before writing a configuration file or header, Kconfiglib now
   compares the previous contents of the file against the new contents,
   and skips the write if there's no change. This avoids updating the
   modification time, and can save work.

   A message like "No change to '.config'" is shown when there's no
   change.

 - .config now has '# end of <menu>' comments to make it easier to see
   where a menu ends. This was taken from a change to the C tools.

 - load_config() and write_(min_)config() now return a message that can
   be printed with print(kconf.load_config()). This allows messages to
   be reused in e.g. the configuration interfaces (nice now that there's
   also a "No change to..." string).

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-06-07 18:20:22 +02:00
Alexey Brodkin
bceaed67b3 simulation: ARC EM in nSIM to use nsim_em.props
This matches a naming scheme used for other flavour of ARC EM
SoC flavors.

Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
2019-06-06 14:20:42 -04:00
Kumar Gala
442df97366 scripts/dts: Remove alias defines for labels
We added generation of aliases for "alt-label" (which was the outer
label of a node) for use with shields and connectors.  However we've
never used these defines and the generation is a bit inconsistent.

This removes generation of defines like for label 'arduino_spi':
	#define ARDUINO_SPI_BASE_ADDRESS ... (already deprecated)
	#define DT_ST_STM32_SPI_FIFO_ARDUINO_SPI_BASE_ADDRESS ...

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-06-06 09:08:16 -06:00
Nathaniel Graff
bd2919dcf1 runners/jlink: Add --tool-opt option for more args
Adds the --tool-opt option to allow additional arguments to be
passed to JLink Commander

Signed-off-by: Nathaniel Graff <nathaniel.graff@sifive.com>
2019-05-30 16:54:02 -04:00
Ulf Magnusson
a1c3cc6660 guiconfig: Add a graphical configuration interface
This is a graphical configuration interface written in Tkinter. Like
menuconfig.py, it supports showing all symbols (with invisible symbols
in red) and jumping directly to symbols. Symbol values can also be
changed directly from the jump-to dialog.

This interface should feel a lot smoother than menuconfig.py on Windows.

When single-menu mode is enabled, a single menu is shown at a time, like
in the terminal menuconfig. Only this mode distinguishes between symbols
defined with 'config' and symbols defined with 'menuconfig'.

Compatible with both Python 2 and Python 3. Has been tested on X11,
Windows, and macOS.

To avoid having to carry around a bunch of GIFs, the image data is
embedded in guiconfig.py. To use separate GIF files instead, change
_USE_EMBEDDED_IMAGES to False. The image files can be found in
https://github.com/ulfalizer/Kconfiglib/tree/screenshots/guiconfig.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-05-30 09:54:06 -04:00
Ulf Magnusson
6cfc13522c menuconfig: Small Space/Enter improvement + fix for obscure bug
Update menuconfig (and Kconfiglib, just to sync) to upstream revision
7e0b9f7ae1, to get these commits in:

  menuconfig: Improve space/enter behavior slightly

  Space toggles value is possible, and enters menus otherwise. Enter
  works the other way around.

  Make this explicit in the code, which also fixes some corner cases,
  like space doing nothing on a y-selected menuconfig symbol.

  -------------------------------------------------------------------

  menuconfig: Fix display issue for unsatisfied-deps selected symbol
  with children

  A symbol with unsatisfied direct dependencies can end up with visible
  children in an implicit submenu if it is selected (though that
  generates a warning), so the optimization in _shown_nodes() isn't
  safe, and causes the child nodes to not be shown outside show-all
  mode.

  Just remove the optimization. Trying things out some more,
  everything's plenty fast enough anyway.

  Checking the direct dependencies of the parent instead would be safe.

The menu path now says "(Top)" instead of "(top menu)" too, which is a
bit more compact. Make the same change in genrest.py.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-05-30 09:54:06 -04:00
Luiz Augusto von Dentz
a3bea8872b Bluetooth: L2CAP: Move fixed channels to its own section in ROM
This changes the declaration of fixed channels to be statically defined
with use of BT_L2CAP_CHANNEL_DEFINE since fixed channels are never
unregistered.

Signed-off-by: Luiz Augusto von Dentz <luiz.von.dentz@intel.com>
2019-05-29 16:31:03 +03:00
Maureen Helm
4de99994f0 scripts: west: Add reset-after-load argument to jlink runner
Adds a new argument to the jlink runner to reset the device after
loading code to flash. This fixes a problem with the lpcxpresso54114
board where it was necessary to manually reset the board to get new code
to start running after the 'ninja flash' command. This new argument is
optional and false by default because there are some cases were we must
not reset after load, such as when we load the application into ITCM on
imx rt devices.

Signed-off-by: Maureen Helm <maureen.helm@nxp.com>
2019-05-25 10:01:30 -04:00
Marti Bolivar
8b9b7e7c8a scripts: avoid unnecessary 'west build --force'
Commit 88fb8bacfb ("scripts: improve west build's board handling")
lets us specify the board with a build.board config or BOARD
environment variable.

However, under some conditions, e.g. if the use has
build.pristine=auto and build.board=some_board, the following fails a
check_force call:

west build samples/hello_world
west build samples/philosophers

The problem is that the check_force wasn't made aware of the other
places a board can come from. Fix that.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-24 11:07:18 +02:00
Jan Van Winkel
6b9e160b5a sanitycheck: Execute binary in output directory
Execute the test binary from the output directory instead of directory
where sanitycheck was started.

This will ensure that any artifact created with a relative path by the
test binary will be placed in the output directory instead of creating
the artifact in the directory where sanitycheck was executed and prevent
any possible conflicts.

Signed-off-by: Jan Van Winkel <jan.van_winkel@dxplore.eu>
2019-05-23 08:37:09 -04:00
Kumar Gala
0d4290a2cb scripts/dts: Fix issue in label generation of registers
If we have something like:

       #address-cells = <1>;
       #size-cells = <0>;

       intc: ioapic@fec00000  {
	       compatible = "intel,ioapic";
	       reg = <0xfec00000 0x100000>;
       };

We should generate:

DT_INTEL_IOAPIC_FEC00000_BASE_ADDRESS_0
DT_INTEL_IOAPIC_FEC00000_BASE_ADDRESS_1

Instead we generated:

DT_INTEL_IOAPIC_FEC00000_BASE_ADDRESS
DT_INTEL_IOAPIC_FEC00000_BASE_ADDRESS_1

This was due to logic deciding if '_0' should be used not taking into
account #address-cells & #size-cells correctly.

Fixes: #16296

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-05-22 17:27:53 -05:00
Antony Pavlov
8245195306 dts: import vendor-prefixes.txt from linux kernel
import linux-v5.1:Documentation/devicetree/bindings/vendor-prefixes.txt

Use vendor-prefixes.txt to check vendor prefixes
used in compatible strings and property names.

Signed-off-by: Antony Pavlov <antonynpavlov@gmail.com>
2019-05-22 12:40:20 -04:00
Antony Pavlov
f08148e640 scripts/checkpatch.pl: fix root dir detection and enable it
Don't try to find files specific for linux kernel.

Signed-off-by: Antony Pavlov <antonynpavlov@gmail.com>
2019-05-22 12:40:20 -04:00
Wentong Wu
c8f43b9c25 scripts: make code relocation work for non-XIP system
on non-XIP system, loading address shouldn't be located on
FLASH memory region.

Fixes: #16090.

Signed-off-by: Wentong Wu <wentong.wu@intel.com>
2019-05-21 15:28:59 -04:00
Wentong Wu
743a184b2d scripts: mpu align for ro section of new memory region on non-XIP system
on non-XIP system, SRAM is the default region, and relocated .data
section and .bss section of SRAM shouldn't be inserted between
_image_rom_start and _image_rom_end, because the memory region between
_image_rom_start and _image_rom_end will construct the mpu ro region.
Also for the newly added memory region on non-XIP system, the
relocated .text secition and .rodata section should also be mpu aligned.

Fixes: #16090.

Signed-off-by: Wentong Wu <wentong.wu@intel.com>
2019-05-21 15:28:59 -04:00
Kumar Gala
fba8ac988c scripts/dts: Remove alias defines for bus parents with an alias
We generated some alias defines for children of a bus in which we had a
path alias for the bus node.  We never used these defines, we don't
recommend they get used (child of busses should use instance defines)
and they were only generated in small handful of cases (for dts that had
path aliases to the bus node - i2c or spi).

Remove this as effectively dead code.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-05-21 09:20:31 -04:00
Marc Herbert
75276a7d97 sanitycheck: when present, point at handler.log instead of run.log
When Zephyr crashes immediately QEMU reports an error immediately. This
is immediately reported by "make run". Then sanitycheck points the user
at the output of "make run". However the error message(s) are in QEMU's
output which is in a different .log file.

To address this situation point the error message at handler.log
instead of run.log if and only if handler.log is not empty.

To reproduce here's an artificial but very simple crash:

  sanitycheck --extra-args=CONFIG_TEST_USERSPACE=n \
    -p qemu_x86 -T tests/kernel/mem_protect/stackprot/

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-05-21 08:28:07 -04:00
Kumar Gala
983b493ac1 scripts/dts: Add support for enum's
Add specific enum generation support related to usb 'maximum-speed'
property.  This will generate a define with _ENUM with the integer
value of the enum as its ordered in the YAML.  The assumption right
now is that there's a matching enum in the code.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-05-17 09:53:20 -05:00
Øyvind Rønningstad
79d2e31cff scripts: mergehex: Add --overlap argument
Allows specifying the 'overlap' argument in IntelHex::merge().
This is identical to the --overlap argument in hexmerge.py, which
is bundled with IntelHex.

Signed-off-by: Øyvind Rønningstad <oyvind.ronningstad@nordicsemi.no>
2019-05-16 07:16:55 -05:00
Andy Doan
cbecaddff1 sanitycheck: Add a "test-only" option
This makes it easy to re-run tests from an existing build directory
w/o trying to rebuild artifacts.

Signed-off-by: Andy Doan <andy@foundries.io>
2019-05-16 07:08:01 -05:00
Andy Doan
79c48849fe sanitycheck: Allow custom arguments for west-flash
This allows you to use west to flash the device and pass custom
arguments to the flash command.

Signed-off-by: Andy Doan <andy@foundries.io>
2019-05-16 07:08:01 -05:00
Marti Bolivar
b3674f596b scripts: fix west TypeError on unknown arguments
Let's not mess with CommandContextError here, as the APIs have gotten
messed around a bit in various versions. Just use log.die() as that
will work with current and future west versions, and is clearer anyway.

Fixes west 247.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-16 14:00:53 +02:00
Jukka Rissanen
89bf1578d9 net: sockets: Add a way to register a socket family handler
Allow automatic handling of registered socket families.

Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
2019-05-10 22:43:27 +03:00
Carles Cufi
66fdd48fdd west: Add completion command
Add a completion command that dumps the contents of a shell
completion file present in the zephyr repository.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-05-10 13:08:27 +02:00
Carles Cufi
f43d306527 west: Add bash completion file
This file was previously located in the west repository, under scripts/.
Since it now includes knowledge about specific behavior ef zephyr
extension commands, we move it here after overhauling it completely.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-05-10 13:08:27 +02:00
Marc Herbert
1c8632cfaa sanitycheck: support symbolic links in ZEPHYR_BASE
Fix issue where sanitycheck wrongly assumed tests inside ZEPHYR_BASE
to be outside ZEPHYR_BASE and dropped the prefix in their name. This
happened when:
- ZEPHYR_BASE contains symbolic link(s), and
- relative --testcase-root argument(s) are passed

To generate unique names, TestCase.get_unique(testcase_root) first
checks whether "testcase_root" starts with ZEPHYR_BASE. Either may or
may not include symbolic links so both must be canonicalized before
comparison. While fixing this method, replace explicit forward slash
"/" and string replace with os.path.relpath() and make a couple other
simplifications and minor pydoc fixes.

Add new canonical_zephyr_base = os.path.realpath(ZEPHYR_BASE) constant
and corresponding comments and guidelines.

The most visible effect of this mismatch was sanitycheck dropping the
--testcase-root prefix from the unique name of tests inside
ZEPHYR_BASE. This means some test names could be not unique anymore
and silently overwrite each other's results, example:

 bash# cd zephyr_dir_with_symlink; export ZEPHYR_BASE=$(pwd)
 ./scripts/sanitycheck -T samples/portability/cmsis_rtos_v1 \
                       -T samples/portability/cmsis_rtos_v2

The more systematic and practical consequence (and how I actually
found this) was test outputs landing in unexpected locations.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-05-07 23:19:35 -04:00
Henrik Brix Andersen
f9dd53624c scripts/dts: allow node to be child on one bus and parent on another
Allow a device tree node to be child on one bus and parent on another
bus (e.g. an I2C slave device with multiple sub-devices).

Signed-off-by: Henrik Brix Andersen <henrik@brixandersen.dk>
2019-05-07 22:33:06 -04:00
Erwan Gouriou
1e314df89a scripts/dts: globals: parse gpio-map properties for connector nodes
gpio-map is a property of "nexus node", defined in dts v0.3.
It allows to describe a pin connector so it can be referenced
through phandles and hence used in expansion device nodes like a
shield header (typically implemented through overlays).

This change implements gpio controller resolution through these maps.
Few assumptions were taken in order to simplify the implementation.
These assumptions bring some limitations to the use of gpio-map
but my understanding is that this should still allow to cover most
use cases.

Assumptions:
-gpio-size is the same for all gpio-controllers referenced in a map
-optional properties gpio-map-mask and gpio-map-pass-thru are
supposed to be omitted
The understanding of this last assumption is that flags provided in
the expansion device node will overwrite the connector flags.

In a latter stage, when need happen, these limitations can be
revisited to unlock fully fledged gpio-map usage.

Fixes #15637

Signed-off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
2019-05-07 22:16:42 -04:00
Ulf Magnusson
9d3435692d scripts: gen_kobject_list: Remove redundant parentheses
Making a clean slate for a pylint test in CI.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-05-07 08:46:41 -07:00
Marti Bolivar
0396b6e64c scripts: west build: propagate verbosity to cmake generator
When run as "west -v build", make sure that the underlying build tool
is run in verbose mode as well (if the generator is known to support
it, which is the case for Unix Makefiles and Ninja based generators).

The per-generator hacks here are needed to support CMake 3.13. If we
move to CMake 3.14 or later, we can just run "cmake --build BUILD -v"
and be done with it.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-07 15:49:16 +02:00
Jun Li
2d5fb6da4c west: fix import error on sign.py
Fix broken imports caused by changes
from its dependencies.

Signed-off-by: Jun Li <jun.r.li@intel.com>
2019-05-07 08:36:29 -04:00
Ulf Magnusson
859c4ed2cd west: Fix unused import and variable
Making a clean slate for a pylint test in CI.

'_' is a common name for non-problematic unused variables in Python.
pylint knows not to flag it.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-05-07 08:12:34 -04:00
Marti Bolivar
83a5fd8d4c scripts: make west build -h fit on a screen
The west build --help output no longer fits in a single page.  Move
details and examples into the documentation, so the -h output doesn't
require scrolling around.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-06 16:30:05 -04:00
Marti Bolivar
3a486a85bc scripts: add --build-opt option to west build
This allows passing arguments to the underlying build tool (ninja,
make, etc.).

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-06 16:30:05 -04:00
Marti Bolivar
8465cf25c8 scripts: add --dry-run flags to west build
Analogously to the Make options with the same names, these print the
commands which would have run without running them.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-06 16:30:05 -04:00
Marti Bolivar
a1ef6962ae scripts: fix up west build verbosity levels
Adjust them so "west build -v" prints ZEPHYR_BASE and any CMake
commands, but none of the other more esoteric bits of information.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-06 16:30:05 -04:00
Marti Bolivar
bbe890ab30 scripts: add build.generator config option for west build
This can be used to override the default CMake generator
permanently. Its values are the same as those acceptable to cmake's -G
option.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-06 16:30:05 -04:00
Marti Bolivar
1f5e6d897f scripts: add --cmake-only to west build
This forces CMake to run, but skips running any build system target
afterwards.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-06 16:30:05 -04:00
Marti Bolivar
88fb8bacfb scripts: improve west build's board handling
- Respect the BOARD environment setting.
- Don't require --force if the board can't be figured out: it might be
  set in CMakeLists.txt, for example. Instead, downgrade to a warning
  which can be disabled with "west config build.board_warn false".
- Add a build.board configuration option used as a final BOARD fallback
  after CACHED_BOARD (in the CMake cache), --board (command line), and
  BOARD (environment).
- Keep the config docs up to date.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-06 16:30:05 -04:00
Marti Bolivar
d15950353d scripts: add helpers to west build for getting config options
These will be used more in a later patch.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-06 16:30:05 -04:00
Marti Bolivar
69099e3d99 scripts: flake8 cleanup for west build
Clean up coding nits to match west's style.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-06 16:30:05 -04:00
Marti Bolivar
acda25716a west: build: use CMAKE_HOME_DIRECTORY to locate source
Rather than relying on APPLICATION_SOURCE_DIR to be in the cache, just
use CMAKE_HOME_DIRECTORY instead.

https://cmake.org/cmake/help/latest/variable/CMAKE_HOME_DIRECTORY.html

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-06 16:30:05 -04:00
Marti Bolivar
4dbf5f1535 scripts: fix and tweak west boards
- This script didn't get fixed when cmake.py was renamed zcmake, so it
  won't run; fix that.
- Change the default format string to '{name}' to keep things simple
- Flake8 lint

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-05-06 16:30:05 -04:00
Maureen Helm
fe4b1ff0c3 boards: rv32m1_vega: Use different openocd config file for each core
Adds a new argument to the openocd runner to optionally specify the
config file. Updates the rv32m1_vega board to use different openocd
config files for the ri5cy and zero-riscy cores.

Signed-off-by: Maureen Helm <maureen.helm@nxp.com>
2019-05-06 14:52:17 -05:00
Carles Cufi
851fa12d51 west: boards: Use the new zcmake module name
Switch to the recently renamed zcmake module when importing the
run_cmake function.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-05-05 12:58:22 +02:00
Marc Herbert
af1090c3bd sanitycheck: clarify who spawns processes and threads and where
Notably fix the wrong comment I added in commit 6f011c95c4: when
testing with QEmu sanitycheck does _not_ spawn QEmu; it relies on "make
run" instead.

Searching the code for "Spawning" now cycles directly to all the places
starting processes and threads.

- Sample -v -v verbose output (lines wrapped for commit message check)

 Spawning QEMUHandler Thread for \
     qemu_x86/samples/hello_world/sample.helloworld 'make run'

- native_posix example:

 Spawning process /home/.../sanity-out/native_posix/\
    /samples/hello_world/sample.helloworld/zephyr/zephyr.exe
 Spawning BinaryHandler Thread for native_posix/\
     samples/hello_world/sample.helloworld

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-05-04 18:29:01 -04:00
Carles Cufi
c9f4bb6799 west: build: Construct CalledProcessError with positional args
Since west's main.py relies on the args tuple with the returncode
and the cmd, create the CalledProcessError using the correct
positional args.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-05-04 20:59:15 +02:00
Jan Van Winkel
4d975dba79 west: Renamed cmake.py to zcmake.py
Renamed cmake.py to zcmake.py to prevent import conflict if cmake is
installed through pip.

Signed-off-by: Jan Van Winkel <jan.van_winkel@dxplore.eu>
2019-05-04 16:40:06 +02:00
Carles Cufi
d6f033e690 west: Add a boards command
Add a new "boards" command that is able to list all the boards in the
upstream tree. There is currently no support for out-of-tree boards.

The command executes cmake to use the built-in CMake script,
boards.cmake, to list the boards, and then stores the information
retrieved and allows the user to present it in a user-definable format.

Fixes https://github.com/zephyrproject-rtos/west/issues/53

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-05-03 21:55:13 +02:00
Carles Cufi
31bdad5299 west: Implement CMake helpers in scripts/west_commands
Move the existing CMake and build functionality from the west repository
to zephyr. The rationale behind this move is that it's very tightly
coupled with the Zephyr build system and is only used by the extension
commands implemented in the zephyr tree.
If additional extension commands in third-party repos want to use the
functionality they can add $ZEPHYR_BASE/scripts/west_commands to the
Python system path.

The implmentations in the west repo will be deprecated.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-05-03 21:55:13 +02:00
Sigvart M. Hovland
06365a5ed1 cmake: zephyr_modules: Add west executable argument
On some systems where you don't have access to `PATH` and you can't
set the `ENV{PATH}` variable. You need to be able to pass the path
to the west executable down to the python script so it is better
for it to be set explicitly than assuming that it exsists as a
part of the PATH/executables in the shell being called.

Signed-off-by: Sigvart M. Hovland <sigvart.hovland@nordicsemi.no>
2019-05-03 14:23:09 -04:00
Maureen Helm
d6d1e27a97 scripts: west_commands: Print error message when jlink exe not found
Adds a check in the jlink runner to look for the jlink executables and
print a more useful error message if they are not found.

Signed-off-by: Maureen Helm <maureen.helm@nxp.com>
2019-05-02 22:50:55 -04:00
Luiz Augusto von Dentz
e0f3ab6bf2 Bluetooth: GATT: Add support for static services
This reintroduces support for static service in the form of a new API,
BT_GATT_SERVICE_DEFINE, and changes the internal services (GAP/GATT)
to be defined as const as they are never register/unregistered.

Internal service needed to be renamed in order to keep the same order
as before since the section elements are sorted by name.

The result is the following (make ram_report):

before:
      gatt.c                                    572     0.66%
        cf_cfg                                   32     0.04%
        db                                        8     0.01%
        db_hash                                  16     0.02%
        db_hash_work                             32     0.04%
        gap_attrs				180     0.21%
        gap_svc                                  12     0.01%
        gatt_attrs                              160     0.18%
        gatt_sc                                  80     0.09%
        gatt_svc                                 12     0.01%
        sc_ccc_cfg                               32     0.04%
        subscriptions                             8     0.01%

after:
      gatt.c                                    210     0.24%
        cf_cfg                                   32     0.04%
        db                                        8     0.01%
        db_hash                                  16     0.02%
        db_hash_work                             32     0.04%
        gatt_sc                                  80     0.09%
        last_static_handle                        2     0.00%
        sc_ccc_cfg                               32     0.04%
        subscriptions                             8     0.01%

Signed-off-by: Luiz Augusto von Dentz <luiz.von.dentz@intel.com>
2019-05-02 08:29:23 +03:00
Flavio Ceolin
bf878ced12 sanitycheck: Fix an error in scan_path exception
The function error expects only one parameter. The excpetion handler in
scan_path was calling this function with multiple parameters instead of
formatting the string.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2019-04-28 13:38:32 -04:00
Torsten Rasmussen
0dd3b421c4 west: zephyr_module: Improved error messaging during west failures
Fixes: #15664

This commit improve error messaging in case `west` list fails.
Previously any error messages and stack trace reported by `west` will
was thrown away by zephyr_module.py.

Now the error, as well as any stack traces, printed by `west` will be
re-printed to the user.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2019-04-27 12:38:32 -07:00
Marc Herbert
4a10eea0eb gen_kobject_list.py: better comments and --help. Zero code change.
- The script can but does not always generate five files, in fact the
  current build invokes it at least three times requesting different
  outputs every time.

- --kobj-types-output produces a code fragment; not a standalone/usable
  header file. It outputs enum constants for a single enum type and not
  several enum types.

- Some outputs include driver instances and others not: clarify which.

- There's an entire and great section in the documentation that took
  me ages to find because it's not referenced anywhere in the --help
  or code. Fixed.

- Highlight the massive duplication in the CMakeLists.txt to save
  déjà vu confusion and minimize future divergence.

- Other minor tweaks.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-04-26 03:21:59 -07:00
Carles Cufi
3a88dce9c5 scripts: west: Run pristine.cmake directly instead of the target
When making a build folder pristine until now we were running the
'pristine' build target. The issue with that is that ninja/make or
whatever build tool is being used might decide to re-run CMake itself if
some of the dependencies have changes. This might trigger an error that
is unfriendly and unnecessary, since the user is explicitly asking for
the build folder to be wiped before starting a fresh build.
To avoid this issue restor to running directly the CMake script that the
'pristine' build target itself uses, so as to make sure that the build
folder is wiped unconditionally regardless of changes made to the tree.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-04-23 13:24:41 -07:00
Anas Nashif
d18ec53575 sanitycheck: show handler in verbose mode
When running in verbose mode, show what handler is being used.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-04-20 14:14:47 -04:00
Anas Nashif
d6476eee3e sanitycheck: handle pid file not being present
Do not try to open non-existing pid files.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-04-20 14:14:47 -04:00
Anas Nashif
042d9b7d83 sanitycheck: fix reporting: build vs. run
If build fails, report a build failure instead of handler error.

Fixes #15272

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-04-20 14:14:47 -04:00
Anas Nashif
654ec598ac sanitycheck: add timestamps to messages
Using the new option --timestamps, any output from sanitycheck will have
a timestamp to help identify bottle necks and monitor execution time.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-04-20 14:14:47 -04:00
Anas Nashif
924987d72f ci: detect changes to sub-board definitions
When detecting changes to boards, make sure we test all board
configurations available in that board directory, not only the main one.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-04-20 14:14:29 -04:00
Patrik Flykt
83de530d5a scripts/coccinelle: Add script for counting identifier length
Add a simple Coccinelle script that counts identifier lengths and
prints out a warning if it is longer than 31 characters.

The script can be run with:
spatch -D report --very-quiet \
--include-headers --recursive-includes \
--cocci-file $ZEPHYR_BASE/scripts/coccinelle/identifier_length.cocci \
--dir $ZEPHYR_BASE \
kernel/

Where '--include-headers' and '--recursive-includes' can be omitted
if neede.

Signed-off-by: Patrik Flykt <patrik.flykt@intel.com>
2019-04-18 12:29:07 -04:00
Andy Ross
aa2b8a1bc7 scripts/sanitycheck: Precompile test-time regular expressions
The Harness handlers for tests were parsing the realtime stream out of
qemu pipes by recompiling and executing every regex for every line (!)
of output from the simulator.  That's a significant CPU load, and it's
(1) in a separate thread not tracked by the JOBS limit and (2)
happening at the worst possible time and contending with the qemu
process for host CPU cycles that it needs to hit its (real world)
timer targets on time.

Compile them just once, please.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2019-04-18 12:21:13 -04:00
Ulf Magnusson
86b0c22dc4 kconfig: Avoid potential issue parsing generated_dts_board.conf
Splitting a string like 'foo="bar=baz"' on '=' will give ['foo', '"bar',
'baz"'] instead of the intended ['foo', '"bar=baz"']. split() with
maxsplit=1 to avoid potential issues.

Not seen in practice. Just some future safety.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-04-18 08:49:46 -04:00
Ulf Magnusson
aec74f692c kconfiglib/menuconfig: Various behavior/UI improvements
Update Kconfiglib and menuconfig to upstream revision 90c5573c19, to get
these commits in:

    Warn for unquoted argument to 'source', etc.

    Print a warning suggesting to add quotes for things like

      source foo/bar/Kconfig
      menu title
      prompt unquoted

    Example warning:

      Kconfig:32: warning: style: quotes recommended around
      'lib/Kconfig.debug' in 'source lib/Kconfig.debug'

    That quoteless syntax is supported for compatibility with old
    versions of the C tools. It only works for a single word.

    ==================================================================

    menuconfig: Include all parents in menu paths

    Previously, symbols not defined with 'menuconfig' with children
    weren't listed in the children's menu paths. It was deliberate, but
    it's probably an anti-feature in retrospect, because it can make it
    harder to find stuff by following the menu path.

    Don't try to be clever and just list all the parent nodes in the
    menu path.

    ==================================================================

    menuconfig: Fix display issue for optional-prompt menuconfigs

    _shown_nodes() needs to check whether invisible 'menuconfig' symbols
    with optional prompts have visible children, so that they can be
    shown outside show-all mode. Previously, only 'config' symbols were
    checked.

    ==================================================================

    menuconfig: Remember last saved/loaded path and improve
    _conf_changed

    Remember the last path that was manually saved/loaded instead of
    reverting back to standard_config_filename() (e.g. .config).

    Remember the path to the last saved minimal configuration separately
    as well.

    Also improve the _conf_changed behavior when loading a .config
    within the interface. Instead of always treating it as needing to be
    saved, check if it's outdated, like for the .config file loaded on
    startup.

    Also make the exit message ("No changes to save", etc.) always
    include the target .config file, which is helpful. Previously, only
    the save message did.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-04-18 08:46:15 -04:00
Carles Cufi
b7c75915e0 west: build: Add new pristine cmd-line and config option
Add a new command-line and build config option, `pristine`, that the
user can pass to `west build` or set in its configuration file(s) in
order to automatically trigger a pristine build on every build or
whenever west considers it required.

The option can take the following values:

- never: Never run the target
- always: Always run the pristine target before building
- auto: Run the pristine target when required

With `auto`, the pristine target will be run when running
west with an existing build folder containing a build system and:

- Selecting a different board from the one currently in the build system
- Selecting a different application from the one currently in the build
  system

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-04-17 10:02:46 -04:00
Carles Cufi
3c6584d6d5 west: build: Use officially documented CMake options
The -B option has always existed but was first officially documented in
CMake 3.13.0. In that same release the -S option was introduced,
replacing the old undocumented -H. Switch to using the officially
documented options.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-04-17 10:02:46 -04:00
Marc Herbert
0f7255c3b8 sanitycheck: order results.csv and discards.csv deterministically
One of the first things needed when comparing builds of tests across
different environments/systems is to make sure the same (sub)tests were
selected and run in the first place. For that purpose sort the output of
--testcase-report and --discard-report as they were in random order.

Actually make the entire class TestInstance sortable by adding a
standard __lt__() method comparing unique instance names; it could be
useful again.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-04-17 08:28:54 -04:00
Marc Herbert
3061c92aaf file2hex.py: new --gzip-mtime option that defaults to zero + test
This makes the output of file2hex.py deterministic by default while also
letting the user set the mtime in the gzip header manually if desired.

Use the option without any argument to restore the previous behavior
that sets the current (and obviously changing) "now" timestamp.

To test:  ./sanitycheck --tag gen_inc_file

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-04-17 08:27:40 -04:00
Anas Nashif
a94f2b0b12 release: update footprint data
Update footprint data for 1.14 release.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-04-15 20:31:52 -04:00
Ulf Magnusson
4f920df29a scripts: Remove checkconfig.py
This is an old script for finding references to undefined Kconfig
symbols that assumes Kconfiglib 1 (an older API).

There's now a different check for references to undefined symbols (see
commit 1d0834b35f ("checks: kconfig: Check for references to undefined
Kconfig symbols") in the ci-tools repo).

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-04-12 08:28:03 -04:00
Marti Bolivar
100a2bba6c scripts: update pyocd in requirements.txt
This includes a bugfix for a pyocd requirement on pyyaml
that could not be satisfied using officially supported
releases.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-04-10 17:52:15 -04:00
Andrew Boie
9e69c54060 sanitycheck: add more info during processing
After the testcase configs are built, there is a step to
filter all the test case information to determine the set
of tests to run.

As this step takes a nontrivial amount of time, add an
informational message about it.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-04-10 13:09:28 -04:00
Torsten Rasmussen
ad026c09b1 cmake: modules: using posix path for Kconfig generated file
Fixes: #15289

Kconfig requires posix style path when sourcing other files.
As abspath in python will use native host style, i.e. backslash '\' in
windows this will cause invalid paths in Kconfig generated file and
thus the file will never be loaded.

This commit uses PurePath to convert the path to posix style, ensuring
Kconfig can load the modules.

Signed-off-by: Torsten Rasmussen <torsten.rasmussen@nordicsemi.no>
2019-04-09 16:47:45 +02:00
Anas Nashif
f6a163fc5c ci: remove unused script merge_junit.py
This script is now part of ci-tools repo.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-04-07 10:26:27 -04:00
Anas Nashif
3ae52624ff license: cleanup: add SPDX Apache-2.0 license identifier
Update the files which contain no license information with the
'Apache-2.0' SPDX license identifier.  Many source files in the tree are
missing licensing information, which makes it harder for compliance
tools to determine the correct license.

By default all files without license information are under the default
license of Zephyr, which is Apache version 2.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-04-07 08:45:22 -04:00
Anas Nashif
8287f7c855 scripts: remove old xtools config files
Those files are outdated. We now have them as part of the sdk-ng git
repo.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-04-07 08:45:22 -04:00
Patrik Flykt
7c0a245d32 arch: Rename reserved function names
Rename reserved function names in arch/ subdirectory. The Python
script gen_priv_stacks.py was updated to follow the 'z_' prefix
naming.

Signed-off-by: Patrik Flykt <patrik.flykt@intel.com>
2019-04-03 17:31:00 -04:00
Andrew Boie
f0835674a3 lib: os: add sys_mutex data type
For systems without userspace enabled, these work the same
as a k_mutex.

For systems with userspace, the sys_mutex may exist in user
memory. It is still tracked as a kernel object, but has an
underlying k_mutex that is looked up in the kernel object
table.

Future enhancements will optimize sys_mutex to not require
syscalls for uncontended sys_mutexes, using atomic ops
instead.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-04-03 13:47:45 -04:00
Andrew Boie
1dc6612d50 userspace: do not track net_context as a kobject
The socket APIs no longer deal with direct net context
pointers.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-04-03 13:47:45 -04:00
Andrew Boie
a00eff731c gen_kobject_list: allow object data to be C code
The data value in a kernel object structure is specific
to that type of object. Allow this to be a reference to
another C symbol or other compiled code by populating as
a string.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-04-03 13:47:45 -04:00
Andrew Boie
c235e167fa elf_helper: support for objects in user memory
Some forthcoming kernel object types like futexes need to
be tracked, but do not contain data that is private to
the kernel.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-04-03 13:47:45 -04:00
Andrew Boie
ce6210e479 gen_priv_stacks.py: fix kobject list
k_stack are orthogonal to thread stacks.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-04-03 13:47:45 -04:00
Anas Nashif
193fb971c2 ci: do not report coverage on pull requests
The code coverage reports on pull requests has been inconsistent and
confusing. Disable them for pull-requests.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-04-01 18:39:17 -04:00
Anas Nashif
366ed11bfa sanitycheck: deal with special charachters in xml
Some logging text and colors were not escaped correctly, make sure we
generate well formed XML reports.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-04-01 12:23:09 -04:00
Anas Nashif
f16e92c000 sanitycheck: count samples in reports
We have not been counting samples in reports. This change lists tests
associated with sample code which in many cases is just verifying output
from the sample and counts as 1 test.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-04-01 12:23:09 -04:00
Carles Cufi
766edd449d cmake: modules: Enclose name and path in quotes
Due to the fact that CMake only supports greedy regexes,
the ':' in a Windows path ('C:\...') was being matched leading
to an invalid split of the line. Quote both the name and the path
to avoid this issue.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-04-01 07:27:23 +08:00
Torsten Rasmussen
bd7569f272 cmake: Extracted Zephyr module processing into python script
Fixes: #14513

This commit move the functionality of extracting zephyr modules into
generated CMake and Kconfig include files from CMake into python.

This allows other tools, especially CI to re-use the zephyr module
functionality.

Signed-off-by: Torsten Rasmussen <torsten.rasmussen@nordicsemi.no>
2019-03-29 23:10:59 -04:00
Marc Herbert
195195a4fc file2hex.py: switch from gzip.compress() to GzipFile()
Zero functional change, this is pure refactoring and preparation for
using the mtime= parameter which the gzip.compress() shortcut does not
make available.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-03-29 08:12:53 -04:00
Patrik Flykt
2fb87b9dd2 scripts/coccinelle: Update unsigned suffix script
Update unsigned suffix script to properly match multiplication.
Fix provided by Himanshu Jha.

Signed-off-by: Patrik Flykt <patrik.flykt@intel.com>
2019-03-28 17:15:58 -05:00
Patrik Flykt
caebf204c6 scripts/coccinelle: Update Coccinelle script for unsigned values
Update the script to detect and update more instances of unsigned
variable assignments when using all four simple rules of arithmetics.

Signed-off-by: Patrik Flykt <patrik.flykt@intel.com>
2019-03-28 17:15:58 -05:00
Ulf Magnusson
a449c98db2 scripts: Fix risky uses of non-raw regex strings in Python scripts
Fixes pylint warnings like this one:

    doc/conf.py:325:0: W1401: Anomalous backslash in string: '\s'.
    String constant might be missing an r prefix.
    (anomalous-backslash-in-string)

The reason for this warning is that backslash escapes are interpreted in
non-raw (non-r-prefixed) strings. For example, '\a' and r'\a' are not
the same string (first one has a single ASCII bell character, second one
has two characters).

It just happens that there's no \s (or \., or \/) escape for example,
and '\s' turns into two characters (as needed for a regex). It's risky
to rely on stuff like that regexes though. Best to make them raw strings
unless they're super trivial.

Also note that '\s' and '\\s' turn into the same string.

Another tip: A literal ' can be put into a string with "blah'blah"
instead of 'blah\'blah'.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-03-28 14:41:32 -05:00
Ulf Magnusson
0e8e92cd87 scripts/dts: Fix bug in dead /memreserve/ code
The /memreserve/ code would crash if it ever ran, because 'name' isn't
defined (seems to be some copy-paste here). There are no /memreserve/s
in Zephyr though, so it works out.

'name' seems to be the node name. Not sure what to put for a
/memreserve/, but make it '<memreserve>' to make it stand out.

Fixes a pylint warning.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-03-28 11:11:13 -05:00
Ulf Magnusson
7ffd628dac scripts: elf_helper: Remove undefined var ref. in ArrayType.__repr__()
self.num_members doesn't exist. This commit just removes the reference,
because I didn't want to guess a proper fix.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-03-28 11:07:42 -05:00
Ulf Magnusson
425998bd79 scripts: elf_helper: Tidy up get_symbols() to eliminate pylint warning
Using a member variable in the dict comprehension was probably a typo
(can't see 'sym' referenced elsewhere). Use a local variable instead.

Made pylint spit out these warnings (which might be spurious though):

    scripts/elf_helper.py:535:24: E0203: Access to member 'sym' before
    its definition line 536 (access-member-before-definition)

    scripts/elf_helper.py:535:39: E0203: Access to member 'sym' before
    its definition line 536 (access-member-before-definition)

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-03-28 11:06:33 -05:00
Ulf Magnusson
d5b0bd14e3 scripts: Remove unused imports in all Python scripts
Discovered with pylint3.

Upstream open-amp PR: https://github.com/OpenAMP/open-amp/pull/168

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-03-28 11:06:20 -05:00
Ulf Magnusson
12ba9dfa52 scripts: Remove unused variables in all Python scripts
Discovered with pylint3.

Use the placeholder name '_' for unproblematic unused variables. It's
what I'm used to, and pylint knows not to flag it.

Python tip:

    for i in range(n):
        some_list.append(0)

can be replaced with

    some_list += n*[0]

Similarly, 3*'\t' gives '\t\t\t'.

(Relevant here because pylint flagged the loop index as unused.)

To do integer division in Python 3, use // instead of /.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-03-28 11:06:20 -05:00
Ulf Magnusson
399c04ce5a scripts/dts: Remove unused variables and imports
Discovered with pylint3.

Use the placeholder name '_' for unproblematic unused variables. It's
what I'm used to, and pylint knows not to flag it.

Also improve the naming a bit in devicetree.py. If a key/value is known
to be a specific thing (like a node), then it's helpful to call it that
instead of something generic like "value".

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-03-28 11:06:01 -05:00
Marc Herbert
682961a382 sanitycheck: document that --save-tests appends to existing file
Fix --help message. Also rename run_report() to save_tests() as it's
used only once by --save-tests and nowhere else. Maybe the code was
shared with some --other-report feature in the past but not any more.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-03-28 09:30:00 -04:00
Andrew Boie
cf91f8cf90 elf_helper: fix debug_die call
debug_die() is not implemented in this class, and indeed we
don't even have a reference to the DWARF DIE object.

This is a fatal error anyway, just raise an exception.

Fixes #14762

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-03-27 16:37:49 -05:00
Marc Herbert
6f011c95c4 sanitycheck: don't generate the top-level Makefile in random order
Also add a "generated by sanitycheck" header to indicate origin and a
warning about the dependency required to actually run the test.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-03-26 22:23:56 -04:00
Ulf Magnusson
e9c1d6d0ed scripts: gen_app_partition: Fix broken typo'd sys.exit()
os.exit() doesn't exist.

Also use the nifty sys.exit(msg) feature, which prints 'msg' to stderr
and exits with status 1.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-03-26 10:19:26 -05:00
Ulf Magnusson
ba312fe844 scripts: Remove unnecessary () around if/while conditions in Python
Not needed in Python. Detected by check C0325 in pylint3.

Also replace an

  if len(tag):

with just

  if tag:

Empty strings, byte strings, lists, etc., are falsy in Python.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-03-26 07:59:59 -05:00
Ulf Magnusson
d83141535c scripts: Remove accidental semicolons in Python scripts
Making a clean slate for some pylint CI tests. Only enabling relatively
uncontroversial stuff.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-03-26 07:57:52 -05:00
Ulf Magnusson
5c8fd84d12 scripts: Remove extra trailing newlines from Python scripts
Fixing all instances so that it can be flagged in a pylint CI check
later.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-03-26 07:53:08 -05:00
Ulf Magnusson
7da0053a0c scripts: gitlint: Fix broken module docstring
Needs to be at the beginning of the file. Fixes a pylint warning:

    scripts/process_gperf.py:26:-1: W0105: String statement has no
    effect (pointless-string-statement)

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-03-26 07:50:06 -05:00
Ulf Magnusson
1a27de0d31 scripts: gen_priv_stacks: Remove unused static kernel object stuff
Might've been a copy-paste from scripts/gen_kobject_list.py.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-03-26 07:47:36 -05:00
Ulf Magnusson
605423f9b1 scripts: process_gperf: Fix broken module docstring
Needs to be at the beginning of the file. Fixes a pylint warning:

    scripts/process_gperf.py:26:-1: W0105: String statement has no
    effect (pointless-string-statement)

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-03-26 07:46:11 -05:00
Marc Herbert
932a33ad75 sanitycheck: fix, expand and clarify test case selection --help
- Fix wrong --test help message
- Provide more examples to clarify naming hierarchy
- Document that --sub-test runs its entire --test
- Point out that save/load options use their own format
- Document that --list-tests is flattened

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-03-18 22:07:25 -07:00
Andrew Boie
7ea211256e userspace: properly namespace handler functions
Now prefixed with z_hdlr_ instead of just hdlr_.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-03-18 09:23:11 -07:00
Ulf Magnusson
958e2d3229 scripts/dts: Refactor Flash code to be less twisty
- Remove DTFlash.extract(), which was just dispatching to either
   _extract_flash() or _extract_code_partition() depending on which
   magic string was passed in. Call them directly instead.

 - Fold constant and globally available parameter values into functions

 - Remove DTFlash._flash_node. It's easy to derive wherever it's needed,
   and it makes it clearer where it comes from (and means functions can
   be called in any order).

 - Remove DTFlash._flash_base_address, which is unused

 - Remove various unused parameters to functions

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-03-15 11:09:49 -05:00
Ulf Magnusson
2a4d8b07aa scripts/dts: Remove weird def_label.split('/') code
def_label is the name used for macros and .conf file values. It can
never have a slash in it.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-03-15 11:09:49 -05:00
Ulf Magnusson
f95a230b82 scripts/dts: Clean up/fix extract/interrupts.py a bit
- Fix broken code that was meant to turn the
   'interrupts'/'interrupts-extended' value into a list if it wasn't.
   list(123) will error out instead of creating [123].

 - Remove weird .split('/') on macro name

 - Rename 'props' to 'vals'. It's the value of a single property.

 - Get rid of a bare 'except:'

 - Rename l_fqn to full_name. Accidentally stumbled upon 'fqn' probably
   standing for "fully qualified name", but it's not obvious.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-03-15 11:09:49 -05:00
Ulf Magnusson
3b64e71221 scripts/dts: Simplify extract/interrupts.py and add some helpers
- Have get_parent_path() return None for the root ('/'). This is handy
   when looping over path components.

 - Move _find_parent_irq_node() out of the class, call it
   parent_irq_node(), and use get_parent_path() in it.

 - Add a global err() function for reporting errors. Use it if a node
   unexpectedly has no interrupt-parent.

   Previously, this would give a Python error instead.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-03-15 11:09:49 -05:00
Ulf Magnusson
a3d3aa58b8 scripts/dts: Remove redundant empty constructors
It's not obligatory for Python classes to have an __init__().

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-03-15 11:09:49 -05:00
Ulf Magnusson
0a3f00ad7c scripts/dts: Rename get_node_label() to node_label()
Just returns a value, with no side effects.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-03-15 11:09:49 -05:00
Kumar Gala
4da0f8b796 linker: Remove unused (OPTIONAL) from linker scripts
(OPTIONAL) was a vestiage from the initial import of the Zephyr code
base and we dont utilize it with the GNU linker.  Additionally, the way
(OPTIONAL) gets defined to nothing creates a linker script that lld
(from llvm) doesn't like.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-03-15 06:42:02 -05:00
David B. Kinder
6000a6205a doc: prepare for improving doc API linking
Linking to API material requires knowing the pecularities of how
doxygen, sphinx, and breathe work. In an attempt to hide some of this
we're preparing the current docs to allow use of configuration defaults
that will let us more simply use a default role that will hunt for a
reference target in the various domains that are available by using a
default "role" of "all".  This will let us use the simple notation
`functionname` or `typename` without fully specifying the reference as
:c:func:`functionname`.

This patch cleans up exising docs that were (incorrectly) using single
backtics where double backtics should have been used, and also found
some typos (such as a space between the role name and the reference,
such as :file: `filename`, and a missing colon such as
c:func:`functionname`)

This is a start to address issue #14313

Signed-off-by: David B. Kinder <david.b.kinder@intel.com>
2019-03-15 05:47:19 +01:00
Michael Scott
4278bf5673 scripts/dts: Fix for PyYAML 5.1 Loader changes
PyYAML 5.1 was just released and it doesn't support !include as
previous versions do.  This breaks our DTS bindings parsing.

Let's fix our extract_dts_include.py script to work with both
3.13 and 5.1.

Also, update the pyyaml requirement to >=3.13 to be sure we're
compatible.

Fixes: https://github.com/zephyrproject-rtos/zephyr/issues/14496

Signed-off-by: Michael Scott <mike@foundries.io>
2019-03-13 23:21:42 -05:00
Kumar Gala
1230ae1e63 scripts/sanitycheck: Add 'vectors' section to whitelist
Add 'vectors' to the whitelisted list of linker sections.  This is
needed by the openisa_rv32m1 SoC.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-03-13 17:02:05 -05:00
Daniel Leung
212ec9a29a linker: sort app shared mem partition by alignment
If CONFIG_MPU_REQUIRES_POWER_OF_TWO_ALIGNMENT is enabled,
the app shared memory partition may cause waste of memory
due to the need for padding.

For example, tests/subsys/jwt and board mps2_an385:

  z_test_mem_partition: addr 0x20000000, size 52
  z_libc_partition    : addr 0x20000040, size 4
  k_mbedtls_partition : addr 0x20008000, size 32736

    ending at 0x2000ffff, taking up 65536 bytes

With power-of-two size and alignment requirement,
k_mbedtls_partition takes up 32KB memory and needs to be
aligned on 32KB boundary. If the above partitions are
ordered as shown, there needs to be a lot of padding
after z_libc_partition before k_mbedtls_partition can
start. In order to minimize padding, these partitions
need to be sort by size in descending order.

After the changes here,	the partitions are:

  k_mbedtls_partition : addr 0x20000000, size 32736
  z_test_mem_partition: addr 0x20008000, size 52
  z_libc_partition    : addr 0x20008040, size 4

    ending at 0x2000805f, taking up 32864 bytes

With the above example, sorting results in a saving
of 32672 bytes of saving.

Fixes #14121

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2019-03-13 18:26:51 +01:00
Erwan Gouriou
7f84787c13 scripts: fix path issue in push-hook script
series-puh-hook.sh script was pushed with local path
instead of ${ZEPHYR_BASE}.
Fix this.

Signed-off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
2019-03-13 11:59:38 -05:00
Marc Herbert
0c465bbec1 sanitycheck: make --subtest and --test mutually exclusive
They're effectively mutually exclusive already because
options.sub_test in main() immediately discards any --test
argument(s). This commit preempts user confusion thanks to this new
message:

 sanitycheck: error: argument --sub-test: not allowed with
    argument -s/--test

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-03-13 01:44:52 +01:00
Carles Cufi
c5daabe190 west_commands: pyocd: Sector erase by default
Due to issues with the implementation of the default auto-erase in
pyocd, sometimes the chip is mass-erased even when not intended. To
avoid this issue, default to forcing sector erasing unless mass erasure
is explicitly requested by the user with additional flash options.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-03-12 16:52:14 +01:00
Andrew Boie
c78c5e6936 userspace: add additional script documentation
We have several scripts used by the build system related
to generating code for system calls, privileged mode stacks,
kernel object metadata, and application shared memory
partitions. Add some overview documentation for each.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-03-12 08:37:58 +01:00
Marc Herbert
edf17591dd sanitycheck: group case selection options in the help message
$ sanitycheck -h # is re-ordered like this:

 < ... all other options ... >

 -C, --coverage        Generate coverage reports. Implies
                       --enable_coverage

 --coverage-platform   COVERAGE_PLATFORM
                       Plarforms to run coverage reports on. This
                       option may be used multiple times.

Test case selection:

 -f, --only-failed     Run only those tests that failed the previous
                       sanity check invocation.

 -s TEST, --test TEST  Run only the specified test cases. These are
                       named by <path to test project relative to
                       --testcase-root>/<testcase.yaml section name>

 --sub-test SUB_TEST   Run only the specified sub-test cases and its
                       parent. These are named by test case name
                       appended by test function, i.e.
                       kernel.mutex.mutex_lock_unlock.
 --list-tests          list all tests.
 -F FILENAME, --load-tests FILENAME
                       Load list of tests to be run from file.
 -E FILENAME, --save-tests FILENAME
                       Save list of tests to be run to file.
 -T TESTCASE_ROOT, --testcase-root TESTCASE_ROOT
                       Base directory to recursively search for test
                       cases. All testcase.yaml files under here will
                       be processed. May be called multiple times.
                       Defaults to the 'samples' and 'tests' directories
                       in the Zephyr tree.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-03-12 08:32:49 +01:00
Patrik Flykt
4344e27c26 all: Update reserved function names
Update reserved function names starting with one underscore, replacing
them as follows:
   '_k_' with 'z_'
   '_K_' with 'Z_'
   '_handler_' with 'z_handl_'
   '_Cstart' with 'z_cstart'
   '_Swap' with 'z_swap'

This renaming is done on both global and those static function names
in kernel/include and include/. Other static function names in kernel/
are renamed by removing the leading underscore. Other function names
not starting with any prefix listed above are renamed starting with
a 'z_' or 'Z_' prefix.

Function names starting with two or three leading underscores are not
automatcally renamed since these names will collide with the variants
with two or three leading underscores.

Various generator scripts have also been updated as well as perf,
linker and usb files. These are
   drivers/serial/uart_handlers.c
   include/linker/kobject-text.ld
   kernel/include/syscall_handler.h
   scripts/gen_kobject_list.py
   scripts/gen_syscall_header.py

Signed-off-by: Patrik Flykt <patrik.flykt@intel.com>
2019-03-11 13:48:42 -04:00
Daniel Leung
c8066c55fc linker: remove references to _k_task_list/_k_event_list
Commit 45a7e5d076 removed scripts/sysgen
and _k_task_list/_k_event_list sections are no longer being used.
So remove them.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2019-03-09 16:01:23 -05:00
Daniel Leung
bd40fc2c90 linker: remove references to no longer exist _k_memory_pool
Commit 73cb9586ce renamed linker
section from _k_memory_pool to _k_mem_pool, but the references
in linker scripts are still there. So remove them now.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2019-03-09 16:01:23 -05:00
Erwan Gouriou
831fbed3a7 scripts: Add hook calllable series-push-hook script
Running checkpatch in pre-commit hook could be a constraint
for day to day work. Though, running it before pushing to
zephyr repo is highly recommended.
Add a  series push script callable from pre-push hook.
Document how to use it in contribute section.

Signed-off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
2019-03-08 16:43:06 -05:00
Carles Cufi
b710177a27 west: commands: build: Specify source dir without a flag
In order to simplify the usage of `west build`, take a positional
argument with the source directory instead of requiring the `-s,
--source-dir` flag. This makes it easier and quicker to invoke west when
building, as well as being consistent with CMake.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-03-08 21:19:42 +01:00
Sebastian Bøe
f96c9bc75a kconfig: Have ninja Re-run CMake when Kconfig sources change
Users often get confused when they change Kconfig sources and then
rebuild only to discover that nothing happens. To fix this we add a
dependency between re-running cmake, and all Kconfig sources, similair
to how touching CMakeLists.txt files cause CMake to re-run.

This fixes #5634

Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
2019-03-07 10:40:00 +01:00
Marc Herbert
f78288be20 gen_kobject_list.py: OrderedDict for < 3.6 determinism
Dictionaries are iterated in a random order by Python 3.5 and before.
This could have caused "Unstable" CI in PR #13921 and maybe others.
Anyway we want builds to be determimistic by default. Explicit
randomness can be added for better coverage but not by default.

1. When running "make kobj_types_h_target" repeatedly one can observe
that the following .h files keep changing in
build/zephyr/include/generated/:

- kobj-types-enum.h
- otype-to-str.h
- otype-to-size.h

Switching kobjects to OrderedDict makes these 3 .h files deterministic.

2. When running this test repeatedly with CONFIG_USERSPACE=y:

  rm build/zephyr/*.gperf && make -C build obj_list

... the dict used for --gperf-output seems to be deterministic, probably
because its keys are all integers (memory addresses). However we can't
take that for granted with Python < 3.6 so out of caution also switch
the output of find_objects() in elf_helper.py to a sorted OrderedDict.

PS: I would normally prefer official Python documentation to
StackOverflow however this one is a good summary and has all the
multiple pointers to the... official Python documentation.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-03-06 20:58:58 -05:00
Marc Herbert
73cb0bf306 extract_dts_includes.py: deterministic flash table with Python < 3.6
To reproduce the issue this addresses, run "cmake -B96b_nitrogen"
twice with Python 3.5. Observe the order of the flash partition table
changing in:
"build{1,2}/zephyr/include/generated/generated_dts_board.conf" and
"generated_dts_board_unfixed.h"

Dictionaries are iterated in a random order by Python 3.5 and before.
This could have caused "Unstable" CI in PR #13921 and maybe others.
Anyway we want builds to be determimistic by default. Explicit
randomness can be added for better coverage but not by default.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-03-06 20:58:58 -05:00
Ulf Magnusson
ac059db362 kconfiglib: Give more helpful hints for missing files
Sigvart pointed out that the error message for missing Kconfig files
(which rambles on a lot about $srctree) can easily kick people off in
the wrong direction.

Update Kconfiglib to upstream revision 9f26eb3ae3 to add the following
commit, which makes the error message point to stuff that's more likely
to be the problem:

    Give more helpful error messages when files are missing

    The current error message talks a lot about $srctree, but $srctree
    is seldom the culprit in practice. More common is 'source
    "$(SOME_ENV_VAR)/foo"`, where SOME_ENV_VAR hasn't been set.

    Include the complete 'source ...' line for missing Kconfig files,
    and mention unset environment variables as a hint. Only mention
    $srctree briefly.

    Also shorten the message when a .config can't be a found a bit. This
    message would usually only be seen when working directly with the
    library.

Example error:

    Kconfig:7: '/Kconfig' not found (in 'source
    "$(SOME_ENV_VAR)/Kconfig"'). Check that environment variables are
    set correctly (e.g. $srctree, which is unset or blank). Also note
    that unset environment variables expand to the empty string.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-03-06 12:07:50 +01:00
Andrew Boie
f084c38d44 userspace: properly namespace linker vars
App shared memory defines a bunch of symbols in the
linker script. Namespace them properly as private
zephyr variables.

The variables which indicate the bounds of the entire
partition now end with "_part_start", "_part_size",
and "_part_end" to make them easy for scripts to
distinguish them from other generated symbols for
data/bss sizes.

Finally, the bss size is not rounded up, this was
causing unnecessary memory to be zeroed.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-03-05 10:02:27 -08:00
Anas Nashif
f4d8eccb70 sanitycheck: fixed handling of retries
When re-running failed tests, do not go through any filtering, just load
the failed tests directly and execute them. The filtering was done based
on default platforms and any tests that were failing on non-default
platforms were ignored.

Fixes #13956

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-03-04 14:08:12 -05:00
Anas Nashif
c9d1bb90d8 ci: add option to specify custom range
This is useful when running the script locally on a set of commits to
reproduce how things are run in CI.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-03-04 14:08:12 -05:00
Anas Nashif
5923760dd5 sanitycheck: handle serial exception
Do not throw excpetion when there is no serial output, just timeout.

Fixes #13541

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-03-04 08:14:29 -05:00
Marc Herbert
6b4cda9b25 run_ci.sh: git log what [base] version is being tested
Zephyr Continuous Integration made the choice of rebasing pull requests
on its fast moving master branch before testing them. Unfortunately the
Shippable interface assumes the other choice[*] and highlights the SHA1
of the pull request _before_ rebasing which is very misleading.  For now
the only way to realize this disconnect is to scrutinize the build logs
and/or this source code (or tribal knowledge).

To mitigate and reduce a bit this confusion, run a short:
  git log -n 5 --oneline --decorate
before and after rebase.

Making it worse, Shippable sub-runs are not pegged to a common base
version. For instance the reason why shippable sub-run 35578.1 passed
while all other 35578.x sub runs failed is very likely because PR #13803
was merged to master in the mean time. This commit would have been able
to prove that.

[*] In an ideal world with infinite time and resources, Continuous
Integration would test pull requests BOTH alone and combined with the
moving tip.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-03-01 09:39:31 +01:00
Marc Herbert
f7e995c075 run_ci.sh: run pytest-3 if found
On (at least) Fedora "pytest" is python2 and python2 only.

Also update remote name in sample invocation from "upstream" to "origin"
because that's how west sets things up by default (and this script
requires west)

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-03-01 09:39:31 +01:00
Paul Sokolovsky
c2a70a2710 scripts: size_report: Fix output of header break line
Was apparently an artifact of Python2 to Python3 conversion. Led to
printing of literal '='*110i instead of a line of ='s.

Signed-off-by: Paul Sokolovsky <paul.sokolovsky@linaro.org>
2019-02-28 09:23:16 -08:00
Marc Herbert
29c19c1b0f filter-known-issues.py: clarify what "new" means
When building incrementally, filter-known-issues.py reports a varying
and totally different set of "new" issues than when building from
scratch. Warnings for unrelated upstream code disappearing and
re-appearing are especially confusing. Expand the messages a bit to
clarify.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-02-28 18:18:30 +01:00
Ulf Magnusson
d3bf4b432c scripts/dts: Remove dead 'aliases' check in insert_defs()
insert_defs() is the only function that adds keys to 'defs', and they
always get added with an 'aliases' key.

This function could get super simple if the duplicate-removal code could
be removed. It only seems to be needed to prevent duplicates for some
(unfortunately tricky) flash-related stuff now.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-27 10:02:35 -06:00
Ulf Magnusson
25ff4b72a9 scripts/dts: Remove global 'structs' variable
Unused.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-27 10:02:35 -06:00
Ulf Magnusson
59a0c4331d scripts/dts: Fix binding_compats/bindings_compat naming mismatch
'binding_compats' makes the most sense, so use that everywhere.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-27 10:02:35 -06:00
Ulf Magnusson
5999f10ba2 scripts/dts: Get rid of deepcopy() and 'except:' in translate_addr()
- No deep copying is needed. The deepcopy() is just a hack to work
   around that the value might not be a list. That deserves a comment at
   least.

 - A catch-all try/except will hide stuff like misspelled variables as
   well. It's usually a bad idea.

Rewrite things to be more explicit, and add some comments.

reg.py was indirectly using the deepcopy() imported in global.py,
because it does 'from global import *'. Have it import deepcopy() itself
instead. reg.py is the only remaining user of deepcopy().

reg.py relying on the deepcopy() import in global.py was super hard to
discover, due to another catch-all try/except around the deepcopy()
call.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-27 10:02:35 -06:00
Ulf Magnusson
9d4c2921ee scripts/dts: Remove redundant 'children' check in create_chosen()
All nodes have a 'children' key.

Also simplify a bit. The loop is copying a dictionary verbatim.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-27 10:02:35 -06:00
Ulf Magnusson
5edeec3dc4 scripts/dts: Simplify and document get_compat()
Get rid of the try/except and the find_parent_prop() call. Also add some
documentation.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-27 10:02:35 -06:00
Ulf Magnusson
e0f55f17e8 scripts/dts: Fix get_parent_path() for top-level nodes
'/foo' was turned into ''. Turn it into '/' instead. That's how the root
is represented in 'reduced' too.

Also remove some code from get_addr_size_cells() that was only there to
work around the bug.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-27 10:02:35 -06:00
Ulf Magnusson
bc73ba6fc3 scripts/dts: Remove get_node_compats()
Unused.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-27 10:02:35 -06:00
Ulf Magnusson
392587b2a5 scripts/dts: Remove redundant 'children' check in create_aliases()
All nodes have a 'children' key.

Also simplify and improve naming a bit.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-27 10:02:35 -06:00
Ulf Magnusson
246a15c4db scripts/dts: Never ignore properties in generate_node_defines()
If a binding specifies 'generate: define' for 'interrupt-names' (like
some bindings do), then that ought to generate #defines for it, but the
scripts/dts code currently hardcodes 'interrupt-names' to be ignored
(along with some other properties).

Maybe the 'generate: define' in those bindings is a mistake, but the
code still ought to respect it. That also gets rid of some mystery code.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-27 10:02:35 -06:00
Andrew Boie
cfde940d6d x86: relocate some arch-specific scripts
These scripts are specific to the x86 port.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-02-25 13:15:37 -08:00
Andrew Boie
4ce652e4b2 userspace: remove APP_SHARED_MEM Kconfig
This is an integral part of userspace and cannot be used
on its own. Fold into the main userspace configuration.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-02-23 07:43:55 -05:00
Ulf Magnusson
79906fc481 scripts/dts: Document the generate_*_defines() functions
Add some short doc comments at the beginning so that people can quickly
get an idea of what they're about.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-22 12:44:16 -06:00
Ulf Magnusson
e04139f506 scripts/dts: Rename extract_property() to generate_prop_defines()
Consistent with generate_defines() and generate_node_defines().

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-22 12:44:16 -06:00
Ulf Magnusson
f5b17d4138 scripts/dts: Call /foo/bar a "path" instead of an "address"
It's confusing that "address" is often used within the same function to
refer to both node paths and e.g. address cells.

Make things easier to understand by calling /foo/bar a path instead.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-22 12:44:16 -06:00
Ulf Magnusson
5b791d1eea scripts/dts: Generate node-level #defines outside extract_property()
extract_property() is meant to generate #defines for a single property,
like 'foo = <1 2 3>'. Currently, it also generates node-level #defines
related to parent buses.

That makes the intent of the code hard to understand, and also means
that identical node-level #defines get redundantly added multiple times
(once per property).

Generate the node-level bus #defines before processing properties, in
generate_node_defines(). Use a new generate_bus_defines() helper.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-22 12:44:16 -06:00
Ulf Magnusson
edc1f6af7a scripts/dts: Move flash partition handling out of property loop
Due to the way the code was structured,
flash.extract_partition(node_path) was called multiple times for the
same 'node_path'.

That must've been a mistake to begin with (but was hard to spot before
the code was cleaned up). Move the generation of per-node #defines out
of the property loop.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-22 12:44:16 -06:00
Carles Cufi
c2d5e7b01c kconfig: Don't load env var if in doc mode
Do not load the GENERATED_DTS_BOARD_CONF if in doc mode, since it will
not defined as it doesn't apply. No need to defined it to a dummy value.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-02-22 10:14:21 +01:00
Andrew Boie
42d1481fb0 gen_mmu_x86: fix overlapping region detection
Fixes: #13515

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-02-21 09:49:19 -08:00
Ulf Magnusson
858701f67a scripts/dts: Remove redundant compat check in generate_defines()
generate_node_defines() immediately returns if the node's 'compatible'
is not in the binding.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-20 12:17:31 -06:00
Ulf Magnusson
75c6d2cd02 scripts/dts: Simplify generate_node_defines()
Merge stuff that no longer needs to be separate. Also improve naming.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-20 12:17:31 -06:00
Ulf Magnusson
797eaa78f6 scripts/dts: Remove redundant extract_property() parameters
Derive 'node_compat' and 'prop_val' (the dictionary for the property
from the YAML binding) inside extract_property().

That gives it just two parameters and makes it clearer that it's just
generating #define's for a single device tree property. 'prop_val' was
only used to look up prop_val['type'].

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-20 12:17:31 -06:00
Ulf Magnusson
e539479d4f scripts/dts: Move prop_names() call into extract_property()
Moves it closer to where it's used, and will allow other simplifications
in generate_node_defines().

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-20 12:17:31 -06:00
Ulf Magnusson
7bb57fe71f scripts/dts: Remove confusing deepcopy() in generate_node_defines()
After lots of reverse-engineering, I understand why deepcopy() is used:

 1) Various functions defined in scripts/dts/extract/* modify the list
    of names in-place

 2) A plain list.copy() won't work when the *-names property happens to
    have a single name in it, because devicetree.py gives a string
    instead of a list in that case

Using deepcopy() to solve (2) is very confusing (especially with no
comments), because no deep copying is actually needed.

Get rid of deepcopy(), add a helper function for fetching the names, and
some comments.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-20 12:17:31 -06:00
Ulf Magnusson
50ab1d8e5f scripts/dts: Clean up create_reduced()
- Remove dead "if 'props' in nodes" check

 - Rename the 'nodes' parameter to 'node'. It represents a single node.

 - Use a defaultdict(int) for 'last_used_id' (makes unset keys give 0)

 - Use a global for 'last_used_id'

 - Move some related code so it appears together

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-20 12:17:31 -06:00
Ulf Magnusson
8c3aabb2c2 scripts/dts: Remove find_node_by_path()
Unused.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-20 12:17:31 -06:00
Ulf Magnusson
07abb05bd9 kconfiglib: Clarify kconfig_filenames doc re. absolute paths
Update Kconfiglib (and menuconfig, just to sync) to upstream revision
99a7af769352b, to add the commit below, for a doc issue reported by
Sebastian Bøe:

  Document that kconfig_filenames keeps absolute paths as-is

  Came up in https://github.com/ulfalizer/Kconfiglib/issues/67.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-20 11:22:17 -06:00
Paul Sokolovsky
26e562c0bd scripts: gitlint: Consistent case for Signed-off-by
The commit header as added by "git commit -s" is "Signed-off-by:",
and not any other casing. While it perhaps case-insensitive, use
canonical casing to not confuse (attentive) users.

Signed-off-by: Paul Sokolovsky <paul.sokolovsky@linaro.org>
2019-02-19 12:12:42 -05:00
Ulf Magnusson
a8c851771c scripts/dts: Remove block in extract_property()
Turn

  if a:
    if b:
      ...

into

  if a and b:
    ...

Simplify some error messages with .format() as well, and get rid of some
redundant str()s. '{0} {1} {1}'.format('foo', 'bar') gives
'foo bar bar'.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-19 07:44:44 -06:00
Ulf Magnusson
af3580c314 scripts/dts: Clean up generate_node_defines() a bit
- Reduce the indentation and remove the awkward line breaks. This is
   possible now that there's no recursion.

 - Be consistent with quotes

 - Move some initialization closer to where it's used

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-19 07:44:44 -06:00
Ulf Magnusson
44467ef8dc scripts/dts: Remove recursion from generate_node_defines()
The recursion was used to implement 'properties:' within 'properties:'
in binding files, which seems to be a dead leftover (and undocumented).

Removing it gets rid of code and makes things more transparent.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-19 07:44:44 -06:00
Ulf Magnusson
b80905fd8e scripts/dts: Inline filter_list in generate_node_defines()
Makes it clear that it's a static list, and moves the values closer to
where they're used.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-19 07:44:44 -06:00
Ulf Magnusson
c20598c84f scripts/dts: Clarify generate_defines() a bit
Use more descriptive naming, get rid of unused 'v' variable, and an
unnecessary None check (None won't appear in get_binding_compats()).

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-19 07:44:44 -06:00
Ulf Magnusson
3c1d9e893b scripts/dts: Remove unnecessary 'reduced' parameter
...in generate_node_defines(). 'reduced' is available as a global.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-19 07:44:44 -06:00
Ulf Magnusson
90c73e3e5a scripts/dts: Rename extract_node_include_info()
...to generate_node_defines().

More explicit. "extract node include info" can be read in many ways:

 - Extract a node's "include info"

 - Extract node and include info

 - etc.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-19 07:44:44 -06:00
Ulf Magnusson
0e40c9615b scripts/dts: Move DT_CHOSEN_* generation into generate_defines()
That way it consistently generates all #define's.

Add some related clarifying comments to main() too.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-19 07:44:44 -06:00
Ulf Magnusson
cbf9afccbe scripts/dts: Rename generate_node_definitions() to generate_defines()
As in #define's. Easier to guess (to me anyway).

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-19 07:44:44 -06:00
Ulf Magnusson
1ca73e33a9 scripts/dts: Use re.fullmatch() instead of re.match(... + '$', ...)
Simpler, and makes it clearer that the regex'ing is deliberate.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-19 07:44:44 -06:00
Ulf Magnusson
95385744f9 scripts/dts: Fix indentation in extract_node_include_info()
The for-loop body was indented by 8 spaces instead of 4.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-19 07:44:44 -06:00
Maureen Helm
6de4578dc6 scripts: Update pyocd version to 0.16.1
Update the pyocd version to pick up a fix for the command 'pyocd erase',
which was broken in 0.15.0 and fixed in 0.16.0. Although we don't have
any in-tree uses of this command in zephyr, there will soon be in-tree
uses in mcuboot when the deprecated 'pyocd-flashtool' and 'pyocd-tool'
commands are replaced with the new unified pyocd commands.

Signed-off-by: Maureen Helm <maureen.helm@nxp.com>
2019-02-18 22:34:25 -05:00
Anas Nashif
f9e73c90f2 sanitycheck: do no overide OVERLAY_CONFIG
We have been overriding OVERLAY_CONFIG coming from tests. Concat all
files and create one variable that we pass to kconfig as list.

Fixes #13320

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-02-18 10:49:08 -05:00
Marc Herbert
d5b2834f58 scripts/*syscalls.py: sort os.walk() for a more deterministic build
Found by "disordered --shuffle-dirents=yes".

Sorting os.walk() in scripts/subfolder_list.py removes the randomness in
the following files:

  build/zephyr/misc/generated/syscalls_subdirs.txt
  build/zephyr/CMakeFiles/syscall_list_h_target.dir/ or build.ninja

Sorting os.walk() in scripts/parse_syscalls.py removes the randomness
in:

  build/zephyr/misc/generated/syscalls.json
  build/zephyr/include/generated/syscall_dispatch.c
  build/zephyr/include/generated/syscall_list.h

Note my (limited so far) testing did *not* observe any randomness in any
object file that this would address; the main purpose here is to remove
a very large amount of noise in diffoscope.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-02-17 07:32:05 -05:00
Kumar Gala
e6393dd22e scripts/sanitycheck: Add 'nocache' section to whitelist
Add 'nocache' to the whitelisted list of linker sections.

Fixes #13449

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-02-16 22:14:34 -05:00
Marti Bolivar
4a0f1f2817 west: sign.py: usability fixes
Fix some usability issues with this command.

- add help making it clear that either imgtool must be installed, or
  the path to imgtool.py must be provided using --tool-path.

- in case people don't read that, print a helpful message if imgtool
  is not installed and --tool-path is not provided.

- in case the build directory is not properly set up for an MCUboot
  chain-loaded image, make the BuildConfiguration inspection more
  robust, printing out errors using check_force() when values are
  missing.

- mark the --tool option required to print usage and avoid a
  RuntimeError if it is not provided.

- make sure we search for the default build directory before checking
  for its existence, in case it was not provided

Signed-off-by: Marti Bolivar <marti@foundries.io>
2019-02-15 21:07:10 +01:00
Marti Bolivar
f13fa537d7 west: fix issue with sign.py
The arguments have to be saved in self.args for check_force's error
path to work.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2019-02-15 21:07:10 +01:00
Tomasz Gorochowik
8adf39aae3 scripts: gen_kobject_list: Fix error log call
A trivial bug that caused the following error when this situation
actually happened:

  Too many thread objects (21)
  Traceback (most recent call last):
    File "/zephyr/scripts/gen_kobject_list.py", line 307, in <module>
      main()
    File "/zephyr/scripts/gen_kobject_list.py", line 281, in main
      -(-thread_counter // 8))
  TypeError: write() takes exactly one argument (2 given)

With this commit, the detected error is printed correctly, e.g:

  Too many thread objects (21)
  Increase CONFIG_MAX_THREAD_BYTES to 3

Signed-off-by: Tomasz Gorochowik <tgorochowik@antmicro.com>
2019-02-15 10:19:52 -08:00
Andrew Boie
eea0f93a83 x86: generate user mode page tables
If kernel page table isolation is enabled, we generate a second
set of page tables. These tables, except for the shared page, have
all non-user pages marked as non-present.

The MMU generation script has been refactored:

- Debugging output has been make significantly simpler and less
  verbose

- Useless globals removed or adjusted

- MMU region list is validated as it is read

- Some tuples unpacked into individual variables to make the
  code easier to read.

- Useless command line option for output binary endian-ness
  remobved

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-02-14 12:46:36 -05:00
Ulf Magnusson
732fa65956 scripts/dts: Rename dict_merge() to merge_properties()
It's special-cased for merging bindings and not a general dictionary
merging function.

Also simplify the documentation a bit.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-14 10:49:44 -06:00
Ulf Magnusson
14de997f25 scripts/dts: dict_merge(): Rename dct/merge_dct to to_dict/from_dict
Makes it easier to remember what's being merged into what. "merge_dct"
is ambiguous.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-14 10:49:44 -06:00
Ulf Magnusson
298b4431b9 scripts/dts: Do title/version/description special-casing in dict_merge()
Simpler to do it there than in merge_included_bindings().

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-14 10:49:44 -06:00
Ulf Magnusson
f0eeb11fc0 scripts/dts: Add helper for reporting !include-related errors
Makes the message end up on stderr along with the exception, and removes
some code.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-14 10:49:44 -06:00
Ulf Magnusson
35b3d440c1 scripts/dts: Replace the Bindings class with plain functions
The Bindings class was only used to implement '!include foo.yaml'
(easier to see after some things were moved out of it). Use plain
functions instead, which might be a bit more transparent and gives
simpler code.

Also remove the recursive '!include' detection for now, which is broken
in that the same .yaml being included twice will always trigger it, even
for non-circular cases.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-14 10:49:44 -06:00
Ulf Magnusson
4a99dbcf9e scripts/dts: Simplify and remove dead code in Bindings._extract_file()
As written, this function could take e.g. '!include bar', find 'foo-bar'
(due to the endswith()), and include it. This is undocumented and
doesn't seem to be used, so I'm guessing that it isn't intentional.

Removing that bug/feature also makes the code much simpler.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-14 10:49:44 -06:00
Ulf Magnusson
32e6565fd7 scripts/dts: Merge Bindings.binding() into load_bindings()
load_bindings() is trivial, and it allows some stuff to simplified, like
the check for no bindings being found, and the assignments to the
extract.globals.{bindings,bus_bindings,bindings_compat}.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-14 10:49:44 -06:00
Ulf Magnusson
706809e9b0 scripts/dts: Turn Bindings._included into a global
This is (probably) a temporary change just to be able to move some stuff
out of the Bindings class.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-14 10:49:44 -06:00
Ulf Magnusson
a1f1969331 scripts/dts: Separate out finding binding (.yaml) files
As a step in moving some functions out of the Bindings class.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-14 10:49:44 -06:00
Ulf Magnusson
d859f4f937 scripts/dts: Move initialization logic into load_bindings()
Makes main() more readable.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-14 10:49:44 -06:00
Ulf Magnusson
e17af82190 scripts/dts: Rename load_yaml_descriptions() to load_bindings()
More explicit, and consistent with other code.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-14 10:49:44 -06:00
Kumar Gala
1f1282afa4 scripts/dts/extract_dts_includes: Limit conf file to DT prefix defines
Only write out to the .conf file if the define starts with "DT_".  The
the conf file should only be used by sanitycheck, west, and
kconfigfunctions at this time.  In the future we should remove it, so
lets limit what's it exposing at this time.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-02-14 10:45:25 -06:00
Kumar Gala
b814af827d scripts/dts/extract: Fix how we determine DT_FLASH_AREA_<X>_DEV
DT_FLASH_AREA_<X>_DEV should be the label of the flash controller.  For
flashes that are "jedec,spi-nor" the flash itself is considered the
controller.  For "soc-nv-flash" the parent of the flash is assumed to be
the controller.

Rework the logic in how we determined DT_FLASH_AREA_<X>_DEV to handle
both cases.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-02-14 05:13:35 -06:00
Ulf Magnusson
4e02590231 scripts/dts: Change 'Error::' to 'Error:'
Typo.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-13 08:53:07 -06:00
Ulf Magnusson
b29a5b0e83 scripts/dts: Add a check_binding_properties() function
Removes some logic from merge_included_bindings() and makes it easier to
follow.

Also remove an outdated comment ('id' no longer exists) and some
comments that might not be helpful.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-13 08:53:07 -06:00
Ulf Magnusson
2b8766db53 scripts/dts: Improve documentation for merge_included_bindings()
Use a plain-English description.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-13 08:53:07 -06:00
Ulf Magnusson
3f8616add9 scripts/dts: Get rid of yaml_list in Bindings.bindings()
The yaml_list dictionary is three regular variables in disguise, as keys
are never added. Use regular variables instead, which is less confusing.

Hopefully the naming makes sense.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-13 08:53:07 -06:00
Ulf Magnusson
9e8c3fdf16 scripts/dts: Fix recursion in merge_included_bindings()
The consistency checks at the beginning of the function were not done
for "leaf" .yaml files.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-13 08:53:07 -06:00
Ulf Magnusson
6a15130b9a scripts/dts: Rename yaml_traverse_inherited()
...to merge_included_bindings().

This gives a better hint to what it does (merges properties from
!include'd files).

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-13 08:53:07 -06:00
Ulf Magnusson
00b2ec05e5 scripts/dts: Document Bindings._include()
Explain that it implements !include.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-13 08:53:07 -06:00
Ulf Magnusson
b98cdc10f6 scripts/dts: Have Bindings._include() always return a list
Makes yaml_traverse_inherited() more straightforward.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-13 08:53:07 -06:00
Ulf Magnusson
105c5b426b scripts/dts: Remove support for !import
Implemented as an undocumented and unused synonym for !include.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-13 08:53:07 -06:00
Ulf Magnusson
339db385ef scripts/dts: Simplify Bindings._include()
Use a list comprehension for the '!include [foo.yaml, bar.yaml]' case.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-13 08:53:07 -06:00
Ulf Magnusson
377a794378 scripts/dts: Remove dead '!include <dict>' implementation
Remove the '!include {foo: bar}' support from Bindings._include(). Only
'!include foo.yaml' and '!include [foo.yaml, bar.yaml]' are documented,
and yaml_traverse_inherited() doesn't implement the dict case either.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-13 08:53:07 -06:00
Sebastian Bøe
1ca1075b20 cmake: kconfig: pass GENERATED_DTS_BOARD_CONF on to kconfig
Make kconfigfunctions.py agnostic to where GENERATED_DTS_BOARD_CONF is
located.

We don't want to encode the path of GENERATED_DTS_BOARD_CONF into more
places than necessary. Also, re-using GENERATED_DTS_BOARD_CONF makes
it easier to change where generated_dts_board.conf is located.

Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
2019-02-12 20:16:47 -05:00
Sebastian Bøe
cd4354317e kconfig: Have Kconfig fail when PROJECT_BINARY_DIR is not set
kconfigfunctions.py will silently set all values to '0' when the
environment variable 'PROJECT_BINARY_DIR' is not set.

But PROJECT_BINARY_DIR not being set indicates an internal error, so
we should instead error out with an appropriate error message.

Silently failing is dangerous and increases debug time. It is better
to fail fast.

Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
2019-02-12 20:16:47 -05:00
Erwan Gouriou
7fce83ff8e kconfigfunctions: Disambiguate reference to DTS generated database
Documentation for kconfigfunctions, mentions they look up elements
in "the DTS generated "conf" style database". This indication
could be cryptic for new zephyr users. Adding the exact name and
path of the file for disambiguation.

Signed-off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
2019-02-12 11:37:53 -06:00
Alexander Wachter
8757e780fc drivers: can: Update userspace handlers
This commit updates the userspace handlers to thle latest version.

Signed-off-by: Alexander Wachter <alexander.wachter@student.tugraz.at>
2019-02-12 09:39:30 -05:00
Anas Nashif
062d05645b ci: tests: do not build all tests when docs/images change
When we change documentation or board images, do not build all tests for
this board. Only add more tests when actual code has changed.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-02-09 17:22:08 -06:00
Anas Nashif
115b811016 ci: do not build all tests when docs/images change
When we change documentation or board images, do not build all tests for
this board. Only add more tests when actual code has changed.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-02-09 17:22:08 -06:00
Kumar Gala
561d6dd1ed scripts/dts/extract_dts_includes: Mark non DT_ defines as deprecated
Mark any define that doesn't start with DT_, LED, SW, or have PWM_LED in
the name as deprecated via __DEPRECATED_MACRO.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-02-09 04:47:59 -06:00
Ulf Magnusson
5c67ebe4a8 scripts/dts: Clean up Bindings.bindings()
Some more work will be done here later (yaml_list could be replaced by a
few plain variables, for example), but this is a start:

 - Extract the compat string from the 'compatible:' line directly in the
   regex, instead of separately

 - Flatten the code, turning

     if a:
       ...
       if b:
         ...
         if c:
           ...

   into

     if not a:
        continue

     ...

     if not b:
        continue

     ...

     if not c:
        continue

     ...

 - Add some comments

 - Use clearer naming

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-08 22:57:05 -06:00
Ulf Magnusson
4e5868f4af scripts/dts: Make all_compats() return a set with all compats
Previously, all_compats() returned a dictionary mapping nodes to their
'compatible' values. This dictionary was then (only) used to generate a
set() of all 'compatible' strings.

Save a step and get rid of some code by having all_compats() return a
set() with all 'compatible' strings directly.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-08 22:57:05 -06:00
Ulf Magnusson
4fcdd134c2 scripts/dts: Improve naming in create_phandles()
k, v -> child_name, child_node

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-08 22:57:05 -06:00
Ulf Magnusson
8c515b25a3 scripts/dts: Remove unused create_phandles() 'handle' parameter
Unused.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-08 22:57:05 -06:00
Ulf Magnusson
ace8450040 scripts/dts: Simplify get_all_compatibles()
- Rename to all_compats(). The "compats" part is consistent with
   elsewhere.

 - Don't require an empty dictionary to be passed in. Build the
   result within the function instead.

 - Use more specific names (k, v -> child_name, child_node)

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-08 22:57:05 -06:00
Ulf Magnusson
cb507889dd scripts/dts: Replace dts['/'] with 'root'
Factors out some code.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-08 22:57:05 -06:00
Ulf Magnusson
b52290fa58 scripts/dts: Rename get_chosen() to create_chosen()
It creates the global 'chosen' dictionary. get_*() makes it sound like
it returns a value.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-08 22:57:05 -06:00
Ulf Magnusson
28c76fd3d8 scripts/dts: Rename get_aliases() to create_aliases()
It creates the global 'aliases' dictionary. get_*() makes it sound like
it returns a value.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-08 22:57:05 -06:00
Ulf Magnusson
6d824feb69 scripts/dts: Rename get_phandles() to create_phandles()
It creates the global 'phandles' dictionary. get_*() makes it sound like
it returns a value.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-08 22:57:05 -06:00
Ulf Magnusson
ccf7de0306 scripts/dts: Rename get_reduced() to create_reduced()
It creates the global 'reduced' dictionary. get_*() makes it sound like
it returns a value.

Fix a comment that might make people think the global dictionaries are
Python lists too.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-08 22:57:05 -06:00
Ulf Magnusson
9eb0d33ca0 scripts/dts: Simplify extract_string_prop()
Check if the node is missing from 'defs' first, and insert it if not.
That way, later code can assume it exists.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-08 22:57:05 -06:00
Kumar Gala
3de6de305f west: sign: Convert FLASH_WRITE_BLOCK_SIZE to DT_
Use DT_FLASH_WRITE_BLOCK_SIZE prefixed defined instead of
FLASH_WRITE_BLOCK_SIZE as the non-DT version is deprecated.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-02-08 22:39:07 -06:00
Kumar Gala
65cd010fc1 west: sign: Convert FLASH_AREA to DT_FLASH_AREA
Use DT_FLASH_AREA prefixed defined instead of FLASH_AREA as the non-DT
version is deprecated.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-02-08 22:39:07 -06:00
Kumar Gala
ec63dca3ea scripts/dts/extract: Fix producing bogus alias if reg has no size
Its possible that a reg property is address only.  If that's the case we
should produce any alias information for the size.  We should check
nr_size_cells to decide to handle the alias case or not.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-02-08 22:39:07 -06:00
Kumar Gala
e3e87f624f scripts/dts/extract_dts_includes: Hand alias to alias for write_conf
If we have an alias pointing to another alias we would get None in the
resulting conf file.  However that alias might exist, so try looking it
up again in the aliases db.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-02-08 22:39:07 -06:00
Kumar Gala
5a20a418a2 scripts/dts/extract: Remove generation for FLASH_LABEL
FLASH_LABEL define isn't used anywhere so remove it.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-02-08 22:39:07 -06:00
Kumar Gala
873e4e950a scripts/dts/extract_dts_includes: Add DT_ prefix to FLASH_AREA defines
We generate the following FLASH prefixed defines:

	FLASH_AREA_MCUBOOT_LABEL
	FLASH_AREA_MCUBOOT_OFFSET
	FLASH_AREA_MCUBOOT_OFFSET_0
	FLASH_AREA_MCUBOOT_READ_ONLY
	FLASH_AREA_MCUBOOT_SIZE
	FLASH_AREA_MCUBOOT_SIZE_0

Now we generate DT_ prefixed versions:

	DT_FLASH_AREA_MCUBOOT_LABEL
	DT_FLASH_AREA_MCUBOOT_OFFSET
	DT_FLASH_AREA_MCUBOOT_OFFSET_0
	DT_FLASH_AREA_MCUBOOT_READ_ONLY
	DT_FLASH_AREA_MCUBOOT_SIZE
	DT_FLASH_AREA_MCUBOOT_SIZE_0

And will deprecate the non DT_ prefixed versions in the future.  We keep
them around as aliases for the time being.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-02-08 22:39:07 -06:00
Kumar Gala
2024f6b2a0 scripts/dts/extract_dts_includes: Add DT_ prefix to FLASH defines
We generate the following FLASH prefixed defines:
	FLASH_LABEL
	FLASH_ERASE_BLOCK_SIZE
	FLASH_WRITE_BLOCK_SIZE

Now we generate DT_ prefixed versions:
	DT_FLASH_LABEL
	DT_FLASH_ERASE_BLOCK_SIZE
	DT_FLASH_WRITE_BLOCK_SIZE

And will deprecate the non DT_ prefixed versions in the future.  We keep
them around as aliases for the time being.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-02-08 22:39:07 -06:00
Torsten Rasmussen
7e9d1bdda4 CMake/Kconfig: CMake and Kconfig build integration
This commit allows for Zephyr modules to be natively integrated into
the build system with CMakeLists.txt and Kconfig files.

The sourcing of module files are done in following order:
- If <module>/zephyr/module.yml exists, use cmake and kconfig settings
  for sourcing of additional file
- Else if <module>/zephyr/CMakeLists.txt exists, source this file into
  CMake build tree and add <module>/zephyr/Kconfig as osource

If none of the above files are present, the project is considered to
not be a Zephyr module

Signed-off-by: Torsten Rasmussen <torsten.rasmussen@nordicsemi.no>
2019-02-08 22:47:02 -05:00
David B. Kinder
1cc8bbb4ae doc: fix misspelling in kconfig doc
Fix typos introduced by PR #13186

Signed-off-by: David B. Kinder <david.b.kinder@intel.com>
2019-02-08 16:33:09 -06:00
Kumar Gala
c7606609be dts: Remove base_label support
We've removed all references to base_label in dts bindings and are
removing the functionality.  We should depend on DT_ prefixed defines
going forward.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-02-08 14:44:42 -06:00
Carles Cufi
acc82cee00 scripts: west: Tweak west build doc
Clarify what "being in a build directory" exactly means.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-02-08 14:58:28 -05:00
Kumar Gala
f2ef52f122 kconfig: kconfigfunctions: update dt_str_val
Clarify the docs for dt_str_val that if the name isn't found we return
and empty string.  Also cleanup the code slightly as we don't need to
escape the double quote.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-02-08 12:02:18 -06:00
Kumar Gala
ff70b3444f dts: Convert CONFIG_ to DT_ symbols for chosen props
Replace generating CONFIG_ symbols with DT_ symbols for chosen
properties like 'zephyr,console' or 'zephyr,bt-mon-uart'.  We now use a
kconfigfunctions (dt_str_val) to extract the info from dts into Kconfig.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-02-08 11:41:26 -06:00
Kumar Gala
2579adea1e kconfig: kconfigfunctions: Add dt_str_val function
Add dt_str_val to extract a string from the dt conf database.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-02-08 10:29:57 -06:00
Kumar Gala
bfaaa6bbe9 dts: Convert CONFIG_CCM to DT_CCM
Since we know do DTS before Kconfig we should try and remove dts from
creating Kconfig namespaced symbols and leave that to Kconfig.  So
rename CONFIG_CCM_<FOO> to DT_CCM_<FOO>.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-02-08 10:29:57 -06:00
Marti Bolivar
5591ca2374 sanitycheck: various fixes for device testing
Device testing has various problems:

- the main thread doesn't halt the serial monitoring thread when an
  error occurs while flashing, meaning flash errors can look like
  timeouts

- serial_line is unbound if the ser.readline() call raises an
  exception, so the following "if serial_line" would raise NameError
  in that case

- the serial monitoring thread is not a daemon, so exceptions or
  errors in the handle() thread that cause thread exit will leave the
  process sitting around until the serial monitoring thread has a
  chance to exit

Fix these issues:

- Rewrite the serial monitoring thread as a select loop over the
  serial port file descriptor and a pipe shared by the main thread and
  serial monitoring thread. If flashing fails, the main thread uses
  the pipe to signal to the serial monitoring thread that it should
  exit.

- Make sure serial_line is always bound when read.

- Mark the serial thread daemonic.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2019-02-08 08:08:32 -05:00
Marti Bolivar
303b52268a sanitycheck: print flash output if VERBOSE
If the user has requested verbose output, then use check_call() when
running the flash target so that the build system output goes to the
controlling terminal's stdout instead of being swallowed.

This makes it easier to debug flashing failures.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2019-02-08 08:08:32 -05:00
Andrew Boie
41f6011c36 userspace: remove APPLICATION_MEMORY feature
This was never a long-term solution, more of a gross hack
to get test cases working until we could figure out a good
end-to-end solution for memory domains that generated
appropriate linker sections. Now that we have this with
the app shared memory feature, and have converted all tests
to remove it, delete this feature.

To date all userspace APIs have been tagged as 'experimental'
which sidesteps deprecation policies.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-02-08 07:04:30 -05:00
Andrew Boie
686bd913c3 gen_app_partitions: add library support
Zephyr may be linked against third-party libraries which
were not part of the build. These may contain globals
which need to end up in a memory partition.

We can now specify the names of these libraries, as well
as a destination partition for their globals.

Some excessively long variables were renamed.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-02-08 07:04:30 -05:00
Andrew Boie
7adff462e7 app_shmem: overhaul partition specification
* K_APP_DMEM_SECTION/K_MEM_BMEM_SECTION macros now exist
  to specifically define the name of the sections for data
  and bss respectively.

* All boards now use the gen_app_partitions.py script, the
  padding hacks for non-power-of-two arches didn't work right
  in all cases. Linker scripts have been updated.

* The defined k_mem_partition is now completely initialized
  at build time. The region data structures now only exist
  to zero BSS.

Based on some work submitted by Adithya Baglody
<adithya.baglody@intel.com>

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-02-08 07:04:30 -05:00
Anas Nashif
45a9786a41 sanitycheck: support filtering based on cmake cache
Parse CMakeCache.txt and filter based on variables defined in CMake.
The CMakeCache.txt parsing is borrowed from west.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-02-07 22:46:10 -06:00
Ulf Magnusson
298f9e1e8a scripts/dts: Rename get_key_value() to define_str()
As in "#define string". More explicit.

Rename the parameters to name/value instead of k/v too.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-07 21:42:36 -06:00
Ulf Magnusson
638e89bb7c scripts/dts: Simplify tabstop logic in write_header()
A string can be repeated n times with n*'foo'.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-07 21:42:36 -06:00
Ulf Magnusson
0e922bff75 scripts/dts: Simplify write_header()
That the 'aliases' key isn't a regular property makes things a bit
confusing. Write the aliases separately, which should be a bit clearer.
That also makes it clear that the order doesn't matter.

The 'aliases' key always exists in defs[node], so use [] instead of
.get(). Also write "'aliases'" instead of "prop" when the value is
known.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-07 21:42:36 -06:00
Ulf Magnusson
98df8b99d6 scripts/dts: Remove unused os.listdir import
Unused.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-07 21:42:36 -06:00
Ulf Magnusson
72f0125b92 scripts/dts: Rename conf/header-writing functions to write_conf/header
Probably easier to understand.

Also rename 'fd' to 'f'. "File descriptor" might make it sound like an
integer.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-07 21:42:36 -06:00
Ulf Magnusson
00c43d4d08 scripts/dts: Simplify output_keyvalue_lines()
That the 'aliases' key isn't a regular property makes things a bit
confusing. Write the aliases separately, which should be a bit clearer.
That also makes it clear that the order doesn't matter.

Also write "'aliases'" instead of "prop" when the value is known.

The .get() seems to be needed to work around a bug, and sometimes you
get DT_...=None in the output. Might look into that later.

extract_string_prop() could generate 'defs' entries without 'aliases'
keys. Make it add an empty 'aliases' dict instead, to remove existence
checks elsewhere.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-07 21:42:36 -06:00
Ulf Magnusson
7b0ac7d303 scripts/dts: Simplify get_parent_address()
Returns the empty string for '/foo' and '/', but the old version did
that too, so maybe it's intended.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-07 21:42:36 -06:00
Ulf Magnusson
a776927250 scripts/dts: Simplify find_parent_prop()
Also replace a .get() where the key is known to exist.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-07 21:42:36 -06:00
Ulf Magnusson
87193b0e1b scripts/dts: Remove generate_node_definitions() return value
Unused.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-07 21:42:36 -06:00
Ulf Magnusson
4217de046b scripts/dts: Inline simple single-caller functions into main()
Inline load_and_parse_dts() and generate_{keyvalue,include}_file().

Also specify UTF-8 for writing files, and fix some mis-indentation.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-07 21:42:36 -06:00
Ulf Magnusson
2528b538a4 scripts/dts: Remove redundant nargs=1 argparse args
Flags take a single argument by default, and removing nargs=1 avoids
getting it as a single-element list.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-07 21:42:36 -06:00
Ulf Magnusson
6f5d5c56f2 scripts/dts: Simplify code around extract_bus_name() call
Get rid of the parent_label variable.

Also avoid using "label" in two different senses (DT label and name of
define macro) within the same comment.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-07 21:42:36 -06:00
Ulf Magnusson
7827c765aa scripts/dts: Rename extract_single() to extract_bus_name()
This matches the only way in which it's used.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-07 21:42:36 -06:00
Ulf Magnusson
6127f0a07f scripts/dts: Simplify extract_single()
Get rid of the 'prop' variable.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-07 21:42:36 -06:00
Ulf Magnusson
c631edddb8 scripts/dts: Inline 'prop' and 'key' params into extract_single()
These are always 'parent-label' (used as a flag) and 'bus-name',
respectively.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-07 21:42:36 -06:00
Ulf Magnusson
636b6be62d scripts/dts: Remove the prop_def variable in extract_single()
Shorter and still clear without it.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-07 21:42:36 -06:00
Ulf Magnusson
f5ee8f65a7 scripts/dts: Remove dead type-checking code in extract_single()
'prop' is always a string.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-07 21:42:36 -06:00
bde3224c91 scripts: bossac: Support flashing at an offset.
The Adafruit SAMD51 boards use a combined BOSSA / U2F bootloader that
resides in the first 16 KiB.  Add support for pulling the application
partition offset from the configuration and using that to offset where
the binary gets flashed.

Signed-off-by: Michael Hope <mlhx@google.com>
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-02-07 15:13:11 -05:00
Findlay Feng
69c15f0ed1 dts: flash_map: Automatically generate of the list of flash area
This Patch add functionality for automatic generation of the flash map
using DTS description. Automatic generation allows to replace
C-hardcoded flash_map.

We generate a set of defines based on the index of a partiion:
	#define DT_FLASH_AREA_<IDX>_OFFSET	0
	#define DT_FLASH_AREA_<IDX>_SIZE	131072
	#define DT_FLASH_AREA_<IDX>_DEV		"FLASH_CTRL"
	#define DT_FLASH_AREA_<IDX>_LABEL	MCUBOOT

Additionally we also define:
	#define DT_FLASH_AREA_NUM		4

and:
	#define DT_FLASH_AREA_<PARTNAME>_ID	0

Signed-off-by: Findlay Feng <i@fengch.me>
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-02-07 10:31:27 -06:00
Marti Bolivar
d1780aac42 scripts: west_commands: add sign command
This command is useful for signing binaries for loading by a
bootloader. At present, only MCUboot's "imgtool" is supported, but it
would be straightforward to add support for additional tools.

Using this command instead of "plain" imgtool avoids looking up any
numbers for the flash write block size, text section offset, or slot
size to get a signed binary. All users need to specify is the location
of the signing key.

This greatly improves usability for those unfamiliar with MCUboot, or
even experienced users who have to deal with multiple flash partition
layouts, boards, etc.

The command works by inspecting state in the Zephyr build system, some
of which is also provided by the runner package.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2019-02-07 08:44:19 -05:00
Marti Bolivar
8c095285d1 scripts: west_commands: refactor run_common.py
Pull out some more common functionality we will need elsewhere into a
separate file.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2019-02-07 08:44:19 -05:00
Marti Bolivar
8c4479994c scripts: west_commands: refactor build.py
This is a prep work patch for adding another command.  Refactor
build.py to use a new Forceable superclass and find_build_dir() helper
routine. Fix a help string while we are here.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2019-02-07 08:44:19 -05:00
Himanshu Jha
b2fa9db074 scripts: coccicheck: Add support for SPFLAGS
SPFLAGS allows to pass additional flags supported by
spatch during transformation.

Both short notation `-f=` and `--sp-flag=` can be used
to pass the flag to the coccicheck.

Signed-off-by: Himanshu Jha <himanshujha199640@gmail.com>
2019-02-07 08:12:20 -05:00
Ulf Magnusson
7df60372c2 scripts/dts: Rename convert_string_to_label() to str_to_label()
Less spammy and as easy to understand ("label" seems a bit confusing
though, but it's consistent with other code).

Also simplify some surrounding code, and clean up the implementation of
str_to_label() a bit.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-07 10:39:37 +01:00
Ulf Magnusson
9c1274b6e8 scripts/dts: Simplify code around insert_defs() calls
Pass the dictionary of definitions directly. Shorter and clearer.

Also get rid of some other temporary variables, a redundant str() call,
and some mysterious commented-out code in _extract_flash().

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-07 10:39:37 +01:00
Ulf Magnusson
e8a077b6a3 scripts/dts: Remove redundant parentheses around 'if' conditions
Not needed in Python.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-07 10:39:37 +01:00
Ulf Magnusson
b278500d94 scripts/dts: Remove dead and confusing pwm/gpio-related code
- Maybe the code is trying to turn integers into single-element list,
   but it won't work (needs to be [value] rather than list(value)).
   Don't think you could ever end up with an integer either.

 - extract_controller() assumes that prop_values is a list, so
   reduced[...].get(prop) should be reduced[...][prop]. get() means it's
   okay for the key to be missing, which makes the code confusing.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-07 10:39:37 +01:00
Ulf Magnusson
d2985b31d8 scripts/dts: Remove dead and confusing code in get_phandles()
- The 'props' key is known to exist (and 'if enabled == ...:' would
    crash if it ever was missing)

  - 'd' is always a dictionary

  - There's no need to test if d['children'] is non-empty before looping
    through it

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-07 10:39:37 +01:00
Ulf Magnusson
23afd94704 scripts/dts: Remove dead and confusing code in get_all_compatibles()
- The 'props' key is known to exist (and 'if enabled == ...:' would
   crash if it ever was missing)

 - 'd' is always a dictionary

 - There's no need to test if d['children'] is non-empty before looping
   through it

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-07 10:39:37 +01:00
Ulf Magnusson
d26cb1fb55 scripts/dts: Remove redundant and confusing dict.get() calls
dict.get(key) signals to people reading the code that you're not sure
whether the key exists. It returns None if it doesn't.

When the key is known to exist, dict[key] should be used. This also
helps catch bugs by raising an exception if the key is missing.

Similarly, whether a key in a dict should be tested with

    if key in dict:

instead of with

    if dict.get(key):

The second version signals that you both want to make sure that the
exists and that it's truthy (e.g., non-empty). That's confusing if a
simple existence check was meant.

There seems to be a bug in output_keyvalue_lines() where

    fd.write("%s=%s\n" % (entry, defs[node].get(a)))

can end up writing '...=None'. Removing the .get() makes it throw an
exception instead. Keep the .get() for now and don't attempt to fix the
bug.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-07 10:39:37 +01:00
Ulf Magnusson
01db6469ca scripts/dts: Remove redundant dict.keys() calls
'x in dict' and 'for x in dict' in Python always uses the keys.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-02-07 10:39:37 +01:00
Andrew Boie
f753ae92fb gen_mmu_x86: remove unused function
Fixes #13096

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-02-06 14:34:20 -08:00
Andrew Boie
2d9bbdf5f3 x86: remove support for non-PAE page tables
PAE tables introduce the NX bit which is very desirable
from a security perspetive, back in 1995.

PAE tables are larger, but we are not targeting x86 memory
protection for RAM constrained devices.

Remove the old style 32-bit tables to make the x86 port
easier to maintain.

Renamed some verbosely named data structures, and fixed
incorrect number of entries for the page directory
pointer table.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-02-05 20:51:21 -08:00
Timon Baetz
4cc355d423 boards: hifive1: Added flash runner
HiFive1 flash runner using OpenOCD and GDB

Signed-off-by: Timon Baetz <timon.baetz@gmail.com>
2019-02-05 14:29:16 -06:00
Anas Nashif
c96c90acf7 sanitycheck: return build.log when we think we crashed
In some cases sanitycheck handles a build error that started with 'make
run' as a handler crash when it actually failed during building. Right
now we try to attach the handler.log to the XML output even if it did
not exist. Check for the file and if it was not found, go back to the
build.log

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-02-05 13:40:49 -05:00
Alberto Escolar Piedras
834b86f8d7 sanitycheck: coverage: Tolerate missing source files
When calling genhtml to generate a coverage report directly from
sanitycheck (-C option), call it with the "--ignore-errors source"
option, so it does not exit when it meets a missing file, but
instead it warns the user and continues.

Fixes #13014

Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
2019-02-03 15:28:53 -05:00
Maureen Helm
987313e102 scripts: west: Update pyocd runner to use unified tool subcommands
pyocd 0.14.0 merged its command-line tools into a unified pyocd tool
with subcommands. The separate command-line tools still remain, but are
deprecated. Update the pyocd runner to use the new unified pyocd tool
with subcommands.

Signed-off-by: Maureen Helm <maureen.helm@nxp.com>
2019-02-02 10:32:30 -05:00
Anas Nashif
12f737163f ci: run_ci.sh: make script run locally
Allow running script locally. For example, on a local tree with local
changes:

./scripts/ci/run_ci.sh -b master -r upstream  -l

This will be have the same way as in CI and creates a test manifest
based on changes.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-02-01 19:04:55 -05:00
Anas Nashif
8a51a4bf26 ci: do not abort if west is already initialised
We should not abort if west was initiaized, someone must have
initialized, i.e. locally and everything is already setup for us and
ready to go.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-02-01 19:04:55 -05:00
Kumar Gala
ce7ed18989 dts: Move CONFIG_SRAM_* & CONFIG_FLASH_* back to Kconfig
dts will now generate DT_SRAM_BASE_ADDRESS, DT_SRAM_SIZE,
DT_FLASH_BASE_ADDRESS, and DT_FLASH_SIZE defines.  Kconfig can utilize
these defines to set defaults for the CONFIG_ variants.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-02-01 10:16:06 -06:00
Carles Cufi
fa26ef02d2 doc: Add KCONFIG_DOC_MODE env var handling
Export a new KCONFIG_DOC_MODE environment variable when building the doc
and invoking Kconfig, so that the functions that expect a build folder
can accordingly return a hardcoded value.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-02-01 10:16:06 -06:00
Kumar Gala
87915cd36f dts: stop generating CONFIG_FLASH_LOAD_{OFFSET,SIZE}
As a step to completing removing use of Kconfig defines in dts files we
need to change how we configure/set CONFIG_FLASH_LOAD_{OFFSET,SIZE}.

Previously we would set them based on how the chosen property
'zephyr,code-partition' was set to.  If 'zephyr,code-partition' wasn't
set we would default the values to 0.  We had some generic overlay logic
which would define 'zephyr,code-partition' based on
CONFIG_BOOTLOADER_MCUBOOT.

Going forward if the DTS has 'zephyr,code-partition' set we will
generate DT_CODE_PARTITION_OFFSET & DT_CODE_PARTITION_SIZE defines.  We
will leave it to Kconfig to set CONFIG_FLASH_LOAD_OFFSET &
CONFIG_FLASH_LOAD_SIZE.

We introduce a python script that allows Kconfig to extract data from
the DTS and thus we can utilize DT_CODE_PARTITION_OFFSET and
DT_CODE_PARTITION_SIZE values to set defaults for
CONFIG_FLASH_LOAD_OFFSET & CONFIG_FLASH_LOAD_SIZE.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-02-01 10:16:06 -06:00
Andrew Boie
c253a686bf app_shmem: auto-initialize partitions
There are no longer per-partition initialization functions.
Instead, we iterate over all of them at boot to set up the
derived k_mem_partitions properly.

Some ARC-specific hacks that should never have been applied
have been removed from the userspace test.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-01-30 23:15:51 -05:00
Carles Cufi
109f80f142 west: runners: nrfjprog: Enable testing of nRF91
Enable the nRF91 suite of tests.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-01-30 20:16:27 +01:00
Carles Cufi
13ce9d94b9 west: runners: nrfjprog: Correct nRF91 unit test reference data
nRF91 only does `--sectorerase` (no UICR) and does not need to
enable pin reset (`--pinresetenable`).

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-01-30 20:16:27 +01:00
Ioannis Glaropoulos
a97dbec8c1 scripts: west: explicitly ask for --sectorerase for nRF9160_pca10090
When flashing nRF9160_pca10090 board with nrfjprog, explicitly
request for --sectorerase, instead of --sectoranduicrerase (used
when flashing boards with nRF52 SOC).

Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
2019-01-30 20:16:27 +01:00
Kumar Gala
7d52cb7d4a scripts/dts/extract: Rework to extract_{controller,cells}
Rework extract_{controller,cells} so they aren't recursive.  As part of
this change we take the flat cell array property and build a proper list
based on what the number of cells each element would represent.

Fixes #12724

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-01-30 09:41:10 -06:00
Kumar Gala
d5b58e1655 scripts/dts/extract: Move flattening of list to one place
Because of differents in dtc versions the output of a list property may
vary.  We would flatten the list of lists to a single list to match how
older dtc versions did things.  Rather than doing that flattening in
multiple places, move it to get_reduced() so we do it just once.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-01-30 09:41:10 -06:00
Anas Nashif
f29087ee1b sanitycheck: Handle data over UART from gcov
Make sure we capture data from gcov and do not timeout before all the
data has been captured. Also report on incomplete data capture.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-01-29 15:03:38 -05:00
Anas Nashif
a34f371e36 scripts: gen_gcov_files: report if data capture is not complete
If we get incomplete data, report on the screen.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-01-29 15:03:38 -05:00
Anas Nashif
47cf4bf544 sanitycheck: add option to point to gcov-tool
When building with older versions of the toolchain we need to use gcov
from the same tool set, otherwise data formar might not be compatible.

Added an option to point to the gcov tool from the same toolchain used
to build the code. By default the host gcov is used.

Relates to #12571

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-01-29 15:03:38 -05:00
Anas Nashif
a4b5a739d9 sanitycheck: link with --coverage only for native_posix
Link with --coverage only when building with the host toolchain.

Relates to #12571

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-01-29 15:03:38 -05:00
Sebastian Bøe
6d8d4442b0 mergehex: Improve the error feedback when merged hex files overlap
When merging hex files, and there is an overlap between the hex files,
display the hex file that failed to merge.

Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
2019-01-29 17:04:42 +01:00
Marti Bolivar
808028b46f scripts: west commands: add text for "west --help"
Although each of these command implementations has good output for
"west <command> --help", the "west --help" output is missing the
one-line descriptions.

Add them by modifying west-commands.yml to use a new 'help' key in
each command name. These need to be kept in sync with the Python
sources. For now just do that by copy/pasting. We could add fancy
logic to load the help from west-commands.yml later if we want.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2019-01-29 10:15:01 +01:00
Marti Bolivar
ecae1e9b17 west_commands: add information on running tests
Add information to the README on how to run the tests, and note that
they are part of Zephyr's CI.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2019-01-29 10:15:01 +01:00
Carles Cufi
89ee604836 scripts: west: Add support for nRF91 in nrfjprog
Extend the list with the new nRF91 SoC.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
2019-01-29 10:15:01 +01:00
Carles Cufi
6b00bc9357 west: Multiple CI fixes for the topic-west branch
In order to be able to merge the topic branch, we require a few
fixes to CI.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-01-29 10:15:01 +01:00
Marti Bolivar
63841df184 scripts: run_ci.sh: add pytest-based testing for west commands
Incorporate these into run_ci.sh the same way that btsim results were
done. This adds a dependency on pytest.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2019-01-29 10:15:01 +01:00
Marti Bolivar
9cecb5df4a scripts: run_ci.sh: remove dead code
Commit 88ece494 ("ci: do not build docs in main CI job") removed the
call to build_docs but not the function itself. That makes it
confusing to try to find where it's called; remove it.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2019-01-29 10:15:01 +01:00
Marti Bolivar
ab82264ace scripts: add west build, flash, and debug commands
West now supports a mechanism for extension commands. Use it to move
the command implementations that are tightly coupled with boards and
the zephyr build system back into the Zephyr repository.

This patch doesn't include test cases. Those will be moved over in a
subsequent patch.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2019-01-29 10:15:01 +01:00
Carles Cufi
6aa87b6f5b ci: Update to the new west model
The new west is able to initialize itself from an existing zephyr clone
without modifying it, adapt the CI script accordingly.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-01-29 10:15:01 +01:00
Anas Nashif
ca51ac916c ci: Use west to clone zephyr
West right now modifies the zephyr tree and points to master. In CI we
need to point to the pull-request HEAD instead, so so whatever shippable
does after cloning with west.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-01-29 10:15:01 +01:00
Carles Cufi
9b4eb37f38 scripts: remove west from scripts/
west will now be installed via pip in order for the bootstrapper to be
decoupled from the west runners.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-01-29 10:15:01 +01:00
Sebastian Bøe
361daf8ebd DeviceTree: Fix generated header comment
The header comment in 'generated_dts_board_unfixed.h' was including
'compatible' as a helpful text.

But something has been broken with how we extract 'compatible' because
it was being evaluated to 'q'. Giving a confusing header text:

Generated include file q

Since the mechanism is broken, and does not appear to be important, we
remove it.

Also, modernize how we generate multi-line strings.

Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
2019-01-25 03:02:33 -06:00
Sebastian Bøe
c23cc26c73 DeviceTree: Convert 'generated_dts_board.h' into a source file
extract_dts_includes.py has been generating DT output and then
concatenating it with fixup header files to create
'generated_dts_board.h'.

In this patch we instead introduce a source file named
'generated_dts_board.h' and have it \#include the appropriate DT
output and fixup files.

This results in a simpler system because users can now read
'generated_dts_board.h' as C source code to see how fixup files and
generated DT output relate to each other. Whereas before they would
have to either read documentation or python code to gain the same
understanding.

Also, it reduces the scope and complexity of one of our most bloated
python scripts, extract_dts_includes.py.

Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
2019-01-25 03:02:33 -06:00
Kumar Gala
f592c528f4 scripts/dts/extract: Add support for generating SPI GPIO CS defines
Add support to generation script to generate defines for GPIO CS
[WIP], need to clarify some issues on how CS's look when we mix GPIO-CS
and HW CS.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-01-23 18:12:24 -06:00
Kumar Gala
4bd4185a2c scripts/dts/extract: Add single extract to extract_{controller,cells}
Add the ability to extract just a single instance from extract_cells and
extract_controller.  This lets us pass in a prop_values like:

cs-gpios =  < 0x02 0x11 0x00 >, < 0x03 0x05 0x00 >;

and extract just index = 1 if we want < 0x03 0x05 0x00 >.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-01-23 18:12:24 -06:00
Kumar Gala
44fd0c5624 scripts/dts/extract: Move extract_{controller,cells} into globals
Move extract_cells and extract_controller functions into globals so they
can be utilized outside of just the main extract_dts_includes script.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-01-23 18:12:24 -06:00
Anas Nashif
939f15195d requirements: update to pyocd 0.15.0
Previous release had some issues with unicode and was failing to install
in our CI. Those issues were fixed in this release.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-01-23 15:42:40 -05:00
Kumar Gala
9948eb036c scripts/dts/extract: Don't warn for 'category' merge of 'required'
We get warnings like:
	foo.yaml('BAH') merge of property 'category': 'required'
	overwrites 'optional'

This warning isn't meaningful as its reasonable to change the 'category'
of a property from 'optional' to 'required'.  So don't warn in the case
anymore.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-01-23 06:17:06 -06:00
Sebastian Bøe
56d7471b0c sanitycheck: Error out when running sanitycheck from Windows
Running sanitycheck from Windows is not supported yet. Unfortunately,
the error message is obscure when a user is unaware of this and runs
it from Windows anyway.

This patch gives an easy to understand error message explaining this
issue.

Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
2019-01-22 07:44:06 -05:00
Anas Nashif
eecd90f1d8 ci: remove check-compliance script
This is now maintained outside of the tree in ci-scripts repo.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-01-18 09:49:56 -05:00
Kumar Gala
32cbc42ce6 scripts/dts/extract: fix handling of non-MMIO SPI flash
The Intel S1000 board has a SPI flash that isn't directly MMIO
addressable.  For this case we shouldn't generate CONFIG_FLASH_SIZE or
CONFIG_FLASH_BASE_ADDRESS.

We use the heuristic of assuming if the flash node #size-cells is 0, its
a SPI flash.  Than if the parent node (SPI controller) has only a single
reg element we assume its a non-MMIO addressable SPI flash.  If there is
more than one reg element we assume the last reg element is the MMIO
address region for access to the flash.

Fixes #12530

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-01-17 21:29:07 -05:00
Anas Nashif
74dbe338d4 sanitycheck: do not keep extending timeout
While trying to catch crashes after test completes, do not keep
extending the timeout which might lead to and endless loop and not
getting out based on set timeout.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-01-17 19:46:45 -05:00
Peter A. Bigot
bfad9721d2 kernel: remove k_alert API
This API was used in only one place in non-test code.  See whether we
can remove it.

Closes #12232

Signed-off-by: Peter A. Bigot <pab@pabigot.com>
2019-01-16 21:34:07 -05:00
Anas Nashif
dbd7649fc0 sanitycheck: add option to specify platforms with coverage support
We want to run coverage on select platforms even if a few do support
that now. Specify which platforms should be enabled using the new option

--coverage-platform

By default we have the posix platform configured. Any additional
platforms can be specified using the option above.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-01-16 06:12:33 -05:00
Anas Nashif
db9592aa3e sanitycheck: convert gcov data to gcda
Collect data generated by built-in gcov over UART.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-01-16 06:12:33 -05:00
Adithya Baglody
131edad483 scripts: gen_gcov_files.py: Added python script to parse uart dump
This script will parse the serial console log file and create the
required gcda files.

Usage:
python3 ${ZEPHYR_BASE}/scripts/gen_gcov_files.py -i console_output.log
Add -v for verbose.

Signed-off-by: Adithya Baglody <adithya.nagaraj.baglody@intel.com>
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-01-16 06:12:33 -05:00
Kumar Gala
8f52956905 scripts: extract_dts_includes: Change arg parsing requirements
Changed arg parsing so you can invoke the script and only generate the
include file or conf file as desired.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-01-15 08:10:31 -06:00
Jan Kowalewski
265895b293 sanitycheck: Add Renode support
The change adds an optional capability to kill process by PID in the
BinaryHandler. At the moment proc.terminate only closed the make
process, also closing the stdin/stdout, but this is impossible to detect
in .NET.

Signed-off-by: Dawid Wojciechowski <dwojciechowski@internships.antmicro.com>
Signed-off-by: Jan Kowalewski <jkowalewski@antmicro.com>
Signed-off-by: Piotr Zierhoffer <pzierhoffer@antmicro.com>
2019-01-14 09:12:07 -05:00
Kumar Gala
35ecc78cdf scripts/dts/extract: Generate unique defines based on compat & instance
Add support to generate defines of the form
DT_<COMPAT>_<INSTANCE>_<PROP>.  The idea is that we can utilize this in
drivers to remove the need for dts_fixup.h.  The <INSTANCE> value is
determined by the script and starts at 0 and counts up for each instance
of a given <COMPAT>.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-01-14 06:33:29 -06:00
Kumar Gala
3b9147df3f scripts/dts/extract: Fliter duplicates values inserted into defs
If an aliases already exists in the defs we should filter it out.  Also
if an alias exists in the new_def we should filter the alias out.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-01-14 06:33:29 -06:00
Kumar Gala
94240aebb2 scripts/dts/extract: Add an instance ID for each compatiable
Add a unique instance starting at 0 for each compatiable in the dts.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-01-14 06:33:29 -06:00
Sebastian Bøe
b72fb4e9e8 cmake: Remove dead code 'FindRegex.cmake'
When Zephyr was building the C implementation of KConfig it used the
build script 'FindRegex.cmake' to satisfy a regex dependency.

Now that we no longer use the C implementation of Kconfig we no longer
use this build script, and it is safe to remove it.

Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
2019-01-14 13:02:03 +01:00
David Leach
afcfa11ce0 scripts: extract_dts_includes: Enhance message information
The information message when processing the yaml files is
cryptic when there is an override. This change adds filenmae
of yaml file being processed and the parent tuple with
the override.

Signed-off-by: David Leach <david.leach@nxp.com>
2019-01-12 15:23:11 -06:00
Maureen Helm
73956a398e scripts: Update pyocd version to 0.14.3
pyocd recently merged its command-line tools into a unified pyocd tool
with subcommands. The separate command-line tools still remain, but are
deprecated. Update the pyocd version required by zephyr in preparation
for updating the pyocd runner to use the new unified tool commands.

Pin pyocd to a specific version rather than allowing any version greater
than or equal to 0.14.0 because there have been a few unstable releases
recently. Version 0.14.3 is what I have been using locally and know to
be stable.

Signed-off-by: Maureen Helm <maureen.helm@nxp.com>
2019-01-11 21:33:28 -06:00
Kumar Gala
e57a596a0c scripts/dts/extract: Support QSPI memory mapped flash
We need to refactor how we determine CONFIG_FLASH_{SIZE,BASE_ADDRESS}
for when we have a QSPI that is memory mapped. "zephyr,flash" is going
to point the actual flash component that will be on a SPI bus and thus
the device node's reg property tells us which CS on the SPI bus its on,
not the memory mapped address/size of the flash.

So we make a few assumptions to handle this case:
1. If the #size-cells for the flash node is 0, we assume its a QSPI
   memory mapped flash
2. That the QSPI memory mapped node (parent of what "zephyr,flash" is
   point to, will contain a reg propery where the second reg pair will
   be the memory mapped region for the flash.

We move handling of CONFIG_FLASH_{SIZE,BASE_ADDRESS} into flash.extract
so its all in one place.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-01-11 13:12:18 -06:00
Marti Bolivar
0b1b4e2d23 scripts: update west to upstream commit f01059a
The main reason to copy into upstream is to bring in a new runner.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2019-01-11 10:33:24 +01:00
Andy Ross
dc4151f8f3 scripts/sanitycheck: Don't build "slow" tests either
Extend the "slow" flag on tests to inhibit building too.  The original
assumption was that building would be fast but running slow, but now
We have tests using a component (OpenThread) that wants to pull and
build software from github for every app.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2019-01-04 14:59:28 -05:00
Andy Ross
9c9162dfd9 sanitycheck: Add --disable-size-report option
The segment size computation has value, but it's actually a very slow
process that parallizes poorly.  It seems to be bound by the Python
GIL doing the parsing, so never sees more than about 150% of CPU in
use even on wildly parallel systems.  And it takes about 75-80
seconds, which is 15-20% of the entire runtime of the test on that
box!

And the only "failure" case this can detect (unexpected sections in
the output file) is now a duplicate of the orphan section warning
we've since enalbled at the linker level.

This defaults to enabling the test to preserve behavior (as I don't
know where all the existing users of the size report are to change
them), but long term we might consider making "disabled" the default
and switching this to an --enable flag.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2019-01-03 16:06:28 -05:00
Ulf Magnusson
cf3cddd1cd menuconfig: Prompt for save when Kconfig files change
Update menuconfig and Kconfiglib to upstream revision 16539b4f223fa, to
add the commit below.

    menuconfig: Prompt for save if a different .config would be saved

    Previously, menuconfig.py only prompted for saving the configuration
    if .config didn't exist or the user changed symbol values within the
    interface.

    Also make it prompt for save if Kconfig symbols have been added,
    removed, or have had their defaults changed, provided it would make
    the saved .config differ from the loaded one.

    This usually won't matter for correctness, because loading an
    outdated configuration performs an implicit olddefconfig, but it's
    less confusing.

    Also add a Kconfig.missing_syms attribute that records all
    assignments to undefined symbols in the most recently loaded .config
    file. This is needed to implement the check for whether the saved
    .config would be different.

    As an unrelated change, always prompt for saving if a .config has
    been loaded from within the menuconfig interface. The intention is
    probably often to save the configuration somewhere else, even if it
    isn't modified.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-01-03 13:55:31 +01:00
Adithya Baglody
65f79cdc21 scripts: gen_relocate_app: Use linker provided size variable.
This seems to solve the issue with GH-12033. There seems to be
some compiler optimization that was causing this issue. This
occurs only when the build is re-run by invoking cmake without
clearing the previous build.

Fixes:GH-12033

Signed-off-by: Adithya Baglody <adithya.nagaraj.baglody@intel.com>
2018-12-29 09:23:34 -05:00
Alberto Escolar Piedras
3980e0cf64 sanitycheck: Use supression file for valgrind and dump log
When running valgrind in sanitycheck, use the suppression file
and dump the log in its own file.

Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
2018-12-21 21:07:18 +01:00
Alberto Escolar Piedras
661bc8206f sanitycheck: Fail test if the process returns != 0
If the test process returns an error (return code != 0),
it should be considered a failure, even if the handler
considers the test passed.
But this is not done when we are killing (SIGTERM) the processes,
as then the return code is not necessarily sensible.

Without this, for example, when running with valgrind, all valgrind
errors are missed.

Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
2018-12-21 21:07:18 +01:00
Alberto Escolar Piedras
ace7bf9508 valgrind: Add error supression file for POSIX arch
Added a valgrind error supression file which can be used when
running in the POSIX arch.

So far we are supressing 2 types of errors:
* One very normal in glibc, which will be triggered in the nrf52_bsim
* 2 due to the posix arch prioritizing terminating fast over terminating
cleanly. The posix arch, when posix_core_clean_up() is called, will
pthread_cancel all Zephyr threads. But it will NOT wait for them to
finnish, in case any of them was hung or there is any other problem
which would prevent it. Therefore, depending on the execution, some
times the threads will be still lingering when the main thread exits
Somethign similar applies to posix_soc_clean_up().
These 2 are not errors, but conscious choices => Therefore we
supress them

Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
2018-12-21 21:07:18 +01:00
Marti Bolivar
15384b79c3 scripts: west: fix 'west debug'
We can't invoke west's main as a subprocess, or control-C from GDB in
a 'west debug' will terminate the program. Import it and run main as a
function in the same process instead.

Clean up some cruft and remove unused code while we're here, to keep
this file shorter.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2018-12-19 17:34:14 +01:00
Flavio Ceolin
8f3459b3db coccinelle: Updating ignore_return to support memcpy
The return of memcpy was being ignored just like memset's return. Just
adding it to coccinelle script.

MISRA-C rule 17.7

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2018-12-19 14:37:25 +01:00
Ulf Magnusson
1581155828 menuconfig: Add show-help mode
Update menuconfig (and Kconfiglib, just to sync) to upstream revision
e629c33d31e22, to add the commit below.

  menuconfig: Add show-help mode

  Pressing F toggles show-help mode, where the help window at the bottom
  displays the help text of the currently selected item, if any. Can be
  handy when browsing through symbols.

  Also mention the different modes that are available in the module
  docstring.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-12-18 13:28:37 +01:00
Kumar Gala
2cc957e860 scripts: extract_dts_includes: Generate defines for chosen props
Generate a set of defines that convey if a given chosen property is used
or not.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2018-12-17 11:26:40 +01:00
Carles Cufi
d9b8b02519 scripts: ci: check-compliance: Fix running out-of-tree
In order for check-compliance.py to be able to run cleanly out-of-tree,
set the repo (to be checked) path based on cwd, and refer to Zephyr and
its tools by the specific path set by ZEPHYR_BASE.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2018-12-14 13:11:24 +01:00
Kumar Gala
252919128b scripts: extract_dts_includes: Rework get_binding
The original version of get_binding didn't handle the case of having a
child node on a SPI or I2C device node.  The example is a SPI flash and
having a partition node under that.

Re-work the binding lookup logic to first look in the master bindings
dict, and if not found there we assume it must be in the bus specific
dictionary, and we can use the parent node to find the bus type.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2018-12-13 08:12:53 -06:00
Anas Nashif
1c65b6b3a4 sanitycheck: filter by device tree configs
In addition to Kconfig values from generated .config, also parse and
filter based on configs generated by device-tree.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-12-11 13:13:26 -06:00
Kumar Gala
17f0977fc9 scripts: extract_dts_includes: Fix handing of no binding
There are cases in which we get the parents compat and thus we might
have a binding available for the actual node.  An example is having a
SPI or I2C controller with a child node, but that node doesn't have a
binding associated with it.  We'll end up getting the parent compat, so
return the parent binding in this case.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2018-12-11 09:23:21 -06:00
Ulf Magnusson
57b28cae2c kconfiglib: Save previous configuration to .config.old
Update Kconfiglib (and menuconfig, just to sync) to upstream revision
094f4a9622046, to add the commit below.

  Save existing configuration to .<filename>.old in write_config()

  Add a default-True 'save_old' flag to write_config(). If 'save_old' is
  True and an existing configuration file is being overwritten, a copy
  of the old configuration file is saved to .<filename>.old (e.g.
  .config.old) in the same directory.

  Errors are ignored, as the old configuration would usually just be a
  nice-to-have, and not essential.

  The same functionality could be added for minimal configuration files
  and headers, but it's probably most useful for configuration files.

Other changes:

  - Parsing performance is improved a bit

  - scripts/kconfig/kconfig.py now prints the path to the merged
    configuration in zephyr/.config, to make it a bit easier to
    discover.

Fixes: #2907

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-12-08 17:14:38 +01:00
Andrew Boie
cd255744d0 userspace: eliminate warning in process_gperf.py
Python 3.7 throws a FutureWarning when it thinks it
has found a nested set in a regular expression.

We weren't actually; just trying to match some brackets.
Escape them instead. No functional difference.

Fixes: #11961

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2018-12-07 20:26:46 -05:00
Anas Nashif
724d6f7909 sanitycheck: add new _TEXT_SECTION_NAME_2 section to whitelist
Add _TEXT_SECTION_NAME_2 to the whitelisted list of linker sections.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-12-07 16:50:24 -05:00
Alberto Escolar Piedras
d8c25d37d9 ci: Allow setting BSIM_OUT_PATH from environment
If BSIM_OUT_PATH was set in the environment use that value

Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
2018-12-07 16:46:54 -05:00
Alberto Escolar Piedras
69772ef7d8 ci: Fail shippable if something fails catastrophically
Before run_ci.sh was introduced, if some command in the
.shippable.yml script failed, Shippable would stop and report
a failure.
When run_ci.sh was introduced we lost this, which is pretty
dangerous if something fails cathastrophically and the xml
reports are not (properly) generated and copied, as then
Shippable reports a pass.
=> Run run_ci.sh in -e mode, so we propagate a bad failure upwards

Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
2018-12-07 16:46:54 -05:00
Alberto Escolar Piedras
539e73f526 ci: Copy results before handling coverage
Coverage parsing has high chances of failing, let's copy
first the results

Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
2018-12-07 16:46:54 -05:00
Adithya Baglody
ff63101c7d scripts: gen_relocate_app.py: Script for relocating files in memory.
This script will relocate .text .data and .bss sections from
required files and places it in the required memory region. This
memory region and file are given to this python script in the form
of a string. Refer to the script for the format of this string and
the procedure to invoke it.

The main goal of this script is to provide a robust way to re-order
the memory contents without actually having to modify the code
(C source code and the linker code).
In simple terms this script will do the job of
__attribute__((section("name"))) for a bunch of files together.

Signed-off-by: Varun Sharma <varun.sharma@intel.com>
Signed-off-by: Adithya Baglody <adithya.nagaraj.baglody@intel.com>
2018-12-07 10:32:41 -05:00
Kumar Gala
471d2f3a0a scripts: extract_dts_includes: Generate device bus info define
For devices on buses like I2C or SPI its possible that a sensor might
support being on either bus type.  In the dts we can tell this by
looking at the parent bus of the device.  Its useful for a driver to
know this so can we build support into the driver accordingly.

For example if the LSM6DSL sensor ("st,lsm6dsl") is in the dts on a spi
bus we now generate:

	#define DT_ST_LPS22HB_PRESS_BUS_SPI	1

Its possible that a system exists in which multiple of the same sensor
exist but on different busses, so drivers should handle that case
accordingly.  For the LSM6DSL example we'd end up with:

	#define DT_ST_LPS22HB_PRESS_BUS_I2C	1
	#define DT_ST_LPS22HB_PRESS_BUS_SPI	1

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2018-12-07 09:08:04 -06:00
Kumar Gala
2442382fac scripts: extract_dts_includes: Support same sensor on different bus
There are a number of cases in which we a sensor can be either connected
on I2C or SPI.  We've been treating these cases as different compatiable
properties, but they really should use the same compatiable and just
determine the info based on the parent bus in the device tree.  We can
now support having two different binding files for the same compatiable
to handle the case of a sensor supporting either I2C or SPI as how its
connected.

We put "sensor" nodes in a bus specific dict bus_bindings[bus] that
we can than lookup later based on the DTS and determining the bus type
that the "sensor" node is on.

Fixes #11375

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2018-12-07 09:08:04 -06:00
Kumar Gala
2893be57ef scripts: extract_dts_includes: Split out yaml_list info
Have the yaml_list now have 'node', 'bus', and 'compat' information
instead of just the 'node' info.  For now we have all YAML nodes under
yaml_list['node'], however this will change to support sensors that
can be on I2C or SPI, since they will have the same compat, but
different YAML nodes.

This also introduces yaml_list['bus'] which has YAML nodes for any
compatiables on a given 'parent' bus (ie, 'i2c', or 'spi').

Finally, we also have yaml_list['compat'] with all the compatiables
that we parsed and matched to a YAML.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2018-12-07 09:08:04 -06:00
Kumar Gala
f34a99532a scripts: extract_dts_includes: Pass node address to get_binding
Change get_binding to take the node address instead of the
compatiable.  This is in prep for having get_binding be able to look
at the parent of the node and determine which bus specific binding to
utilize (for cases like sensors that support both I2C & SPI).

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2018-12-07 09:08:04 -06:00
Kumar Gala
112b0f5eff scripts: extract_dts_includes: Add accessor function to access binding
Add accessor functions get_binding and get_binding_compats to
encapsulate access to binding database info.  Cleanup all direct access
to binding info and replace with calls to get_binding() and
get_binding_compats()

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2018-12-07 09:08:04 -06:00
Kumar Gala
35afcc9c12 scripts: extract_dts_includes: remove passing yaml param to functions
Various functions like add_prop_aliases, extract_single_prop,
extract_single, reg.extract, compatible.extract, default.extract, and
flash.extract don't use the yaml argument that is passed to them.
Remove passing the argument, this is the first step in making access to
the yaml node information go through an accessor function.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2018-12-07 09:08:04 -06:00
Kumar Gala
b7f5101e79 scripts: extract_dts_includes: Handle YAML inheritance when we load
Move the handling of 'inherits' in the YAML to right when we load.  This
allows us to remove yaml_collapse function and will allow us to look
into the yaml right after we load it for things like if the yaml is for
a child bus.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2018-12-07 09:08:04 -06:00
Jakub Rzeszutko
f8178dcb05 shell: remove Console dependencies
Removed Console dependencies from shell uart backend.
Generated define: CONFIG_UART_SHELL_ON_DEV_NAME for each board.

Fixes #10191

Signed-off-by: Jakub Rzeszutko <jakub.rzeszutko@nordicsemi.no>
2018-12-07 12:11:11 +01:00
Anas Nashif
c562e4247d ci: remove bsim result options
This option is not being used anymore.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-12-06 14:24:41 -05:00
Anas Nashif
c3e98056ea ci: set second pass options correctly
Use correct 3rd pass env variable.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-12-06 14:24:41 -05:00
Himanshu Jha
200e847ace scripts: coccicheck: Improve continuous run operation
The current implementation of continuous run operation using
command `./scripts/coccicheck` i.e., without specifying any options,
`coccicheck` default runs in `report` mode with all available
coccinelle scripts present at `scripts/coccinelle/`.

Not all scripts have report mode implemented in them, which
leads to failure of coccicheck.

With this new implementation we choose whatever available mode
is present in coccinelle script and pass it to MODE variable
without stopping continuous coverage.

And perhaps if there are plans to add `coccicheck` as a sanity
checker in future to the Zephyr automated CI, then certainly we
want the warnings/errors produced by scripts to be less verbose
to the users.

Therefore, in this new implementation we prioritise the modes as:

1. report
2. context
3. patch
and lastly falling to
4. org

Lastly, in order to differentiate between outputs of various
coccinelle scripts being run, `x------x` separator has been used
to make reports mode readable.

Signed-off-by: Himanshu Jha <himanshujha199640@gmail.com>
2018-12-05 15:14:05 -05:00
Kumar Gala
459ad673ba scripts: extract_dts_includes: Fix DeprecationWarning
With python3.7 we get the following warning:

	extract_dts_includes.py:496: DeprecationWarning: Using or
	importing the ABCs from 'collections' instead of from
	'collections.abc' is deprecated, and in 3.8 it will stop
	working.

Fix this by using 'from collections.abc import Mapping' instead of
'import collections'.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2018-12-04 22:54:46 -05:00
Patrik Flykt
1b48cb6dc1 scripts/coccinelle: Add Coccinelle script for unsigned values
Add a Coccinelle script that adds an 'U' to values assigned to
unsigned variables, according ot MISRA-C rule 7.2.

Add a 'report' mode to the script that can be used by developer/CI
and a 'patch' mode that should do the heavy lifting.

Signed-off-by: Patrik Flykt <patrik.flykt@intel.com>
2018-12-04 22:51:56 -05:00
Håkon Øye Amundsen
81c6662d23 cmake: flash: scripts: Include externally built hex files
Allow user to add externally built hex files to the cmake property
HEX_FILES_TO_MERGE. The hex files in this list will be merged
with the hex file generated when building the zephyr application.

This allows users to leverage the application configuration
available in Kconfig and CMake to help decide what hex file
should be merged with the zephyr application.

Signed-off-by: Håkon Øye Amundsen <haakon.amundsen@nordicsemi.no>
2018-12-04 19:04:13 +01:00
Anas Nashif
3a361ecdc7 ci: do not build btsim if not installed
Do not attempt to build btsim if software is not installed.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-12-03 19:28:01 -05:00
Anas Nashif
e81c902488 ci: PULL_REQUEST is reserved, use something else
Use PULL_REQUEST_NR instead of PULL_REQUEST which is reserved by
shippable.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-12-03 18:45:26 -05:00
Andrew Boie
e48bc56b11 elf_helper: skip SYS_INIT() "devices"
SYS_INIT instantiates a device struct, but this is really
just used to run some functions at boot, it does not correspond
to a device driver belonging to a subsystem. Don't put them in
the kernel object table.

These are easy to filter since they are all named with the
_SYS_NAME macro.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2018-12-03 11:44:01 -08:00
Anas Nashif
d1bca4e056 ci: use set -x for verbosity
set -v was too verbose, so change to -x and remove extra sourcing of
zephyr-env.sh.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-12-03 13:00:02 -05:00
Anas Nashif
45cbc81160 ci: pass pull request number to CI script
For further processing, pass pull request number to the CI script.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-12-03 13:00:02 -05:00
Anas Nashif
d95ce03c1d ci: define BSIM_* variables in ci script directly
Rather than pass them as options, just set it directly in the CI script.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-12-03 13:00:02 -05:00
Anas Nashif
a0a1bffd1c ci: move pos processing to the CI script
Move post processing to script and remove duplicatated code.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-12-03 13:00:02 -05:00
Andrew Boie
c7b7363da8 userspace: add additional debug to elf_helper
Devices are identified as belonging to a particular subsystem
by looking at device->driver_api.

Print some debug information if this is NULL or points to an
unrecognized API struct.

This is normal in a lot of cases, for example any use of SYS_INIT().
However, for real devices this may be an indication of mis-
configuration.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2018-12-03 09:26:20 -08:00
Ioannis Glaropoulos
560357b5e6 scripts: add missing lincense in scripts/gen_app_partitions.py
Add missing lincense header in scripts/gen_app_partitions.py

Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
2018-11-30 15:22:58 +01:00
Ioannis Glaropoulos
92f9b24ac9 arch: arm: make _image_ram_start point to the beginning of sram
Move the definition of _image_ram_start at the beginning
of the RAMMABLE (SRAM) region, so it points to the actual
start of RAM linker sections.

Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
2018-11-30 15:22:58 +01:00
Ioannis Glaropoulos
62c5894bc1 arch: arm: properly enforce MPU-aware linker section alignment
This commit standardizes and simplifies the way we enforce
linker section alignment, to comply with minimum alignment
requirement for MPU, if we build Zephyr with MPU support:
- it enforces alignment with the minimum MPU granularity at
  the beginning and end of linker sections that require to
  be protected by MPU,
- it enforces alignment with size if required by the MPU
  architecture.

Particularly for the Application Memory section, the commit
simplifies how the proper alignment is enforced, removing
the need of calculating the alignment with a post-linker
python script. It also removes the need for an additional
section for padding.

For the Application Shared Memory section(s), the commit
enforces minimum alignment besides the requirement for
alignment with size (for the respective MPUs) and fixes
a bug where the app_data_align was erronously used in the
scipts for auto-generating the linker scripts.

Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
2018-11-30 15:22:58 +01:00
Andrew Boie
c2e01dff3f workqueues: don't put k_work in special section
There's no current need for this and it makes work items
declared with K_WORK_DEFINE() inaccessible to user mode.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2018-11-29 09:21:18 -08:00
Himanshu Jha
40b5442699 coccinelle: Add script to remove unnecessary return variable
This script helps to remove bogus intermediate local variable
used in functions to store return value and instead return
directly while saving few bits of memory.

Signed-off-by: Himanshu Jha <himanshujha199640@gmail.com>
2018-11-28 11:54:01 -08:00
Anas Nashif
8d84f9229c gitlint: do not start with subsys:
Commit messages should not start with literal "subsys:", instead, spell
out the actual subsystem name.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-11-28 09:22:01 -08:00
Ulf Magnusson
c1f4d677f2 menuconfig: Improve behavior for named choices included multiple times
Update menuconfig to upstream revision 38dff36b0f97b, to improve the
behavior for a case reported by Øyvind Rønningstad.

Upstream commit message:

    menuconfig: Only list duplicated choice symbols once

    When a Kconfig file defined a named choice and was included multiple
    times, the choice symbols were listed multiple times in the
    menuconfig as well (due to commit 17d7c1e ("menuconfig: Show all
    symbols at each menu location for multi.def. choices")).

    That's probably not what you want. Tweak it so that each symbol is
    only shown once, with the prompt that was used for it at whatever
    choice definition location is entered.

    Also change how the choice selection is displayed before the choice
    is entered, so that the prompt used for the selected symbol at that
    particular location is used. Previously, the prompt at the first
    definition location for the symbol was always used.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-11-28 16:58:28 +01:00
Henrik Brix Andersen
9e8c9ca8fb subsys: fb: add support for generating CFB font headers at build time
Add script and cmake functions for automatically generating Character
Frame Buffer (CFB) font header files from image files, TrueType, or
OpenType font files.

Signed-off-by: Henrik Brix Andersen <henrik@brixandersen.dk>
2018-11-28 11:46:00 +01:00
Ulf Magnusson
9347b0f28f kconfiglib/menuconfig: Various improvements
Update Kconfiglib and menuconfig to upstream revision e47d7eff1012e, to
add some small fixes/improvements/changes:

 - Raise errors for extra trailing tokens anywhere. They were silently
   ignored in some places:

     * end{if,menu,choice} <extra tokens>

     * {default,select,...} FOO <extra tokens>  (though e.g.
       'default FOO if' raised an error)

 - Rephrase the warning when selecting a choice symbol to make it
   clearer that select never has any effect on choice symbols.

 - Display empty menus with '----' instead of '---> (empty)'. This
   matches the C tools. It might be less confusing for symbols defined
   with 'menuconfig', which is where you most often get empty menus
   (when the symbol is n).

 - Speed up parsing performance

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-11-26 14:07:36 +01:00
Bobby Noelte
9cf8b3cd20 sanitycheck: fix false passed on localized error message
Fix false passed on localized error message in make invocation.

Fixes #8348

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-11-21 20:04:19 -05:00
Marti Bolivar
c41703a020 scripts: west: be more explicit in copy warning message
Print the exact path to the copy of west that was run, to further
avoid confusion. Explain exactly what is going away and why.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2018-11-21 09:43:45 -05:00
Alberto Escolar Piedras
2d580a8339 ci: Add missing option to control placement of BT tests results file
Added a missing option in run_ci.sh to control the placement of the
xml results file. (Before it was just propagated thru the enviroment)
+
Changed .shippable to use this option when callign it.

Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
2018-11-21 09:42:46 -05:00
Anas Nashif
53982b7e27 Revert "ci: do not build btsim if we are not going to run it"
This reverts commit 6d5ac1c089.

Looks like this was not correctly done, building btsim is needed on all
nodes.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-11-21 09:12:21 -05:00
Anas Nashif
88ece4944b ci: do not build docs in main CI job
Building docs and running various checks is now being done in another CI
job that runs in parallel and reports directly to the PR.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-11-21 09:12:21 -05:00
Anas Nashif
f114a1383d sanitycheck: outdir should not be relative to ZEPHYR_BASE
We should leave ZEPHYR_BASE alone and create the test output wherever
we are running.
This will support running sanitycheck on test roots other than zephyr's
main git repo.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-11-21 15:05:07 +01:00
Anas Nashif
f18ad9dd4b sanitycheck: support running tests out of the tree
right now we assume we are running the tests inside the Zephyr tree and
things do not work well when someone tries to use sanitycheck on tests
outside the tree with a different test root. Cleanup the name handling
and support running outside of Zephyr.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-11-21 15:05:07 +01:00
Anas Nashif
ccb57d766b ci: fix skipped testcase status posting
Skipped tests were not evaluated correctly.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-11-20 18:14:19 -05:00
Anas Nashif
4c0e3e8697 ci: overhaul check-compliance.py script
- Support sending status to github
- Support license checking
- Make every test a class
- Use junitparser instead of custom xml for junit output

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-11-20 16:36:10 -05:00
Anas Nashif
6d5ac1c089 ci: do not build btsim if we are not going to run it
We were building this on every node.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-11-20 16:36:10 -05:00
Anas Nashif
10970a60c0 sanitycheck: remove unused platforms keyword
This keyword had no effect and was being copied over to many samples
errornously.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-11-19 15:03:55 -05:00
Bobby Noelte
1bf1d6a267 scripts: extract_dts_includes: fix cell name readout from yaml
Default for a unavailable cell name was an empty list, whereas a
given name is a string. Leads to a runtime error if the name is given.
Make the default an empty string.

Names for cells are taken from the yaml binding. The key used is
"#cells" which clashes if there are different cell based properties
for a device. Readout names from the more specific "#xxx-cells" if
given and fall back to "#cells" if not given.

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
2018-11-19 08:35:10 -06:00
Ulf Magnusson
37fcec9833 kconfiglib/menuconfig/kconfig.py: Various improvements
Update Kconfiglib and menuconfig to upstream revision d3866958c7685, to
add various improvements:

 - Support HOME and END in the jump-to dialog in the menuconfig. END can
   be handy as choices, menus, and comments appear at the end.

 - Add more fine-grained warning controls for multiple assignments to a
   symbol in configuration files. Use it to simplify kconfig.py a bit.

   Clean up kconfig.py a bit in other ways too, e.g. by removing unused
   imports.

 - Improve Kconfig parsing performance slightly

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-11-19 12:18:23 +01:00
Marti Bolivar
18e6c10339 scripts: print notice that west's copy is a copy
One of the most common issues I've had to deal with as people are
starting to try west is that bash has cached the 'west' binary to the
version in zephyr/scripts/west after they ran zephyr-env.sh on Unix
operating systems.

This makes people (correctly) think they are running an older version
when they get output about missing clone commands, etc., but doesn't
give them any hints about why that happened, because most people don't
know that west is already in widespread use for flashing and
debugging.

To make fielding support for these users easier while we're
transitioning, print a notice at the end of the copy informing users
what just happened. We can make it more forceful as time goes on.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2018-11-17 00:49:12 -05:00
Andrew Boie
f253d0779d k_mem_slab: track as a kernel object
We aren't going to allow any user mode access to the
k_mem_slab APIs, but in some cases (specifically in the
case of the I2S subsystem) we need to allow user mode
to assign a memory slab to a particular driver.

This will let us verfiy (in supervisor mode) that a provided
k_mem_slab pointer is really a k_mem_slab, and know its
initialization state, and have permissions assigned to it.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2018-11-15 16:20:36 -05:00
Anas Nashif
3cbffefb26 sanitycheck: use overlays to enable coverage
Set COVERAGE option using an overlay instead of piping it through CMAKE
which breaks dependecies, i.e. if someone wants to disable that option
on the application level.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-11-15 08:31:34 -05:00
Anas Nashif
8d72bb909f sanitycheck: remove dependency of coverage on native_posix
Make it more generic and depend on Kconfig taking care of dependencies.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-11-15 08:31:34 -05:00
Anas Nashif
4df6c565d2 sanitycheck: ignore generated headers in outdir
With this, we will get results for the same file for every build (tests
and samples).

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-11-15 08:31:34 -05:00
Anas Nashif
7dd19ea74d sanitycheck: filter by test fixtures
Support specifying a test fixture attached to a board to filter in/out
tests that require this fixture to be able to run a test.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-11-14 23:42:45 -05:00
Himanshu Jha
817f4b374f coccinelle: Suppress reports/warnings for ext/
The following addition `depends on !(file in "ext")` allows
to exclude `ext/` warnings reported by coccinelle scripts.

Signed-off-by: Himanshu Jha <himanshujha199640@gmail.com>
2018-11-14 19:20:34 -05:00
Andrzej Głąbek
a7c430f36f scripts: dts: Use DT_ prefix in generated labels
Almost all labels generated by the extracting script are now prefixed
with DT_. The only exceptions are:
- stuff with 'base_label' specified in yaml bindings
- items specified by 'regs_config' and 'name_config' dictionaries
  in globals.py module
- FLASH related labels generated by flash.extract() called separately
  from generate_node_definitions(), e.g. FLASH_WRITE_BLOCK_SIZE -
  these are used directly, not through fixups, from existing code
  so I didn't want to touch them now

Labels generated for aliases are additionally prefixed with information
from the 'compatible' property, e.g. DT_GPIO_LEDS_LED0_* is generated
instead of LED0_*. To provide backward compatibility for code that uses
LEDx_* and SWx_* labels in their previous forms, a command line option
named 'old-alias-names' is added to the extraction script. This option
causes that the labels for aliases are generated in both old and new
forms. Currently this option is always enabled in dts.cmake.

Signed-off-by: Andrzej Głąbek <andrzej.glabek@nordicsemi.no>
2018-11-13 10:44:42 -06:00
Alberto Escolar Piedras
917711fd8a ci: Run bsim_bt tests and collect coverage results
Compile and run the tests avaliable in bsim_bt
and collect the coverage results into the coverage report.

Also, detect if bsim's component folder already contains the
nRF52 HW models, and if it does instead of trying to fetch
them again (which will fail) check that the right versio is
present. This should ease testing locally.

Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
2018-11-13 11:14:19 -05:00
Sebastian Bøe
03ed0950e5 sanitycheck: Allow skipping the 'unrecognized section' test
Allow the user to disable the "unrecognized section" test. I can see
multiple use-cases for disabling the test.

If orphan sections exist and are dynamically or unpredictably named
the unrecognized section test will fail.

If out-of-tree sections exist, one might want to temporarily disable
the "unrecognized section" test until one has made it recognized.

The test is disabled through a CLI flag.

Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
2018-11-13 11:13:33 -05:00
Sebastian Bøe
5a2ddf37d9 msys: Drop support for building in an MSYS environment
MSYS support was added as a stop-gap while native windows support was
unsupported. Now that Native windows support is stable we can drop
support for MSYS.

Dropping support for MSYS fixes #11260 and allows us to spend more
resources on native windows support.

Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
2018-11-13 10:28:33 +01:00
Anas Nashif
043b0a6a28 ci: build docs on last node
Distribute the load and build docs correctly on the last node.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-11-11 08:34:12 -05:00
Anas Nashif
0a63947abc ci: move ci scripting from .shippable to a script
move CI scripting to a dedicated script. For example, to simulate what
is run in CI when a pull request is submitted:

./scripts/ci/run_ci.sh -b master -r upstream  -p

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-11-10 13:09:00 -05:00
Anas Nashif
399281beb8 scripts: remove unused ci scripts
Remove old and unused scripts from the tree.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-11-10 13:09:00 -05:00
Erwan Gouriou
eaac6d3cf9 scripts/dts: extract_dts_includes: fix multi-compat bus parent
Case when bus parent has more than one compatible was not treated
correctly. Use get_compat() method which returns first compat
in case several compats are available

Fixes #11121

Signed-off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
2018-11-10 11:45:53 -06:00
Andrew Boie
506f15c381 interrupts: simplify position of sw ISR table
We now place the linker directives for the SW ISR table
in the common linker scripts, instead of repeating it
everywhere.

The table will be placed in RAM if dynamic interrupts are
enabled.

A dedicated section is used, as this data must not move
in between build phases.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2018-11-10 11:01:22 -05:00
Andrew Boie
7bac15f2ff x86: add dynamic interrupt support
If dynamic interrupts are enabled, a set of trampoline stubs
are generated which transfer control to a common dynamic
interrupt handler function, which then looks up the proper
handler and parameter and then executes the interrupt.

Based on the prior x86 dynamic interrupt implementation which
was removed from the kernel some time ago, and adapted to
changes in the common interrupt handling code, build system,
and IDT generation tools.

An alternative approach could be to read the currently executing
vector out of the APIC, but this is a much slower operation.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2018-11-10 11:01:22 -05:00
Mikkel Jakobsen
9c8a6099a7 scripts: locked pyocd version to 0.12.0 in requirements.txt
The recently released version 0.13.0 of pyocd does not install cleanly
in linux and prevents installation of requirements.txt. Locking pyocd to
version 0.12.0 is a temporary workaround until problem is solved by
pyocd maintainers.

Signed-off-by: Mikkel Jakobsen <mikkel.aunsbjerg@prevas.dk>
2018-11-10 10:06:17 -05:00
Himanshu Jha
58781d5782 coccinelle: Add script to check legal usage of sizeof expression
sizeof when applied to a pointer typed expression gives the size of
the pointer and not the size of the object associated with the pointer
expression leading to errors.

This scripts checks for inconsistencies where sizeof is incorrectly
used, especially while calculating size of memory to be allocated in
memory allocating functions.

Eg:

-	memset(pStr, 0, sizeof(pStr));
+	memset(pStr, 0, sizeof(*pStr));

Signed-off-by: Himanshu Jha <himanshujha199640@gmail.com>
2018-11-07 10:34:24 -05:00
Himanshu Jha
1e3e10cfa8 coccinelle: Add script to remove redundant semicolon
This script allowes to remove the redundant semicolon from
`if`, `switch`, `while`, `for` statements.

Signed-off-by: Himanshu Jha <himanshujha199640@gmail.com>
2018-11-07 10:34:06 -05:00
Himanshu Jha
6f2c7eed49 coccinelle: Add script to find cases of unsigned < 0
Unsigned expressions cannot be less than zero and presence
of such practices very likely indicates a bug.

Signed-off-by: Himanshu Jha <himanshujha199640@gmail.com>
2018-11-07 10:33:43 -05:00
Himanshu Jha
1df011c8e2 coccinelle: Add script to find cases of missing locks
This script finds cases of missing locks in the code.
There is some possibilty of false positives in cases
where a particular function is supposed to exit with
the lock held or there is any preceding function call
that releases the lock.

Suggested-by: Julia Lawall <julia.lawall@lip6.fr>
Signed-off-by: Himanshu Jha <himanshujha199640@gmail.com>
2018-11-07 10:32:59 -05:00
Himanshu Jha
6f98ae38a2 coccinelle: irq_lock.cocci: Update script
* Provide a single liner description at top using `///`
  comments. This one liner is displayed when using `--verbose`
  mode of coccicheck.

* Specify Condidence level property adhering to coccinelle.rst
  section "Proposing new semantic patches".

* Add virtual patch rule to handle the patch case when coccicheck
  is supplied with `--mode=patch` option.

* Add `depends on !(file in "ext")` to ignore reports from `ext/`
  directory.

* Simplify rule to use disjunctions and use "exists" to match any
  available control path.

Suggested-by: Julia Lawall <julia.lawall@lip6.fr>
Signed-off-by: Himanshu Jha <himanshujha199640@gmail.com>
2018-11-07 10:31:41 -05:00
Himanshu Jha
ff5a8a85e1 coccinelle: ignore_return.cocci: Update script
* Provide a single liner description at top using `///`
  comments. This one liner is displayed when using `--verbose`
  mode of coccicheck.

* Specify Condidence level property adhering to coccinelle.rst
  section "Proposing new semantic patches".

* Add virtual patch rule to handle the patch case when coccicheck
  is supplied with `--mode=patch` option.

* Add `depends on !(file in "ext")` to ignore reports from `ext/`
  directory.

* Simplify patch rule to reduce effort in reconstruction since
  logically we are only *adding* void cast.

Signed-off-by: Himanshu Jha <himanshujha199640@gmail.com>
2018-11-07 10:30:11 -05:00
Himanshu Jha
6aaba6842b coccinelle: Add script to report cases of NULL dereference
This script finds cases of NULL dereferences where a variable
is dereferenced under a NULL test, even though it is known
to be NULL.

Signed-off-by: Himanshu Jha <himanshujha199640@gmail.com>
2018-11-07 10:28:58 -05:00
Himanshu Jha
060414704b coccinelle: unsigned_shift.cocci: Update script
* Use `expression` metavariable instead of `constant` to catch
  missing cases such as:

diff -u -p a/subsys/net/ip/rpl.c b/subsys/net/ip/rpl.c
--- a/subsys/net/ip/rpl.c
+++ b/subsys/net/ip/rpl.c
@@ -686,7 +686,7 @@ static void new_dio_interval(struct net_
 {
 	u32_t time;

-	time = 1 << instance->dio_interval_current;
+	time = BIT(instance->dio_interval_current);

* Provide a single liner description at top using `///`
  comments. This one liner is displayed when using `--verbose`
  mode of coccicheck.

* Specify Condidence level property adhering to coccinelle.rst
  section "Proposing new semantic patches".

* Add virtual patch rule to handle the patch case when coccicheck
  is supplied with `--mode=patch` option.

* Edit patch rule to remove redundant parentheses in the output.

* Add `depends on !(file in "ext")` to ignore reports from `ext/`
  directory.

Signed-off-by: Himanshu Jha <himanshujha199640@gmail.com>
2018-11-07 10:27:44 -05:00
Kumar Gala
7c2a8a4678 dts/extract_dts_includes.py: Expose flash partition extract function
Allow directly calling the flash partition extraction function and
remove passing any arguments that the function doesn't actually use.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2018-11-06 07:42:39 -06:00
Kumar Gala
6fd3fbbd22 dts/extract_dts_includes.py: Rework flash partition extraction
Reworked flash partition extraction so we call it when we see a
node that is 'partiton@' instead of per property.  We also handle the
'read-only' property in the flash partition extraction function which
lets us remove 'use-property-label' handling.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2018-11-06 07:42:39 -06:00
Kumar Gala
577df2dbe3 dts/extract_dts_includes.py: remove use of 'use-property-label' in regs
We don't have any cases that need 'use-property-label' related to
handling of reg extraction so lets remove it.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2018-11-06 07:42:39 -06:00
Kumar Gala
30960e61bf scripts/dts: extract_dts_includes: Add support for client pwms
Add support to the extract script to handle 'pwms' client properties.
This is pretty much identical to how we hand 'gpios'.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2018-11-05 14:01:21 -06:00
Anas Nashif
a4c368ef07 sanitycheck: only log printable chars
New Qemu is emitting binary charachters which is disturbing generation
of XML, so make sure we only log printable charachters.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-11-05 11:00:38 -05:00
Ulf Magnusson
dc6b0c83d2 menuconfig: Add two small error reporting and UI improvements
Update menuconfig and Kconfiglib to upstream revision 68426efeae6aa, to
add two minor menuconfig improvements and a small bugfix:

 - Expected errors, like syntax errors in Kconfig files, no longer
   generate a Python backtrace. Just the error message is shown.

 - Entering a choice now places the cursor on the selected symbol, if
   any.

 - Having two consecutive empty 'if's (which could also appear e.g. due
   to osource) crashed the menuconfig when entering show-all mode, due
   to the 'if' removal logic in Kconfiglib failing to remove one of
   them. All configurations of 'if's are now correctly removed.

   This error was found while reading the code.

Some minor optimizations and some internal cleanup is included as well.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-11-04 22:00:56 +01:00
Kumar Gala
a834c53e91 scripts: dts: extract: Fix handling of prop being a list of lists
Before dtc 1.4.7 we'd get something like the following for an gpios
property:

reg = <1 2 3 4>;

After dtc 1.4.7 we get:

cs-gpios = <0x05 0x0d 0x00>, < 0x06 0x00 0x00>;

We should handle both cases in the extract reg handling code.  So if
we see a list of lists, we flatten it to a single list to normalize
the property.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2018-11-02 23:11:33 -04:00
Kumar Gala
73c7cd184b size_report: cleanup up detection and error handling
We first check to see if we have a valid elf file before doing anything
else.  Otherwise we'd could get script errors instead of just notifying
the user the file doesn't exist.

Signed-off-by: Kumar Gala <kumar.gala@gmail.com>
2018-10-29 10:15:17 -04:00
Kumar Gala
f6af3ed14a size_report: Mark arguments as required
We need a minimum of --outdir, --objdump, and --nm set for the script to
work so mark these arguments as required.

Signed-off-by: Kumar Gala <kumar.gala@gmail.com>
2018-10-29 10:15:17 -04:00
Kumar Gala
a4173329d4 size_report: convert to use argparser as optparse has been deprecated
Do a straight forward conversion from optparse to argparser.  All
of our other python scripts use argparser already.

Signed-off-by: Kumar Gala <kumar.gala@gmail.com>
2018-10-29 10:15:17 -04:00
Ulf Magnusson
85f8c0d4eb menuconfig: Add jump-to for choices, menus, and comments
Update menuconfig (and Kconfiglib, just to sync) to upstream revision
bf1701b36634b, to add this commit:

    menuconfig: Add jump-to for choices, menus, and comments

    For choices, search the name and the prompt. This is the same as for
    symbols, except names are optional (and rare) for choices.

    For menus and comments, search the prompt (title text).

    When jumping to a non-empty choice or menu, jump into it instead of
    jumping to its menu node. If show-all mode is off and there are
    visible items in the choice/menu, then jump to the first visible
    node. Otherwise, enable show-all and jump to the first node.

Previously, only symbols could be jumped to.

Various other small fixes/improvements are included too:

 - The "no range constraints" text was dropped from the input dialog
   when settings int/hex symbols without an active 'range'. It might be
   more confusing than helpful.

 - A crash when pressing Ctrl-F (the view-help shortcut) with no matches
   in the jump-to dialog was fixed

 - Some gnome-terminal shoddiness was worked around to remove minor
   jumpiness when reducing the height of the terminal

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-10-27 15:56:36 -04:00
Erwan Gouriou
94d3b6a211 scripts/dts: extract: fix 'compatbible' property typo
Replace "Compatibles" with "compatibles".

Signed-off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
2018-10-25 08:02:33 +01:00
Erwan Gouriou
ba79b2f43f scripts/dts: extract: Provide a value to generated compat #define
Compat define is generated without value. This should not harm,
but since generated flag is a "Kconfig like" define, we migth need it
to hold a value.

Signed-off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
2018-10-25 08:02:33 +01:00
Erwan Gouriou
a6716aef15 scripts/dts: extract_dts_includes: fix alias creation for subnodes
Heuristic for populating an alias string for a subnode
of an alias node was wrong as there was no check if
alias already existed and string created was not in line
with device tree specfication (allowed chars: [0-9],[a-z], '-').

Fix these two points.

Signed-off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
2018-10-24 18:17:09 +01:00
Kumar Gala
f6a30ff070 dts/extract_dts_includes.py: refactor get_node_label_string
We can replace the method get_node_label_string in DTDirective with the
global function get_node_label.  This way we have a single place that
labels for defines are being generated.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2018-10-24 10:41:25 +01:00
Kumar Gala
193ad8ef9f dts/extract_dts_includes.py: refactor get_node_label
Don't based the compat to get_node_label, we can just figure out based
on the node address we are passed.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2018-10-24 10:41:25 +01:00
Kumar Gala
168c2dec95 dts/extract_dts_includes.py: Move label override handling
Refactor the handling of 'use-property-label' into extract_prop since
its the only place its needed.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2018-10-24 10:41:25 +01:00
Kumar Gala
25ec17683f scripts/dts/extract: Change compat define prefix
Drop CONFIG_ and just use DT_ prefix for compatiable generated defines.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2018-10-24 10:40:46 +01:00
Adithya Baglody
1fa8cf9279 linker: arm: place priv.noinit at the end of RAM
This puts the priviledged stack at the end of RAM.
This combines PR #10507 and #10542.

Fixes #10473
Fixes #10474
Fixes #10515

Signed-off-by: Adithya Baglody <adithya.nagaraj.baglody@intel.com>
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2018-10-19 16:11:34 -04:00
Kumar Gala
b61b0cc631 dts/extract_dts_includes.py: Filter out standard properties
Standard device tree properties like 'reg-names', 'interrupt-names', and
'phandle' should be handled by other extract mechanisms or ignored.  So
filter them out before we even call extract_property

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2018-10-19 10:01:51 -05:00
Anas Nashif
924a4e7000 sanitycheck: support requirements on env variables
Some boards depend on environment variables so we want to make sure we
do not attempt to build boards requiring additional setup.

Add the section below into the board YAML file, sanitycheck will check
the environment and will only run tests on that board if the variables
are defined.

env:
  - VAR1
  - VAR2

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-10-19 10:57:54 -04:00
Kumar Gala
fc5e23365e dts: Add proper handling for boolean properties
We've never handled boolean properties well, if we had a boolean that
existed we'd generate a define of 'True' and if the boolean didn't exist
than nothing would be generated.  So there was no easy way to tell that
the boolean wasn't defined.

Now if we mark a property as boolean in the yaml, we will generate a
define for it regardless if it exists or not.  If it exists we'll
set the value to 1, and if it doesn't to 0.

Fixes #8376

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2018-10-19 07:55:34 -05:00
Kumar Gala
527a69ce62 dts/extract/globals.py: Add function to get list of compats
Add a function that given a node address we get a list of compats back
instead of just the first.  This is in prep for eDTS support.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2018-10-18 12:57:32 -05:00
Kumar Gala
3c14bbe884 scripts: dts: extract: Fix handling of reg prop being a list of lists
Before dtc 1.4.7 we'd get something like the following for an reg
property:

reg = <1 2 3 4>;

After dtc 1.4.7 we get:

reg = <1 2>, <3 4>;

We should handle both cases in the extract reg handling code.  So if
we see a list of lists, we flatten it to a single list to normalize
the property.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2018-10-18 06:59:13 -05:00
Paul Sokolovsky
b9248a17ac scripts: extract_dts_includes: Read files as utf-8
Python's open() by default using "system character encoding", which
may vary from system to system (e.g. LOCALE=C aka "ascii" on bare-OS
systems, something on Windows, etc.). But Zephyr files are utf-8, so
read them as such, by explicitly specifying encoding.

This is similar to changes earlier done to another script in 94620bd.

Signed-off-by: Paul Sokolovsky <paul.sokolovsky@linaro.org>
2018-10-17 13:56:25 +02:00
Johann Fischer
fcffebfe15 sanitycheck: whitelist font entry section
Whitelist font entry section.

Signed-off-by: Johann Fischer <j.fischer@phytec.de>
2018-10-16 14:54:47 -04:00
Alberto Escolar Piedras
3777b4452d sanitycheck: nf52_bsim testcases to run with other native
To be able to collect all coverage reports from the "native"
applications in the first CI instance, let's also place the
nrf52_bsim testcases first. The order now is:
 unit_testing
 native_posix
 *_bsim
 anything else

Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
2018-10-15 09:36:37 -04:00
Anas Nashif
2c7636bc88 sanitycheck: fix skipped element in junit output
The skipped element was left empty, fill it with details to make parsers
happy.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-10-13 11:35:45 -04:00
Anas Nashif
1ea5d7b472 sanitycheck: run tests based on sub-testcases
run tests based on sub-testcases, for example:
  sanitycheck  --sub-test net.app.app_tcp4_client_hostname_fail

will run all the tests in tests/net/app/net.app.

Useful for re-running tests that are reported in testrail and in reports
generated by sanitycheck.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-10-13 11:35:45 -04:00
Anas Nashif
07d54c042e sanitycheck: force configured toolchain
Do not check for board/toolchain combination and if a certain toolchain
is configured for a board, run with whatever toolchain we have.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-10-13 11:35:45 -04:00
Sebastian Bøe
5681f87fd7 sanitycheck: Support legacy variant 'gccarmemb'
CMake will translate from the toolchain variant 'gccarmemb' to
'gnuarmemb', but sanitycheck does not. This causes inconsistent and
therefore confusing behaviour between CMake and sanitycheck.

Until gccarmemb is dropped support for, do the same translation with
sanitycheck.

Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
2018-10-12 11:05:07 -04:00
Anas Nashif
a3be0945fe west: runner: fix naming and paths to xt-ocd
Fix naming of boards supported by this runner.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-10-11 10:49:56 -04:00
Erwan Gouriou
76cba5d5ea scripts: cleanup unwanted updstreamed file
scripts/dts_sanity/edts_ref.json_save has been pushed by error
in commmit a429104095
Clean it up.

Signed-off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
2018-10-11 09:24:02 -05:00
Kumar Gala
011b93011e dts: yaml: remove unused id field
The 'id' field was never used and tended to just have the compat of the
node.  Lets remove it and removed some code in extract_dts_includes.py
related to it.  Added a warning if 'id' is set in a yaml so we can
remove it going forward.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2018-10-11 08:03:44 -05:00
Praful Swarnakar
cf89e2883d sanitycheck: Add fixture option for external hardware dependecy
Add a new option as fixture in harness configurations for utilizing
sanitycheck to identify test cases that require external hardware
such as sensor, ble, networking for validation. The config will be
added to yaml files with unique fixture name to identify each hardware
and allow automation to trigger test execution on setup having the
specific fixture enabled. Also, remove the default required for type and
regex configs that is not essential in case of ztest based test cases.

Signed-off-by: Praful Swarnakar <praful.swarnakar@intel.com>
2018-10-10 19:11:29 -04:00
Ulf Magnusson
5cfe06b6bc genrest/menuconfig: Fix crash for promptless choices
Update menuconfig (and Kconfiglib, just to sync) to upstream revision
256e5b3e38e92 to get the fix below in, for an issue in an external
project. Also update genrest.py with a similar fix (the genrest issue
was what originally prompted it).

    menuconfig: Improve/fix promptless choice handling

    The code assumed that all parent (interface) menus always have a
    prompt, which is false for items in promptless choices. This led to
    a crash e.g. when viewing the symbol information for a symbol within
    a promptless choice.

    Promptless choices with children can show up "legitimately" when
    people define choices in multiple locations to add symbols, though
    this is broken in the C tools.

    Use standard_sc_expr_str(node.item) instead of the non-existing
    prompt for promptless choices. That way they show up as
    '<choice (name if any>)>', which is consistent with how they're
    shown elsewhere.

    This commit also changes how choice names are displayed in
    show-name/show-all mode, to the standard_sc_expr_str() format.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-10-10 16:16:28 +02:00
Erwan Gouriou
a429104095 ext/hal/st: stm32f2: dos2unix and remove trailing white spaces
Format all cube packages to the same format before update to new
versions:
-Apply dos2unix
-Remove trailing white spaces

Signed-off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
2018-10-09 12:03:17 -04:00
Kumar Gala
c922788d0a scripts/dts/extract: Add support for translating range properties
For now we've used range properties as a pass through.  However range
properties can translate from one address space to another.  This is
typically used one address spaces translate from one physical bus to
another (For example going from PCI address space to internal SoC memory
map).  However, we can also use this for cases where we want to reduce
duplication (For example with ARMv8-M for secure v non-secure MMIO
registers).

'ranges' takes either the form of:
	ranges; /* pass through translation */
or:
	ranges = <child-bus-address parent-bus-address length>;

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2018-10-05 13:19:39 -05:00
Kumar Gala
31340d8cdd scripts/dts/extract: Re-implement get_addr_size_cells
We use to walk to the full tree in search of addr/size cells.  We should
only have to look at the parent of the node for these properties.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2018-10-05 09:59:07 -05:00
Kumar Gala
6a8d5952d0 scripts/dts: Add help function to get address/size cells for a node
Given the path address to a node return the address/size cells.  In
theory this should just be looking up the #{address,size}-cells in
the parent node, we'll make that cleanup next.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2018-10-05 09:59:07 -05:00
Kumar Gala
bf0001242c scripts/dts/extract: Add get_parent_address helper function
We have a few places that we want the path to the parent, so lets add a
helper function for that.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2018-10-05 09:59:07 -05:00
Mieszko Mierunski
316f8604b7 script: dts: Fix alias generation for reg property.
In case of having more than one reg, aliases would not generate
properly. Number of register at the end of define was missing.

Signed-off-by: Mieszko Mierunski <mieszko.mierunski@nordicsemi.no>
2018-10-04 07:54:48 -05:00
Himanshu Jha
ab35ebe17d coccinelle: Add script to enforce ARRAY_SIZE macro usage
This script detects cases where ARRAY_SIZE can be used such as
where there is a division of sizeof the array by the sizeof its first
element or by any indexed element or the element type. It replaces the
division of the two sizeofs by ARRAY_SIZE helper macro.

Signed-off-by: Himanshu Jha <himanshujha199640@gmail.com>
2018-10-02 14:03:39 -07:00
Himanshu Jha
dce9141fb1 Coccinelle: Add support for Coccinelle infrastructure
'coccicheck' target is used to initiate Coccinelle checker. It
can be called via four different modes that generate different
output corresponding the mode.

The four modes are:

* 'patch' proposes a fix, when possible.
* 'report' generates a list in the following format:
	file:line:column-column: message
* 'context' highlights lines of interest and their context in
   a diff-like style. Lines of interest are indicated with '-'.
* 'org' generates a report in the Org mode format of Emacs.

The 'coccicheck' front-end can be called with suitable arguments:

* --mode=<mode>: specify the mode for processsing.
* --cocci=<path/to/foo.cocci>: specify the SmPL file to use.
* --verbose=<1>: enable verbose output.

Run `./scripts/coccicheck --help` for more detailed info
on various options available.

With the above arguments the coccinelle transformation engine
runs on the entire/part of the source code tree depending on
various options supplied to the coccicheck sanity checker.

More detailed documentation can be found at:
doc/application/coccinelle.rst

Cc: Julia Lawall <Julia.Lawall@lip6.fr>
Signed-off-by: Himanshu Jha <himanshujha199640@gmail.com>
2018-10-02 14:03:39 -07:00
Aurelien Jarno
872719e9a6 scripts/dts/extract/flash.py: fix load offset
When the code partition is not at the beginning of the flash, the load
offset is wrongly computed. The address in the device tree is already
relative to the beginning of the parent node, ie the beginning of the
flash memory space. There is therefore no need to subtract it.

Signed-off-by: Aurelien Jarno <aurelien@aurel32.net>
2018-10-02 10:00:04 -07:00
David Leach
dc2acd6b94 scripts: extract: globals.py: Fix node name parsing
The node name parsing for the _LABEL #define does not parse the unique
node portion in the same fashion between @<unit_address> and _<number>.
This change normalizes the two modes.

Signed-off-by: David Leach <david.leach@nxp.com>
2018-10-02 09:45:39 -05:00
Ulf Magnusson
905e65d694 Kconfig: Improve error messages for mismatched endchoice/endif/endmenu
Update Kconfiglib (and menuconfig, just to sync) to upstream revision
8087311cd91be to get the following fix in, for an issue reported by
pfalcon:

    Give clearer errors for bad endchoice/endif/endmenu nesting

    An endchoice/endif/endmenu with no corresponding choice/if/menu
    generated a cryptic 'unrecognized construct' parse error. Improve
    the error message so that the problem is pointed out explicitly:

      kconfiglib.KconfigError: Kconfig:37: couldn't parse 'endmenu': no
      corresponding 'menu'

    Reported in https://github.com/ulfalizer/Kconfiglib/issues/56.

This update also adds support for user-defined preprocessor functions in
Python, which could potentially be handy for DTS/Kconfig integration.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-09-29 22:01:00 +02:00
Adithya Baglody
10c6a0c1aa subsys: app_memory: Fixed the build issue wrt ARC.
The linker script is updated to support the new method of size
calculation.

Signed-off-by: Adithya Baglody <adithya.nagaraj.baglody@intel.com>
2018-09-28 08:09:23 +05:30
Flavio Ceolin
6fc84feaf2 kernel: syscalls: Change handlers namespace
According C99 the first 31 characters of an identifier must be unique.
Shortening the namespace of the generated objects to achieve it.

C99 - 5.2.4.1
MISRA-C rule 5.1

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2018-09-28 07:58:19 +05:30
Mark Ruvald Pedersen
f0e2e1bccb portability: Remove C99-illegal semicolons from the global scope
This commit touches the C codebase and the python syscall generator.

The Z_GENLIST-macros expand to whole functions. Once expanded by the
preprocessor we notice a semicolon is put after the function body. But
ISO C99 does not allow extra ‘;’ outside of a function. Though this is
accepted by GCC with GNU extensions, it is not by Clang.

Signed-off-by: Mark Ruvald Pedersen <mped@oticon.com>
2018-09-28 07:57:28 +05:30
Marti Bolivar
ddad53abe2 scripts: add colorama to requirements.txt
This got missed in the latest west code import.

Fixes #10241.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2018-09-27 05:52:39 +05:30
Marti Bolivar
9e3edbbd80 scripts: fix west launcher for macOS
The $(readlink -f) call in scripts/west doesn't work on macOS.

Let's just ensure compatibility on all the platforms by moving
scripts/west-launcher.py to scripts/west, making it executable on
macOS and Linux, and launching it from the Windows doskey macro.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2018-09-26 00:33:45 +02:00
Marti Bolivar
55b462cdfa scripts: update west to latest upstream version.
Update to the latest west. This includes a new 'attach' command. There
are also multi-repo commands, but those won't get exposed to the user
unless they install Zephyr using "west init" + "west fetch" (and not,
say, "git clone").

Replace the launchers; they now detect whether zephyr is part of a
multi-repo installation, and run the west code in its own repository
if that is the case.

This also requires an update to:

- the flash/debug CMakeLists.txt, as the new west package is no longer
  executable as a module and must have its main script run by the
  interpreter instead.

- the documentation, to reflect a rename and with a hack to fix
  the automodule directive in flash-debug.rst for now

Signed-off-by: Marti Bolivar <marti@foundries.io>
2018-09-25 17:51:22 +02:00
Anas Nashif
ba4643b778 sanitycheck: remove special ansi charachters from xml
XML does not like special ansi charachters (coming from logger), so
remove them.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-09-23 13:04:05 -04:00
Anas Nashif
0f55538762 gitlint: accept "'" in committer names
Allow "'" in names, for example xxx O'xxx.

Fixes #10152

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-09-21 12:49:23 -04:00
Kumar Gala
c57f3a09bc scripts/dts/extract: Fix handling of value extraction based on cells
If size or address cells happened to be 2 we'd get the wrong value since
we where shift the least significant 32-bit's up.  Adjust the math based
on the value of the cell amount to fix things.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2018-09-21 07:23:49 -07:00
Bobby Noelte
14ced2f021 scripts/dts/extract: Cleanup reg extraction
Rename address / size cell variable to clarify that its the number
of address / size cells.

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2018-09-21 07:23:49 -07:00
Erwan Gouriou
cc054c8044 scripts/dts/extract: factorize 'reg' prop extraction
reg class is updated to take into account whole use cases
of reg extraction so extract_reg_prop could be removed.

Signed-off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
2018-09-21 07:23:49 -07:00
Erwan Gouriou
8245b0c1a2 scripts/dts: extract_dts_includes: remove unused 'names' argument
Parameter 'names' is provided as argument of several property
extract functions. Thought it is actually used only once.
Remove when not needed.

Signed-off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2018-09-21 07:23:49 -07:00
Adithya Baglody
ae92f2badf subsys: app_memory: Fixed the size calculation for power of 2 MPUs
The size calculation for power of 2 MPUs were incorrect.
The calculation was not taking into account the amount of padding
the linker does when doing the required alignment. Hence the size
being calculated was completely incorrect.

With this patch the code now is optimized and the size of
partitions is now provided by the linker.

Signed-off-by: Adithya Baglody <adithya.nagaraj.baglody@intel.com>
2018-09-20 11:25:53 -04:00
Krzysztof Chruscinski
6aed72e487 shell: Shell subsystem reimplementation
New shell support features like:
- multi-instance
- command tree
- static and dynamic commands
- multiline
- help print function
- smart tab (autocompletion)
- meta-keys
- history, wildcards etc.
- generic transport (initially, uart present)

Signed-off-by: Jakub Rzeszutko <jakub.rzeszutko@nordicsemi.no>
Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
Signed-off-by: Piotr Zięcik <piotr.ziecik@nordicsemi.no>
2018-09-19 09:30:29 -04:00
Flavio Ceolin
4218d5f8f0 kernel: Make If statement have essentially Boolean type
Make if statement using pointers explicitly check whether the value is
NULL or not.

The C standard does not say that the null pointer is the same as the
pointer to memory address 0 and because of this is a good practice
always compare with the macro NULL.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2018-09-18 13:57:15 -04:00
Flavio Ceolin
a7fffa9e00 headers: Fix headers guards
Any word started with underscore followed by and uppercase letter or a
second underscore is a reserved word according with C99.

With have *many* violations on Zephyr's code, this commit is tackling
only the violations caused by headers guards. It also takes the
opportunity to normalize them using the filename in uppercase and
replacing dot with underscore. e.g file.h -> FILE_H

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2018-09-17 15:49:26 -04:00
Anas Nashif
2e7bdb6be2 scripts: gen_priv_stacks.py: run through flake8
Follow pep8 style based on flake8 script.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-09-17 09:40:02 -04:00
Anas Nashif
7a08b2b353 scripts: gen_kobject_list.py: run through flake8
Follow pep8 style based on flake8 script.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-09-17 09:40:02 -04:00
Anas Nashif
6af68b3623 scripts: elf_helper.py: run through flake8
Follow pep8 style based on flake8 script.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-09-17 09:40:02 -04:00
Ulf Magnusson
e0f1d77f5c menuconfig: Fix some minor graphical glitching and add custom styles
Update menuconfig.py to upstream revision 35a60b786c646.

This fixes some minor graphical glitching, with parts of lines that
don't fit the terminal wrapping around to the next line, and some
headings disappearing.

This version also adds supports for custom style definitions,
contributed by Mitja Horvat (punkfluid). Colors can be specified both by
number and in #RRGGBB notation. Color accuracy varies depending on
terminal capabilities, but will usually be good.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-09-17 09:55:12 +02:00
Flavio Ceolin
da49f2e440 coccicnelle: Ignore return of memset
The return of memset is never checked. This patch explicitly ignore
the return to avoid MISRA-C violations.

The only directory excluded directory was ext/* since it contains
only imported code.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2018-09-14 16:55:37 -04:00
Flavio Ceolin
91b6123932 kernel: syscall: Explicitly ignoring not used return
Some syscacll return value through parameters and for these functions
the return of _arch_syscall_invoke* are not used.

MISRA requires that all return values be checked.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2018-09-14 16:55:37 -04:00
Carles Cufi
5f102fbfca scripts: ci: Set SOC_DIR for check-compliance.py
To avoid having to set it externally, set the SOC_DIR environment
variable directly in the compliance script.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2018-09-14 17:45:56 +02:00
Erwan Gouriou
c5ada39411 scripts: extract_dts_includes.py: generate 'compatible' CONFIG flags
Generate CONFIG_DT_COMPAT flags using available nodes
'compatible' property.
Store them in a bogus node_address key of defs dict so
duplicates could be removed.

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
Signed-off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
2018-09-14 08:02:53 -05:00
Anas Nashif
81b272119a docs: fixed documenation pointers
Fixed URL to documentation, now latest docs are under /latest/..
Fixes #9932

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-09-12 12:03:18 -04:00
Flavio Ceolin
3259ac08ca kernel: userspace: Sanitize switch usage
MISRA-C requires that every switch clause has a break instruction.
Changing gen_kobject_list script to generates compliance code.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2018-09-12 10:05:06 -04:00