Commit graph

5615 commits

Author SHA1 Message Date
Anas Nashif
dbd7649fc0 sanitycheck: add option to specify platforms with coverage support
We want to run coverage on select platforms even if a few do support
that now. Specify which platforms should be enabled using the new option

--coverage-platform

By default we have the posix platform configured. Any additional
platforms can be specified using the option above.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-01-16 06:12:33 -05:00
Anas Nashif
db9592aa3e sanitycheck: convert gcov data to gcda
Collect data generated by built-in gcov over UART.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-01-16 06:12:33 -05:00
Adithya Baglody
131edad483 scripts: gen_gcov_files.py: Added python script to parse uart dump
This script will parse the serial console log file and create the
required gcda files.

Usage:
python3 ${ZEPHYR_BASE}/scripts/gen_gcov_files.py -i console_output.log
Add -v for verbose.

Signed-off-by: Adithya Baglody <adithya.nagaraj.baglody@intel.com>
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2019-01-16 06:12:33 -05:00
Kumar Gala
8f52956905 scripts: extract_dts_includes: Change arg parsing requirements
Changed arg parsing so you can invoke the script and only generate the
include file or conf file as desired.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-01-15 08:10:31 -06:00
Jan Kowalewski
265895b293 sanitycheck: Add Renode support
The change adds an optional capability to kill process by PID in the
BinaryHandler. At the moment proc.terminate only closed the make
process, also closing the stdin/stdout, but this is impossible to detect
in .NET.

Signed-off-by: Dawid Wojciechowski <dwojciechowski@internships.antmicro.com>
Signed-off-by: Jan Kowalewski <jkowalewski@antmicro.com>
Signed-off-by: Piotr Zierhoffer <pzierhoffer@antmicro.com>
2019-01-14 09:12:07 -05:00
Kumar Gala
35ecc78cdf scripts/dts/extract: Generate unique defines based on compat & instance
Add support to generate defines of the form
DT_<COMPAT>_<INSTANCE>_<PROP>.  The idea is that we can utilize this in
drivers to remove the need for dts_fixup.h.  The <INSTANCE> value is
determined by the script and starts at 0 and counts up for each instance
of a given <COMPAT>.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-01-14 06:33:29 -06:00
Kumar Gala
3b9147df3f scripts/dts/extract: Fliter duplicates values inserted into defs
If an aliases already exists in the defs we should filter it out.  Also
if an alias exists in the new_def we should filter the alias out.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-01-14 06:33:29 -06:00
Kumar Gala
94240aebb2 scripts/dts/extract: Add an instance ID for each compatiable
Add a unique instance starting at 0 for each compatiable in the dts.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-01-14 06:33:29 -06:00
Sebastian Bøe
b72fb4e9e8 cmake: Remove dead code 'FindRegex.cmake'
When Zephyr was building the C implementation of KConfig it used the
build script 'FindRegex.cmake' to satisfy a regex dependency.

Now that we no longer use the C implementation of Kconfig we no longer
use this build script, and it is safe to remove it.

Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
2019-01-14 13:02:03 +01:00
David Leach
afcfa11ce0 scripts: extract_dts_includes: Enhance message information
The information message when processing the yaml files is
cryptic when there is an override. This change adds filenmae
of yaml file being processed and the parent tuple with
the override.

Signed-off-by: David Leach <david.leach@nxp.com>
2019-01-12 15:23:11 -06:00
Maureen Helm
73956a398e scripts: Update pyocd version to 0.14.3
pyocd recently merged its command-line tools into a unified pyocd tool
with subcommands. The separate command-line tools still remain, but are
deprecated. Update the pyocd version required by zephyr in preparation
for updating the pyocd runner to use the new unified tool commands.

Pin pyocd to a specific version rather than allowing any version greater
than or equal to 0.14.0 because there have been a few unstable releases
recently. Version 0.14.3 is what I have been using locally and know to
be stable.

Signed-off-by: Maureen Helm <maureen.helm@nxp.com>
2019-01-11 21:33:28 -06:00
Kumar Gala
e57a596a0c scripts/dts/extract: Support QSPI memory mapped flash
We need to refactor how we determine CONFIG_FLASH_{SIZE,BASE_ADDRESS}
for when we have a QSPI that is memory mapped. "zephyr,flash" is going
to point the actual flash component that will be on a SPI bus and thus
the device node's reg property tells us which CS on the SPI bus its on,
not the memory mapped address/size of the flash.

So we make a few assumptions to handle this case:
1. If the #size-cells for the flash node is 0, we assume its a QSPI
   memory mapped flash
2. That the QSPI memory mapped node (parent of what "zephyr,flash" is
   point to, will contain a reg propery where the second reg pair will
   be the memory mapped region for the flash.

We move handling of CONFIG_FLASH_{SIZE,BASE_ADDRESS} into flash.extract
so its all in one place.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-01-11 13:12:18 -06:00
Marti Bolivar
0b1b4e2d23 scripts: update west to upstream commit f01059a
The main reason to copy into upstream is to bring in a new runner.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2019-01-11 10:33:24 +01:00
Andy Ross
dc4151f8f3 scripts/sanitycheck: Don't build "slow" tests either
Extend the "slow" flag on tests to inhibit building too.  The original
assumption was that building would be fast but running slow, but now
We have tests using a component (OpenThread) that wants to pull and
build software from github for every app.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2019-01-04 14:59:28 -05:00
Andy Ross
9c9162dfd9 sanitycheck: Add --disable-size-report option
The segment size computation has value, but it's actually a very slow
process that parallizes poorly.  It seems to be bound by the Python
GIL doing the parsing, so never sees more than about 150% of CPU in
use even on wildly parallel systems.  And it takes about 75-80
seconds, which is 15-20% of the entire runtime of the test on that
box!

And the only "failure" case this can detect (unexpected sections in
the output file) is now a duplicate of the orphan section warning
we've since enalbled at the linker level.

This defaults to enabling the test to preserve behavior (as I don't
know where all the existing users of the size report are to change
them), but long term we might consider making "disabled" the default
and switching this to an --enable flag.

Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
2019-01-03 16:06:28 -05:00
Ulf Magnusson
cf3cddd1cd menuconfig: Prompt for save when Kconfig files change
Update menuconfig and Kconfiglib to upstream revision 16539b4f223fa, to
add the commit below.

    menuconfig: Prompt for save if a different .config would be saved

    Previously, menuconfig.py only prompted for saving the configuration
    if .config didn't exist or the user changed symbol values within the
    interface.

    Also make it prompt for save if Kconfig symbols have been added,
    removed, or have had their defaults changed, provided it would make
    the saved .config differ from the loaded one.

    This usually won't matter for correctness, because loading an
    outdated configuration performs an implicit olddefconfig, but it's
    less confusing.

    Also add a Kconfig.missing_syms attribute that records all
    assignments to undefined symbols in the most recently loaded .config
    file. This is needed to implement the check for whether the saved
    .config would be different.

    As an unrelated change, always prompt for saving if a .config has
    been loaded from within the menuconfig interface. The intention is
    probably often to save the configuration somewhere else, even if it
    isn't modified.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-01-03 13:55:31 +01:00
Adithya Baglody
65f79cdc21 scripts: gen_relocate_app: Use linker provided size variable.
This seems to solve the issue with GH-12033. There seems to be
some compiler optimization that was causing this issue. This
occurs only when the build is re-run by invoking cmake without
clearing the previous build.

Fixes:GH-12033

Signed-off-by: Adithya Baglody <adithya.nagaraj.baglody@intel.com>
2018-12-29 09:23:34 -05:00
Alberto Escolar Piedras
3980e0cf64 sanitycheck: Use supression file for valgrind and dump log
When running valgrind in sanitycheck, use the suppression file
and dump the log in its own file.

Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
2018-12-21 21:07:18 +01:00
Alberto Escolar Piedras
661bc8206f sanitycheck: Fail test if the process returns != 0
If the test process returns an error (return code != 0),
it should be considered a failure, even if the handler
considers the test passed.
But this is not done when we are killing (SIGTERM) the processes,
as then the return code is not necessarily sensible.

Without this, for example, when running with valgrind, all valgrind
errors are missed.

Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
2018-12-21 21:07:18 +01:00
Alberto Escolar Piedras
ace7bf9508 valgrind: Add error supression file for POSIX arch
Added a valgrind error supression file which can be used when
running in the POSIX arch.

So far we are supressing 2 types of errors:
* One very normal in glibc, which will be triggered in the nrf52_bsim
* 2 due to the posix arch prioritizing terminating fast over terminating
cleanly. The posix arch, when posix_core_clean_up() is called, will
pthread_cancel all Zephyr threads. But it will NOT wait for them to
finnish, in case any of them was hung or there is any other problem
which would prevent it. Therefore, depending on the execution, some
times the threads will be still lingering when the main thread exits
Somethign similar applies to posix_soc_clean_up().
These 2 are not errors, but conscious choices => Therefore we
supress them

Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
2018-12-21 21:07:18 +01:00
Marti Bolivar
15384b79c3 scripts: west: fix 'west debug'
We can't invoke west's main as a subprocess, or control-C from GDB in
a 'west debug' will terminate the program. Import it and run main as a
function in the same process instead.

Clean up some cruft and remove unused code while we're here, to keep
this file shorter.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2018-12-19 17:34:14 +01:00
Flavio Ceolin
8f3459b3db coccinelle: Updating ignore_return to support memcpy
The return of memcpy was being ignored just like memset's return. Just
adding it to coccinelle script.

MISRA-C rule 17.7

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2018-12-19 14:37:25 +01:00
Ulf Magnusson
1581155828 menuconfig: Add show-help mode
Update menuconfig (and Kconfiglib, just to sync) to upstream revision
e629c33d31e22, to add the commit below.

  menuconfig: Add show-help mode

  Pressing F toggles show-help mode, where the help window at the bottom
  displays the help text of the currently selected item, if any. Can be
  handy when browsing through symbols.

  Also mention the different modes that are available in the module
  docstring.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-12-18 13:28:37 +01:00
Kumar Gala
2cc957e860 scripts: extract_dts_includes: Generate defines for chosen props
Generate a set of defines that convey if a given chosen property is used
or not.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2018-12-17 11:26:40 +01:00
Carles Cufi
d9b8b02519 scripts: ci: check-compliance: Fix running out-of-tree
In order for check-compliance.py to be able to run cleanly out-of-tree,
set the repo (to be checked) path based on cwd, and refer to Zephyr and
its tools by the specific path set by ZEPHYR_BASE.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2018-12-14 13:11:24 +01:00
Kumar Gala
252919128b scripts: extract_dts_includes: Rework get_binding
The original version of get_binding didn't handle the case of having a
child node on a SPI or I2C device node.  The example is a SPI flash and
having a partition node under that.

Re-work the binding lookup logic to first look in the master bindings
dict, and if not found there we assume it must be in the bus specific
dictionary, and we can use the parent node to find the bus type.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2018-12-13 08:12:53 -06:00
Anas Nashif
1c65b6b3a4 sanitycheck: filter by device tree configs
In addition to Kconfig values from generated .config, also parse and
filter based on configs generated by device-tree.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-12-11 13:13:26 -06:00
Kumar Gala
17f0977fc9 scripts: extract_dts_includes: Fix handing of no binding
There are cases in which we get the parents compat and thus we might
have a binding available for the actual node.  An example is having a
SPI or I2C controller with a child node, but that node doesn't have a
binding associated with it.  We'll end up getting the parent compat, so
return the parent binding in this case.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2018-12-11 09:23:21 -06:00
Ulf Magnusson
57b28cae2c kconfiglib: Save previous configuration to .config.old
Update Kconfiglib (and menuconfig, just to sync) to upstream revision
094f4a9622046, to add the commit below.

  Save existing configuration to .<filename>.old in write_config()

  Add a default-True 'save_old' flag to write_config(). If 'save_old' is
  True and an existing configuration file is being overwritten, a copy
  of the old configuration file is saved to .<filename>.old (e.g.
  .config.old) in the same directory.

  Errors are ignored, as the old configuration would usually just be a
  nice-to-have, and not essential.

  The same functionality could be added for minimal configuration files
  and headers, but it's probably most useful for configuration files.

Other changes:

  - Parsing performance is improved a bit

  - scripts/kconfig/kconfig.py now prints the path to the merged
    configuration in zephyr/.config, to make it a bit easier to
    discover.

Fixes: #2907

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-12-08 17:14:38 +01:00
Andrew Boie
cd255744d0 userspace: eliminate warning in process_gperf.py
Python 3.7 throws a FutureWarning when it thinks it
has found a nested set in a regular expression.

We weren't actually; just trying to match some brackets.
Escape them instead. No functional difference.

Fixes: #11961

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2018-12-07 20:26:46 -05:00
Anas Nashif
724d6f7909 sanitycheck: add new _TEXT_SECTION_NAME_2 section to whitelist
Add _TEXT_SECTION_NAME_2 to the whitelisted list of linker sections.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-12-07 16:50:24 -05:00
Alberto Escolar Piedras
d8c25d37d9 ci: Allow setting BSIM_OUT_PATH from environment
If BSIM_OUT_PATH was set in the environment use that value

Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
2018-12-07 16:46:54 -05:00
Alberto Escolar Piedras
69772ef7d8 ci: Fail shippable if something fails catastrophically
Before run_ci.sh was introduced, if some command in the
.shippable.yml script failed, Shippable would stop and report
a failure.
When run_ci.sh was introduced we lost this, which is pretty
dangerous if something fails cathastrophically and the xml
reports are not (properly) generated and copied, as then
Shippable reports a pass.
=> Run run_ci.sh in -e mode, so we propagate a bad failure upwards

Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
2018-12-07 16:46:54 -05:00
Alberto Escolar Piedras
539e73f526 ci: Copy results before handling coverage
Coverage parsing has high chances of failing, let's copy
first the results

Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
2018-12-07 16:46:54 -05:00
Adithya Baglody
ff63101c7d scripts: gen_relocate_app.py: Script for relocating files in memory.
This script will relocate .text .data and .bss sections from
required files and places it in the required memory region. This
memory region and file are given to this python script in the form
of a string. Refer to the script for the format of this string and
the procedure to invoke it.

The main goal of this script is to provide a robust way to re-order
the memory contents without actually having to modify the code
(C source code and the linker code).
In simple terms this script will do the job of
__attribute__((section("name"))) for a bunch of files together.

Signed-off-by: Varun Sharma <varun.sharma@intel.com>
Signed-off-by: Adithya Baglody <adithya.nagaraj.baglody@intel.com>
2018-12-07 10:32:41 -05:00
Kumar Gala
471d2f3a0a scripts: extract_dts_includes: Generate device bus info define
For devices on buses like I2C or SPI its possible that a sensor might
support being on either bus type.  In the dts we can tell this by
looking at the parent bus of the device.  Its useful for a driver to
know this so can we build support into the driver accordingly.

For example if the LSM6DSL sensor ("st,lsm6dsl") is in the dts on a spi
bus we now generate:

	#define DT_ST_LPS22HB_PRESS_BUS_SPI	1

Its possible that a system exists in which multiple of the same sensor
exist but on different busses, so drivers should handle that case
accordingly.  For the LSM6DSL example we'd end up with:

	#define DT_ST_LPS22HB_PRESS_BUS_I2C	1
	#define DT_ST_LPS22HB_PRESS_BUS_SPI	1

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2018-12-07 09:08:04 -06:00
Kumar Gala
2442382fac scripts: extract_dts_includes: Support same sensor on different bus
There are a number of cases in which we a sensor can be either connected
on I2C or SPI.  We've been treating these cases as different compatiable
properties, but they really should use the same compatiable and just
determine the info based on the parent bus in the device tree.  We can
now support having two different binding files for the same compatiable
to handle the case of a sensor supporting either I2C or SPI as how its
connected.

We put "sensor" nodes in a bus specific dict bus_bindings[bus] that
we can than lookup later based on the DTS and determining the bus type
that the "sensor" node is on.

Fixes #11375

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2018-12-07 09:08:04 -06:00
Kumar Gala
2893be57ef scripts: extract_dts_includes: Split out yaml_list info
Have the yaml_list now have 'node', 'bus', and 'compat' information
instead of just the 'node' info.  For now we have all YAML nodes under
yaml_list['node'], however this will change to support sensors that
can be on I2C or SPI, since they will have the same compat, but
different YAML nodes.

This also introduces yaml_list['bus'] which has YAML nodes for any
compatiables on a given 'parent' bus (ie, 'i2c', or 'spi').

Finally, we also have yaml_list['compat'] with all the compatiables
that we parsed and matched to a YAML.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2018-12-07 09:08:04 -06:00
Kumar Gala
f34a99532a scripts: extract_dts_includes: Pass node address to get_binding
Change get_binding to take the node address instead of the
compatiable.  This is in prep for having get_binding be able to look
at the parent of the node and determine which bus specific binding to
utilize (for cases like sensors that support both I2C & SPI).

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2018-12-07 09:08:04 -06:00
Kumar Gala
112b0f5eff scripts: extract_dts_includes: Add accessor function to access binding
Add accessor functions get_binding and get_binding_compats to
encapsulate access to binding database info.  Cleanup all direct access
to binding info and replace with calls to get_binding() and
get_binding_compats()

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2018-12-07 09:08:04 -06:00
Kumar Gala
35afcc9c12 scripts: extract_dts_includes: remove passing yaml param to functions
Various functions like add_prop_aliases, extract_single_prop,
extract_single, reg.extract, compatible.extract, default.extract, and
flash.extract don't use the yaml argument that is passed to them.
Remove passing the argument, this is the first step in making access to
the yaml node information go through an accessor function.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2018-12-07 09:08:04 -06:00
Kumar Gala
b7f5101e79 scripts: extract_dts_includes: Handle YAML inheritance when we load
Move the handling of 'inherits' in the YAML to right when we load.  This
allows us to remove yaml_collapse function and will allow us to look
into the yaml right after we load it for things like if the yaml is for
a child bus.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2018-12-07 09:08:04 -06:00
Jakub Rzeszutko
f8178dcb05 shell: remove Console dependencies
Removed Console dependencies from shell uart backend.
Generated define: CONFIG_UART_SHELL_ON_DEV_NAME for each board.

Fixes #10191

Signed-off-by: Jakub Rzeszutko <jakub.rzeszutko@nordicsemi.no>
2018-12-07 12:11:11 +01:00
Anas Nashif
c562e4247d ci: remove bsim result options
This option is not being used anymore.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-12-06 14:24:41 -05:00
Anas Nashif
c3e98056ea ci: set second pass options correctly
Use correct 3rd pass env variable.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-12-06 14:24:41 -05:00
Himanshu Jha
200e847ace scripts: coccicheck: Improve continuous run operation
The current implementation of continuous run operation using
command `./scripts/coccicheck` i.e., without specifying any options,
`coccicheck` default runs in `report` mode with all available
coccinelle scripts present at `scripts/coccinelle/`.

Not all scripts have report mode implemented in them, which
leads to failure of coccicheck.

With this new implementation we choose whatever available mode
is present in coccinelle script and pass it to MODE variable
without stopping continuous coverage.

And perhaps if there are plans to add `coccicheck` as a sanity
checker in future to the Zephyr automated CI, then certainly we
want the warnings/errors produced by scripts to be less verbose
to the users.

Therefore, in this new implementation we prioritise the modes as:

1. report
2. context
3. patch
and lastly falling to
4. org

Lastly, in order to differentiate between outputs of various
coccinelle scripts being run, `x------x` separator has been used
to make reports mode readable.

Signed-off-by: Himanshu Jha <himanshujha199640@gmail.com>
2018-12-05 15:14:05 -05:00
Kumar Gala
459ad673ba scripts: extract_dts_includes: Fix DeprecationWarning
With python3.7 we get the following warning:

	extract_dts_includes.py:496: DeprecationWarning: Using or
	importing the ABCs from 'collections' instead of from
	'collections.abc' is deprecated, and in 3.8 it will stop
	working.

Fix this by using 'from collections.abc import Mapping' instead of
'import collections'.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2018-12-04 22:54:46 -05:00
Patrik Flykt
1b48cb6dc1 scripts/coccinelle: Add Coccinelle script for unsigned values
Add a Coccinelle script that adds an 'U' to values assigned to
unsigned variables, according ot MISRA-C rule 7.2.

Add a 'report' mode to the script that can be used by developer/CI
and a 'patch' mode that should do the heavy lifting.

Signed-off-by: Patrik Flykt <patrik.flykt@intel.com>
2018-12-04 22:51:56 -05:00
Håkon Øye Amundsen
81c6662d23 cmake: flash: scripts: Include externally built hex files
Allow user to add externally built hex files to the cmake property
HEX_FILES_TO_MERGE. The hex files in this list will be merged
with the hex file generated when building the zephyr application.

This allows users to leverage the application configuration
available in Kconfig and CMake to help decide what hex file
should be merged with the zephyr application.

Signed-off-by: Håkon Øye Amundsen <haakon.amundsen@nordicsemi.no>
2018-12-04 19:04:13 +01:00
Anas Nashif
3a361ecdc7 ci: do not build btsim if not installed
Do not attempt to build btsim if software is not installed.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-12-03 19:28:01 -05:00