Commit graph

5615 commits

Author SHA1 Message Date
Juan Manuel Torres Palma
dce32622cd scripts: fix tags.sh to current source tree
Update tags.sh tool to generate appropriate files to use with cscope and
ctags programs.

Current implementation generates tags for all but samples and it's
possible to filter by architecture.

Signed-off-by: Juan Manuel Torres Palma <j.m.torrespalma@gmail.com>
2018-01-15 18:30:28 -05:00
Sebastian Bøe
2612744352 cmake: Don't specify a C standard when building Kconfig
Specifying a C standard triggered a compiler warning on Ubuntu (gcc
5.4.0) and a compiler error on Mac OS 10.12.6. Omit specifying the
standard and let the host toolchain use it's default instead. Tested
on Mac OS and Ubuntu 16.04.3.

This fixes #5640

Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
2018-01-12 08:05:35 -05:00
Carles Cufi
d92769b849 scripts: kconfig: Replace Kconfig exectuables with Python
We have been using a fork of the Linux kernel's Kconfig system to
configure the Zephyr tree. The issue is that this is a native tool
written in C that is not easy to compile for Windows. This patch
replaces the use of the conf executable with kconfig.py, a script that
uses Kconfiglib to generate the .config and autoconf.h files required to
compile Zephyr.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2018-01-12 07:29:05 -05:00
Carles Cufi
f4639f186a scripts: kconfig: Prefer later defaults
This commit mirrors c6390d559f in
Kconfiglib instead of Kconfig.

Origin: https://github.com/carlescufi/Kconfiglib/tree/zephyr

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2018-01-12 07:29:05 -05:00
Carles Cufi
33bbecb946 scripts: kconfig: Add support for wildcards and globbing
Some projects use wildcards when sourcing a Kconfig file. Add
support for globbing the files that match the wildcards and process
them one by one.

Origin: https://github.com/carlescufi/Kconfiglib/tree/zephyr

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2018-01-12 07:29:05 -05:00
Carles Cufi
591eb575ea scripts: kconfig: Import Kconfiglib
Import Kconfiglib, the Python Kconfig parsing library.

Origin: https://github.com/ulfalizer/Kconfiglib
Revision: 8d30e5bb1ad5cab16d1226cc5cd3a03d64664f5d

Note that this will in time replace doc/scripts/genrest/kconfiglib.py,
which is an earlier version and should not live in that folder.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2018-01-12 07:29:05 -05:00
Carles Cufi
907735dd58 scripts: parse_syscalls: Multiplatform separators
In order to support both "/" and "\" as OS path separators,
use the correct Python os.sep helper.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2018-01-11 10:48:56 -05:00
Anas Nashif
abfed53525 gitlint: fix check for line_count
We should check if the commits line is less than, not less/equal...

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-01-11 10:48:03 -05:00
David B. Kinder
77d877c0d9 doc: update doc tools versions
I've been successfully using the latest sphinx/breathe/docutils
and doxygen versions for local doc build testing. The CI system
already uses the latest doxygen, so this patch updates the
pip-installed sphinx, breathe, and docutils tools too.

Signed-off-by: David B. Kinder <david.b.kinder@intel.com>
2018-01-11 06:51:22 -05:00
Anas Nashif
2dd5cef811 gitlint: do not allow title-only commit messages
It has been agreed in the project TSC to reject commit messages without
any content. Every commit message needs some explaination beyond what
was put in the title, even the most trivial ones.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-01-11 06:39:42 -05:00
Sebastian Bøe
4971d2a084 cmake: Fix "make VERBOSE=1"
It is desired behaviour that when 'make VERBOSE=1' is invoked, the
underlying scripts will be invoked in 'verbose mode', e.g. they have
the flag --verbose passed to them.

This patch modifies all the underlying scripts in the build system to
inject --verbose into the command line when the environment variable
VERBOSE is set.

The environment variable VERBOSE is a CMake concept. CMake will
generate Makefile's that read this environment variable and try to
behave accordingly. Unfortunately, the generated ninja build systems
behave differently and will have to be invoked like this:

VERBOSE=1 ninja -v

This fixes #4851

Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
2018-01-10 22:14:53 -05:00
Sebastian Bøe
c6f367d251 cmake: Give descriptive error to user when cloned with core.autocrlf
Windows users have on multiple occasions cloned Zephyr using a Windows
git client. It seems that the windows git client defaults to
converting line endings from LF to CRLF when cloning repo's. This
breaks at least one of Zephyr's tools (Kconfig).

This patch introduces a sanity check of the environment for MSYS
users.

Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
2018-01-10 22:12:27 -05:00
Anas Nashif
674bb28040 sanitycheck: filter on CONFIG_ASSERT
CONFIG_ASSERT is being set by cmake, so it is not possible to filter
using the generated config, add this as a standalone filter.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-01-09 11:13:08 -05:00
Anas Nashif
1a5bba72da sanitycheck: move native_posix to the top
Generating coverage data is split over two CI jobs which means the
service will need to merge results and reports wrong coverage data when
only 1 job is finished. This puts the native_posix board first making
sure we run on the first job and generate data in one place.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-01-05 23:00:50 -05:00
Paul Sokolovsky
100474d99d scripts: sanitycheck: If error happens accessing YAML data, go on
Extend exception handling to cover not just YAML loading, but any
error while accessing parsed data too. That may catch e.g. schema
mismatch errors (for folks who don't have pykwalify installed, which
is optional). So, now error will be logged, but processing of other
tests will continue.

For example, I had a local, uncommitted test which wasn't converted
per 23f81eeb42 and caused:

Traceback (most recent call last):
  File "./scripts/sanitycheck", line 2456, in <module>
    main()
  File "./scripts/sanitycheck", line 2324, in main
    options.outdir, options.coverage)
  File "./scripts/sanitycheck", line 1445, in __init__
    for name in parsed_data.tests.keys():
AttributeError: 'list' object has no attribute 'keys'

Signed-off-by: Paul Sokolovsky <paul.sokolovsky@linaro.org>
2018-01-05 09:48:11 -05:00
Anas Nashif
1df7e9a018 script: remove unused script: gcc-version.sh
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-01-03 19:03:41 -05:00
Anas Nashif
7256553955 scripts: python: cleanup script and fix PEP8 issues
Ran scripts through flake8 and fixed issues.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-01-03 19:03:41 -05:00
Anas Nashif
712d34513e sanitycheck: encode logs correctly
We were encoding logs twice.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-01-02 11:54:02 -05:00
Anas Nashif
43e4127df0 sanitycheck: remove unused map
This map is not being used anymore.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-01-02 11:54:02 -05:00
Anas Nashif
a8a1388ae4 sanitycheck: support ninja builder
Add an option to support building with ninja instead of make.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-01-02 11:54:02 -05:00
Anas Nashif
4f02888b58 sanitycheck: use global command line options variable
Now use glabal options instead of passing them around as function
arguments.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-01-02 11:54:02 -05:00
Anas Nashif
e10b651e2d sanitycheck: declare command line options global
We have been passing around options from one function to the next making
it very difficult to add a new option easily and requiring changes to
man function prototypes.
This declated the parsed command line options global and renames args to
options. args is being used elsewhere and this was confusing.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-01-02 11:54:02 -05:00
Anas Nashif
b0f3ae0e98 sanitycheck: harness support
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2017-12-28 20:24:29 -05:00
Anas Nashif
ab940163ee tests: add a new harness keyword to schema
This keyword would mean that a special harness is needed to run the
tests sucessfully. This can be as simple as a loopback wiring or a
complete hardware test setup for sensor and IO testing. It is free form
initially and would be changed to be an enum once we have more values in
place.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2017-12-28 20:24:29 -05:00
Anas Nashif
262e4a4b8b sanitycheck: work around linking posix port
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
2017-12-27 14:16:08 -05:00
Anas Nashif
cc16422538 sanitycheck: add native handler
Support running native applications in sanitycheck using a new handler.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2017-12-27 14:16:08 -05:00
Anas Nashif
2f4e170eb2 sanitycheck: support native build with *.exe extension
search for .exe file when running native applications in sanitycheck.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2017-12-27 14:16:08 -05:00
Anas Nashif
6b55598ad4 board: posix: add native_posix board definition
Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2017-12-27 14:16:08 -05:00
Carles Cufi
fdd94d3bd8 scripts: debug: J-Link script tweaks
* Add the interface speed option
* Select USB connection explicitly
* Use "J-Link" consistently

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2017-12-19 16:57:36 -05:00
Sebastian Bøe
8b2213f670 nrfjprog_py: Use RuntimeError instead of Exception
Be more specific with the exception type.

https://github.com/zephyrproject-rtos/zephyr/pull/5432#issuecomment-352492089

Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
2017-12-19 12:45:34 -05:00
Sebastian Bøe
5056ab24fb runner: nrfjprog: Improve error messages
When a debugger is already connected to the JLink debug adapter
nrfjprog.py would incorrectly detect that the snr is '0' and try to
flash a device with that snr.

Also, when there were no boards connected, nrfjprog.py would
incorrectly state that there were multiple boards connected.

This patch improves the error feedback so that a user can more easily
debug why he can't flash his device.

Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
2017-12-18 21:46:12 -05:00
Marti Bolivar
86e6e674a5 scripts: zephyr_flash_debug.py: swallow exceptions by default
For readability, swallow exceptions unless --verbose is given on the
command line. Add a printline to direct the user how to ensure that's
set in case more information out of the flash script is desired.

Signed-off-by: Marti Bolivar <marti@opensourcefoundries.com>
2017-12-18 21:45:48 -05:00
Marti Bolivar
209e2676b4 scripts: runner: add internal DEBUG to core
Add a debug-only global that prevents commands from running, and just
prints what would have been run.

Signed-off-by: Marti Bolivar <marti@opensourcefoundries.com>
2017-12-15 09:57:30 -05:00
Marti Bolivar
ec01e0f876 scripts: zephyr_flash_debug: add 'runners' command
This just prints the available runner classes.

Signed-off-by: Marti Bolivar <marti@opensourcefoundries.com>
2017-12-15 09:57:30 -05:00
Marti Bolivar
57d5717bc6 scripts: runner: let users copy/paste failed commands
Print a copy/pastable version of any command which fails, to help
debugging.

Signed-off-by: Marti Bolivar <marti@opensourcefoundries.com>
2017-12-15 09:57:30 -05:00
Marti Bolivar
ad8b006bf2 scripts: runner: pyocd: support pyocd-flashtool options
Allow the user to specify additional options for pyocd-flashtool. This
grants finer-grained control to individual applications, e.g. to
perform a chip erase instead of a sector erase when flashing.

Signed-off-by: Marti Bolivar <marti@opensourcefoundries.com>
2017-12-15 09:57:30 -05:00
Marti Bolivar
ccacaab49f scripts: runner: pyocd: support DT-based flashing
Enable DT support in the pyOCD flasher, and turn it on in the build
system.

Signed-off-by: Marti Bolivar <marti@opensourcefoundries.com>
2017-12-15 09:57:30 -05:00
Marti Bolivar
1645d372d2 scripts: runner: dfu-util: add --dfuse-modifiers option
This gives callers finer-grained control over the flash. For example,
giving --dfuse-modifiers=force:mass-erase allows a chip-erase rather
than just erasing the sectors which need to be overwritten with the image.

Signed-off-by: Marti Bolivar <marti@opensourcefoundries.com>
2017-12-15 09:57:30 -05:00
Marti Bolivar
3ebc2e46b2 scripts: runner: dfu-util: support DT-based flashing for DfuSe
Enable DT support in the dfu-util flasher when the target is a
DfuSe (DFU + ST extensions) device.

Untangling DfuSe-specific options (currently, the default is 'leave',
to immediately start running after the flashing is done) from the
actual address makes this cleaner, and sets up a subsequent patch to
let callers set DfuSe options.

It also lets us fix an unnecessary printline when flashing DfuSe
devices. There's no need to reset those, since the 'leave' modifier
starts execution immediately.

Signed-off-by: Marti Bolivar <marti@opensourcefoundries.com>
2017-12-15 09:57:30 -05:00
Marti Bolivar
3830bc51f2 scripts: runner: add infrastructure for DT-based flashing
Add the necessary infrastructure to the runner core to support
computing flash addresses based on the devicetree. Specifically, add:

- a new RunnerCaps capability, flash_addr, which lets runners declare
  when they support flashing to an arbitrary address

- a common --dt-flash option to all runner command line parsers which
  support this capability, which lets users request flash addresses to
  be computed from device tree

- a ZephyrBinaryRunner helper method, get_flash_address(), which is
  the common code needed to compute a flash address from device
  tree (or return a default value if non-DT based flashing is
  requested). This relies on the BuildConfiguration parser introduced
  in an earlier patch.

Subsequent patches will use this functionality in individual runners.

Signed-off-by: Marti Bolivar <marti@opensourcefoundries.com>
2017-12-15 09:57:30 -05:00
Marti Bolivar
7a3462de7a scripts: runner: use arguments, not environment vars
The various runners (flash/debug scripts) use environment variables to
take arguments. This is legacy behavior which is not desirable.

Use command line arguments instead.

Note: this leaves more general environment variables with publicly
      documented behavior in place for now, for compatibility, e.g.:

      ZEPHYR_FLASH_OVER_DFU, OPENSDA_FW, ESP_IDF_PATH, PYOCD_DAPARG

For example, when using dfu-util to flash arduino_101, instead of
setting DFUUTIL_PID, DFUUTIL_ALT, and DFUUTIL_IMG environment
variables, have the script invocation look like this:

  python3 .../zephyr_flash_debug.py dfu-util flash \
          [common arguments omitted] \
          --pid=8087:0aba --alt=x86_app \
          --img=.../build/zephyr/zephyr.bin

Make similar changes for other runners (openocd, etc.) and
targets (debug, debugserver).

To implement this in the scripts:

  - have the individual scripts/support/runner/some-runner.py files
    register their own command line arguments

  - teach them to construct instances from arguments, not the
    environment

  - have zephyr_flash_debug.py request runners to register command
    line argument parsers, and handle arguments

In the build system:

  - add a new board_runner_args() extension function that board.cmake
    files can use to add to the zephyr_flash_debug.py command line

  - adjust cmake/flash/CMakeLists.txt to invoke with arguments

  - add new helper include files for each runner (like
    boards/common/dfu-util.board.cmake, etc.), which add default
    options as needed and then add on overrides from
    board_runner_args() calls

  - update board.cmake files to use the new includes and extension

This implied some tweaking when using openocd to make the CMake string
escaping and unescaping work properly.

Signed-off-by: Marti Bolivar <marti@opensourcefoundries.com>
2017-12-15 09:57:30 -05:00
Marti Bolivar
f4ace33daf scripts: runner: add build-time .conf parser
This parses generated_dts_board.conf and auto.conf, given a build
directory.

Signed-off-by: Marti Bolivar <marti@opensourcefoundries.com>
2017-12-15 09:57:30 -05:00
Marti Bolivar
cd8d43b1c9 scripts: runner: generalize commands to "capabilities"
Some configuration options or device tree nodes affect the way that
runners ought to behave, but there's no good way for them to report
whether they can handle them.

One motivating example is CONFIG_FLASH_LOAD_OFFSET, as influenced by
the zephyr,code-partition chosen node in the DT for architectures
where CONFIG_HAS_FLASH_LOAD_OFFSET=y.

If CONFIG_FLASH_LOAD_OFFSET is nonzero, the 'flash' command ought to
place the kernel at that address offset from the device flash's start
address. Runners don't support this right now, which should be
fixed. However, we don't want to mandate support for this feature,
since not all targets need it.

We need to let runners declare what their capabilities are. Make it so
by adding a RunnerCaps class to the runner core. This currently just
states which commands a runner can handle, but can be generalized to
implement the above use case.

Signed-off-by: Marti Bolivar <marti@opensourcefoundries.com>
2017-12-15 09:57:30 -05:00
Marti Bolivar
e33ec242fd scripts: refactor flash/debug scripts to remove "shell"
The Python-based runners have replaced the old shell scripts. Refactor
the build system accordingly:

- FLASH_SCRIPT is now BOARD_FLASH_RUNNER
- DEBUG_SCRIPT is now BOARD_DEBUG_RUNNER

The values, rather than being the names of files, are now the names of
runners in scripts/support/runner. They are still short, descriptive
names like "openocd", "jlink", "em-starterkit", etc.

Adjust the zephyr_flash_debug.py call and runner internals
accordingly. Have each runner class report a name and the commands it
can handle. This lets us move some boilerplate from each do_run()
method into the common run() routine, and enables further improvements
in future patches.

The handles_command() method is temporary, and will be replaced by a
more general mechanism for describing runner capabilities in a
subsequent patch. The initial use case for extending this is to add
device tree awareness to the runners.

To try to avoid user confusion, abort the configuration if an
xxx_SCRIPT is defined.

Signed-off-by: Marti Bolivar <marti@opensourcefoundries.com>
2017-12-15 09:57:30 -05:00
Marti Bolivar
0fba4bf20c scripts: runner: refactor run() implementation
Have the subclasses implement a do_run() method instead, which run()
delegates to. This will make it possible to handle common
functionality in the superclass before runner-specific methods are
called. It is a prerequisite for tasks like loading the build time
configuration to add device tree awareness.

Signed-off-by: Marti Bolivar <marti@opensourcefoundries.com>
2017-12-15 09:57:30 -05:00
Marti Bolivar
5b0167ae79 scripts: runner: pyocd: fix stale comment
This isn't just a flasher anymore.

Signed-off-by: Marti Bolivar <marti@opensourcefoundries.com>
2017-12-15 09:57:30 -05:00
Marti Bolivar
36aee10bef scripts: zephyr_flash_debug: simple improvements
Delete an unused import to make flake8 happy. Update the module
docstring.

Signed-off-by: Marti Bolivar <marti@opensourcefoundries.com>
2017-12-15 09:57:30 -05:00
Marti Bolivar
607a817a4a scripts: extract_dts_includes: fix potential undefined read
Only read 'status' when it has been assigned in compress_nodes().

Signed-off-by: Marti Bolivar <marti@opensourcefoundries.com>
2017-12-15 09:57:30 -05:00
Erwan Gouriou
fa47748ca3 scripts: extract_dts_inculdes: treat 'parent: bus' attribute
When node has 'bus' as 'parent' attribute, change base label to
include parent address in label prefix.

Besides, generates a "_BUS_NAME" define which equals parent label.

Signed-Off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
2017-12-14 11:30:44 -06:00
Erwan Gouriou
568fc9d91f scripts: extract_dts_include: update to match new bindings path
Following move of dts bindings from common/yaml to bindings,
update yaml file search routine.

Signed-Off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
2017-12-14 11:30:44 -06:00