Fixes: #40389
This commit aligns the use of the following Zephyr build variables:
- CONF_FILE
- DTC_OVERLAY_FILE
- OVERLAY_CONFIG
- WEST_DIR
- ZEPHYR_BOARD_ALIASES
- ZEPHYR_EXTRA_MODULES
- ZEPHYR_MODULES
so that they can be set in same manner and with same order of
preference.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
When using CONFIG_NPCX_IMAGE_OUTPUT_HEX, set the hex_file
runners_yaml_props_target property, so that west flash will flash the
correct file.
Change the cmake flash rules to populate the hex_file value in the
runners.yaml file if either CONFIG_BUILD_OUTPUT_HEX is enabled or some
cmake file set the runners_yaml_props_target hex_file property.
Update the npcx9m6f_evb instructions now that the filename is implicit.
Signed-off-by: Jeremy Bettis <jbettis@google.com>
Currently there is no way to support running a board on multiple
emulation platforms nor to choose a desired emulation platform for the
simulation to be run on. This commit introduces a new
SUPPORTED_EMU_PLATFORMS list, which defines available emulation
platforms for a given board.
Fixes#12375.
Signed-off-by: Filip Kokosinski <fkokosinski@antmicro.com>
Zephyr thread awareness is available for openocd but boards don't
have debuggers configuration. This configure OpenOCD runner
automatically to complete configuration.
User still require enable CONFIG_DEBUG_THREAD_INFO=y to visualize
thread debug information.
Signed-off-by: Gerson Fernando Budke <nandojve@gmail.com>
Today, there is a build target is added for each runner: flash, debug,
debugserver, attach.
And those runners will have a dependency to Zephyr logical target that
is built before invoking `west <runner>`.
This design has some flaws, mainly that additional dependencies directly
on the target will not be built when running `west <runner>` directly.
That generator expressions cannot be used for the DEPENDS argument.
Instead, the build target `<runner>` will not have any dependencies, and
will raise a build error if a dependency is added to the target.
Due to how `add_dependencies()` work, this must be done as a build time
check, and not configure time check.
`west <runner>` will invoke a build before executing the runner, and
this way ensure the build target is up-to-date, which again removes the
need for a dedicated `west_<runner>_target`.
It also minimizes the risk of developer errors, as developers no longer
need to consider the need for adding additional dependencies.
If a custom target is part of the default `all` build, then it's ensured
to be up-to-date.
Fixes: Issue reported on slack.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
When CONFIG_MCUBOOT_SIGNATURE_KEY_FILE is set and the build system
successfully creates signed outputs, use them as the default artifacts
to flash.
This flashes the non-confirmed variant regardless of
CONFIG_MCUBOOT_GENERATE_CONFIRMED_IMAGE, allowing POST code in the
application to confirm the image or not. Since this is used primarily
from development, I'd hesitate to assume the confirmed image is good.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
This commit message is a bit of a novel mostly:
- because the issues involved are longstanding
- as evidence this is not a capricious refactoring
The runners.core.RunnerConfig Python class holds common configuration
values used by multiple runners, such as the location of the build
outputs and board directory.
The runners code, first written in 2017-ish, replaced various shell
scripts that got this information from the environment. Avoiding
environment variables was a requirement, however. It's ghastly to set
environment variables for a single command invocation on Windows, and
the whole thing was part of a larger push to make Zephyr development
on Windows better.
I had a hammer (the argparse module). Finding a replacement naturally
looked like a nail, so the information that ends up in RunnerConfig
got shunted from the build system to Python in the form of 'west
flash' / 'west debug' command line options like '--board-dir',
'--elf-file', etc.
I initially stored the options and their values in the CMake cache.
This was chosen in hopes the build system maintainer would like
the strategy (which worked).
I knew the command line arguments approach was a bit hacky (this
wasn't a nail), but I also honestly didn't have a better idea at the
time.
It did indeed cause issues:
- users don't know that just because they specify --bin-file on the
command line doesn't mean that their runner respects the option, and
have gotten confused trying to flash alternate files, usually for
chain-loading by MCUboot (for example, see #15961)
- common options weren't possible to pass via board.cmake files
(#22563, fixed partly via introduction of runners.yaml and the west
flash/debug commands no longer relying on the cache)
- it is confusing that "west flash --help" prints information about
openocd related options even when the user's board has no openocd
support. The same could be said about gdb in potential future use
cases where debugging occurs via some other tool.
Over time, they've caused enough users enough problems that
improvements are a priority.
To work towards this, put these values into runners.yaml using a new
'config: ...' key/value instead of command line options.
For example, instead of this in the generated runners.yaml file:
args:
common:
- --hex-file=.../zephyr.hex
we now have:
config:
hex_file: zephyr.hex
and similarly for other values.
In Python, we still support the command line options, but they are not
generated by the build system for any in-tree boards. Further work is
needed to deprecate the confusing ones (like --hex-file) and move the
runner-specific host tool related options (like --openocd) to the
runners that need them.
Individual board.cmake files should now influence these values by
overriding the relevant target properties of the
runners_yaml_props_target.
For example, instead of:
board_runner_args(foo "--hex-file=bar.hex")
Do this:
set_target_properties(runners_yaml_props_target PROPERTIES
hex_file bar.hex)
This change additionally allows us to stitch cmake/mcuboot.cmake and
the runners together easily by having mcuboot.cmake override the
properties that set the hex or bin file to flash. (The command line
arguments are still supported as-is.)
Combined with 98e0c95d91ae16f14e4997fb64ccdf0956595712 ("build:
auto-generate signed mcuboot binaries"), this will allow users to
build and flash images to be chain loaded by mcuboot in a way that
avoids calling 'west sign' and passing 'west flash' its output files
entirely.
While we are here, rename runner_yml_write to runners_yaml_append().
This function doesn't actually write anything, and we're here
refactoring this file anyway, so we might as well improve the
situation while we're at it.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Rename it to "runners_yaml_props_target", because the file name is
runners.yaml, not runner.yml.
Move it to the boilerplate definition so that we can modify it before
cmake/flash/CMakeLists.txt gets run.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Commit c1f80db44a removed various
now-unused CMake cache variables, but it missed the ones that begin
with ZEPHYR_RUNNER_ARGS. Delete those now.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Resolve a TODO item that followed the introduction of runners.yaml
files by removing the now-unused CMake cache variables that they
obsolete.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
This commit creates a list of a phony targets for each runner, that is:
`west_flash_depends`, `west_debug_depends`, and so on.
Those targets has identical dependencies as CMake runner target.
flash, debug, debugserver, attach targets.
As example `ninja flash` correctly ensures dependencies are taken into
consideration before calling `west flash`.
Unfortunately, calling `west flash` directly would not re-run the flash
dependencies, cause `west flash` would only build the default CMake
target.
Now, `west flash` calls the phony `west_flash_depends` target, ensuring
all deps are up-to-date before flashing (unless --skip-rebuild is given)
The same is true for the other mentioned runners.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
This commit renames `FLASH_DEPS` to `RUNNERS_DEPS`.
The current name `FLASH_DEPS` is misleading in the sense that this
depency is added to all runner targets, flash, debug, debugserver,
attach, and not only the flash runners.
Therefore this is now named `RUNNERS_DEPS` instead.
Similar, zephyr_property_target now contains the property
FLASH_DEPENDENCIES, DEBUG_DEPENDENCIES, and so on, so that additional
dependencies can be added for each runner.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Using zephyr_target_property::FLASH_DEPENDENCIES to fetch additional
dependencies to the flash operation.
The properties are fetched using a generator expression which allows
users of Zephyr to add dependencies both before and after the flash
target has been defined.
Dependencies can be other targets that must be build / custom commands
which must be executed before the flash operation. Or it can be targets
which must be built.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Signed-off-by: Haakon Oeye Amundsen <haakon.amundsen@nordicsemi.no>
If a runner had no args, the format of the generated runners.yaml
was invalid due to missing indentation.
This commit fixes this issue by adding the required indentation.
This fixes issue #24215
Signed-off-by: Håkon Øye Amundsen <haakon.amundsen@nordicsemi.no>
This allows users to modify the contents of the data being
written after the cmake/flash/CMakeLists.txt has been
executed.
Signed-off-by: Håkon Øye Amundsen <haakon.amundsen@nordicsemi.no>
With the introduction of ZephyrConfig.cmake all parts of CMake code
should rely on the CMake ZEPHYR_BASE variable instead of the environment
setting.
This ensures that after the first CMake invocation, then all subsequent
invocation in same build folder will use same zephyr base.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Using zephyr_target_property::FLASH_DEPENDENCIES to fetch additional
dependencies to the flash operation.
The properties are fetched using a generator expression which allows
users of Zephyr to add dependencies both before and after the flash
target has been defined.
Dependencies can be other targets that must be build / custom commands
which must be executed before the flash operation. Or it can be targets
which must be built.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Signed-off-by: Haakon Oeye Amundsen <haakon.amundsen@nordicsemi.no>
There are two problems with the way runners.yaml is being created:
1. The dictionary which contains the arguments for each runner is
using the runner's name converted to a C identifier instead of the
runner's name itself. That causes west flash to fail when the two
are different, e.g. for 'misc-flasher' (runner name), which is
different than 'misc_flasher' (runner name as C identifier)
2. We need to make sure that the dictionary key maps to an empty list
if there are no arguments, which normally doesn't happen since the
runner usually at least takes the path of the file to flash or debug.
It does happen in the case of misc-flasher, though, since the whole
point of that runner is that it's an escape hatch for people with
out of tree scripts that nevertheless want 'west flash' integration
for things like sanitycheck device testing.
3. A copy/paste error is setting the debug runner to the flash runner.
Fix them all.
Fixes: #23004
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
The YAML contents mirror the values in the ZEPHYR_RUNNER_CONFIG
variables, but they are phrased in terms of command line arguments.
This makes it possible for Python to intermix them with
runner-specific arguments, which is a step towards being able to set
arguments like --bin-file via board_set_runner_args(). The next step
is to handle them in Python too.
Move the RUNNERS_VERBOSE setting closer to its use while at it, to
preserve readability.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Of the 1.7s or so that CMake spends in Configure-time there is 0.1s
spent to determine whether west can flash or not. This is significant
compared to the usefulness of flash-check.
AFAICT when installing west there is no longer a two-step procedure
where first a bootstrapper that can't flash is installed, and then a
proper west is installed, so the risk of having installed the
bootstrapper is no longer something we should spend 100ms testing for.
This patch removes the check, thereby speeding up CMake by 100ms.
Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
Update the files which contain no license information with the
'Apache-2.0' SPDX license identifier. Many source files in the tree are
missing licensing information, which makes it harder for compliance
tools to determine the correct license.
By default all files without license information are under the default
license of Zephyr, which is Apache version 2.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Since we install west from pip and it becomes an executable, use it
directly assuming it's in the user's PATH.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Signed-off-by: Torsten Rasmussen <torsten.rasmussen@nordicsemi.no>
This patch adds a dependency from the 'flash' target to
the 'mergehex' target IF files to be merged are configured.
Signed-off-by: Håkon Øye Amundsen <haakon.amundsen@nordicsemi.no>
Allow user to add externally built hex files to the cmake property
HEX_FILES_TO_MERGE. The hex files in this list will be merged
with the hex file generated when building the zephyr application.
This allows users to leverage the application configuration
available in Kconfig and CMake to help decide what hex file
should be merged with the zephyr application.
Signed-off-by: Håkon Øye Amundsen <haakon.amundsen@nordicsemi.no>
Adds a build system target for the west attach command, which starts a
debugging session without programming the flash.
Signed-off-by: Maureen Helm <maureen.helm@nxp.com>
Update to the latest west. This includes a new 'attach' command. There
are also multi-repo commands, but those won't get exposed to the user
unless they install Zephyr using "west init" + "west fetch" (and not,
say, "git clone").
Replace the launchers; they now detect whether zephyr is part of a
multi-repo installation, and run the west code in its own repository
if that is the case.
This also requires an update to:
- the flash/debug CMakeLists.txt, as the new west package is no longer
executable as a module and must have its main script run by the
interpreter instead.
- the documentation, to reflect a rename and with a hack to fix
the automodule directive in flash-debug.rst for now
Signed-off-by: Marti Bolivar <marti@foundries.io>
Continue better integration of the runner subpackage into west by
moving the common runner configuration options into the command
core. This allows commands like "west flash -h" to display help for
common overrides like --kernel-hex.
Adjustments needed to make this happen are:
- Change the build system to separate common configuration values from
runner-specific options and arguments
- Prepare the runner core by defining a new RunnerConfig class that
represents the common configuration, and accepting that from a new
create() method, which replaces create_from_args().
- Convert all concrete runner classes to use the new style of
argument parsing and initialization.
- Group the command options appropriately for help output readability
There's still a bit of tool-specific stuff in the common
configuration (gdb and openocd configuration in particular); a more
generic way to deal with that will be necessary to better support
things like non-GDB debuggers, but that's out of scope of this patch.
All the runner-specific options are still in the runner packge, which
currently prevents them from being included in "west flash -h" etc.
Fixing that is also out of scope of this patch.
This has the ancillary benefit of getting rid of the legacy 'debug'
argument to ZephyrBinaryRunner, which is no longer appropriate since
verbose debug logging is handled by log.py in west.
Signed-off-by: Marti Bolivar <marti@opensourcefoundries.com>
Since, by design, the west 'flash', 'debug', and 'debugserver'
commands operate the same way as the Makefile targets when given no
arguments, so just delegate to west from cmake/flash/CMakeLists.txt
instead of invoking zephyr_flash_debug.py by hand.
Removing the old script is the first step towards being able to clean
up the command line argument handling in the runner package, which was
always kind of a hack and can be improved now that runner is part of a
larger tool.
Signed-off-by: Marti Bolivar <marti@opensourcefoundries.com>
This is a stepping-stone to adding runner functionality into west
itself.
Since all of the runner tools assume a Zephyr build directory layout,
this doesn't put anything generic into a Zephyr-specific tool.
Make minimal adjustments to zephyr_flash_debug.py to keep existing
build system targets working unmodified.
Signed-off-by: Marti Bolivar <marti@opensourcefoundries.com>
Persist all the important information needed by the runner package to
the CMake cache. This serves as a parseable record for various tools
which need to understand how to run the binary.
In particular, it will be used by the west tool, which will be
introduced in subsequent patches.
Signed-off-by: Marti Bolivar <marti@opensourcefoundries.com>
Fixes: 5958
Ninja by default will suppress command output and input. This is a
problem for the 'flash', 'debug', etc. targets, which may need to
interact with the user (and which display interactive output like
progress bars).
To fix this, add the USES_TERMINAL option to their target
definitions. This puts them in the console pool on Ninja, allowing the
commands to interact with the console.
References:
https://ninja-build.org/manual.html#_the_literal_console_literal_poolhttps://cmake.org/cmake/help/v3.8/command/add_custom_target.html
Signed-off-by: Marti Bolivar <marti@opensourcefoundries.com>
Instead of accessing the environment variable ZEPHYR_BASE every time we
require accessing the source code root, use an intermediate variable
that has OS path separators correctly set to '/' to avoid issues on
Windows.
Note: This removes the ZEPHYR_SOURCE_DIR CMake variable. External
applications using that will need to change to use the new ZEPHYR_BASE
variable.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
The various runners (flash/debug scripts) use environment variables to
take arguments. This is legacy behavior which is not desirable.
Use command line arguments instead.
Note: this leaves more general environment variables with publicly
documented behavior in place for now, for compatibility, e.g.:
ZEPHYR_FLASH_OVER_DFU, OPENSDA_FW, ESP_IDF_PATH, PYOCD_DAPARG
For example, when using dfu-util to flash arduino_101, instead of
setting DFUUTIL_PID, DFUUTIL_ALT, and DFUUTIL_IMG environment
variables, have the script invocation look like this:
python3 .../zephyr_flash_debug.py dfu-util flash \
[common arguments omitted] \
--pid=8087:0aba --alt=x86_app \
--img=.../build/zephyr/zephyr.bin
Make similar changes for other runners (openocd, etc.) and
targets (debug, debugserver).
To implement this in the scripts:
- have the individual scripts/support/runner/some-runner.py files
register their own command line arguments
- teach them to construct instances from arguments, not the
environment
- have zephyr_flash_debug.py request runners to register command
line argument parsers, and handle arguments
In the build system:
- add a new board_runner_args() extension function that board.cmake
files can use to add to the zephyr_flash_debug.py command line
- adjust cmake/flash/CMakeLists.txt to invoke with arguments
- add new helper include files for each runner (like
boards/common/dfu-util.board.cmake, etc.), which add default
options as needed and then add on overrides from
board_runner_args() calls
- update board.cmake files to use the new includes and extension
This implied some tweaking when using openocd to make the CMake string
escaping and unescaping work properly.
Signed-off-by: Marti Bolivar <marti@opensourcefoundries.com>
The Python-based runners have replaced the old shell scripts. Refactor
the build system accordingly:
- FLASH_SCRIPT is now BOARD_FLASH_RUNNER
- DEBUG_SCRIPT is now BOARD_DEBUG_RUNNER
The values, rather than being the names of files, are now the names of
runners in scripts/support/runner. They are still short, descriptive
names like "openocd", "jlink", "em-starterkit", etc.
Adjust the zephyr_flash_debug.py call and runner internals
accordingly. Have each runner class report a name and the commands it
can handle. This lets us move some boilerplate from each do_run()
method into the common run() routine, and enables further improvements
in future patches.
The handles_command() method is temporary, and will be replaced by a
more general mechanism for describing runner capabilities in a
subsequent patch. The initial use case for extending this is to add
device tree awareness to the runners.
To try to avoid user confusion, abort the configuration if an
xxx_SCRIPT is defined.
Signed-off-by: Marti Bolivar <marti@opensourcefoundries.com>
We append a path to the FLASH_SCRIPT/DEBUG_SCRIPT that was bogus and not
really needed by zephyr_flash_debug.py. So lets remove it since its
just confusing.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Some boards define multiple configuration which all are maintained under
the same board directory. The flasher was looking for an openocd.cfg
based on the board name, which can't be found for such boards.
Use the variable BOARD_DIR provided by cmake instead of trying to
assemble the board directory location on our own.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
The old way of constructing FLASH_SCRIPT_ENV_VARS was corrupting the
values that were passed to the flasher. This new method is the
standard way of creating a dictionary/hashmap in CMake and does not
suffer from the same problem.
This fixes
https://github.com/zephyrproject-rtos/zephyr/issues/4844#event-1334599401
Signed-off-by: Sebastian Boe <sebastian.boe@nordicsemi.no>
Introducing CMake is an important step in a larger effort to make
Zephyr easy to use for application developers working on different
platforms with different development environment needs.
Simplified, this change retains Kconfig as-is, and replaces all
Makefiles with CMakeLists.txt. The DSL-like Make language that KBuild
offers is replaced by a set of CMake extentions. These extentions have
either provided simple one-to-one translations of KBuild features or
introduced new concepts that replace KBuild concepts.
This is a breaking change for existing test infrastructure and build
scripts that are maintained out-of-tree. But for FW itself, no porting
should be necessary.
For users that just want to continue their work with minimal
disruption the following should suffice:
Install CMake 3.8.2+
Port any out-of-tree Makefiles to CMake.
Learn the absolute minimum about the new command line interface:
$ cd samples/hello_world
$ mkdir build && cd build
$ cmake -DBOARD=nrf52_pca10040 ..
$ cd build
$ make
PR: zephyrproject-rtos#4692
docs: http://docs.zephyrproject.org/getting_started/getting_started.html
Signed-off-by: Sebastian Boe <sebastian.boe@nordicsemi.no>
