Remove the boards and shields lists from the 'usage' target output.
That might have been readable at some point long ago in Zephyr's
history, when only a few boards were available, but right now it's
obscuring the high level targets we really want 'usage' to print.
Instead, add 'boards' and 'shields' targets which the user can run to
get those lists, and reference them from the 'usage' output. This
makes 'usage' squintable again. We use the new list_boards.py script
from the 'boards' target.
Reference the 'help' target from 'usage' as well, and drop the
recommendation that people run '--target help' from the 'west build
--help' output for the 'west build --target' option. The canonical
place to look is 'usage' now.
Use the new list_boards.py code from 'west boards' as well, which
allows us to add the board's directory as a format string key, in
addition to its name and architecture.
Keep west-completion.bash up to date. While doing that, I noticed that
a bunch of references to this file refer to a stale location, so fix
those too.
Finally, the 'usage' output is what we print for a failed board or
shield lookup, so that needs to be updated also. Handle that by
invoking boards.cmake and a new shields.cmake in CMake script mode to
print the relevant output.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
rimage dropped its "-m" parameter and switched over to using "-c"
for a configuration file, including a target name.
Add support for extended manifest for all cAVS versions.
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
It's fine to flash an image with UICR data as long as the user has
given any of the --erase, --recover, or --force options. Silence the
error correctly.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
As of today we check for gdb presence in 'do_create' method and
raise an exception in case of gdb is missing. That makes nsim
runner unusable without gdb even for the commands which don't use
it (like 'flash' command).
Fix that.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Runner implementations are only allowed to unconditionally import
modules in the python standard library. They are not allowed to import
anything that comes from pip or other third party sources unless they
catch the ImportError and gracefully do nothing.
Fix the imports in the mdb runner accordingly, sorting the imports
into sections to make it clearer what's stdlib, what's runners, and
what's third party while we're here.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Rework the runner to improve various issues.
Every board.cmake file for an nRF SoC target is repeating boilerplate
needed for the nrfjprog runner's --nrf-family argument. The
information we need to decide the --nrf-family is already available in
Kconfig, so just get it from there instead. Keep the --nrf-family
argument around for compatibility, though.
This cuts boilerplate burden for board maintainers.
We also need to revisit how this runner handles recovery to fix it
in nRF53 and keep things consistent everywhere else.
To cleanly handle additional readback protection features in nRF53,
add a --recover option that does an 'nrfjprog --recover' before
flashing. Keep the behavior consistent across SoCs by supporting it on
those too. Because this is expected to be a bit tricky for users to
understand, check if a --recover is needed if the 'nrfjprog --program'
fails because of protection, and tell the user how to fix it.
Finally, instead of performing a separate 'nrfjprog --eraseall', just
give --chiperase to 'nrfjprog --program' process's arguments instead
of --sectorerase. This is cleaner, resulting in fewer subprocesses and
avoiding an extra chip reset.
Having a separate 'west flash --recover' option doubles the number of
test cases if we want to keep exhaustively enumerating them. That
doesn't feel worthwhile, so update the test cases by picking a
representative subset of the possibilities. Each test now has enough
state that it's worth wrapping it up in a named tuple for readability.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
The current bossac runner have multiple entries for flash offset
parameter. Remove offset parameter from command line and rework
all infraestructure to get offset from device tree. Add proper
verification routines to validate configurations on board and
device tree entries to fix SDK compatibility checks.
Fixes#29312.
Signed-off-by: Gerson Fernando Budke <nandojve@gmail.com>
Add is_extended_samba_protocol method to verify if board is using SAM-BA
extended protocol (Arduino/Adafruit UF2). It allows enforce serial line
speed to 1200 baud automatically. This avoid add entries on board.cmake
file.
Signed-off-by: Gerson Fernando Budke <nandojve@gmail.com>
mdb binary starts several subproceses and one of them is cld process.
In runners/mdb.py we record process id of cld on each mdb launch
to terminate simulator correctly later. However we can finish test
and terminate mdb before the cld process was found (so cld won't
be terminated correctly by sanitycheck infrastructure). It may happen
if we launch mdb on fast host machine.
That leads to several issues. First of all we get ugly error in
sanitycheck output:
------------------------>8--------------------------------
FileNotFoundError: [Errno 2] No such file or directory: '/xxxx/mdb.pid'
------------------------>8--------------------------------
Secondly (and it's more important) we terminate simulator incorrectly.
We terminate mdb leaving cld process alive, running and consuming one
cpu core permanently (until we kill it manually)
So, let's increase granularity of lookups and don't wait extra 0.5
seconds before the first lookup.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
I keep seeing people miss the --context option in the "west flash
--help" output.
This option is very important for understanding the runner-specific
options and state, and missing it means people get confused about what
west flash, debug, etc. can do and are doing.
Try to avoid this problem by adding a big fat banner about the
omission of runner-specific options in the main help output, and
provide more hints about how to use --context.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Record MDB's CLD process pid to file so this process can be
terminated by sanitycheck infrastructure.
Update mdb runners test to be able to handle changes.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
mdb runner is quite special as it can be used to run Zephyr on
both simulator (nSIM) and real hardware.
However it is really misleading as same command (west flash)
will run Zephyr in simulation for one board and try to run it
on HW for another board. Things are getting worse for boards
supporting both runs in simulation and on real hardware.
Let's split mdb runner for mdb-hw (for runs on HW) and mdb-nsim
(for runs in simulation) runners.
Signed-off-by: Eugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
The current stty command uses a hard code value of 1200. This is not
compliant with SAM-BA specs and may create compatibility problems. Add
an optional speed argument with 115200 as default value following SAM-BA
specifications. All boards that needs a different speed should define
board_runner_args(bossac "--speed=<value>") with value as required
speed.
Signed-off-by: Gerson Fernando Budke <nandojve@gmail.com>
It's not clear why this error is here. The "formats" array seems to
be limited to "bin" and "hex" only, but every signing tool is going to
have its own idea of what format to emit and what ingredients need to
be used to do that.
In particular, rimage (used for the Intel Audio DSPs) doesn't use nor
generate zephyr.bin (it's very large), so it trips over this failure.
Just present the possibly-empty list of output formats to the Signer
object and let it make the decision about whether an empty formats
list is an error.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
The west build command has historically tried not to rm -rf
directories that don't look like zephyr build directories. The way it
does this is by checking for the presence of a CMake cache
with a Zephyr-specific variable (ZEPHYR_TOOLCHAIN_VARIANT) in it.
The problem with this approach is that if the build system fails
before this cache variable is set, the directory doesn't look like a
zephyr build directory, and therefore west build won't make it
pristine even with --pristine=always, even though build directories
resulting from failed runs like that are almost certainly
irrecoverably broken and need to be made pristine before anything will
work.
This leads to users having to rm -rf their directories manually, which
is not so nice.
To avoid this from happening, just check for ZEPHYR_BASE, which is
set early on in ZephyrConfig.cmake in 'modern' zephyr build systems.
Keep the ZEPHYR_TOOLCHAIN_VARIANT check in place for compatibility.
We could consider being less selective and just using shutil.rmtree()
whenever we have --pristine=always, but that would be a bigger
behavioral change than I'm comfortable doing without a good reason.
Fixes: #28876
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
BOSSA takes the offset within flash memory, not the absolute address.
This doesn't matter on most boards as the flash starts at zero but
does matter on the Due as the flash starts at +512 KiB.
Signed-off-by: Michael Hope <mlhx@google.com>
Inspect the hex file with intelhex, and fail if the hex file has any
contents in the UICR area(s).
family == 'NRF52' still always does --sectoranduicrerase, but this
option is not available on other families.
Add --force command line option to proceed with flashing instead of
failing.
Signed-off-by: Øyvind Rønningstad <oyvind.ronningstad@nordicsemi.no>
Commit 3124c02 ("cmake: flash/debug: refactor runner configuration")
changed the way the hex_file and elf_file inputs in the RunnerConfig
object are created. In particular, they are now host-style paths.
This breaks flashing with openocd on Windows, which doesn't handle that
properly. Fix that by "casting" the internal hex_file and elf_file
attributes to POSIX paths.
Fixes: #28138
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
I've repeatedly seen that people are not aware of the existence of
this configuration option.
I've been using build.pristine=auto daily for years and never had any
problems. I've also asked around on Slack a couple of times over
various points to see if anybody objects to making this change. Nobody
has, so let's just turn it on by default.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
This commit message is a bit of a novel mostly:
- because the issues involved are longstanding
- as evidence this is not a capricious refactoring
The runners.core.RunnerConfig Python class holds common configuration
values used by multiple runners, such as the location of the build
outputs and board directory.
The runners code, first written in 2017-ish, replaced various shell
scripts that got this information from the environment. Avoiding
environment variables was a requirement, however. It's ghastly to set
environment variables for a single command invocation on Windows, and
the whole thing was part of a larger push to make Zephyr development
on Windows better.
I had a hammer (the argparse module). Finding a replacement naturally
looked like a nail, so the information that ends up in RunnerConfig
got shunted from the build system to Python in the form of 'west
flash' / 'west debug' command line options like '--board-dir',
'--elf-file', etc.
I initially stored the options and their values in the CMake cache.
This was chosen in hopes the build system maintainer would like
the strategy (which worked).
I knew the command line arguments approach was a bit hacky (this
wasn't a nail), but I also honestly didn't have a better idea at the
time.
It did indeed cause issues:
- users don't know that just because they specify --bin-file on the
command line doesn't mean that their runner respects the option, and
have gotten confused trying to flash alternate files, usually for
chain-loading by MCUboot (for example, see #15961)
- common options weren't possible to pass via board.cmake files
(#22563, fixed partly via introduction of runners.yaml and the west
flash/debug commands no longer relying on the cache)
- it is confusing that "west flash --help" prints information about
openocd related options even when the user's board has no openocd
support. The same could be said about gdb in potential future use
cases where debugging occurs via some other tool.
Over time, they've caused enough users enough problems that
improvements are a priority.
To work towards this, put these values into runners.yaml using a new
'config: ...' key/value instead of command line options.
For example, instead of this in the generated runners.yaml file:
args:
common:
- --hex-file=.../zephyr.hex
we now have:
config:
hex_file: zephyr.hex
and similarly for other values.
In Python, we still support the command line options, but they are not
generated by the build system for any in-tree boards. Further work is
needed to deprecate the confusing ones (like --hex-file) and move the
runner-specific host tool related options (like --openocd) to the
runners that need them.
Individual board.cmake files should now influence these values by
overriding the relevant target properties of the
runners_yaml_props_target.
For example, instead of:
board_runner_args(foo "--hex-file=bar.hex")
Do this:
set_target_properties(runners_yaml_props_target PROPERTIES
hex_file bar.hex)
This change additionally allows us to stitch cmake/mcuboot.cmake and
the runners together easily by having mcuboot.cmake override the
properties that set the hex or bin file to flash. (The command line
arguments are still supported as-is.)
Combined with 98e0c95d91ae16f14e4997fb64ccdf0956595712 ("build:
auto-generate signed mcuboot binaries"), this will allow users to
build and flash images to be chain loaded by mcuboot in a way that
avoids calling 'west sign' and passing 'west flash' its output files
entirely.
While we are here, rename runner_yml_write to runners_yaml_append().
This function doesn't actually write anything, and we're here
refactoring this file anyway, so we might as well improve the
situation while we're at it.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
The 'runner_config' variable name is particularly misleading because
there is a class called RunnerConfig, and that variable does not
contain one.
Rename it to 'runners_yaml' since it contains the parsed contents of
the runners.yaml file. Rename the variable that refers to the path
itself to 'runners_yaml_path'. No functional changes expected.
This is prep work for redoing how actual RunnerConfig objects get
made.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Before venturing off into refactoring land, let's make sure we have a
type checker running on the main APIs used by the various runners so
we don't miss anything obvious.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Use mypy to type check the runners package.
The test procedure is now annoying enough to replicate locally that
I'm going to wrap it in a script. Do this for both UNIX and Windows
environments by writing that script in Python.
Keep the GitHub workflow up to date so we now get mypy results in CI.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
This is needed for sanitycheck hardware maps which take the serial
number directly from USB metadata.
Signed-off-by: Øyvind Rønningstad <oyvind.ronningstad@nordicsemi.no>
When MCUBoot is built using the swap-move strategy, the secondary slot
can be one sector smaller than the primary slot, because the primary
slot's last "useful" image sector must be "reserved" for the move
operation; this impacts the generation of padded images, so when
signing an image, the proper size to use is the secondary slot's,
unless a secondary image is not defined which is the case when using
single image DFU.
Signed-off-by: Fabio Utzig <fabio.utzig@nordicsemi.no>
Add support for the J-Link Commander "-NoGui 1" command line parameter
in the West J-Link runner.
This command line parameter suppresses GUI dialogs (except for license
dialogs) in J-Link Commander starting from v6.80.
Signed-off-by: Henrik Brix Andersen <hebad@vestas.com>
If a runner supports the --erase option, make it so that the user can
add --no-erase to the command line to explicitly disable it.
Add a diagnostic at info() level whenever this option is requested.
The intent of this commit is to make it more obvious when a mass erase
was requested, especially in situations (like MCUboot with
CONFIG_MCUBOOT_TRY_MASS_ERASE=y) where this option may be turned on by
default.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Fixes: #27375
This is a cleanup of the Zephyr CMake package export.
The code has been simplified so that the export now happens through a
CMake script. This avoids several generated CMake build files compared
to previous export mode, and thus removes the need for a CMake pristine
script.
A benefit of this cleanup is that it also fixes#27375.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Avoid use of the cached_runner_config() helper, which we have a TODO
item to get rid of. Adjust the output and do some other minor
improvements.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Rolling yet another parser turns out to be the best way to let west
extensions respond to Kconfig values. See source code comments in the
patch for details.
The sanitylib library has some similar functionality but it isn't
exactly the same, and it gets strings wrong. For example, that parser
can't handle this option:
CONFIG_FOO="he said \"no\" to me"
This one can, and it has a couple of other features we'll find useful
for west extensions eventually besides.
(Not to mention that sanitylib also rolled its own CMake cache parser,
which also exists in west_commands.)
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
