Commit graph

2587 commits

Author SHA1 Message Date
Michael Hope 447cad97df scripts: bossac: automatically pull the flash offset from the config
BOSSAC uses a bootloader, so pull the flash address from DeviceTree or
the config and use if the version of BOSSAC supports offsets.

Signed-off-by: Michael Hope <mlhx@google.com>
2020-06-28 15:48:04 +02:00
Michael Hope 54c2744790 scripts: bossac: add support for bossac 1.9.1
BOSSA 1.7 has built-in bootloader offset handling while 1.9.1
requires the user to supply the offset.  Add support for both by
sniffing the help output and warn the user if the flags needs adding.

Related to #22062

Signed-off-by: Michael Hope <mlhx@google.com>
2020-06-28 15:47:59 +02:00
Wentong Wu 6fae53c044 sanitycheck: handle timeouts correctly with qemu
In some cases we were not handling return code correctly and evaluating
timeouts as a pass. Report failure correctly.

Fixes #26065

Signed-off-by: Wentong Wu <wentong.wu@intel.com>
2020-06-25 11:05:29 -04:00
Martí Bolívar bd491fbb97 runners: promote --erase to common option
Move the --erase option to core.py from nrfjprog.py and jlink.py,
where it is currently supported.

Using the RunnerCaps option enforcement mechanism introduced earlier,
enforce that it will only be given to runners that support it.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-06-25 12:14:25 +02:00
Martí Bolívar f8e8e9229d runners: enforce RunnerCaps via create() indirection
Require all implementations to provide a do_create(), a new
ZephyrBinaryRunner abstract class method, and make create() itself
concrete.

This allows us to enforce common conventions related to individual
runner capabilities as each runner provides to the core via
RunnerCaps.

For now, just enforce that:

- common options related to capabilities are always added, so runners
  can't reuse them for different ends

- common options provided for runners which don't support them emit
  sensible error messages that should be easy to diagnose and support

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-06-25 12:14:25 +02:00
Martí Bolívar 7b93bd54d4 scripts: runners: clean up super() calls
We can just call super() instead of super(MyClassName, self). The
original extra verbosity is likely due to old habits of mine from
Python 2 which are no longer necessary, but got copy/pasted around.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-06-25 12:14:25 +02:00
Andrew Boie 45979dafb4 linker: add iterable section macros
For iterable areas defined with Z_STRUCT_SECTION_ITERABLE(),
the corresponding output section in the linker script is just
boilerplate. Add macros to make these definitions simpler.

Unfortunately, we have a fair number of iterable sections not
defined with Z_STRUCT_SECTION_ITERABLE(), this patch does not
address this.

The output sections are all named <struct name>_area, update
sanitylib.py with this.

sys_sem with no userspace, and k_lifo/k_fifo are special cases
where different data types that are all equivalent need to be
put in the same iterable area. Add
Z_STRUCT_SECTION_ITERABLE_ALTERNATE() for this special case.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-06-24 17:09:22 -04:00
Daniel Leung 2959ed35c1 west: runners/misc-flasher: fix passing extra args to script
The argparse module, by default, complains about non-defined
arguments. This prevents passing arguments prefixed with '-'
or '--' to the target script (e.g. calling another Python
script using argparse). This changes the misc-flasher script
so that any arguments not recognized by west will be passed
to the target script.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-06-24 08:51:53 -04:00
Grzegorz Szymaszek 0a3e3958df west: runners: stm32flash: convert start_addr to string (concatenation)
The start_addr attribute of Stm32flashBinaryRunner is an integer. It
must be converted to a string before being concatenated with a colon and
the (already converted to a string) size to erase or write.

Signed-off-by: Grzegorz Szymaszek <gszymaszek@short.pl>
2020-06-19 15:18:08 +02:00
Jordan Yates 9c39a93014 west: runners: jlink: Disable DAP after flashing
Disables the Debug-Access-Port of the microcontroller after flashing.
If not disabled, the DAP consumes ~1.6mA until the debugger disables it
or a hard power cycle is applied.

The DAP is typically automatically disabled after flashing, but if other
instances of JLink software are running (not connected), it will not be.

The added command resets the value of the CTRL/STAT register of the DAP.
This clears the CSYSPWRUPREQ and CDBGPRWUPREQ bits, leaving the debug
hardware free to power off the appropriate hardware. In no way does it
hinder the ability to later connect to the device for debugging.

This resolves the jlink portion of #26139

Signed-off-by: Jordan Yates <jordan.yates@data61.csiro.au>
2020-06-17 19:26:54 +02:00
Carles Cufi 88f25df9ac Bluetooth: smp: Warn on enabling several Bluetooth options
Add warning about enabling the options below so that users are
aware that this is a security risk.

- CONFIG_BT_DEBUG_SMP
- CONFIG_BT_DEBUG_KEYS
- CONFIG_BT_OOB_DATA_FIXED
- CONFIG_BT_USE_DEBUG_KEYS
- CONFIG_BT_STORE_DEBUG_KEYS

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2020-06-17 17:14:33 +03:00
Aastha Grover cf72fe8fe9 tests: sanitycheck: Testcases for load_from_file and apply_filters
test_testsuite_class.py: Added testcases for load_From_file and
apply_filters functions of Class Testsuite.
conftest.py: Modified class_testsuite fixture to create the
outdir as temporary directory which gets deleted after execution
of testcases.
test_data/sanitycheck.csv: load_from_file function uses this
customized file to load the failed or last run testcases.
test_data/sanitycheck_keyerror.csv: file used by
test_load_from_file function to raise the appropriate error
if there is a keyerror.
scripts/requirements-build-test.txt: added mock & csv python libraries

Signed-off-by: Aastha Grover <aastha.grover@intel.com>
2020-06-15 22:27:18 -04:00
Aastha Grover 27435ce810 scripts: tests: sanitycheck: Add more testcases for Class TestSuite
test_testsuite_class.py: Added testcases for add_configurations(),
get_all_testcases(), get_toolchain() and get_platforms() functions
test_data/board_config: board root directory for all board
configurations. This is also to add the platform configurations
in a list which is used for further testing.
conftest.py: Added fixtures for all_testcases_dict (returns a
dictionary of testcases) and platforms_list (returns a list of
all platforms using add_configurations function).

Signed-off-by: Aastha Grover <aastha.grover@intel.com>
2020-06-15 22:27:18 -04:00
Ruslan Mstoi e406c9c448 scripts: gen_gcov_files.py: add argument help text
Convert the description comment at the top of the file to a
documentation string. That string then maybe viewed with the --help
argument. Also, rework a bit the documentation string: remove usage,
since that is provided by the help message of the argparse module.

Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
2020-06-13 16:33:22 -04:00
Ruslan Mstoi 69b07e3156 scripts: gen_gcov_files.py: PEP 8 style fixes
Fix PEP 8 coding style issues reported by pycodestyle.

Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
2020-06-13 16:33:22 -04:00
Anas Nashif adb6a89be5 sanitycheck: capture non-ztest results
We were missing test results from non-ztest tests in the target reports.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-06-12 11:50:26 -04:00
Anas Nashif 288ae28c13 ci: add compliance checking via GH actions
Move check_compliance script to main tree and adapt/use with GH actions.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-06-12 07:53:09 -04:00
Flavio Ceolin b429b12609 hardening: Add MCUMGR_CMD_FS_MGMT recommendation
MCMUMGR file system management is discouraged in production, just
adding it to the recommendation list.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2020-06-12 11:15:24 +02:00
Kumar Gala 9ca541a504 linker: Remove deprecated Kconfig options related to linker scripts
Remove Kconfig, linker script, and related bits associated with
CUSTOM_RODATA_LD, CUSTOM_RWDATA_LD, CUSTOM_SECTIONS_LD,
SOC_NOINIT_LD, SOC_RODATA_LD, and SOC_RWDATA_LD options that have been
deprecated since Zephyr 2.2.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-06-12 11:14:28 +02:00
Ruslan Mstoi a8e0655833 scripts: file2hex.py: PEP 8 style fix
Fix pycodestyle PEP 8 issue:

E127 continuation line over-indented for visual indent

Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
2020-06-12 11:09:58 +02:00
Ruslan Mstoi 1bae76770b scripts: file2hex.py: add argument help text
Convert the description comment at the top of the file to a
documentation string. That string then maybe viewed with the --help
argument. Also, rework a bit the documentation string.

Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
2020-06-12 11:09:58 +02:00
Torsten Rasmussen 3917ee51a9 scripts: zephyr modules: introducing Zephyr module dependency handling
This commit introduces the possibility of adding dependencies between
Zephyr modules to ensure that a Zephyr module on which other modules
depends is processed first.

The dependency chain is ordered using a topological sort.

This allows to add dependencies to a zephyr/module.yml as:

build:
  cmake: .
  depends:
    - fatfs

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2020-06-12 11:07:58 +02:00
Wayne Ren 7cbe5c3e18 scripts: add unit test for mdb runner
add unit test for mdb runner

Signed-off-by: Wayne Ren <wei.ren@synopsys.com>
2020-06-11 10:02:11 -04:00
Wayne Ren 1506db5098 scripts: add the runner script for metaware debugger
* add the runner script for metaware debugger(mdb).
* mdb is required for SMP case
* mdb also can provides a GUI interface

Signed-off-by: Wayne Ren <wei.ren@synopsys.com>
2020-06-11 10:02:11 -04:00
Peter Bigot 3c36b42d03 sanitycheck: add notes key to schema
Prior to addition of schema validation running sanitycheck
--generate-hw-map would preserve the value of any field in an existing
map that it didn't update.  These fields now cause a parsing error.
Add notes as an optional key where information relevant to the board
can be added to provide useful context (such as which serial port
corresponds to the console, or why a non-standard runner is required).

Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
2020-06-10 12:03:17 +02:00
Peter Bigot 3cdd5b4be4 sanitycheck: add missing probe_id key to schema
probe_id is used as an optional key that overrides id as the source
of board_id when configured to use a runner that is not the default.

Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
2020-06-10 12:03:17 +02:00
Kumar Gala 491bbad8e8 ci: Update for buildkite environment
* Tweak west_setup:
  - log `west update` to a file to reduce noise in log
  - use `west forall` + `git reset` to make sure files are checked out
    (this is to handle a possible module cache)
* Output when we start sanity_check.  Add a banner for when we run
  sanity_check so it's a bit easier to find in console logs

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-06-09 15:28:48 -04:00
Ruslan Mstoi 7d8adddd2e scripts: process_gperf: typo fix
Fix double "e" in "repreesentation"

Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
2020-06-09 10:40:58 +02:00
Ruslan Mstoi 84822e3086 scripts: gen_syscalls: typo fix
Fix "defing" as "defining"

Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
2020-06-09 10:39:49 +02:00
Spoorthy Priya Yerabolu d434dfc22c sanitylib: Adding small change to enable create_overlay() testing
Returning the content from create_overlay() in TestInstance class

Signed-off-by: Spoorthy Priya Yerabolu <spoorthy.priya.yerabolu@intel.com>
2020-06-08 16:38:48 -04:00
Spoorthy Priya Yerabolu 9ac30b595c scripts: tests: sanitycheck: Add testcases for TestInstance Class
test_testinstance.py: Add testcases for check_build_or_run()
create_overlay() and calculate_sizes()
conftest.py: Module for common pytest fixtures
test_data/board_config: Adding test data for board configurations

Signed-off-by: Spoorthy Priya Yerabolu <spoorthy.priya.yerabolu@intel.com>
2020-06-08 16:38:48 -04:00
Kumar Gala 93da8dc478 Revert "checkpatch: update checkpatch to warn about C99 type usage"
Now that we are standardizing C99 integer types we can revert the commit
that warned about C99 type usage.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-06-08 08:23:57 -05:00
Kumar Gala a1b77fd589 zephyr: replace zephyr integer types with C99 types
git grep -l 'u\(8\|16\|32\|64\)_t' | \
		xargs sed -i "s/u\(8\|16\|32\|64\)_t/uint\1_t/g"
	git grep -l 's\(8\|16\|32\|64\)_t' | \
		xargs sed -i "s/s\(8\|16\|32\|64\)_t/int\1_t/g"

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-06-08 08:23:57 -05:00
Andrew Boie be919d3bf7 userspace: improve dynamic object allocation
We now have a low-level function z_dynamic_object_create()
which is not a system call and is used for installing
kernel objects that are not supported by k_object_alloc().

Checking for valid object type enumeration values moved
completely to the implementation function.

A few debug messages and comments were improved.

Futexes and sys_mutexes are now properly excluded from
dynamic generation.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-06-03 22:33:32 +02:00
Andrew Boie 299ec8f1b5 userspace: net sockets are kernel objects
Any data structure declaration tagged with __net_socket will end up
in the kernel object table with type K_OBJ_NET_SOCKET. These all
correspond to objects which are associated with socket file
descriptors and can handle the socket vtable API.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-06-03 22:33:32 +02:00
Andrew Boie 5960119f16 scripts: parse_syscalls: generalize struct tags
Now we can build up lists of data structures matching a list
of particular tags, with __subsystem being just one case.

Relax searches to also look inside C files, since struct
prototypes may be declared there as well.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-06-03 22:33:32 +02:00
Andrew Boie 455e178b3b scripts: gen_kobject_list: generalize obj alloc
Instead of handling this ad hoc, generalize which kobject
types can be allocated.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-06-03 22:33:32 +02:00
Andrew Boie 378024c510 userspace: add z_is_in_user_syscall()
Certain types of system call validation may need to be pushed
deeper in the implementation and not performed in the verification
function. If such checks are only pertinent when the caller was
from user mode, we need an API to detect this situation.

This is implemented by having thread->syscall_frame be non-NULL
only while a user system call is in progress. The template for the
system call marshalling functions is changed to clear this value
on exit.

A test is added to prove that this works.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-06-03 22:33:32 +02:00
Martí Bolívar e4a761cffe devicetree: add migration guide documentation and tests
Add test cases that verify various bits and pieces of the legacy
devicetree macros match the new APIs.

Writing these test cases without giving rise to deprecated macro
warnings which might break people's CI if they build with -Werror
requires turning off the __WARN() generation in
devicetree_legacy_unfixed.h. The entire file is deprecated at this
point and must be explicitly enabled with an opt-in Kconfig option, so
there isn't any harm in doing this.

Nevertheless, take a minimally invasive approach to avoiding __WARN()
generation in gen_legacy_defines.py, to avoid the possibility of
breakage. This code is basically frozen anyway, so hacks like this
won't cause maintainability problems since it isn't being actively
maintained.

Use the new tests as fodder for a migration guide from the old API in
the documentation.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-05-28 22:12:38 +02:00
Martí Bolívar 9d36b4f6e4 doc: some devicetree fixes and updates
Some updates to the reference page for the "core" APIs, and associated
follow-ups in the guides:

- centralize documentation of chosen zephyr nodes in a non-legacy
  file, provide a reference to them from the intro page in the guide
- review doxygen docstrings and correct errors for generic APIs
- add introductory text to each section in the API reference
- add missing hardware-specific pages

Documentation for layers built on top of these is mostly left to future
commits, but I do have a smattering of fixes in the guides that I
noticed while I was doing this.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-05-26 18:12:00 +02:00
Wentong Wu 6c9d4a5c57 scripts: use debug function instead of debug_die to dump objs
In step 4 of find_kobjects, use func debug instead of debug_die
to dump debug info to avoid dump wrong info.

Signed-off-by: Wentong Wu <wentong.wu@intel.com>
2020-05-25 18:07:20 +02:00
Wentong Wu 0bf5113c6b Revert "scripts: add DW_AT_abstract_origin check"
This reverts commit 2593a919ee.

This commit wasn't the root cause of the found bug in dwarf analyze.

Signed-off-by: Wentong Wu <wentong.wu@intel.com>
2020-05-25 18:07:20 +02:00
Maciej Perkowski b2fa99caf8 sanitycheck: Add filtered test cases to the JUnit full report
If a test instance from .yaml file was skipped due to being on
a filtered list then there are no entries about it in the
sanitycheck_report.xml for a given platform. This commit fills
instance.results with skipped test cases also for skipped instances so
those test cases are visible when the full report is generated. Tested
for ./tests/kernel/mem_protect/userspace on nrf52dk_nrf52832 where
userspace.gap_filling is filtered out.

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2020-05-25 15:52:17 +02:00
Anas Nashif 3b86f13fab sanitycheck: add fixture support to hardware map
It is now possible to add a list of fixture a platform supports which is
matched to testcases requesting fixtures to be avaiable to be able to
run.

For example:

- available: true
  connected: true
  id: 0240000026334e450015400f5e0e000b4eb1000097969900
  platform: frdm_k64f
  product: DAPLink CMSIS-DAP
  runner: pyocd
  serial: /dev/ttyACM9
  fixtures:
    - gpio_loopback

Fixes #24943

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-05-24 20:25:51 +02:00
Anas Nashif cefb9a170f sanitycheck: support fixtures
Add fixtures to hardware map schema.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-05-24 20:25:51 +02:00
Anas Nashif ce8c12eb5a sanitycheck: cleanup fixture processing
Cleanup fixture processing and allow ztest testcases to support
harness_config with fixture definition.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-05-24 20:25:51 +02:00
Anas Nashif 77946fa835 sanitycheck: remove wrong message about --qemu-testing
--qemu-testing does not exist.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-05-21 22:47:36 -04:00
Ruslan Mstoi 56b65e43de scripts: gen_offset_header: add argument help text
Help text is set to documentation string of the script.

Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
2020-05-21 14:45:24 +02:00
Ruslan Mstoi 4695eb4e1d scripts: net: Fix incorrect error messages
If environment variables ZEPHYR_BASE or NET_TOOLS_BASE are unset, an
incorrect message is printed saying that "it is set, but it is not a
directory":

$ZEPHYR_BASE is unset
$ZEPHYR_BASE is set, but it is not a directory
$NET_TOOLS_BASE is unset, no net-tools found
$NET_TOOLS_BASE set, but it is not a directory

This patch fixes that issue.

Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
2020-05-21 14:45:07 +02:00
Martí Bolívar 414ed86269 edtlib: fix default type for interrupts property
The name of the interrupts property is typo-ed in the python script.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-05-20 10:41:07 +02:00