Commit graph

2639 commits

Author SHA1 Message Date
Martí Bolívar 469f53c210 scripts: sanitycheck: import edtlib
This is a cargo-culted attempt to make an error observed in poorly
understood CI circumstances go away when loading edt.pickle, by making
edtlib visible in sys.modules before loading the pickle file.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-07-24 17:58:03 -04:00
Wentong Wu 517633c101 scripts: sanitylib.py: add exception check for psutil
Add exception check for psutil when running CI with QEMU.

Signed-off-by: Wentong Wu <wentong.wu@intel.com>
2020-07-24 14:26:33 -04:00
Anas Nashif 8d8875b752 scripts: add maintainer script
Adding the get_maintainer.py script

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
2020-07-24 17:07:48 +02:00
Kumar Gala feeda05323 ci: Add a debug out of what tests we are going to build
Add a cat of the test_file.txt so we can see exactly what tests are
expected to be built and run.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-07-24 12:10:15 +02:00
Martí Bolívar 9861e5d22a sanitylib: log test exception tracebacks
This aids in debugging.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-07-23 13:36:22 -05:00
Martí Bolívar 9c92baa73f devicetree: use edt.pickle more
Consolidate creation of edtlib.EDT objects from a build directory's
devicetree into one place by loading it from build/zephyr/edt.pickle
everywhere. A previous commit creates edt.pickle from gen_defines.py.

In addition to probably speeding things up slightly by not reparsing
the devicetree, the main benefit of this approach is creating a single
point of truth for the bindings directories and warnings
configuration, meaning we don't have to worry about them getting out
of sync while being passed around between devicetree creation and
usage time.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-07-23 09:19:45 -05:00
Jan Van Winkel 716eea0b65 scripts: Increased minimal pyelftools version
Increased minimal pyelftools version to 0.26 else
scripts/footprint/size_report will fail to import LocationExpr from
elftools.dwarf.locationlists

Signed-off-by: Jan Van Winkel <jan.van_winkel@dxplore.eu>
2020-07-23 07:49:35 -04:00
Martí Bolívar 006319f043 scripts: kconfigfunctions: add a comment
I can never remember what the contents of the kconfigfunctions dict is
and have to look it up every time. I suspect I'm not alone, and not
everyone who uses these will know how to look up the answer. Add a
comment explaining the value.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-07-23 11:00:39 +02:00
Martí Bolívar 24677f93bc kconfig: add dt_path_enabled, dt_alias_enabled
There is already a way to check if a node is enabled based on its node
label, but we don't have an equivalent way to do that for a path or an
alias. Add them. These rely on the same underlying edtlib machinery to
get their jobs done, but are being treated differently within Kconfig
in order to match distinctions between paths and aliases made in the
devicetree.h API.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-07-23 11:00:39 +02:00
Luiz Augusto von Dentz 75f47a56b5 Bluetooth: Add config option to disable security checks
This adds CONFIG_BT_CONN_DISABLE_SECURITY which can be used to disable
security checks for incoming requests enabling to test accessing GATT
attributes and L2CAP channels that would otherwise require
encryption/authentication in order to be accessed.

It depends on BT_TESTING to indicate to the users that this is a
testing feature which shall not be used in production.

Signed-off-by: Luiz Augusto von Dentz <luiz.von.dentz@intel.com>
2020-07-23 11:00:04 +02:00
Maureen Helm 8c7cef6da0 runners: Implement --erase option in pyocd runner
Implements the common runner option to mass erase before flashing in the
pyocd runner.

Signed-off-by: Maureen Helm <maureen.helm@nxp.com>
2020-07-23 10:30:47 +02:00
Anas Nashif e47866c530 sanitycheck: fix --device-testing without serial-pty
serial-pty is an extension that is not supported in the hardware map, so
we need to work around the fact that it is not part of the map when
dealing with normal serial.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-07-22 16:57:14 -05:00
Andrei Emeltchenko d8b845be4e sanitycheck: Add serial_pty script command line option
Add option to provide to sanitycheck argument for creating
pseudoterminal. Used with hardware without serial console connected.
A pseudoterminal is used to make a sanitycheck believe that it
interacts with a terminal although it actually interacts with the
script.

E.g "sanitycheck --device-testing --device-serial-pty <script>"

Signed-off-by: Andrei Emeltchenko <andrei.emeltchenko@intel.com>
2020-07-21 20:19:33 -04:00
Anas Nashif e8e367ada4 sanitycheck: support only_tags for boards
Support running/building only specific tags and ignoring everything
else.

Adding this to a platform yaml file will enable tests with one of of
tags listed to be executed on the platform.

This is useful for special platform configurations used for testing
specific features for example.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-07-20 17:44:16 -04:00
Ruslan Mstoi 6f26ca0b60 scripts: tests: Add tests for subfolder_list.py
This commit implements tests of subfolder_list.py

Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
2020-07-20 17:31:53 -04:00
Ruslan Mstoi ab20641054 scripts: subfolder_list.py: refactor the code
Refactor subfolder_list.py to enable unit testing. This means split the
implementation into separate logical functions. Additionally, modify the
code to improve PEP 8 compatibility and pylint result.

Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
2020-07-20 17:31:53 -04:00
Carles Cufi 02dea9279b scripts: modules: Fix initialization of variable
The projects variable needs to be initialized with the list of modules
provided via cmd-line arguments.

This is a regression introduced by
ef3c5e5516.

Fixes #26948.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2020-07-20 22:39:09 +02:00
Anas Nashif 5092541e2b sanitycheck: disable erroring on warnings
--warnings-as-errors was basically doing nothing, it is the default.
Replace this with an option to disable erroring on warning:

 -W, --disable-warnings-as-errors

Fixes #26910

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-07-17 13:03:36 -05:00
Anas Nashif 06052927be sanitycheck: capture fails correctly on qemu crashes
We still have cases where qemu crashes are not being correctly captured
in the on-screen summary, however they are captured in the final
generated report.

Fixes #26679

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-07-16 15:44:17 -04:00
Daniel Leung ba5f627815 scripts: size_report: rework to use pyelftools...
...instead of GNU binutils to extract DWARF information.
This is now a bit more portable across OS and toolchain.
One bouns is that this nows with on qemu_x86_64.

A few differences are:
() z_mrsh_* which are aliases to handler_no_syscalls() are now
   dropped as they are mapped to the same address, so that they
   are not counted multiple times.
() Static functions and variables with same names should now be
   attributed to the correct source files instead of being
   accumulated into the same symbol of one file (e.g. multiple
   thread_entry() in kernel tests).
() The totals for ROM and RAM are calculated from the
   corresponding sections. Previous script includes the debug
   sections as total ROM size which is not entirely correct.

Fixes #22996

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2020-07-16 12:14:22 +02:00
Spoorthy Priya Yerabolu 0e599be5fc sanitycheck: Updating README file
Updated readme with coverage commands and
organization of tests

Signed-off-by: Spoorthy Priya Yerabolu <spoorthy.priya.yerabolu@intel.com>
2020-07-09 16:03:57 -05:00
Anas Nashif dc43c298a2 sanitycheck: do not retry build errors with --retry-failed
restore how --only-failed works by allow rebuilds in case of build
errors, however, do not rebuild when --retry-failed is used, which is a
CI usecase and nothing would change in the second iteration.

Fixes #26685

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-07-09 16:46:24 -04:00
Stephanos Ioannidis 3322489d22 config: Rename TEXT_SECTION_OFFSET to ROM_START_OFFSET
The `TEXT_SECTION_OFFSET` symbol is used to specify the offset between
the beginning of the ROM area and the address of the first ROM section.

This commit renames `TEXT_SECTION_OFFSET` to `ROM_START_OFFSET` because
the first ROM section is not always the `.text` section.

Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
2020-07-09 14:02:38 -04:00
Christian Taedcke 3dbe9f2960 sanitycheck: native: Add option to enable UBSAN
Add option for native platform to enable undefined behaviour sanitizer.

Signed-off-by: Christian Taedcke <christian.taedcke@lemonbeat.com>
2020-07-09 07:49:57 -04:00
Aastha Grover cb0f65e410 requirements-build-test.txt: Updating mock module version
Updating mock module version to >=4.0.1 as sanitycheck testsuite
might fail in some systems due to old version.

Signed-off-by: Aastha Grover <aastha.grover@intel.com>
2020-07-09 06:26:28 -04:00
Anas Nashif b8c7de6d9a actions: update daily version for testing
Update versions.json with the daily commit that device testing will be
performed on.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-07-09 11:29:56 +02:00
Torsten Rasmussen ef3c5e5516 cmake: west: invoke west using same python as rest of build system
When running CMake, then Python3 will be used.
This is detected through FindPython3, with a preference for using the
python or python3 in path, if any of those matches the required Python
minimal version in Zephyr.

It is also possible for users to specify a different Python, as example
by using:
`cmake -DPYTHON_PREFER=/usr/bin/python3.x`

However, when running `west` as native command, then west will be
invoked on linux based on the python defined in:
`west` launcher, which could be: `#!/usr/bin/python3.y`

Thus there could be mismatch in Pythons used for `west` and the python
used for other scripts.

This is even worse on windows, where a user might experience:
```
>.\opt\bin\Scripts\west.exe --version
Traceback (most recent call last):
  File "C:\Python37\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  ...
  File "C:\Python37\lib\socket.py", line 49, in <module>
    import _socket
ImportError: Module use of python38.dll conflicts with this version of
Python.
```

when testing out a newer Python, but the python in path is still a 3.7.

By importing `west` into zephyr_module.py and by using, as example
`python -c "from west.util import west_topdir; print(topdir())"`
we ensure the same python is used in all python scripts.

Also it allows the user to control the python to use for west.

It also ensures that the west version being tested, is also the version
being used, where old code would test the version imported by python,
but using the west in path (which could be a different version)

If the west version installed in the current Python, and west invocation
is using a different Python interpreter, then an additional help text
is printed, to easier assist users with debugging.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2020-07-08 23:23:04 -04:00
Kumar Gala 4e2988deeb edtlib: Fold _binding_compat into _init_compat2binding
We only have one use of _binding_compat and it doesn't need self, so
just fold it into _init_compat2binding.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-07-08 22:03:24 -04:00
Kumar Gala 6bf761fc0a dts: Remove support for deprecated DTS binding syntax
We deprecated a number of aspects of the DTS binding syntax in Zephyr
2.1.  Remove the support for the deprecated syntax.  Remove from docs
about the deprecated syntax as well.

Removed reference in release-notes-2.1.rst to legacy_binding_syntax
since that anchor doesn't exist anymore.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-07-08 22:03:24 -04:00
Martí Bolívar d8698cbdc6 sanitylib: pacify pylint
According to our compliance checks, we have 'useless-suppression'
and 'unused-import' on our hands:

W0611: Unused CLoader imported from yaml as Loader
W0611: Unused Loader imported from yaml
I0021: Useless suppression of 'unsubscriptable-object'

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-07-08 20:19:02 -04:00
Martí Bolívar 148542c080 build: remove dead stores to environment
Now that kconfigfunctions.py is loading the EDT object from a pickle
file, I can't find any other reads of the environment variables it was
previously using to do that.

(The CMake variables with the same names are read in different places;
I'm specifically talking about environment variables here.)

Remove the dead stores.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-07-08 14:02:44 -05:00
Martí Bolívar 269f350487 kconfig: load edt from edt.pickle
This saves time and avoids the need to pass additional state around in
the environment to recreate the edt exactly.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-07-08 14:02:44 -05:00
Martí Bolívar 533f451e7e dts: marshal the EDT object for later use
We need to save and restore the devicetree data to generate optimized
dependency information later on in the build, in particular during the
final application link.

Make this happen by pickling the EDT object in BUILD_DIR/edt.pickle.

The existence of this file is an implementation detail, so do not add
it to the documentation.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-07-08 14:02:44 -05:00
Martí Bolívar e76b7205ef dts: edtlib: allow pickling/unpickling EDT objects
We have a use case for saving the EDT object to be able to open it up
again later. It would be convenient to be able to do this with the
pickle module from stdlib.

The only thing stopping us from doing that appears to be the open
reference to sys.stderr that's held the edt object even after
EDT.__init__ exits. However, there doesn't seem to be a need to keep
holding on to this object, and in fact it would be a little bit nicer
to drop the reference in case something else (even in the same Python
process that created it originally) wants the EDT object around, but
might want the warn file closed if its refcount zeroes out.

Just drop the reference at the end of __init__ and make EDT._warn()
throw an exception if it's attempted to be used after the constructor
exits.

Make pickle-ability an API guarantee so we can treat any regressions
as bugs going forward.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-07-08 14:02:44 -05:00
Martí Bolívar e05c94e334 dts: edtlib: fix reg / unit addr warn location
I can't see a good reason to be doing this in the Node class's
unit_addr accessor. Move it up to the edtlib initialization so it only
happens once.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-07-08 14:02:44 -05:00
Anas Nashif ae61b7edd6 sanitycheck: Use the C LibYAML parser if available
Use the C LibYAML parser if available, rather than the Python parser. It
is much faster.

This is a clean and rebased version of PR #20210 by
Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>

Co-authored-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-07-08 13:34:45 -05:00
Anas Nashif 869ca055c5 sanitycheck: handle qemu crashes
Deal with qemu crashes and do not register pass status as the default.
In case we do not have a status, report unknown.

This now captures different states from qemu that were not captured
before.

Fixes #26679

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-07-07 16:04:32 -07:00
Erwan Gouriou 2339fa00f3 scripts: sanitylib.py: Add support for STLINK-V3 probe
Recent ST boards embed the new ST-Link probe V3.
It is advertised as "STLINK-V3", update sanitylmib to take it
into account.
In handle function, it is proposed to treat it separately as current
"STM32 STLink" as processing might differ in next future (hla_serial
deprecation).

Signed-off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
2020-07-07 16:41:09 -04:00
Øyvind Rønningstad 4813f469f0 sanitycheck: Improve handling of duplicate devices
Some devices have multiple serial ports, which show as duplicates in the
hardware map. This doesn't work well when using --generate-hardware-map
to regenerate an existing map. E.g. nrf5340pdk_nrf5340_cpuapp by default
prints to the 3rd of 3 devices. If it shows up on /dev/ttyACM0,
/dev/ttyACM1, and /dev/ttyACM2, printout will come on /dev/ttyACM2.
nrf9160dk_nrf9160 also provides 3 devices, but prints to the 1st.

This patch sorts the devices by the serial port and matches multiple
duplicate entries one-to-one to retain the ordering. This way, the
correct device can be given the platform name and the others can be kept
as "unknown" so that --device-testing understands correctly
1) that there is only one device (not 3)
2) where the serial output will come.

Signed-off-by: Øyvind Rønningstad <oyvind.ronningstad@nordicsemi.no>
2020-07-06 10:27:22 -04:00
Aastha Grover 8213a1509f sanitycheck: Testcases for report functions of TestSuite Class
test_reporting_testsuite.py: Adding testcases for reporting
functions of Class testSuite.
test_testsuite_class.py: Testcase for add_instances function
conftest.py: Added fixture for instances_fixture

Signed-off-by: Aastha Grover <aastha.grover@intel.com>
2020-07-03 12:20:35 -04:00
Aastha Grover dcbd915759 sanitylib: Fix discard_report to check self.discards dict
Modifying discard_report function to fix the check for
self.discards dictionary in try block. self.discards
is None is dead code which didnot check if the dictionary was
empty. Changed it to if not self.discards.

Signed-off-by: Aastha Grover <aastha.grover@intel.com>
2020-07-03 12:20:35 -04:00
Anas Nashif 3cb1b437d2 ci: do not run sanitycheck on linter config changes
Do not run sanitycheck on linter configuration changes.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-07-03 07:58:31 -04:00
Anas Nashif f04461e8d0 sanitycheck: fix --failed-only handling
- Report build errors as errors, not test failures
- Do not try and build/run tests with build failures
- Fix issue with empty reports when running --only-failed
- Report build errors in the detailed and target reports

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-07-03 07:25:49 -04:00
Michael Hope bb8277962d scripts: bossac: automatically pull the flash offset from the config
BOSSAC uses a bootloader, so pull the flash address from DeviceTree or
the config and use if the version of BOSSAC supports offsets.

Signed-off-by: Michael Hope <mlhx@google.com>
2020-07-02 08:23:03 -04:00
Michael Hope 4ac195e0af scripts: bossac: add support for bossac 1.9.1
BOSSA 1.7 has built-in bootloader offset handling while 1.9.1
requires the user to supply the offset.  Add support for both by
sniffing the help output and warn the user if the flags needs adding.

Related to #22062

Signed-off-by: Michael Hope <mlhx@google.com>
2020-07-02 08:23:03 -04:00
Torsten Rasmussen 13642fe013 cmake: west: west flash dependencies in sync with CMake flash target
This commit creates a list of a phony targets for each runner, that is:
`west_flash_depends`, `west_debug_depends`, and so on.
Those targets has identical dependencies as CMake runner target.
flash, debug, debugserver, attach targets.

As example `ninja flash` correctly ensures dependencies are taken into
consideration before calling `west flash`.

Unfortunately, calling `west flash` directly would not re-run the flash
dependencies, cause `west flash` would only build the default CMake
target.

Now, `west flash` calls the phony `west_flash_depends` target, ensuring
all deps are up-to-date before flashing (unless --skip-rebuild is given)

The same is true for the other mentioned runners.

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
2020-07-01 07:58:10 -04:00
Anas Nashif b5973930b1 sanitycheck: set selected_plaforms when loading tests from file
Missed the case we use in CI where we pre-generate the testcase list and
load it, this was resulting in empty test reports in CI.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-06-30 18:34:01 -04:00
Spoorthy Priya Yerabolu 473ed3412f scripts: tests: sanitycheck: Add testcases for TestCase class
test_testinstance.py: Add testcases to scan file and path for
sub testcases
test_data/testcases: Also added the ztest test files
test_testsuite_class.py: changed get_all_tests() to match
count of sub testcases in ztest files

Signed-off-by: Spoorthy Priya Yerabolu <spoorthy.priya.yerabolu@intel.com>
2020-06-29 13:15:00 -04:00
Spoorthy Priya Yerabolu ad4d4fc7d1 scripts: tests: sanitycheck: changes to sanitylib to sort the warnings
While testing the warnings in scan_file() format is not the same
everytime we run tests
Added sorted with reverse flag so that warnings are same for
every run

Signed-off-by: Spoorthy Priya Yerabolu <spoorthy.priya.yerabolu@intel.com>
2020-06-29 13:15:00 -04:00
Anas Nashif 33c21e1f7d ci: do not skip BT simulator tests
BT simulator tests run standalone and independent of sanitycheck and
might be triggered by tests that are not managed by sanitycheck, so
leave it alone.

Fixes #26508

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-06-29 12:16:37 -04:00