Add the ability to map file/dir paths of a PR to twister TAG. We
introduce scripts/ci/tags.yaml to conveys which files are associated
with which tag.
Since not all file/tags will be specified in tags.yaml we use the
combination of the files modified list and the tags.yaml information to
determine which tags can be excluded (ie if the file list doesn't match
any file in tags.yaml for a given tag listed there, we can that exclude
it).
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Treat all test_file_* files the same and just explicitly set the header
in the final test_file.txt. This makes the logic the same regardless of
what paths tests are coming from.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
If the local option (-l) was specified we were ignoring the results
of what_changed.py and always setting a "FULL" build. Only explicitly
set a FULL run if we are building for a commit to the tree (local and
PR builds should respect the output of what_changed.py).
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Since we use 'main' now, rename the example comment usage to use
that branch name instead of 'master'.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Add a very simple check that verifies that new error numbers are added
according to the errno.h file in newlib, in order to maintain
compatibility with it.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Added SAMPLE_MODULE_LOG_LEVEL_DBG as an exception. Define is used
in the log_api test suite.
Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
This scripts receives the same parameter of what_changed.py. And run
coccinelle scripts for code guideline compliance in the given git
commits. e.g: ./guideline_check.py --commits origin/master..HEAD
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
BabbleSim execution was (AFAICT mistakenly) disabled in
dda6a5ee90d0f5b31e7d269761dc2ab1a2f8510c. Re-enable it so we run them in
CI again.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
If files are changed for architecture code, then only run CI on those
architectures.
Combine all scripts into one and make it simpler.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
This commit lets the Kconfig compliance check run from top of current
git repo instead of only executing inside ZEPHYR_BASE.
This extends the usability of the Kconfig compliance check.
Together with this possibility, a KconfigBasic mode has been added which
will execute the same verification for symbols in the Kconfig tree but
will not check for undefined symbols outside Kconfig.
This is needed as running Kconfig undef check outside the tree in
non-Zephyr repos might pickup `CONFIG_` named symbols in other code.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Changes to .buildkite/daily.yml don't make any sense to run twister
for since issues will only be seen when the daily builds run so add
it to the twister_ignore.txt list.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
The introduction of Zephyr module glue code in the Zephyr repository
introduces a Kconfig variable in the form of:
`config ZEPHYR_<MODULE_NAME>_MODULE`.
All Kconfig variables go into `autoconf.h`, therefore it is necessary
to sanitize the Kconfig variable, so that it does not contain special
characters. To ensure consistent variable name, then the module name
will be sanitized in all variable use in both Kconfig and CMake.
The sanitization is done be replacing all special characters with an
underscore, `_`.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Fixes: #31436
This fix adds ZEPHYR_<MODULE_NAME>_KCONFIG variable to compliance check.
This ensures that Zephyr Kconfig glue code is included in the
compliance check, and thus remove the issue reported in #31436.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Likely an accident when commit 288ae28c13 moved the script to the main
repo. Stayed unnoticed because this handler rarely ever runs.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
We made a similar change for the nightly builds, but the PR builds
invoke run_ci.sh, so we need the same change here.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Fixes: #28462
This commit allows shields to be defined in other BOARD_ROOTs, either
using `-DBOARD_ROOT=<path>` or a Zephyr module defined BOARD_ROOT.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Introduce TagoIO IoT Cloud HTTP post client example. This explorer
Zephyr network resources to demonstrate an end to end application.
The TagoIO allows that any user can test on a easy way Ethernet, WIFI
and Modem (PPP) with BSD sockets. The example provides overlays to
configure WIFI and Modem.
The application consists an a pseudo temperature sensor that sends
periodically data to TagoIO IoT Cloud platform. The data can be
visualized on a web browser dashboard, cellphone or tablet. The
steps to configure TagoIO are described on the example documentation.
Special Variables:
- CONFIG_TAGOIO_DEVICE_TOKEN DEVID token generated by TagoIO
- CONFIG_TAGOIO_HTTP_WIFI_SSID SSID when using WIFI
- CONFIG_TAGOIO_HTTP_WIFI_PSK PASSWD when using WIFI
- CONFIG_MODEM_GSM_UART_NAME UART label when using MODEM
- CONFIG_MODEM_GSM_APN APN when using MODEM
Signed-off-by: Gerson Fernando Budke <nandojve@gmail.com>
Add date and if a commit should be considered for weekly testing.
This will still work with the old format.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Use the pytest test framework in the dtlib.py and edtlib.py test
suites (testdtlib.py and testedtlib.py respectively).
The goal here is not to change what is being tested. The existing test
suite is excellent and very thorough.
However, it is made up of executable scripts where all of the tests
are run using a hand-rolled framework in a single function per file.
This is a bit all-or-nothing and prevents various nice features
available in the de-facto standard pytest test framework from being
used.
In particular, pytest can:
- drop into a debugger (pdb) when there is a problem
- accept a pattern which specifies a subset of tests to run
- print very detailed error messages about the actual and expected
results in various traceback formats from brief to very verbose
- gather coverage data for the python scripts being tested (via plugin)
- run tests in parallel (via plugin)
- It's easy in pytest to run tests with temporary directories
using the tmp_path and other fixtures. This us avoid
temporarily dirtying the working tree as is done now.
Moving to pytest lets us leverage all of these things without any loss
in ease of use (in fact, some things are nicer in pytest):
- Any function that starts with "test_" is automatically picked up and
run. No need for rolling up lists of functions into a test suite.
- Tests are written using ordinary Python 'assert'
statements.
- Pytest magic unpacks the AST of failed asserts to print details on
what went wrong in really nice ways. For example, it will show you
exactly what parts of two strings that are expected to be equal
differ.
For the most part, this is a pretty mechanical conversion:
- extract helpers and test cases into separate functions
- insert temporary paths and adjust tests accordingly to not match
file names exactly
- use 'assert CONDITION' instead of 'if not CONDITION: fail()'
There are a few cases where making this happen required slightly
larger changes than that, but they are limited.
Move the checks from check_compliance.py to a new GitHub workflow,
removing hacks that are no longer needed.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
This commit introduces support for multiple SOC_ROOT.
This means that additional SOC_ROOTs specified using -DSOC_ROOT as
argument to CMake will be forming a list together with ${ZEPHYR_BASE}.
This allows for greater flexibility, as developers can now specify
multiple out-of-tree SoCs and not worry about the SoC used for the
board they compile for.
Also it avoid code, such as:
if(BOARD STREQUAL my_board_using_out_of_tree_soc)
set(SOC_ROOT some/out/of/tree/soc/path)
endif()
in application CMakeLists.txt.
Finally, allowing multiple SOC_ROOTs prepares for specifying SOC_ROOTs
in Zephyr modules.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Now that kconfigfunctions.py is loading the EDT object from a pickle
file, I can't find any other reads of the environment variables it was
previously using to do that.
(The CMake variables with the same names are read in different places;
I'm specifically talking about environment variables here.)
Remove the dead stores.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
BT simulator tests run standalone and independent of sanitycheck and
might be triggered by tests that are not managed by sanitycheck, so
leave it alone.
Fixes#26508
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
* Tweak west_setup:
- log `west update` to a file to reduce noise in log
- use `west forall` + `git reset` to make sure files are checked out
(this is to handle a possible module cache)
* Output when we start sanity_check. Add a banner for when we run
sanity_check so it's a bit easier to find in console logs
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
When files were added, matching did not work and we were counting added
files twice. Use sets to make sure we have a unique list of files.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
if we are changing a test or a sample, there is not need to run
everything, it is enough to just build/run the changed test/sample.
Also, some files have impact on the code and if they are being changed,
do not run fully sanitycheck.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Script to detect if full sanitycheck should run or if we can skip it and
just run the code that actuallt changed (in samples/tests).
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
When merging files, make sure to remove header rows from each file and
just keep one header. This will stop us from parsing header rows as
tests.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Replace running west command tests in run_ci.sh with a github workflow.
This provides some benefits in that we can run the west command tests on
multiple python versions and host OSes (linux, mac and windows).
Also have the benefit that the tests are only run on modifications to
files in scripts/west_commands/ or scripts/west-commands.yml.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
So that the EDTT tool could be used by this script,
set its path so other scripts could find it
Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
With timestamps enabled, the log in shippable is very difficult to read,
so remove those and enable them when needed for debugging only.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
If modules announce they have tests, run sanitycheck on those modules.
This will run on first matrix node and as the last step after running
regular testsuite.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Fetch HW models from a new west module.
And, remove all pre-west glue which was used to:
* Fetch them in CI
* Validate their vesion
* Modify the include path and link to them
Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
Make sure we cleanup the manifest files at the right spot of the CI
process. When not building a PR, behavior is different.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
For cases like zephyr-test or for other reasons the repo dir might not
be called zephyr. Just use the basename of the dir instead of assuming
its called zephyr.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
pylint does not like how this library works and generates spurious
warnings like
scripts/ci/get_modified_tests.py:55:13: E1121: Too many positional
arguments for function call (too-many-function-args)
scripts/ci/get_modified_tests.py:57:13: E1123: Unexpected keyword
argument '_tty_out' in function call (unexpected-keyword-arg)
These warnings are useful enough to want to have enabled in the upcoming
pylint CI check, so suppress them here.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
When running run_ci.sh locally with:
./scripts/ci/run_ci.sh -l -b master -R upstream/master..
we export BSIM_OUT_PATH unconditionally which causes sanitycheck to fail
on nrf52_bsim (it depends on BSIM_OUT_PATH variable).
Check if the path defined in BSIM_OUT_PATH is available and unset env.
variable if it is not.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
The main change is the elimination of the bootstrapper, a design flaw
/ misfeature.
Update the documentation to be compatible with the 0.6.x releases as
well. This has to be done atomically, as there were incompatible
changes. Make use of the versionchanged and versionadded directives
to begin keeping track of how these APIs are evolving.
(Note that west 0.6.0 will remain compatible with the extension
commands in Zephyr v1.14 LTS as long as that is still alive. This
change is targeted towards Zephyr 2.0 users.)
This requires a bump in the shippable container and allows us to
simplify the west_commands test procedure.
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
Found a few annoying typos and figured I better run script and
fix anything it can find, here are the results...
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
We not analysing coverage data at all in CI right now. Disable this
while we figure out a better solution for reporting data.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
When detecting changes to boards, make sure we test all board
configurations available in that board directory, not only the main one.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Update the files which contain no license information with the
'Apache-2.0' SPDX license identifier. Many source files in the tree are
missing licensing information, which makes it harder for compliance
tools to determine the correct license.
By default all files without license information are under the default
license of Zephyr, which is Apache version 2.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
The code coverage reports on pull requests has been inconsistent and
confusing. Disable them for pull-requests.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Fixes pylint warnings like this one:
doc/conf.py:325:0: W1401: Anomalous backslash in string: '\s'.
String constant might be missing an r prefix.
(anomalous-backslash-in-string)
The reason for this warning is that backslash escapes are interpreted in
non-raw (non-r-prefixed) strings. For example, '\a' and r'\a' are not
the same string (first one has a single ASCII bell character, second one
has two characters).
It just happens that there's no \s (or \., or \/) escape for example,
and '\s' turns into two characters (as needed for a regex). It's risky
to rely on stuff like that regexes though. Best to make them raw strings
unless they're super trivial.
Also note that '\s' and '\\s' turn into the same string.
Another tip: A literal ' can be put into a string with "blah'blah"
instead of 'blah\'blah'.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
This is useful when running the script locally on a set of commits to
reproduce how things are run in CI.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Zephyr Continuous Integration made the choice of rebasing pull requests
on its fast moving master branch before testing them. Unfortunately the
Shippable interface assumes the other choice[*] and highlights the SHA1
of the pull request _before_ rebasing which is very misleading. For now
the only way to realize this disconnect is to scrutinize the build logs
and/or this source code (or tribal knowledge).
To mitigate and reduce a bit this confusion, run a short:
git log -n 5 --oneline --decorate
before and after rebase.
Making it worse, Shippable sub-runs are not pegged to a common base
version. For instance the reason why shippable sub-run 35578.1 passed
while all other 35578.x sub runs failed is very likely because PR #13803
was merged to master in the mean time. This commit would have been able
to prove that.
[*] In an ideal world with infinite time and resources, Continuous
Integration would test pull requests BOTH alone and combined with the
moving tip.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
On (at least) Fedora "pytest" is python2 and python2 only.
Also update remote name in sample invocation from "upstream" to "origin"
because that's how west sets things up by default (and this script
requires west)
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
When we change documentation or board images, do not build all tests for
this board. Only add more tests when actual code has changed.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
When we change documentation or board images, do not build all tests for
this board. Only add more tests when actual code has changed.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Allow running script locally. For example, on a local tree with local
changes:
./scripts/ci/run_ci.sh -b master -r upstream -l
This will be have the same way as in CI and creates a test manifest
based on changes.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
We should not abort if west was initiaized, someone must have
initialized, i.e. locally and everything is already setup for us and
ready to go.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Incorporate these into run_ci.sh the same way that btsim results were
done. This adds a dependency on pytest.
Signed-off-by: Marti Bolivar <marti@foundries.io>
Commit 88ece494 ("ci: do not build docs in main CI job") removed the
call to build_docs but not the function itself. That makes it
confusing to try to find where it's called; remove it.
Signed-off-by: Marti Bolivar <marti@foundries.io>
The new west is able to initialize itself from an existing zephyr clone
without modifying it, adapt the CI script accordingly.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
West right now modifies the zephyr tree and points to master. In CI we
need to point to the pull-request HEAD instead, so so whatever shippable
does after cloning with west.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
In order for check-compliance.py to be able to run cleanly out-of-tree,
set the repo (to be checked) path based on cwd, and refer to Zephyr and
its tools by the specific path set by ZEPHYR_BASE.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Before run_ci.sh was introduced, if some command in the
.shippable.yml script failed, Shippable would stop and report
a failure.
When run_ci.sh was introduced we lost this, which is pretty
dangerous if something fails cathastrophically and the xml
reports are not (properly) generated and copied, as then
Shippable reports a pass.
=> Run run_ci.sh in -e mode, so we propagate a bad failure upwards
Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
Added a missing option in run_ci.sh to control the placement of the
xml results file. (Before it was just propagated thru the enviroment)
+
Changed .shippable to use this option when callign it.
Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
This reverts commit 6d5ac1c089.
Looks like this was not correctly done, building btsim is needed on all
nodes.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Building docs and running various checks is now being done in another CI
job that runs in parallel and reports directly to the PR.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
- Support sending status to github
- Support license checking
- Make every test a class
- Use junitparser instead of custom xml for junit output
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Compile and run the tests avaliable in bsim_bt
and collect the coverage results into the coverage report.
Also, detect if bsim's component folder already contains the
nRF52 HW models, and if it does instead of trying to fetch
them again (which will fail) check that the right versio is
present. This should ease testing locally.
Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
move CI scripting to a dedicated script. For example, to simulate what
is run in CI when a pull request is submitted:
./scripts/ci/run_ci.sh -b master -r upstream -p
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
To avoid having to set it externally, set the SOC_DIR environment
variable directly in the compliance script.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
The prefixes might be a leftover from the old 'option env="..."' symbols
(which are no longer needed). Since environment variables can be
referenced directly now, there's no point in having a prefix.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Threw an exception about Kconfig files not being found. They need to be
looked up relative to ZEPHYR_BASE.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Use the new KCONFIG_STRICT Kconfiglib functionality, which makes the
library itself warn for references to undefined Kconfig symbols.
Also get rid of the chdir() hack. Kconfiglib now only looks up Kconfig
files relative to $srctree when it's set, so the hack is no longer
needed.
Remove list_undef_kconfig_refs.py. It's no longer needed.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
External projects might run check-compliance.py from a subdirectory,
while list_undef_kconfig_refs.py expects to find the top-level Kconfig
file in the current directory.
Due to obscure Kconfig ugliness (explained in a comment), the working
directory needs to be changed here, instead of using 'srctree'.
(Thinking of changing that Kconfig behavior so that it only applies to
.config files. Suspect it's been a bug in the C tools all along for
Kconfig files.)
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Update Kconfiglib to get upstream commit eb6c21a9b33a2 ("Turn
MenuNode/Symbol/Choice.referenced() into a @property") in. It converts
the MenuNode.referenced() function into a property, which makes the API
more consistent (read-only stuff uses properties).
Also update scripts/ci/list_undef_kconfig_refs.py to access .referenced
as a property.
Piggyback a small is_num() simplification.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Add a helper module scripts/ci/list_undef_kconfig_refs.py that searches
the entire Kconfig tree and reports any references to undefined Kconfig
symbols. Use it to add a new check to scripts/ci/check-compliance.py.
Also allow list_undef_kconfig_refs.py to be run standalone.
Example error:
Error: Found references to undefined Kconfig symbols:
BAR
===
- Referenced at Kconfig:12:
config FOO
bool
depends on BAR
- Referenced at Kconfig:16:
menu "menu"
depends on BAR
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Allow check-compliance.py to be run from any path with a valid Git tree,
removing the unnecessary and artificial limitation imposed in the case
of gitlint.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Make sure committers have correct and valid git settings and verify that
the committer idenity matches one of the signed-off-by entries.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
The file was opened as text, to avoid issues with UTF-8 in the future,
make it all binary and encode to UTF-8 correctly.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Check if any board files have changed and build more tests with this
board to uncover any build regressions.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
This addition would catch tests being added or modified and would run
those tests with --all and catch issues with non default board
configurations before they get merged into the tree.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Add initial .shippable.yml for CI integration on github and related
scripts.
Change-Id: I095d125e780bba980e635e218205c8741e753a8e
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
