Application log level is not detected correctly as the option is defined
using a template, so it can't be grepped.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
Add specialized tests that execute another test indirectly and ensure
that the results report errors correctly. See the README.rst file
provided with this commit for details.
Signed-off-by: Yuval Peress <peress@google.com>
There were a bunch of old command-line arguments that haven't been used
in a while, remove them.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Tweak check_compliance.py to allow user to run it w/o having
ZEPHYR_BASE set. This is similar to what we do for twister.
Signed-off-by: Kumar Gala <galak@kernel.org>
This is this initial sysbuild guide on what sysbuild is and how to build
a sample using sysbuild.
It provides an architectural overview of sysbuild.
Descriptions on how to use `west build` or `cmake` + `ninja` to build
projects with the sysbuild infrastructure.
Flashing is described through the use of `west flash`.
Extending sysbuild with additional Zephyr based applications are
described, and reference to CMake documentation for including non-Zephyr
based applications are provided.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
There is a breaking change in Junitparser 'TestCase.result' between
v1.x and v2.x.
Update check_compliance.py minimally to take into account this change.
This avoids relying on an outdated package.
Signed-off-by: Henri Xavier <datacomos@huawei.com>
In cases of global changes where 100s of nodes are launched, i.e. on
samples and tests (more than 20 tests/samples changed), do a full
covrage run.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Remove uncrustify configuration file in favor of using clang-format, a
tool that is nowadays more popular/powerful for code formatting.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
This commit fixes the broken links to the Contribution Guidelines in
the Compliance Check script.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
Now that we have partially oerhauled twister in the tree, modify actions
with new options:
- not reporting filtered tests is now default
- output is json, not csv
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
twister now exports json instead of csv, so we make the testplan read
the json data and generate the consolidated plan for the workflow.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Adding LOG_BACKEND_MOCK_OUTPUT_DEFAULT &
LOG_BACKEND_MOCK_OUTPUT_SYST to UNDEF_KCONFIG_WHITELIST as Kconfig
options generated by a template under testcases are not
identified by Compliance check by default.
Signed-off-by: Aastha Grover <aastha.grover@intel.com>
When doing global changes, like typo fixes or header changes, keep
default scope and do not build each test for each platform.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
This reverts commit e61c534e52, which
enabled commit message checking via the checkpatch script.
The checkpatch commit message checking is currently causing more
problems that it fixes, for example:
1. the rules that are supposed to be specific to the "header" (commit
message) are being applied to the diffs.
2. there are some rules that do not fully make sense for the Zephyr
and the way we have been doing things.
Note that we currently have the 'gitlint' checking the commit messages,
so reverting this feature does not completely take away the commit
message checking in the CI.
If we are to ever re-introduce this feature, all of the aforementioned
issues need to be addressed, and the effects and ramifications of
enabling this feature must be thoroughly analysed.
Signed-off-by: Stephanos Ioannidis <root@stephanos.io>
Change CheckPatch() compliance test to call checkpatch.pl directly
rather than through git diff. This has the advantage of including the
commit message in what is being checked, which is useful to catch stuff
like spurious Gerrit commit ids.
The --mailback does not seem to work correctly in this mode, but since
the output is filtered by checking the command exit code anyway, that
option is redundant, so drop that as well.
Signed-off-by: Fabio Baltieri <fabiobaltieri@google.com>
Use the common code in scripts/list_boards.py for finding matching
boards in scripts/ci/test_plan.py. This has the benefit of not trying to
use board revisions files as board names.
Signed-off-by: Henrik Brix Andersen <hebad@vestas.com>
Kconfig symbols must not be defined in Kconfig.defconfig.* files. Such
files are meant to change the default value of certain options.
Signed-off-by: Gerard Marull-Paretas <gerard.marull@nordicsemi.no>
Need a better way or tagging for this area, we have been introducing
regressions that were undetected due to those filters.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
If we are invoking testplan for only one board, do not bother with other
board related changes, just generate plan for the specified platform.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
We were missing smaller sets and skipping tests when the set is smaller
than the allowed number of tests per node.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Improve calculation of matrix and move calculations from workflow to the
testplan script. We now generate a file that can be parsed by the action
with the data needed to start twister with the right number of nodes.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Do not invoke --integration when dealing with one platform only and
generate testplan only for the needed platforms.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Merged 3 files used to generate the test plan per PR based on the
changes. 2 python scripts and a shell script are now all merged into 1
python script that generates the input file for twister based on a list
of changed files by the PR.
This remove lots of old and obsolete code and simplifies things a bit,
no need anymore for an intermediate script to call twister, we call it
directly in the workflow and use the new test_plan script to generate
the test plan.
This also reenables the recently disabled tag based filtering which had
a bug, bug is resolved in this new implementation.
On push events, we now run twister without the --integration option to
catch any issues in the main branch that were not caught in PRs. PRs
continue to run with --integration enabled. This event (push) is now run
on 15 builders due to the increased size.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
This action replaces current buildkite workflow and uses github actions
to build and run tests in the zephyr tree using twister. The main
differences to current builtkite workflow:
- the action handles all 3 events: pull requests, push and schedule
- the action determines size of matrix (number of build hosts) based on
the change with a minimum of 1 builder. If more tests are built/run
due to changes to boards or tests/samples, the matrix size is
increased. This will avoid timeouts when running over capacity due to
board/test changes.
- We use ccache and store cache files on amazon S3 for more flexibility
- Results are collected per build host and merged in the final step and
failures are posted into github action check runs.
- It runs on more powerful instances that can handle more load.
Currently we have 10 build hosts per run (that can increase depending
on number of tests run) and can deliver results within 1 hour.
- the action can deal with non code changes and will not allocate more
than required to deal with changes to documentation and other files
that do not require running twister
The goal long-term is better integrate this workflow with other actions
and not run unncessarily if other workflows have failed, for example, if
commit message is bogus, we should stop at that check, to avoid wasting
resources given that the commit message will have to be fixed anyways
which would later trigger another run on the same code.
Currently there is 1 open issue with this action related to a github
workflow bug where the final results are not posted to the same workflow
and might appear under other workflows. Github is working on this bug.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Many tests and CI activties are being missed by excluding tests
mistakingly when running twister.
This is visibile when you change one or more tests in kernel/ for
example, twister does not run those tests that have changed at all and
marking the PR as tested and ready to be merged.
Temporary fix for #40235.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Fixes: #38463
With 6be1b2af9b then Zephyr shields and
soc Kconfig are always sourced from Kconfig directly.
However, check_compliance.py generates Kconfig files for sourcing the
same files thus sourcing the same files twice.
The end result is a lot of Kconfig warnings as described in #38463.
Removing the generated Kconfig files as this is no longer needed after
6be1b2af9b has been introduced.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
New docs for cavs_v25 describe building a Linux kernel for a
Chromebook and need to talk about these kconfigs.
Signed-off-by: Andy Ross <andrew.j.ross@intel.com>
Support for ARM Compiler C library.
This commit add support for the ARM Compiler C libary in:
- Kconfig
- libc/armstdc
A new Kconfig symbol is added to allow a toolchain to specify if they
support linking with the minimal C library.
Also the CMake variable `TOOLCHAIN_HAS_NEWLIB` is exported to Kconfig
so that CONFIG_NEWLIB_LIBS can only be enabled if the toolchain has
newlib.
The armclang toolchain selects the CMake scatter file generator and
disables support for the LD linker template which is not supported by
armlink.
For the ARM Compiler C library, a corresponding lib/libc/armstc/ folder
with a minimal implementation to work with the ARM Compiler C library
is added.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
The filter_py() function is handed a bunch of file names and is
expected to give back just the python files.
Its input is the output from a 'git diff' command which normally when
used in the vanilla zephyr repository just prints regular files.
In situations where we're checking compliance on a repository with
submodules, however, filter_py() can be given directories in the
'fnames' list from the git output.
In the interests of being defensive and sharing infrastructure, just
handle this case in filter_py().
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
Take the int-pin information (i.e. what pin between INT1
and INT2 the drdy is attached to) directly from DT.
Signed-off-by: Armando Visconti <armando.visconti@st.com>
Remove includes from the tag file. If we have a change to an include
that implies some API change which may show up somewhere else so we
want to build as many tests/samples as possible in that case.
Also added a comment about keeing entries sorted alphabetically.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
We were testing modified_tags.args before we called get_twister_opt.py
so it would never have been created and thus twister_exclude_tag_opt
would never get set correctly. Move the check to after we call
get_twister_opt.py
Also we need to remove modified_tags.args once we are done with it.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Add the ability to map file/dir paths of a PR to twister TAG. We
introduce scripts/ci/tags.yaml to conveys which files are associated
with which tag.
Since not all file/tags will be specified in tags.yaml we use the
combination of the files modified list and the tags.yaml information to
determine which tags can be excluded (ie if the file list doesn't match
any file in tags.yaml for a given tag listed there, we can that exclude
it).
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Treat all test_file_* files the same and just explicitly set the header
in the final test_file.txt. This makes the logic the same regardless of
what paths tests are coming from.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
If the local option (-l) was specified we were ignoring the results
of what_changed.py and always setting a "FULL" build. Only explicitly
set a FULL run if we are building for a commit to the tree (local and
PR builds should respect the output of what_changed.py).
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Since we use 'main' now, rename the example comment usage to use
that branch name instead of 'master'.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Add a very simple check that verifies that new error numbers are added
according to the errno.h file in newlib, in order to maintain
compatibility with it.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Added SAMPLE_MODULE_LOG_LEVEL_DBG as an exception. Define is used
in the log_api test suite.
Signed-off-by: Krzysztof Chruscinski <krzysztof.chruscinski@nordicsemi.no>
This scripts receives the same parameter of what_changed.py. And run
coccinelle scripts for code guideline compliance in the given git
commits. e.g: ./guideline_check.py --commits origin/master..HEAD
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
BabbleSim execution was (AFAICT mistakenly) disabled in
dda6a5ee90d0f5b31e7d269761dc2ab1a2f8510c. Re-enable it so we run them in
CI again.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
If files are changed for architecture code, then only run CI on those
architectures.
Combine all scripts into one and make it simpler.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
This commit lets the Kconfig compliance check run from top of current
git repo instead of only executing inside ZEPHYR_BASE.
This extends the usability of the Kconfig compliance check.
Together with this possibility, a KconfigBasic mode has been added which
will execute the same verification for symbols in the Kconfig tree but
will not check for undefined symbols outside Kconfig.
This is needed as running Kconfig undef check outside the tree in
non-Zephyr repos might pickup `CONFIG_` named symbols in other code.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Changes to .buildkite/daily.yml don't make any sense to run twister
for since issues will only be seen when the daily builds run so add
it to the twister_ignore.txt list.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
The introduction of Zephyr module glue code in the Zephyr repository
introduces a Kconfig variable in the form of:
`config ZEPHYR_<MODULE_NAME>_MODULE`.
All Kconfig variables go into `autoconf.h`, therefore it is necessary
to sanitize the Kconfig variable, so that it does not contain special
characters. To ensure consistent variable name, then the module name
will be sanitized in all variable use in both Kconfig and CMake.
The sanitization is done be replacing all special characters with an
underscore, `_`.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Fixes: #31436
This fix adds ZEPHYR_<MODULE_NAME>_KCONFIG variable to compliance check.
This ensures that Zephyr Kconfig glue code is included in the
compliance check, and thus remove the issue reported in #31436.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Likely an accident when commit 288ae28c13 moved the script to the main
repo. Stayed unnoticed because this handler rarely ever runs.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
We made a similar change for the nightly builds, but the PR builds
invoke run_ci.sh, so we need the same change here.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
Fixes: #28462
This commit allows shields to be defined in other BOARD_ROOTs, either
using `-DBOARD_ROOT=<path>` or a Zephyr module defined BOARD_ROOT.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Introduce TagoIO IoT Cloud HTTP post client example. This explorer
Zephyr network resources to demonstrate an end to end application.
The TagoIO allows that any user can test on a easy way Ethernet, WIFI
and Modem (PPP) with BSD sockets. The example provides overlays to
configure WIFI and Modem.
The application consists an a pseudo temperature sensor that sends
periodically data to TagoIO IoT Cloud platform. The data can be
visualized on a web browser dashboard, cellphone or tablet. The
steps to configure TagoIO are described on the example documentation.
Special Variables:
- CONFIG_TAGOIO_DEVICE_TOKEN DEVID token generated by TagoIO
- CONFIG_TAGOIO_HTTP_WIFI_SSID SSID when using WIFI
- CONFIG_TAGOIO_HTTP_WIFI_PSK PASSWD when using WIFI
- CONFIG_MODEM_GSM_UART_NAME UART label when using MODEM
- CONFIG_MODEM_GSM_APN APN when using MODEM
Signed-off-by: Gerson Fernando Budke <nandojve@gmail.com>
Add date and if a commit should be considered for weekly testing.
This will still work with the old format.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Use the pytest test framework in the dtlib.py and edtlib.py test
suites (testdtlib.py and testedtlib.py respectively).
The goal here is not to change what is being tested. The existing test
suite is excellent and very thorough.
However, it is made up of executable scripts where all of the tests
are run using a hand-rolled framework in a single function per file.
This is a bit all-or-nothing and prevents various nice features
available in the de-facto standard pytest test framework from being
used.
In particular, pytest can:
- drop into a debugger (pdb) when there is a problem
- accept a pattern which specifies a subset of tests to run
- print very detailed error messages about the actual and expected
results in various traceback formats from brief to very verbose
- gather coverage data for the python scripts being tested (via plugin)
- run tests in parallel (via plugin)
- It's easy in pytest to run tests with temporary directories
using the tmp_path and other fixtures. This us avoid
temporarily dirtying the working tree as is done now.
Moving to pytest lets us leverage all of these things without any loss
in ease of use (in fact, some things are nicer in pytest):
- Any function that starts with "test_" is automatically picked up and
run. No need for rolling up lists of functions into a test suite.
- Tests are written using ordinary Python 'assert'
statements.
- Pytest magic unpacks the AST of failed asserts to print details on
what went wrong in really nice ways. For example, it will show you
exactly what parts of two strings that are expected to be equal
differ.
For the most part, this is a pretty mechanical conversion:
- extract helpers and test cases into separate functions
- insert temporary paths and adjust tests accordingly to not match
file names exactly
- use 'assert CONDITION' instead of 'if not CONDITION: fail()'
There are a few cases where making this happen required slightly
larger changes than that, but they are limited.
Move the checks from check_compliance.py to a new GitHub workflow,
removing hacks that are no longer needed.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
This commit introduces support for multiple SOC_ROOT.
This means that additional SOC_ROOTs specified using -DSOC_ROOT as
argument to CMake will be forming a list together with ${ZEPHYR_BASE}.
This allows for greater flexibility, as developers can now specify
multiple out-of-tree SoCs and not worry about the SoC used for the
board they compile for.
Also it avoid code, such as:
if(BOARD STREQUAL my_board_using_out_of_tree_soc)
set(SOC_ROOT some/out/of/tree/soc/path)
endif()
in application CMakeLists.txt.
Finally, allowing multiple SOC_ROOTs prepares for specifying SOC_ROOTs
in Zephyr modules.
Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>
Now that kconfigfunctions.py is loading the EDT object from a pickle
file, I can't find any other reads of the environment variables it was
previously using to do that.
(The CMake variables with the same names are read in different places;
I'm specifically talking about environment variables here.)
Remove the dead stores.
Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
BT simulator tests run standalone and independent of sanitycheck and
might be triggered by tests that are not managed by sanitycheck, so
leave it alone.
Fixes#26508
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
* Tweak west_setup:
- log `west update` to a file to reduce noise in log
- use `west forall` + `git reset` to make sure files are checked out
(this is to handle a possible module cache)
* Output when we start sanity_check. Add a banner for when we run
sanity_check so it's a bit easier to find in console logs
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
When files were added, matching did not work and we were counting added
files twice. Use sets to make sure we have a unique list of files.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
if we are changing a test or a sample, there is not need to run
everything, it is enough to just build/run the changed test/sample.
Also, some files have impact on the code and if they are being changed,
do not run fully sanitycheck.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Script to detect if full sanitycheck should run or if we can skip it and
just run the code that actuallt changed (in samples/tests).
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
When merging files, make sure to remove header rows from each file and
just keep one header. This will stop us from parsing header rows as
tests.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Replace running west command tests in run_ci.sh with a github workflow.
This provides some benefits in that we can run the west command tests on
multiple python versions and host OSes (linux, mac and windows).
Also have the benefit that the tests are only run on modifications to
files in scripts/west_commands/ or scripts/west-commands.yml.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
So that the EDTT tool could be used by this script,
set its path so other scripts could find it
Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
With timestamps enabled, the log in shippable is very difficult to read,
so remove those and enable them when needed for debugging only.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
If modules announce they have tests, run sanitycheck on those modules.
This will run on first matrix node and as the last step after running
regular testsuite.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Fetch HW models from a new west module.
And, remove all pre-west glue which was used to:
* Fetch them in CI
* Validate their vesion
* Modify the include path and link to them
Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
Make sure we cleanup the manifest files at the right spot of the CI
process. When not building a PR, behavior is different.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
For cases like zephyr-test or for other reasons the repo dir might not
be called zephyr. Just use the basename of the dir instead of assuming
its called zephyr.
Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
pylint does not like how this library works and generates spurious
warnings like
scripts/ci/get_modified_tests.py:55:13: E1121: Too many positional
arguments for function call (too-many-function-args)
scripts/ci/get_modified_tests.py:57:13: E1123: Unexpected keyword
argument '_tty_out' in function call (unexpected-keyword-arg)
These warnings are useful enough to want to have enabled in the upcoming
pylint CI check, so suppress them here.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
When running run_ci.sh locally with:
./scripts/ci/run_ci.sh -l -b master -R upstream/master..
we export BSIM_OUT_PATH unconditionally which causes sanitycheck to fail
on nrf52_bsim (it depends on BSIM_OUT_PATH variable).
Check if the path defined in BSIM_OUT_PATH is available and unset env.
variable if it is not.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
The main change is the elimination of the bootstrapper, a design flaw
/ misfeature.
Update the documentation to be compatible with the 0.6.x releases as
well. This has to be done atomically, as there were incompatible
changes. Make use of the versionchanged and versionadded directives
to begin keeping track of how these APIs are evolving.
(Note that west 0.6.0 will remain compatible with the extension
commands in Zephyr v1.14 LTS as long as that is still alive. This
change is targeted towards Zephyr 2.0 users.)
This requires a bump in the shippable container and allows us to
simplify the west_commands test procedure.
Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
Found a few annoying typos and figured I better run script and
fix anything it can find, here are the results...
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
We not analysing coverage data at all in CI right now. Disable this
while we figure out a better solution for reporting data.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
When detecting changes to boards, make sure we test all board
configurations available in that board directory, not only the main one.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Update the files which contain no license information with the
'Apache-2.0' SPDX license identifier. Many source files in the tree are
missing licensing information, which makes it harder for compliance
tools to determine the correct license.
By default all files without license information are under the default
license of Zephyr, which is Apache version 2.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
The code coverage reports on pull requests has been inconsistent and
confusing. Disable them for pull-requests.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Fixes pylint warnings like this one:
doc/conf.py:325:0: W1401: Anomalous backslash in string: '\s'.
String constant might be missing an r prefix.
(anomalous-backslash-in-string)
The reason for this warning is that backslash escapes are interpreted in
non-raw (non-r-prefixed) strings. For example, '\a' and r'\a' are not
the same string (first one has a single ASCII bell character, second one
has two characters).
It just happens that there's no \s (or \., or \/) escape for example,
and '\s' turns into two characters (as needed for a regex). It's risky
to rely on stuff like that regexes though. Best to make them raw strings
unless they're super trivial.
Also note that '\s' and '\\s' turn into the same string.
Another tip: A literal ' can be put into a string with "blah'blah"
instead of 'blah\'blah'.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
This is useful when running the script locally on a set of commits to
reproduce how things are run in CI.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Zephyr Continuous Integration made the choice of rebasing pull requests
on its fast moving master branch before testing them. Unfortunately the
Shippable interface assumes the other choice[*] and highlights the SHA1
of the pull request _before_ rebasing which is very misleading. For now
the only way to realize this disconnect is to scrutinize the build logs
and/or this source code (or tribal knowledge).
To mitigate and reduce a bit this confusion, run a short:
git log -n 5 --oneline --decorate
before and after rebase.
Making it worse, Shippable sub-runs are not pegged to a common base
version. For instance the reason why shippable sub-run 35578.1 passed
while all other 35578.x sub runs failed is very likely because PR #13803
was merged to master in the mean time. This commit would have been able
to prove that.
[*] In an ideal world with infinite time and resources, Continuous
Integration would test pull requests BOTH alone and combined with the
moving tip.
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
On (at least) Fedora "pytest" is python2 and python2 only.
Also update remote name in sample invocation from "upstream" to "origin"
because that's how west sets things up by default (and this script
requires west)
Signed-off-by: Marc Herbert <marc.herbert@intel.com>
When we change documentation or board images, do not build all tests for
this board. Only add more tests when actual code has changed.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
When we change documentation or board images, do not build all tests for
this board. Only add more tests when actual code has changed.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Allow running script locally. For example, on a local tree with local
changes:
./scripts/ci/run_ci.sh -b master -r upstream -l
This will be have the same way as in CI and creates a test manifest
based on changes.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
We should not abort if west was initiaized, someone must have
initialized, i.e. locally and everything is already setup for us and
ready to go.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Incorporate these into run_ci.sh the same way that btsim results were
done. This adds a dependency on pytest.
Signed-off-by: Marti Bolivar <marti@foundries.io>
Commit 88ece494 ("ci: do not build docs in main CI job") removed the
call to build_docs but not the function itself. That makes it
confusing to try to find where it's called; remove it.
Signed-off-by: Marti Bolivar <marti@foundries.io>
The new west is able to initialize itself from an existing zephyr clone
without modifying it, adapt the CI script accordingly.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
West right now modifies the zephyr tree and points to master. In CI we
need to point to the pull-request HEAD instead, so so whatever shippable
does after cloning with west.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
In order for check-compliance.py to be able to run cleanly out-of-tree,
set the repo (to be checked) path based on cwd, and refer to Zephyr and
its tools by the specific path set by ZEPHYR_BASE.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Before run_ci.sh was introduced, if some command in the
.shippable.yml script failed, Shippable would stop and report
a failure.
When run_ci.sh was introduced we lost this, which is pretty
dangerous if something fails cathastrophically and the xml
reports are not (properly) generated and copied, as then
Shippable reports a pass.
=> Run run_ci.sh in -e mode, so we propagate a bad failure upwards
Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
Added a missing option in run_ci.sh to control the placement of the
xml results file. (Before it was just propagated thru the enviroment)
+
Changed .shippable to use this option when callign it.
Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
This reverts commit 6d5ac1c089.
Looks like this was not correctly done, building btsim is needed on all
nodes.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Building docs and running various checks is now being done in another CI
job that runs in parallel and reports directly to the PR.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
- Support sending status to github
- Support license checking
- Make every test a class
- Use junitparser instead of custom xml for junit output
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Compile and run the tests avaliable in bsim_bt
and collect the coverage results into the coverage report.
Also, detect if bsim's component folder already contains the
nRF52 HW models, and if it does instead of trying to fetch
them again (which will fail) check that the right versio is
present. This should ease testing locally.
Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
move CI scripting to a dedicated script. For example, to simulate what
is run in CI when a pull request is submitted:
./scripts/ci/run_ci.sh -b master -r upstream -p
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
To avoid having to set it externally, set the SOC_DIR environment
variable directly in the compliance script.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
The prefixes might be a leftover from the old 'option env="..."' symbols
(which are no longer needed). Since environment variables can be
referenced directly now, there's no point in having a prefix.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Threw an exception about Kconfig files not being found. They need to be
looked up relative to ZEPHYR_BASE.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Use the new KCONFIG_STRICT Kconfiglib functionality, which makes the
library itself warn for references to undefined Kconfig symbols.
Also get rid of the chdir() hack. Kconfiglib now only looks up Kconfig
files relative to $srctree when it's set, so the hack is no longer
needed.
Remove list_undef_kconfig_refs.py. It's no longer needed.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
External projects might run check-compliance.py from a subdirectory,
while list_undef_kconfig_refs.py expects to find the top-level Kconfig
file in the current directory.
Due to obscure Kconfig ugliness (explained in a comment), the working
directory needs to be changed here, instead of using 'srctree'.
(Thinking of changing that Kconfig behavior so that it only applies to
.config files. Suspect it's been a bug in the C tools all along for
Kconfig files.)
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Update Kconfiglib to get upstream commit eb6c21a9b33a2 ("Turn
MenuNode/Symbol/Choice.referenced() into a @property") in. It converts
the MenuNode.referenced() function into a property, which makes the API
more consistent (read-only stuff uses properties).
Also update scripts/ci/list_undef_kconfig_refs.py to access .referenced
as a property.
Piggyback a small is_num() simplification.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Add a helper module scripts/ci/list_undef_kconfig_refs.py that searches
the entire Kconfig tree and reports any references to undefined Kconfig
symbols. Use it to add a new check to scripts/ci/check-compliance.py.
Also allow list_undef_kconfig_refs.py to be run standalone.
Example error:
Error: Found references to undefined Kconfig symbols:
BAR
===
- Referenced at Kconfig:12:
config FOO
bool
depends on BAR
- Referenced at Kconfig:16:
menu "menu"
depends on BAR
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Allow check-compliance.py to be run from any path with a valid Git tree,
removing the unnecessary and artificial limitation imposed in the case
of gitlint.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Make sure committers have correct and valid git settings and verify that
the committer idenity matches one of the signed-off-by entries.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
The file was opened as text, to avoid issues with UTF-8 in the future,
make it all binary and encode to UTF-8 correctly.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Check if any board files have changed and build more tests with this
board to uncover any build regressions.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
This addition would catch tests being added or modified and would run
those tests with --all and catch issues with non default board
configurations before they get merged into the tree.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Add initial .shippable.yml for CI integration on github and related
scripts.
Change-Id: I095d125e780bba980e635e218205c8741e753a8e
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
