doc: rename sanitycheck -> twister
Change documentation to use new name for sanitycheck. Signed-off-by: Anas Nashif <anas.nashif@intel.com>
This commit is contained in:
parent
b18f311b3e
commit
99e6a8b999
10 changed files with 54 additions and 54 deletions
|
@ -293,23 +293,23 @@ Note, gitlint only checks HEAD (the most recent commit), so you should run it
|
|||
after each commit, or use the ``--commits`` option to specify a commit range
|
||||
covering all the development patches to be submitted.
|
||||
|
||||
sanitycheck
|
||||
===========
|
||||
twister
|
||||
=======
|
||||
|
||||
.. note::
|
||||
sanitycheck does not currently run on Windows.
|
||||
twister does not currently run on Windows.
|
||||
|
||||
To verify that your changes did not break any tests or samples, please run the
|
||||
``sanitycheck`` script locally before submitting your pull request to GitHub. To
|
||||
``twister`` script locally before submitting your pull request to GitHub. To
|
||||
run the same tests the CI system runs, follow these steps from within your
|
||||
local Zephyr source working directory:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
source zephyr-env.sh
|
||||
./scripts/sanitycheck
|
||||
./scripts/twister
|
||||
|
||||
The above will execute the basic sanitycheck script, which will run various
|
||||
The above will execute the basic twister script, which will run various
|
||||
kernel tests using the QEMU emulator. It will also do some build tests on
|
||||
various samples with advanced features that can't run in QEMU.
|
||||
|
||||
|
@ -474,7 +474,7 @@ workflow here:
|
|||
git checkout -b fix_out_of_date_patch origin/net
|
||||
|
||||
#. Make changes, test locally, change, test, test again, ... (Check out the
|
||||
prior chapter on `sanitycheck`_ as well).
|
||||
prior chapter on `twister`_ as well).
|
||||
|
||||
#. When things look good, start the pull request process by adding your changed
|
||||
files::
|
||||
|
|
|
@ -213,7 +213,7 @@ The CI infrastructure currently runs the following tests:
|
|||
- Run ''checkpatch'' for code style issues (can vote -1 on errors; see note)
|
||||
- Gitlint: Git commit style based on project requirements
|
||||
- License Check: Check for conflicting licenses
|
||||
- Run ''sanitycheck'' script
|
||||
- Run ''twister'' script
|
||||
|
||||
- Run kernel tests in QEMU (can vote -1 on errors)
|
||||
- Build various samples for different boards (can vote -1 on errors)
|
||||
|
|
|
@ -132,24 +132,24 @@ You may postprocess these with your preferred tools. For example:
|
|||
Sanitycheck coverage reports
|
||||
****************************
|
||||
|
||||
Zephyr's :ref:`sanitycheck script <sanitycheck_script>` can automatically
|
||||
Zephyr's :ref:`twister script <twister_script>` can automatically
|
||||
generate a coverage report from the tests which were executed.
|
||||
You just need to invoke it with the ``--coverage`` command line option.
|
||||
|
||||
For example, you may invoke::
|
||||
|
||||
$ sanitycheck --coverage -p qemu_x86 -T tests/kernel
|
||||
$ twister --coverage -p qemu_x86 -T tests/kernel
|
||||
|
||||
or::
|
||||
|
||||
$ sanitycheck --coverage -p native_posix -T tests/bluetooth
|
||||
$ twister --coverage -p native_posix -T tests/bluetooth
|
||||
|
||||
which will produce ``sanity-out/coverage/index.html`` with the report.
|
||||
|
||||
The process differs for unit tests, which are built with the host
|
||||
toolchain and require a different board::
|
||||
|
||||
$ sanitycheck --coverage -p unit_testing -T tests/unit
|
||||
$ twister --coverage -p unit_testing -T tests/unit
|
||||
|
||||
which produces a report in the same location as non-unit testing.
|
||||
|
||||
|
|
|
@ -29,7 +29,7 @@ Examples
|
|||
Example remaining work
|
||||
======================
|
||||
|
||||
- Zephyr's :ref:`sanitycheck_script` currently use :file:`board.yaml` files to
|
||||
- Zephyr's :ref:`twister_script` currently use :file:`board.yaml` files to
|
||||
determine the hardware supported by a board. This should be obtained from
|
||||
devicetree instead.
|
||||
|
||||
|
|
|
@ -241,7 +241,7 @@ Updating or restoring ST-Link firmware
|
|||
|
||||
ST-Link firmware can be updated using `STM32CubeProgrammer Tool`_.
|
||||
It is usually useful when facing flashing issues, for instance when using
|
||||
sanitycheck's device-testing option.
|
||||
twister's device-testing option.
|
||||
|
||||
Once installed, you can update attached board ST-Link firmware with the
|
||||
following command
|
||||
|
@ -250,9 +250,9 @@ following command
|
|||
|
||||
s java -jar ~/STMicroelectronics/STM32Cube/STM32CubeProgrammer/Drivers/FirmwareUpgrade/STLinkUpgrade.jar -sn <board_uid>
|
||||
|
||||
Where board_uid can be obtained using sanitycheck's generate-hardware-map
|
||||
option. For more information about sanitycheck and available options, see
|
||||
:ref:`sanitycheck_script`.
|
||||
Where board_uid can be obtained using twister's generate-hardware-map
|
||||
option. For more information about twister and available options, see
|
||||
:ref:`twister_script`.
|
||||
|
||||
.. _jlink-external-debug-probe:
|
||||
|
||||
|
|
|
@ -330,7 +330,7 @@ Integration tests:
|
|||
* should verify basic usage of the module (configuration,
|
||||
functional APIs, etc.) that is integrated with Zephyr.
|
||||
* shall be built and executed (for example in QEMU) as part of
|
||||
sanitycheck runs in pull requests that introduce changes in module
|
||||
twister runs in pull requests that introduce changes in module
|
||||
repositories.
|
||||
|
||||
.. note::
|
||||
|
@ -540,10 +540,10 @@ Sanitycheck
|
|||
===========
|
||||
|
||||
To execute both tests and samples available in modules, the Zephyr test runner
|
||||
(sanitycheck) should be pointed to the directories containing those samples and
|
||||
(twister) should be pointed to the directories containing those samples and
|
||||
tests. This can be done by specifying the path to both samples and tests in the
|
||||
:file:`zephyr/module.yml` file. Additionally, if a module defines out of tree
|
||||
boards, the module file can point sanitycheck to the path where those files
|
||||
boards, the module file can point twister to the path where those files
|
||||
are maintained in the module. For example:
|
||||
|
||||
|
||||
|
|
|
@ -180,7 +180,7 @@ The optional files are:
|
|||
of your board. You only need this if you're :ref:`contributing-your-board` to
|
||||
Zephyr.
|
||||
- :file:`plank.yaml`: a YAML file with miscellaneous metadata used by the
|
||||
:ref:`sanitycheck_script`.
|
||||
:ref:`twister_script`.
|
||||
|
||||
.. _default_board_configuration:
|
||||
|
||||
|
|
|
@ -7,4 +7,4 @@ Testing
|
|||
:maxdepth: 1
|
||||
|
||||
ztest
|
||||
sanitycheck
|
||||
twister
|
||||
|
|
|
@ -1,7 +1,7 @@
|
|||
.. _sanitycheck_script:
|
||||
.. _twister_script:
|
||||
|
||||
Sanity Tests
|
||||
#############
|
||||
Test Runner (Twister)
|
||||
#####################
|
||||
|
||||
This script scans for the set of unit test applications in the git repository
|
||||
and attempts to execute them. By default, it tries to build each test
|
||||
|
@ -11,16 +11,16 @@ The default options will build the majority of the tests on a defined set of
|
|||
boards and will run in an emulated environment if available for the
|
||||
architecture or configuration being tested.
|
||||
|
||||
In normal use, sanitycheck runs a limited set of kernel tests (inside
|
||||
an emulator). Because of its limited test execution coverage, sanitycheck
|
||||
In normal use, twister runs a limited set of kernel tests (inside
|
||||
an emulator). Because of its limited test execution coverage, twister
|
||||
cannot guarantee local changes will succeed in the full build
|
||||
environment, but it does sufficient testing by building samples and
|
||||
tests for different boards and different configurations to help keep the
|
||||
complete code tree buildable.
|
||||
|
||||
When using (at least) one ``-v`` option, sanitycheck's console output
|
||||
When using (at least) one ``-v`` option, twister's console output
|
||||
shows for every test how the test is run (qemu, native_posix, etc.) or
|
||||
whether the binary was just built. There are a few reasons why sanitycheck
|
||||
whether the binary was just built. There are a few reasons why twister
|
||||
only builds a test and doesn't run it:
|
||||
|
||||
- The test is marked as ``build_only: true`` in its ``.yaml``
|
||||
|
@ -28,7 +28,7 @@ only builds a test and doesn't run it:
|
|||
- The test configuration has defined a ``harness`` but you don't have
|
||||
it or haven't set it up.
|
||||
- The target device is not connected and not available for flashing
|
||||
- You or some higher level automation invoked sanitycheck with
|
||||
- You or some higher level automation invoked twister with
|
||||
``--build-only``.
|
||||
|
||||
These also affect the outputs of ``--testcase-report`` and
|
||||
|
@ -39,21 +39,21 @@ To run the script in the local tree, follow the steps below:
|
|||
::
|
||||
|
||||
$ source zephyr-env.sh
|
||||
$ ./scripts/sanitycheck
|
||||
$ ./scripts/twister
|
||||
|
||||
If you have a system with a large number of cores, you can build and run
|
||||
all possible tests using the following options:
|
||||
|
||||
::
|
||||
|
||||
$ ./scripts/sanitycheck --all --enable-slow
|
||||
$ ./scripts/twister --all --enable-slow
|
||||
|
||||
This will build for all available boards and run all applicable tests in
|
||||
a simulated (for example QEMU) environment.
|
||||
|
||||
The list of command line options supported by sanitycheck can be viewed using::
|
||||
The list of command line options supported by twister can be viewed using::
|
||||
|
||||
$ ./scripts/sanitycheck --help
|
||||
$ ./scripts/twister --help
|
||||
|
||||
|
||||
|
||||
|
@ -149,7 +149,7 @@ testing:
|
|||
|
||||
default: [True|False]:
|
||||
This is a default board, it will tested with the highest priority and is
|
||||
covered when invoking the simplified sanitycheck without any additional
|
||||
covered when invoking the simplified twister without any additional
|
||||
arguments.
|
||||
ignore_tags:
|
||||
Do not attempt to build (and therefore run) tests marked with this list of
|
||||
|
@ -311,7 +311,7 @@ platform_allow: <list of platforms>
|
|||
only be run on the allowed platform and nothing else.
|
||||
|
||||
integration_platforms: <YML list of platforms/boards>
|
||||
This option limits the scope to the listed platforms when sanitycheck is
|
||||
This option limits the scope to the listed platforms when twister is
|
||||
invoked with the --integration option. Use this instead of
|
||||
platform_allow if the goal is to limit scope due to timing or
|
||||
resource constraints.
|
||||
|
@ -320,7 +320,7 @@ platform_exclude: <list of platforms>
|
|||
Set of platforms that this test case should not run on.
|
||||
|
||||
extra_sections: <list of extra binary sections>
|
||||
When computing sizes, sanitycheck will report errors if it finds
|
||||
When computing sizes, twister will report errors if it finds
|
||||
extra, unexpected sections in the Zephyr binary unless they are named
|
||||
here. They will not be included in the size calculation.
|
||||
|
||||
|
@ -457,7 +457,7 @@ filter: <expression>
|
|||
The set of test cases that actually run depends on directives in the testcase
|
||||
filed and options passed in on the command line. If there is any confusion,
|
||||
running with -v or examining the discard report
|
||||
(:file:`sanitycheck_discard.csv`) can help show why particular test cases were
|
||||
(:file:`twister_discard.csv`) can help show why particular test cases were
|
||||
skipped.
|
||||
|
||||
Metrics (such as pass/fail state and binary size) for the last code
|
||||
|
@ -475,7 +475,7 @@ Running in Integration Mode
|
|||
|
||||
This mode is used in continuous integration (CI) and other automated
|
||||
environments used to give developers fast feedback on changes. The mode can
|
||||
be activated using the --integration option of sanitycheck and narrows down
|
||||
be activated using the --integration option of twister and narrows down
|
||||
the scope of builds and tests if applicable to platforms defined under the
|
||||
integration keyword in the testcase definition file (testcase.yaml and
|
||||
sample.yaml).
|
||||
|
@ -485,21 +485,21 @@ Running Tests on Hardware
|
|||
*************************
|
||||
|
||||
Beside being able to run tests in QEMU and other simulated environments,
|
||||
sanitycheck supports running most of the tests on real devices and produces
|
||||
twister supports running most of the tests on real devices and produces
|
||||
reports for each run with detailed FAIL/PASS results.
|
||||
|
||||
|
||||
Executing tests on a single device
|
||||
===================================
|
||||
|
||||
To use this feature on a single connected device, run sanitycheck with
|
||||
To use this feature on a single connected device, run twister with
|
||||
the following new options::
|
||||
|
||||
scripts/sanitycheck --device-testing --device-serial /dev/ttyACM0 -p \
|
||||
scripts/twister --device-testing --device-serial /dev/ttyACM0 -p \
|
||||
frdm_k64f -T tests/kernel
|
||||
|
||||
The ``--device-serial`` option denotes the serial device the board is connected to.
|
||||
This needs to be accessible by the user running sanitycheck. You can run this on
|
||||
This needs to be accessible by the user running twister. You can run this on
|
||||
only one board at a time, specified using the ``--platform`` option.
|
||||
|
||||
|
||||
|
@ -511,7 +511,7 @@ hardware map needs to be created with all connected devices and their
|
|||
details such as the serial device and their IDs if available. Run the following
|
||||
command to produce the hardware map::
|
||||
|
||||
./scripts/sanitycheck --generate-hardware-map map.yml
|
||||
./scripts/twister --generate-hardware-map map.yml
|
||||
|
||||
The generated hardware map file (map.yml) will have the list of connected
|
||||
devices, for example::
|
||||
|
@ -555,9 +555,9 @@ it for every run to get the correct serial devices and status of the devices.
|
|||
With the hardware map ready, you can run any tests by pointing to the map
|
||||
file::
|
||||
|
||||
./scripts/sanitycheck --device-testing --hardware-map map.yml -T samples/hello_world/
|
||||
./scripts/twister --device-testing --hardware-map map.yml -T samples/hello_world/
|
||||
|
||||
The above command will result in sanitycheck building tests for the platforms
|
||||
The above command will result in twister building tests for the platforms
|
||||
defined in the hardware map and subsequently flashing and running the tests
|
||||
on those platforms.
|
||||
|
||||
|
@ -588,7 +588,7 @@ Fixtures are defined in the hardware map file as a list::
|
|||
runner: pyocd
|
||||
serial: /dev/ttyACM9
|
||||
|
||||
When running `sanitycheck` with ``--device-testing``, the configured fixture
|
||||
When running `twister` with ``--device-testing``, the configured fixture
|
||||
in the hardware map file will be matched to testcases requesting the same fixtures
|
||||
and these tests will be executed on the boards that provide this fixture.
|
||||
|
|
@ -15,11 +15,11 @@ Quick start - Integration testing
|
|||
|
||||
A simple working base is located at :zephyr_file:`samples/testing/integration`. Just
|
||||
copy the files to ``tests/`` and edit them for your needs. The test will then
|
||||
be automatically built and run by the sanitycheck script. If you are testing
|
||||
be automatically built and run by the twister script. If you are testing
|
||||
the **bar** component of **foo**, you should copy the sample folder to
|
||||
``tests/foo/bar``. It can then be tested with::
|
||||
|
||||
./scripts/sanitycheck -s tests/foo/bar/test-identifier
|
||||
./scripts/twister -s tests/foo/bar/test-identifier
|
||||
|
||||
|
||||
In the example above ``tests/foo/bar`` signifies the path to the test and the
|
||||
|
@ -27,7 +27,7 @@ In the example above ``tests/foo/bar`` signifies the path to the test and the
|
|||
|
||||
To run all tests defined in a test project, run::
|
||||
|
||||
./scripts/sanitycheck -T tests/foo/bar/
|
||||
./scripts/twister -T tests/foo/bar/
|
||||
|
||||
The sample contains the following files:
|
||||
|
||||
|
@ -95,14 +95,14 @@ Listing Tests
|
|||
|
||||
Tests (test projects) in the Zephyr tree consist of many testcases that run as
|
||||
part of a project and test similar functionality, for example an API or a
|
||||
feature. The ``sanitycheck`` script can parse the testcases in all
|
||||
feature. The ``twister`` script can parse the testcases in all
|
||||
test projects or a subset of them, and can generate reports on a granular
|
||||
level, i.e. if cases have passed or failed or if they were blocked or skipped.
|
||||
|
||||
Sanitycheck parses the source files looking for test case names, so you
|
||||
can list all kernel test cases, for example, by entering::
|
||||
|
||||
sanitycheck --list-tests -T tests/kernel
|
||||
twister --list-tests -T tests/kernel
|
||||
|
||||
Skipping Tests
|
||||
==============
|
||||
|
@ -161,7 +161,7 @@ that interaction.
|
|||
Best practices for declaring the test suite
|
||||
===========================================
|
||||
|
||||
*sanitycheck* and other validation tools need to obtain the list of
|
||||
*twister* and other validation tools need to obtain the list of
|
||||
subcases that a Zephyr *ztest* test image will expose.
|
||||
|
||||
.. admonition:: Rationale
|
||||
|
@ -207,7 +207,7 @@ Here is a generic template for a test showing the expected use of
|
|||
ztest_run_test_suite(common);
|
||||
}
|
||||
|
||||
For *sanitycheck* to parse source files and create a list of subcases,
|
||||
For *twister* to parse source files and create a list of subcases,
|
||||
the declarations of :func:`ztest_test_suite` must follow a few rules:
|
||||
|
||||
- one declaration per line
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue