doc: rename sanitycheck -> twister

Change documentation to use new name for sanitycheck.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
This commit is contained in:
Anas Nashif 2020-12-07 11:46:56 -05:00
commit 99e6a8b999
10 changed files with 54 additions and 54 deletions

View file

@ -293,23 +293,23 @@ Note, gitlint only checks HEAD (the most recent commit), so you should run it
after each commit, or use the ``--commits`` option to specify a commit range
covering all the development patches to be submitted.
sanitycheck
===========
twister
=======
.. note::
sanitycheck does not currently run on Windows.
twister does not currently run on Windows.
To verify that your changes did not break any tests or samples, please run the
``sanitycheck`` script locally before submitting your pull request to GitHub. To
``twister`` script locally before submitting your pull request to GitHub. To
run the same tests the CI system runs, follow these steps from within your
local Zephyr source working directory:
.. code-block:: console
source zephyr-env.sh
./scripts/sanitycheck
./scripts/twister
The above will execute the basic sanitycheck script, which will run various
The above will execute the basic twister script, which will run various
kernel tests using the QEMU emulator. It will also do some build tests on
various samples with advanced features that can't run in QEMU.
@ -474,7 +474,7 @@ workflow here:
git checkout -b fix_out_of_date_patch origin/net
#. Make changes, test locally, change, test, test again, ... (Check out the
prior chapter on `sanitycheck`_ as well).
prior chapter on `twister`_ as well).
#. When things look good, start the pull request process by adding your changed
files::

View file

@ -213,7 +213,7 @@ The CI infrastructure currently runs the following tests:
- Run ''checkpatch'' for code style issues (can vote -1 on errors; see note)
- Gitlint: Git commit style based on project requirements
- License Check: Check for conflicting licenses
- Run ''sanitycheck'' script
- Run ''twister'' script
- Run kernel tests in QEMU (can vote -1 on errors)
- Build various samples for different boards (can vote -1 on errors)

View file

@ -132,24 +132,24 @@ You may postprocess these with your preferred tools. For example:
Sanitycheck coverage reports
****************************
Zephyr's :ref:`sanitycheck script <sanitycheck_script>` can automatically
Zephyr's :ref:`twister script <twister_script>` can automatically
generate a coverage report from the tests which were executed.
You just need to invoke it with the ``--coverage`` command line option.
For example, you may invoke::
$ sanitycheck --coverage -p qemu_x86 -T tests/kernel
$ twister --coverage -p qemu_x86 -T tests/kernel
or::
$ sanitycheck --coverage -p native_posix -T tests/bluetooth
$ twister --coverage -p native_posix -T tests/bluetooth
which will produce ``sanity-out/coverage/index.html`` with the report.
The process differs for unit tests, which are built with the host
toolchain and require a different board::
$ sanitycheck --coverage -p unit_testing -T tests/unit
$ twister --coverage -p unit_testing -T tests/unit
which produces a report in the same location as non-unit testing.

View file

@ -29,7 +29,7 @@ Examples
Example remaining work
======================
- Zephyr's :ref:`sanitycheck_script` currently use :file:`board.yaml` files to
- Zephyr's :ref:`twister_script` currently use :file:`board.yaml` files to
determine the hardware supported by a board. This should be obtained from
devicetree instead.

View file

@ -241,7 +241,7 @@ Updating or restoring ST-Link firmware
ST-Link firmware can be updated using `STM32CubeProgrammer Tool`_.
It is usually useful when facing flashing issues, for instance when using
sanitycheck's device-testing option.
twister's device-testing option.
Once installed, you can update attached board ST-Link firmware with the
following command
@ -250,9 +250,9 @@ following command
s java -jar ~/STMicroelectronics/STM32Cube/STM32CubeProgrammer/Drivers/FirmwareUpgrade/STLinkUpgrade.jar -sn <board_uid>
Where board_uid can be obtained using sanitycheck's generate-hardware-map
option. For more information about sanitycheck and available options, see
:ref:`sanitycheck_script`.
Where board_uid can be obtained using twister's generate-hardware-map
option. For more information about twister and available options, see
:ref:`twister_script`.
.. _jlink-external-debug-probe:

View file

@ -330,7 +330,7 @@ Integration tests:
* should verify basic usage of the module (configuration,
functional APIs, etc.) that is integrated with Zephyr.
* shall be built and executed (for example in QEMU) as part of
sanitycheck runs in pull requests that introduce changes in module
twister runs in pull requests that introduce changes in module
repositories.
.. note::
@ -540,10 +540,10 @@ Sanitycheck
===========
To execute both tests and samples available in modules, the Zephyr test runner
(sanitycheck) should be pointed to the directories containing those samples and
(twister) should be pointed to the directories containing those samples and
tests. This can be done by specifying the path to both samples and tests in the
:file:`zephyr/module.yml` file. Additionally, if a module defines out of tree
boards, the module file can point sanitycheck to the path where those files
boards, the module file can point twister to the path where those files
are maintained in the module. For example:

View file

@ -180,7 +180,7 @@ The optional files are:
of your board. You only need this if you're :ref:`contributing-your-board` to
Zephyr.
- :file:`plank.yaml`: a YAML file with miscellaneous metadata used by the
:ref:`sanitycheck_script`.
:ref:`twister_script`.
.. _default_board_configuration:

View file

@ -7,4 +7,4 @@ Testing
:maxdepth: 1
ztest
sanitycheck
twister

View file

@ -1,7 +1,7 @@
.. _sanitycheck_script:
.. _twister_script:
Sanity Tests
#############
Test Runner (Twister)
#####################
This script scans for the set of unit test applications in the git repository
and attempts to execute them. By default, it tries to build each test
@ -11,16 +11,16 @@ The default options will build the majority of the tests on a defined set of
boards and will run in an emulated environment if available for the
architecture or configuration being tested.
In normal use, sanitycheck runs a limited set of kernel tests (inside
an emulator). Because of its limited test execution coverage, sanitycheck
In normal use, twister runs a limited set of kernel tests (inside
an emulator). Because of its limited test execution coverage, twister
cannot guarantee local changes will succeed in the full build
environment, but it does sufficient testing by building samples and
tests for different boards and different configurations to help keep the
complete code tree buildable.
When using (at least) one ``-v`` option, sanitycheck's console output
When using (at least) one ``-v`` option, twister's console output
shows for every test how the test is run (qemu, native_posix, etc.) or
whether the binary was just built. There are a few reasons why sanitycheck
whether the binary was just built. There are a few reasons why twister
only builds a test and doesn't run it:
- The test is marked as ``build_only: true`` in its ``.yaml``
@ -28,7 +28,7 @@ only builds a test and doesn't run it:
- The test configuration has defined a ``harness`` but you don't have
it or haven't set it up.
- The target device is not connected and not available for flashing
- You or some higher level automation invoked sanitycheck with
- You or some higher level automation invoked twister with
``--build-only``.
These also affect the outputs of ``--testcase-report`` and
@ -39,21 +39,21 @@ To run the script in the local tree, follow the steps below:
::
$ source zephyr-env.sh
$ ./scripts/sanitycheck
$ ./scripts/twister
If you have a system with a large number of cores, you can build and run
all possible tests using the following options:
::
$ ./scripts/sanitycheck --all --enable-slow
$ ./scripts/twister --all --enable-slow
This will build for all available boards and run all applicable tests in
a simulated (for example QEMU) environment.
The list of command line options supported by sanitycheck can be viewed using::
The list of command line options supported by twister can be viewed using::
$ ./scripts/sanitycheck --help
$ ./scripts/twister --help
@ -149,7 +149,7 @@ testing:
default: [True|False]:
This is a default board, it will tested with the highest priority and is
covered when invoking the simplified sanitycheck without any additional
covered when invoking the simplified twister without any additional
arguments.
ignore_tags:
Do not attempt to build (and therefore run) tests marked with this list of
@ -311,7 +311,7 @@ platform_allow: <list of platforms>
only be run on the allowed platform and nothing else.
integration_platforms: <YML list of platforms/boards>
This option limits the scope to the listed platforms when sanitycheck is
This option limits the scope to the listed platforms when twister is
invoked with the --integration option. Use this instead of
platform_allow if the goal is to limit scope due to timing or
resource constraints.
@ -320,7 +320,7 @@ platform_exclude: <list of platforms>
Set of platforms that this test case should not run on.
extra_sections: <list of extra binary sections>
When computing sizes, sanitycheck will report errors if it finds
When computing sizes, twister will report errors if it finds
extra, unexpected sections in the Zephyr binary unless they are named
here. They will not be included in the size calculation.
@ -457,7 +457,7 @@ filter: <expression>
The set of test cases that actually run depends on directives in the testcase
filed and options passed in on the command line. If there is any confusion,
running with -v or examining the discard report
(:file:`sanitycheck_discard.csv`) can help show why particular test cases were
(:file:`twister_discard.csv`) can help show why particular test cases were
skipped.
Metrics (such as pass/fail state and binary size) for the last code
@ -475,7 +475,7 @@ Running in Integration Mode
This mode is used in continuous integration (CI) and other automated
environments used to give developers fast feedback on changes. The mode can
be activated using the --integration option of sanitycheck and narrows down
be activated using the --integration option of twister and narrows down
the scope of builds and tests if applicable to platforms defined under the
integration keyword in the testcase definition file (testcase.yaml and
sample.yaml).
@ -485,21 +485,21 @@ Running Tests on Hardware
*************************
Beside being able to run tests in QEMU and other simulated environments,
sanitycheck supports running most of the tests on real devices and produces
twister supports running most of the tests on real devices and produces
reports for each run with detailed FAIL/PASS results.
Executing tests on a single device
===================================
To use this feature on a single connected device, run sanitycheck with
To use this feature on a single connected device, run twister with
the following new options::
scripts/sanitycheck --device-testing --device-serial /dev/ttyACM0 -p \
scripts/twister --device-testing --device-serial /dev/ttyACM0 -p \
frdm_k64f -T tests/kernel
The ``--device-serial`` option denotes the serial device the board is connected to.
This needs to be accessible by the user running sanitycheck. You can run this on
This needs to be accessible by the user running twister. You can run this on
only one board at a time, specified using the ``--platform`` option.
@ -511,7 +511,7 @@ hardware map needs to be created with all connected devices and their
details such as the serial device and their IDs if available. Run the following
command to produce the hardware map::
./scripts/sanitycheck --generate-hardware-map map.yml
./scripts/twister --generate-hardware-map map.yml
The generated hardware map file (map.yml) will have the list of connected
devices, for example::
@ -555,9 +555,9 @@ it for every run to get the correct serial devices and status of the devices.
With the hardware map ready, you can run any tests by pointing to the map
file::
./scripts/sanitycheck --device-testing --hardware-map map.yml -T samples/hello_world/
./scripts/twister --device-testing --hardware-map map.yml -T samples/hello_world/
The above command will result in sanitycheck building tests for the platforms
The above command will result in twister building tests for the platforms
defined in the hardware map and subsequently flashing and running the tests
on those platforms.
@ -588,7 +588,7 @@ Fixtures are defined in the hardware map file as a list::
runner: pyocd
serial: /dev/ttyACM9
When running `sanitycheck` with ``--device-testing``, the configured fixture
When running `twister` with ``--device-testing``, the configured fixture
in the hardware map file will be matched to testcases requesting the same fixtures
and these tests will be executed on the boards that provide this fixture.

View file

@ -15,11 +15,11 @@ Quick start - Integration testing
A simple working base is located at :zephyr_file:`samples/testing/integration`. Just
copy the files to ``tests/`` and edit them for your needs. The test will then
be automatically built and run by the sanitycheck script. If you are testing
be automatically built and run by the twister script. If you are testing
the **bar** component of **foo**, you should copy the sample folder to
``tests/foo/bar``. It can then be tested with::
./scripts/sanitycheck -s tests/foo/bar/test-identifier
./scripts/twister -s tests/foo/bar/test-identifier
In the example above ``tests/foo/bar`` signifies the path to the test and the
@ -27,7 +27,7 @@ In the example above ``tests/foo/bar`` signifies the path to the test and the
To run all tests defined in a test project, run::
./scripts/sanitycheck -T tests/foo/bar/
./scripts/twister -T tests/foo/bar/
The sample contains the following files:
@ -95,14 +95,14 @@ Listing Tests
Tests (test projects) in the Zephyr tree consist of many testcases that run as
part of a project and test similar functionality, for example an API or a
feature. The ``sanitycheck`` script can parse the testcases in all
feature. The ``twister`` script can parse the testcases in all
test projects or a subset of them, and can generate reports on a granular
level, i.e. if cases have passed or failed or if they were blocked or skipped.
Sanitycheck parses the source files looking for test case names, so you
can list all kernel test cases, for example, by entering::
sanitycheck --list-tests -T tests/kernel
twister --list-tests -T tests/kernel
Skipping Tests
==============
@ -161,7 +161,7 @@ that interaction.
Best practices for declaring the test suite
===========================================
*sanitycheck* and other validation tools need to obtain the list of
*twister* and other validation tools need to obtain the list of
subcases that a Zephyr *ztest* test image will expose.
.. admonition:: Rationale
@ -207,7 +207,7 @@ Here is a generic template for a test showing the expected use of
ztest_run_test_suite(common);
}
For *sanitycheck* to parse source files and create a list of subcases,
For *twister* to parse source files and create a list of subcases,
the declarations of :func:`ztest_test_suite` must follow a few rules:
- one declaration per line