michaelh/zephyr
1
0
Fork 0
Code Releases Activity

doc: improve west's repo-tool.rst

Browse source 
Various improvements to the west repo-tool.rst page and pages that
link to it:

Rename the page title to "Basics", since it documents, well the
basics, including built-in commands like "help" an "config" that are
not directly related to multiple repositories.

The "multi-repo" term was invented before we started using
"workspace". Drop it from the text and rework things using words like
"workspace", "basics", or "built-ins" instead, which read better.

We've been using west for a long enough time that justifying its
existence prominently at the start of this page is no longer
necessary; move that to the "dustbin of history" page (why.rst).

Try harder to clarify exactly what a "project" is, along with other
workspace related clarifications.

Improve the documentation for the west init and west update commands,
and create :ref: targets for linking directly to them for convenience
elsewhere in the docs.

Slim down the usages for other built-in commands, so we don't have to
keep those up to date as carefully each release. This is about to be
important for west 0.9, which is going to change the detailed
semantics for the "[PROJECT ...]" part of each command synopsis in a
way that will be inconvenient to duplicate for each of these.

Move the "Topologies supported" content lower down. Try to help the
reading flow by letting people get familiar with a workspace and what
you can do with it using the Zephyr GSG workspace that they probably
already have before overwhelming them with other possibilities and
choices.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
This commit is contained in:
Martí Bolívar 2021-01-06 00:30:12 -08:00 • committed by Carles Cufí
commit 7097e1785d
4 changed files with 268 additions and 204 deletions
Download patch file Download diff file Expand all files Collapse all files

2
doc/guides/west/config.rst
View file

@ -142,7 +142,7 @@ commands are documented in the pages for those commands.
* - ``update.fetch``
- String, one of ``"smart"`` (the default behavior starting in v0.6.1) or
``"always"`` (the previous behavior). If set to ``"smart"``, the
:ref:`west update <west-multi-repo-cmds>` command will skip fetching
:ref:`west-update` command will skip fetching
from project remotes when those projects' revisions in the manifest file
are SHAs or tags which are already available locally. The ``"always"``
behavior is to unconditionally fetch from the remote.

2
doc/guides/west/release-notes.rst
View file

@ -149,7 +149,7 @@ v0.6.1
The user-visible features in this point release are:
- The `west update <west-multi-repo-cmds>` command has a new ``--fetch``
- The :ref:`west-update` command has a new ``--fetch``
command line flag and ``update.fetch`` :ref:`configuration option
<west-config>`. The default value, "smart", skips fetching SHAs and tags
which are available locally.

464
doc/guides/west/repo-tool.rst
View file

@ -1,79 +1,289 @@
.. _west-multi-repo:
Multiple Repository Management
##############################
Basics
######
This page introduces basic concepts related to west and its multiple repository
management features, and gives an overview of the associated commands. See
:ref:`west-history` and `Zephyr issue #6770`_ for additional discussion,
rationale, and motivation.
.. note::
West's multi-repo commands are meant to augment Git in minor ways for
multi-repo work, not to replace it. For tasks that only operate on one
repository, just use plain Git commands.
This page introduces west's basic concepts and built-in commands along with
references to further reading.
.. _west-workspace:
Introduction
************
West's built-in commands allow you to work with projects composed of multiple
Git repositories installed under a common parent directory, which we call a
*west workspace* (before west 0.7, this was called a *west installation*). This
works similarly to `Git Submodules
<https://git-scm.com/book/en/v2/Git-Tools-Submodules>`_ and Google's `repo
<https://gerrit.googlesource.com/git-repo/>`_.
West's built-in commands allow you to work with *projects* (Git
repositories) under a common *workspace* directory. You can create a
workspace using the :ref:`west-init` command.
A west workspace is the result of running the ``west init`` command, which
is described in more detail below. For upstream Zephyr, the workspace looks
like this:
If you've followed the upstream Zephyr getting started guide, your
workspace looks like this:
.. code-block:: none
zephyrproject/ # west topdir
├── .west/
│ └── config
├── zephyr/ # .git/ │
│ ├── west.yml # manifest │ never modified by west
│ └── [... other files ...] │
├── .west/ # marks the location of the topdir
│ └── config # per-workspace local configuration file
├── zephyr/ # .git/ repo │ the manifest repository,
│ ├── west.yml # manifest file │ never modified by west
│ └── [... other files ...] │ after creation
├── modules/
│ └── lib/
│ └── tinycbor/ # .git/ project
├── net-tools/ # .git/ project
└── [ ... other projects ...]
Above, :file:`zephyrproject` is the name of the west workspace's root
directory. This name is just an example -- it could be anything, like ``z``,
``my-zephyr-workspace``, etc. The file :file:`.west/config` is the
workspace's :ref:`local configuration file <west-config>`.
Above, :file:`zephyrproject` is the name of the workspace's top level
directory, or *topdir*. (The name :file:`zephyrproject` is just an example
-- it could be anything, like ``z``, ``my-zephyr-workspace``, etc.)
The topdir contains the :file:`.west` directory. When west needs to find
the topdir, it searches for :file:`.west`, and uses its parent directory.
The search starts from the current working directory (and starts again from
the location in the :envvar:`ZEPHYR_BASE` environment variable as a
fallback if that fails). The file :file:`.west/config` is the workspace's
:ref:`local configuration file <west-config>`.
Every west workspace contains exactly one *manifest repository*, which is a
Git repository containing a file named :file:`west.yml`, which is the *west
manifest*. The location of the manifest repository is given by the
:ref:`manifest.path configuration option <west-config-index>` in the local
configuration file. The manifest file, along with west's configuration files,
controls the workspace's behavior. For upstream Zephyr, :file:`zephyr` is
the manifest repository, but you can configure west to use any Git repository
in the workspace as the manifest repository. The only requirement is that it
contains a valid manifest file. See :ref:`west-manifests` for more details on
what this means.
Git repository containing a *manifest file*. The location of the manifest
repository is given by the :ref:`manifest.path configuration option
<west-config-index>` in the local configuration file.
Both of the :file:`tinycbor` and :file:`net-tools` directories are *projects*
managed by west, and configured in the manifest file. A west workspace can
contain arbitrarily many projects. As shown above, projects can be located
anywhere in the workspace. They don't have to be subdirectories of the
manifest directory, and they can be inside of arbitrary subdirectories inside
the workspace's root directory. By default, the Zephyr build system uses
west to get the locations of all the projects in the workspace, so any code
they contain can be used by applications. This behavior can be overridden using
the ``ZEPHYR_MODULES`` CMake variable; see :ref:`modules` for details.
For upstream Zephyr, :file:`zephyr` is the manifest repository, but you can
configure west to use any Git repository in the workspace as the manifest
repository. The only requirement is that it contains a valid manifest file.
See :ref:`west-topologies` for information on other options, and
:ref:`west-manifests` for details on the manifest file format.
Finally, any repository managed by a west workspace can contain
:ref:`extension commands <west-extensions>`, which are extra west commands
provided by that project. This includes the manifest repository and any project
repository.
The manifest file is a YAML file that defines *projects*, which are the
additional Git repositories in the workspace managed by west. The manifest
file is named :file:`west.yml` by default; this can be overridden using the
``manifest.file`` local configuration option.
You use the :ref:`west-update` command to update the workspace's projects
based on the contents of the manifest file. The manifest file controls
things like where the project should be placed within the workspace, what
URL to clone it from if it's missing, and what Git revision should be
checked out locally.
Projects can be located anywhere inside the workspace, but they may not
"escape" it. Project repositories need not be located in subdirectories of
the manifest repository or as immediate subdirectories of the topdir.
However, projects must have paths inside the workspace. (You may replace a
project's repository directory within the workspace with a symbolic link to
elsewhere on your computer, but west will not do this for you.)
A workspace can contain additional Git repositories or other files and
directories not managed by west. West basically ignores anything in the
workspace except :file:`.west`, the manifest repository, and the projects
specified in the manifest file.
For upstream Zephyr, ``tinycbor`` and ``net-tools`` are projects. They are
specified in the manifest file :file:`zephyr/west.yml`. This file specifies
that ``tinycbor`` is located in the :file:`modules/lib/tinycbor` directory
beneath the workspace topdir. By default, the Zephyr :ref:`build system
<build_overview>` uses west to get the locations of all the projects in the
workspace, so any code they contain can be used as :ref:`modules`.
Finally, any repository managed by a west workspace (either the manifest
repository or any project repository) can define :ref:`west-extensions`.
Extensions are extra commands not built into west that you can run when
using that workspace.
The zephyr repository uses this feature to provide Zephyr-specific commands
like :ref:`west build <west-building>`. Defining these as extensions keeps
west's core agnostic to the specifics of any workspace's Zephyr version,
etc.
.. _west-struct:
Structure
*********
West's code is distributed via PyPI in a Python package named ``west``.
This distribution includes a launcher executable, which is also named
``west`` (or ``west.exe`` on Windows).
When west is installed, the launcher is placed by :file:`pip3` somewhere in
the user's filesystem (exactly where depends on the operating system, but
should be on the ``PATH`` :ref:`environment variable <env_vars>`). This
launcher is the command-line entry point to running both built-in commmands
like ``west init``, ``west update``, along with any extensions discovered
in the workspace.
In addition to its command-line interface, you can also use west's Python
APIs directly. See :ref:`west-apis` for details.
.. _west-manifest-rev:
The ``manifest-rev`` branch
***************************
West creates and controls a Git branch named ``manifest-rev`` in each
project. This branch points to the revision that the manifest file
specified for the project at the time :ref:`west-update` was last run.
Other workspace management commands may use ``manifest-rev`` as a reference
point for the upstream revision as of this latest update. Among other
purposes, the ``manifest-rev`` branch allows the manifest file to use SHAs
as project revisions.
Although ``manifest-rev`` is a normal Git branch, west will recreate and/or
reset it on the next update. For this reason, it is **dangerous**
to check it out or otherwise modify it yourself. For instance, any commits
you manually add to this branch may be lost the next time you run ``west
update``. Instead, check out a local branch with another name, and either
rebase it on top of a new ``manifest-rev``, or merge ``manifest-rev`` into
it.
.. note::
West does not create a ``manifest-rev`` branch in the manifest repository,
since west does not manage the manifest repository's branches or revisions.
.. _west-built-in-cmds:
Built-in Commands
*****************
This section gives an overview of west's built-in commands.
Some commands are related to Git commands with the same name, but operate
on the entire workspace. For example, ``west diff`` shows local changes in
multiple Git repositories in the workspace.
Some commands take projects as arguments. These arguments can be project
names as specified in the manifest file, or (as a fallback) paths to them
on the local file system. Omitting project arguments to commands which
accept them (such as ``west list``, ``west forall``, etc.) usually defaults
to using all projects in the manifest file plus the manifest repository
itself.
The following documentation does not exhaustively describe all commands.
For additional help, run ``west <command> -h`` (e.g. ``west init -h``).
.. _west-init:
``west init``
=============
This command creates a west workspace. It can be used in two ways:
1. Cloning a new manifest repository from a remote URL
2. Creating a workspace around an existing local manifest repository
**Option 1**: to clone a new manifest repository from a remote URL, use:
.. code-block:: none
west init [-m URL] [--mr REVISION] [--mf FILE] [directory]
The new workspace is created in the given :file:`directory`, creating a new
:file:`.west` inside this directory. You can give the manifest URL using
the ``-m`` switch, the initial revision to check out using ``--mr``, and
the location of the manifest file within the repository using ``--mf``.
For example, running:
.. code-block:: shell
west init -m https://github.com/zephyrproject-rtos/zephyr --mr v1.14.0 zp
would clone the upstream official zephyr repository into :file:`zp/zephyr`,
and check out the ``v1.14.0`` release. This command creates
:file:`zp/.west`, and set the ``manifest.path`` :ref:`configuration option
<west-config>` to ``zephyr`` to record the location of the manifest
repository in the workspace. The default manifest file location is used.
The ``-m`` option defaults to
``https://github.com/zephyrproject-rtos/zephyr``. The ``--mr`` option
defaults to ``master``. The ``--mf`` option defaults to ``west.yml``. If no
``directory`` is given, the current working directory is used.
**Option 2**: to create a workspace around an existing local manifest
repository, use:
.. code-block:: none
west init -l [--mf FILE] directory
This creates :file:`.west` **next to** :file:`directory` in the file
system, and sets ``manifest.path`` to ``directory``.
As above, ``--mf`` defaults to ``west.yml``.
The ``west init`` command does not clone any of the projects defined in the
manifest file, regardless of whether ``-l`` is given. To do that, use
``west update``.
.. _west-update:
``west update``
---------------
This command clones and updates the projects specified in the :term:`west
manifest` file.
.. code-block:: none
west update [-h] [--stats] [-f {always,smart}] [-k] [-r] [PROJECT ...]
By default, this command:
#. Parses the manifest file, usually :file:`west.yml`
#. Clones any project repositories that are not already present locally
#. Fetches any project revisions which are not already pulled from the
remote
#. Sets each project's :ref:`manifest-rev <west-manifest-rev>` branch to the
revision specified for that project in the manifest file
#. Checks out each ``manifest-rev`` in local working trees, as `detached
HEADs <https://git-scm.com/docs/git-checkout#_detached_head>`_
To operate on a subset of projects only, specify them using the ``PROJECT``
positional arguments, which can be either project names as given in the
manifest file, or paths to the local project clones.
To force this command to fetch from project remotes even if the revisions
appear to be available locally, either use ``--fetch always`` or set the
``update.fetch`` :ref:`configuration option <west-config>` to ``"always"``.
For safety, ``west update`` uses ``git checkout --detach`` to check out a
detached ``HEAD`` at the manifest revision for each updated project,
leaving behind any branches which were already checked out. This is
typically a safe operation that will not modify any of your local branches.
If you would rather rebase any locally checked out branches instead, use
the ``-r`` (``--rebase``) option.
If you would like ``west update`` to keep local branches checked out as
long as they point to commits that are descendants of the new
``manifest-rev``, use the ``-k`` (``--keep-descendants``) option.
.. _west-multi-repo-misc:
Other Repository Management Commands
====================================
West has a few more commands for managing the repositories in the
workspace, which are summarized here. Run ``west <command> -h`` for
detailed help.
- ``west list``: print a line of information about each project in the
manifest, according to a format string
- ``west manifest``: manage the manifest file. See :ref:`west-manifest-cmd`.
- ``west diff``: run ``git diff`` in local project repositories
- ``west status``: run ``git status`` in local project repositories
- ``west forall``: run an arbitrary command in local project repositories
Additional Commands
===================
Finally, here is a summary of other built-in commands.
- ``west config``: get or set :ref:`configuration options <west-config>`
- ``west topdir``: print the top level directory of the west workspace
- ``west help``: get help about a command, or print information about all
commands in the workspace, including :ref:`west-extensions`
.. _west-topologies:
Topologies supported
********************
@ -250,156 +460,6 @@ v2.2.0 and its modules, then add the ``app1`` and ``app2`` projects:
You can also do this "by hand" by copy/pasting :file:`zephyr/west.yml`
as shown :ref:`above <west-t2>` for the T2 topology, with the same caveats.
.. _west-struct:
West Structure
**************
West's code is distributed via PyPI in a Python package named ``west``. See
:ref:`west-apis` for API documentation.
This distribution also includes a launcher executable, also named ``west``.
When west is installed, the launcher is placed by :file:`pip3` somewhere in the
user's filesystem (exactly where depends on the operating system, but should be
on the ``PATH`` :ref:`environment variable <env_vars>`). This launcher is the
command-line entry point.
.. _west-manifest-rev:
The ``manifest-rev`` branch
***************************
West creates a branch named ``manifest-rev`` in each project, pointing to the
commit the project's revision resolves to. The branch is updated whenever
project data is fetched by ``west update``. Other multi-repo commands also use
``manifest-rev`` as a reference for the upstream revision as of the most recent
update. See :ref:`west-multi-repo-cmds`, below, for more information.
``manifest-rev`` is a normal Git branch, but if you delete or otherwise modify
it, west will recreate and/or reset it as if with ``git reset --hard`` on the
next update (though ``git update-ref`` is used internally). For this reason, it
is normally a **bad idea to modify it yourself**. ``manifest-rev`` was added to
allow SHAs as project revisions in the manifest, and to give a consistent
reference for the current upstream revision regardless of how the manifest
changes over time.
.. note::
West does not create a ``manifest-rev`` branch in the manifest repository,
since west does not manage the manifest repository's branches or revisions.
.. _west-multi-repo-cmds:
Multi-Repo Commands
*******************
This section gives a quick overview of the multi-repo commands, split up by
functionality. Some commands loosely mimic the corresponding Git command, but
in a multi-repo context (e.g. ``west diff`` shows local changes on all
repositories).
Project arguments can be the names of projects in the manifest, or (as
fallback) paths to them. Omitting project arguments to commands which accept a
list of projects (such as ``west list``, ``west forall``, etc.) usually
defaults to using all projects in the manifest file plus the manifest
repository itself.
For help on individual commands, run ``west <command> -h`` (e.g. ``west diff
-h``).
Main Commands
=============
The ``west init`` and ``west update`` multi-repo commands are the most
important to understand.
- ``west init [-l] [-m URL] [--mr REVISION] [PATH]``: create a west
workspace in directory :file:`PATH` (i.e. :file:`.west` etc. will be
created there). If the ``PATH`` argument is not given, the current working
directory is used. This command does not clone any of the projects in the
manifest; that is done the next time ``west update`` is run.
This command can be invoked in two ways:
1. If you already have a local clone of the zephyr repository and want to
create a west workspace around it, you can use the ``-l`` switch to
pass its path to west, as in: ``west init -l path/to/zephyr``. This is
the only reason to use ``-l``.
2. Otherwise, omit ``-l`` to create a new workspace from a remote manifest
repository. You can give the manifest URL using the ``-m`` switch, and its
revision using ``--mr``. For example, invoking west with: ``west init -m
https://github.com/zephyrproject-rtos/zephyr --mr v1.15.0`` would clone
the upstream official zephyr repository at the tagged release v1.15.0
(``-m`` defaults to https://github.com/zephyrproject-rtos/zephyr, and
the ``-mr`` default is overridden to ``v1.15.0``).
- ``west update [--fetch {always,smart}] [--rebase] [--keep-descendants]
[PROJECT ...]``: clone and update the specified projects based
on the current :term:`west manifest`.
By default, this command:
#. Parses the manifest file, :file:`west.yml`
#. Clones any project repositories that are not already present locally
#. Fetches any project revisions in the manifest file which are not already
pulled from the remote
#. Sets each project's :ref:`manifest-rev <west-manifest-rev>` branch to the
current manifest revision
#. Checks out those revisions in local working trees
To operate on a subset of projects only, specify them using the ``PROJECT``
positional arguments, which can be either project names as given in the
manifest file, or paths to the local project clones.
To force this command to fetch from project remotes even if the revisions
appear to be available locally, either use ``--fetch always`` or set the
``update.fetch`` :ref:`configuration option <west-config>` to ``"always"``.
For safety, ``west update`` uses ``git checkout --detach`` to check out a
detached ``HEAD`` at the manifest revision for each updated project, leaving
behind any branches which were already checked out. This is typically a safe
operation that will not modify any of your local branches. See the help for
the ``--rebase`` / ``-r`` and ``--keep-descendants`` / ``-k`` options for
ways to influence this.
.. _west-multi-repo-misc:
Miscellaneous Commands
======================
West has a few more commands for managing the multi-repo, which are briefly
discussed here. Run ``west <command> -h`` for detailed help.
- ``west forall -c COMMAND [PROJECT ...]``: Runs the shell command ``COMMAND``
within the top-level repository directory of each of the specified projects
(default: all cloned projects). If ``COMMAND`` consists of more than one
word, it must be quoted to prevent it from being split up by the shell.
To run an arbitrary Git command in each project, use something like ``west
forall -c 'git <command> --options'``. Note that ``west forall`` can be used
to run any command, though, not just Git commands.
- ``west help <command>``: this is equivalent to ``west <command> -h``.
- ``west list [-f FORMAT] [PROJECT ...]``: Lists project information from the
manifest file, such as URL, revision, path, etc. The printed information can
be controlled using the ``-f`` option.
- ``west manifest``: Manipulates manifest files. See :ref:`west-manifest-cmd`.
- ``west manifest --validate``: Ensure the current manifest file is
well-formed. Print information about what's wrong and fail the process in
case of error.
- ``west diff [PROJECT ...]``: Runs a multi-repo ``git diff``
for the specified projects.
- ``west status [PROJECT ...]``: Like ``west diff``, for
running ``git status``.
- ``west topdir``: Prints the top directory of the west workspace.
Private repositories
********************

4
doc/guides/west/why.rst
View file

@ -36,6 +36,10 @@ The basic requirements are:
Rationale for a custom tool
***************************
Some of west's features are similar to those provided by
`Git Submodules <https://git-scm.com/book/en/v2/Git-Tools-Submodules>`_ and
Google's `repo <https://gerrit.googlesource.com/git-repo/>`_.
Existing tools were considered during west's initial design and development.
None were found suitable for Zephyr's requirements. In particular, these were
examined in detail: