488294368f
When this west doc page was written, we didn't have any documentation for modules. We do now, so point to it instead of CMake. Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
400 lines
15 KiB
ReStructuredText
400 lines
15 KiB
ReStructuredText
.. _west-multi-repo:
|
|
|
|
Multiple Repository Management
|
|
##############################
|
|
|
|
This page introduces basic concepts related to west and its multiple repository
|
|
management features, and gives an overview of the associated commands. See
|
|
:ref:`west-history` and `Zephyr issue #6770`_ for additional discussion,
|
|
rationale, and motivation.
|
|
|
|
.. note::
|
|
|
|
West's multi-repo commands are meant to augment Git in minor ways for
|
|
multi-repo work, not to replace it. For tasks that only operate on one
|
|
repository, just use plain Git commands.
|
|
|
|
.. _west-workspace:
|
|
|
|
Introduction
|
|
************
|
|
|
|
West's built-in commands allow you to work with projects composed of multiple
|
|
Git repositories installed under a common parent directory, which we call a
|
|
*west workspace* (before west 0.7, this was called a *west installation*). This
|
|
works similarly to `Git Submodules
|
|
<https://git-scm.com/book/en/v2/Git-Tools-Submodules>`_ and Google's `repo
|
|
<https://gerrit.googlesource.com/git-repo/>`_.
|
|
|
|
A west workspace is the result of running the ``west init`` command, which
|
|
is described in more detail below. For upstream Zephyr, the workspace looks
|
|
like this:
|
|
|
|
.. code-block:: none
|
|
|
|
zephyrproject
|
|
├── .west
|
|
│ └── config
|
|
├── zephyr
|
|
│ ├── west.yml
|
|
│ └── [... other files ...]
|
|
├── modules
|
|
│ └── lib
|
|
│ └── tinycbor
|
|
├── net-tools
|
|
└── [ ... other projects ...]
|
|
|
|
Above, :file:`zephyrproject` is the name of the west workspace's root
|
|
directory. This name is just an example -- it could be anything, like ``z``,
|
|
``my-zephyr-workspace``, etc. The file :file:`.west/config` is the
|
|
workspace's :ref:`local configuration file <west-config>`.
|
|
|
|
Every west workspace contains exactly one *manifest repository*, which is a
|
|
Git repository containing a file named :file:`west.yml`, which is the *west
|
|
manifest*. The location of the manifest repository is given by the
|
|
:ref:`manifest.path configuration option <west-config-index>` in the local
|
|
configuration file. The manifest file, along with west's configuration files,
|
|
controls the workspace's behavior. For upstream Zephyr, :file:`zephyr` is
|
|
the manifest repository, but you can configure west to use any Git repository
|
|
in the workspace as the manifest repository. The only requirement is that it
|
|
contains a valid manifest file. See :ref:`west-manifests` for more details on
|
|
what this means.
|
|
|
|
Both of the :file:`tinycbor` and :file:`net-tools` directories are *projects*
|
|
managed by west, and configured in the manifest file. A west workspace can
|
|
contain arbitrarily many projects. As shown above, projects can be located
|
|
anywhere in the workspace. They don't have to be subdirectories of the
|
|
manifest directory, and they can be inside of arbitrary subdirectories inside
|
|
the workspace's root directory. By default, the Zephyr build system uses
|
|
west to get the locations of all the projects in the workspace, so any code
|
|
they contain can be used by applications. This behavior can be overridden using
|
|
the ``ZEPHYR_MODULES`` CMake variable; see :ref:`modules` for details.
|
|
|
|
Finally, any repository managed by a west workspace can contain
|
|
:ref:`extension commands <west-extensions>`, which are extra west commands
|
|
provided by that project. This includes the manifest repository and any project
|
|
repository.
|
|
|
|
Topologies supported
|
|
********************
|
|
|
|
The following are example source code topologies supported by west.
|
|
|
|
- T1: star topology, zephyr is the manifest repository
|
|
- T2: star topology, a Zephyr application is the manifest repository
|
|
- T3: forest topology, freestanding manifest repository
|
|
|
|
T1: Star topology, zephyr is the manifest repository
|
|
====================================================
|
|
|
|
- The zephyr repository acts as the central repository and specifies
|
|
its :ref:`modules` in its :file:`west.yml`
|
|
- Analogy with existing mechanisms: Git submodules with zephyr as the
|
|
super-project
|
|
|
|
This is the default. See :ref:`west-workspace` for how mainline Zephyr is an
|
|
example of this topology.
|
|
|
|
.. _west-t2:
|
|
|
|
T2: Star topology, application is the manifest repository
|
|
=========================================================
|
|
|
|
- Useful for those focused on a single application
|
|
- A repository containing a Zephyr application acts as the central repository
|
|
and names other projects required to build it in its :file:`west.yml`. This
|
|
includes the zephyr repository and any modules.
|
|
- Analogy with existing mechanisms: Git submodules with the application as
|
|
the super-project, zephyr and other projects as submodules
|
|
|
|
A workspace using this topology looks like this:
|
|
|
|
.. code-block:: none
|
|
|
|
west-workspace
|
|
├── application
|
|
│ ├── CMakeLists.txt
|
|
│ ├── prj.conf
|
|
│ ├── src
|
|
│ │ └── main.c
|
|
│ └── west.yml
|
|
├── modules
|
|
│ └── lib
|
|
│ └── tinycbor
|
|
└── zephyr
|
|
|
|
Here is an example :file:`application/west.yml` which uses
|
|
:ref:`west-manifest-import`, available since west 0.7, to import Zephyr v2.2.0
|
|
and its modules into the application manifest file:
|
|
|
|
.. code-block:: yaml
|
|
|
|
# Example T2 west.yml, using manifest imports.
|
|
manifest:
|
|
remotes:
|
|
- name: zephyrproject-rtos
|
|
url-base: https://github.com/zephyrproject-rtos
|
|
projects:
|
|
- name: zephyr
|
|
remote: zephyrproject-rtos
|
|
revision: v2.2.0
|
|
import: true
|
|
self:
|
|
path: application
|
|
|
|
You can still selectively "override" individual Zephyr modules if you use
|
|
``import:`` in this way; see :ref:`west-manifest-ex1.3` for an example.
|
|
|
|
Another way to do the same thing is to copy/paste :file:`zephyr/west.yml`
|
|
to :file:`application/west.yml`, adding an entry for the zephyr
|
|
project itself, like this:
|
|
|
|
.. code-block:: yaml
|
|
|
|
# Equivalent to the above, but with manually maintained Zephyr modules.
|
|
manifest:
|
|
remotes:
|
|
- name: zephyrproject-rtos
|
|
url-base: https://github.com/zephyrproject-rtos
|
|
defaults:
|
|
remote: zephyrproject-rtos
|
|
projects:
|
|
- name: zephyr
|
|
revision: v2.2.0
|
|
west-commands: scripts/west-commands.yml
|
|
- name: net-tools
|
|
revision: some-sha-goes-here
|
|
path: tools/net-tools
|
|
# ... other Zephyr modules go here ...
|
|
self:
|
|
path: application
|
|
|
|
(The ``west-commands`` is there for :ref:`west-build-flash-debug` and other
|
|
Zephyr-specific :ref:`west-extensions`. It's not necessary when using
|
|
``import``.)
|
|
|
|
The main advantage to using ``import`` is not having to track the revisions of
|
|
imported projects separately. In the above example, using ``import`` means
|
|
Zephyr's :ref:`module <modules>` versions are automatically determined from the
|
|
:file:`zephyr/west.yml` revision, instead of having to be copy/pasted (and
|
|
maintained) on their own.
|
|
|
|
T3: Forest topology
|
|
===================
|
|
|
|
- Useful for those supporting multiple independent applications or downstream
|
|
distributions with no "central" repository
|
|
- A dedicated manifest repository which contains no Zephyr source code,
|
|
and specifies a list of projects all at the same "level"
|
|
- Analogy with existing mechanisms: Google repo-based source distribution
|
|
|
|
A workspace using this topology looks like this:
|
|
|
|
.. code-block:: none
|
|
|
|
west-workspace
|
|
├── app1
|
|
│ ├── CMakeLists.txt
|
|
│ ├── prj.conf
|
|
│ └── src
|
|
│ └── main.c
|
|
├── app2
|
|
│ ├── CMakeLists.txt
|
|
│ ├── prj.conf
|
|
│ └── src
|
|
│ └── main.c
|
|
├── manifest-repo
|
|
│ └── west.yml
|
|
├── modules
|
|
│ └── lib
|
|
│ └── tinycbor
|
|
└── zephyr
|
|
|
|
Here is an example T3 :file:`manifest-repo/west.yml` which uses
|
|
:ref:`west-manifest-import`, available since west 0.7, to import Zephyr
|
|
v2.2.0 and its modules, then add the ``app1`` and ``app2`` projects:
|
|
|
|
.. code-block:: yaml
|
|
|
|
manifest:
|
|
remotes:
|
|
- name: zephyrproject-rtos
|
|
url-base: https://github.com/zephyrproject-rtos
|
|
- name: your-git-server
|
|
url-base: https://git.example.com/your-company
|
|
defaults:
|
|
remote: your-git-server
|
|
projects:
|
|
- name: zephyr
|
|
remote: zephyrproject-rtos
|
|
revision: v2.2.0
|
|
import: true
|
|
- name: app1
|
|
revision: SOME_SHA_OR_BRANCH_OR_TAG
|
|
- name: app2
|
|
revision: ANOTHER_SHA_OR_BRANCH_OR_TAG
|
|
self:
|
|
path: manifest-repo
|
|
|
|
You can also do this "by hand" by copy/pasting :file:`zephyr/west.yml`
|
|
as shown :ref:`above <west-t2>` for the T2 topology, with the same caveats.
|
|
|
|
.. _west-struct:
|
|
|
|
West Structure
|
|
**************
|
|
|
|
West's code is distributed via PyPI in a Python package named ``west``. See
|
|
:ref:`west-apis` for API documentation.
|
|
|
|
This distribution also includes a launcher executable, also named ``west``.
|
|
When west is installed, the launcher is placed by :file:`pip3` somewhere in the
|
|
user's filesystem (exactly where depends on the operating system, but should be
|
|
on the ``PATH`` :ref:`environment variable <env_vars>`). This launcher is the
|
|
command-line entry point.
|
|
|
|
.. _west-manifest-rev:
|
|
|
|
The ``manifest-rev`` branch
|
|
***************************
|
|
|
|
West creates a branch named ``manifest-rev`` in each project, pointing to the
|
|
commit the project's revision resolves to. The branch is updated whenever
|
|
project data is fetched by ``west update``. Other multi-repo commands also use
|
|
``manifest-rev`` as a reference for the upstream revision as of the most recent
|
|
update. See :ref:`west-multi-repo-cmds`, below, for more information.
|
|
|
|
``manifest-rev`` is a normal Git branch, but if you delete or otherwise modify
|
|
it, west will recreate and/or reset it as if with ``git reset --hard`` on the
|
|
next update (though ``git update-ref`` is used internally). For this reason, it
|
|
is normally a **bad idea to modify it yourself**. ``manifest-rev`` was added to
|
|
allow SHAs as project revisions in the manifest, and to give a consistent
|
|
reference for the current upstream revision regardless of how the manifest
|
|
changes over time.
|
|
|
|
.. note::
|
|
|
|
West does not create a ``manifest-rev`` branch in the manifest repository,
|
|
since west does not manage the manifest repository's branches or revisions.
|
|
|
|
.. _west-multi-repo-cmds:
|
|
|
|
Multi-Repo Commands
|
|
*******************
|
|
|
|
This section gives a quick overview of the multi-repo commands, split up by
|
|
functionality. Some commands loosely mimic the corresponding Git command, but
|
|
in a multi-repo context (e.g. ``west diff`` shows local changes on all
|
|
repositories).
|
|
|
|
Project arguments can be the names of projects in the manifest, or (as
|
|
fallback) paths to them. Omitting project arguments to commands which accept a
|
|
list of projects (such as ``west list``, ``west forall``, etc.) usually
|
|
defaults to using all projects in the manifest file plus the manifest
|
|
repository itself.
|
|
|
|
For help on individual commands, run ``west <command> -h`` (e.g. ``west diff
|
|
-h``).
|
|
|
|
Main Commands
|
|
=============
|
|
|
|
The ``west init`` and ``west update`` multi-repo commands are the most
|
|
important to understand.
|
|
|
|
- ``west init [-l] [-m URL] [--mr REVISION] [PATH]``: create a west
|
|
workspace in directory :file:`PATH` (i.e. :file:`.west` etc. will be
|
|
created there). If the ``PATH`` argument is not given, the current working
|
|
directory is used. This command does not clone any of the projects in the
|
|
manifest; that is done the next time ``west update`` is run.
|
|
|
|
This command can be invoked in two ways:
|
|
|
|
1. If you already have a local clone of the zephyr repository and want to
|
|
create a west workspace around it, you can use the ``-l`` switch to
|
|
pass its path to west, as in: ``west init -l path/to/zephyr``. This is
|
|
the only reason to use ``-l``.
|
|
|
|
2. Otherwise, omit ``-l`` to create a new workspace from a remote manifest
|
|
repository. You can give the manifest URL using the ``-m`` switch, and its
|
|
revision using ``--mr``. For example, invoking west with: ``west init -m
|
|
https://github.com/zephyrproject-rtos/zephyr --mr v1.15.0`` would clone
|
|
the upstream official zephyr repository at the tagged release v1.15.0
|
|
(``-m`` defaults to https://github.com/zephyrproject-rtos/zephyr, and
|
|
the ``-mr`` default is overridden to ``v1.15.0``).
|
|
|
|
- ``west update [--fetch {always,smart}] [--rebase] [--keep-descendants]
|
|
[PROJECT ...]``: clone and update the specified projects based
|
|
on the current :term:`west manifest`.
|
|
|
|
By default, this command:
|
|
|
|
#. Parses the manifest file, :file:`west.yml`
|
|
#. Clones any project repositories that are not already present locally
|
|
#. Fetches any project revisions in the manifest file which are not already
|
|
pulled from the remote
|
|
#. Sets each project's :ref:`manifest-rev <west-manifest-rev>` branch to the
|
|
current manifest revision
|
|
#. Checks out those revisions in local working trees
|
|
|
|
To operate on a subset of projects only, specify them using the ``PROJECT``
|
|
positional arguments, which can be either project names as given in the
|
|
manifest file, or paths to the local project clones.
|
|
|
|
To force this command to fetch from project remotes even if the revisions
|
|
appear to be available locally, either use ``--fetch always`` or set the
|
|
``update.fetch`` :ref:`configuration option <west-config>` to ``"always"``.
|
|
|
|
For safety, ``west update`` uses ``git checkout --detach`` to check out a
|
|
detached ``HEAD`` at the manifest revision for each updated project, leaving
|
|
behind any branches which were already checked out. This is typically a safe
|
|
operation that will not modify any of your local branches. See the help for
|
|
the ``--rebase`` / ``-r`` and ``--keep-descendants`` / ``-k`` options for
|
|
ways to influence this.
|
|
|
|
.. _west-multi-repo-misc:
|
|
|
|
Miscellaneous Commands
|
|
======================
|
|
|
|
West has a few more commands for managing the multi-repo, which are briefly
|
|
discussed here. Run ``west <command> -h`` for detailed help.
|
|
|
|
- ``west forall -c COMMAND [PROJECT ...]``: Runs the shell command ``COMMAND``
|
|
within the top-level repository directory of each of the specified projects
|
|
(default: all cloned projects). If ``COMMAND`` consists of more than one
|
|
word, it must be quoted to prevent it from being split up by the shell.
|
|
|
|
To run an arbitrary Git command in each project, use something like ``west
|
|
forall -c 'git <command> --options'``. Note that ``west forall`` can be used
|
|
to run any command, though, not just Git commands.
|
|
|
|
- ``west help <command>``: this is equivalent to ``west <command> -h``.
|
|
|
|
- ``west list [-f FORMAT] [PROJECT ...]``: Lists project information from the
|
|
manifest file, such as URL, revision, path, etc. The printed information can
|
|
be controlled using the ``-f`` option.
|
|
|
|
- ``west manifest``: Manipulates manifest files. See :ref:`west-manifest-cmd`.
|
|
|
|
- ``west manifest --validate``: Ensure the current manifest file is
|
|
well-formed. Print information about what's wrong and fail the process in
|
|
case of error.
|
|
|
|
- ``west diff [PROJECT ...]``: Runs a multi-repo ``git diff``
|
|
for the specified projects.
|
|
|
|
- ``west status [PROJECT ...]``: Like ``west diff``, for
|
|
running ``git status``.
|
|
|
|
- ``west topdir``: Prints the top directory of the west workspace.
|
|
|
|
.. _PyPI:
|
|
https://pypi.org/project/west/
|
|
|
|
.. _Zephyr issue #6770:
|
|
https://github.com/zephyrproject-rtos/zephyr/issues/6770
|
|
|
|
.. _namespace package:
|
|
https://www.python.org/dev/peps/pep-0420/
|