doc: Update documentation to describe use of find_package(Zephyr)

This commit changes the current boilerplate include description and
instead describes the use of find_package(Zephyr)

It also add a section covering additional possibilities when using
find_package(Zephyr).

- Search order
- Zephyr repository application description
- Zephyr workspace application description
- Zephyr freestanding application description
- Environment ZEPHYR_BASE setting
- Multiple Zephyr and preference setting

Signed-off-by: Torsten Rasmussen <Torsten.Rasmussen@nordicsemi.no>

cmake/app/boilerplate.cmake

@@ -1,10 +1,18 @@
 # SPDX-License-Identifier: Apache-2.0
 
 # This file must be included into the toplevel CMakeLists.txt file of
-# Zephyr applications, e.g. zephyr/samples/hello_world/CMakeLists.txt
+# Zephyr applications.
-# must start with the line:
+# Zephyr CMake package automatically includes this file when CMake function
+# find_package() is used.
 #
-# include(${ZEPHYR_BASE}/cmake/app/boilerplate.cmake NO_POLICY_SCOPE)
+# To ensure this file is loaded in a Zephyr application it must start with
+# one of those lines:
+#
+# find_package(Zephyr)
+# find_package(Zephyr HINTS $ENV{ZEPHYR_BASE})
+#
+# The `HINTS $ENV{ZEPHYR_BASE}` variant is required for any application inside
+# the Zephyr repository.
 #
 # It exists to reduce boilerplate code that Zephyr expects to be in
 # application CMakeLists.txt code.

doc/application/index.rst

@@ -162,6 +162,10 @@ subdirectories which are not described here.
 :file:`tests`
     Test code and benchmarks for Zephyr features.
 
+:file:`share`
+    Additional architecture independent data. Currently containing Zephyr CMake
+    package.
+
 
 Creating an Application
 ***********************
@@ -206,28 +210,36 @@ Follow these steps to create a new application directory. (Refer to
 
    .. code-block:: cmake
 
-      # Boilerplate code, which pulls in the Zephyr build system.
+      # Find Zephyr. This also loads Zephyr's build system.
       cmake_minimum_required(VERSION 3.13.1)
-      include($ENV{ZEPHYR_BASE}/cmake/app/boilerplate.cmake NO_POLICY_SCOPE)
+      find_package(Zephyr)
       project(my_zephyr_app)
 
       # Add your source file to the "app" target. This must come after
-      # the boilerplate code, which defines the target.
+      # find_package(Zephyr) which defines the target.
       target_sources(app PRIVATE src/main.c)
 
-   The boilerplate code sets the minimum CMake version and pulls in the Zephyr
+   ``find_package(Zephyr)`` sets the minimum CMake version and pulls in the
-   build system, which creates a CMake target named ``app``. Adding sources
+   Zephyr build system, which creates a CMake target named ``app`` (see
-   to this target is how you include them in the build.
+   :ref:`cmake_pkg`). Adding sources to this target is how you include them in
+   the build.
 
-   .. note:: ``cmake_minimum_required()`` is also invoked from
+   .. note:: ``cmake_minimum_required()`` is also invoked by the Zephyr package.
-             :file:`boilerplate.cmake`. The most recent of the two
+              The most recent of the two versions will be enforced by CMake.
-             versions will be enforced by CMake.
 
 #. Set Kconfig configuration options. See :ref:`application-kconfig`.
 
 #. Configure any devicetree overlays needed by your application.
    See :ref:`set-devicetree-overlays`.
 
+.. note::
+
+   ``include($ENV{ZEPHYR_BASE}/cmake/app/boilerplate.cmake NO_POLICY_SCOPE)``
+   is still supported for backward compatibility with older applications.
+   Including ``boilerplate.cmake`` directly in the sample still requires to run
+   ``source zephyr-env.sh`` or execute ``zephyr-env.cmd`` before building the
+   application.
+
 .. _important-build-vars:
 
 Important Build System Variables
@@ -248,13 +260,10 @@ should know about.
    * As :ref:`env_vars`.
    * As a ``set(<VARIABLE> <VALUE>)`` statement in your :file:`CMakeLists.txt`
 
-* :makevar:`ZEPHYR_BASE`: Sets the path to the directory containing Zephyr,
+* :makevar:`ZEPHYR_BASE`: Zephyr base variable used by the build system.
-  which is needed by the build system's boilerplate file.  This is an
+  ``find_package(Zephyr)`` will automatically set this as a cached CMake
-  environment variable set by the :file:`zephyr-env.sh` script on Linux/macOS
+  variable. But ``ZEPHYR_BASE`` can also be set as an environment variable in
-  or :file:`zephyr-env.cmd` on Windows, as you learned when getting started
+  order to force CMake to use a specific Zephyr installation.
-  with Zephyr in :ref:`getting_started_run_sample`. You can also set
-  :makevar:`ZEPHYR_BASE` explicitly, but then you won't get the other features
-  provided by those scripts.
 
 * :makevar:`BOARD`: Selects the board that the application's build
   will use for the default configuration.  See :ref:`boards` for
@@ -412,6 +421,8 @@ described above.)
   kernel binary. Other binary output formats, such as :file:`.hex` and
   :file:`.bin`, are also supported.
 
+.. _application_rebuild:
+
 Rebuilding an Application
 =========================
 
@@ -1122,15 +1133,20 @@ Make sure to follow these steps in order.
    found elsewhere if the CMake variable :makevar:`KCONFIG_ROOT` is
    set with an absolute path.
 
-#. Now include the mandatory boilerplate that integrates the
+#. Specify that the application requires Zephyr on a new line, **after any
-   application with the Zephyr build system on a new line, **after any
    lines added from the steps above**:
 
    .. code-block:: cmake
 
-      include($ENV{ZEPHYR_BASE}/cmake/app/boilerplate.cmake NO_POLICY_SCOPE)
+      find_package(Zephyr)
       project(my_zephyr_app)
 
+   .. note:: ``find_package(Zephyr HINTS $ENV{ZEPHYR_BASE})`` can be used if
+             enforcing a specific Zephyr installation by explicitly
+             setting the ``ZEPHYR_BASE`` environment variable should be
+             supported. All samples in Zephyr supports the ``ZEPHYR_BASE``
+             environment variable.
+
 #. Now add any application source files to the 'app' target
    library, each on their own line, like so:
 
@@ -1144,7 +1160,7 @@ Below is a simple example :file:`CMakeList.txt`:
 
    set(BOARD qemu_x86)
 
-   include($ENV{ZEPHYR_BASE}/cmake/app/boilerplate.cmake NO_POLICY_SCOPE)
+   find_package(Zephyr)
    project(my_zephyr_app)
 
    target_sources(app PRIVATE src/main.c)

doc/getting_started/index.rst

@@ -246,6 +246,36 @@ directory using west:
          cd zephyrproject
          west update
 
+.. rst-class:: numbered-step
+
+Export Zephyr CMake package
+***************************
+
+Exporting Zephyr as a :ref:`cmake_pkg` makes it possible for CMake to automatically find and load
+boilerplate code for building a Zephyr application.
+
+Zephyr CMake package is exported with the following command
+
+.. tabs::
+
+   .. group-tab:: Ubuntu
+
+      .. code-block:: bash
+
+         west zephyr-export
+
+   .. group-tab:: macOS
+
+      .. code-block:: bash
+
+         west zephyr-export
+
+   .. group-tab:: Windows
+
+      .. code-block:: bat
+
+         west zephyr-export
+
 .. _install_py_requirements:
 
 .. rst-class:: numbered-step
@@ -367,31 +397,6 @@ are properly set up for Zephyr development.
       compatible with your board, a good alternative to try is the
       :ref:`Hello World sample <hello_world>`.
 
-#. Set build environment variables:
-
-   .. tabs::
-
-      .. group-tab:: Ubuntu
-
-         .. code-block:: bash
-
-            cd ~/zephyrproject/zephyr
-            source zephyr-env.sh
-
-      .. group-tab:: macOS
-
-         .. code-block:: bash
-
-            cd ~/zephyrproject/zephyr
-            source zephyr-env.sh
-
-      .. group-tab:: Windows
-
-         .. code-block:: bat
-
-            cd %HOMEPATH%/zephyrproject/zephyr
-            zephyr-env.cmd
-
 #. Build the blinky sample. Specify **your board name**
    (see :ref:`boards`) in the command below:
 

doc/getting_started/installation_linux.rst

@@ -280,8 +280,7 @@ As already noted above, the SDK also includes prebuilt host tools.  To use the
 SDK's prebuilt host tools with a toolchain from another source, keep the
 :envvar:`ZEPHYR_SDK_INSTALL_DIR` environment variable set to the Zephyr SDK
 installation directory. To build without the Zephyr SDK's prebuilt host tools,
-the :envvar:`ZEPHYR_SDK_INSTALL_DIR` environment variable must be unset before
+the :envvar:`ZEPHYR_SDK_INSTALL_DIR` environment variable must be unset.
-you run ``source zephyr-env.sh`` later on in the Getting Started Guide.
 
 To make sure this variable is unset, run:
 

doc/guides/beyond-GSG.rst

@@ -122,6 +122,12 @@ repo.
    repository, switch branches in it, or perform a ``git bisect`` inside of
    it.
 
+Export Zephyr CMake package
+***************************
+
+The :ref:`cmake_pkg` can be exported to CMake's user package registry if it has
+not already been done as part of :ref:`getting_started`.
+
 Build and Run an Application
 ****************************
 
@@ -149,16 +155,6 @@ a list of supported boards.
 
       cd zephyrproject/zephyr
 
-#. Set up your build environment variables:
-
-   .. code-block:: console
-
-      # Linux and macOS
-      source zephyr-env.sh
-
-      # Windows
-      zephyr-env.cmd
-
 #. Build the blinky sample for the ``reel_board``:
 
    .. zephyr-app-commands::

doc/guides/index.rst

@@ -31,3 +31,4 @@ User and Developer Guides
    tracing/index
    west/index
    optimizations/index
+   zephyr_cmake_package.rst

doc/guides/zephyr_cmake_package.rst

@@ -0,0 +1,430 @@
+.. _cmake_pkg:
+
+Zephyr CMake Package
+####################
+
+The Zephyr `CMake package`_ is a convenient way to create a Zephyr-based application.
+
+The Zephyr CMake package ensures that CMake can automatically select a Zephyr to use for building
+the application, whether it is a Zephyr repository application, Zephyr workspace application, or a
+Zephyr freestanding application.
+
+When developing a Zephyr-based application, then a developer simply needs to write
+``find_package(Zephyr)`` in the beginning of the application :file:`CMakeLists.txt` file.
+
+To use the Zephyr CMake package it must first be exported to the `CMake user package registry`_.
+This is means creating a reference to the current Zephyr installation inside the
+CMake user package registry.
+
+
+.. tabs::
+
+   .. group-tab:: Ubuntu
+
+      In Linux, the CMake user package registry is found in:
+
+      ``~/.cmake/package/Zephyr``
+
+   .. group-tab:: macOS
+
+      In macOS, the CMake user package registry is found in:
+
+      ``~/.cmake/package/Zephyr``
+
+   .. group-tab:: Windows
+
+      In Windows, the CMake user package registry is found in:
+
+      ``HKEY_CURRENT_USER\Software\Kitware\CMake\Packages\Zephyr``
+
+
+The Zephyr CMake package allows CMake to automatically find a Zephyr base.
+One or more Zephyr installations must be exported.
+Exporting multiple Zephyr installations may be useful when developing or testing
+Zephyr freestanding applications, Zephyr workspace application with vendor forks, etc..
+
+
+Zephyr CMake package export (west)
+**********************************
+
+When installing Zephyr using :ref:`west <get_the_code>` then it is recommended
+to export Zephyr using ``west zephyr-export``.
+
+
+Zephyr CMake package export (without west)
+******************************************
+
+Zephyr CMake package is exported to the CMake user package registry using the following commands:
+
+.. code-block:: bash
+
+   cmake -S <PATH-TO-ZEPHYR>/share/zephyr-package/cmake -B <PATH-TO-ZEPHYR>/share/zephyr-package/cmake
+   cmake --build <PATH-TO-ZEPHYR>/share/zephyr-package/cmake --target pristine
+
+This will export the current Zephyr to the CMake user package registry and remove the temporary
+files generated by CMake during export.
+
+.. _zephyr_cmake_package_zephyr_base:
+
+
+Zephyr application structure
+****************************
+
+An application can be placed anywhere on your disk, but to better understand how the Zephyr
+package is used, we will name three specific layouts.
+
+
+Zephyr repository application
+=============================
+
+A Zephyr repository has the following structure:
+
+.. code-block:: none
+
+   <projects>/zephyr-workspace
+   └── zephyr
+        ├── arch
+        ├── boards
+        ├── cmake
+        ├── samples
+        │    ├── hello_world
+        │    └── ...
+        ├── tests
+        └── ...
+
+Any application located inside this tree, is simply referred to as a Zephyr repository application.
+In this example ``hello_world`` is a Zephyr repository application.
+
+
+Zephyr workspace application
+============================
+
+A Zephyr workspace has the following structure:
+
+.. code-block:: none
+
+   <projects>/zephyr-workspace
+   ├── zephyr
+   ├── bootloader
+   ├── modules
+   ├── tools
+   ├── <vendor/private-repositories>
+   └── my_applications
+        └── my_first_app
+
+Any application located in such workspace, but outside the Zephyr repository itself, is referred to
+as a Zephyr workspace application.
+In this example ``my_first_app`` is a Zephyr workspace application.
+
+.. note:: The root of a Zephyr workspace is identical to ``west topdir`` if the workspace was
+          installed using ``west``
+
+
+Zephyr freestanding application
+===============================
+
+A Zephyr freestanding application is a Zephyr application located outside of a Zephyr workspace.
+
+
+.. code-block:: none
+
+   <projects>/zephyr-workspace
+   ├── zephyr
+   ├── bootloader
+   └── ...
+
+   <home>/app
+   ├── CMakeLists.txt
+   ├── prj.conf
+   └── src
+       └── main.c
+
+In this example ``app`` is a Zephyr freestanding application.
+
+
+Zephyr Base Environment Setting
+*******************************
+
+The Zephyr CMake package search functionality allows for explicitly specifying
+a Zephyr base using an environment variable.
+
+To do this, use the following ``find_package()`` syntax:
+
+.. code-block:: cmake
+
+   find_package(Zephyr HINTS $ENV{ZEPHYR_BASE})
+
+This syntax instructs CMake to first search for Zephyr using the Zephyr base environment setting
+:envvar:`ZEPHYR_BASE` and then use the normal search paths.
+
+.. _zephyr_cmake_search_order:
+
+Zephyr CMake Package Search Order
+*********************************
+
+When Zephyr base environment setting is not used for searching, the Zephyr installation matching
+the following criteria will be used:
+
+* A Zephyr repository application will use the Zephyr in which it is located.
+  For example:
+
+  .. code-block:: none
+
+        <projects>/zephyr-workspace/zephyr
+        └── samples
+            └── hello_world
+
+  in this example, ``hello_world`` will use ``<projects>/zephyr-workspace/zephyr``.
+
+
+* Zephyr workspace application will use the Zephyr that share the same workspace.
+  For example:
+
+  .. code-block:: none
+
+     <projects>/zephyr-workspace
+     ├── zephyr
+     ├── ...
+     └── my_applications
+          └── my_first_app
+
+  in this example, ``my_first_app`` will use ``<projects>/zephyr-workspace/zephyr`` as this Zephyr
+  is located in the same workspace as the Zephyr workspace application.
+
+
+* Zephyr freestanding application will use the Zephyr registered in the CMake user package registry.
+  For example:
+
+  .. code-block:: none
+
+     <projects>/zephyr-workspace-1
+     └── zephyr                       (Not exported to CMake)
+
+     <projects>/zephyr-workspace-2
+     └── zephyr                       (Exported to CMake)
+
+     <home>/app
+     ├── CMakeLists.txt
+     ├── prj.conf
+     └── src
+         └── main.c
+
+  in this example, only ``<projects>/zephyr-workspace-2/zephyr`` is exported to the CMake package
+  registry and therefore this Zephyr will be used by the Zephyr freestanding application
+  ``<home>/app``.
+
+  If user wants to test the application with ``<projects>/zephyr-workspace-1/zephyr``, this can be
+  done by using the Zephyr Base environment setting, meaning set
+  ``ZEPHYR_BASE=<projects>/zephyr-workspace-1/zephyr``, before
+  running CMake.
+
+  .. note::
+
+     The Zephyr package selected on the first CMake invocation will be used for all subsequent
+     builds. To change the Zephyr package, for example to test the application using Zephyr base
+     environment setting, then it is necessary to do a pristine build first
+     (See :ref:`application_rebuild`).
+
+Zephyr CMake Package Version
+****************************
+
+When writing an application then it is possible to specify a Zephyr version number ``x.y.z`` that
+must be used in order to build the application.
+
+Specifying a version is especially useful for a Zephyr freestanding application as it ensures the
+application is built with a minimal Zephyr version.
+
+It also helps CMake to select the correct Zephyr to use for building, when there are multiple
+Zephyr installations in the system.
+
+For example:
+
+  .. code-block:: cmake
+
+     cmake_minimum_required(VERSION 3.13.1)
+     find_package(Zephyr 2.2.0)
+     project(app)
+
+will require ``app`` to be built with Zephyr 2.2.0 as minimum.
+CMake will search all exported candidates to find a Zephyr installation which matches this version
+criteria.
+
+Thus it is possible to have multiple Zephyr installations and have CMake automatically select
+between them based on the version number provided, see `CMake package version`_ for details.
+
+For example:
+
+.. code-block:: none
+
+   <projects>/zephyr-workspace-2.a
+   └── zephyr                       (Exported to CMake)
+
+   <projects>/zephyr-workspace-2.b
+   └── zephyr                       (Exported to CMake)
+
+   <home>/app
+   ├── CMakeLists.txt
+   ├── prj.conf
+   └── src
+       └── main.c
+
+in this case, there are two released versions of Zephyr installed at their own workspaces.
+Workspace 2.a and 2.b, corresponding to the Zephyr version.
+
+To ensure ``app`` is built with minimum version ``2.a`` the following ``find_package``
+syntax may be used:
+
+.. code-block:: cmake
+
+   cmake_minimum_required(VERSION 3.13.1)
+   find_package(Zephyr 2.a)
+   project(app)
+
+
+Note that both ``2.a`` and ``2.b`` fulfill this requirement.
+
+CMake also supports the keyword ``EXACT``, to ensure an exact version is used, if that is required.
+In this case, the application CMakeLists.txt could be written as:
+
+.. code-block:: cmake
+
+   cmake_minimum_required(VERSION 3.13.1)
+   find_package(Zephyr 2.a EXACT)
+   project(app)
+
+In case no Zephyr is found which satisfies the version required, as example, the application specifies
+
+.. code-block:: cmake
+
+   cmake_minimum_required(VERSION 3.13.1)
+   find_package(Zephyr 2.z)
+   project(app)
+
+then an error similar to below will be printed::
+
+  Could not find a configuration file for package "Zephyr" that is compatible
+  with requested version "2.z".
+
+  The following configuration files were considered but not accepted:
+
+    <projects>/zephyr-workspace-2.a/zephyr/share/zephyr-package/cmake/ZephyrConfig.cmake, version: 2.a.0
+    <projects>/zephyr-workspace-2.b/zephyr/share/zephyr-package/cmake/ZephyrConfig.cmake, version: 2.b.0
+
+
+.. note:: It can also be beneficial to specify a version number for Zephyr repository applications
+          and Zephyr workspace applications. Specifying a version in those cases ensures the
+	  application will only build if the Zephyr repository or workspace is matching.
+	  This can be useful to avoid accidental builds when only part of a workspace has been
+	  updated.
+
+
+Multiple Zephyr Installations (Zephyr workspace)
+************************************************
+
+Testing out a new Zephyr version, while at the same time keeping the existing Zephyr in the
+workspace untouched is sometimes beneficial.
+
+Or having both an upstream Zephyr, Vendor specific, and a custom Zephyr in same workspace.
+
+For example:
+
+.. code-block:: none
+
+   <projects>/zephyr-workspace
+   ├── zephyr
+   ├── zephyr-vendor
+   ├── zephyr-custom
+   ├── ...
+   └── my_applications
+        └── my_first_app
+
+
+in this setup, ``find_package(Zephyr)`` has the following order of precedence for selecting
+which Zephyr to use:
+
+* Project name: ``zephyr``
+* First project, when Zephyr projects are ordered lexicographical, in this case.
+
+  * ``zephyr-custom``
+  * ``zephyr-vendor``
+
+This means that ``my_first_app`` will use ``<projects>/zephyr-workspace/zephyr``.
+
+It is possible to specify a Zephyr preference list in the application.
+
+A Zephyr preference list can be specified as:
+
+.. code-block:: cmake
+
+   cmake_minimum_required(VERSION 3.13.1)
+
+   set(ZEPHYR_PREFER "zephyr-custom" "zephyr-vendor")
+   find_package(Zephyr)
+
+   project(my_first_app)
+
+
+the ``ZEPHYR_PREFER`` is a list, allowing for multiple Zephyrs.
+If a Zephyr is specified in the list, but not found in the system, it is simply ignored and
+``find_package(Zephyr)`` will continue to the next candidate.
+
+
+This allows for temporary creation of a new Zephyr release to be tested, without touching current
+Zephyr. When testing is done, the ``zephyr-test`` folder can simply be removed.
+Such a CMakeLists.txt could look as:
+
+.. code-block:: cmake
+
+   cmake_minimum_required(VERSION 3.13.1)
+
+   set(ZEPHYR_PREFER "zephyr-test")
+   find_package(Zephyr)
+
+   project(my_first_app)
+
+Zephyr CMake package source code
+********************************
+
+The Zephyr CMake package source code in
+``<PATH-TO-ZEPHYR>/share/zephyr-package/cmake`` contains the CMake config package
+which is used by CMake ``find_package`` function.
+
+It also contains code for exporting Zephyr as a CMake config package.
+
+The following is an overview of those files
+
+:file:`CMakeLists.txt`
+    The CMakeLists.txt file for the CMake build system which is responsible for
+    exporting Zephyr as a package to the CMake user package registry.
+
+:file:`ZephyrConfigVersion.cmake`
+    The Zephyr package version file. This file is called by CMake to determine
+    if this installation fulfils the requirements specified by user when calling
+    ``find_package(Zephyr ...)``. It is also responsible for detection of Zephyr
+    repository or workspace only installations.
+
+:file:`ZephyrUnittestConfigVersion.cmake`
+    Same responsibility as ``ZephyrConfigVersion.cmake``, but for unit tests.
+    Includes ``ZephyrConfigVersion.cmake``.
+
+:file:`ZephyrConfig.cmake`
+    The Zephyr package file. This file is called by CMake to for the package
+    meeting which fulfils the requirements specified by user when calling
+    ``find_package(Zephyr ...)``. This file is responsible for sourcing of
+    boilerplate code.
+
+:file:`ZephyrUnittestConfig.cmake`
+    Same responsibility as ``ZephyrConfig.cmake``, but for unit tests.
+    Includes ``ZephyrConfig.cmake``.
+
+:file:`zephyr_package_search.cmake`
+   Common file used for detection of Zephyr repository and workspace candidates.
+   Used by ``ZephyrConfigVersion.cmake`` and ``ZephyrConfig.cmake`` for common code.
+
+:file:`pristine.cmake`
+   Pristine file for removing all files created by CMake during configure and generator time when
+   exporting Zephyr CMake package. Running pristine keeps all package related files mentioned above.
+
+.. _CMake package: https://cmake.org/cmake/help/latest/manual/cmake-packages.7.html
+.. _CMake user package registry: https://cmake.org/cmake/help/latest/manual/cmake-packages.7.html#user-package-registry
+.. _CMake package version: https://cmake.org/cmake/help/latest/command/find_package.html#version-selection