doc: API: Document the modification of stable APIs

In order to make technical progress possible, some stable APIs
may sometimes need to be modified in a non backwards-compatible way.
Describe the process for integrating such changes in the documentation.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>

doc/development_process/api_lifecycle.rst

@@ -77,9 +77,90 @@ An API can be declared ``stable`` after fulfilling the following requirements:
 - Stable APIs can get backward compatible updates, bug fixes and security fixes
   at any time.
 
+.. _stable_api_changes:
+
+Introducing incompatible changes
+================================
+
+A stable API, as described above strives to remain backwards-compatible through
+its life-cycle. There are however cases where fulfilling this objective prevents
+technical progress or is simply unfeasible without unreasonable burden on the
+maintenance of the API and its implementation(s).
+
+An incompatible change is defined as one that forces users to modify their
+existing code in order to maintain the current behavior of their application.
+The need for recompilation of applications (without changing the application
+itself) is not considered an incompatible change.
+
+In order to restrict and control the introduction of a change that breaks the
+promise of backwards compatibility the following steps must be followed whenever
+such a change is considered necessary in order to accept it in the project:
+
+#. An :ref:`RFC issue <rfcs>` must be opened on GitHub with the following
+   content:
+
+   .. code-block:: none
+
+      Title:     RFC: API Change: <subsystem>
+      Contents:  - Problem Description:
+                   - Background information on why the change is required
+                 - Proposed Change (detailed):
+                   - Brief description of the API change
+                 - Detailed RFC:
+                   - Function call changes
+                   - Device Tree changes (source and bindings)
+                   - Kconfig option changes
+                 - Dependencies:
+                   - Impact to users of the API, including the steps required
+                     to adapt out-of-tree users of the API to the change
+
+   Instead of a written description of the changes, the RFC issue may link to a
+   Pull Request containing those changes in code form.
+#. The RFC issue must be labeled with the GitHub ``Stable API Change`` label
+#. The RFC issue must be submitted for discussion in the next `Zephyr API meeting`_
+#. An email must be sent to the ``devel`` mailing list with a subject identical
+   to the RFC issue title and that links to the RFC issue
+
+The RFC will then receive feedback through issue comments and will also be
+discussed in the Zephyr API meeting, where the stakeholders and the community at
+large will have a chance to discuss it in detail.
+
+Finally, and if not done as part of the first step, a Pull Request must be
+opened on GitHub. It is left to the person proposing the change to decide
+whether to introduce both the RFC and the Pull Request at the same time or to
+wait until the RFC has gathered consensus enough so that the implementation can
+proceed with confidence that it will be accepted.
+The Pull Request must include the following:
+
+- A title that matches the RFC issue
+- A link to the RFC issue
+- The actual changes to the API
+
+  - Changes to the API header file
+  - Changes to the API implementation(s)
+  - Changes to the relevant API documentation
+  - Changes to Device Tree source and bindings
+
+- The changes required to adapt in-tree users of the API to the change.
+  Depending on the scope of this task this might require additional help from
+  the corresponding maintainers
+- An entry in the "API Changes" section of the release notes for the next
+  upcoming release
+- The labels ``API``, ``Stable API Change`` and ``Release Notes``, as well as
+  any others that are applicable
+
+Once the steps above have been completed, the outcome of the proposal will
+depend on the approval of the actual Pull Request by the maintainer of the
+corresponding subystem. As with any other Pull Request, the author can request
+for it to be discussed and ultimately even voted on in the `Zephyr TSC meeting`_.
+
+If the Pull Request is merged then an email must be sent to the ``devel`` and
+``user`` mailing lists informing them of the change.
+
 .. note::
 
-   Incompatible changes will be announced in the release notes.
+   Incompatible changes will be announced in the "API Changes" section of the
+   release notes.
 
 Deprecated
 ***********
@@ -87,6 +168,8 @@ Deprecated
 .. note::
 
    Unstable APIs can be removed without deprecation at any time.
+   Deprecation and removal of APIs will be announced in the "API Changes"
+   section of the release notes.
 
 The following are the requirements for deprecating an existing API:
 
@@ -100,6 +183,8 @@ The following are the requirements for deprecating an existing API:
 
   - Mark as deprecated
   - Document the deprecation
+  - Include the deprecation in the "API Changes" of the release notes for the
+    next upcoming release
   - Code using the deprecated API needs to be modified to remove usage of said
     API
   - The change needs to be atomic and bisectable
@@ -134,3 +219,6 @@ the release notes, mailing lists, Github issues and pull-requests.
 If it's not OK to remove the API, the maintainers will continue to support
 migration and update the roadmap with the aim to remove the API in the next
 release.
+
+.. _`Zephyr TSC meeting`: https://github.com/zephyrproject-rtos/zephyr/wiki/Zephyr-Committee-and-Working-Group-Meetings#technical-steering-committee-tsc
+.. _`Zephyr API meeting`: https://github.com/zephyrproject-rtos/zephyr/wiki/Zephyr-Committee-and-Working-Group-Meetings#zephyr-api-meeting

doc/development_process/dev_env_and_tools.rst

@@ -105,14 +105,14 @@ Security
 Changes that appear to have an impact to the overall security of the system need
 to be reviewed by a security expert from the security working group.
 
-TSC
+TSC and Working Groups
-++++
+++++++++++++++++++++++
 
 Changes that introduce new features or functionality or change the way the
 overall system works need to be reviewed by the TSC or the responsible Working
-Group. For example for API changes, the API working group needs to be consulted
+Group. For example for :ref:`stable API changes <stable_api_changes>`, the
-and made aware of such changes.
+proposal needs to be presented in the API meeting so that the relevant
-
+stakeholders are made aware of the change.
 
 A Pull-Request should have an Assignee
 =======================================
@@ -278,6 +278,16 @@ Description    The issue is to be discussed in the following
 
 .. _`dev-review/TSC meeting`: https://github.com/zephyrproject-rtos/zephyr/wiki/Zephyr-Committee-and-Working-Group-Meetings
 
+Stable API changes
+==================
+
+=============  ===============================================================
+Labels         ``Stable API Change``
+Applicable to  PRs  and issues
+Description    The issue or PR describes a change to a stable API. See
+               additional information in :ref:`stable_api_changes`
+=============  ===============================================================
+
 Minimum PR review time
 ======================
 

doc/development_process/proposals.rst

@@ -49,6 +49,8 @@ by any project member or the community. You should contact an assigned owner if
 you'd like to discuss or contribute to that feature's implementation
 
 
+.. _rfcs:
+
 Proposals and RFCs
 *******************
 
@@ -67,6 +69,7 @@ examples that would benefit from an RFC are:
 
 - A new feature that creates new API surface area, and would require a feature
   flag if introduced.
+- The modification of an existing stable API
 - The removal of features that already shipped as part of Zephyr.
 - The introduction of new idiomatic usage or conventions, even if they do not
   include code changes to Zephyr itself.
@@ -88,6 +91,17 @@ Some changes do not require an RFC:
 - Addition of new boards, SoCs or drivers to existing subsystems
 - ...
 
+The process in itself consists in creating a GitHub issue with the :ref:`RFC
+label <gh_labels>` that documents the proposal thoroughly. There is an `RFC
+template`_ included in the main Zephyr GitHub repository that serves as a
+guideline to write a new RFC.
+
+As with Pull Requests, RFCs might require discussion in the context of one of
+the `Zephyr meetings`_ in order to move it forward in cases where there is
+either disagreement or not enough voiced opinions in order to proceed. Make sure
+to either label it appropriately or include it in the corresponding GitHub
+project in order for it to be examined during the next meeting.
+
 Roadmap and Release Plans
 *************************
 
@@ -132,3 +146,6 @@ and main stakeholders of the project can make progress on.
 
 Items labeled as ``features`` are short or long term release items that shall
 have an assignee and a milestone set.
+
+.. _`RFC template`: https://github.com/zephyrproject-rtos/zephyr/blob/master/.github/ISSUE_TEMPLATE/rfc---proposal.md
+.. _`Zephyr meetings`: https://github.com/zephyrproject-rtos/zephyr/wiki/Zephyr-Committee-and-Working-Group-Meetings