michaelh/zephyr
1
0
Fork 0
Code Releases Activity

doc: add more details about memory domains

Browse source 
We now more throroughly discuss memory domains, thread
resource pools, and automatic memory domains.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
This commit is contained in:
Andrew Boie 2019-08-15 16:10:49 -07:00 • committed by Anas Nashif
commit 714e37fb78
3 changed files with 316 additions and 114 deletions
Download patch file Download diff file Expand all files Collapse all files

3
doc/reference/usermode/index.rst
View file

@ -182,9 +182,8 @@ for execution after the kernel starts:
.. toctree::
:maxdepth: 2
memory_domain.rst
kernelobjects.rst
syscalls.rst
memory_domain.rst
mpu_stack_objects.rst
mpu_userspace.rst
usermode_sharedmem.rst

352
doc/reference/usermode/memory_domain.rst
View file

@ -1,40 +1,316 @@
.. _memory_domain:
Memory Domain
#############
Memory Protection Design
########################
The memory domain APIs are used by unprivileged threads to share data to
the threads in the same memory domain and protect sensitive data from threads
outside their domain. Memory domains are not only used for improving security,
but are also useful for debugging (unexpected access would cause an exception).
Zephyr's memory protection design is geared towards microcontrollers with MPU
(Memory Protection Unit) hardware. We do support some architectures which have
a paged MMU (Memory Management Unit), but in that case the MMU is used like
an MPU with an identity page table.
Since architectures generally have constraints on how many partitions can be
defined, and the size/alignment of each partition, users may need to group
related data together using linker sections.
All of the discussion below will be using MPU terminology; systems with MMUs
can be considered to have an MPU with an unlimited number of programmable
regions.
.. contents::
:local:
:depth: 2
There are a few different levels on how memory access is configured when
Zephyr memory protection features are enabled, which we will describe here:
Concepts
********
Boot Time Memory Configuration
******************************
A memory domain contains some number of memory partitions.
A memory partition is a memory region (might be RAM, peripheral registers,
or flash, for example) with specific attributes (access permission, e.g.
privileged read/write, unprivileged read-only, or execute never).
Memory partitions are defined by a set of underlying MPU regions
or MMU tables. A thread belongs to a single memory domain at
any point in time but a memory domain may contain multiple threads.
Threads in the same memory domain have the same access permissions
to the memory partitions belonging to the memory domain. New threads
will inherit any memory domain configuration from the parent thread.
This is the configuration of the MPU after the kernel has started up. It should
contain the following:
Implementation
- Any configuration of memory regions which need to have special caching or
write-back policies for basic hardware and driver function. Note that most
MPUs have the concept of a default memory access policy map, which can be
enabled as a "background" mapping for any area of memory that doesn't
have an MPU region configuring it. It is strongly recommended to use this
to maximize the number of available MPU regions for the end user. On
ARMv7-M/ARMv8-M this is called the System Address Map, other CPUs may
have similar capabilities.
- A read-only, executable region or regions for program text and ro-data, that
is accessible to user mode. This could be further sub-divided into a
read-only region for ro-data, and a read-only, executable region for text, but
this will require an additional MPU region. This is required so that
threads running in user mode can read ro-data and fetch instructions.
- Depending on configuration, user-accessible read-write regions to support
extra features like GCOV, HEP, etc.
Assuming there is a background map which allows supervisor mode to access any
memory it needs, and regions are defined which grant user mode access to
text/ro-data, this is sufficient for the boot time configuration.
Hardware Stack Overflow
***********************
``CONFIG_HW_STACK_PROTECTION`` is an optional feature which detects stack
buffer overflows when the system is running in supervisor mode. This
catches issues when the entire stack buffer has overflowed, and not
individual stack frames, use compiler-assisted CONFIG_STACK_CANARIES for that.
Like any crash in supervisor mode, no guarantees can be made about the overall
health of the system after a supervisor mode stack overflow, and any instances
of this should be treated as a serious error. However it's still very useful to
know when these overflows happen, as without robust detection logic the system
will either crash in mysterious ways or behave in an undefined manner when the
stack buffer overflows.
Some systems implement this feature by creating at runtime a 'guard' MPU region
which is set to be read-only and is at either the beginning or immediately
preceding the supervisor mode stack buffer. If the stack overflows an
exception will be generated.
This feature is optional and is not required to catch stack overflows in user
mode; disabling this may free 1-2 MPU regions depending on the MPU design.
Other systems may have dedicated CPU support for catching stack overflows
and no extra MPU regions will be required.
Thread Stack
************
Any thread running in user mode will need access to its own stack buffer.
On context switch into a user mode thread, a dedicated MPU region will be
programmed with the bounds of the stack buffer. A thread exceeding its stack
buffer will start pushing data onto memory it doesn't have access to and a
memory access violation exception will be generated.
Thread Resource Pools
*********************
A small subset of kernel APIs, invoked as system calls, require heap memory
allocations. This memory is used only by the kernel and is not accessible
directly by user mode. In order to use these system calls, invoking threads
must assign themselves to a resource pool, which is a k_mem_pool object.
Memory is drawn from a thread's resource pool using :c:func:`z_thread_malloc()`
and freed with :c:func:`k_free()`.
The APIs which use resource pools are as follows, with any alternatives
noted for users who do not want heap allocations within their application:
- :c:func:`k_stack_alloc_init()` sets up a k_stack with its storage
buffer allocated out of a resource pool instead of a buffer provided by the
user. An alternative is to declare k_stacks that are automatically
initialized at boot with :c:macro:`K_STACK_DEFINE()`, or to initialize the
k_stack in supervisor mode with :c:func:`k_stack_init()`.
- :c:func:`k_pipe_alloc_init()` sets up a k_pipe object with its
storage buffer allocated out of a resource pool instead of a buffer provided
by the user. An alternative is to declare k_pipes that are automatically
initialized at boot with :c:macro:`K_PIPE_DEFINE()`, or to initialize the
k_pipe in supervisor mode with :c:func:`k_pipe_init()`.
- :c:func:`k_msgq_alloc_init()` sets up a k_msgq object with its
storage buffer allocated out of a resource pool instead of a buffer provided
by the user. An alternative is to declare a k_msgq that is automatically
initialized at boot with :c:macro:`K_MSGQ_DEFINE()`, or to initialize the
k_msgq in supervisor mode with :c:func:`k_msgq_init()`.
- :c:func:`k_poll()` when invoked from user mode, needs to make a kernel-side
copy of the provided events array while waiting for an event. This copy is
freed when :c:func:`k_poll()` returns for any reason.
- :c:func:`k_queue_alloc_prepend()` and :c:func:`k_queue_alloc_append()`
allocate a container structure to place the data in, since the internal
bookkeeping information that defines the queue cannot be placed in the
memory provided by the user.
- :c:func:`k_object_alloc()` allows for entire kernel objects to be
dynamically allocated at runtime and a usable pointer to them returned to
the caller.
The relevant API is :c:func:`k_thread_resource_pool_assign()` which assigns
a k_mem_pool to draw these allocations from for the target thread.
If the system heap is enabled, then the system heap may be used with
:c:func:`k_thread_system_pool_assign()`, but it is preferable for different
logical applications running on the system to have their own pools.
Memory Domains
**************
The kernel ensures that any user thread will have access to its own stack
buffer, plus program text and read-only data. The memory domain APIs are the
way to grant access to additional blocks of memory to a user thread.
Conceptually, a memory domain is a collection of some number of memory
partitions. The maximum number of memory partitions in a domain
is limited by the number of available MPU regions. This is why it is important
to minimize the number of boot-time MPU regions.
Memory domains are *not* intended to control access to memory from supervisor
mode. In some cases this may be unavoidable; for example some architectures do
not allow for the definition of regions which are read-only to user mode but
read-write to supervisor mode. A great deal of care must be taken when working
with such regions to not unintentionally cause the kernel to crash when
accessing such a region.
Memory domain APIs are only available to supervisor mode. The only control
user mode has over memory domains is that any user thread's child threads
will automatically become members of the parent's domain.
Memory Partitions
=================
Each memory partition consists of a memory address, a size,
and access attributes. It is intended that memory partitions are used to
control access to system memory. Defining memory partitions are subject
to the following constraints:
- The partition must represent a memory region that can be programmed by
the underlying memory management hardware, and needs to conform to any
underlying hardware constraints. For example, many MPU-based systems require
that partitions be sized to some power of two, and aligned to their own
size. For MMU-based systems, the partition must be aligned to a page and
the size some multiple of the page size.
- Partitions within the same memory domain may not overlap each other. There is
no notion of precedence among partitions within a memory domain. Partitions
within a memory domain are assumed to have a higher precedence than any
boot-time memory regions, however whether a memory domain partition can
overlap a boot-time memory region is architecture specific.
- The same partition may be specified in multiple memory domains. For example
there may be a shared memory area that multiple domains grant access to.
- Care must be taken in determining what memory to expose in a partition.
It is not appropriate to provide direct user mode access to any memory
containing private kernel data.
- Memory domain partitions are intended to control access to system RAM.
Configuration of memory partitions which do not correspond to RAM
may not be supported by the architecture; this is true for MMU-based systems.
There are two ways to define memory partitions: either manually or
automatically.
Manual Memory Partitions
------------------------
The following code declares a global array buf, and then declares
a read-write partition for it which may be added to a domain:
.. code-block:: c
u8_t __aligned(32) buf[32];
K_MEM_PARTITION_DEFINE(my_partition, buf, sizeof(buf),
K_MEM_PARTITION_P_RW_U_RW);
This does not scale particularly well when we are trying to contain multiple
objects spread out across several C files into a single partition.
Automatic Memory Partitions
---------------------------
Automatic memory partitions are created by the build system. All globals
which need to be placed inside a partition are tagged with their destination
partition. The build system will then coalesce all of these into a single
contiguous block of memory, zero any BSS variables at boot, and define
a memory partition of appropriate base address and size which contains all
the tagged data.
Automatic memory partitions are only configured as read-write
regions. They are defined with :c:macro:`K_APPMEM_PARTITION_DEFINE()`.
Global variables are then routed to this partition using
:c:macro:`K_APP_DMEM()` for initialized data and :c:macro:`K_APP_BMEM()` for
BSS.
.. code-block:: c
#include <app_memory/app_memdomain.h>
/* Declare a k_mem_partition "my_partition" that is read-write to
* user mode. Note that we do not specify a base address or size.
*/
K_APPMEM_PARTITION_DEFINE(my_partition);
/* The global variable var1 will be inside the bounds of my_partition
* and be initialized with 37 at boot.
*/
K_APP_DMEM(my_partition) int var1 = 37;
/* The global variable var2 will be inside the bounds of my_partition
* and be zeroed at boot size K_APP_BMEM() was used, indicating a BSS
* variable.
*/
K_APP_BMEM(my_partition) int var2;
The build system will ensure that the base address of my_partition will
be properly aligned, and the total size of the region conforms to the memory
management hardware requirments, adding padding if necessary.
If multiple partitions are being created, a variadic preprocessor macro can be
used as provided in ``app_macro_support.h``:
.. code-block:: c
FOR_EACH(K_APPMEM_PARTITION_DEFINE, part0, part1, part2);
There are some kernel objects which are defined by macros and take an argument
for a destination section. A good example of these are sys_mem_pools, which
are heap objects. The destination section name for an automatic partition
can be obtained with :c:macro:`K_APP_DMEM_SECTION()` and
:c:macro:`K_APP_BMEM_SECTION()` respectively for initialized data and BSS:
.. code-block:: c
/* Declare automatic memory partition foo_partition */
K_APPMEM_PARTITION_DEFINE(foo_partition);
/* Section argument for the destination section obtained via
* K_APP_DMEM_SECTION()
*/
SYS_MEM_POOL_DEFINE(foo_pool, NULL, BLK_SIZE_MIN, BLK_SIZE_MAX,
BLK_NUM_MAX, BLK_ALIGN,
K_APP_DMEM_SECTION(foo_partition));
Automatic Partitions for Static Library Globals
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The build-time logic for setting up automatic memory partitions is in
``scripts/gen_app_partitions.py``. If a static library is linked into Zephyr,
it is possible to route all the globals in that library to a specific
memory partition with the ``--library`` argument.
For example, if the Newlib C library is enabled, the Newlib globals all need
to be placed in ``z_libc_partition``. The invocation of the script in the
top-level ``CMakeLists.txt`` adds the following:
.. code-block:: none
gen_app_partitions.py ... --library libc.a z_libc_partition ..
There is no support for expressing this in the project-level configuration
or build files; the toplevel ``CMakeLists.txt`` must be edited.
Pre-defined Memory Partitions
-----------------------------
There are a few memory partitions which are pre-defined by the system:
- ``z_malloc_partition`` - This partition contains the system-wide pool of
memory used by libc malloc(). Due to possible starvation issues, it is
not recommended to draw heap memory from a global pool, instead
it is better to define various sys_mem_pool objects and assign them
to specific memory domains.
- ``z_libc_partition`` - Contains globals required by the C library and runtime.
Required if using newlib. Required if using minimal libc with
``CONFIG_STACK_CANARIES`` enabled.
Library-specific partitions are listed in ``include/app_memory/partitions.h``.
For example, to use the MBEDTLS library from user mode, the
``k_mbedtls_partition`` must be added to the domain.
Memory Domain Usage
===================
Create a Memory Domain
======================
----------------------
A memory domain is defined using a variable of type
:c:type:`struct k_mem_domain`. It must then be initialized by calling
@ -49,7 +325,7 @@ The following code defines and initializes an empty memory domain.
k_mem_domain_init(&app0_domain, 0, NULL);
Add Memory Partitions into a Memory Domain
==========================================
------------------------------------------
There are two ways to add memory partitions into a memory domain.
@ -97,20 +373,22 @@ memory domain one by one.
The maximum number of memory partitions is limited by the maximum
number of MPU regions or the maximum number of MMU tables.
Add Threads into a Memory Domain
================================
Memory Domain Assignment
------------------------
Adding threads into a memory domain grants threads permission to access
the memory partitions in the memory domain.
The following code shows how to add threads into a memory domain.
Any thread may join a memory domain, and any memory domain may have multiple
threads assigned to it. Threads are assigned to memory domains with an API
call:
.. code-block:: c
k_mem_domain_add_thread(&app0_domain, app_thread_id);
In addition, if a thread is a member of a memory domain, and it creates a
child thread, that thread will belong to the domain as well.
Remove a Memory Partition from a Memory Domain
==============================================
----------------------------------------------
The following code shows how to remove a memory partition from a memory
domain.
@ -124,7 +402,7 @@ that matches the given parameter and removes that partition from the
memory domain.
Remove a Thread from the Memory Domain
======================================
--------------------------------------
The following code shows how to remove a thread from the memory domain.
@ -133,7 +411,7 @@ The following code shows how to remove a thread from the memory domain.
k_mem_domain_remove_thread(app_thread_id);
Destroy a Memory Domain
=======================
-----------------------
The following code shows how to destroy a memory domain.
@ -142,7 +420,7 @@ The following code shows how to destroy a memory domain.
k_mem_domain_destroy(&app0_domain);
Available Partition Attributes
==============================
------------------------------
When defining a partition, we need to set access permission attributes
to the partition. Since the access control of memory partitions relies on

75
doc/reference/usermode/usermode_sharedmem.rst
View file

@ -1,75 +0,0 @@
.. _usermode_sharedmem:
Application Shared Memory
#########################
.. note::
In this document, we will cover the basic usage of enabling shared
memory using a template around app_memory subsystem.
Overview
********
The use of subsystem app_memory in userspace allows control of
shared memory between threads. The foundation of the implementation
consists of memory domains and partitions. Memory partitions are created
and used in the definition of variable to group them into a
common space. The memory partitions are linked to domains
that are then assigned to a thread. The process allows selective
access to memory from a thread and sharing of memory between two
threads by assigning a partition to two different domains. By using
the shared memory template, code to protect memory can be used
on different platform without the application needing to implement
specific handlers for each platform. Note the developer should understand
the hardware limitations in context to the maximum number of memory
partitions available to a thread. Specifically processors with MPU's
cannot support the same number of partitions as a MMU.
This specific implementation adds a wrapper to simplify the programmers
task of using the app_memory subsystem through the use of macros and
a python script to generate the linker script. The linker script provides
the proper alignment for processors requiring power of two boundaries.
Without the wrapper, a developer is required to implement custom
linker scripts for each processor in the project.
The general usage is as follows. Include app_memory/app_memdomain.h
in the userspace source file. Mark the variable to be placed in
a memory partition. The two markers are for data and bss respectively:
K_APP_DMEM(id) and K_APP_BMEM(id). The id is used as the partition name.
The resulting section name can be seen in the linker.map as
"data_smem_id" and "data_smem_idb".
To create a k_mem_partition, call the macro K_APPMEM_PARTITION_DEFINE(part0)
where "part0" is the name then used to refer to that partition. The
standard memory domain APIs may be used to add it to domains; the declared
name is a k_mem_partition symbol.
Example:
.. code-block:: c
/* create partition at top of file outside functions */
K_APPMEM_PARTITION_DEFINE(part0);
/* create domain */
struct k_mem_domain dom0;
/* assign variables to the domain */
K_APP_DMEM(part0) int var1;
K_APP_BMEM(part0) static volatile int var2;
int main()
{
k_mem_domain_init(&dom0, 0, NULL)
k_mem_domain_add_partition(&dom0, part0);
k_mem_domain_add_thread(&dom0, k_current_get());
...
}
If multiple partitions are being created, a variadic
preprocessor macro can be used as provided in
app_macro_support.h:
.. code-block:: c
FOR_EACH(K_APPMEM_PARTITION_DEFINE, part0, part1, part2);