userspace: add additional script documentation

We have several scripts used by the build system related
to generating code for system calls, privileged mode stacks,
kernel object metadata, and application shared memory
partitions. Add some overview documentation for each.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>

scripts/gen_app_partitions.py

@@ -4,6 +4,34 @@
 #
 # SPDX-License-Identifier: Apache-2.0
 
+"""
+Script to generate a linker script organizing application memory partitions
+
+Applications may declare build-time memory domain partitions with
+K_APPMEM_PARTITION_DEFINE, and assign globals to them using K_APP_DMEM
+or K_APP_BMEM macros. For each of these partitions, we need to
+route all their data into appropriately-sized memory areas which meet the
+size/alignment constraints of the memory protection hardware.
+
+This linker script is created very early in the build process, before
+the build attempts to link the kernel binary, as the linker script this
+tool generates is a necessary pre-condition for kernel linking. We extract
+the set of memory partitions to generate by looking for variables which
+have been assigned to input sections that follow a defined naming convention.
+We also allow entire libraries to be pulled in to assign their globals
+to a particular memory partition via command line directives.
+
+This script takes as inputs:
+
+- The base directory to look for compiled objects
+- key/value pairs mapping static library files to what partitions their globals
+  should end up in.
+
+The output is a linker script fragment containing the definition of the
+app shared memory section, which is further divided, for each partition
+found, into data and BSS for each partition.
+"""
+
 import sys
 import argparse
 import os

scripts/gen_kobject_list.py

@@ -3,6 +3,50 @@
 # Copyright (c) 2017 Intel Corporation
 #
 # SPDX-License-Identifier: Apache-2.0
+"""
+Script to generate gperf tables of kernel object metadata
+
+User mode threads making system calls reference kernel objects by memory
+address, as the kernel/driver APIs in Zephyr are the same for both user
+and supervisor contexts. It is necessary for the kernel to be able to
+validate accesses to kernel objects to make the following assertions:
+
+    - That the memory address points to a kernel object
+
+    - The kernel object is of the expected type for the API being invoked
+
+    - The kernel object is of the expected initialization state
+
+    - The calling thread has sufficient permissions on the object
+
+The zephyr build generates an intermediate ELF binary, zephyr_prebuilt.elf,
+which this script scans looking for kernel objects by examining the DWARF
+debug information to look for instances of data structures that are considered
+kernel objects. For device drivers, the API struct pointer populated at build
+time is also examined to disambiguate between various device driver instances
+since they are all 'struct device'.
+
+The result of this script is five generated files:
+
+    - A gperf script to generate the hash table mapping kernel object memory
+      addresses to kernel object metadata, used to track permissions,
+      object type, initialization state, and any object-specific data.
+
+    - A header file containing generated macros for validating driver instances
+      inside the system call handlers for the driver subsystem APIs.
+
+    - A header file defining enumerated types for all the different kernel
+      object types.
+
+    - A C code fragment, included by kernel/userspace.c, for printing
+      human-readable representations of kernel object types in the
+      otype_to_str() function.
+
+    - A C code fragment, included by kernel/userspace.c, for mapping
+      kernel object types to the sizes of those kernel objects, used for
+      allocating instances of them at runtime (CONFIG_DYNAMIC_OBJECTS)
+      in the obj_size_get() function.
+"""
 
 import sys
 import argparse

scripts/gen_priv_stacks.py

@@ -4,6 +4,23 @@
 #
 # SPDX-License-Identifier: Apache-2.0
 
+"""
+Script to generate gperf tables mapping threads to their privileged mode stacks
+
+Some MPU devices require that memory region definitions be aligned to their
+own size, which must be a power of two. This introduces difficulties in
+reserving memory for the thread's supervisor mode stack inline with the
+K_THREAD_STACK_DEFINE() macro.
+
+Instead, the stack used when a user thread elevates privileges is allocated
+elsewhere in memory, and a gperf table is created to be able to quickly
+determine where the supervisor mode stack is in memory. This is accomplished
+by scanning the DWARF debug information in zephyr_prebuilt.elf, identifying
+instances of 'struct k_thread', and emitting a gperf configuration file which
+allocates memory for each thread's privileged stack and creates the table
+mapping thread addresses to these stacks.
+"""
+
 import sys
 import argparse
 import struct

scripts/gen_syscall_header.py

@@ -4,6 +4,17 @@
 #
 # SPDX-License-Identifier: Apache-2.0
 
+"""
+Generation script for syscall_macros.h
+
+The generation of macros for invoking system calls of various number
+of arguments, in different execution types (supervisor only, user only,
+mixed supervisor/user code) is tedious and repetitive. Rather than writing
+by hand, this script generates it.
+
+This script has no inputs, and emits the generated header to stdout.
+"""
+
 import sys
 from enum import Enum
 

scripts/gen_syscalls.py

@@ -4,6 +4,25 @@
 #
 # SPDX-License-Identifier: Apache-2.0
 
+"""
+Script to generate system call invocation macros
+
+This script parses the system call metadata JSON file emitted by
+parse_syscalls.py to create several files:
+
+- A file containing weak aliases of any potentially unimplemented system calls,
+  as well as the system call dispatch table, which maps system call type IDs
+  to their handler functions.
+
+- A header file defing the system call type IDs, as well as function
+  prototypes for all system call handler functions.
+
+- A directory containing header files. Each header corresponds to a header
+  that was identified as containing system call declarations. These
+  generated headers contain the inline invocation functions for each system
+  call in that header.
+"""
+
 import sys
 import re
 import argparse

scripts/parse_syscalls.py

@@ -4,6 +4,22 @@
 #
 # SPDX-License-Identifier: Apache-2.0
 
+"""
+Script to scan Zephyr include directories and emit system call metadata
+
+System calls require a great deal of boilerplate code in order to implement
+completely. This script is the first step in the build system's process of
+auto-generating this code by doing a text scan of directories containing
+header files, and building up a database of system calls and their
+function call prototypes. This information is emitted to a generated
+JSON file for further processing.
+
+If the output JSON file already exists, its contents are checked against
+what information this script would have outputted; if the result is that the
+file would be unchanged, it is not modified to prevent unnecessary
+incremental builds.
+"""
+
 import sys
 import re
 import argparse