Commit graph

2639 commits

Author SHA1 Message Date
Anas Nashif
81b272119a docs: fixed documenation pointers
Fixed URL to documentation, now latest docs are under /latest/..
Fixes #9932

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-09-12 12:03:18 -04:00
Flavio Ceolin
3259ac08ca kernel: userspace: Sanitize switch usage
MISRA-C requires that every switch clause has a break instruction.
Changing gen_kobject_list script to generates compliance code.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2018-09-12 10:05:06 -04:00
Marti Bolivar
5cf80e4e22 scripts: requirements.txt: fix PyYAML for python 3.7
PyYAML 3.13 includes a fix necessary to install on Python 3.7:

    3.13 (2018-07-05)

    Rebuild wheels using latest Cython for Python 3.7 support.

Support it by accepting any PyYAML at least as recent as the current
version in requirements.txt, currently 3.12.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2018-09-11 20:58:09 -04:00
Ulf Magnusson
d0e8752a52 Kconfig: Rename $ENV_VAR_{ARCH,BOARD_DIR} to $ARCH/$BOARD_DIR
The prefixes might be a leftover from the old 'option env="..."' symbols
(which are no longer needed). Since environment variables can be
referenced directly now, there's no point in having a prefix.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-09-11 19:17:25 -04:00
Ulf Magnusson
213a88f7ff Kconfig: Support macro expansion within symbol names
Update Kconfiglib to upstream revision 2be02fac78527 to add support for
expanding macros within symbol names, extending the existing Kconfig
preprocessor
(https://github.com/torvalds/linux/blob/master/Documentation/kbuild/
kconfig-macro-language.txt). Some minor optimizations are included as
well.

This makes it possible to add Kconfig templates to avoid code
repetition, e.g. like below:

Kconfig.log_template:

    choice
    	prompt "Max compiled-in log level for $(module-str)"

    config $(module)_LOG_LEVEL_OFF
    	bool "Off"

    config $(module)_LOG_LEVEL_ERR
    	bool "Error"

    config $(module)_LOG_LEVEL_WRN
    	bool "Warning"

    config $(module)_LOG_LEVEL_INF
    	bool "Info"

    config $(module)_LOG_LEVEL_DBG
    	bool "Debug"

    endchoice

    config $(module)_LOG_LEVEL
    	int
    	default 0 if $(module)_LOG_LEVEL_OFF
    	default 1 if $(module)_LOG_LEVEL_ERR
    	default 2 if $(module)_LOG_LEVEL_WRN
    	default 3 if $(module)_LOG_LEVEL_INF
    	default 4 if $(module)_LOG_LEVEL_DBG

Using the template:

    module = FOO
    module-str = foo
    source "Kconfig.log_template"

    ...

    module = BAR
    module-str = bar
    source "Kconfig.log_template"

This feature might create harder-to-read Kconfig files if abused, so it
should probably be used sparingly.

Fixes: #9761

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-09-11 16:08:38 +02:00
Anas Nashif
38a795074f sanitycheck: update release data
Update footprint data for release.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-09-09 21:07:19 -04:00
Kumar Gala
5824b81176 scripts: dts: extract: Fix handling of irq prop being a list of lists
Before dtc 1.4.7 we'd get something like the following for an interrupt
property in which #interrupt-cells is 2.

interrupts = <1 2 3 4>;

After dtc 1.4.7 we get:

interrupts = <1 2>, <3 4>;

We should handle both cases in the extract interrupt handling code.  So
if we see a list of lists, we flatten it to a single list to normalize
the property.

Fixes: #9558

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2018-09-06 09:46:03 -05:00
Ulf Magnusson
af6af0fb8a scripts: kconfig: Put a blank line before warnings and errors
Since the warnings generated by this script are so spammy, it can be
difficult to tell where they start and end. Put a blank line before each
one to make it clearer.

Also put blank lines around the non-whitelisted-warning error.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-09-04 14:28:15 -04:00
Ulf Magnusson
49fa4d5627 scripts: kconfig: Use unique_{defined_syms,choices} and clean up a bit
- Use Kconfig.unique_{defined_syms,choices} to avoid redundant checks
   for symbols/choices defined in multiple locations. These were
   recently added to Kconfiglib.

 - Remove the comment about the alldefconfig starting state. It probably
   isn't useful here.

 - Print the messages about loading configuration files just before they
   are actually loaded. That looks less confusing if one of them fails
   to load.

 - Line-wrap the error message about non-whitelisted warnings.

 - Misc. other minor code cleanup.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-09-04 14:28:15 -04:00
Istvan Bisz
1642c45717 scripts: dts: extract: clock.py fix node alias name
A node alias name should be applied to the all property names.

Before this patch, the alias name is applied erroneously
just for CLOCK_CONTROLLER, skipping the CLOCK_BITS and CLOCK_BUS
property names.

Signed-off-by: Istvan Bisz <istvan.bisz@t-online.hu>
2018-09-03 12:46:44 -04:00
Anas Nashif
77837e8107 sanitycheck: do not abort logging on faults
We have been dropping lines after finding a fault which resulted in
missing information in the log. Make sure we continue and only report
failure at the end of the execution.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-08-30 15:05:30 -04:00
Anas Nashif
39ae72b4cb sanitycheck: capture delayed faults
Do not close console after PASS is reported, wait a bit for any
remaining messages from the tests, sometimes we have faults that need to
be parsed.
This now works for Qemu handler, support for other handlers to follow.

Fixes #9646

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-08-30 15:05:30 -04:00
Ulf Magnusson
9f8d429048 menuconfig: Fix a case of visible symbols not being shown
Update menuconfig (and Kconfiglib, just to sync) to upstream revision
78682a8e3c4fe to get the following fix in, which could cause certain
visible symbols to not show up in the menuconfig interface:

    menuconfig: Always show implicit submenus with visible nodes

    Currently, the symbol BAR below (which ends up indented in an
    implicit submenu) is shown only if DEP is non-n (or if show-all mode
    is enabled):

      config FOO
            bool "foo" if DEP
            default y

      config BAR
            bool "bar" if FOO

    This is bad, because it hides visible symbols from the interface.
    The assumption was that an implicit submenu (which is only created
    if the parent has a prompt) would never have visible items when the
    parent item is invisible, but prompt-specific conditions and
    select/imply can break that assumption.

    Fix it by always showing implicit submenus with visible nodes, along
    with the parent node. If the parent node is invisible, show it in
    red, like in show-all mode (which happens automatically). That's
    probably better than having mysteriously indented nodes when the
    parent is invisible.

(Some other tweaks were made later to avoid showing red outside show-all
mode, because it might look confusing/broken.)

A new color scheme for the menuconfig has been added as well,
contributed by Mitja Horvat (pinkfluid). It can be enabled by setting
the environment variable MENUCONFIG_THEME to 'aquatic'.

The default theme has been tweaked to use bold text for the selected
item too.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-08-30 14:26:05 +02:00
Daniel Leung
9945e7fda0 doc: add ability to generate PDF
This adds new targets to generate build documentation through
LaTEX to PDF.

There are a few notes:

1. pdflatex complains about the tex file generated by doxygen
   so it needs to be fixed with a Python script before feeding
   in through pdflatex.

2. SVG files are not recognized by pdflatex so they are converted
   to known good format on the fly, only for producing PDF. This
   uses the libRSVG's rsvg-convert tool.

Relates to #6782.

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2018-08-29 18:16:56 -04:00
Bobby Noelte
5a16b9020f dts: bindings: scan application dir for bindings
In addtion to zephyr/dts/bindings als scan the dts/bindings
directory within the application source directory for bindings.

Allows to have application specific bindings and drivers.

Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
2018-08-27 12:50:13 -04:00
Ulf Magnusson
c1f54cc5fd Kconfig: Show include paths in menuconfig and documentation
Update Kconfiglib and menuconfig to upstream revision a28bc4da9762e,
which adds include path information to menuconfig, showing how the
Kconfig file of the symbol got 'source'd in addition to showing the
definition location.

Add include path information to the auto-generated Kconfig documentation
too.

Some small Kconfiglib bugs are fixed to, like error reporting for
recursive 'source's being broken (crashing instead of printing the
error), a minor file descriptor leak, and syntax checking not catching
extra trailing tokens after 'if <expr>' and 'depends on <expr>'.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-08-26 09:47:20 -07:00
Ulf Magnusson
d67095ba59 Kconfiglib: Make header symbol order match .config files again
Commit f6bf897780 ("kconfiglib: Add preprocessor and two warnings")
accidentally made the ordering of #define's in generated header files
random (dependent on Python's set() implementation). Minimal
configuration files (which can be saved from the menuconfig) were
affected too.

The intent is for the order to match .config files, which in turn match
symbol definition order. This doesn't affect behavior in any way, but
makes things more readable (and is handier if files are checked in).

Update Kconfiglib to upstream revision a3252f620fad9 to make the
ordering match again. Tests have been added upstream.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-08-22 15:29:18 -07:00
Oleg Zhurakivskyy
f3bc967e49 sanitycheck: Overcommit the default number of jobs
Overcommit the default number of jobs by factor 2
to keep the original behavior intact.

Signed-off-by: Oleg Zhurakivskyy <oleg.zhurakivskyy@intel.com>
2018-08-19 13:12:15 -07:00
Oleg Zhurakivskyy
99aacd9808 sanitycheck: Rename CPU_COUNTS to JOBS
This would correspond the help and avoid cpus/threads confusion.

Signed-off-by: Oleg Zhurakivskyy <oleg.zhurakivskyy@intel.com>
2018-08-19 13:12:15 -07:00
Oleg Zhurakivskyy
c97054c1c9 sanitycheck: Fix the logic for jobs
multiprocessing.cpu_count() already returns the number of threads
(not cores), no need to multiply it by 2.

Signed-off-by: Oleg Zhurakivskyy <oleg.zhurakivskyy@intel.com>
2018-08-19 13:12:15 -07:00
Oleg Zhurakivskyy
42822083c4 sanitycheck: Get ZEPHYR_BASE only once
Also clean up ZEPHYR_BASE discovery.

Signed-off-by: Oleg Zhurakivskyy <oleg.zhurakivskyy@intel.com>
2018-08-19 13:12:15 -07:00
Ulf Magnusson
4fe581ccc8 check-compliance: Fix undef. Kconfig symbol check for external projects
Threw an exception about Kconfig files not being found. They need to be
looked up relative to ZEPHYR_BASE.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-08-18 10:38:52 +02:00
Ulf Magnusson
a56be6f58d Kconfiglib: Fix $srctree behavior for the top-level Kconfig file
Due a design goof, the top-level Kconfig file passed to
Kconfig.__init__() wasn't looked up relative to $srctree. This broke a
planned fix for an issue with the CI check for references to undefined
Kconfig files in external projects.

Update Kconfiglib to upstream revision 953cc12464 to fix it.

Also update doc/CMakeLists.txt to pass 'Kconfig' instead of '../Kconfig'
to genrest.py. $srctree is already set to $ZEPHYR_BASE.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-08-18 10:38:52 +02:00
Flavio Ceolin
8aec087268 kernel: Fix bitwise operators with unsigned operators
Bitwise operators should be used only with unsigned integer operands
because the result os bitwise operations on signed integers are
implementation-defined.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2018-08-16 19:47:41 -07:00
Flavio Ceolin
0866d18d03 irq: Fix irq_lock api usage
irq_lock returns an unsigned int, though, several places was using
signed int. This commit fix this behaviour.

In order to avoid this error happens again, a coccinelle script was
added and can be used to check violations.

Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
2018-08-16 19:47:41 -07:00
Anas Nashif
b4bdd66924 sanitycheck: exit on exceptions
Exit on exceptions with crashing.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-08-16 13:22:10 -07:00
Nathaniel Graff
0b60f3839d dts: Fix get_compat when parent compat is a list
When a DTS node has no 'compatible' property and its parent node has a
list for its compatible property, return only the first element of the
parent's compatible list.

Signed-off-by: Nathaniel Graff <nathaniel.graff@sifive.com>
2018-08-16 06:22:39 -07:00
Adithya Baglody
f8a8bdf392 scripts: gen_app_smem.py: Removing a needless file.
No longer needed. Superseded by scripts/gen_app_partitions.py

Signed-off-by: Adithya Baglody <adithya.nagaraj.baglody@intel.com>
2018-08-14 05:22:54 -07:00
Adithya Baglody
c69fb0d016 userspace: app_shared_mem: Fixed incorrect implementation.
This feature was failing on a default ARM core MPU. The linker
script that was getting created was not able to align the required
partitions at prebuilt time.

The old implementation relied on the prebuilt to finish then
extract the size information which was then used to align the regions.
This fails because the size of the alignment and the fill in the
linker needs to be available at prebuilt time else it cant manage
the final elf file generation. We cant have 2 different sizes of
prebuilt and final elf file.

This implementation will get the alignment requirements met at
prebuilt time.

Signed-off-by: Adithya Baglody <adithya.nagaraj.baglody@intel.com>
2018-08-14 05:22:54 -07:00
Ulf Magnusson
f24d4bb369 check-compliance: Simplify undefined Kconfig symbol check
Use the new KCONFIG_STRICT Kconfiglib functionality, which makes the
library itself warn for references to undefined Kconfig symbols.

Also get rid of the chdir() hack. Kconfiglib now only looks up Kconfig
files relative to $srctree when it's set, so the hack is no longer
needed.

Remove list_undef_kconfig_refs.py. It's no longer needed.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-08-13 19:31:46 -07:00
Ulf Magnusson
6686efb838 kconfiglib/menuconfig: Add functionality and improve symbol information
Update Kconfiglib and menuconfig to upstream revision 6aea8d06b637e.

Kconfiglib changes:

 - Kconfig files are now looked up just relative to $srctree (if set).
   Previously, each source'd Kconfig file was also looked up in the
   current directory, and Kconfig files there could override Kconfig
   files in $srctree. This is what the C tools do.

   I'm pretty sure that behavior was a bug in the C tools all along, and
   only meant for .config files. It caused problems (and an ugly
   workaround) for the undefined Kconfig symbol CI check in an external
   project.

   The new behavior also saves a bunch of open()'s, though it's probably
   not noticeable.

 - Setting the KCONFIG_STRICT environment variable to 'y' now makes
   Kconfiglib itself warn for references to undefined symbols. This
   isn't safe in general, as some projects use multiple Kconfig trees
   with shared Kconfig files (e.g. the Linux kernel).

   This will be used to simplify the undefined Kconfig symbol CI check.

 - It's now possible to customize how symbols and choices are printed
   within expressions.

   This will be used to make the RST link generation in genrest.py less
   hacky.

 - Instead of having 'gsource', a plain 'source' is now globbing, and
   requires at least one match. There's also 'osource', for when it's
   okay for a glob pattern to match no files.

   'gsource' had the design flaw that there was no way to require at
   least one file to match. I plan on replacing all 'gsource' statements
   with plain 'source's later, but 'gsource' is still supported for
   backwards compatibility.

 - def_int, def_hex, and def_string are now available as a Kconfig
   extensions, analogous to def_bool (set type and add default).

 - Misc. internal cleanup.

menuconfig changes:

 - Boolean value hints are no longer shown to the right of defaults in
   the symbol information for int/hex symbols. Stuff like
   '- 74 (value: "n")' wasn't helpful, and looked confusing.

 - Symbol information has been made more compact in general, e.g. by
   skipping value hints where they aren't helpful
   ('FOO(=y)' instead of 'FOO(=y) (=y)'), and by shortening stuff like
   '(value: "y")' to just '(=y)'.

 - Misc. internal cleanup.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-08-13 19:31:46 -07:00
Sebastian Bøe
1a40990b2d syscalls: Define the syscall id's with '#define' instead of enum
We use the code generator 'gen_syscalls.py' to assign numeric
id's to each syscall. These id's have been defined using an enum
like this:

enum {
	K_SYSCALL_ADC_DISABLE,
	K_SYSCALL_ADC_ENABLE,
	K_SYSCALL_LIMIT
};

but enums can not be included by assembly files. So we have been
compiling the enum values and then extracting them into #define's when
needed.

In this situation there happen to not be any benefits of using
'enum' over #define's so we can simplify by initially defining
them with #define instead.

Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
2018-08-13 16:43:40 -07:00
Andrew Boie
09c22cc45d userspace: add net_context as a kernel object
Socket APIs pass pointers to these disguised as file descriptors.
This lets us effectively validate them.

Kernel objects now can have Kconfig dependencies specified, in case
certain structs are not available in all configurations.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2018-08-13 07:19:39 -07:00
Daniel Egger
f3a6e8aa89 sanitycheck: Recommend installing "ply" instead of "python3-ply"
Since a distributions name for the ply package may change, the only
constant name which can be used for a recommendation is the name on
PyPi which happens to be "ply".

Signed-off-by: Daniel Egger <daniel@eggers-club.de>
2018-08-12 11:39:36 +02:00
Ulf Magnusson
ec3eff57e0 Kconfig: Use the first default with a satisfied condition
Up until now, Zephyr has patched Kconfig to use the last 'default' with
a satisfied condition, instead of the first one. I'm not sure why the
patch was added (it predates Kconfiglib), but I suspect it's related to
Kconfig.defconfig files.

There are at least three problems with the patch:

  1. It's inconsistent with how Kconfig works in other projects, which
     might confuse newcomers.

  2. Due to oversights, earlier 'range' properties are still preferred,
     as well as earlier 'default' properties on choices.

     In addition to being inconsistent, this makes it impossible to
     override 'range' properties and choice 'default' properties if the
     base definition of the symbol/choice already has 'range'/'default'
     properties.

     I've seen errors caused by the inconsistency, and I suspect there
     are more.

  3. A fork of Kconfiglib that adds the patch needs to be maintained.

Get rid of the patch and go back to standard Kconfig behavior, as
follows:

  1. Include the Kconfig.defconfig files first instead of last in
     Kconfig.zephyr.

  2. Include boards/Kconfig and arch/<arch>/Kconfig first instead of
     last in arch/Kconfig.

  3. Include arch/<arch>/soc/*/Kconfig first instead of last in
     arch/<arch>/Kconfig.

  4. Swap a few other 'source's to preserve behavior for some scattered
     symbols with multiple definitions.

     Swap 'source's in some no-op cases too, where it might match the
     intent.

  5. Reverse the defaults on symbol definitions that have more than one
     default.

     Skip defaults that are mutually exclusive, e.g. where each default
     has an 'if <some board>' condition. They are already safe.

  6. Remove the prefer-later-defaults patch from Kconfiglib.

Testing was done with a Python script that lists all Kconfig
symbols/choices with multiple defaults, along with a whitelist of fixed
symbols. The script also verifies that there are no "unreachable"
defaults hidden by defaults without conditions

As an additional test, zephyr/.config was generated before and after the
change for several samples and checked to be identical (after sorting).

This commit includes some default-related cleanups as well:

  - Simplify some symbol definitions, e.g. where a default has 'if FOO'
    when the symbol already has 'depends on FOO'.

  - Remove some redundant 'default ""' for string symbols. This is the
    implicit default.

Piggyback fixes for swapped ranges on BT_L2CAP_RX_MTU and
BT_L2CAP_TX_MTU (caused by confusing inconsistency).

Piggyback some fixes for style nits too, e.g. unindented help texts.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-08-10 12:38:28 -07:00
Daniel Leung
e58b65427e kernel: threads: assign index no. to dynamically created threads
Kernel threads created at build time have unique indexes to map them
into various bitarrays. This patch extends these indexes to
dynamically created threads where the associated  kernel objects are
allocated at runtime.

Fixes: #9081

Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2018-08-09 09:20:14 -07:00
Erwan Gouriou
69a211fe00 scripts: extract_dts_includes: use deepcopcy when accessing reduced
Variables 'names' is copying value from 'reduced' dict and gives
illusion that any computation could be done on copied data.
Problem is that later computation on 'names' will modify data in
'reduced'.
Use deepcopy in order to avoid erasing thing in reduced.

Signed-off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
2018-08-06 09:44:21 -05:00
Erwan Gouriou
8e7403504a scripts/dts: extract_dts_includes: makes 'defs' and 'structs' global
'defs' variable is used and passed as a parameter in the vast
majority of the functions of extract_dts_includes script.
Set this variable global
Do the same with 'struct' variable for upcoming change.

Signed-off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
2018-08-06 09:44:21 -05:00
Ulf Magnusson
5402662dd9 cmake: kconfig: Fix rerunning cmake after Kconfig warnings
Commit b3d165f ("scripts: kconfig: Handle warnings generated
during evaluation") made it common for kconfig.py to fail after writing
zephyr/.config. This confuses the configuration fragment checksum logic
in cmake/kconfig.cmake, because it expects the saved checksum file to
exist if zephyr/.config exists.

The end result is a CMake error when rerunning the configuration after
non-whitelisted Kconfig warnings.

Fix it by only writing zephyr/.config (and zephyr/include/autoconf.h) in
kconfig.py if there are no warnings-turned-errors.

Also check if the saved checksum file exists in kconfig.cmake before
trying to open it. Normally this shouldn't happen though.

Move the writing of the checksum file to before writing zephyr/.config
as well. That way, zephyr/.config only gets written if the other
operations succeed.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-08-02 19:15:26 +02:00
Ulf Magnusson
b3d165f3c2 scripts: kconfig: Handle warnings generated during evaluation
Warnings generated during symbol evaluation were accidentally ignored,
due to checking for warnings before writing .config and autoconf.h
(which indirectly evaluates all symbols).

Move the warning checking code to after writing the configuration to
catch such warnings. kconfig.py still gets rerun if any
warnings-turned-errors show up.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2018-07-31 08:26:28 -04:00
Marti Bolivar
94be858f31 scripts: west: sync with upstream
This brings in content from commits in the upstream West
repository. Since both the version here and upstream have diverged,
this synchronizes them again.

Signed-off-by: Marti Bolivar <marti@foundries.io>
2018-07-30 17:56:21 -04:00
Anas Nashif
9930633454 kconfig: remove whitelisting for ARC_INIT
ARC_INIT was whitelisted in kconfig, removing now that the issue is
fixed.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-07-30 15:17:51 -04:00
Maureen Helm
d7d1090422 doc: scripts: Updates for pyocd v0.11.1
pyocd recently added support for python 3 so we can now remove the
python 2 package requirements. It also merged Zephyr thread awareness
upstream, so we can remove the reference to my pyocd pull request.

Tested debugging and flashing on Linux and Windows.

Signed-off-by: Maureen Helm <maureen.helm@nxp.com>
2018-07-27 00:54:55 +02:00
Andrew Boie
353acf4aae gen_syscalls.py: do not output data to stdout
There's no particularly good reason to have one kind of
output from this script to be sent to stdout instead of
a filename specified by parameter, and it makes it
annoying to add debug print() statements.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2018-07-26 00:53:45 -04:00
Shawn Mosley
573f32b6d2 userspace: compartmentalized app memory organization
Summary: revised attempt at addressing issue 6290.  The
following provides an alternative to using
CONFIG_APPLICATION_MEMORY by compartmentalizing data into
Memory Domains.  Dependent on MPU limitations, supports
compartmentalized Memory Domains for 1...N logical
applications.  This is considered an initial attempt at
designing flexible compartmentalized Memory Domains for
multiple logical applications and, with the provided python
script and edited CMakeLists.txt, provides support for power
of 2 aligned MPU architectures.

Overview: The current patch uses qualifiers to group data into
subsections.  The qualifier usage allows for dynamic subsection
creation and affords the developer a large amount of flexibility
in the grouping, naming, and size of the resulting partitions and
domains that are built on these subsections. By additional macro
calls, functions are created that help calculate the size,
address, and permissions for the subsections and enable the
developer to control application data in specified partitions and
memory domains.

Background: Initial attempts focused on creating a single
section in the linker script that then contained internally
grouped variables/data to allow MPU/MMU alignment and protection.
This did not provide additional functionality beyond
CONFIG_APPLICATION_MEMORY as we were unable to reliably group
data or determine their grouping via exported linker symbols.
Thus, the resulting decision was made to dynamically create
subsections using the current qualifier method. An attempt to
group the data by object file was tested, but found that this
broke applications such as ztest where two object files are
created: ztest and main.  This also creates an issue of grouping
the two object files together in the same memory domain while
also allowing for compartmenting other data among threads.

Because it is not possible to know a) the name of the partition
and thus the symbol in the linker, b) the size of all the data
in the subsection, nor c) the overall number of partitions
created by the developer, it was not feasible to align the
subsections at compile time without using dynamically generated
linker script for MPU architectures requiring power of 2
alignment.

In order to provide support for MPU architectures that require a
power of 2 alignment, a python script is run at build prior to
when linker_priv_stacks.cmd is generated.  This script scans the
built object files for all possible partitions and the names given
to them. It then generates a linker file (app_smem.ld) that is
included in the main linker.ld file.  This app_smem.ld allows the
compiler and linker to then create each subsection and align to
the next power of 2.

Usage:
 - Requires: app_memory/app_memdomain.h .
 - _app_dmem(id) marks a variable to be placed into a data
section for memory partition id.
 - _app_bmem(id) marks a variable to be placed into a bss
section for memory partition id.
 - These are seen in the linker.map as "data_smem_id" and
"data_smem_idb".
 - To create a k_mem_partition, call the macro
app_mem_partition(part0) where "part0" is the name then used to
refer to that partition. This macro only creates a function and
necessary data structures for the later "initialization".
 - To create a memory domain for the partition, the macro
app_mem_domain(dom0) is called where "dom0" is the name then
used for the memory domain.
 - To initialize the partition (effectively adding the partition
to a linked list), init_part_part0() is called. This is followed
by init_app_memory(), which walks all partitions in the linked
list and calculates the sizes for each partition.
 - Once the partition is initialized, the domain can be
initialized with init_domain_dom0(part0) which initializes the
domain with partition part0.
 - After the domain has been initialized, the current thread
can be added using add_thread_dom0(k_current_get()).
 - The code used in ztests ans kernel/init has been added under
a conditional #ifdef to isolate the code from other tests.
The userspace test CMakeLists.txt file has commands to insert
the CONFIG_APP_SHARED_MEM definition into the required build
targets.
  Example:
        /* create partition at top of file outside functions */
        app_mem_partition(part0);
        /* create domain */
        app_mem_domain(dom0);
        _app_dmem(dom0) int var1;
        _app_bmem(dom0) static volatile int var2;

        int main()
        {
                init_part_part0();
                init_app_memory();
                init_domain_dom0(part0);
                add_thread_dom0(k_current_get());
                ...
        }

 - If multiple partitions are being created, a variadic
preprocessor macro can be used as provided in
app_macro_support.h:

        FOR_EACH(app_mem_partition, part0, part1, part2);

or, for multiple domains, similarly:

        FOR_EACH(app_mem_domain, dom0, dom1);

Similarly, the init_part_* can also be used in the macro:

        FOR_EACH(init_part, part0, part1, part2);

Testing:
 - This has been successfully tested on qemu_x86 and the
ARM frdm_k64f board.  It compiles and builds power of 2
aligned subsections for the linker script on the 96b_carbon
boards.  These power of 2 alignments have been checked by
hand and are viewable in the zephyr.map file that is
produced during build. However, due to a shortage of
available MPU regions on the 96b_carbon board, we are unable
to test this.
 - When run on the 96b_carbon board, the test suite will
enter execution, but each individaul test will fail due to
an MPU FAULT.  This is expected as the required number of
MPU regions exceeds the number allowed due to the static
allocation. As the MPU driver does not detect this issue,
the fault occurs because the data being accessed has been
placed outside the active MPU region.
 - This now compiles successfully for the ARC boards
em_starterkit_em7d and em_starterkit_em7d_v22. However,
as we lack ARC hardware to run this build on, we are unable
to test this build.

Current known issues:
1) While the script and edited CMakeLists.txt creates the
ability to align to the next power of 2, this does not
address the shortage of available MPU regions on certain
devices (e.g. 96b_carbon).  In testing the APB and PPB
regions were commented out.
2) checkpatch.pl lists several issues regarding the
following:
a) Complex macros. The FOR_EACH macros as defined in
app_macro_support.h are listed as complex macros needing
parentheses.  Adding parentheses breaks their
functionality, and we have otherwise been unable to
resolve the reported error.
b) __aligned() preferred. The _app_dmem_pad() and
_app_bmem_pad() macros give warnings that __aligned()
is preferred. Prior iterations had this implementation,
which resulted in errors due to "complex macros".
c) Trailing semicolon. The macro init_part(name) has
a trailing semicolon as the semicolon is needed for the
inlined macro call that is generated when this macro
expands.

Update: updated to alternative CONFIG_APPLCATION_MEMORY.
Added config option CONFIG_APP_SHARED_MEM to enable a new section
app_smem to contain the shared memory component.  This commit
seperates the Kconfig definition from the definition used for the
conditional code.  The change is in response to changes in the
way the build system treats definitions.  The python script used
to generate a linker script for app_smem was also midified to
simplify the alignment directives.  A default linker script
app_smem.ld was added to remove the conditional includes dependency
on CONFIG_APP_SHARED_MEM.  By addining the default linker script
the prebuild stages link properly prior to the python script running

Signed-off-by: Joshua Domagalski <jedomag@tycho.nsa.gov>
Signed-off-by: Shawn Mosley <smmosle@tycho.nsa.gov>
2018-07-25 12:02:01 -07:00
Andrew Boie
a698e84a76 userspace: adjust syscall generation scripts
parse_syscalls.py was doing too much and was generating
derived and partial string output information that was
completed later by gen_syscalls.py.

Now parse_syscalls.py just breaks up system call information into
non-derived data which is fully processed by gen_syscalls.py.

The goal is to ease maintenance of system call generation with
all the mechanism on what to do with system call information in
one script location rather than two of them.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2018-07-24 14:39:38 -04:00
Anas Nashif
f3d48e1cce sanitycheck: allow blacklisting boards
Add new kwyboard to board definition to allow blacklisting boards. This
is needed when a board is broken causing CI to fail without a fix in
sight.

Add:

sanitycheck: false

to the board yaml file to disable the board. By default, the value is
set to true.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-07-24 09:58:18 -04:00
Anas Nashif
d2b3c75403 Revert "sanitycheck: set state correctly in case of a crash"
This reverts commit d74a56bd63.

Still not soving the issue and introducing additional problems.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-07-23 12:58:31 -04:00
Anas Nashif
d74a56bd63 sanitycheck: set state correctly in case of a crash
sanitycheck not printing QEMU console in some cases where a crash
happens and when state is not set correctly.

Fixes #9061

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-07-22 22:14:10 -04:00
Anas Nashif
e24350c775 sanitycheck: do not run if we do not have nsimdrv
Check if we have nSim simulator in path, if not, just test build without
attempting to run.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2018-07-19 18:59:07 -04:00