Do not close console after PASS is reported, wait a bit for any
remaining messages from the tests, sometimes we have faults that need to
be parsed.
This now works for Qemu handler, support for other handlers to follow.
Fixes#9646
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
Update menuconfig (and Kconfiglib, just to sync) to upstream revision
78682a8e3c4fe to get the following fix in, which could cause certain
visible symbols to not show up in the menuconfig interface:
menuconfig: Always show implicit submenus with visible nodes
Currently, the symbol BAR below (which ends up indented in an
implicit submenu) is shown only if DEP is non-n (or if show-all mode
is enabled):
config FOO
bool "foo" if DEP
default y
config BAR
bool "bar" if FOO
This is bad, because it hides visible symbols from the interface.
The assumption was that an implicit submenu (which is only created
if the parent has a prompt) would never have visible items when the
parent item is invisible, but prompt-specific conditions and
select/imply can break that assumption.
Fix it by always showing implicit submenus with visible nodes, along
with the parent node. If the parent node is invisible, show it in
red, like in show-all mode (which happens automatically). That's
probably better than having mysteriously indented nodes when the
parent is invisible.
(Some other tweaks were made later to avoid showing red outside show-all
mode, because it might look confusing/broken.)
A new color scheme for the menuconfig has been added as well,
contributed by Mitja Horvat (pinkfluid). It can be enabled by setting
the environment variable MENUCONFIG_THEME to 'aquatic'.
The default theme has been tweaked to use bold text for the selected
item too.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
This adds new targets to generate build documentation through
LaTEX to PDF.
There are a few notes:
1. pdflatex complains about the tex file generated by doxygen
so it needs to be fixed with a Python script before feeding
in through pdflatex.
2. SVG files are not recognized by pdflatex so they are converted
to known good format on the fly, only for producing PDF. This
uses the libRSVG's rsvg-convert tool.
Relates to #6782.
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
In addtion to zephyr/dts/bindings als scan the dts/bindings
directory within the application source directory for bindings.
Allows to have application specific bindings and drivers.
Signed-off-by: Bobby Noelte <b0661n0e17e@gmail.com>
Update Kconfiglib and menuconfig to upstream revision a28bc4da9762e,
which adds include path information to menuconfig, showing how the
Kconfig file of the symbol got 'source'd in addition to showing the
definition location.
Add include path information to the auto-generated Kconfig documentation
too.
Some small Kconfiglib bugs are fixed to, like error reporting for
recursive 'source's being broken (crashing instead of printing the
error), a minor file descriptor leak, and syntax checking not catching
extra trailing tokens after 'if <expr>' and 'depends on <expr>'.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Commit f6bf897780 ("kconfiglib: Add preprocessor and two warnings")
accidentally made the ordering of #define's in generated header files
random (dependent on Python's set() implementation). Minimal
configuration files (which can be saved from the menuconfig) were
affected too.
The intent is for the order to match .config files, which in turn match
symbol definition order. This doesn't affect behavior in any way, but
makes things more readable (and is handier if files are checked in).
Update Kconfiglib to upstream revision a3252f620fad9 to make the
ordering match again. Tests have been added upstream.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
multiprocessing.cpu_count() already returns the number of threads
(not cores), no need to multiply it by 2.
Signed-off-by: Oleg Zhurakivskyy <oleg.zhurakivskyy@intel.com>
Threw an exception about Kconfig files not being found. They need to be
looked up relative to ZEPHYR_BASE.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Due a design goof, the top-level Kconfig file passed to
Kconfig.__init__() wasn't looked up relative to $srctree. This broke a
planned fix for an issue with the CI check for references to undefined
Kconfig files in external projects.
Update Kconfiglib to upstream revision 953cc12464 to fix it.
Also update doc/CMakeLists.txt to pass 'Kconfig' instead of '../Kconfig'
to genrest.py. $srctree is already set to $ZEPHYR_BASE.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Bitwise operators should be used only with unsigned integer operands
because the result os bitwise operations on signed integers are
implementation-defined.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
irq_lock returns an unsigned int, though, several places was using
signed int. This commit fix this behaviour.
In order to avoid this error happens again, a coccinelle script was
added and can be used to check violations.
Signed-off-by: Flavio Ceolin <flavio.ceolin@intel.com>
When a DTS node has no 'compatible' property and its parent node has a
list for its compatible property, return only the first element of the
parent's compatible list.
Signed-off-by: Nathaniel Graff <nathaniel.graff@sifive.com>
This feature was failing on a default ARM core MPU. The linker
script that was getting created was not able to align the required
partitions at prebuilt time.
The old implementation relied on the prebuilt to finish then
extract the size information which was then used to align the regions.
This fails because the size of the alignment and the fill in the
linker needs to be available at prebuilt time else it cant manage
the final elf file generation. We cant have 2 different sizes of
prebuilt and final elf file.
This implementation will get the alignment requirements met at
prebuilt time.
Signed-off-by: Adithya Baglody <adithya.nagaraj.baglody@intel.com>
Use the new KCONFIG_STRICT Kconfiglib functionality, which makes the
library itself warn for references to undefined Kconfig symbols.
Also get rid of the chdir() hack. Kconfiglib now only looks up Kconfig
files relative to $srctree when it's set, so the hack is no longer
needed.
Remove list_undef_kconfig_refs.py. It's no longer needed.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Update Kconfiglib and menuconfig to upstream revision 6aea8d06b637e.
Kconfiglib changes:
- Kconfig files are now looked up just relative to $srctree (if set).
Previously, each source'd Kconfig file was also looked up in the
current directory, and Kconfig files there could override Kconfig
files in $srctree. This is what the C tools do.
I'm pretty sure that behavior was a bug in the C tools all along, and
only meant for .config files. It caused problems (and an ugly
workaround) for the undefined Kconfig symbol CI check in an external
project.
The new behavior also saves a bunch of open()'s, though it's probably
not noticeable.
- Setting the KCONFIG_STRICT environment variable to 'y' now makes
Kconfiglib itself warn for references to undefined symbols. This
isn't safe in general, as some projects use multiple Kconfig trees
with shared Kconfig files (e.g. the Linux kernel).
This will be used to simplify the undefined Kconfig symbol CI check.
- It's now possible to customize how symbols and choices are printed
within expressions.
This will be used to make the RST link generation in genrest.py less
hacky.
- Instead of having 'gsource', a plain 'source' is now globbing, and
requires at least one match. There's also 'osource', for when it's
okay for a glob pattern to match no files.
'gsource' had the design flaw that there was no way to require at
least one file to match. I plan on replacing all 'gsource' statements
with plain 'source's later, but 'gsource' is still supported for
backwards compatibility.
- def_int, def_hex, and def_string are now available as a Kconfig
extensions, analogous to def_bool (set type and add default).
- Misc. internal cleanup.
menuconfig changes:
- Boolean value hints are no longer shown to the right of defaults in
the symbol information for int/hex symbols. Stuff like
'- 74 (value: "n")' wasn't helpful, and looked confusing.
- Symbol information has been made more compact in general, e.g. by
skipping value hints where they aren't helpful
('FOO(=y)' instead of 'FOO(=y) (=y)'), and by shortening stuff like
'(value: "y")' to just '(=y)'.
- Misc. internal cleanup.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
We use the code generator 'gen_syscalls.py' to assign numeric
id's to each syscall. These id's have been defined using an enum
like this:
enum {
K_SYSCALL_ADC_DISABLE,
K_SYSCALL_ADC_ENABLE,
K_SYSCALL_LIMIT
};
but enums can not be included by assembly files. So we have been
compiling the enum values and then extracting them into #define's when
needed.
In this situation there happen to not be any benefits of using
'enum' over #define's so we can simplify by initially defining
them with #define instead.
Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
Socket APIs pass pointers to these disguised as file descriptors.
This lets us effectively validate them.
Kernel objects now can have Kconfig dependencies specified, in case
certain structs are not available in all configurations.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Since a distributions name for the ply package may change, the only
constant name which can be used for a recommendation is the name on
PyPi which happens to be "ply".
Signed-off-by: Daniel Egger <daniel@eggers-club.de>
Up until now, Zephyr has patched Kconfig to use the last 'default' with
a satisfied condition, instead of the first one. I'm not sure why the
patch was added (it predates Kconfiglib), but I suspect it's related to
Kconfig.defconfig files.
There are at least three problems with the patch:
1. It's inconsistent with how Kconfig works in other projects, which
might confuse newcomers.
2. Due to oversights, earlier 'range' properties are still preferred,
as well as earlier 'default' properties on choices.
In addition to being inconsistent, this makes it impossible to
override 'range' properties and choice 'default' properties if the
base definition of the symbol/choice already has 'range'/'default'
properties.
I've seen errors caused by the inconsistency, and I suspect there
are more.
3. A fork of Kconfiglib that adds the patch needs to be maintained.
Get rid of the patch and go back to standard Kconfig behavior, as
follows:
1. Include the Kconfig.defconfig files first instead of last in
Kconfig.zephyr.
2. Include boards/Kconfig and arch/<arch>/Kconfig first instead of
last in arch/Kconfig.
3. Include arch/<arch>/soc/*/Kconfig first instead of last in
arch/<arch>/Kconfig.
4. Swap a few other 'source's to preserve behavior for some scattered
symbols with multiple definitions.
Swap 'source's in some no-op cases too, where it might match the
intent.
5. Reverse the defaults on symbol definitions that have more than one
default.
Skip defaults that are mutually exclusive, e.g. where each default
has an 'if <some board>' condition. They are already safe.
6. Remove the prefer-later-defaults patch from Kconfiglib.
Testing was done with a Python script that lists all Kconfig
symbols/choices with multiple defaults, along with a whitelist of fixed
symbols. The script also verifies that there are no "unreachable"
defaults hidden by defaults without conditions
As an additional test, zephyr/.config was generated before and after the
change for several samples and checked to be identical (after sorting).
This commit includes some default-related cleanups as well:
- Simplify some symbol definitions, e.g. where a default has 'if FOO'
when the symbol already has 'depends on FOO'.
- Remove some redundant 'default ""' for string symbols. This is the
implicit default.
Piggyback fixes for swapped ranges on BT_L2CAP_RX_MTU and
BT_L2CAP_TX_MTU (caused by confusing inconsistency).
Piggyback some fixes for style nits too, e.g. unindented help texts.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Kernel threads created at build time have unique indexes to map them
into various bitarrays. This patch extends these indexes to
dynamically created threads where the associated kernel objects are
allocated at runtime.
Fixes: #9081
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
Variables 'names' is copying value from 'reduced' dict and gives
illusion that any computation could be done on copied data.
Problem is that later computation on 'names' will modify data in
'reduced'.
Use deepcopy in order to avoid erasing thing in reduced.
Signed-off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
'defs' variable is used and passed as a parameter in the vast
majority of the functions of extract_dts_includes script.
Set this variable global
Do the same with 'struct' variable for upcoming change.
Signed-off-by: Erwan Gouriou <erwan.gouriou@linaro.org>
Commit b3d165f ("scripts: kconfig: Handle warnings generated
during evaluation") made it common for kconfig.py to fail after writing
zephyr/.config. This confuses the configuration fragment checksum logic
in cmake/kconfig.cmake, because it expects the saved checksum file to
exist if zephyr/.config exists.
The end result is a CMake error when rerunning the configuration after
non-whitelisted Kconfig warnings.
Fix it by only writing zephyr/.config (and zephyr/include/autoconf.h) in
kconfig.py if there are no warnings-turned-errors.
Also check if the saved checksum file exists in kconfig.cmake before
trying to open it. Normally this shouldn't happen though.
Move the writing of the checksum file to before writing zephyr/.config
as well. That way, zephyr/.config only gets written if the other
operations succeed.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
Warnings generated during symbol evaluation were accidentally ignored,
due to checking for warnings before writing .config and autoconf.h
(which indirectly evaluates all symbols).
Move the warning checking code to after writing the configuration to
catch such warnings. kconfig.py still gets rerun if any
warnings-turned-errors show up.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
This brings in content from commits in the upstream West
repository. Since both the version here and upstream have diverged,
this synchronizes them again.
Signed-off-by: Marti Bolivar <marti@foundries.io>
pyocd recently added support for python 3 so we can now remove the
python 2 package requirements. It also merged Zephyr thread awareness
upstream, so we can remove the reference to my pyocd pull request.
Tested debugging and flashing on Linux and Windows.
Signed-off-by: Maureen Helm <maureen.helm@nxp.com>
There's no particularly good reason to have one kind of
output from this script to be sent to stdout instead of
a filename specified by parameter, and it makes it
annoying to add debug print() statements.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Summary: revised attempt at addressing issue 6290. The
following provides an alternative to using
CONFIG_APPLICATION_MEMORY by compartmentalizing data into
Memory Domains. Dependent on MPU limitations, supports
compartmentalized Memory Domains for 1...N logical
applications. This is considered an initial attempt at
designing flexible compartmentalized Memory Domains for
multiple logical applications and, with the provided python
script and edited CMakeLists.txt, provides support for power
of 2 aligned MPU architectures.
Overview: The current patch uses qualifiers to group data into
subsections. The qualifier usage allows for dynamic subsection
creation and affords the developer a large amount of flexibility
in the grouping, naming, and size of the resulting partitions and
domains that are built on these subsections. By additional macro
calls, functions are created that help calculate the size,
address, and permissions for the subsections and enable the
developer to control application data in specified partitions and
memory domains.
Background: Initial attempts focused on creating a single
section in the linker script that then contained internally
grouped variables/data to allow MPU/MMU alignment and protection.
This did not provide additional functionality beyond
CONFIG_APPLICATION_MEMORY as we were unable to reliably group
data or determine their grouping via exported linker symbols.
Thus, the resulting decision was made to dynamically create
subsections using the current qualifier method. An attempt to
group the data by object file was tested, but found that this
broke applications such as ztest where two object files are
created: ztest and main. This also creates an issue of grouping
the two object files together in the same memory domain while
also allowing for compartmenting other data among threads.
Because it is not possible to know a) the name of the partition
and thus the symbol in the linker, b) the size of all the data
in the subsection, nor c) the overall number of partitions
created by the developer, it was not feasible to align the
subsections at compile time without using dynamically generated
linker script for MPU architectures requiring power of 2
alignment.
In order to provide support for MPU architectures that require a
power of 2 alignment, a python script is run at build prior to
when linker_priv_stacks.cmd is generated. This script scans the
built object files for all possible partitions and the names given
to them. It then generates a linker file (app_smem.ld) that is
included in the main linker.ld file. This app_smem.ld allows the
compiler and linker to then create each subsection and align to
the next power of 2.
Usage:
- Requires: app_memory/app_memdomain.h .
- _app_dmem(id) marks a variable to be placed into a data
section for memory partition id.
- _app_bmem(id) marks a variable to be placed into a bss
section for memory partition id.
- These are seen in the linker.map as "data_smem_id" and
"data_smem_idb".
- To create a k_mem_partition, call the macro
app_mem_partition(part0) where "part0" is the name then used to
refer to that partition. This macro only creates a function and
necessary data structures for the later "initialization".
- To create a memory domain for the partition, the macro
app_mem_domain(dom0) is called where "dom0" is the name then
used for the memory domain.
- To initialize the partition (effectively adding the partition
to a linked list), init_part_part0() is called. This is followed
by init_app_memory(), which walks all partitions in the linked
list and calculates the sizes for each partition.
- Once the partition is initialized, the domain can be
initialized with init_domain_dom0(part0) which initializes the
domain with partition part0.
- After the domain has been initialized, the current thread
can be added using add_thread_dom0(k_current_get()).
- The code used in ztests ans kernel/init has been added under
a conditional #ifdef to isolate the code from other tests.
The userspace test CMakeLists.txt file has commands to insert
the CONFIG_APP_SHARED_MEM definition into the required build
targets.
Example:
/* create partition at top of file outside functions */
app_mem_partition(part0);
/* create domain */
app_mem_domain(dom0);
_app_dmem(dom0) int var1;
_app_bmem(dom0) static volatile int var2;
int main()
{
init_part_part0();
init_app_memory();
init_domain_dom0(part0);
add_thread_dom0(k_current_get());
...
}
- If multiple partitions are being created, a variadic
preprocessor macro can be used as provided in
app_macro_support.h:
FOR_EACH(app_mem_partition, part0, part1, part2);
or, for multiple domains, similarly:
FOR_EACH(app_mem_domain, dom0, dom1);
Similarly, the init_part_* can also be used in the macro:
FOR_EACH(init_part, part0, part1, part2);
Testing:
- This has been successfully tested on qemu_x86 and the
ARM frdm_k64f board. It compiles and builds power of 2
aligned subsections for the linker script on the 96b_carbon
boards. These power of 2 alignments have been checked by
hand and are viewable in the zephyr.map file that is
produced during build. However, due to a shortage of
available MPU regions on the 96b_carbon board, we are unable
to test this.
- When run on the 96b_carbon board, the test suite will
enter execution, but each individaul test will fail due to
an MPU FAULT. This is expected as the required number of
MPU regions exceeds the number allowed due to the static
allocation. As the MPU driver does not detect this issue,
the fault occurs because the data being accessed has been
placed outside the active MPU region.
- This now compiles successfully for the ARC boards
em_starterkit_em7d and em_starterkit_em7d_v22. However,
as we lack ARC hardware to run this build on, we are unable
to test this build.
Current known issues:
1) While the script and edited CMakeLists.txt creates the
ability to align to the next power of 2, this does not
address the shortage of available MPU regions on certain
devices (e.g. 96b_carbon). In testing the APB and PPB
regions were commented out.
2) checkpatch.pl lists several issues regarding the
following:
a) Complex macros. The FOR_EACH macros as defined in
app_macro_support.h are listed as complex macros needing
parentheses. Adding parentheses breaks their
functionality, and we have otherwise been unable to
resolve the reported error.
b) __aligned() preferred. The _app_dmem_pad() and
_app_bmem_pad() macros give warnings that __aligned()
is preferred. Prior iterations had this implementation,
which resulted in errors due to "complex macros".
c) Trailing semicolon. The macro init_part(name) has
a trailing semicolon as the semicolon is needed for the
inlined macro call that is generated when this macro
expands.
Update: updated to alternative CONFIG_APPLCATION_MEMORY.
Added config option CONFIG_APP_SHARED_MEM to enable a new section
app_smem to contain the shared memory component. This commit
seperates the Kconfig definition from the definition used for the
conditional code. The change is in response to changes in the
way the build system treats definitions. The python script used
to generate a linker script for app_smem was also midified to
simplify the alignment directives. A default linker script
app_smem.ld was added to remove the conditional includes dependency
on CONFIG_APP_SHARED_MEM. By addining the default linker script
the prebuild stages link properly prior to the python script running
Signed-off-by: Joshua Domagalski <jedomag@tycho.nsa.gov>
Signed-off-by: Shawn Mosley <smmosle@tycho.nsa.gov>
parse_syscalls.py was doing too much and was generating
derived and partial string output information that was
completed later by gen_syscalls.py.
Now parse_syscalls.py just breaks up system call information into
non-derived data which is fully processed by gen_syscalls.py.
The goal is to ease maintenance of system call generation with
all the mechanism on what to do with system call information in
one script location rather than two of them.
Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
Add new kwyboard to board definition to allow blacklisting boards. This
is needed when a board is broken causing CI to fail without a fix in
sight.
Add:
sanitycheck: false
to the board yaml file to disable the board. By default, the value is
set to true.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
sanitycheck not printing QEMU console in some cases where a crash
happens and when state is not set correctly.
Fixes#9061
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
The two handlers were doing pretty much the same with minor differences,
unify them into one single handler BinaryHandler that will be able to
handle additional targets.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
refactor all add_goal routines in to one single function that can handle
multiple platforms. Move everything to one single add_goal function.
Signed-off-by: Anas Nashif <anas.nashif@intel.com>
For native builds it does not make much sense to calculate
the size of the suposed RAM or ROM, or to check that new
unexpected sections did not appear.
So let's save the time and not do it.
Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
$srctree was changed to an absolute path when the documentation building
was switched over to CMake, which uncovered a bug in Kconfiglib that
caused symbols in gsource'd files to show up with absolute paths in the
auto-generated Kconfig documentation.
This commit adds upstream commit ac692af07a123 ("Fix absolute $srctree
prefixes showing up on gsource'd files"), which fixes it.
Upstream commit message:
When using gsource with $srctree set to an absolute path, the $srctree
prefix would show up in MenuNode.filename, trickling its way into e.g.
generated documentation.
This was due to a broken test: os.path.isabs() was checked after
joining the pattern with $srctree, making it mistake an absolute
$srctree for an absolute path in the Kconfig file.
Fix the test.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
In order to get rid of doc/scripts/filter-doc-log.sh to make building
the documentation cross-platform, include the functionality in there in
the already existing filter-known-issues.py.
Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
Update Kconfiglib to upstream revision 547fced630611 to get a new
Kconfig preprocessor in, documented in
https://github.com/torvalds/linux/blob/master/Documentation/kbuild/kconfig-macro-language.txt.
The preprocessor allows shell functions to be run directly from Kconfig.
Things like 'default "prefix-$(shell,some-cmd)' and
'depends on (success,some-cmd)' are supported.
The preprocessor might come in handy for Kconfig/DTS integration. I'm
thinking of extending it so that Python functions can be called as well.
There's also two new warnings:
- Trying to use an int/hex symbol like a bool symbol in an expression
(where it will always evaluate to n)
- Having a 'default' on an int/hex symbol that lies outside an active
'range'.
The parser is more strict now as well (due to dropping some hacks for 3+
year old kernel versions).
A related fix for scripts/kconfig.py is also included:
The comment above the whitelist lies. I accidentally changed the warning
text for the select-with-unsatisfied-dependencies warning while
generalizing it (for m-valued dependencies, which you'd never get in
Zephyr).
Update the whitelist to detect the new warning text.
Last-minute piggyback:
Include a change that improved the parse time for U-Boot from 4 seconds
to about 0.6 seconds, related to symbols defined in multiple locations.
It might be helpful for Zephyr as well, as it also uses a lot of symbols
defined in multiple locations.
Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
