Commit graph

5778 commits

Author SHA1 Message Date
Peter A. Bigot
8b4d529706 scripts: gen_cfb_font_header: correct last character
DELETE is not a printable character.

Signed-off-by: Peter A. Bigot <pab@pabigot.com>
2019-07-11 20:20:59 -07:00
Kumar Gala
31052837ed scripts/dts: cleanup 'generation' usage
* Remove dead code that referenced 'generation' but didn't do anything
  with it
* Replace looking at 'generation' with a simple check for property
  starting with # (for things like #address-cells, etc)  or ending in
  -map (for things like gpio-map) to skip

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-07-11 06:51:17 -04:00
Peter A. Bigot
79765b8ad7 scripts/dts: fix support for uint8-array property values
uint8-array is the name for what the devicetree specification calls a
bytestring.

The original parsing code treated square brackets as equivalent to
quotes or angle brackets, failing to interpret elements as hex-encoded.
This was a bug, corrected in this patch by processing content as a
sequence of hex-encoded bytes with arbitrary whitespace.

The original generating code emitted the property as individual
elements.  Replace that with generation of a single structure
initializer, which is more useful in cases where the length of a
property value may vary between nodes.

Signed-off-by: Peter A. Bigot <pab@pabigot.com>
2019-07-11 06:50:27 -04:00
Manivannan Sadhasivam
7ac767b880 scripts: dts: Match alpha numeric property values
For using alpha numeric property values in a devicetree node, we
need to match the values starts with a number. Current scenario will
return the value as a numeric literal if it starts with a number. This
will not work for a compatible like, "96b-ls-con" which is proposed in
issue #15598.

Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
2019-07-11 06:15:22 -04:00
Michael Scott
4ca54390f4 sanitycheck: handle west-flash dependency for device-testing
sanitycheck --help mentions that west-flash requires device-testing
to be enabled (and indeed it does because DeviceHandler will never
be called where west_flash option is used.  Let's generate an error
if west-flash is used w/o specifying device-testing.

Also cleanup help text which looks odd in both sanitycheck --help
and in the file itself.

Signed-off-by: Michael Scott <mike@foundries.io>
2019-07-10 08:13:57 -07:00
Michael Scott
421ce46f4b sanitycheck: add west-runner parameter
Several boards have multiple runners setup.  We need a way to specify
which runner to use with sanitycheck.  Introduce --west-runner option.

Signed-off-by: Michael Scott <mike@foundries.io>
2019-07-10 08:13:57 -07:00
Kumar Gala
8083f8a6b0 scripts/dts: deprecate DT_<COMPAT>_<INSTANCE ID>_BUS_<BUS>
We don't have any uses of this form of define so deprecate it for now.
If needed this should be DT_INST_<INSTANCE ID>_<COMPAT>_BUS_<BUS>.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-07-10 06:24:39 -05:00
Kumar Gala
81072b53a6 dts/scripts: mark all non DT_ prefix deprecated
Now that we've converted LED and SW to use DT_ prefix we can mark the
non-DT_ prefixed versions as deprecated.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-07-10 05:23:24 -05:00
Kumar Gala
5c78b93c6b scripts/dts: Mark 'non-S' forms of cell and controllers as deprecated
Now that we've converted all _GPIO_ to _GPIOS_ we can mark the _GPIO_
form as deprecated (same for _PWM_ / _PWMS_).

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-07-10 05:23:24 -05:00
Kumar Gala
e5d838081c dts: Fix handling of fixed-clock
A number of minor issues with the 'fixed-clock' support:
* Fix the #clock-cells to be 0
* Fix nxp_ke1xf.dtsi to set #clock-cells 0 and the clock reference to
  only be a phandle.
* Fix the generation script to only generate what it should for a
  'fixed-clock'

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-07-10 05:23:07 -05:00
Marc Herbert
c7633de038 sanitycheck: fix priority of --extra-args=CONFIG_ over testcase.yaml
sanitycheck takes any "extra_config" list found in the testcase.yaml
file and generates an "overlay" file from it. This file is placed in
the per-test build directory and passed to cmake/kconfig.cmake through a
-DOVERLAY_CONF= option set in the (also) generated sanity-out/Makefile.

This commit moves this generated config overlay to a subdirectory one
level down from the build directory, otherwise kconfig.cmake picks it
up *twice*: once from the -DOVERLAY_CONF= option already mentioned above
and a second time because kconfig.cmake scans the build directory and
blindly picks up ALL files ending with .conf[*]. The second pickup is
problematic because kconfig.cmake currently gives it the top precedence,
even higher than anything the user espressed with --extra-args=CONFIG_*

Here's a quick and simple demonstration of the issue fixed by this
commit:

  cd $ZEPHYR_BASE/samples/net/sockets/net_mgmt/
  sanitycheck -T. -p qemu_x86 -b -v # --extra-args=CONFIG_USERSPACE=y|n
  grep CONFIG_USERSPACE $(find sanity-out/ -name .config)

  .net_mgmt.kernelmode/zephyr/.config: # CONFIG_USERSPACE is not set
    .net_mgmt.usermode/zephyr/.config: CONFIG_USERSPACE=y

  grep 'Merged configuration' $(find sanity-out/ -name build.log)

Without this commit, attemps to override anything with
--extra-args=CONFIG_ are silently dropped on the floor.

For more background this issue was found while using the recipe in
commit message 4afcc0f8af

[*] picking up all .conf files is debatable but a much bigger debate
with backward compatibility implications. This small fix makes
absolutely zero difference to anyone or anything not using sanitycheck.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-07-09 19:16:30 -07:00
Andrew Boie
49cf4867c7 sanitycheck: select gcov tool based on target
Use PATH gcov if no SDK in use, or native-posix derived
targets are in use.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-07-09 10:20:31 +02:00
Wentong Wu
859ca42006 scripts: elf_helper: correct kernel object address check range
kernel object should not located in app_smem area.

Fixes: #17338.

Signed-off-by: Wentong Wu <wentong.wu@intel.com>
2019-07-08 13:28:13 -07:00
Marc Herbert
e5cedcae43 sanitycheck help: how to spot tests that were not run and just built
There are a few reasons why sanitycheck will only build a test and not
run it: list them in the developer guide. Also lists the --options that
provide that information and update their --help message.

A couple other --help fixes.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-07-08 08:07:57 -04:00
Marc Herbert
d7ca903b72 sanitycheck: replace cmake -H "not meant for public use" with -S
https://www.mail-archive.com/cmake-developers@cmake.org/msg16769.html

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-07-08 08:04:39 -04:00
Sebastian Bøe
b8d4a77ecd size_report: Don't crash when observing stripped symbols
When stripped symbols are present in the build size_report has crashed
due to it not being aware of this case.

This patch causes stripped symbols to be ignored instead, allowing
analysis of the non-stripped symbols.

It is left as future work to somehow display how much space stripped
symbols are taking up.

Stripped symbols can be present when third-party binaries are linked
into the build.

Signed-off-by: Sebastian Bøe <sebastian.boe@nordicsemi.no>
2019-07-05 14:18:49 -04:00
Marc Herbert
9e57338efd sanitycheck: fix --jobs help message + other cosmetic fixes
The --jobs default was recently changed in commit 9f4f57eed3, update
its help message.

Add the hopefully last missing verbose("Spawning...") statement.

Fix comment updated in commit 095b82a301.

Replace two tags with whitespace.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-07-03 23:04:16 -07:00
Andrew Boie
1578ef71b1 sanitycheck: workaround file limits
sanitycheck is opening an insane number of file descriptors
simultaneously as it opens up communication pipes with
every test that supports emulation, on every emulated
board target.

Increase the resource limit on open files until this code
can be properly refactored.

Workaround for: #17239

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-07-03 16:49:21 -04:00
Andrew Boie
8047a6f4d0 sanitycheck: make coverage reports easier to do
- The --gcov-tool argument now has a reasonable default
  if the Zephyr SDK is in use.
- --coverage-platform, if unspecified, defaults to what
  was passed to --platform
- --coverage implies --enable-slow, so that tests with
  the 'slow' tag are built and run.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-07-02 22:56:14 -04:00
Andrew Boie
095b82a301 sanitycheck: correct timeout extension comment
This wasn't discussing why there is extra time for when
code coverage is enabled.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-07-02 22:54:24 -04:00
Andrew Boie
9f4f57eed3 sanitycheck: don't multiply CPU count
We have a number of timing sensitive tests which run
correctly on a much more frequent basis if the system
is not so heavily loaded. Instead of squeezing a few
more crumbs of performance by doubling the CPU count,
just use the number of CPUs reported by the system.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-07-02 22:51:24 -04:00
Andrew Boie
d62e229680 sanitycheck: extend coverage timeout
If GCOV coverage is enabled, the coverage dump happens after
"PROJECT EXECUTION SUCCESSFUL" is printed. In some cases,
the additional time added was not enough to capture all the
GCOV output on a heavily loaded system before the emulator
gets killed.

Ideally, the decision to kill the emulator needs to be smarter
and less race-prone, but that can wait for a future
enhancement.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2019-07-02 20:30:41 -04:00
Alberto Escolar Piedras
b91f37494b sanitycheck: Minor optimization in BinaryHandler
Do not search for the valgrind binary (which) if valgrind
was not selected to be used.

Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
2019-07-02 18:49:44 -04:00
Alberto Escolar Piedras
649368c018 sanitycheck: Record time used in BinaryHandler
So the time used to run boards which use the BinaryHandler can be
reported, record the time used from spawning the process until
it finnishes or is killed.
The BinaryHandler is used by the "native" boards, unit tests,
nsim and Renode.

Signed-off-by: Alberto Escolar Piedras <alpi@oticon.com>
2019-07-02 18:49:44 -04:00
Andrzej Głąbek
bb8dddfa99 scripts: dts: Add extraction of clock frequency for fixed clock
This patch adds generation of `*_FIXED_CLOCK_FREQUENCY` macros for
clock consuming nodes that are provided with a fixed rate clock.

Signed-off-by: Andrzej Głąbek <andrzej.glabek@nordicsemi.no>
2019-06-28 09:45:35 -05:00
Andrzej Głąbek
c782ba82ae scripts: dts: Add special handling of '+' in 'str_to_label' function
This patch is needed in order to get compilable macros for compatibles
like "arm,cortex-m0+".

Signed-off-by: Andrzej Głąbek <andrzej.glabek@nordicsemi.no>
2019-06-28 07:32:01 -05:00
Marc Herbert
eefea9da9c gen_app_partitions: deterministic _and_ decreasing app_smem_*.ld
Commit 212ec9a29a / feature #14121 already ordered partitions by
decreasing size, however it was common in samples/userspace/shared_mem/
/sample.kernel.memory_protection.shared_mem for two partitions to have
the same size and be randomly ordered between them. This adds the
partition name as a second sort key.

Unlike previous attempt in commit 725abdf430 this doesn't use the
partition name as the first (and only) key and doesn't break the
decreasing size order.  Huge thanks to Sigvart Hovland for spotting this
in a post-merge but prompt code review.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-06-27 18:06:30 -07:00
Marc Herbert
85bc0d2fe5 Revert "gen_app_partitions.py: make generated/app_smem_*.ld files deterministic"
This reverts commit 725abdf430 which did get rid of randomness in the
order of the partition _names_ as claimed but regressed commit
212ec9a29a / feature #14121 and broke the previous size order which I
missed. Huge thanks to Sigvart Hovland for spotting this in a post-merge
but prompt code review. Proper fix in the next commit.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-06-27 18:06:30 -07:00
Marc Herbert
725abdf430 gen_app_partitions.py: make generated/app_smem_*.ld files deterministic
Dictionaries are not ordered in Python 3.5 and before, so building twice
in a row could lead to a different partition order, different
build/zephyr/include/generated/app_smem_*.ld files and different
binaries.

Fix with a minor change to the "for" loop in the output function:
make it iterate on sorted(partitions.items()) instead of the raw and
randomly ordered partitions dictionary.

It is easy to reproduce the issue even without downgrading to an
obsolete Python version; pick a test like samples/userspace/shared_mem/
and simply change the code to this:

--- a/scripts/gen_app_partitions.py
+++ b/scripts/gen_app_partitions.py
@@ -159,10 +159,12 @@ def parse_elf_file(partitions):
                     partitions[partition_name][SZ] += size

+import random
 def generate_final_linker(linker_file, partitions):
     string = linker_start_seq
     size_string = ''
-    for partition, item in sorted(partitions.items()):
+    for partition, item in sorted(partitions.items(),
+                                  key=lambda x: random.random()):
         string += data_template.format(partition)
         if LIB in item:
             for lib in item[LIB]:

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-06-27 07:18:09 -04:00
Marc Herbert
6ccd026f7c gen_relocate_app.py: make generated/linker_relocate.ld deterministic
Dictionaries are not ordered in Python 3.5 and before, so building twice
${ZEPHYR_BASE}/samples/application_development/code_relocation/
in a row could lead to a different sections order, different
build/zephyr/include/generated/linker_relocate.d and code_relocation.c
and different binaries.

Fix with a minor change to three "for" loops in the output functions:
make them iterate on sorted(list of sections) instead of the raw and
randomly ordered dictionaries of sections.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
2019-06-27 07:18:09 -04:00
Carles Cufi
41f1f648f6 west: runners: Guess build folder
When using a build folder format with build.dir-fmt that includes any
parameters that need resolving, the west runners cannot find the folder
since the required information (board, source dir or app) is not
available.
Add a very simple heuristic to support the case where a build folder
starts with a hardcoded prefix (for example 'build/') and a single build
is present under that prefix.
The heuristic is gated behind a new configuration option:
build.guess-dir

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-06-26 11:50:26 -04:00
Carles Cufi
3b3bbc1ee0 Bluetooth: hci_uart: Use DT for the hci_uart UART
Use Device Tree,and in particular a new 'bt-c2h-uart' to select which
UART is being used to communicate with an external BLE Host when acting
as a Controller.

Signed-off-by: Carles Cufi <carles.cufi@nordicsemi.no>
2019-06-26 12:37:04 +02:00
Marti Bolivar
814bc7ba8e scripts: remove runner related cruft
There's some old cache variables that don't matter anymore in here.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-06-26 01:25:54 +02:00
Marti Bolivar
f08935fa73 scripts: west_commands: improve run_common error handling
Print a friendlier error message on ValueError, but don't throw away
the stack trace.

Move another call to log.die().

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-06-26 01:25:54 +02:00
Marti Bolivar
e6873b8d7e scripts: tweak west build output
Follow along with changes made in west flash/debug/etc to make it
easier to see the output steps visually.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-06-26 01:25:54 +02:00
Marti Bolivar
ec8dbf34dc scripts: tweak runner-related command output
Having common log handlers now lets us improve our logging output so
that info messages are prefixed with the runner they came from, and
doing something similar with the high level steps as we go, like this:

   -- west <command>: using runners
   -- runners.RUNNER_NAME: doing something
   <output from RUNNER_NAME subprocesses go here>
   -- runners.RUNNER_NAME: all done, bye

We can also colorize the west output to make it stand out better from
subprocesses, using the same output formatting style that west
commands like west list do.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-06-26 01:25:54 +02:00
Marti Bolivar
ddce583ca2 scripts: west_commands: decouple runners pkg from west
I've had some requests to be able to use code in the runners package
without having west installed.

It turns out to be pretty easy to make this happen, as west is
currently only used for west.log and some trivial helper methods:

- To replace west log, use the standard logging module
- Add an appropriate handler for each runner's logger in
  run_common.py which delegates to west.log, to keep
  output working as expected.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-06-26 01:25:54 +02:00
Marti Bolivar
d176cc36ad scripts: runners: update ZephyrBinaryRunner docstring
Fix some things that are incorrect and try to make it more clear.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-06-26 01:25:54 +02:00
Marti Bolivar
6440514bec scripts: west_commands: fix help string
The option name is wrong.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-06-26 01:25:54 +02:00
Marti Bolivar
725e98e300 scripts: fix pyocd runner --flash-opt help
This doesn't work. Fix the example.

Signed-off-by: Marti Bolivar <marti.bolivar@nordicsemi.no>
2019-06-25 22:33:02 +02:00
Kumar Gala
95f28eb764 scripts/dts: Remove DT_CHOSEN generation
We never used DT_CHOSEN so lets remove this and add it back if/when
there's an actual need for it.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-06-25 08:43:58 -05:00
Kumar Gala
08a5f9f8eb scripts/dts: Add ability to mark cell/ctrl defines as deprecated
Add support so that we can flag any "defines" associated with a call to
either extract_cells or extract_controller as deprecated.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-06-25 08:43:35 -05:00
Kumar Gala
79b59e8205 scripts/dts: Remove 'use-prop-name' from clock generation
We never set 'use-prop-name' on clock bindings so lets just always
use CLOCK_CONTROLLER as the define name we generate.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-06-25 08:42:31 -05:00
Kumar Gala
593d628e32 dts/extract: Remove 'use-prop-name' from cell and controller handling
We have 'use-prop-name' flag in the bindings which is specifically used
for GPIO properties to control if we get "GPIO" or "GPIOS" as the
generated define name.

Lets remove the inconsistancy and use "GPIOS" as the preferred name as
this matches the DTS property name.  Towards that we will generate both
forms and remove support for 'use-prop-name'.

This also impacts "PWM" generation.  So we'll have "PWM" and "PWMS"

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-06-25 08:40:33 -05:00
Jukka Rissanen
c9aaab74db net: ptp: clock: Add usermode support to ptp_clock_get()
It is useful that the ptp_clock_get() function can be called from
the userspace. Create also unit test for calling that function
from userspace.

Signed-off-by: Jukka Rissanen <jukka.rissanen@linux.intel.com>
2019-06-25 15:22:51 +03:00
Wentong Wu
5611e92347 kernel: add futex support
A k_futex is a lightweight mutual exclusion primitive designed
to minimize kernel involvement. Uncontended operation relies
only on atomic access to shared memory. k_futex structure lives
in application memory. And when using futexes, the majority of
the synchronization operations are performed in user mode. A
user-mode thread employs the futex wait system call only when
it is likely that the program has to block for a longer time
until the condition becomes true. When the condition comes true,
futex wake operation will be used to wake up one or more threads
waiting on that futex.

This patch implements two futex operations: k_futex_wait and
k_futex_wake. For k_futex_wait, the comparison with the expected
value, and starting to sleep are performed atomically to prevent
lost wake-ups. If different context changed futex's value after
the calling use-mode thread decided to block himself based on
the old value, the comparison will help observing the value
change and will not start to sleep. And for k_futex_wake, it
will wake at most num_waiters of the waiters that are sleeping
on that futex. But no guarantees are made on which threads are
woken, that means scheduling priority is not taken into
consideration.

Fixes: #14493.

Signed-off-by: Wentong Wu <wentong.wu@intel.com>
2019-06-24 15:38:21 -07:00
Ulf Magnusson
a570b40252 genrest: De-spam docs by skipping direct deps. in more places
Similar deal to commit cc14c40a2d ("kconfiglib: Unclutter symbol
strings, avoid redundant writes, misc.").

Hide the direct dependencies in the defaults, selects, and implies
sections. Do the same in menuconfig/guiconfig as well.

This uses a new Kconfiglib API, so update Kconfiglib to upstream
revision 164ef007a8. This also includes some minor optimizations and
cleanups.

Signed-off-by: Ulf Magnusson <Ulf.Magnusson@nordicsemi.no>
2019-06-24 12:14:06 -07:00
Ioannis Glaropoulos
88959e75dd scripts: gen_priv_stacks.py: use the right alignment for priv stacks
The privilege stacks are not sandboxed inside an MPU region,
so they do not have to be aligned with the stack buffer size.
We fix this by using the PRIVILEGE_STACK_ALIGN macro, which
is defined in arch.h and reflects the minimum alignment
requirement for privilege stack buffers.

Signed-off-by: Ioannis Glaropoulos <Ioannis.Glaropoulos@nordicsemi.no>
2019-06-24 10:16:57 -07:00
Kumar Gala
74818bcb49 scripts/dts: Treat array always as a list
If the type of property is a 'array' we should generate defines as
if its a list even if theres only a single element in the list.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-06-24 07:16:59 -04:00
Kumar Gala
344737d759 scripts/dts: Treat string-list always as a list
If the type of property is a 'string-list' we should generate defines as
if its a list even if theres only a single element in the list.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2019-06-22 10:03:17 -05:00