Commit graph

5615 commits

Author SHA1 Message Date
Wayne Ren
7cbe5c3e18 scripts: add unit test for mdb runner
add unit test for mdb runner

Signed-off-by: Wayne Ren <wei.ren@synopsys.com>
2020-06-11 10:02:11 -04:00
Wayne Ren
1506db5098 scripts: add the runner script for metaware debugger
* add the runner script for metaware debugger(mdb).
* mdb is required for SMP case
* mdb also can provides a GUI interface

Signed-off-by: Wayne Ren <wei.ren@synopsys.com>
2020-06-11 10:02:11 -04:00
Peter Bigot
3c36b42d03 sanitycheck: add notes key to schema
Prior to addition of schema validation running sanitycheck
--generate-hw-map would preserve the value of any field in an existing
map that it didn't update.  These fields now cause a parsing error.
Add notes as an optional key where information relevant to the board
can be added to provide useful context (such as which serial port
corresponds to the console, or why a non-standard runner is required).

Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
2020-06-10 12:03:17 +02:00
Peter Bigot
3cdd5b4be4 sanitycheck: add missing probe_id key to schema
probe_id is used as an optional key that overrides id as the source
of board_id when configured to use a runner that is not the default.

Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
2020-06-10 12:03:17 +02:00
Kumar Gala
491bbad8e8 ci: Update for buildkite environment
* Tweak west_setup:
  - log `west update` to a file to reduce noise in log
  - use `west forall` + `git reset` to make sure files are checked out
    (this is to handle a possible module cache)
* Output when we start sanity_check.  Add a banner for when we run
  sanity_check so it's a bit easier to find in console logs

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-06-09 15:28:48 -04:00
Ruslan Mstoi
7d8adddd2e scripts: process_gperf: typo fix
Fix double "e" in "repreesentation"

Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
2020-06-09 10:40:58 +02:00
Ruslan Mstoi
84822e3086 scripts: gen_syscalls: typo fix
Fix "defing" as "defining"

Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
2020-06-09 10:39:49 +02:00
Spoorthy Priya Yerabolu
d434dfc22c sanitylib: Adding small change to enable create_overlay() testing
Returning the content from create_overlay() in TestInstance class

Signed-off-by: Spoorthy Priya Yerabolu <spoorthy.priya.yerabolu@intel.com>
2020-06-08 16:38:48 -04:00
Spoorthy Priya Yerabolu
9ac30b595c scripts: tests: sanitycheck: Add testcases for TestInstance Class
test_testinstance.py: Add testcases for check_build_or_run()
create_overlay() and calculate_sizes()
conftest.py: Module for common pytest fixtures
test_data/board_config: Adding test data for board configurations

Signed-off-by: Spoorthy Priya Yerabolu <spoorthy.priya.yerabolu@intel.com>
2020-06-08 16:38:48 -04:00
Kumar Gala
93da8dc478 Revert "checkpatch: update checkpatch to warn about C99 type usage"
Now that we are standardizing C99 integer types we can revert the commit
that warned about C99 type usage.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-06-08 08:23:57 -05:00
Kumar Gala
a1b77fd589 zephyr: replace zephyr integer types with C99 types
git grep -l 'u\(8\|16\|32\|64\)_t' | \
		xargs sed -i "s/u\(8\|16\|32\|64\)_t/uint\1_t/g"
	git grep -l 's\(8\|16\|32\|64\)_t' | \
		xargs sed -i "s/s\(8\|16\|32\|64\)_t/int\1_t/g"

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-06-08 08:23:57 -05:00
Andrew Boie
be919d3bf7 userspace: improve dynamic object allocation
We now have a low-level function z_dynamic_object_create()
which is not a system call and is used for installing
kernel objects that are not supported by k_object_alloc().

Checking for valid object type enumeration values moved
completely to the implementation function.

A few debug messages and comments were improved.

Futexes and sys_mutexes are now properly excluded from
dynamic generation.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-06-03 22:33:32 +02:00
Andrew Boie
299ec8f1b5 userspace: net sockets are kernel objects
Any data structure declaration tagged with __net_socket will end up
in the kernel object table with type K_OBJ_NET_SOCKET. These all
correspond to objects which are associated with socket file
descriptors and can handle the socket vtable API.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-06-03 22:33:32 +02:00
Andrew Boie
5960119f16 scripts: parse_syscalls: generalize struct tags
Now we can build up lists of data structures matching a list
of particular tags, with __subsystem being just one case.

Relax searches to also look inside C files, since struct
prototypes may be declared there as well.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-06-03 22:33:32 +02:00
Andrew Boie
455e178b3b scripts: gen_kobject_list: generalize obj alloc
Instead of handling this ad hoc, generalize which kobject
types can be allocated.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-06-03 22:33:32 +02:00
Andrew Boie
378024c510 userspace: add z_is_in_user_syscall()
Certain types of system call validation may need to be pushed
deeper in the implementation and not performed in the verification
function. If such checks are only pertinent when the caller was
from user mode, we need an API to detect this situation.

This is implemented by having thread->syscall_frame be non-NULL
only while a user system call is in progress. The template for the
system call marshalling functions is changed to clear this value
on exit.

A test is added to prove that this works.

Signed-off-by: Andrew Boie <andrew.p.boie@intel.com>
2020-06-03 22:33:32 +02:00
Martí Bolívar
e4a761cffe devicetree: add migration guide documentation and tests
Add test cases that verify various bits and pieces of the legacy
devicetree macros match the new APIs.

Writing these test cases without giving rise to deprecated macro
warnings which might break people's CI if they build with -Werror
requires turning off the __WARN() generation in
devicetree_legacy_unfixed.h. The entire file is deprecated at this
point and must be explicitly enabled with an opt-in Kconfig option, so
there isn't any harm in doing this.

Nevertheless, take a minimally invasive approach to avoiding __WARN()
generation in gen_legacy_defines.py, to avoid the possibility of
breakage. This code is basically frozen anyway, so hacks like this
won't cause maintainability problems since it isn't being actively
maintained.

Use the new tests as fodder for a migration guide from the old API in
the documentation.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-05-28 22:12:38 +02:00
Martí Bolívar
9d36b4f6e4 doc: some devicetree fixes and updates
Some updates to the reference page for the "core" APIs, and associated
follow-ups in the guides:

- centralize documentation of chosen zephyr nodes in a non-legacy
  file, provide a reference to them from the intro page in the guide
- review doxygen docstrings and correct errors for generic APIs
- add introductory text to each section in the API reference
- add missing hardware-specific pages

Documentation for layers built on top of these is mostly left to future
commits, but I do have a smattering of fixes in the guides that I
noticed while I was doing this.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-05-26 18:12:00 +02:00
Wentong Wu
6c9d4a5c57 scripts: use debug function instead of debug_die to dump objs
In step 4 of find_kobjects, use func debug instead of debug_die
to dump debug info to avoid dump wrong info.

Signed-off-by: Wentong Wu <wentong.wu@intel.com>
2020-05-25 18:07:20 +02:00
Wentong Wu
0bf5113c6b Revert "scripts: add DW_AT_abstract_origin check"
This reverts commit 2593a919ee.

This commit wasn't the root cause of the found bug in dwarf analyze.

Signed-off-by: Wentong Wu <wentong.wu@intel.com>
2020-05-25 18:07:20 +02:00
Maciej Perkowski
b2fa99caf8 sanitycheck: Add filtered test cases to the JUnit full report
If a test instance from .yaml file was skipped due to being on
a filtered list then there are no entries about it in the
sanitycheck_report.xml for a given platform. This commit fills
instance.results with skipped test cases also for skipped instances so
those test cases are visible when the full report is generated. Tested
for ./tests/kernel/mem_protect/userspace on nrf52dk_nrf52832 where
userspace.gap_filling is filtered out.

Signed-off-by: Maciej Perkowski <Maciej.Perkowski@nordicsemi.no>
2020-05-25 15:52:17 +02:00
Anas Nashif
3b86f13fab sanitycheck: add fixture support to hardware map
It is now possible to add a list of fixture a platform supports which is
matched to testcases requesting fixtures to be avaiable to be able to
run.

For example:

- available: true
  connected: true
  id: 0240000026334e450015400f5e0e000b4eb1000097969900
  platform: frdm_k64f
  product: DAPLink CMSIS-DAP
  runner: pyocd
  serial: /dev/ttyACM9
  fixtures:
    - gpio_loopback

Fixes #24943

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-05-24 20:25:51 +02:00
Anas Nashif
cefb9a170f sanitycheck: support fixtures
Add fixtures to hardware map schema.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-05-24 20:25:51 +02:00
Anas Nashif
ce8c12eb5a sanitycheck: cleanup fixture processing
Cleanup fixture processing and allow ztest testcases to support
harness_config with fixture definition.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-05-24 20:25:51 +02:00
Anas Nashif
77946fa835 sanitycheck: remove wrong message about --qemu-testing
--qemu-testing does not exist.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-05-21 22:47:36 -04:00
Ruslan Mstoi
56b65e43de scripts: gen_offset_header: add argument help text
Help text is set to documentation string of the script.

Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
2020-05-21 14:45:24 +02:00
Ruslan Mstoi
4695eb4e1d scripts: net: Fix incorrect error messages
If environment variables ZEPHYR_BASE or NET_TOOLS_BASE are unset, an
incorrect message is printed saying that "it is set, but it is not a
directory":

$ZEPHYR_BASE is unset
$ZEPHYR_BASE is set, but it is not a directory
$NET_TOOLS_BASE is unset, no net-tools found
$NET_TOOLS_BASE set, but it is not a directory

This patch fixes that issue.

Signed-off-by: Ruslan Mstoi <ruslan.mstoi@intel.com>
2020-05-21 14:45:07 +02:00
Martí Bolívar
414ed86269 edtlib: fix default type for interrupts property
The name of the interrupts property is typo-ed in the python script.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-05-20 10:41:07 +02:00
Aastha Grover
6f0a0926e3 scripts: tests: sanitycheck: Add testcases for schema validation.
test_sanitycheck.py: Add testcases for correct & incorrect
testcase/platform yaml schemas.

Signed-off-by: Aastha Grover <aastha.grover@intel.com>
2020-05-20 10:35:35 +02:00
Aastha Grover
5948ab6cf1 sanitycheck: add testcases for add_testcases function of testsuite class
test_testsuite_class.py: Add testcases for add_testcases function
of testsuite class in sanitycheck.
test_data/testcases/tests & /samples : Testcase root directory
to add all the testcases & to test add_testcases function.
conftest.py: Module for common pytest fixtures, also used for
passing data from one testcase to another.
Note: conftest.py has a class_testsuite fixture where board_root is
defined as the directory which will be added in a separate PR.

Signed-off-by: Aastha Grover <aastha.grover@intel.com>
2020-05-20 10:35:35 +02:00
Wentong Wu
2593a919ee scripts: add DW_AT_abstract_origin check
According to below rule which's from DWARF5 sepc, if the
attribute can't be founded in given DIE, check more entry
associated by DW_AT_abstract_origin.

For the purposes of determining whether a debugging information
entry has a particular attribute (such as DW_AT_name), if
debugging information entry A has a DW_AT_specification or
DW_AT_abstract_origin attribute pointing to another debugging
information entry B, any attributes of B are considered to be
part of A.

Signed-off-by: Wentong Wu <wentong.wu@intel.com>
2020-05-20 10:33:14 +02:00
Aastha Grover
a0ae53410e sanitylib: Remove hardcoded sample.yaml & testcase.yaml in TestSuite class
This fix creates class variables SAMPLE_FILENAME & TESTCASE_FILENAME
in Class TestSuite and remove the hardcoded sample.yaml & testcase.yaml
from add_testcases function. This makes testing for sanitylib script
easier so that shippable do not detect the test_data for sanitylib
as actual zephyr testcases.

Signed-off-by: Aastha Grover <aastha.grover@intel.com>
2020-05-14 22:48:09 +02:00
Wentong Wu
0d619aefd8 sanitycheck: Add cpu time for QEMUHandler
Add cpu time for QEMUHandler because the guest virtual time
in QEMU icount mode isn't host time and it's maintained by
counting guest instructions, we use QEMU process CPU time to
mostly simulate the time of guest OS.

Signed-off-by: Wentong Wu <wentong.wu@intel.com>
2020-05-14 13:52:07 +02:00
Wentong Wu
7ec57b4666 scripts: sanitylib.py: enhancement for QEMUHandler
Sometimes QEMU can't handle SIGTERM signal correctly and just
run as normal, in that case kill -9 QEMU process directly and
leave sanitycheck judge the testing result by console output.

For failures caused by other reasons, terminate all releated
processes to avoid CI waits 2 hours and reports timeout.

Signed-off-by: Wentong Wu <wentong.wu@intel.com>
2020-05-14 13:52:07 +02:00
Kumar Gala
bd97378870 devicetree: Add support for fixed-partitions
Add DT_NODE_BY_FIXED_PARTITION_LABEL that given a "label" in any
fixed-partitions map will return the node_id for that partition node.

Add DT_NODE_HAS_FIXED_PARTITION_LABEL that will test if a given
fixed-partitions "label" is valid.

Add DT_FIXED_PARTITION_ID that will return an unique ordinal value for
the partition give a node_id to the partition.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-05-13 21:22:53 +02:00
Kumar Gala
4a5a90aef8 gen_defines: Rework implementation of write_child_functions
Rework how write_child_functions to match how we do the code for
DT_FOREACH_OKAY_INST.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-05-13 21:22:53 +02:00
Kumar Gala
3a68566d40 edtlib: add a flag for old fixed-partitions behavior
allow the old generator to act as it did.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-05-13 21:22:53 +02:00
Kumar Gala
058842b353 edtlib: handle match of fixed-partition on any bus
If we have a fixed-partition on a flash device that is for example on
a spi controller we will not get a binding match currently.  This is
because we expect a match between both the compatible and the fact that
fixed-partition node is a decendant of the spi bus.

To address this we treat fixed-partitions as if they are on no bus.
This has the effect of causing a binding match as well as ensuring that
when we process the fixed-partition node we will do anything special to
it because of the bus it happens to be under (for example SPI CS_GPIO
processing).

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-05-13 21:22:53 +02:00
Kumar Gala
06edcb11db edtlib: Initialize node.compats earlier
Setup node.compats right after we create the Node.  This allows access
to the compats information in _bus_node.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-05-13 21:22:53 +02:00
Peter Bigot
49e8c7080f scripts/coccinelle: add script to preserve const qualifier on config_info
Drivers cast the device config_info pointer to a driver-specific
structure.  The referenced object is const-qualified; make sure the
cast doesn't inadvertently remove that qualifier.

Signed-off-by: Peter Bigot <peter.bigot@nordicsemi.no>
2020-05-13 18:21:52 +02:00
Anas Nashif
44f7ba0781 sanitycheck: detect wrongly named tests and fail
Tests should always start with test_. If not, they will not be in the
report.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-05-13 18:16:07 +02:00
Anas Nashif
4258d8d34b sanitycheck: give summary about built vs executed
It is important to see that some tests are only building and not
executing...

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-05-11 17:00:26 +02:00
Anas Nashif
91fd68dbbf sanitycheck: also look for cpp files when scanning
look for *.c* instead of just *.c files when scanning or testcases.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-05-11 17:00:26 +02:00
Anas Nashif
bb28035a82 sanitycheck: support exporting instances
Enable exporting tests per platform.

sanitycheck --export-tests out.txt  -p qemu_x86 -T tests/

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-05-11 17:00:26 +02:00
Anas Nashif
1a5defa924 sanitycheck: support --force-platform
This option allows running tests on platforms that would be excluded
based on the test definition.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-05-11 17:00:26 +02:00
Anas Nashif
60ae507783 sanitycheck: move export-tests handling and deduplicate code
Reuse the same code used for other options.

Signed-off-by: Anas Nashif <anas.nashif@intel.com>
2020-05-11 17:00:26 +02:00
Martí Bolívar
0b5c58fcab scripts: runners: handle uninstalled dependencies
Catch ImportError whenever a non-standard module import fails from any
runners that do one. Complain at runtime about it if the user actually
needs the runner.

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-05-09 13:05:48 +02:00
Christopher Friedt
2c0eecaa5e posix arch: build on aarch64 / allow host-specific cmake includes
This change enables specific compiler and linker options to be used in
the case that an arch/posix/os.arch.cmake file exists.

Note: os and arch in the above case are evaluations of
CMAKE_HOST_SYSTEM_NAME and CMAKE_HOST_SYSTEM_PROCESSOR.

Otherwise, the existing "generic" compiler and linker flags in
arch/posix/CMakeLists.txt are used.

Additional flags and checks are provided in
arch/posix/Linux.aarch64.cmake.

Added scripts/user_wordsize.py to detect if userspace is 64-bit or
32-bit, which should be consistent with the value of CONFIG_64BIT
for Aarch64 on Linux.

Fixes #24842

Signed-off-by: Christopher Friedt <chrisfriedt@gmail.com>
2020-05-09 12:17:24 +02:00
Kumar Gala
6a2cb94f6f sanitycheck: suppress warnings from EDT
We don't need sanitycheck's use of EDT to report warnings, we'll get
them from the build system.  So the warnings are just duplication and
noise, thus lets always suppress them.

Signed-off-by: Kumar Gala <kumar.gala@linaro.org>
2020-05-08 21:54:02 -04:00
Martí Bolívar
7e0eed9235 devicetree: allow access to all nodes
Usually, we want to operate only on "available" device
nodes ("available" means "status is okay and a matching binding is
found"), but that's not true in all cases.

Sometimes we want to operate on special nodes without matching
bindings, such as those describing memory.

To handle the distinction, change various additional devicetree APIs
making it clear that they operate only on available device nodes,
adjusting gen_defines and devicetree.h implementation details
accordingly:

- emit macros for all existing nodes in gen_defines.py, regardless
  of status or matching binding
- rename DT_NUM_INST to DT_NUM_INST_STATUS_OKAY
- rename DT_NODE_HAS_COMPAT to DT_NODE_HAS_COMPAT_STATUS_OKAY
- rename DT_INST_FOREACH to DT_INST_FOREACH_STATUS_OKAY
- rename DT_ANY_INST_ON_BUS to DT_ANY_INST_ON_BUS_STATUS_OKAY
- rewrite DT_HAS_NODE_STATUS_OKAY in terms of a new DT_NODE_HAS_STATUS
- resurrect DT_HAS_NODE in the form of DT_NODE_EXISTS
- remove DT_COMPAT_ON_BUS as a public API
- use the new default_prop_types edtlib parameter

Signed-off-by: Martí Bolívar <marti.bolivar@nordicsemi.no>
2020-05-08 19:37:18 -05:00