michaelh/zephyr
1
0
Fork 0
Code Releases Activity

twister: Fix missing testcases with error status

Browse source 
If timeout error was occured during performing testsuite on QEMU or
other simulator (e.g. Renode), information about this error will be not
placed in final report. It can be fixed, by set status of testcases,
which were not performed due to this error as "BLOCK".

Simmilar solution is alredy used in DeviceHandler when Twister works
with real platform. So I think that it is resonable to use the same
approach, when tests are performed on simulators.

I move loop which iterate through all testcases and set their tests
status as BLOCK into the parent Handler class, because the same
activity is performed in each children class (BinaryHandler,
DeviceHandler and QEMUHandler).

Fixes #38756

Signed-off-by: Piotr Golyzniak <piotr.golyzniak@nordicsemi.no>
This commit is contained in:
Piotr Golyzniak 2021-09-23 00:14:04 +02:00 • committed by Anas Nashif
commit 0e1ecb373a
1 changed files with 13 additions and 3 deletions
Download patch file Download diff file Expand all files Collapse all files

16
scripts/pylib/twister/twisterlib.py
View file

@ -436,6 +436,16 @@ class Handler:
proc.kill()
self.terminated = True
def add_missing_testscases(self, harness):
"""
If testsuite was broken by some error (e.g. timeout) it is necessary to
add information about next testcases, which were not be
performed due to this error.
"""
for c in self.instance.testcase.cases:
if c not in harness.tests:
harness.tests[c] = "BLOCK"
class BinaryHandler(Handler):
def __init__(self, instance, type_str):
@ -590,6 +600,7 @@ class BinaryHandler(Handler):
else:
self.set_state("timeout", handler_time)
self.instance.reason = "Timeout"
self.add_missing_testscases(harness)
self.record(harness)
@ -859,9 +870,7 @@ class DeviceHandler(Handler):
handler_time = time.time() - start_time
if out_state in ["timeout", "flash_error"]:
for c in self.instance.testcase.cases:
if c not in harness.tests:
harness.tests[c] = "BLOCK"
self.add_missing_testscases(harness)
if out_state == "timeout":
self.instance.reason = "Timeout"
@ -1152,6 +1161,7 @@ class QEMUHandler(Handler):
self.instance.reason = "Timeout"
else:
self.instance.reason = "Exited with {}".format(self.returncode)
self.add_missing_testscases(harness)
def get_fifo(self):
return self.fifo_fn