twister: Fix missing testcases with error status
If timeout error was occured during performing testsuite on QEMU or other simulator (e.g. Renode), information about this error will be not placed in final report. It can be fixed, by set status of testcases, which were not performed due to this error as "BLOCK". Simmilar solution is alredy used in DeviceHandler when Twister works with real platform. So I think that it is resonable to use the same approach, when tests are performed on simulators. I move loop which iterate through all testcases and set their tests status as BLOCK into the parent Handler class, because the same activity is performed in each children class (BinaryHandler, DeviceHandler and QEMUHandler). Fixes #38756 Signed-off-by: Piotr Golyzniak <piotr.golyzniak@nordicsemi.no>
This commit is contained in:
parent
cf6bb4d962
commit
0e1ecb373a
1 changed files with 13 additions and 3 deletions
|
@ -436,6 +436,16 @@ class Handler:
|
|||
proc.kill()
|
||||
self.terminated = True
|
||||
|
||||
def add_missing_testscases(self, harness):
|
||||
"""
|
||||
If testsuite was broken by some error (e.g. timeout) it is necessary to
|
||||
add information about next testcases, which were not be
|
||||
performed due to this error.
|
||||
"""
|
||||
for c in self.instance.testcase.cases:
|
||||
if c not in harness.tests:
|
||||
harness.tests[c] = "BLOCK"
|
||||
|
||||
|
||||
class BinaryHandler(Handler):
|
||||
def __init__(self, instance, type_str):
|
||||
|
@ -590,6 +600,7 @@ class BinaryHandler(Handler):
|
|||
else:
|
||||
self.set_state("timeout", handler_time)
|
||||
self.instance.reason = "Timeout"
|
||||
self.add_missing_testscases(harness)
|
||||
|
||||
self.record(harness)
|
||||
|
||||
|
@ -859,9 +870,7 @@ class DeviceHandler(Handler):
|
|||
handler_time = time.time() - start_time
|
||||
|
||||
if out_state in ["timeout", "flash_error"]:
|
||||
for c in self.instance.testcase.cases:
|
||||
if c not in harness.tests:
|
||||
harness.tests[c] = "BLOCK"
|
||||
self.add_missing_testscases(harness)
|
||||
|
||||
if out_state == "timeout":
|
||||
self.instance.reason = "Timeout"
|
||||
|
@ -1152,6 +1161,7 @@ class QEMUHandler(Handler):
|
|||
self.instance.reason = "Timeout"
|
||||
else:
|
||||
self.instance.reason = "Exited with {}".format(self.returncode)
|
||||
self.add_missing_testscases(harness)
|
||||
|
||||
def get_fifo(self):
|
||||
return self.fifo_fn
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue