gen_kobject_list.py: OrderedDict for < 3.6 determinism

Dictionaries are iterated in a random order by Python 3.5 and before.
This could have caused "Unstable" CI in PR #13921 and maybe others.
Anyway we want builds to be determimistic by default. Explicit
randomness can be added for better coverage but not by default.

1. When running "make kobj_types_h_target" repeatedly one can observe
that the following .h files keep changing in
build/zephyr/include/generated/:

- kobj-types-enum.h
- otype-to-str.h
- otype-to-size.h

Switching kobjects to OrderedDict makes these 3 .h files deterministic.

2. When running this test repeatedly with CONFIG_USERSPACE=y:

  rm build/zephyr/*.gperf && make -C build obj_list

... the dict used for --gperf-output seems to be deterministic, probably
because its keys are all integers (memory addresses). However we can't
take that for granted with Python < 3.6 so out of caution also switch
the output of find_objects() in elf_helper.py to a sorted OrderedDict.

PS: I would normally prefer official Python documentation to
StackOverflow however this one is a good summary and has all the
multiple pointers to the... official Python documentation.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
This commit is contained in:
Marc Herbert 2019-03-05 14:31:44 -08:00 committed by Anas Nashif
commit f78288be20
3 changed files with 39 additions and 16 deletions

View file

@ -9,6 +9,8 @@ import os
import struct
from distutils.version import LooseVersion
from collections import OrderedDict
import elftools
from elftools.elf.elffile import ELFFile
from elftools.elf.sections import SymbolTableSection
@ -518,7 +520,14 @@ class ElfHelper:
ret[addr] = ko
self.debug("found %d kernel object instances total" % len(ret))
return ret
# 1. Before python 3.7 dict order is not guaranteed. With Python
# 3.5 it doesn't seem random with *integer* keys but can't
# rely on that.
# 2. OrderedDict means _insertion_ order, so not enough because
# built from other (random!) dicts: need to _sort_ first.
# 3. Sorting memory address looks good.
return OrderedDict(sorted(ret.items()))
def get_symbols(self):
for section in self.elf.iter_sections():