test: Support tests which can only be run manually
At present we normally write tests either in Python or in C. But most
Python tests end up doing a lot of checks which would be better done in C.
Checks done in C are orders of magnitude faster and it is possible to get
full access to U-Boot's internal workings, rather than just relying on
the command line.
The model is to have a Python test set up some things and then use C code
(in a unit test) to check that they were done correctly. But we don't want
those checks to happen as part of normal test running, since each C unit
tests is dependent on the associate Python tests, so cannot run without
it.
To acheive this, add a new UT_TESTF_MANUAL flag to use with the C 'check'
tests, so that they can be skipped by default when the 'ut' command is
used. Require that tests have a name ending with '_norun', so that pytest
knows to skip them.
Signed-off-by: Simon Glass <sjg@chromium.org>
diff --git a/test/py/conftest.py b/test/py/conftest.py
index 304e931..fc9dd3a 100644
--- a/test/py/conftest.py
+++ b/test/py/conftest.py
@@ -289,7 +289,13 @@
m = re_ut_test_list.search(l)
if not m:
continue
- vals.append(m.group(1) + ' ' + m.group(2))
+ suite, name = m.groups()
+
+ # Tests marked with _norun should only be run manually using 'ut -f'
+ if name.endswith('_norun'):
+ continue
+
+ vals.append(f'{suite} {name}')
ids = ['ut_' + s.replace(' ', '_') for s in vals]
metafunc.parametrize(fixture_name, vals, ids=ids)