Skip to content

Commit 21a552d

Browse files
committed
Warn if frameworks exceed the maximum number of test mutations
> The number of test mutations will be limited to 10. #8420 This change shows a warning message for frameworks if the maximum is reached.
1 parent 46de0d4 commit 21a552d

File tree

1 file changed

+8
-0
lines changed

1 file changed

+8
-0
lines changed

toolset/utils/metadata.py

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -191,6 +191,14 @@ def parse_config(self, config, directory):
191191
% config['framework'],
192192
color=Fore.YELLOW)
193193

194+
maximum_tests = 10
195+
if len(tests_to_run) > maximum_tests:
196+
message = [
197+
"Framework %s defines %s tests in benchmark_config.json (max is %s)."
198+
% (config['framework'], len(tests_to_run), maximum_tests),
199+
"Contact maintainers and remove deprecated or discarded ones to make room."
200+
]
201+
log("\n".join(message), color=Fore.YELLOW)
194202
# Check that each test configuration is acceptable
195203
# Throw exceptions if a field is missing, or how to improve the field
196204
for test_name, test_keys in test.items():

0 commit comments

Comments
 (0)