Low: cts: Consolidate tests in cts-lab result output#4062
Low: cts: Consolidate tests in cts-lab result output#4062clumens merged 1 commit intoClusterLabs:mainfrom
Conversation
It's possible to run the same test multiple times, like so:
--choose "ComponentFail ComponentFail"
If you do, the test summary will list each one individually:
Jan 31 09:28:13 Test ComponentFail {'calls': 1, 'failure': 0, 'skipped': 0, 'auditfail': 0}
Jan 31 09:28:13 Test ComponentFail {'calls': 1, 'failure': 0, 'skipped': 0, 'auditfail': 0}
Instead, these can be condensed into a single row:
Feb 23 13:42:38 rhel9-ctslab-exec Test ComponentFail {'calls': 2, 'failure': 0, 'skipped': 0, 'auditfail': 0}
Note that I've chosen to leave the detailed results as they are. This
is because the details results may contain other information that
differs from run to run, even of the same test, and we may want to see
that.
Fixes T1018
| stat_summary[test.name][key] += test.stats[key] | ||
|
|
||
| self._cm.log(f"{f'Test {test.name}':<25} {stat_filter!r}") | ||
| for (name, summary) in stat_summary.items(): |
There was a problem hiding this comment.
Order isn't guaranteed here. I think for consistency with the other cts-lab modes, we should sort according to the order in tests.test_list. That becomes a bit more complicated than it should be, because self.tests is set to the full list of tests in the other modes -- for this mode (Sequence), it's set to the list of tests that were actually specified on the command line.
It would also be reasonable to sort based on the order in which the tests were specified, though that's also a tad ugly with this current dict-based implementation.
I briefly looked into doing this. I don't think we should hold up this PR further over it. There are a number of changes that would be helpful, to do this cleanly. I really shouldn't let myself spend more time on it.
2 participants
It's possible to run the same test multiple times, like so:
--choose "ComponentFail ComponentFail"
If you do, the test summary will list each one individually:
Jan 31 09:28:13 Test ComponentFail {'calls': 1, 'failure': 0, 'skipped': 0, 'auditfail': 0}
Jan 31 09:28:13 Test ComponentFail {'calls': 1, 'failure': 0, 'skipped': 0, 'auditfail': 0}
Instead, these can be condensed into a single row:
Feb 23 13:42:38 rhel9-ctslab-exec Test ComponentFail {'calls': 2, 'failure': 0, 'skipped': 0, 'auditfail': 0}
Note that I've chosen to leave the detailed results as they are. This is because the details results may contain other information that differs from run to run, even of the same test, and we may want to see that.
Fixes T1018