Univention Bugzilla – Bug 51991
Benchmark for CFG driven tests
Last modified: 2021-05-07 18:20:03 CEST
We want to gather further data about the execution time of individual steps in our current test setup. Many tests are configured by our own 'CFG files', which contain a list of commands to be executed. These are the 'steps' which we want to further measure. We will later try to aggregate these results into nice graphical representations. These will probably help to identify performance bottlenecks and altogether a tool to prioritize possible tuning measures. We could also heuristically check how long individual steps usually take and abort test steps if they got slower than usual for any reason.
I reverted that patch because it breaks ASYNC in ucs-ec2-tools [4.4-5] b65146bca6 Bug #51991 test: Disable benchmark test/utils/utils.sh | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-) [4.4-6] d40ae550bb Bug #51991 test: Disable benchmark test/utils/utils.sh | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-) [5.0-0] 3c95753118 Bug #51991 test: Disable benchmark test/utils/utils.sh | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-)
As ucs-ec2-tools executes those `.cfg` files maybe add the code there. Categorization for "run": - unique run number, e.g. Jenkins JOB id - git branch name, e.g. `4.4-6` - path to .cfg file within git, e.g. `test/scenarios/autotest-091-master-s4.cfg` Sub-Categorization: - section name, .e.g `master092` - phase, e.g. `command1` - step, e.g. `1.` Data to store per command: - command executed, e.g. ` utils.sh; basic_setup` - start - end - [duration := end - start] - exit code Data stored per run: - all collected log files - individual test names, their results [and output] Also have a look at <https://logz.io/blog/open-source-monitoring-tools/>. - ucs-ec2-tools must not block if the service is temporarily unavailable (--on-unavailable=abort|block|retry-X-times|to-local-file|skip) - the service should accept arbitrary messages (log-data, XML, data.gz, ...) - data should not expire by default
https://git.knut.univention.de/dist/repo-ng/-/issues/43