This project can be tailored to both Bachelor and Master level.
Testing of complex products requires a surprisingly tricky approach. The thesis “Corner cases and possible improvements of traditional automated testing” has pointed out solutions to a few of them. It described how the Seven-Dimensional matrix can be decomposed and individual cells handled, and results composed back. In this thesis, the student should focus on the actions that take place once some part of the final cell finishes. These can be notifications to the maintainer of the suite or to the author of the test, generation of steps to reproduce, generation of reproducer, generating a more specific job, a rerun of the job, a rerun of the job with a different setup, removal of the job, analysis of the history of failure extracting stack traces, pointing out a commit causing the regression, environment and standard output/error. All leading to a simple “test passed” or “that exact line really failed, because…” without any human touch, with focus on test case diversity – unified approach should be taken for a single test, for benchamrk or for cumulative tests and benchmarks.