This is a Master level project.
Project Thoth is a recommendation engine for Python software stacks. As Python is becoming de-facto the language of choice for many data scientists and machine learning engineers, Thoth aggregates information about popular Python machine learning and data science packages, such as TensorFlow, Pytorch and many others – this aggregation is done on different levels. One such data aggregation is done in a pipeline that is capable of installing and testing Python packages into different operating systems with a different set of software present in it (different versions of Python interpreter, different versions of glibc, …).
The goal of this thesis is to evaluate the performance impact of various parts of machine learning libraries. The performance can be measured on the library level, as well as checking different aspects of the performance when dependencies of the software stack are adjusted (e.g. NumPy for TensorFlow, MKL for Intel TensorFlow builds, …)
Thesis overview:
- Get familiar with project Thoth and its goals.
- Get familiar with Thoth’s Amun service and Thoth’s dependency monkey.
- Get familiar with the input and output of Thoth’s Amun service.
- Get familiar with an already existing set of “Performance Indicators” used in project Thoth.
- Study TensorFlow and PyTorch architecture, spot parts relevant for performance analyses.
- Based on the aggregated data set, design and build one or more predictive models that can predict performance of AI/ML libraries.
- Evaluate the accuracy of predictive models created and discuss possible future improvements and additional changes that could be made.