Optimizing Hyperparameters for Machine Learning Algorithms in Production (PDF)

Machine learning (ML) offers the potential to train data-based models and therefore to extract knowledge from data. Due to an increase in networking and digitalization, data and consequently the application of ML are growing in production. The creation of ML models includes several tasks that need to be conducted within data integration, data preparation, modeling, and deployment.
One key design decision in this context is the selection of the hyperparameters of an ML algorithm - regardless of whether this task is conducted manually by a data scientist or automatically by an AutoML system. Therefore, data scientists and AutoML systems rely on hyperparameter optimization (HPO) techniques: algorithms that automatically identify good hyperparameters for ML algorithms. The selection of the HPO technique is of great relevance, since it can improve the final performance of an ML model by up to 62 % and reduce its errors by up to 95 %, compared to computing with default values.
As the selection of the HPO technique depends on different domain-specific influences, it becomes more and more popular to use decision support systems to facilitate this selection. Since no approach exists, which covers the requirements from the production domain, the main research question of this thesis was: Can a decision support system be developed that supports in the selecting of HPO techniques in the production domain?