Hyperparameter Tuning

Hyperparameter tuning is key area in machine learning and is an active area of research. Machine learning algorithms almost always contain hyperparameters (a couple to a couple dozen parameters that control the model complexity and training behavior) which have significant impact on model performance and therefore need to be tuned. Searching the parameter space in a brute force manner can be prohibitively costly. Algorithm-assisted hyperparameter tuning addresses this. This is an active area of research which looks for the methods that will find the best values of hyperparameters automatically and as fast as possible. In this presentation I will talk about Bayesian hyperparameter tuning algorithms, their types and characteristics and briefly mention the latest research in hyperparameter tuning.