Dissertation · 2018 · Open access
Extending Hyperband with Model-Based Sampling Strategies
Open access LMU (Ludwid Maxmilian's Universitat Munchen) · Published 1 January 2018 · DOI 10.5282/ubm/epub.59111
In brief
“Extending Hyperband with Model-Based Sampling Strategies” is a 2018 dissertation by Niklas Klein, published in Open access LMU (Ludwid Maxmilian's Universitat Munchen). OpenAlex counts 1 citations as of 19 November 2026. It is open access: the full text is free to read at Open access LMU (Ludwid Maxmilian's Universitat Munchen).
- Citations
- 1
- Most cited in
- 2024 (1)
- References
- 26
- Authors
- 1
Abstract
We address the problem of hyperparameter optimization for machine learning algorithms.This is a difficult task for two main reasons: On one hand, the function modelled by the machine learning algorithm is usually derivative free (black-box).On the other hand, function evaluations are typically very expensive in terms of their computational time.The focus of this thesis lies mainly on the hyperparameters of gradient boosting models and neural networks.Both methods contain a large amount of interdependent hyperparameters.This makes the optimization especially difficult.Simple optimization approaches such as grid or random search generally do not provide satisfying results.We are going to examine more sophisticated strategies like Bayesian Optimization which try to model the underlying black-box function in order to find a better configuration.A more recent approach is given by Hyperband, which is the eponym of this thesis.In essence, it adaptively allocates resources across a set of randomly sampled configurations based on their performances.We will explore this algorithm in detail and eventually propose a possible way of combining the advantages of model-based optimization and Hyperband.Afterwards, we present our implementation of Hyperband which we based on the R6 class system.This object oriented variant provides a very generic framework which enables the user to easily modify the conventional Hyperband algorithm.We finally present the results of a benchmark experiment where we compare the aforementioned combination of Hyperband and MBO with the original algorithm.Our findings show that for boosting models, both variants perform about equal (for few training iterations, the hybrid optimizer outperforms the conventional Hyperband while for more training iterations, it is the other way around).When considering neural networks, for which Hyperband was initially intended, both Hyperband-based methods outshine the base line model (e.g. a Random Search).Particularly striking is the fact that even though our hybrid approach generated an overhead (through the additional computations resulting from the model-based procedure), it generally needed less computational time than the conventional Hyperband.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Klein, N. (2018). Extending Hyperband with Model-Based Sampling Strategies. Open access LMU (Ludwid Maxmilian's Universitat Munchen). https://doi.org/10.5282/ubm/epub.59111
MLA 9
Klein, Niklas. “Extending Hyperband with Model-Based Sampling Strategies.” Open access LMU (Ludwid Maxmilian's Universitat Munchen), 2018. https://doi.org/10.5282/ubm/epub.59111.
Chicago
Klein, Niklas. 2018. “Extending Hyperband with Model-Based Sampling Strategies.” Open access LMU (Ludwid Maxmilian's Universitat Munchen). https://doi.org/10.5282/ubm/epub.59111.
Harvard
Klein, N. (2018) ‘Extending Hyperband with Model-Based Sampling Strategies’, Open access LMU (Ludwid Maxmilian's Universitat Munchen). doi:10.5282/ubm/epub.59111.
IEEE
N. Klein, “Extending Hyperband with Model-Based Sampling Strategies,” Open access LMU (Ludwid Maxmilian's Universitat Munchen), 2018, doi: 10.5282/ubm/epub.59111.
BibTeX
@phdthesis{klein2018extending,
title = {{Extending Hyperband with Model-Based Sampling Strategies}},
author = {Klein, Niklas},
year = {2018},
doi = {10.5282/ubm/epub.59111},
url = {https://doi.org/10.5282/ubm/epub.59111}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Topic and keywords
Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Hyperband
- hyperparameter optimization
- model-based optimization
- Bayesian optimization
- gradient boosting
- artificial neural networks
- black-box optimization
- expensive function evaluations
Work with this paper
Most-cited papers in Computer Science
- ImageNet classification with deep convolutional neural networksAlex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton · Communications of the ACM · 2017 · 114,726 citations
- Long Short-Term MemorySepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 109,097 citations
- A Mathematical Theory of CommunicationClaude E. Shannon · Bell System Technical Journal · 1948 · 91,736 citations
- Deep learningYann LeCun, Yoshua Bengio, Geoffrey E. Hinton · Nature · 2015 · 90,226 citations
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksShaoqing Ren, Kaiming He, Ross Girshick and 1 more · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2016 · 74,143 citations
Questions about this paper
How do I cite this paper in APA style?
Klein, N. (2018). Extending Hyperband with Model-Based Sampling Strategies. Open access LMU (Ludwid Maxmilian's Universitat Munchen). https://doi.org/10.5282/ubm/epub.59111
How many times has this paper been cited?
1 times, according to OpenAlex on 19 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access at Open access LMU (Ludwid Maxmilian's Universitat Munchen): sign in and open the PDF from this page.
Who wrote this paper?
Niklas Klein.
What is the DOI of this paper?
10.5282/ubm/epub.59111, permanently at https://doi.org/10.5282/ubm/epub.59111. Sign in and open it from this page.
Bibliographic data from OpenAlex (CC0), last checked 19 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.