Dissertation · 2018 · Open access

Extending Hyperband with Model-Based Sampling Strategies

Niklas Klein

Open access LMU (Ludwid Maxmilian's Universitat Munchen) · Published 1 January 2018 · DOI 10.5282/ubm/epub.59111

Open PDFView at the publisherCite this paper

In brief

“Extending Hyperband with Model-Based Sampling Strategies” is a 2018 dissertation by Niklas Klein, published in Open access LMU (Ludwid Maxmilian's Universitat Munchen). OpenAlex counts 1 citations as of 19 November 2026. It is open access: the full text is free to read at Open access LMU (Ludwid Maxmilian's Universitat Munchen).

Citations
1
Most cited in
2024 (1)
References
26
Authors
1

Abstract

We address the problem of hyperparameter optimization for machine learning algorithms.This is a difficult task for two main reasons: On one hand, the function modelled by the machine learning algorithm is usually derivative free (black-box).On the other hand, function evaluations are typically very expensive in terms of their computational time.The focus of this thesis lies mainly on the hyperparameters of gradient boosting models and neural networks.Both methods contain a large amount of interdependent hyperparameters.This makes the optimization especially difficult.Simple optimization approaches such as grid or random search generally do not provide satisfying results.We are going to examine more sophisticated strategies like Bayesian Optimization which try to model the underlying black-box function in order to find a better configuration.A more recent approach is given by Hyperband, which is the eponym of this thesis.In essence, it adaptively allocates resources across a set of randomly sampled configurations based on their performances.We will explore this algorithm in detail and eventually propose a possible way of combining the advantages of model-based optimization and Hyperband.Afterwards, we present our implementation of Hyperband which we based on the R6 class system.This object oriented variant provides a very generic framework which enables the user to easily modify the conventional Hyperband algorithm.We finally present the results of a benchmark experiment where we compare the aforementioned combination of Hyperband and MBO with the original algorithm.Our findings show that for boosting models, both variants perform about equal (for few training iterations, the hybrid optimizer outperforms the conventional Hyperband while for more training iterations, it is the other way around).When considering neural networks, for which Hyperband was initially intended, both Hyperband-based methods outshine the base line model (e.g. a Random Search).Particularly striking is the fact that even though our hybrid approach generated an overhead (through the additional computations resulting from the model-based procedure), it generally needed less computational time than the conventional Hyperband.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Klein, N. (2018). Extending Hyperband with Model-Based Sampling Strategies. Open access LMU (Ludwid Maxmilian's Universitat Munchen). https://doi.org/10.5282/ubm/epub.59111

MLA 9

Klein, Niklas. “Extending Hyperband with Model-Based Sampling Strategies.” Open access LMU (Ludwid Maxmilian's Universitat Munchen), 2018. https://doi.org/10.5282/ubm/epub.59111.

Chicago

Klein, Niklas. 2018. “Extending Hyperband with Model-Based Sampling Strategies.” Open access LMU (Ludwid Maxmilian's Universitat Munchen). https://doi.org/10.5282/ubm/epub.59111.

Harvard

Klein, N. (2018) ‘Extending Hyperband with Model-Based Sampling Strategies’, Open access LMU (Ludwid Maxmilian's Universitat Munchen). doi:10.5282/ubm/epub.59111.

IEEE

N. Klein, “Extending Hyperband with Model-Based Sampling Strategies,” Open access LMU (Ludwid Maxmilian's Universitat Munchen), 2018, doi: 10.5282/ubm/epub.59111.

BibTeX

@phdthesis{klein2018extending,
  title     = {{Extending Hyperband with Model-Based Sampling Strategies}},
  author    = {Klein, Niklas},
  year      = {2018},
  doi       = {10.5282/ubm/epub.59111},
  url       = {https://doi.org/10.5282/ubm/epub.59111}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Topic and keywords

Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification

  • Hyperband
  • hyperparameter optimization
  • model-based optimization
  • Bayesian optimization
  • gradient boosting
  • artificial neural networks
  • black-box optimization
  • expensive function evaluations

Work with this paper

Most-cited papers in Computer Science

All highly cited Computer Science papers

Questions about this paper

How do I cite this paper in APA style?

Klein, N. (2018). Extending Hyperband with Model-Based Sampling Strategies. Open access LMU (Ludwid Maxmilian's Universitat Munchen). https://doi.org/10.5282/ubm/epub.59111

How many times has this paper been cited?

1 times, according to OpenAlex on 19 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at Open access LMU (Ludwid Maxmilian's Universitat Munchen): sign in and open the PDF from this page.

Who wrote this paper?

Niklas Klein.

What is the DOI of this paper?

10.5282/ubm/epub.59111, permanently at https://doi.org/10.5282/ubm/epub.59111. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 19 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.