Dissertation · 2020 · Open access

Practical Bayesian optimisation for hyperparameter tuning

Ahsan Alvi

Oxford University Research Archive (ORA) (University of Oxford) · Published 1 January 2020

Open PDFView at the publisherCite this paper

In brief

“Practical Bayesian optimisation for hyperparameter tuning” is a 2020 dissertation by Ahsan Alvi, published in Oxford University Research Archive (ORA) (University of Oxford). OpenAlex counts 3 citations as of 19 November 2026. It is open access: the full text is free to read at Oxford University Research Archive (ORA) (University of Oxford).

Citations
3
Most cited in
2023 (1)
References
19
Authors
1

Abstract

Advances in machine learning have had, and continue to have, a profound effect on scientific research and industrial activities. We are able to uncover insights contained within large troves of data and develop models to solve problems that seemed infeasible until recently. Before we can train a model, we have to define high-level properties, also known as the model’s hyperparameters. Examples include the architecture of a neural network, the class and parameters of a stochastic optimisation algorithm, the number of trees in a random forest or the kernel in a Gaussian process. One of the key challenges in developing good machine learning models is choosing good values for these hyperparameters. Evaluating the quality of a set of hyperparameters is usually computationally costly, as it requires us to first train a model specified by the hyperparameters before we can evaluate a score. The computational costs of scoring a configuration means that we want to perform this hyperparameter search with as few trials as possible. Bayesian optimisation (BO) is a general-purpose efficient global optimisation technique that has proven to be remarkably effective for many expensive optimisation tasks, including hyperparameter tuning. This thesis brings together multiple contributions aimed at making BO more practically applicable, focussing on uncertain-input measurements as well as model hyperparameter tuning. The case of data with input noise is relevant when we are modelling (or finding extremal values of) a quantity defined on a spatial grid. We undertake a comparative study evaluating existing uncertain-input GP methods, and develop a novel approach based on the unscented transform that outperforms existing uncertain-input methods. We then address the question of parallelising BO to make optimal use of available computing resources. We develop a novel class of methods for asynchronous parallel BO, and demonstrate empirically the benefits of our approach, as well as more generally showing the advantages of asynchronous over synchronous BO. Finally, we propose a new framework for dealing with a mixture of categorical and continuous hyperparameters. With few exceptions, model hyperparameters consist of both categorical and continuous variables, but BO approaches have mainly focussed on continuous search spaces. We present a novel framework that combines multi-armed bandits with continuous BO methods and leverages each of their strengths to achieve state-of-the-art performance on real-world optimisation tasks.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Alvi, A. (2020). Practical Bayesian optimisation for hyperparameter tuning. Oxford University Research Archive (ORA) (University of Oxford). https://ora.ox.ac.uk/objects/uuid:08e836e1-74bc-4b6e-956c-703197398654/files/d5712m656c

MLA 9

Alvi, Ahsan. “Practical Bayesian optimisation for hyperparameter tuning.” Oxford University Research Archive (ORA) (University of Oxford), 2020. https://ora.ox.ac.uk/objects/uuid:08e836e1-74bc-4b6e-956c-703197398654/files/d5712m656c.

Chicago

Alvi, Ahsan. 2020. “Practical Bayesian optimisation for hyperparameter tuning.” Oxford University Research Archive (ORA) (University of Oxford). https://ora.ox.ac.uk/objects/uuid:08e836e1-74bc-4b6e-956c-703197398654/files/d5712m656c.

Harvard

Alvi, A. (2020) ‘Practical Bayesian optimisation for hyperparameter tuning’, Oxford University Research Archive (ORA) (University of Oxford). Available at: https://ora.ox.ac.uk/objects/uuid:08e836e1-74bc-4b6e-956c-703197398654/files/d5712m656c.

IEEE

A. Alvi, “Practical Bayesian optimisation for hyperparameter tuning,” Oxford University Research Archive (ORA) (University of Oxford), 2020.

BibTeX

@phdthesis{alvi2020practical,
  title     = {{Practical Bayesian optimisation for hyperparameter tuning}},
  author    = {Alvi, Ahsan},
  year      = {2020},
  url       = {https://ora.ox.ac.uk/objects/uuid:08e836e1-74bc-4b6e-956c-703197398654/files/d5712m656c}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Citations by year

YearCitationsShare of the best year
202311
202411
202511

Topic and keywords

Social Sciences › Decision Sciences › Management Science and Operations Research › Advanced Bandit Algorithms Research

  • Bayesian optimization
  • hyperparameter optimization
  • unscented transform
  • multi-armed bandits
  • expensive function evaluations
  • random forest
  • neural architecture search

Work with this paper

Most-cited papers in Decision Sciences

All highly cited Decision Sciences papers

Questions about this paper

How do I cite this paper in APA style?

Alvi, A. (2020). Practical Bayesian optimisation for hyperparameter tuning. Oxford University Research Archive (ORA) (University of Oxford). https://ora.ox.ac.uk/objects/uuid:08e836e1-74bc-4b6e-956c-703197398654/files/d5712m656c

How many times has this paper been cited?

3 times, according to OpenAlex on 19 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at Oxford University Research Archive (ORA) (University of Oxford): sign in and open the PDF from this page.

Who wrote this paper?

Ahsan Alvi.

Bibliographic data from OpenAlex (CC0), last checked 19 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.