Dissertation · 2020 · Open access
Practical Bayesian optimisation for hyperparameter tuning
Oxford University Research Archive (ORA) (University of Oxford) · Published 1 January 2020
In brief
“Practical Bayesian optimisation for hyperparameter tuning” is a 2020 dissertation by Ahsan Alvi, published in Oxford University Research Archive (ORA) (University of Oxford). OpenAlex counts 3 citations as of 19 November 2026. It is open access: the full text is free to read at Oxford University Research Archive (ORA) (University of Oxford).
- Citations
- 3
- Most cited in
- 2023 (1)
- References
- 19
- Authors
- 1
Abstract
Advances in machine learning have had, and continue to have, a profound effect on scientific research and industrial activities. We are able to uncover insights contained within large troves of data and develop models to solve problems that seemed infeasible until recently. Before we can train a model, we have to define high-level properties, also known as the model’s hyperparameters. Examples include the architecture of a neural network, the class and parameters of a stochastic optimisation algorithm, the number of trees in a random forest or the kernel in a Gaussian process. One of the key challenges in developing good machine learning models is choosing good values for these hyperparameters. Evaluating the quality of a set of hyperparameters is usually computationally costly, as it requires us to first train a model specified by the hyperparameters before we can evaluate a score. The computational costs of scoring a configuration means that we want to perform this hyperparameter search with as few trials as possible. Bayesian optimisation (BO) is a general-purpose efficient global optimisation technique that has proven to be remarkably effective for many expensive optimisation tasks, including hyperparameter tuning. This thesis brings together multiple contributions aimed at making BO more practically applicable, focussing on uncertain-input measurements as well as model hyperparameter tuning. The case of data with input noise is relevant when we are modelling (or finding extremal values of) a quantity defined on a spatial grid. We undertake a comparative study evaluating existing uncertain-input GP methods, and develop a novel approach based on the unscented transform that outperforms existing uncertain-input methods. We then address the question of parallelising BO to make optimal use of available computing resources. We develop a novel class of methods for asynchronous parallel BO, and demonstrate empirically the benefits of our approach, as well as more generally showing the advantages of asynchronous over synchronous BO. Finally, we propose a new framework for dealing with a mixture of categorical and continuous hyperparameters. With few exceptions, model hyperparameters consist of both categorical and continuous variables, but BO approaches have mainly focussed on continuous search spaces. We present a novel framework that combines multi-armed bandits with continuous BO methods and leverages each of their strengths to achieve state-of-the-art performance on real-world optimisation tasks.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Alvi, A. (2020). Practical Bayesian optimisation for hyperparameter tuning. Oxford University Research Archive (ORA) (University of Oxford). https://ora.ox.ac.uk/objects/uuid:08e836e1-74bc-4b6e-956c-703197398654/files/d5712m656c
MLA 9
Alvi, Ahsan. “Practical Bayesian optimisation for hyperparameter tuning.” Oxford University Research Archive (ORA) (University of Oxford), 2020. https://ora.ox.ac.uk/objects/uuid:08e836e1-74bc-4b6e-956c-703197398654/files/d5712m656c.
Chicago
Alvi, Ahsan. 2020. “Practical Bayesian optimisation for hyperparameter tuning.” Oxford University Research Archive (ORA) (University of Oxford). https://ora.ox.ac.uk/objects/uuid:08e836e1-74bc-4b6e-956c-703197398654/files/d5712m656c.
Harvard
Alvi, A. (2020) ‘Practical Bayesian optimisation for hyperparameter tuning’, Oxford University Research Archive (ORA) (University of Oxford). Available at: https://ora.ox.ac.uk/objects/uuid:08e836e1-74bc-4b6e-956c-703197398654/files/d5712m656c.
IEEE
A. Alvi, “Practical Bayesian optimisation for hyperparameter tuning,” Oxford University Research Archive (ORA) (University of Oxford), 2020.
BibTeX
@phdthesis{alvi2020practical,
title = {{Practical Bayesian optimisation for hyperparameter tuning}},
author = {Alvi, Ahsan},
year = {2020},
url = {https://ora.ox.ac.uk/objects/uuid:08e836e1-74bc-4b6e-956c-703197398654/files/d5712m656c}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Citations by year
| Year | Citations | Share of the best year |
|---|---|---|
| 2023 | 1 | |
| 2024 | 1 | |
| 2025 | 1 |
Topic and keywords
Social Sciences › Decision Sciences › Management Science and Operations Research › Advanced Bandit Algorithms Research
- Bayesian optimization
- hyperparameter optimization
- unscented transform
- multi-armed bandits
- expensive function evaluations
- random forest
- neural architecture search
Work with this paper
Most-cited papers in Decision Sciences
- The PRISMA 2020 statement: an updated guideline for reporting systematic reviewsMatthew James Page, Joanne E. McKenzie, Patrick M. Bossuyt and 23 more · BMJ · 2021 · 144,919 citations
- Evaluating Structural Equation Models with Unobservable Variables and Measurement ErrorClaes Fornell, David F. Larcker · Journal of Marketing Research · 1981 · 118,336 citations
- Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternativesLi‐tze Hu, Peter M. Bentler · Structural Equation Modeling A Multidisciplinary Journal · 1999 · 117,461 citations
- Common method biases in behavioral research: A critical review of the literature and recommended remedies.Philip M. Podsakoff, SCOTT B. MACKENZIE, Jeong Yeon Lee and 1 more · Journal of Applied Psychology · 2003 · 89,248 citations
- Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statementDavid Moher, Alessandro Liberati, Jennifer Tetzlaff and 2 more · BMJ · 2009 · 85,524 citations
Questions about this paper
How do I cite this paper in APA style?
Alvi, A. (2020). Practical Bayesian optimisation for hyperparameter tuning. Oxford University Research Archive (ORA) (University of Oxford). https://ora.ox.ac.uk/objects/uuid:08e836e1-74bc-4b6e-956c-703197398654/files/d5712m656c
How many times has this paper been cited?
3 times, according to OpenAlex on 19 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access at Oxford University Research Archive (ORA) (University of Oxford): sign in and open the PDF from this page.
Who wrote this paper?
Ahsan Alvi.
Bibliographic data from OpenAlex (CC0), last checked 19 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.