Dissertation · 2020 · Open access

Sample-efficient Optimization Using Neural Networks

Mashall Aryan

Open Access Victoria University of Wellington | Te Herenga Waka (Figshare) · Published 1 January 2020 · DOI 10.26686/wgtn.17147828

Open PDFView at the publisherCite this paper

In brief

“Sample-efficient Optimization Using Neural Networks” is a 2020 dissertation by Mashall Aryan, published in Open Access Victoria University of Wellington | Te Herenga Waka (Figshare). OpenAlex counts 0 citations as of 19 November 2026. It is open access: the full text is free to read at Open Access Victoria University of Wellington | Te Herenga Waka (Figshare).

Citations
0
References
188
Authors
1

Abstract

The solution to many science and engineering problems includes identifying the minimum or maximum of an unknown continuous function whose evaluation inflicts non-negligible costs in terms of resources such as money, time, human attention or computational processing. In such a case, the choice of new points to evaluate is critical. A successful approach has been to choose these points by considering a distribution over plausible surfaces, conditioned on all previous points and their evaluations. In this sequential bi-step strategy, also known as Bayesian Optimization, first a prior is defined over possible functions and updated to a posterior in the light of available observations. Then using this posterior, namely the surrogate model, an infill criterion is formed and utilized to find the next location to sample from. By far the most common prior distribution and infill criterion are Gaussian Process and Expected Improvement, respectively. The popularity of Gaussian Processes in Bayesian optimization is partially due to their ability to represent the posterior in closed form. Nevertheless, the Gaussian Process is afflicted with several shortcomings that directly affect its performance. For example, inference scales poorly with the amount of data, numerical stability degrades with the number of data points, and strong assumptions about the observation model are required, which might not be consistent with reality. These drawbacks encourage us to seek better alternatives. This thesis studies the application of Neural Networks to enhance Bayesian Optimization. It proposes several Bayesian optimization methods that use neural networks either as their surrogates or in the infill criterion. This thesis introduces a novel Bayesian Optimization method in which Bayesian Neural Networks are used as a surrogate. This has reduced the computational complexity of inference in surrogate from cubic (on the number of observation) in GP to linear. Different variations of Bayesian Neural Networks (BNN) are put into practice and inferred using a Monte Carlo sampling. The results show that Monte Carlo Bayesian Neural Network surrogate could performed better than, or at least comparably to the Gaussian Process-based Bayesian optimization methods on a set of benchmark problems. This work develops a fast Bayesian Optimization method with an efficient surrogate building process. This new Bayesian Optimization algorithm utilizes Bayesian Random-Vector Functional Link Networks as surrogate. In this family of models the inference is only performed on a small subset of the entire model parameters and the rest are randomly drawn from a prior. The proposed methods are tested on a set of benchmark continuous functions and hyperparameter optimization problems and the results show the proposed methods are competitive…

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Aryan, M. (2020). Sample-efficient Optimization Using Neural Networks. Open Access Victoria University of Wellington | Te Herenga Waka (Figshare). https://doi.org/10.26686/wgtn.17147828

MLA 9

Aryan, Mashall. “Sample-efficient Optimization Using Neural Networks.” Open Access Victoria University of Wellington | Te Herenga Waka (Figshare), 2020. https://doi.org/10.26686/wgtn.17147828.

Chicago

Aryan, Mashall. 2020. “Sample-efficient Optimization Using Neural Networks.” Open Access Victoria University of Wellington | Te Herenga Waka (Figshare). https://doi.org/10.26686/wgtn.17147828.

Harvard

Aryan, M. (2020) ‘Sample-efficient Optimization Using Neural Networks’, Open Access Victoria University of Wellington | Te Herenga Waka (Figshare). doi:10.26686/wgtn.17147828.

IEEE

M. Aryan, “Sample-efficient Optimization Using Neural Networks,” Open Access Victoria University of Wellington | Te Herenga Waka (Figshare), 2020, doi: 10.26686/wgtn.17147828.

BibTeX

@phdthesis{aryan2020sample,
  title     = {{Sample-efficient Optimization Using Neural Networks}},
  author    = {Aryan, Mashall},
  year      = {2020},
  doi       = {10.26686/wgtn.17147828},
  url       = {https://doi.org/10.26686/wgtn.17147828}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Topic and keywords

Social Sciences › Decision Sciences › Management Science and Operations Research › Advanced Bandit Algorithms Research

  • Bayesian optimization
  • neural network surrogate model
  • Bayesian neural networks
  • infill criterion
  • expected improvement
  • Kriging surrogate model
  • random vector functional link network
  • hyperparameter optimization

Work with this paper

Most-cited papers in Decision Sciences

All highly cited Decision Sciences papers

Questions about this paper

How do I cite this paper in APA style?

Aryan, M. (2020). Sample-efficient Optimization Using Neural Networks. Open Access Victoria University of Wellington | Te Herenga Waka (Figshare). https://doi.org/10.26686/wgtn.17147828

How many times has this paper been cited?

0 times, according to OpenAlex on 19 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at Open Access Victoria University of Wellington | Te Herenga Waka (Figshare): sign in and open the PDF from this page.

Who wrote this paper?

Mashall Aryan.

What is the DOI of this paper?

10.26686/wgtn.17147828, permanently at https://doi.org/10.26686/wgtn.17147828. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 19 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.