Conference paper · 2017 · Open access

Efficient Hyperparameter Optimization for Deep Learning Algorithms Using Deterministic RBF Surrogates

Ilija Ilievski, Taimoor Akhtar, Jiashi Feng, Christine A. Shoemaker

Proceedings of the AAAI Conference on Artificial Intelligence, 31(1) · Published 12 February 2017 · DOI 10.1609/aaai.v31i1.10647

Open full textCite this paper

In brief

“Efficient Hyperparameter Optimization for Deep Learning Algorithms Using Deterministic RBF Surrogates” is a 2017 conference paper by Ilija Ilievski and 3 co-authors, published in Proceedings of the AAAI Conference on Artificial Intelligence. OpenAlex counts 166 citations as of 20 November 2026, which puts it in the top 7% of works published in 2017 by citations. It is open access: the full text is free to read at Proceedings of the AAAI Conference on Artificial Intelligence.

Citations
166
Most cited in
2020 (34)
Field-weighted impact
7.49
References
33
Authors
4

Abstract

Automatically searching for optimal hyperparameter configurations is of crucial importance for applying deep learning algorithms in practice. Recently, Bayesian optimization has been proposed for optimizing hyperparameters of various machine learning algorithms. Those methods adopt probabilistic surrogate models like Gaussian processes to approximate and minimize the validation error function of hyperparameter values. However, probabilistic surrogates require accurate estimates of sufficient statistics (e.g., covariance) of the error distribution and thus need many function evaluations with a sizeable number of hyperparameters. This makes them inefficient for optimizing hyperparameters of deep learning algorithms, which are highly expensive to evaluate. In this work, we propose a new deterministic and efficient hyperparameter optimization method that employs radial basis functions as error surrogates. The proposed mixed integer algorithm, called HORD, searches the surrogate for the most promising hyperparameter values through dynamic coordinate search and requires many fewer function evaluations. HORD does well in low dimensions but it is exceptionally better in higher dimensions. Extensive evaluations on MNIST and CIFAR-10 for four deep neural networks demonstrate HORD significantly outperforms the well-established Bayesian optimization methods such as GP, SMAC, and TPE. For instance, on average, HORD is more than 6 times faster than GP-EI in obtaining the best configuration of 19 hyperparameters.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Ilievski, I., Akhtar, T., Feng, J., & Shoemaker, C. A. (2017). Efficient Hyperparameter Optimization for Deep Learning Algorithms Using Deterministic RBF Surrogates. Proceedings of the AAAI Conference on Artificial Intelligence, 31(1). https://doi.org/10.1609/aaai.v31i1.10647

MLA 9

Ilievski, Ilija, et al. “Efficient Hyperparameter Optimization for Deep Learning Algorithms Using Deterministic RBF Surrogates.” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 31, no. 1, 2017. https://doi.org/10.1609/aaai.v31i1.10647.

Chicago

Ilievski, Ilija, Taimoor Akhtar, Jiashi Feng, and Christine A. Shoemaker. 2017. “Efficient Hyperparameter Optimization for Deep Learning Algorithms Using Deterministic RBF Surrogates.” Proceedings of the AAAI Conference on Artificial Intelligence 31 (1). https://doi.org/10.1609/aaai.v31i1.10647.

Harvard

Ilievski, I. et al. (2017) ‘Efficient Hyperparameter Optimization for Deep Learning Algorithms Using Deterministic RBF Surrogates’, Proceedings of the AAAI Conference on Artificial Intelligence, 31(1). doi:10.1609/aaai.v31i1.10647.

IEEE

I. Ilievski, T. Akhtar, J. Feng, and C. A. Shoemaker, “Efficient Hyperparameter Optimization for Deep Learning Algorithms Using Deterministic RBF Surrogates,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 31, no. 1, 2017, doi: 10.1609/aaai.v31i1.10647.

BibTeX

@inproceedings{ilievski2017efficient,
  title     = {{Efficient Hyperparameter Optimization for Deep Learning Algorithms Using Deterministic RBF Surrogates}},
  author    = {Ilievski, Ilija and Akhtar, Taimoor and Feng, Jiashi and Shoemaker, Christine A.},
  booktitle = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year      = {2017},
  volume    = {31},
  number    = {1},
  doi       = {10.1609/aaai.v31i1.10647},
  url       = {https://doi.org/10.1609/aaai.v31i1.10647}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Citations by year

YearCitationsShare of the best year
201744
201888
20193131
20203434
20212323
20222020
20231111
20241515
20251414
202644

Its field-weighted citation impact is 7.49: it has been cited 7.49 times as often as the average work of the same field, type and year (1.00 is the world average).

Topic and keywords

Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification

  • hyperparameter optimization
  • deep learning
  • radial basis function surrogate
  • deterministic optimization
  • mixed integer programming
  • coordinate search
  • expensive function evaluations
  • Bayesian optimization comparison

Work with this paper

Most-cited papers in Computer Science

All highly cited Computer Science papers

Questions about this paper

How do I cite this paper in APA style?

Ilievski, I., Akhtar, T., Feng, J., & Shoemaker, C. A. (2017). Efficient Hyperparameter Optimization for Deep Learning Algorithms Using Deterministic RBF Surrogates. Proceedings of the AAAI Conference on Artificial Intelligence, 31(1). https://doi.org/10.1609/aaai.v31i1.10647

How many times has this paper been cited?

166 times, according to OpenAlex on 20 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at Proceedings of the AAAI Conference on Artificial Intelligence: sign in and open the full text from this page.

Who wrote this paper?

Ilija Ilievski, Taimoor Akhtar, Jiashi Feng, Christine A. Shoemaker.

What is the DOI of this paper?

10.1609/aaai.v31i1.10647, permanently at https://doi.org/10.1609/aaai.v31i1.10647. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 20 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.