Conference paper · 2017 · Open access
Efficient Hyperparameter Optimization for Deep Learning Algorithms Using Deterministic RBF Surrogates
Proceedings of the AAAI Conference on Artificial Intelligence, 31(1) · Published 12 February 2017 · DOI 10.1609/aaai.v31i1.10647
In brief
“Efficient Hyperparameter Optimization for Deep Learning Algorithms Using Deterministic RBF Surrogates” is a 2017 conference paper by Ilija Ilievski and 3 co-authors, published in Proceedings of the AAAI Conference on Artificial Intelligence. OpenAlex counts 166 citations as of 20 November 2026, which puts it in the top 7% of works published in 2017 by citations. It is open access: the full text is free to read at Proceedings of the AAAI Conference on Artificial Intelligence.
- Citations
- 166
- Most cited in
- 2020 (34)
- Field-weighted impact
- 7.49
- References
- 33
- Authors
- 4
Abstract
Automatically searching for optimal hyperparameter configurations is of crucial importance for applying deep learning algorithms in practice. Recently, Bayesian optimization has been proposed for optimizing hyperparameters of various machine learning algorithms. Those methods adopt probabilistic surrogate models like Gaussian processes to approximate and minimize the validation error function of hyperparameter values. However, probabilistic surrogates require accurate estimates of sufficient statistics (e.g., covariance) of the error distribution and thus need many function evaluations with a sizeable number of hyperparameters. This makes them inefficient for optimizing hyperparameters of deep learning algorithms, which are highly expensive to evaluate. In this work, we propose a new deterministic and efficient hyperparameter optimization method that employs radial basis functions as error surrogates. The proposed mixed integer algorithm, called HORD, searches the surrogate for the most promising hyperparameter values through dynamic coordinate search and requires many fewer function evaluations. HORD does well in low dimensions but it is exceptionally better in higher dimensions. Extensive evaluations on MNIST and CIFAR-10 for four deep neural networks demonstrate HORD significantly outperforms the well-established Bayesian optimization methods such as GP, SMAC, and TPE. For instance, on average, HORD is more than 6 times faster than GP-EI in obtaining the best configuration of 19 hyperparameters.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Ilievski, I., Akhtar, T., Feng, J., & Shoemaker, C. A. (2017). Efficient Hyperparameter Optimization for Deep Learning Algorithms Using Deterministic RBF Surrogates. Proceedings of the AAAI Conference on Artificial Intelligence, 31(1). https://doi.org/10.1609/aaai.v31i1.10647
MLA 9
Ilievski, Ilija, et al. “Efficient Hyperparameter Optimization for Deep Learning Algorithms Using Deterministic RBF Surrogates.” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 31, no. 1, 2017. https://doi.org/10.1609/aaai.v31i1.10647.
Chicago
Ilievski, Ilija, Taimoor Akhtar, Jiashi Feng, and Christine A. Shoemaker. 2017. “Efficient Hyperparameter Optimization for Deep Learning Algorithms Using Deterministic RBF Surrogates.” Proceedings of the AAAI Conference on Artificial Intelligence 31 (1). https://doi.org/10.1609/aaai.v31i1.10647.
Harvard
Ilievski, I. et al. (2017) ‘Efficient Hyperparameter Optimization for Deep Learning Algorithms Using Deterministic RBF Surrogates’, Proceedings of the AAAI Conference on Artificial Intelligence, 31(1). doi:10.1609/aaai.v31i1.10647.
IEEE
I. Ilievski, T. Akhtar, J. Feng, and C. A. Shoemaker, “Efficient Hyperparameter Optimization for Deep Learning Algorithms Using Deterministic RBF Surrogates,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 31, no. 1, 2017, doi: 10.1609/aaai.v31i1.10647.
BibTeX
@inproceedings{ilievski2017efficient,
title = {{Efficient Hyperparameter Optimization for Deep Learning Algorithms Using Deterministic RBF Surrogates}},
author = {Ilievski, Ilija and Akhtar, Taimoor and Feng, Jiashi and Shoemaker, Christine A.},
booktitle = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2017},
volume = {31},
number = {1},
doi = {10.1609/aaai.v31i1.10647},
url = {https://doi.org/10.1609/aaai.v31i1.10647}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Citations by year
| Year | Citations | Share of the best year |
|---|---|---|
| 2017 | 4 | |
| 2018 | 8 | |
| 2019 | 31 | |
| 2020 | 34 | |
| 2021 | 23 | |
| 2022 | 20 | |
| 2023 | 11 | |
| 2024 | 15 | |
| 2025 | 14 | |
| 2026 | 4 |
Its field-weighted citation impact is 7.49: it has been cited 7.49 times as often as the average work of the same field, type and year (1.00 is the world average).
Topic and keywords
Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- hyperparameter optimization
- deep learning
- radial basis function surrogate
- deterministic optimization
- mixed integer programming
- coordinate search
- expensive function evaluations
- Bayesian optimization comparison
Work with this paper
Most-cited papers in Computer Science
- ImageNet classification with deep convolutional neural networksAlex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton · Communications of the ACM · 2017 · 114,726 citations
- Long Short-Term MemorySepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 109,097 citations
- A Mathematical Theory of CommunicationClaude E. Shannon · Bell System Technical Journal · 1948 · 91,736 citations
- Deep learningYann LeCun, Yoshua Bengio, Geoffrey E. Hinton · Nature · 2015 · 90,226 citations
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksShaoqing Ren, Kaiming He, Ross Girshick and 1 more · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2016 · 74,143 citations
Questions about this paper
How do I cite this paper in APA style?
Ilievski, I., Akhtar, T., Feng, J., & Shoemaker, C. A. (2017). Efficient Hyperparameter Optimization for Deep Learning Algorithms Using Deterministic RBF Surrogates. Proceedings of the AAAI Conference on Artificial Intelligence, 31(1). https://doi.org/10.1609/aaai.v31i1.10647
How many times has this paper been cited?
166 times, according to OpenAlex on 20 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access at Proceedings of the AAAI Conference on Artificial Intelligence: sign in and open the full text from this page.
Who wrote this paper?
Ilija Ilievski, Taimoor Akhtar, Jiashi Feng, Christine A. Shoemaker.
What is the DOI of this paper?
10.1609/aaai.v31i1.10647, permanently at https://doi.org/10.1609/aaai.v31i1.10647. Sign in and open it from this page.
Bibliographic data from OpenAlex (CC0), last checked 20 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.