Journal article · 2020 · Open access

Basic Enhancement Strategies When Using Bayesian Optimization for Hyperparameter Tuning of Deep Neural Networks

Hyunghun Cho, Yongjin Kim, Eunjung Lee, Daeyoung Choi, Yongjae Lee, Wonjong Rhee

IEEE Access, 8, 52588–52608 · Published 1 January 2020 · DOI 10.1109/access.2020.2981072

Open full textCite this paper

In brief

“Basic Enhancement Strategies When Using Bayesian Optimization for Hyperparameter Tuning of Deep Neural Networks” is a 2020 journal article by Hyunghun Cho and 5 co-authors, published in IEEE Access. OpenAlex counts 333 citations as of 19 November 2026, which puts it in the top 4% of works published in 2020 by citations. It is open access under a CC BY licence: the full text is free to read at IEEE Access.

Citations
333
Most cited in
2024 (91)
Field-weighted impact
8.43
References
50
Authors
6

Abstract

Compared to the traditional machine learning models, deep neural networks (DNN) are known to be highly sensitive to the choice of hyperparameters. While the required time and effort for manual tuning has been rapidly decreasing for the well developed and commonly used DNN architectures, undoubtedly DNN hyperparameter optimization will continue to be a major burden whenever a new DNN architecture needs to be designed, a new task needs to be solved, a new dataset needs to be addressed, or an existing DNN needs to be improved further. For hyperparameter optimization of general machine learning problems, numerous automated solutions have been developed where some of the most popular solutions are based on Bayesian Optimization (BO). In this work, we analyze four fundamental strategies for enhancing BO when it is used for DNN hyperparameter optimization. Specifically, diversification, early termination, parallelization, and cost function transformation are investigated. Based on the analysis, we provide a simple yet robust algorithm for DNN hyperparameter optimization - DEEP-BO (Diversified, Early-termination-Enabled, and Parallel Bayesian Optimization). When evaluated over six DNN benchmarks, DEEP-BO mostly outperformed well-known solutions including GP-Hedge, BOHB, and the speed-up variants that use Median Stopping Rule or Learning Curve Extrapolation. In fact, DEEP-BO consistently provided the top, or at least close to the top, performance over all the benchmark types that we have tested. This indicates that DEEP-BO is a robust solution compared to the existing solutions. The DEEP-BO code is publicly available at https://github.com/snu-adsl/DEEP-BO.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Cho, H., Kim, Y., Lee, E., Choi, D., Lee, Y., & Rhee, W. (2020). Basic Enhancement Strategies When Using Bayesian Optimization for Hyperparameter Tuning of Deep Neural Networks. IEEE Access, 8, 52588–52608. https://doi.org/10.1109/access.2020.2981072

MLA 9

Cho, Hyunghun, et al. “Basic Enhancement Strategies When Using Bayesian Optimization for Hyperparameter Tuning of Deep Neural Networks.” IEEE Access, vol. 8, 2020, pp. 52588–52608. https://doi.org/10.1109/access.2020.2981072.

Chicago

Cho, Hyunghun, Yongjin Kim, Eunjung Lee, Daeyoung Choi, Yongjae Lee, and Wonjong Rhee. 2020. “Basic Enhancement Strategies When Using Bayesian Optimization for Hyperparameter Tuning of Deep Neural Networks.” IEEE Access 8: 52588–52608. https://doi.org/10.1109/access.2020.2981072.

Harvard

Cho, H. et al. (2020) ‘Basic Enhancement Strategies When Using Bayesian Optimization for Hyperparameter Tuning of Deep Neural Networks’, IEEE Access, 8, pp. 52588–52608. doi:10.1109/access.2020.2981072.

IEEE

H. Cho, Y. Kim, E. Lee, D. Choi, Y. Lee, and W. Rhee, “Basic Enhancement Strategies When Using Bayesian Optimization for Hyperparameter Tuning of Deep Neural Networks,” IEEE Access, vol. 8, pp. 52588–52608, 2020, doi: 10.1109/access.2020.2981072.

BibTeX

@article{cho2020basic,
  title     = {{Basic Enhancement Strategies When Using Bayesian Optimization for Hyperparameter Tuning of Deep Neural Networks}},
  author    = {Cho, Hyunghun and Kim, Yongjin and Lee, Eunjung and Choi, Daeyoung and Lee, Yongjae and Rhee, Wonjong},
  journal   = {IEEE Access},
  year      = {2020},
  volume    = {8},
  pages     = {52588--52608},
  doi       = {10.1109/access.2020.2981072},
  url       = {https://doi.org/10.1109/access.2020.2981072}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Citations by year

YearCitationsShare of the best year
202044
20212020
20223535
20234949
20249191
20257878
20265656

Its field-weighted citation impact is 8.43: it has been cited 8.43 times as often as the average work of the same field, type and year (1.00 is the world average).

Topic and keywords

Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification

  • Bayesian optimization
  • hyperparameter optimization
  • deep neural network
  • diversification
  • early termination
  • parallelization
  • automated machine learning

Work with this paper

Most-cited papers in Computer Science

All highly cited Computer Science papers

Questions about this paper

How do I cite this paper in APA style?

Cho, H., Kim, Y., Lee, E., Choi, D., Lee, Y., & Rhee, W. (2020). Basic Enhancement Strategies When Using Bayesian Optimization for Hyperparameter Tuning of Deep Neural Networks. IEEE Access, 8, 52588–52608. https://doi.org/10.1109/access.2020.2981072

How many times has this paper been cited?

333 times, according to OpenAlex on 19 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at IEEE Access: sign in and open the full text from this page.

Who wrote this paper?

Hyunghun Cho, Yongjin Kim, Eunjung Lee, Daeyoung Choi, Yongjae Lee, Wonjong Rhee.

What is the DOI of this paper?

10.1109/access.2020.2981072, permanently at https://doi.org/10.1109/access.2020.2981072. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 19 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.