Preprint · 2026 · Open access

Pitfalls and Remedies for Multi-Task Bayesian Optimization

Carl Hvarfner, Samuel Daulton, Maximilian Balandat, Eytan Bakshy

arXiv (Cornell University) · Published 10 July 2026 · DOI 10.48550/arxiv.2607.09073

Open PDFView at the publisherCite this paper

In brief

“Pitfalls and Remedies for Multi-Task Bayesian Optimization” is a 2026 preprint by Carl Hvarfner and 3 co-authors, published in arXiv (Cornell University). OpenAlex counts 3 citations as of 19 November 2026. It is open access: the full text is free to read at arXiv (Cornell University).

Citations
3
Most cited in
2026 (3)
References
23
Authors
4

Abstract

Bayesian optimization routinely warm-starts a target experiment with data from related source tasks, and the multi-task Gaussian process is the textbook surrogate for the job. We revisit this default in a controlled setting and find that it misestimates the cross-task correlation even in the simplest non-trivial case, affinely related source and target tasks, where a working transfer learning method should obviously succeed. We trace the failure to two independent structural mechanisms. Per-task standardization, the textbook fix for the affine slice ambiguity, propagates a finite-sample alignment error into the recovered correlation. The marginal likelihood itself identifies the correlation only at a per-sample rate that a Gaussian process at non-overlapping designs further dilutes. We propose three conservative remedies that follow from the analysis: promoting per-task means and scales to model parameters, restricting the task covariance to non-negative correlations, and co-locating part of the source and target designs. Across synthetic multi-task problems and surrogate-based hyperparameter tuning transfer, these remedies recover the target-only baseline on the simple instances, while the broader failure persists on harder instances and across most rank-based and latent-context variants.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Hvarfner, C., Daulton, S., Balandat, M., & Bakshy, E. (2026). Pitfalls and Remedies for Multi-Task Bayesian Optimization. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2607.09073

MLA 9

Hvarfner, Carl, et al. “Pitfalls and Remedies for Multi-Task Bayesian Optimization.” arXiv (Cornell University), 2026. https://doi.org/10.48550/arxiv.2607.09073.

Chicago

Hvarfner, Carl, Samuel Daulton, Maximilian Balandat, and Eytan Bakshy. 2026. “Pitfalls and Remedies for Multi-Task Bayesian Optimization.” arXiv (Cornell University). https://doi.org/10.48550/arxiv.2607.09073.

Harvard

Hvarfner, C. et al. (2026) ‘Pitfalls and Remedies for Multi-Task Bayesian Optimization’, arXiv (Cornell University). doi:10.48550/arxiv.2607.09073.

IEEE

C. Hvarfner, S. Daulton, M. Balandat, and E. Bakshy, “Pitfalls and Remedies for Multi-Task Bayesian Optimization,” arXiv (Cornell University), 2026, doi: 10.48550/arxiv.2607.09073.

BibTeX

@misc{hvarfner2026pitfalls,
  title     = {{Pitfalls and Remedies for Multi-Task Bayesian Optimization}},
  author    = {Hvarfner, Carl and Daulton, Samuel and Balandat, Maximilian and Bakshy, Eytan},
  howpublished = {arXiv (Cornell University)},
  year      = {2026},
  doi       = {10.48550/arxiv.2607.09073},
  url       = {https://doi.org/10.48550/arxiv.2607.09073}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Topic and keywords

Social Sciences › Decision Sciences › Management Science and Operations Research › Advanced Bandit Algorithms Research

  • multi-task Gaussian process
  • transfer learning
  • warm start
  • marginal likelihood

Work with this paper

Most-cited papers in Decision Sciences

All highly cited Decision Sciences papers

Questions about this paper

How do I cite this paper in APA style?

Hvarfner, C., Daulton, S., Balandat, M., & Bakshy, E. (2026). Pitfalls and Remedies for Multi-Task Bayesian Optimization. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2607.09073

How many times has this paper been cited?

3 times, according to OpenAlex on 19 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at arXiv (Cornell University): sign in and open the PDF from this page.

Who wrote this paper?

Carl Hvarfner, Samuel Daulton, Maximilian Balandat, Eytan Bakshy.

What is the DOI of this paper?

10.48550/arxiv.2607.09073, permanently at https://doi.org/10.48550/arxiv.2607.09073. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 19 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.