Preprint · 2024 · Open access

Why do Random Forests Work? Understanding Tree Ensembles as Self-Regularizing Adaptive Smoothers

Alicia Curth, Alan Jeffares, Mihaela van der Schaar

arXiv (Cornell University) · Published 2 February 2024 · DOI 10.48550/arxiv.2402.01502

Free PDFView at the publisherCite this paper

In brief

“Why do Random Forests Work? Understanding Tree Ensembles as Self-Regularizing Adaptive Smoothers” is a 2024 preprint by Alicia Curth and 2 co-authors, published in arXiv (Cornell University). OpenAlex counts 34 citations as of 16 November 2026. It is open access: the full text is free to read at arXiv (Cornell University).

Citations
34
Most cited in
2025 (14)
References
2
Authors
3

Abstract

Despite their remarkable effectiveness and broad application, the drivers of success underlying ensembles of trees are still not fully understood. In this paper, we highlight how interpreting tree ensembles as adaptive and self-regularizing smoothers can provide new intuition and deeper insight to this topic. We use this perspective to show that, when studied as smoothers, randomized tree ensembles not only make predictions that are quantifiably more smooth than the predictions of the individual trees they consist of, but also further regulate their smoothness at test-time based on the dissimilarity between testing and training inputs. First, we use this insight to revisit, refine and reconcile two recent explanations of forest success by providing a new way of quantifying the conjectured behaviors of tree ensembles objectively by measuring the effective degree of smoothing they imply. Then, we move beyond existing explanations for the mechanisms by which tree ensembles improve upon individual trees and challenge the popular wisdom that the superior performance of forests should be understood as a consequence of variance reduction alone. We argue that the current high-level dichotomy into bias- and variance-reduction prevalent in statistics is insufficient to understand tree ensembles -- because the prevailing definition of bias does not capture differences in the expressivity of the hypothesis classes formed by trees and forests. Instead, we show that forests can improve upon trees by three distinct mechanisms that are usually implicitly entangled. In particular, we demonstrate that the smoothing effect of ensembling can reduce variance in predictions due to noise in outcome generation, reduce variability in the quality of the learned function given fixed input data and reduce potential bias in learnable functions by enriching the available hypothesis space.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Curth, A., Jeffares, A., & van der Schaar, M. (2024). Why do Random Forests Work? Understanding Tree Ensembles as Self-Regularizing Adaptive Smoothers. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2402.01502

MLA 9

Curth, Alicia, et al. “Why do Random Forests Work? Understanding Tree Ensembles as Self-Regularizing Adaptive Smoothers.” arXiv (Cornell University), 2024. https://doi.org/10.48550/arxiv.2402.01502.

Chicago

Curth, Alicia, Alan Jeffares, and Mihaela van der Schaar. 2024. “Why do Random Forests Work? Understanding Tree Ensembles as Self-Regularizing Adaptive Smoothers.” arXiv (Cornell University). https://doi.org/10.48550/arxiv.2402.01502.

Harvard

Curth, A., Jeffares, A. and van der Schaar, M. (2024) ‘Why do Random Forests Work? Understanding Tree Ensembles as Self-Regularizing Adaptive Smoothers’, arXiv (Cornell University). doi:10.48550/arxiv.2402.01502.

IEEE

A. Curth, A. Jeffares, and M. van der Schaar, “Why do Random Forests Work? Understanding Tree Ensembles as Self-Regularizing Adaptive Smoothers,” arXiv (Cornell University), 2024, doi: 10.48550/arxiv.2402.01502.

BibTeX

@misc{curth2024random,
  title     = {{Why do Random Forests Work? Understanding Tree Ensembles as Self-Regularizing Adaptive Smoothers}},
  author    = {Curth, Alicia and Jeffares, Alan and van der Schaar, Mihaela},
  howpublished = {arXiv (Cornell University)},
  year      = {2024},
  doi       = {10.48550/arxiv.2402.01502},
  url       = {https://doi.org/10.48550/arxiv.2402.01502}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Citations by year

YearCitationsShare of the best year
202488
20251414
20261212

Topic and keywords

Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification

  • random forest
  • decision tree ensembles
  • adaptive smoothing
  • self-regularization
  • bias-variance trade-off
  • variance reduction
  • bias mitigation

Work with this paper

Most-cited papers in Computer Science

All highly cited Computer Science papers

Questions about this paper

How do I cite this paper in APA style?

Curth, A., Jeffares, A., & van der Schaar, M. (2024). Why do Random Forests Work? Understanding Tree Ensembles as Self-Regularizing Adaptive Smoothers. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2402.01502

How many times has this paper been cited?

34 times, according to OpenAlex on 16 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access: read the free full text at arXiv (Cornell University).

Who wrote this paper?

Alicia Curth, Alan Jeffares, Mihaela van der Schaar.

What is the DOI of this paper?

10.48550/arxiv.2402.01502. The permanent link is https://doi.org/10.48550/arxiv.2402.01502.

Bibliographic data from OpenAlex (CC0), last checked 16 November 2026. ResearchGuru does not host this paper; the links go to the publisher or the repository that does.