Journal article · 2026 · Open access

Molecular deep learning at the edge of chemical space

Derek van Tilborg, Luke Rossen, Francesca Grisoni

Nature Machine Intelligence, 8(4), 575–587 · Published 22 April 2026 · DOI 10.1038/s42256-026-01216-w

Open PDFView at the publisherCite this paper

In brief

“Molecular deep learning at the edge of chemical space” is a 2026 journal article by Derek van Tilborg and 2 co-authors, published in Nature Machine Intelligence. OpenAlex counts 34 citations as of 17 November 2026, which puts it in the top 5% of works published in 2026 by citations. It is open access under a CC BY-NC-ND licence: the full text is free to read at Nature Machine Intelligence.

Citations
34
Most cited in
2026 (30)
Field-weighted impact
41.12
References
82
Authors
3

Abstract

Molecular machine learning models often fail to generalize beyond the chemical space of their training data, limiting their ability to reliably perform predictions on structurally novel bioactive molecules. Here, to advance the ability of machine learning to go beyond the ‘edge’ of their training chemical space, we introduce a joint modelling approach that combines molecular property prediction with molecular reconstruction. This approach allows the introduction of unfamiliarity, a reconstruction-based metric that enables the estimation of model generalizability. Via a systematic analysis spanning more than 30 bioactivity datasets, we demonstrate that unfamiliarity not only effectively identifies out-of-distribution molecules but also serves as a reliable predictor of classifier performance. Even when faced with the presence of strong distribution shifts on large-scale molecular libraries, unfamiliarity yields robust and meaningful molecular insights that go unnoticed by traditional methods. Finally, we experimentally validate unfamiliarity-based molecule screening in the wet lab for two clinically relevant kinases, discovering seven compounds with low micromolar potency and limited similarity to training molecules. This demonstrates that unfamiliarity can extend the reach of machine learning beyond the edge of the charted chemical space, advancing the discovery of diverse and structurally novel molecules. van Tilborg et al. introduce a deep learning model trained to simultaneously predict bioactivity and reconstruct molecules. The model provides a principled measure of distribution shifts, experimentally validated by discovering active inhibitors for two clinically relevant kinase targets.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

van Tilborg, D., Rossen, L., & Grisoni, F. (2026). Molecular deep learning at the edge of chemical space. Nature Machine Intelligence, 8(4), 575–587. https://doi.org/10.1038/s42256-026-01216-w

MLA 9

van Tilborg, Derek, et al. “Molecular deep learning at the edge of chemical space.” Nature Machine Intelligence, vol. 8, no. 4, 2026, pp. 575–587. https://doi.org/10.1038/s42256-026-01216-w.

Chicago

van Tilborg, Derek, Luke Rossen, and Francesca Grisoni. 2026. “Molecular deep learning at the edge of chemical space.” Nature Machine Intelligence 8 (4): 575–587. https://doi.org/10.1038/s42256-026-01216-w.

Harvard

van Tilborg, D., Rossen, L. and Grisoni, F. (2026) ‘Molecular deep learning at the edge of chemical space’, Nature Machine Intelligence, 8(4), pp. 575–587. doi:10.1038/s42256-026-01216-w.

IEEE

D. van Tilborg, L. Rossen, and F. Grisoni, “Molecular deep learning at the edge of chemical space,” Nature Machine Intelligence, vol. 8, no. 4, pp. 575–587, 2026, doi: 10.1038/s42256-026-01216-w.

BibTeX

@article{vantilborg2026molecular,
  title     = {{Molecular deep learning at the edge of chemical space}},
  author    = {van Tilborg, Derek and Rossen, Luke and Grisoni, Francesca},
  journal   = {Nature Machine Intelligence},
  year      = {2026},
  volume    = {8},
  number    = {4},
  pages     = {575--587},
  doi       = {10.1038/s42256-026-01216-w},
  url       = {https://doi.org/10.1038/s42256-026-01216-w}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Topic and keywords

Life Sciences › Pharmacology, Toxicology and Pharmaceutics › Drug Discovery › Computational Drug Discovery Methods

  • molecular machine learning
  • out-of-distribution generalization
  • molecular reconstruction
  • distribution shift
  • bioactivity prediction
  • kinase inhibitors
  • chemical space

Work with this paper

Most-cited papers in Pharmacology, Toxicology and Pharmaceutics

All highly cited Pharmacology, Toxicology and Pharmaceutics papers

Questions about this paper

How do I cite this paper in APA style?

van Tilborg, D., Rossen, L., & Grisoni, F. (2026). Molecular deep learning at the edge of chemical space. Nature Machine Intelligence, 8(4), 575–587. https://doi.org/10.1038/s42256-026-01216-w

How many times has this paper been cited?

34 times, according to OpenAlex on 17 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at Nature Machine Intelligence: sign in and open the PDF from this page.

Who wrote this paper?

Derek van Tilborg, Luke Rossen, Francesca Grisoni.

What is the DOI of this paper?

10.1038/s42256-026-01216-w, permanently at https://doi.org/10.1038/s42256-026-01216-w. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 17 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.