Journal article · 2025 · Open access

Benchmarking transformer-based models for medical record de-identification in a single center multi-specialty evaluation

Rachel Yi Ling Kuo, Andrew A. S. Soltan, Ciaran O’Hanlon, Alan Hasanic, David Andrew Clifton, Gary S. Collins, Dominic Furniss, David W. Eyre

iScience, 28(12), 113732 · Published 9 October 2025 · DOI 10.1016/j.isci.2025.113732

Open full textCite this paper

In brief

“Benchmarking transformer-based models for medical record de-identification in a single center multi-specialty evaluation” is a 2025 journal article by Rachel Yi Ling Kuo and 7 co-authors, published in iScience. OpenAlex counts 11 citations as of 19 November 2026. It is open access under a CC BY licence: the full text is free to read at iScience.

Citations
11
Most cited in
2026 (10)
Field-weighted impact
2.06
References
31
Authors
8

Abstract

Protecting patient confidentiality is central to enabling research using electronic health records. Automated text de-identification offers a scalable alternative to manual redaction. However, different approaches vary in accuracy and adaptability. We evaluated four transformer-based, task-specific models and five large language models on 3,650 clinical records spanning general and specialty datasets from a UK hospital group. Records were dual-annotated by clinicians, allowing precise comparison of performance. The Microsoft Azure de-identification service achieved the highest F1 score, approaching clinician performance, while fine-tuned AnonCAT and GPT-4-0125 with few-shot prompting also performed strongly. Smaller LLMs frequently over-redacted or produced hallucinatory content, limiting interpretability. Task-specific models demonstrated greater stability across datasets, while low-level adaptation improved performance in both model classes. These findings highlight that automated de-identification systems can provide effective support for large-scale sharing of clinical records, but success depends on careful model choice, adaptation strategies, and safeguards to ensure robust data utility and privacy.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Kuo, R. Y. L., Soltan, A. A. S., O’Hanlon, C., Hasanic, A., Clifton, D. A., Collins, G. S., Furniss, D., & Eyre, D. W. (2025). Benchmarking transformer-based models for medical record de-identification in a single center multi-specialty evaluation. iScience, 28(12), 113732. https://doi.org/10.1016/j.isci.2025.113732

MLA 9

Kuo, Rachel Yi Ling, et al. “Benchmarking transformer-based models for medical record de-identification in a single center multi-specialty evaluation.” iScience, vol. 28, no. 12, 2025, p. 113732. https://doi.org/10.1016/j.isci.2025.113732.

Chicago

Kuo, Rachel Yi Ling, Andrew A. S. Soltan, Ciaran O’Hanlon, Alan Hasanic, David Andrew Clifton, Gary S. Collins, Dominic Furniss, and David W. Eyre. 2025. “Benchmarking transformer-based models for medical record de-identification in a single center multi-specialty evaluation.” iScience 28 (12): 113732. https://doi.org/10.1016/j.isci.2025.113732.

Harvard

Kuo, R. Y. L. et al. (2025) ‘Benchmarking transformer-based models for medical record de-identification in a single center multi-specialty evaluation’, iScience, 28(12), p. 113732. doi:10.1016/j.isci.2025.113732.

IEEE

R. Y. L. Kuo et al., “Benchmarking transformer-based models for medical record de-identification in a single center multi-specialty evaluation,” iScience, vol. 28, no. 12, p. 113732, 2025, doi: 10.1016/j.isci.2025.113732.

BibTeX

@article{kuo2025benchmarking,
  title     = {{Benchmarking transformer-based models for medical record de-identification in a single center multi-specialty evaluation}},
  author    = {Kuo, Rachel Yi Ling and Soltan, Andrew A. S. and O’Hanlon, Ciaran and Hasanic, Alan and Clifton, David Andrew and Collins, Gary S. and Furniss, Dominic and Eyre, David W.},
  journal   = {iScience},
  year      = {2025},
  volume    = {28},
  number    = {12},
  pages     = {113732},
  doi       = {10.1016/j.isci.2025.113732},
  url       = {https://doi.org/10.1016/j.isci.2025.113732}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Topic and keywords

Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data

  • transformer models
  • large language models
  • electronic health records
  • patient confidentiality
  • clinical text annotation
  • GPT-4
  • few-shot prompting
  • hallucinations

Work with this paper

Most-cited papers in Computer Science

All highly cited Computer Science papers

Questions about this paper

How do I cite this paper in APA style?

Kuo, R. Y. L., Soltan, A. A. S., O’Hanlon, C., Hasanic, A., Clifton, D. A., Collins, G. S., Furniss, D., & Eyre, D. W. (2025). Benchmarking transformer-based models for medical record de-identification in a single center multi-specialty evaluation. iScience, 28(12), 113732. https://doi.org/10.1016/j.isci.2025.113732

How many times has this paper been cited?

11 times, according to OpenAlex on 19 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at iScience: sign in and open the full text from this page.

Who wrote this paper?

Rachel Yi Ling Kuo, Andrew A. S. Soltan, Ciaran O’Hanlon, Alan Hasanic, David Andrew Clifton, Gary S. Collins, Dominic Furniss, David W. Eyre.

What is the DOI of this paper?

10.1016/j.isci.2025.113732, permanently at https://doi.org/10.1016/j.isci.2025.113732. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 19 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.