Preprint · 2026 · Open access
SHIELD: A Diverse Clinical Note Dataset and Distilled Small Language Models for Enterprise-Scale De-identification
arXiv (Cornell University) · Published 5 May 2026 · DOI 10.48550/arxiv.2605.03301
In brief
“SHIELD: A Diverse Clinical Note Dataset and Distilled Small Language Models for Enterprise-Scale De-identification” is a 2026 preprint by Jose Posada and 3 co-authors, published in arXiv (Cornell University). OpenAlex counts 6 citations as of 19 November 2026. It is open access: the full text is free to read at arXiv (Cornell University).
- Citations
- 6
- Most cited in
- 2026 (6)
- References
- 32
- Authors
- 4
Abstract
De-identification of clinical text is a prerequisite for the secondary use of electronic health records. Existing public benchmarks such as the i2b2 2006 and 2014 corpora are over a decade old and lack the semantic and demographic diversity of modern clinical narratives. Large Language Models (LLMs) reach state-of-the-art zero-shot extraction, but their use at enterprise scale is limited by computational cost and by hospital data governance that restricts sending Protected Health Information (PHI) to cloud APIs. We introduce SHIELD (Synthetic Human-annotated Identifier-replaced Entries for Learning and De-identification), a diverse clinical note dataset of 1,381 notes with 10,229 gold-standard PHI spans across 9 categories, built with set-cover diversity sampling across demographic and document-type strata and human-in-the-loop adjudication. We evaluate four LLMs (two proprietary, two open-weight) to establish a performance ceiling on SHIELD, then show that a teacher-student distillation framework transfers these capabilities into locally deployable Small Language Models. Our best distilled model reaches micro-averaged span-level precision of 0.89 and recall of 0.88 while running on standard workstation hardware. It trails its cloud teacher on per-category recall (0.90 vs. 0.81 macro-averaged) but remains competitive given its lower cost and on-premise deployability. Cross-dataset evaluation shows that diversity-trained models generalize well on universal structured PHI categories, while institution-specific entities remain hard to transfer in both directions, which suggests pairing broad-coverage models with specialized models for high-volume, semi-structured note types. We publicly release the SHIELD dataset and the distilled DeBERTa v3 model to provide an accurate, cost-effective de-identification pipeline deployable entirely behind institutional firewalls.
Cite this paper
Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.
APA 7
Posada, J., Love, D., Datta, S., & Desai, P. (2026). SHIELD: A Diverse Clinical Note Dataset and Distilled Small Language Models for Enterprise-Scale De-identification. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2605.03301
MLA 9
Posada, Jose, et al. “SHIELD: A Diverse Clinical Note Dataset and Distilled Small Language Models for Enterprise-Scale De-identification.” arXiv (Cornell University), 2026. https://doi.org/10.48550/arxiv.2605.03301.
Chicago
Posada, Jose, David Love, Somalee Datta, and Priya Desai. 2026. “SHIELD: A Diverse Clinical Note Dataset and Distilled Small Language Models for Enterprise-Scale De-identification.” arXiv (Cornell University). https://doi.org/10.48550/arxiv.2605.03301.
Harvard
Posada, J. et al. (2026) ‘SHIELD: A Diverse Clinical Note Dataset and Distilled Small Language Models for Enterprise-Scale De-identification’, arXiv (Cornell University). doi:10.48550/arxiv.2605.03301.
IEEE
J. Posada, D. Love, S. Datta, and P. Desai, “SHIELD: A Diverse Clinical Note Dataset and Distilled Small Language Models for Enterprise-Scale De-identification,” arXiv (Cornell University), 2026, doi: 10.48550/arxiv.2605.03301.
BibTeX
@misc{posada2026shield,
title = {{SHIELD: A Diverse Clinical Note Dataset and Distilled Small Language Models for Enterprise-Scale De-identification}},
author = {Posada, Jose and Love, David and Datta, Somalee and Desai, Priya},
howpublished = {arXiv (Cornell University)},
year = {2026},
doi = {10.48550/arxiv.2605.03301},
url = {https://doi.org/10.48550/arxiv.2605.03301}
}
Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)
Topic and keywords
Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- protected health information
- clinical notes
- knowledge distillation
- small language models
- on-device deployment
- demographic diversity
- human-in-the-loop annotation
- DeBERTa
Work with this paper
Most-cited papers in Computer Science
- ImageNet classification with deep convolutional neural networksAlex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton · Communications of the ACM · 2017 · 114,726 citations
- Long Short-Term MemorySepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 109,097 citations
- A Mathematical Theory of CommunicationClaude E. Shannon · Bell System Technical Journal · 1948 · 91,736 citations
- Deep learningYann LeCun, Yoshua Bengio, Geoffrey E. Hinton · Nature · 2015 · 90,226 citations
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal NetworksShaoqing Ren, Kaiming He, Ross Girshick and 1 more · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2016 · 74,143 citations
Questions about this paper
How do I cite this paper in APA style?
Posada, J., Love, D., Datta, S., & Desai, P. (2026). SHIELD: A Diverse Clinical Note Dataset and Distilled Small Language Models for Enterprise-Scale De-identification. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2605.03301
How many times has this paper been cited?
6 times, according to OpenAlex on 19 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.
Is this paper free to read?
Yes. It is open access at arXiv (Cornell University): sign in and open the PDF from this page.
Who wrote this paper?
Jose Posada, David Love, Somalee Datta, Priya Desai.
What is the DOI of this paper?
10.48550/arxiv.2605.03301, permanently at https://doi.org/10.48550/arxiv.2605.03301. Sign in and open it from this page.
Bibliographic data from OpenAlex (CC0), last checked 19 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.