Preprint · 2026 · Open access

SHIELD: A Diverse Clinical Note Dataset and Distilled Small Language Models for Enterprise-Scale De-identification

Jose Posada, David Love, Somalee Datta, Priya Desai

arXiv (Cornell University) · Published 5 May 2026 · DOI 10.48550/arxiv.2605.03301

Open PDFView at the publisherCite this paper

In brief

“SHIELD: A Diverse Clinical Note Dataset and Distilled Small Language Models for Enterprise-Scale De-identification” is a 2026 preprint by Jose Posada and 3 co-authors, published in arXiv (Cornell University). OpenAlex counts 6 citations as of 19 November 2026. It is open access: the full text is free to read at arXiv (Cornell University).

Citations
6
Most cited in
2026 (6)
References
32
Authors
4

Abstract

De-identification of clinical text is a prerequisite for the secondary use of electronic health records. Existing public benchmarks such as the i2b2 2006 and 2014 corpora are over a decade old and lack the semantic and demographic diversity of modern clinical narratives. Large Language Models (LLMs) reach state-of-the-art zero-shot extraction, but their use at enterprise scale is limited by computational cost and by hospital data governance that restricts sending Protected Health Information (PHI) to cloud APIs. We introduce SHIELD (Synthetic Human-annotated Identifier-replaced Entries for Learning and De-identification), a diverse clinical note dataset of 1,381 notes with 10,229 gold-standard PHI spans across 9 categories, built with set-cover diversity sampling across demographic and document-type strata and human-in-the-loop adjudication. We evaluate four LLMs (two proprietary, two open-weight) to establish a performance ceiling on SHIELD, then show that a teacher-student distillation framework transfers these capabilities into locally deployable Small Language Models. Our best distilled model reaches micro-averaged span-level precision of 0.89 and recall of 0.88 while running on standard workstation hardware. It trails its cloud teacher on per-category recall (0.90 vs. 0.81 macro-averaged) but remains competitive given its lower cost and on-premise deployability. Cross-dataset evaluation shows that diversity-trained models generalize well on universal structured PHI categories, while institution-specific entities remain hard to transfer in both directions, which suggests pairing broad-coverage models with specialized models for high-volume, semi-structured note types. We publicly release the SHIELD dataset and the distilled DeBERTa v3 model to provide an accurate, cost-effective de-identification pipeline deployable entirely behind institutional firewalls.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Posada, J., Love, D., Datta, S., & Desai, P. (2026). SHIELD: A Diverse Clinical Note Dataset and Distilled Small Language Models for Enterprise-Scale De-identification. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2605.03301

MLA 9

Posada, Jose, et al. “SHIELD: A Diverse Clinical Note Dataset and Distilled Small Language Models for Enterprise-Scale De-identification.” arXiv (Cornell University), 2026. https://doi.org/10.48550/arxiv.2605.03301.

Chicago

Posada, Jose, David Love, Somalee Datta, and Priya Desai. 2026. “SHIELD: A Diverse Clinical Note Dataset and Distilled Small Language Models for Enterprise-Scale De-identification.” arXiv (Cornell University). https://doi.org/10.48550/arxiv.2605.03301.

Harvard

Posada, J. et al. (2026) ‘SHIELD: A Diverse Clinical Note Dataset and Distilled Small Language Models for Enterprise-Scale De-identification’, arXiv (Cornell University). doi:10.48550/arxiv.2605.03301.

IEEE

J. Posada, D. Love, S. Datta, and P. Desai, “SHIELD: A Diverse Clinical Note Dataset and Distilled Small Language Models for Enterprise-Scale De-identification,” arXiv (Cornell University), 2026, doi: 10.48550/arxiv.2605.03301.

BibTeX

@misc{posada2026shield,
  title     = {{SHIELD: A Diverse Clinical Note Dataset and Distilled Small Language Models for Enterprise-Scale De-identification}},
  author    = {Posada, Jose and Love, David and Datta, Somalee and Desai, Priya},
  howpublished = {arXiv (Cornell University)},
  year      = {2026},
  doi       = {10.48550/arxiv.2605.03301},
  url       = {https://doi.org/10.48550/arxiv.2605.03301}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Topic and keywords

Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data

  • protected health information
  • clinical notes
  • knowledge distillation
  • small language models
  • on-device deployment
  • demographic diversity
  • human-in-the-loop annotation
  • DeBERTa

Work with this paper

Most-cited papers in Computer Science

All highly cited Computer Science papers

Questions about this paper

How do I cite this paper in APA style?

Posada, J., Love, D., Datta, S., & Desai, P. (2026). SHIELD: A Diverse Clinical Note Dataset and Distilled Small Language Models for Enterprise-Scale De-identification. arXiv (Cornell University). https://doi.org/10.48550/arxiv.2605.03301

How many times has this paper been cited?

6 times, according to OpenAlex on 19 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at arXiv (Cornell University): sign in and open the PDF from this page.

Who wrote this paper?

Jose Posada, David Love, Somalee Datta, Priya Desai.

What is the DOI of this paper?

10.48550/arxiv.2605.03301, permanently at https://doi.org/10.48550/arxiv.2605.03301. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 19 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.