Journal article · 2017 · Open access

Automated de-identification of clinical free-text: IJPDS (2017) Issue 1, Vol 1:324 Proceedings of the IPDLN Conference (August 2016)

Azad Dehghan, Tom Liptrot, Catherine M. O’Hara, Matthew Barker-Hewitt, Daniel Tibble, Goran Nenadić

International Journal for Population Data Science, 1(1) · Published 19 April 2017 · DOI 10.23889/ijpds.v1i1.345

Open PDFView at the publisherCite this paper

In brief

“Automated de-identification of clinical free-text: IJPDS (2017) Issue 1, Vol 1:324 Proceedings of the IPDLN Conference (August 2016)” is a 2017 journal article by Azad Dehghan and 5 co-authors, published in International Journal for Population Data Science. OpenAlex counts 2 citations as of 19 November 2026. It is open access under a CC BY-NC-ND licence: the full text is free to read at International Journal for Population Data Science.

Citations
2
Most cited in
2021 (1)
Authors
6

Abstract

ObjectivesIncreasing interest to use unstructured electronic patient records for research has attracted attention to automated de-identification methods to conduct large scale removal of Personal Identifiable Information (PII). PII mainly include identifiable information such as person names, dates (e.g., date of birth), reference numbers (e.g., hospital number, NHS number), locations (e.g., hospital names, addresses), contacts (e.g., telephone, e-mail), occupation, age, and other identity information (ethnical, religion, sexual) mentioned in a private context. De-identification of clinical free-text remains crucial to enable large-scale data access for health research while adhering to legal (Data Protection Act 1998) and ethical obligations. Here we present a computational method developed to automatically remove PII from clinical text. ApproachIn order to automatically identify PII in clinical text, we have developed and validated a Natural Language Processing (NLP) method which combine knowledge- (lexical dictionaries and rules) and data-driven (linear-chain conditional random fields) techniques. In addition, we have designed a novel two-pass recognition approach that uses the output of the initial pass to create patient-level and run-time dictionaries used to identify PII mentions that lack specific contextual clues considered by the initial entity extraction modules. The labelled data used to model and validate our techniques were generated using six human annotators and two distinct types of free-text from The Christie NHS Foundation Trust: (1) clinical correspondence (400 documents) and (2) clinical notes (1,300 documents). ResultsThe de-identification approach was developed and validated using a 60/40 percent split between the development and test datasets. The preliminary results show that our method achieves 97% and 93% token-level F1-measure on clinical correspondence and clinical notes respectively. In addition, the proposed two-pass recognition method was found particularly effective for longitudinal records. Notably, the performances are comparable to human benchmarks (using inter annotator agreements) of 97% and 90% F1 respectively. ConclusionsWe have developed and validated a state-of-the-art method that matches human benchmarks in identification and removal of PII from free-text clinical records. The method has been further validated across multiple institutions and countries (United States and United Kingdom), where we have identified a notable NLP challenge of cross-dataset adaption and have proposed using active learning methods to address this problem. The algorithm, including an active learning component, will be provided as open source to the healthcare community.

Cite this paper

Formatted from the catalogue record. Check names, capitals and page numbers against the paper before you submit.

APA 7

Dehghan, A., Liptrot, T., O’Hara, C. M., Barker-Hewitt, M., Tibble, D., & Nenadić, G. (2017). Automated de-identification of clinical free-text: IJPDS (2017) Issue 1, Vol 1:324 Proceedings of the IPDLN Conference (August 2016). International Journal for Population Data Science, 1(1). https://doi.org/10.23889/ijpds.v1i1.345

MLA 9

Dehghan, Azad, et al. “Automated de-identification of clinical free-text: IJPDS (2017) Issue 1, Vol 1:324 Proceedings of the IPDLN Conference (August 2016).” International Journal for Population Data Science, vol. 1, no. 1, 2017. https://doi.org/10.23889/ijpds.v1i1.345.

Chicago

Dehghan, Azad, Tom Liptrot, Catherine M. O’Hara, Matthew Barker-Hewitt, Daniel Tibble, and Goran Nenadić. 2017. “Automated de-identification of clinical free-text: IJPDS (2017) Issue 1, Vol 1:324 Proceedings of the IPDLN Conference (August 2016).” International Journal for Population Data Science 1 (1). https://doi.org/10.23889/ijpds.v1i1.345.

Harvard

Dehghan, A. et al. (2017) ‘Automated de-identification of clinical free-text: IJPDS (2017) Issue 1, Vol 1:324 Proceedings of the IPDLN Conference (August 2016)’, International Journal for Population Data Science, 1(1). doi:10.23889/ijpds.v1i1.345.

IEEE

A. Dehghan, T. Liptrot, C. M. O’Hara, M. Barker-Hewitt, D. Tibble, and G. Nenadić, “Automated de-identification of clinical free-text: IJPDS (2017) Issue 1, Vol 1:324 Proceedings of the IPDLN Conference (August 2016),” International Journal for Population Data Science, vol. 1, no. 1, 2017, doi: 10.23889/ijpds.v1i1.345.

BibTeX

@article{dehghan2017automated,
  title     = {{Automated de-identification of clinical free-text: IJPDS (2017) Issue 1, Vol 1:324 Proceedings of the IPDLN Conference (August 2016)}},
  author    = {Dehghan, Azad and Liptrot, Tom and O’Hara, Catherine M. and Barker-Hewitt, Matthew and Tibble, Daniel and Nenadić, Goran},
  journal   = {International Journal for Population Data Science},
  year      = {2017},
  volume    = {1},
  number    = {1},
  doi       = {10.23889/ijpds.v1i1.345},
  url       = {https://doi.org/10.23889/ijpds.v1i1.345}
}

Download for a reference manager: BibTeX (.bib) · RIS for Zotero, Mendeley and EndNote (.ris)

Topic and keywords

Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data

  • personally identifiable information
  • clinical text
  • clinical notes
  • entity recognition
  • linear-chain conditional random fields
  • lexical dictionary
  • cross-dataset adaptation
  • active learning

Work with this paper

Most-cited papers in Computer Science

All highly cited Computer Science papers

Questions about this paper

How do I cite this paper in APA style?

Dehghan, A., Liptrot, T., O’Hara, C. M., Barker-Hewitt, M., Tibble, D., & Nenadić, G. (2017). Automated de-identification of clinical free-text: IJPDS (2017) Issue 1, Vol 1:324 Proceedings of the IPDLN Conference (August 2016). International Journal for Population Data Science, 1(1). https://doi.org/10.23889/ijpds.v1i1.345

How many times has this paper been cited?

2 times, according to OpenAlex on 19 November 2026. Google Scholar and Scopus count different sets of sources, so their numbers differ.

Is this paper free to read?

Yes. It is open access at International Journal for Population Data Science: sign in and open the PDF from this page.

Who wrote this paper?

Azad Dehghan, Tom Liptrot, Catherine M. O’Hara, Matthew Barker-Hewitt, Daniel Tibble, Goran Nenadić.

What is the DOI of this paper?

10.23889/ijpds.v1i1.345, permanently at https://doi.org/10.23889/ijpds.v1i1.345. Sign in and open it from this page.

Bibliographic data from OpenAlex (CC0), last checked 19 November 2026. ResearchGuru does not host this paper; once you are signed in, the links open it at the publisher or the repository that does.