Evaluation of automated deidentification of general practice free text health records
Project Details
The aim of this project is to develop, test, and evaluate different methods to de-identify textual data sets from general practice electronic medical records (EMR) (optimise de-identification methods) so these data can be securely extracted for further analysis using natural language processing (NLP).
Patron ID: PAT028
Project Lead:
Text analysis is an emerging field in health data research. Use of textual health data for research purposes has been limited in Australia to date, due to concerns around the personal identifying information they may contain. The aim of this project is to develop, test, and evaluate different methods to de-identify textual data sets from general practice electronic medical records (EMR) (optimise de-identification methods) so these data can be securely extracted for further analysis using natural language processing (NLP).
This project will utilise the VicREN/Patron Manager to seek consent from up to 10 practices contributing data to the Patron data repository to allow the extraction of free text data from their EMR, specifically progress notes and pathology test results. The data will be extracted once and destroyed after five years of last publication, as per ethics requirements – i.e. consent from practices will not extend to the free text fields being regularly extracted and included in the Patron dataset. Data from textual fields will be extracted after usual GRHANITE® de-identification processes have been employed; that is, the researchers will be working only with de-identified data. In this project, the researchers will also work to optimise the de-identification methods of all fields extracted by GRHANITE®; further information is below under ‘Study design’.
This project is part of a broader collaboration between multiple departments within the University of Melbourne: the School of Computing and Information Systems (Engineering); the Department of General Practice, the School of Population and Global Health and the Centre for Digital Transformation of Health (MDHS); and the Melbourne Data Analytics Platform (MDAP). The broad study includes evaluation of de-identification methods as applied to both general practice and hospital data.
The results of this project will facilitate larger scale use of clinical textual data (for example, patient progress notes) for research purposes in Australia, and will provide opportunities for future research with world-class collaborators within the University and externally.
Research Outcomes
- Brief information flyer (PDF)
- Information flyer and participant consent form (PDF)
- Information flyer and participant consent form (Word)
Project Outcome
This project analysed different methods of anonymisation of GP free text notes. The project was able to determine that a Large Language Model approach preformed optimally. This was published and the final Stanford LLM model incorporated into our GRHANITE anonymisation system.
Research Publications
- El-Hayek, C., Barzegar, S., Faux, N., Doyle, K., Pillai, P., Mutch, S. J., Vaisey, A., Ward, R., Sanci, L., Dunn, A. G., Hellard, M. E., Hocking, J. S., Verspoor, K., & Boyle, D. I. R. (2023). An evaluation of existing text de-identification tools for use with patient progress notes from Australian general practice. International Journal of Medical Informatics, 173, 105021. ISSN 1386-5056. doi: 10.1016/j.ijmedinf.2023.105021.
- Klapaukh, R., El-Hayek, C., Boyle, D. Do transformers generalise better than bespoke tools for anonymisation? Informatics in Medicine Unlocked (2025). https://doi.org/10.1016/j.imu.2024.101607
- Klapaukh R., Boyle,D (2024) CensorCheck: A Tool for Evaluating Protected Health Information Detection Systems. EBook: Studies in Health Technology and Informatics Volume 318: Health. Innovation. Community: It Starts With Us Pages 174-175 https://doi.org/10.3233/SHTI240914