Balancing Privacy Protection and Reproducible Entity Extraction in Real-World Data in the Development of a Framework for Deidentified Japanese Electronic Health Record Narratives Using BERT: Retrospective Observational Study
JMIR Medical Informatics · 2026-11-06 · 14 · e97411-e97411
DOI: 10.2196/97411 ↗Abstract
Abstract Background Pharmacoepidemiologic studies using real-world data often lack clinical context embedded in free-text electronic health records (EHRs). However, sharing EHR narratives is restricted by privacy requirements, and conventional deidentification approaches may reduce analytic utility and limit auditability. Objective This study aims to develop and evaluate a framework that (1) extracts structured cancer-related entities from Japanese EHR free text using BERT (Bidirectional Encoder Representations from Transformers)–based natural language processing and (2) generates deidentified analytic text that preserves contextual information for post hoc auditing under privacy constraints. Methods This retrospective observational study used the DATuM IDEA database, which integrates EHR narratives with structured records (claims, prescriptions, and procedure/surgery records) from 2 hospitals in Japan from January 1, 2019, through June 30, 2025. Deterministic linkage was performed within ICI (Integrated Clinical Care Informatics Inc) prior to deidentification. Downstream recoverability analyses were conducted within a governance-constrained analyzable text cohort derived after deidentification and preprocessing. We quantified deidentification using masking rates and evaluated downstream recoverability of TNM (tumor-node-metastasis) or stage-related mentions, internal logical consistency (M1 vs Stage IV), and proxy-based concordance using structured data treatment proxies (Anatomical Therapeutic Chemical [ATC] code L for systemic anticancer therapy and procedure-name keywords for cancer-directed interventions). A stratified manual audit of 400 deidentified narratives primarily inspected for apparent residual direct identifiers and secondarily for retention of TNM and stage outputs. Results Among 10,214 patients with neoplasms ( International Statistical Classification of Diseases, Tenth Revision [ ICD-10 ] C00-D48), linked progress notes were available for 4383 patients (82,863 records). The analyzable text cohort (Layer C) comprised 3689 patients and 38,841 documents, of which 1339 patients (1339/3689, 36.3%) and 11,914 documents (11,914/38,841, 30.7%) had at least one recoverable TNM or stage mention (Layer D). Marginal document-level recoverability in Layer C was 24.2% for T, N, and M elements and 23.6% for stage. Conditional on Layer D, it was 78.8% for T, 78.7% for N, 78.8% for M, and 76.8% for stage. Median masking rates ranged from 9.9% to 15% across major cancer categories. M1 and Stage IV mention indicators showed an overall agreement of 94.7%, positive percent agreement of 54.8%, and negative percent agreement of approximately 99%. Proxy-based concordance analysis showed positive predictive values of 0.700 for Stage IV using the systemic therapy proxy and 0.031 for early-stage classification using the procedure proxy. Manual audit demonstrated retention of intended TNM/stage-related outputs after normalization and identified no apparent residual direct identifiers in the audited deidentified narratives. Conclusions This framework generates deidentified Japanese EHR narratives that preserve contextual structure for auditing while supporting structured entity extraction, addressing the trade-off between privacy protection and reproducibility in real-world data research.
EUVIMED