BERT-based natural language processing analysis of French CT reports: Application to the measurement of the positivity rate for pulmonary embolism

Abstract

Introduction

Appropriate use of imaging, especially ionizing exams such as CT, is a major concern, and decision-support tools aim to reduce unnecessary requests. Measuring their effect requires the positive yield of exams, which means analysing report content — hard without structured reports, which are rarely used in emergency settings.

NLP methods have long been used to detect findings in free-text CT reports. BERT brings state-of-the-art biomedical text mining and can be adapted to a domain and a language without labels. This work builds a BERT-based pipeline for French chest CT reports and uses it to measure the diagnostic yield of CT requested for suspected pulmonary embolism.

Method

2.1Population and reports

All CT reports from Hospices Civils de Lyon between January 2018 and December 2019 were collected, written by a large and diverse panel of radiologists and structured in four parts: clinical referral, technique, findings and conclusion. Reports were anonymized, with IRB approval and waived consent.

2.2Two-stage sentence classification

Instead of classifying each pathology directly, a first classifier separates findings sentences into non-chest, healthy chest and pathological chest; a second one assigns pathological chest sentences to 11 pathology types. Splitting the task this way handles the strong imbalance between categories.

2.3Language model and training

The French CamemBERT model is first adapted to radiology language by unsupervised masked language modelling on report sentences, then fine-tuned in a supervised way into the two classifiers.

2.4Pulmonary embolism detection

In the 2018 test set, chest CT exams are found with stemmed keywords in the technique section (e.g. “thor”, “card”, “pulm”, “TAVI”, “coroscanner”), and exams requested for PE with keywords in the clinical referral (e.g. “embo”, “dimer”, “phlebi”). Two experienced radiologists then rated each selected report as requested for PE or not, and positive for PE or not. A report is NLP-positive when at least one of its findings sentences is classified as pulmonary embolism.

Results

3.1Domain-adapted language model

Fine-tuning CamemBERT on 50,350 validation sentences cut perplexity from 4,539.01 to 1.39.

3.2Sentence classification

3.3Pulmonary embolism and positivity rate

Key findings

• Near-perfect PE detection: 99.13% accuracy, 98.28% sensitivity and 99.29% specificity against radiologists’ reading.
• Positivity rate: 15.8% of the 4,042 exams requested for suspected PE were positive, consistent with a recent large French cohort.
• Errors reflect uncertainty: most discrepancies come from uncertain or ambiguous wording, such as doubtful distal defects or acute versus chronic PE.

Discussion & limitations

Contextual embeddings such as BERT outperform non-contextual methods for capturing syntax and for downstream tasks, and the approach transfers to other languages through open-source pretrained models; the French model can serve other tasks in French-speaking countries. Earlier European studies reported comparable PE detection from French and German reports with maximum-entropy and CNN-based models.

The classifiers are mono-label, although a sentence can describe several findings; focusing on the findings section, where findings are usually described separately, limits the impact. Medical uncertainty in reports remains hard to model, and the study did not compare against simpler NLP methods such as NegEx or FastContext.

Related links

1. BERT-based natural language processing analysis of French CT reports: Application to the measurement of the positivity rate for pulmonary embolism, Research in Diagnostic and Interventional Imaging 6 (2023) 100027. Open access under CC BY-NC-ND 4.0.
2. L. Martin et al., CamemBERT: a Tasty French Language Model, ACL 2020.
3. J. Devlin et al., BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, NAACL 2019.

BibTeX

@article{djahnine2023bertctreports,
  title = {BERT-based natural language processing analysis of French CT reports: Application to the measurement of the positivity rate for pulmonary embolism},
  author = {E. Jupin-Delevaux and A. Djahnine and F. Talbot and A. Richard and S. Gouttard and A. Mansuy and P. Douek and S. A. Si-Mohamed and L. Boussel},
  journal = {Research in Diagnostic and Interventional Imaging (ReDII)},
  year = {2023},
  doi = {10.1016/j.redii.2023.100027}
}