Weakly-Supervised Pathology Detection and Localization in 3D Chest CT

Abstract

Method

The goal is to distinguish four semiological signs present in most pulmonary pathologies, plus normal lung tissue: LLDA (low lung density area, covering emphysema and cysts), CONS (consolidation, where air is replaced by fluid or tissue), NOD (nodules and masses) and IP (interstitial pattern, including honeycombing, ground glass and reticulation).

A patch-based approach is used for three reasons: lower annotation cost with localized supervision, lower memory use than full volumes, and a built-in proxy for localization. It also suits contrastive pretraining, which needs a large number of samples: patches multiply the samples available from each scan.

2.1Contrastive pretraining (SimCLR)

1. Patch sampling. Random 50 × 50 × 50 patches containing at least 80% lung, using an automatic lung segmentation.
2. Two correlated views. A transformation module T randomly alters each patch twice, x_i = T(x) and x_j = T(x), using domain-specific transformations.
3. Encoder. CT-Net, a 2.5D alternative to 3D ResNets that allows ImageNet initialization: the patch is read as stacks of three-channel images by a 2D ResNet-50, combined, reduced by 3D convolutions and projected to a 2048-d representation h.
4. Contrastive loss. A 2-layer MLP head g maps h to a 1024-d vector z, and the NT-Xent loss pulls the two views of the same patch together while pushing apart the other 2(N − 1) samples in the batch. The head is discarded after pretraining.

Pretraining runs on 256,000 random patches with Adam (learning rate 10⁻³), 1,000 epochs, batch size 256, L2 regularization 10⁻⁶ and cosine decay without restarts.

2.2Choosing the data transformations

Eight transformations common in medical imaging were studied: mirror, 90° rotation, zoom (random crop and resize), colour distortion, Gaussian blur, Laplacian, masking with the lung mask, and a random lung mask from another patient. Following Chen et al., each of the 56 ordered pairs was evaluated with a frozen pretrained encoder over 10 runs.

Key findings

• Zoom is key: the best pairs all include zoom — mirror then zoom reaches 82.0%, zoom then random mask 80.2%, rotation then zoom 80.1%.
• Intensity changes hurt: Gaussian blur with Laplacian drops to 55.9%, and colour distortion consistently degrades accuracy (down to 66.5%), since Hounsfield values are essential to tell the signs apart.
• Final choice: random mask, mirror, 90° rotation and zoom, applied sequentially; the same set is used as data augmentation for the random and ImageNet baselines.

2.3Patch classifier

A 5-output classification head is added to the pretrained encoder and fine-tuned on 3,281 labelled patches (1,295 NORMAL, 550 LLDA, 377 CONS, 784 NOD, 275 IP) with categorical cross-entropy and dropout, SGD (learning rate 10⁻⁴), 500 epochs, batch size 256 and cosine decay. Six configurations are compared: random or ImageNet initialization trained end to end (with or without augmentation), and SimCLR pretraining with frozen weights or end-to-end fine-tuning.

2.4Patient-level classification and 3D localization

The patch classifier is swept over the whole scan with 50³ patches overlapping by 10 voxels, producing a likelihood map for each of the 5 labels, smoothed by a per-class Gaussian convolution. Because training patches cover at least 80% lung, border patches create a domain shift at inference; normal-lung filling fills the regions outside the lung mask with healthy lung texture matched to the scan's dose, contrast and slice thickness. The maps are then binarized and filtered, and a threshold T on the volume of pathological regions decides whether the exam is abnormal. T is chosen on the validation ROC curve: T = 7.5 cm³.

Experiments

3.1Data

An internal dataset of 3,029 chest CT scans of unique patients from Hospices Civils de Lyon (2019), acquired on Philips, Siemens, Toshiba and GE scanners and selected by an expert radiologist for relevant abnormalities, image quality and diversity of semiological signs. Lungs are segmented automatically; voxels are windowed to [−1000, +200] HU and normalized to [−1, +1].

Metrics: precision, recall, F1, AUROC and DeLong p-values, at patch and patient level, in multi-label and binary settings. Localization is assessed qualitatively by an experienced radiologist on 100 random test cases, scoring completeness and false positives from 1 to 4.

3.2Representation learning

(a) Random initialization

(b) ImageNet initialization

(c) SimCLR, frozen weights

(d) SimCLR, fine-tuned

Key findings

• Structured latent space: clusters matching each semiological sign appear clearly only after SimCLR pretraining and fine-tuning; random and ImageNet encoders mix the classes.

3.3Patch-level classification

The patch test set contains 236 LLDA, 162 CONS, 336 NOD, 118 IP and 555 NORMAL patches. SimCLR pretraining with fine-tuning reaches an F1 score of 0.943, against 0.677 for random weights, 0.792 for ImageNet and 0.902 for frozen SimCLR weights.

Key findings

• Label efficiency: SimCLR with fine-tuning outperforms every other strategy at every level of supervision, and is already strong from 30% of the labels.
• No plateau: all strategies keep improving with more labels, and the gap in favour of SimCLR fine-tuning holds.

3.4Patient-level classification

Key findings

• Pretraining matters: SimCLR fine-tuning with normal-lung filling reaches AUROC 0.931 on multi-label classification, against 0.651 from random initialization and 0.767 from ImageNet.
• Normal-lung filling helps every configuration, notably for nodules and interstitial pattern.
• Diffuse vs focal: diffuse signs are detected best (IP 0.974, LLDA 0.952) and focal ones are harder (NOD 0.853, CONS 0.791), in line with previous studies.
• Triage: normal versus abnormal reaches AUROC 0.963 with F1 0.974.

3.5Qualitative results: 3D localization

Key findings

• Completeness: a useful alert in 88.8% of findings — 62.6% complete, 16.2% clinically relevant and 10.1% partial; only 11.1% insufficient.
• False positives: 88.3% of findings usable — 47.1% with no false positives, 26.1% with few and 15.1% with significant ones; 11.7% unusable.

3.6Comparison with prior work

Conclusion & limitations

Combining self-supervised pretraining with fine-tuning on a small set of labelled patches exploits large amounts of unlabelled data while keeping the precision of targeted supervision. Aggregating detected signs at patient level, together with simultaneous 3D localization, identifies and visualizes the affected regions, supporting the detection of incidental findings. Patch-level labels are easier and less ambiguous to annotate than whole exams, and tie each label directly to the relevant image features.

The patient-level aggregation remains simple. Evaluating on larger and more diverse datasets, other imaging settings and real clinical workflows is needed to confirm robustness and generalizability.

Related links

1. Weakly-Supervised learning based Pathology Detection and Localization in 3D Chest Scans, Medical Physics, 2024.
2. Automatic Detection of Pathologies in Conventional Scanner Data, PhD thesis, INSA Lyon, 2024 — chapter 3 (source of the tables and figures on this page).
3. Tailored 3D CT contrastive pretraining to improve pulmonary pathology classification, IEEE ICSP, 2022 — the earlier contrastive pretraining study.
4. T. Chen et al., A Simple Framework for Contrastive Learning of Visual Representations (SimCLR), ICML 2020.
5. R. L. Draelos et al., Machine-learning-based multiple abnormality prediction with large-scale chest CT volumes (CT-Net), Medical Image Analysis, 2021.

BibTeX

@article{djahnine2024weakly,
  title = {Weakly-Supervised learning based Pathology Detection and Localization in 3D Chest Scans},
  author = {Djahnine, A. and Jupin-Delevaux, E. and Nempont, O. and Craighero, F. and
            Si-Mohamed, S. A. and Cottin, V. and Douek, P. and Popoff, A. and Boussel, L.},
  journal = {Medical Physics},
  volume = {51},
  number = {11},
  pages = {8272--8282},
  year = {2024}
}