Why the necessity for scalable picture de-identification
Medical pictures, resembling X-rays and MRIs, apart from aiding in prognosis, therapy planning, and illness monitoring, are more and more getting used past particular person affected person care to tell broader medical analysis, public well being coverage, and the event of recent AI-powered diagnostic instruments. This secondary use of medical data, whereas immensely helpful, must endure de-identification of protected well being data (PHI) to safeguard affected person privateness and adjust to laws like HIPAA.
The rising scale of medical picture datasets necessitates dependable and environment friendly de-identification strategies, guaranteeing that the pictures could be safely and ethically used to advance medical science. To this finish, we current the Pixels Resolution Accelerator with a Spark ML Pipeline leveraging Imaginative and prescient Language Fashions (VLM) in parallel to de-identify medical pictures within the extensively used format, Digital Imaging and Communications in Drugs (DICOM).
A DICOM file accommodates each pictures and metadata textual content (learn extra right here). Right here, we deal with our new characteristic de-identifying pictures. It’s price noting that Pixels, our DICOM toolkit, additionally de-identifies metadata along with scalable DICOM ingestion, segmentation, all inside an internet software.
How one can de-identify PHI burned in DICOM pictures
After putting in the Pixels python bundle, run the DicomPhiPipeline as such:
It reads in a DICOM file path in a column in a Spark dataframe and outputs 2 columns:
- a response from a VLM (laid out in endpoint)
- a DICOM file path with PHI masked
As a part of DicomPhiPipeline, redaction is carried out utilizing EasyOCR. Redaction could be carried out independently of VLM PHI detection (redact_even_if_undetected=True) or carried out conditionally on VLM PHI detection (redact_even_if_undetected=False). We advocate the latter as EasyOCR tends to over-redact non-PHI. Conditioning on pictures the VLM has detected as PHI-positive, EasyOCR can be much less more likely to redact the non-PHI pictures.
Comparability with different PHI detection strategies
The competitors
We examined Pixels’ picture PHI detection pipeline with a business vendor and a extensively used open supply answer, Presidio. Each the seller and Presidio used OCR to first extract the textual content from the pictures after which apply a language mannequin to categorise if the textual content was PHI or not. The built-in OCR additionally segmented delicate textual content and utilized a fill masks inside these bounding containers.
Moreover, we in contrast a number of VLMs: GPT-4o, Claude 3.7 Sonnet, and open-source Llama 4 Maverick.
Datasets
The comparability was executed on public DICOM datasets, MIDI-B the place we downsampled to 70 pictures to create a balanced dataset with roughly equal variety of pictures with PHI and with out.
Outcomes
| Job: PHI detection in DICOM pictures | MIDI-B (70) | ||||
|---|---|---|---|---|---|
| Resolution | Value Estimates per 100k pictures | Recall | Precision | Specificity | NPV |
| ISV (business) | $4,400 per 30 days pay as you go | 1.0 | 0.71 | 0.93 | 1.0 |
| Presidio (OSS) | $0 | 0.7 | 0.7 | 0.95 | 0.95 |
| Claude 3.7 Sonnet | $270 | 1.0 | 1.0 | 1.0 | 1.0 |
| GPT-4o | $150 | 1.0 | 1.0 | 1.0 | 1.0 |
| Llama 4 Maverick (OSS) | $45 | 1.0 | 0.91 | 0.98 | 1.0 |
Each Claude 3.7 Sonnet and GPT-4o had excellent PHI detection efficiency. Llama 4 Maverick had 100% recall however 91% precision because it typically mis-identifies non-PHI textual content on the picture as PHI. Nonetheless, Llama 4-Maverick nonetheless gives good efficiency particularly for customers who lean in the direction of over-redaction to keep away from lacking any PHI. In such a case, it has zero false omission charge of PHI (i.e. NPV near 1) and recall of 1 so it might be a superb stability between efficiency and value.
In our checks, we used Presidio and the business answer out-of-the-box with default settings. We observed that efficiency when it comes to each accuracy and velocity was extremely depending on the OCR alternative. It’s probably their efficiency could possibly be improved with alternate options resembling Azure Doc Intelligence.
Why it really works
We surveyed the literature on de-identifying burn-in textual content on medical pictures and realized from the reported success of utilizing OCR, LLMs (e.g. BERT, Bi-LSTM, GPT) and/or VLMs. Our determination to make use of VLM to detect PHI and EasyOCR to detect textual content bounding containers was guided by the success reported by Truong et al. 2025.
VLM change conventional OCR poor at textual content recognition and sometimes introduce typos
In most de-identification strategies reported, OCR was typically used as step one to extract textual content from pictures enter right into a LLM. Nonetheless, we noticed that OCR instruments like tessaract and EasyOCR have been typically poor and sluggish at textual content recognition (i.e. studying), typically mis-reading sure characters and inadvertently introducing typos and compromising downstream PHI detection. To mitigate this, we used a VLM to learn burn-in textual content and classify if the textual content was PHI; the VLMs have been surprisingly good at this.
EasyOCR to detect bounding containers for redaction when VLMs can’t alter pictures
Nonetheless, VLMs can’t output redacted pictures. Thus, we used OCR to do what it did finest, i.e. detect textual content, to offer the bounding field coordinates for subsequent masking. It’s price noting that though there have been latest makes an attempt to fine-tune a VLM to output bounding field coordinates Chen et al. 2025, we opted for an easier answer assembling off-the-shelf instruments (VLM, EasyOCR) as an alternative.
Spark parallelism for production-grade scalability
Whereas Databricks had a batch inferencing functionality with LLMs (ai_functions), it at the moment lacks assist for VLMs. As such, we applied a scalable model for VLM and EasyOCR utilizing Pandas UDF. Working with a big pharmaceutical buyer, Spark parallelism sped up their de-identification course of from 105 minutes to six minutes for a trial run of 1000 DICOM frames! Scaling as much as their full workload of 100,000 DICOM frames, the velocity up and value financial savings have been vital.
Abstract
Given the ability, ease and economics of VLMs as demonstrated by the Pixels 2.0 answer accelerator add-ons, it’s not solely possible however prudent to guard your crucial medical research and associated picture research with scalable PHI detection.
Whereas Pixels is designed for DICOM recordsdata, we discovered our prospects adapting it for different picture codecs in JPEG, Entire Slide Pictures, SVS and so forth.
The updates are posted to our github repo so now is an effective time to replace or check out the Databricks Pixels 2.0 answer accelerator. Attain out to your Databricks account crew to debate your imaging information processing and AI/ML use instances. The authors could be comfortable to listen to from you over LinkedIn if we haven’t already been launched.