auditing-deidentification-runs

Auditing de-identification runs

deidentify(..., audit=True) returns an AuditReport instead of the rewritten text: a deterministic, PHI-free record of every redaction decision — offsets, label, detector confidence and threshold, the action taken, content hashes, model provenance, and projected residual risk. Sign it to make it tamper-evident, retain it as compliance evidence, and verify it later without ever touching the original text. Runs on-device.

When to use this skill

Use it whenever a de-identification needs to be defensible: regulatory retention, internal review, reproducibility checks, or proving to an auditor that identifiers were detected and handled — all without storing or exposing the PHI itself.

Quick start

import openmed

note = "Patient John Doe (MRN 1234567) seen 2024-03-02. SSN 123-45-6789."

# Returns an AuditReport, NOT a DeidentificationResult, when audit=True.
report = openmed.deidentify(note, policy="hipaa_safe_harbor", audit=True)

# Make it tamper-evident with a release HMAC key (keep the key in a vault).
report.sign(b"my-release-hmac-key", key_id="release-2026")

# Persist the no-PHI report (no plaintext identifiers inside).
import json
with open("deid_audit.json", "w") as fh:
    json.dump(report.to_dict(), fh, indent=2)

# Later: verify integrity. Optionally bind to the exact texts via their hashes.
ok = report.verify(b"my-release-hmac-key", original_text=note)
assert ok

What an AuditReport contains

AuditReport (from openmed.core.audit) carries no plaintext PHI. Key fields:

Field	Meaning
policy	policy profile name in effect
resolved_profile	the concrete settings applied (method, model, thresholds, language, sweep flags)
detectors	list[DetectorInfo] — provenance of each detector
safety_sweep	structured-ID sweep metadata (patterns version, source)
spans	list[AuditSpan] — one entry per detected identifier
thresholds	per-label confidence thresholds used
residual_risk	projected leakage / re-identification risk summary
openmed_version	library version that produced the report
manifest_hash	hash of the model manifest used
document_length	character length of the input
input_hash	sha256: hash of the original text
deidentified_text_hash	sha256: hash of the de-identified output
repro_hash	deterministic hash over the canonical payload
signature	AuditSignature once .sign() is called (else None)

AuditSpan (per identifier, no plaintext): start, end, label, canonical_label, sources, confidence, threshold, action, surrogate, text_hash (hash of the span text), plus evidence and a redacted context.

DetectorInfo: source, model_id, model_format, commit, metadata.

AuditSignature: key_id, algorithm (HMAC-SHA256), value (hex digest).

Why audits store offsets + hashes + provenance, never plaintext

The whole point of de-identification is to not retain identifiers. An audit log full of raw names and MRNs would itself be a PHI breach. So OpenMed records:

• offsets (start/end) — where an identifier was, not what it was;
• hashes (input_hash, deidentified_text_hash, per-span text_hash) — prove what was processed without revealing it;
• provenance (detectors, manifest_hash, openmed_version) — which model/version made each call, for reproducibility;
• residual risk (residual_risk) — projected leakage so a reviewer can judge whether the output is safe to release.

This lets an auditor confirm coverage and integrity while the report stays shareable.

Workflow

1. Run with audit=True and a policy= to get an AuditReport.
2. Inspect residual_risk before releasing the output — non-zero projected leakage means review (cross-check with auditing-safe-harbor-checklist).
3. Sign with report.sign(key, key_id=...) using a secret release HMAC key stored in a vault (never in the repo).
4. Persist report.to_dict() (JSON). For a reviewer view without full context, use report.export_review_bundle().
5. Verify on retrieval with report.verify(key). Pass original_text= / deidentified_text= to also confirm the texts match the recorded hashes.
6. Retain per your retention schedule alongside the de-identified output (but keep any reversible mapping separate and encrypted).

Hand-off to / from OpenMed

• Produce the de-id: deidentifying-clinical-text (deidentify); add audit=True to get the report from the same call.
• Coverage check: auditing-safe-harbor-checklist maps span labels to the 18 HIPAA identifier categories and flags gaps.
• Policies: configuring-privacy-policies — resolved_profile reflects the chosen policy=.
• Other surfaces: MCP openmed_deidentify and REST POST /pii/deidentify drive the same engine; audit output is available through them.

Edge cases & gotchas

• audit=True changes the return type to AuditReport. You get the report, not .deidentified_text. Run a separate normal call if you also need the text.
• An unsigned report is not tamper-evident. .sign() is a deliberate second step; signature is None until you call it. Empty/None keys are rejected.
• verify needs the same key. It checks the HMAC and the repro_hash; optionally it re-hashes texts you pass to confirm they are the audited ones.
• Never put plaintext PHI back into the report. Do not stuff raw identifiers into metadata/evidence; the design is hash-and-offset only.
• Store the signing key in a secret manager, not in source or the report.
• Residual risk is advisory, not a pass/verdict. Combine it with the Safe Harbor checklist and human review for release decisions.

Standards & references

• HIPAA de-identification & documentation, 45 CFR 164.514(b): https://www.hhs.gov/hipaa/for-professionals/privacy/special-topics/de-identification/index.html
• HMAC-SHA256: RFC 2104 / FIPS 198-1.
• OpenMed source: openmed/core/audit.py (AuditReport, AuditSpan, AuditSignature, DetectorInfo, sign, verify), openmed/core/pii.py (deidentify(audit=True)).