---
name: pii_redaction_pack
description: Redact PII from documents and data files before sharing: validated detection for emails, SSNs, cards, IBANs, phones and context-gated birth dates, four replacement strategies (replace, mask, keyed hash, consistent pseudonym), and a verification pass that catches metadata, tracked changes, and hidden parts. Use when someone asks to redact, scrub, anonymize, de-identify, or clean a document, spreadsheet, log, or export before sending it out, or to check whether a shared file still leaks personal data.
---

# PII Redaction Pack

## Purpose

Make a file safe to hand to someone else, and prove it. The pack ships a tested stdlib-only scanner and redactor (`bin/redact.py`) plus a verification checklist (`references/verification-checklist.md`) that covers the places redaction usually fails: metadata, tracked changes, comments, embedded objects, and black-box-over-text PDF fakes.

Detection is pattern plus validation, not guesswork:
- Credit cards: 13 to 19 digits, separators allowed, must pass the Luhn check digit test. A digit run that fails Luhn is not flagged as a card.
- US SSNs: must pass SSA assignment rules. Areas 000, 666, and 900 to 999, group 00, and serial 0000 are rejected, so order numbers in SSN shape stay alone.
- IBANs: must pass the mod-97 check.
- Birth dates: flagged only next to a context word (born, dob, date of birth). A bare date is usually a deadline, not an identity.
- Names and street addresses have no reliable pattern. Supply them as a known-literal list (`--names`) and the tool finds every case-insensitive occurrence. Add house patterns (account numbers, case IDs) as regex lines with `--extra`.
- For name detection at scale in free text, Microsoft Presidio adds NER-based PERSON and LOCATION recognizers on top of the same pattern approach. Use it when the literal list is not enough, and still run the verification pass after.

## Workflow

1. **Scope first.** Ask who receives the file and what they actually need from it. Redact to that need. A support ticket needs the error and the timeline, rarely the customer's name, card, or birth date.
2. **Scan the original.** `python3 bin/redact.py scan input.txt --names names.txt` prints counts, line numbers, and masked previews as JSON. Exit code 2 means findings. Reviews show previews only, never full values, so the scan report itself does not become a leak.
3. **Pick a strategy** (table below) and redact to a new file. The tool refuses to overwrite its input.
   `python3 bin/redact.py redact input.txt -o output.txt --strategy pseudonym --names names.txt`
4. **Strip the container.** For office files, PDFs, and images, body redaction is half the job. Follow the metadata and hidden-content sections of `references/verification-checklist.md` (ExifTool plus qpdf re-linearize for PDFs, accept tracked changes and clear document properties for office files).
5. **Verify.** `python3 bin/redact.py verify output.txt --names names.txt` must exit 0. For docx, xlsx, pptx, odt, pdf, and images, `verify` also scans the container parts and returns metadata warnings with exit code 2. Then work the rest of the checklist by eye.
6. **Ship only the verified output.** Keep the original private. Keep the scan report, which holds counts and types only, with your own records.

Replacement strategies, following the options NIST SP 800-188 lists for removing direct identifiers:

| Strategy | Output looks like | Use when |
|---|---|---|
| replace | `[REDACTED:EMAIL]` | Default. Recipient needs the shape of the document, not the values. |
| mask | `***-***-0147`, card keeps last 4 | Support and billing talks where the last 4 confirm identity. |
| hash | `[HASH:EMAIL:58645683fdf9f92d]` | Joining records across files without sharing values. Keyed HMAC-SHA-256, key in `PII_HASH_KEY`. The tool refuses a bare unkeyed hash, which is easy to reverse for low-entropy values like SSNs. |
| pseudonym | `[EMAIL-1]`, `[KNOWN_LITERAL-1]` | Case notes and research text where the same person appears many times and the thread must stay readable. Consistent within one run. |

Worked example, tested end to end. Input line: `SSN 078-05-1120. Card 4111 1111 1111 1111.` The scan flags both (078-05-1120 passes SSA rules, the card passes Luhn). Pseudonym redaction gives `SSN [US_SSN-1]. Card [CREDIT_CARD-1].` A lookalike line, `Order total 1234 5678 9012 3456`, is left untouched because that number fails Luhn.

## Output Contract

- The redacted file, at a new path, never overwriting the source.
- A scan or verify report as JSON: counts by type, line numbers, masked previews, metadata warnings. No full detected values in any report.
- A short note to the user naming the strategy used, the counts removed, the verification result, and any checklist item that could not be checked (for example, a PDF that needs a tool not installed).

## Operating Rules

- Never send, upload, or share the unredacted original while "checking" something. All work happens on local copies.
- Black rectangles over text are not redaction. In a PDF, delete the underlying text or rasterize the page, then verify by extracting text and scanning it.
- Masking keeps part of a value readable by design. Use it only when the recipient has a real reason to see the last 4. Otherwise replace or pseudonymize.
- Quasi-identifiers count. ZIP code, birth date, and sex in combination can single out a person with no name present. If a data file keeps all three, say so plainly and let the owner decide.
- Hashing without a key is not protection for short values. The tool enforces a key; do not work around it with a plain SHA-256 of an SSN.
- Automated detection misses things: names written as initials, values spelled out in words, handwriting in scans. The eye pass in the checklist is mandatory, not optional polish.
- If verification still finds PII and you cannot remove it with the tools at hand, stop and report what remains and where. Never ship a file described as redacted when the verify pass failed.
- This pack reduces sharing risk. It does not by itself make a dataset de-identified under HIPAA, GDPR, or any other rule, and it never gets described that way.