suppression_list_management v1.0.0

name: suppression_list_management description: Run suppression lists for outbound email: one canonical store with address, domain, and person level entries, scoped by stream, fed by unsubscribes, complaints, and bounces, enforced by a hard pre-send check, with a full audit history. Use when setting up or cleaning up suppression, scrubbing a send list, wiring unsubscribe or feedback loop feeds, investigating why an address was or was not mailed, migrating ESPs, or auditing opt-out handling.

Suppression List Management

Purpose

A suppression list is the record of who must not be mailed, and why. It fails in a predictable way: each tool keeps its own copy, the copies drift, and a person who opted out in one system gets mailed from another. This skill runs suppression as one canonical store owned by the sender, checked before every send, fed by every opt-out source, and auditable after the fact.

The checker in bin/suppression_check.py is the working core: it holds the store, scrubs a recipient file before a send, handles hashed partner lists, and writes the audit log. Use it directly, or copy its data model into a database (schema in references/data-model-and-scopes.md).

Workflow

1. Set up one canonical store

• Create the store before the first send, not after the first complaint: python3 bin/suppression_check.py init --store suppressions.csv
• The store lives with the sender, not inside the ESP. ESP lists (SendGrid bounces, blocks, invalid, spam reports, global and group unsubscribes; Amazon SES account level list) are feeds into the canonical store, never the only copy. A provider switch then carries the history with you.
• One store serves every sending tool: sequencer, marketing platform, transactional provider, and any manual sends. If a tool cannot check the store at send time, it does not send until its list has been scrubbed against the store in step 4.
• Keep the store minimal: pattern, type, scope, reason, source, date added, expiry if any, short notes. No names, no engagement data, no free text about the person. A suppression entry only needs enough to match and to explain itself.

2. Model the three levels and the scope

• Address level is the default: one normalized email address (trim, lowercase). Do not collapse plus tags or Gmail dots unless the sending policy says so in writing; those are delivery variants, and collapsing them silently widens a suppression.
• Domain level blocks a whole domain. Reserve it for cases with a clear basis: a company that asked for no mail to any employee, a domain that does not exist, a disposable or test domain, or a partner instruction. Domain entries do real damage when added casually, so they need a note saying who asked and when.
• Person level blocks a person across addresses, using an opaque person id from the CRM (for example crm:4821), never a name in the pattern field. Use it for do not contact requests and legal objections, where the request covers the person, not one inbox. The recipient file carries the same person id so the checker can match it.
• Scope decides which mail stops. global stops every stream, including transactional. A stream scope such as marketing stops only that stream. Rules for scope:
• hard_bounce, complaint, invalid, manual_do_not_contact, and legal_objection are always global.
• unsubscribe is scoped to the stream or group the person left, unless they chose unsubscribe from all, which is global. A marketing unsubscribe must not block a password reset. A global unsubscribe must not be downgraded to one stream.
• soft_bounce_threshold is scoped to the stream that produced the bounces, unless the address also hard bounced.
• Full data model, SQL schema, and scope matrix: references/data-model-and-scopes.md.

3. Wire every source in, with a deadline for each

Each source lands in the store with its reason and source name. Process on the tightest clock that applies, not the loosest:

| Source | Reason written | Clock | |---|---|---| | One-click unsubscribe POST (RFC 8058) | unsubscribe | Immediate in code; providers require honouring within 2 days, CAN-SPAM allows 10 business days, use the 2 day outer limit only as a failure alarm | | Footer or preference centre unsubscribe | unsubscribe (or global if they picked all) | Same day; same store write as the header path | | Plain reply asking to stop | unsubscribe or manual_do_not_contact | Same business day; a reply sitting in one inbox is still an opt-out | | Spam complaint via feedback loop (ARF) or ESP webhook | complaint, global | Immediate; never mail a complainant again on any stream except legally required notices handled outside this system | | Hard bounce (permanent 5xx: unknown user, no such domain) | hard_bounce, global | After the first confirmed occurrence | | Soft bounce (temporary 4xx: mailbox full, server down) | soft_bounce_threshold only after repeated bounce days | Count days, not retries; common practice is 3 to 5 bounce days for generic temporary failures, 7 to 10 for mailbox full; a delivery or a click resets the active counter, the raw events stay logged | | Imported or partner list | partner, hashed match at check time | Scrub at import and again at send time; receive hashes (SHA-256 of the normalized address), never plaintext addresses from a partner | | Manual request (support ticket, sales call, legal) | manual_do_not_contact or legal_objection, global | Same business day, ticket id goes in notes |

Source handling rules:

• Unsubscribe headers. Marketing and subscribed mail carries List-Unsubscribe with one HTTPS URI plus List-Unsubscribe-Post: List-Unsubscribe=One-Click, both covered by the DKIM signature. The endpoint unsubscribes on POST with body List-Unsubscribe=One-Click, with no login, no confirmation page, and no redirect. A GET shows a page and changes nothing; scanners and prefetchers fire GETs. The body still carries a visible unsubscribe link.
• Complaint feeds. Feedback loops deliver ARF reports from mailbox providers; ESP webhooks deliver the same event as spamreport or complaint. Gmail is the exception: its feedback loop is aggregate only, keyed by the Feedback-ID header, so use it to find a bad campaign, not to suppress individuals. Dedupe webhook deliveries: providers retry and batch, so the write must be idempotent on address plus reason plus scope.
• Bounce feeds. Pull ESP suppressions through the API on a schedule as well as by webhook. A checker that reads only one ESP list (say bounces) and misses spam reports gives a confident wrong answer. Reconcile all streams, then write what you found into the canonical store.
• Policy failures are not address failures. Authentication failures, throttling, blocklist and reputation rejections say something about the sender, not the recipient. Log them as sending incidents. Never write a suppression entry from them.

4. Enforce with a hard pre-send check

• Every send path checks the store before dispatch. The check sits at the lowest layer that all sends pass through, so no caller can skip it. A suppressed address is dropped and the drop is logged; it is never sent and then explained.
• Run the file checker for list based sends:

python3 bin/suppression_check.py check --store suppressions.csv \
  --recipients campaign-2026-10-14.csv --stream marketing \
  --out campaign-2026-10-14.checked.csv --hash-file partner-hashes.txt

Output: one row per recipient with ALLOW or SUPPRESS, the matched pattern, match type (address, domain, person, hash), reason, and source. Send only the ALLOW rows. Add --fail-on-suppress in a pipeline to stop the job when any suppression is found, if the team prefers a human look over an automatic drop.

• Scrub at two moments: when a list is imported or built, and again right before the send. The second scrub catches opt-outs that arrived while the campaign was being prepared.
• Normalize the same way on both sides: trim and lowercase addresses and domains before storing and before checking. A case mismatch is the classic silent miss.
• Seed test: keep two seed addresses of your own, add one to the store, and run a check before any new sending tool goes live. If the seeded suppressed address comes back ALLOW, the tool is not wired to the store, whatever its settings page claims.

5. Handle removals and resubscribes with evidence

• Removing a suppression is a controlled action, never a quiet delete. The checker requires --evidence and writes the removed row into the audit log before deleting it from the active store.
• Allowed evidence by reason:
• unsubscribe, legal_objection, manual_do_not_contact: a new, dated consent or a direct written request from the person, recorded in the CRM. A salesperson's memory is not evidence.
• complaint: do not remove for marketing. If the person writes in asking to return, treat it as a new consent, scope it to the stream they named, and note the original complaint date.
• hard_bounce, invalid: an address correction (the old entry stays on the wrong address) or a successful verified delivery to the corrected address. Never remove the entry from the bad address itself.
• soft_bounce_threshold: a successful delivery resets it; record the delivery as the evidence.
• partner: removal belongs to the partner who supplied it; ask them to drop the hash.
• Resubscribe flows write a fresh consent record first, then the removal. Order matters in the audit log.

6. Audit on a schedule and after every incident

Run the audits in references/audits-and-incidents.md. Short version:

• Weekly: reconcile the canonical store against each ESP list both ways. Entries in the ESP but not the store mean a feed is broken. Entries in the store but not the ESP are normal (the store is wider) unless the ESP is the sending tool for that stream and cannot see them, which means the pre-send check is the only thing standing between that entry and a send.
• Monthly: sample 20 recent sends and trace each recipient back through the check report. Sample 20 suppression entries and confirm each has a reason, a source, and a date that match a real event.
• After any complaint from a person who says they already opted out: freeze the stream, find the opt-out event, find which check should have caught it, fix that gap, and only then resume. The worked incident in the reference file shows the trace format.

Output Contract

For a setup or cleanup job, deliver:

1. The store file (or database table) with the header from references/data-model-and-scopes.md, plus the count of entries by reason and scope.
2. A source map: each opt-out, complaint, and bounce source, where it writes, and the clock it runs on.
3. A check report for the next scheduled send, showing allowed and suppressed counts with reasons.
4. The audit log path and the last reconciliation date for each ESP list.

For a single scrub request, deliver the checked file, the suppressed rows with reasons, and one line stating which store version (file date or row count) the check ran against.

For an incident, deliver the trace: opt-out event, store write (or its absence), check report for the send in question, root cause stated as a specific broken step, and the fix applied.

Operating Rules

1. One canonical store. Any second list is a cache or a feed, and it says so in its name or header.
2. No send without a check. If the check cannot run, the send waits. A broken checker is a stop, not a bypass.
3. Complaint and hard bounce entries are global and permanent unless the removal evidence in step 5 exists. Do not expire them on a timer.
4. Unsubscribe entries do not expire. Soft bounce and partner entries may carry an expiry; when they do, the date and the reason for it are in the entry.
5. Store the minimum: no names in patterns, no notes about a person's life or business, ticket ids and dates only.
6. Never send a partner your plaintext suppression list. Share SHA-256 hashes of normalized addresses, one per line, and scrub partner lists the same way.
7. Legal clocks are outer limits, not targets: CAN-SPAM 10 business days, Gmail and Yahoo 2 days for one-click, CASL 10 business days with a link that stays valid at least 60 days. Build for immediate, alarm at the provider limit.
8. GDPR objection to marketing is absolute: stop the marketing. Keeping the minimal entry so the address is not re-imported is the correct handling, and privacy copy should say that plainly instead of promising total deletion.
9. Numbers quoted to a team (complaint rate, bounce rate) come from the provider postmaster or ESP report in front of you. Targets to run to: spam complaint rate under 0.1 percent, with 0.3 percent as the provider enforcement line; investigate any campaign over 2 percent hard bounces before sending again.
10. Every add, remove, and check writes an audit record. If it is not in the audit log, it did not happen, and an auditor should treat it that way.