test_data_management v1.0.0

name: test_data_management description: Designs and builds reliable test data: factory functions, fixtures, seeded databases, and PII-safe scrubbing pipelines. Use when a test suite needs realistic data setup, when tests are flaky from shared data, or when production data must be anonymized before reaching a test environment.

Purpose

Give an agent a complete playbook for test data: how to create it (factories vs fixtures), how to isolate it per test, how to seed databases reproducibly, and how to scrub production data so PII never reaches a test environment.

Workflow

1. Diagnose the data need. Ask: what does the failing or missing test actually require? One record? A realistic distribution? A full production-like database? Pick the lightest mechanism that satisfies it.
2. Choose the mechanism.

• Dynamic entity data (users, orders, sessions a test creates and mutates): factory function with sensible defaults and per-test overrides.
• Static reference data (country codes, currency lists, config): version-controlled fixture files (JSON/YAML).
• Production-shaped data at scale: one-way masked copy of production, never the live database, with a declarative per-column masking policy.

1. Make it deterministic. Seeded random generators, fixed timestamps, explicit IDs. No wall-clock or unseeded randomness anywhere the test asserts on.
2. Make each test own its data. Create what the test needs, assert, clean up. Rollback-wrapping transactions or teardown hooks are the default cleanup path. Never rely on rows left behind by another test.
3. Verify. Run bin/pii_scan.py over every fixture and seeded-dump before it ships. Run the suite twice in a row and in parallel; both must pass with identical results.

Output Contract

• A factory module (see references/factory-patterns.md) or fixture files in a version-controlled tests/fixtures/ directory.
• A seeding path that runs identically on every machine (documented seed order, fixed seeds).
• A PII-scrubbing policy or scan report for any data derived from production (references/pii-scrubbing.md, bin/pii_scan.py).
• Evidence: two consecutive full-suite runs, one of them parallel, all green.

Operating Rules

1. Factories over fixtures for dynamic data. Fixtures for static reference data only. A third near-identical fixture file (e.g. user-admin-inactive.json) means the factory is missing a trait; add the trait instead.
2. Minimal data, maximal signal. A search test needs one searchable record, not a full user profile with billing address and order history. Override only what the test cares about.
3. No shared mutable state. Never hard-code IDs (userId: '1'); use factory sequences or seeded UUIDs so parallel runs cannot collide.
4. Real PII never touches test environments. Generate synthetic data with Faker-style libraries, or mask production data on a one-way copy. Free-text columns, logs, and support fields get scanned too, not just the obvious name/email columns.
5. Deterministic substitution for masked data. The same source value must map to the same fake value in every table, so joins and foreign keys survive. Scrambled emails must still parse; fake card numbers must pass checksums; formats stay valid.
6. Seeded DBs are seeded once, and reproducibly. One explicit seed order, fixed seeds, checked-in migration state. For integration tests where runner and app use different databases, seed through the app's API rather than direct inserts.
7. Cleanup is part of creation. Every factory or seed helper defines its own teardown. Wrap DB tests in transactions rolled back after each test where the framework allows it.