api_test_mode_design v1.0.0

name: api_test_mode_design description: "Designing the test and sandbox mode integrators build against: isolation from live data, test keys and magic values that trigger each outcome, deterministic events, data reset, rate limits that match live in shape, and the path from test to live with no code changes beyond keys. Use when building or auditing an API test mode, sandbox, or test credentials system, or when an integration works in test but breaks in live."

API Test Mode Design

Purpose

Build a test mode an integrator can trust. The test is simple: if code passes in test mode, it should pass in live mode for the same inputs, except for real money, real people, and real side effects. Everything else, auth, validation, errors, events, limits, and timing rules, should behave the same.

This skill ships three working pieces: a test mode checklist in references/test-mode-checklist.md, a magic value table template in references/magic-value-table.md, and a worked go live check in references/go-live-check.md, plus bin/check_go_live.py to scan a codebase for leftover test keys and test endpoints before launch.

Workflow

1. Pick the isolation model and write it down. Choose logical isolation (one stack, test keys route to separate tables or an is_test flag) or physical isolation (separate database, queues, and workers for test). Physical isolation is the default for payments, health, messaging, or anything that can contact a real person. If you pick logical isolation, list the leak paths you will guard: live reports must filter test rows, test mode must never send a real email, SMS, charge, or shipment, and a single query bug must not be able to mix the two datasets in a response.

1. Make the mode visible in the credential. Issue separate test and live keys with the mode in the prefix, following the pattern Stripe uses: pk_test_ and sk_test_ for test, pk_live_ and sk_live_ for live, with restricted variants scoped per resource. A test key must never read or write a live object, and a live key must never see a test object. Return a clear error on a mode mismatch instead of an empty result, because an empty result reads as no data and hides the bug for days.

1. Mirror production behavior, not a simplified version of it. Test mode runs the same code, the same validation, the same error envelope, the same auth flow, and the same token lifespan as live. If live tokens expire in 60 minutes, test tokens expire in 60 minutes. If live requires a second factor, a signature, or mutual TLS, test requires it too. Keep a written differences list, and keep it short. Every entry on that list is a place an integrator will be surprised in production, so each one needs a reason and an owner.

1. Give every outcome a magic value. For each success and each failure an integrator must handle, reserve one deterministic input that always triggers it: a card number, phone number, amount, email, or header. Stripe reserves 4242424242424242 for success, 4000000000000002 for a generic decline, and 4000000000009995 for insufficient funds. Twilio reserves +15005550006 for a valid and available number, +15005550001 for an invalid number, and +15005550000 for an unavailable number. Fill in references/magic-value-table.md for your API before you publish the docs. A magic value must work with test credentials only, must never charge or contact anyone, and must return the same error code live callers will see for the real version of that failure.

1. Make time and events controllable. Long cycles cannot be tested in real time. Provide a test clock or simulation control that freezes time for test objects and advances it on demand, the way Stripe test clocks advance a subscription through trial end, renewal, and failed payment, firing the same webhooks live would fire. Provide an event trigger for each webhook type, a delivery log that shows the payload, the target, and the response status, and a replay button or endpoint for a single delivery. State the retry schedule for test and live, and flag any place they differ.

1. Provide reset and seed, not just an empty box. Add a reset that one person can run without a support ticket: a dashboard button and a meta endpoint such as POST /sandbox/reset that CI can call. Reset returns the account to a known state in seconds. Seed new test accounts with a small golden dataset, a few customers, one completed order, one failed payment, one refund, so the first list call shows something real. Generate that data. Never copy production data into test, not even scrubbed by hand.

1. Match the limit contract. Test mode returns the same rate limit headers, the same 429 shape, and the same Retry-After behavior as live. The quota number may be lower in test to control abuse, but if it is, publish both numbers side by side. An integrator who never sees a 429 in test will not write backoff code, and will learn about limits from their first production incident.

1. Make go live a key swap, then prove it. The only change from test to live is the key and, if you use one, the base URL. No code branch on mode inside business logic beyond the data routing layer. Before launch, run the worked check in references/go-live-check.md and run bin/check_go_live.py over the deploy artifact. Recreate any objects the integration depends on, products, prices, plans, webhook endpoints, and templates, in live mode, because test objects do not carry over. Verify one small real transaction and one real webhook delivery end to end.

Output Contract

• An isolation decision in writing, with the leak paths for logical isolation or the drift controls for physical isolation.
• A key scheme with the mode in the prefix, separate test and live credentials, and a hard wall between test and live objects.
• A completed magic value table covering every success, decline, validation failure, auth failure, limit, and async outcome the API can produce, each with its input, expected status, error code, and resulting event.
• Test controls for time, events, reset, and seed data, each callable from the dashboard and, where CI needs it, from an API.
• A differences list between test and live. An empty list is the goal. Every remaining entry has a reason, an owner, and a note in the public docs.
• A go live run using references/go-live-check.md with evidence: scanner output, the list of live objects recreated, and the result of one real transaction plus one real webhook delivery.

Operating Rules

1. No real side effects in test mode, ever. No real charges, emails to real addresses, SMS to real numbers, shipments, or calls to a live third party in its live mode. Route test traffic to simulators or to the third party in its own test mode, and prove it with a test that uses a real looking address and confirms nothing arrived.

1. Deterministic beats realistic. The same magic input gives the same result every run, in the same order, with the same error code. No random failures, random latency, or random data in the path an automated test asserts on. Put realism in the seed data, not in the outcomes.

1. Validation runs before the magic shortcut. A request with a magic value still passes auth, schema validation, and permission checks first, the way Twilio test credentials still validate input as if it were live. A magic value that bypasses validation teaches integrators code that fails in live.

1. Document the limits of test mode in the open. If test mode cannot do something live can, say so on the testing page, next to the magic values. Twilio states plainly that SMS and calls made with test credentials do not trigger status callbacks. A hidden gap costs more trust than a listed one.

1. Test mode is a product surface, not a staging leftover. It gets the same deploy pipeline as live, deployed first, with contract tests against the same OpenAPI document run against both modes. If a change ships to live that has not shipped to test, that is a release bug.

1. Never let test data age into clutter. Reset is cheap, retention is short, and test PII entered by integrators is deleted on a published schedule. A test account with three years of debris fails in ways no one can reproduce.

1. Reject designs that fail the swap test. If going live needs a code change, a config rewrite beyond keys and base URL, a new webhook URL shape, or a different SDK call, the test mode is not finished. Fix the mode, not the integrator's launch day.

References

• references/test-mode-checklist.md for the full build and audit checklist, grouped by isolation, keys, parity, magic values, events, reset, limits, and go live.
• references/magic-value-table.md for the fill in template, with worked rows from Stripe and Twilio.
• references/go-live-check.md for a worked go live run, step by step, with a filled example.