contract_testing v1.0.0

name: "contract_testing" description: "Consumer driven contract testing for APIs: writing consumer expectations, publishing and verifying contracts in CI, checking provider changes against every consumer before deploy, and handling contract versioning when an API evolves. Triggers when a provider change might break consumers, microservices need integration safety without end to end tests, someone mentions Pact, a Pact Broker, can-i-deploy, or a field rename or type change ships between services."

Contract Testing

Purpose

Catch a provider change that breaks a real consumer in the provider CI run, minutes after the change, instead of in production or in a shared staging environment. Each consumer records the exact requests it sends and the exact parts of the response it uses. That record is the contract. The provider replays every consumer contract against its real code on every change, and a deploy gate asks whether the version about to ship is compatible with everything already deployed.

Contract testing is not a replacement for unit tests, and it is not end to end testing. It covers one boundary at a time: one consumer, one provider, the interactions between them. Schema validation against an OpenAPI document is a useful companion, not a substitute. A schema check proves a response matches what the provider promised in general. A contract proves it still matches what a named consumer actually depends on, including value shape, such as a status string that quietly changed case.

Workflow

1. Pick the boundary that hurts. Start with the service boundary that caused the most recent integration incident or the one teams argue about most. One consumer and one provider, working end to end, beats ten half wired pairs.
2. Write the consumer test through the real client. The consumer test calls its own API client code against a mock provider, never a raw HTTP call typed inside the test. A test that bypasses the client proves nothing about the client. With Pact this is given for the provider state, uponReceiving for the interaction, withRequest, and willRespondWith, and running the test writes the contract JSON.
3. Pin only what the consumer uses, with matchers. Use type matchers for values, such as like or integer, regex matchers for formats and value sets, and eachLike with a minimum for arrays. Pin a literal value only when the literal is the behaviour, such as an error code the client branches on. Literal pinning of names, dates, and amounts makes contracts fail on harmless data changes, and teams delete brittle contracts.
4. Name provider states like scenarios, and parameterise them. A state reads "order 42 exists" for a person, and in Pact specification v3 and later it carries params, such as an id, so the provider handler can build exactly that data. Keep state names stable across consumers that mean the same setup.
5. Publish the contract from consumer CI. Publish on every merge build with the commit SHA as the consumer version and the branch recorded. Share through a Pact Broker, or PactFlow for the hosted version, so contracts and verification results live in one queryable place. A contract file passed around in chat or left in a build artifact is not a system.
6. Verify from the provider, with state handlers. The provider starts its real code, stubs its own downstream dependencies, and replays each interaction. Before each interaction the verifier sends the provider state to the state handler, which seeds the data the interaction assumes, and a teardown call cleans up after. Publish the verification result back to the Broker with the provider version set to the commit SHA. Unpublished results cannot gate anything.
7. Turn on pending and work in progress pacts. Pending means a contract with changed content gets verified and reported without failing the provider build, until it has passed once. Work in progress means contracts from feature branches the provider was never configured for also get verified, in pending mode. Together they give consumers feedback on new expectations without breaking the provider build over a change nobody agreed to yet.
8. Gate deploys with can-i-deploy. Before any deploy, either side runs pact-broker can-i-deploy --pacticipant <name> --version <commit SHA> --to-environment production and a non-zero exit stops the deploy. Record each successful deployment with record-deployment so the gate knows what is live. Full CI wiring lives in references/ci-and-broker.md.
9. Evolve by adding, then removing. Add the new field beside the old one, update consumers, and remove the old field only when no contract still pins it and no deployed consumer version needs it. Run bin/contract_check.py diff on any contract change in review to name breaking changes before CI does. The full rule set for what counts as breaking lives in references/versioning-and-evolution.md.
10. Run the bundled two service example before adopting anything. examples/provider.py is a working orders provider, examples/consumer_client.py is a checkout consumer that depends on four response properties, and examples/contracts/orders.pact.json is its contract. Verify the good provider, then verify the broken one, and watch the same contract catch a field rename and a status casing change that both pass normal unit tests.

Worked example: the breaking change catch

Start both providers and run the verifier against each one:

python3 examples/provider.py --port 8081
python3 examples/provider.py --port 8082 --broken
python3 bin/contract_check.py verify --contract examples/contracts/orders.pact.json --base-url http://127.0.0.1:8081 --state-url http://127.0.0.1:8081/_pact/state
python3 bin/contract_check.py verify --contract examples/contracts/orders.pact.json --base-url http://127.0.0.1:8082 --state-url http://127.0.0.1:8082/_pact/state
python3 bin/contract_check.py diff --old examples/contracts/orders.pact.json --new examples/contracts/orders.pact.v2-breaking.json
python3 examples/consumer_test.py

Verified output from a real run of this code: the good provider passes 2 of 2 interactions and exits 0. The broken provider, which renames total to totalAmount and returns status PAID instead of paid, fails the first interaction with "field missing in provider response" for total and a regex mismatch on status, passes 1 of 2, and exits 1. The consumer client run against the broken provider crashes with a KeyError on total, which is the production failure the contract exists to move into CI. The diff reports the removed total field as breaking and exits 1.

The bundled verifier is a teaching and smoke test harness written in stdlib Python. It implements the same replay, provider state, matcher, and diff ideas with no install step, so a small team or a repo without Pact yet can run a real check today. Production systems with several consumers should use Pact itself, which has libraries for the major languages and supports HTTP and message contracts, plus the Broker for versioned storage and the can-i-deploy gate.

Output Contract

• One contract per consumer and provider pair, generated by running the consumer test through the real client, with matchers instead of literal values wherever the exact value is not the behaviour under test.
• Provider verification that runs on every provider pull request, covers the main branch contracts and every deployed consumer version, publishes its result to the Broker, and has a state handler for every provider state named in any contract.
• A can-i-deploy gate in both pipelines and a record-deployment step after every successful deploy.
• For any proposed API change, a written verdict: breaking or not breaking, which named consumers it affects, and the add then remove plan if it is breaking. No change ships on "it should be fine".

Operating Rules

1. Consumers own the contract, providers own verification. The consumer decides what it needs. The provider proves it still provides it. A provider team editing a consumer contract to make verification pass has destroyed the signal. Negotiate the change, then let the consumer team change its own contract.
2. A consumer with no contract is invisible. Verification only protects consumers that publish. When a new consumer appears, its first contract is part of done, not a follow up task.
3. Never bypass the gate quietly. If can-i-deploy says no, the answer is to fix the incompatibility or ship the additive step first. A forced deploy that breaks a consumer gets written up like any other incident.
4. Contracts test interactions, not business logic. If a test needs to prove a total is calculated correctly, that is a provider unit test. The contract only proves the total arrives, as a number, under the agreed name. Mixing the two makes contracts slow and brittle.
5. Stub the provider dependencies, never the provider. Verification runs real provider code. Its database is seeded by state handlers and its downstream calls are stubbed. Mocking the provider itself turns verification into a test of the mock.
6. Keep verification fast enough to run on every pull request. If verification takes long enough that people batch changes or skip it, the coverage is fiction. Profile state setup first, it is the usual cost.
7. For message based systems, contract the payload. Pact message contracts cover queues and event streams the same way: the consumer defines the message shape it can handle, the provider verifies it can produce it, and the transport stays out of the contract.
8. Know when a lighter tool fits better. A single team with one OpenAPI document and no separate consumers may get enough from schema testing plus a breaking change diff on the spec. Say so instead of selling them a Broker. Contract testing earns its cost when separate consumers, separate teams, or separate deploy schedules exist.