load_testing_basics v1.0.0

name: load_testing_basics description: "Design and run load tests for a web service: pick the test type, build realistic scenarios, run with k6, read latency percentiles, and pinpoint the real bottleneck. Triggers on 'load test', 'performance test', 'stress test', 'soak test', 'spike test', 'find the bottleneck', or requests to validate a service before launch." metadata: { "includeInPrompt": true }

Load Testing Basics

Purpose

Turn "will it hold up?" into a real test with a pass/fail answer: define performance goals first, model realistic traffic, run a staged load, read the numbers honestly, and find the component that actually breaks before launch does.

Workflow

1. Set measurable goals (before anything else)

Write SLOs as numbers, not adjectives. "Fast" is not a goal.

• p95 < <budget> and p99 < <budget> per endpoint (from business requirements, not industry averages)
• Error rate: under 1% (stricter for money paths)
• Sustained throughput: expected RPS at peak, plus the stress target above it
• Put the goals in the test script's threshold block so the tool itself decides pass/fail.

2. Choose the test type

See references/patterns.md for the full table. The three most common:

• Load: ramp to the expected peak, hold, verify goals. The default pre-launch test.
• Stress: push past expected peak to find the breaking point.
• Soak: hold normal load for hours to catch memory leaks and slow degradation. Start with a smoke run (1-2 min, 1-5 VUs) to prove the script and endpoints work before burning time on long runs.

3. Model realistic traffic

A load test is only as honest as its scenario. Include:

• Endpoint ratios matching production (the writes and DB-heavy paths, not just cheap GETs)
• Method mix, payload sizes, auth vs anonymous, cacheable vs not
• Think time between requests (sleep()) so VUs behave like people, not a flood
• Data volume like production: queries fast on 1k rows can crawl at 10M
• Warm up the system (JIT, caches, connection pools) before measuring

4. Pick the load pattern

Load pattern goes in the script's stages. Common ones:

• Ramp-hold-down: [{2m, 50}, {5m, 50}, {2m, 0}] (the standard load test)
• Step: hold each level a few minutes, shows where degradation starts
• Spike: jump 10x for seconds, drop back, tests recovery Full library with k6 code in references/patterns.md. Prefer an open workload model (arrival-rate executors in k6) for capacity tests: traffic arrives at its own rate regardless of response time. Closed VU models slow their arrival rate when the system slows down, which can hide degradation.

5. Run it

Use the template in assets/load-test.template.js:

bash
k6 run --out json=results.json load-test.js

Set the shell timeout longer than the total stage duration. Never point load at third-party endpoints or production without explicit permission and a kill switch.

6. Read the results

Read percentiles, not averages. A p50 of 100ms with a p99 of 8s means 1 in 100 requests is terrible.

• Latency climbs with load while CPU is pinned: compute-bound; scale or optimize the hot path
• Latency flat but errors climb from the start: capacity wall (thread pool, DB connection pool, rate limit); check pool usage and queue time
• p99 spikes while p95 is fine: fan-out calls or GC pauses; profile the tail path
• Memory grows over hours under soak: leak; two heap snapshots under steady load and diff the retained set
• Throughput flatlines while the load generator's CPU is maxed: the bottleneck is your own tool. Distribute the load.

7. Find the bottleneck (report, then fix)

Correlate the failing metric with system metrics, in this order:

1. App: threads/workers saturated? queue time rising?
2. DB: slow queries, missing indexes, pool exhaustion, locks
3. Downstream: external API latency, 429s, retry amplification
4. Infra: CPU throttled (limits on K8s), memory OOMs, disk I/O, network Build a prediction matrix BEFORE the test (component, expected load, known limit, predicted risk, metric to watch); the matrix turns a surprise failure into a confirmed-or-rejected hypothesis. Template in references/patterns.md.

Output Contract

Every run reports:

• The goals it tested (numbers), the pattern used, total duration
• Measured p50/p95/p99 per endpoint, error rate, achieved throughput
• PASS or FAIL against the encoded thresholds, with the threshold that broke
• The identified bottleneck with supporting evidence (metric + component), not a guess
• Next step: what to fix or what follow-up test to run

Operating Rules

1. Verify the script with a smoke run before any long test.
2. Every factual claim in a report (limits, defaults, thresholds) comes from the run output or a cited source. Unverifiable = cut it or frame as judgment.
3. Change one thing between runs and keep the environment stable, or the comparison means nothing.
4. Encode thresholds in the script. Do not eyeball pass/fail from a summary.
5. Test only systems you own. Production testing needs written permission and off-peak hours.