security-regression-tests

• SKILL.mdcore · 6,168 bytes
• SKILL.mdcore · 6,168 bytes
• metadata.jsonasset · 1,233 bytes
• references/test-by-class.mddoc · 5,810 bytes
• references/where-verification-runs.mddoc · 2,175 bytes

<!-- Native skill bundle for Claude Code. Generated by `secure-vibe dev regenerate`. -->
<!-- Do not edit by hand; the source of truth is skills/security-regression-tests/SKILL.md. -->

# Security Regression Tests

Turning a confirmed and fixed finding into a permanent guard: proving the test fails without the fix, asserting the effect and not only the status code, seeding the two principals an authorization test needs, making a timing or out-of-band proof deterministic enough for CI, and keeping the test in a job that can actually block. Use once a security finding has been confirmed and fixed, or when a security test is failing, skipped, or about to be deleted.

## ALWAYS

- **Prove the test fails without the fix.** Revert the patch, or mutate the guard, and confirm the new test goes red — then restore. A test written after the fix can be vacuously green: wrong route, wrong fixture user, wrong assertion target, a typo in the payload. It will stay green forever and nobody will learn anything from it. This is the one step that distinguishes a guard from a decoration, and it is the one most often skipped.
- Assert **both directions**: the attack input now yields the secure outcome, *and* a legitimate input still succeeds. A deny-only test passes equally well against an endpoint that is broken for everyone, so it will hold the line and tell you nothing when someone disables the feature entirely.
- Assert the **effect**, not only the status code. A `403` proves the request was refused; it does not prove the write did not land somewhere else, that the record was not returned in a different shape, or that a queue message was not emitted. Check the state the attack was trying to reach.
- Seed the **fixtures the assertion depends on** before writing it. A cross-tenant test needs two principals and two owned resources, deterministically created — if both test users end up in the same tenant, the deny assertion is untestable and passes for the wrong reason.
- Make it **deterministic and offline**. Freeze the clock for expiry and TTL assertions, stub the HTTP client, dialer or resolver rather than reaching the network, fix random seeds, and avoid sleep-based timing assertions and state shared between tests. Note that a canonical SSRF payload such as `http://169.254.169.254/latest/meta-data/` is a *live, reachable* address on a cloud CI runner, so the naive test both leaves the sandbox and returns a different verdict on a laptop.