Individual skill
prompt-regression-suite
Design a regression test suite that catches an LLM feature getting worse when the prompt, model, or context changes. Use when asked to stop prompt changes breaking production, set up golden tests or CI gates for an LLM feature, or test a model/prompt upgrade before shipping it. Produces a golden case set, per-case pass criteria, CI gate thresholds, and a triage protocol for failures. For designing first-time evaluation of a new feature use ai-eval-plan instead.
skills/prompt-regression-suite/SKILL.md
Source commit f7c90bc09ee05dc25cf257bdb1826f2c2552c2cb
What was checked
- Matches the observed repository HEAD
- Yes
- Scanner rules are current
- Yes
- Repository scan complete
- No
- Skill scan complete
- No
- Observed behavior
- Network or broad shell
- Last checked
- Nov 9, 2026
These are observations at the recorded commit, not a guarantee of safety. Review the source before running it.
Install this skill
Choose an agent in your Stack or copy a command below. Commands fetch the pinned commit and copy only this skill directory. Existing destinations are refused. Copying a command does not install anything.
An install command is unavailable: current rules, complete repository and skill scans, an eligible behavior level and a permitted license are required.
Repository context
- Repository stars (not skill ratings)
- 1,538
- Repository last updated
- Nov 9, 2026
- License
- MIT
Observed agent integration: No specific integration evidence. Standard SKILL.md installation does not imply verified compatibility.
- product-management Repository topic
- productivity Repository topic
- skills Repository topic
- claude-skills Repository topic
- agent-skills Repository topic
- chatgpt Repository topic
- claude-code Repository topic
- cursor Repository topic
- gemini Repository topic
- prompt-engineering Repository topic
- design Derived category
- testing Derived category
Files and excerpt
File metadata and permitted, redacted excerpts come from the recorded scan. Source links are pinned to that commit.
Browse files
SKILL.mdcore · 5,299 bytes
Read excerpt
# Prompt Regression Suite Skill
Every prompt tweak, model upgrade, and context change is a deploy. This skill designs the suite that runs on each one and answers a single question: *did anything that used to work stop working?*
## What This Skill Produces
- A **golden case set**: curated inputs with per-case pass criteria
- **Scoring methods** per case class (exact, rubric-judge, property checks)
- **CI gate thresholds** — what blocks a merge vs. what warns
- A **failure triage protocol** — flaky vs. regressed vs. golden-set-wrong
## Required Inputs
Ask for (if not already provided):