Individual skill
model-evaluation-report
Produces a rigorous, honest evaluation report for an ML model - real baselines, business-matched metrics with confidence intervals, slice-level error analysis, calibration checks, and a go/no-go recommendation. Use when someone asks "is this model good enough to ship", "write up the eval for this model", "did we actually beat the old model", or before any model promotion, stakeholder demo, or retraining decision. Do NOT use for evaluating LLM or prompt outputs - use llm-evaluation instead; for analyzing randomized product experiments use ab-test-analyzer; for documenting a shipped model for consumers use model-card-writer.
skills/model-evaluation-report/SKILL.md
Source commit 97a13607d669157a4a16d5ba79b77f208319f7b9
What was checked
- Matches the observed repository HEAD
- Yes
- Scanner rules are current
- No
- Repository scan complete
- No
- Skill scan complete
- Yes
- Observed behavior
- Not checked yet
- Last checked
- Nov 6, 2026
These are observations at the recorded commit, not a guarantee of safety. Review the source before running it.
Install this skill
Choose an agent in your Stack or copy a command below. Commands fetch the pinned commit and copy only this skill directory. Existing destinations are refused. Copying a command does not install anything.
An install command is unavailable: current rules, complete repository and skill scans, an eligible behavior level and a permitted license are required.
Repository context
- Repository stars (not skill ratings)
- 24
- Repository last updated
- Oct 13, 2026
- License
- MIT
Observed agent integration: No specific integration evidence. Standard SKILL.md installation does not imply verified compatibility.
- claude Repository topic
- claude-skills Repository topic
- skill-me Repository topic
- testing Derived category
Files and excerpt
File metadata and permitted, redacted excerpts come from the recorded scan. Source links are pinned to that commit.
Browse files
SKILL.mdcore · 7,549 bytes
Read excerpt
# Model Evaluation Report
A model that beats a naive baseline on aggregate accuracy can still be worse than useless in production. Honest evaluation means choosing the right metric before looking at results, comparing against baselines that actually matter, and finding the slices where the model fails - because those slices are where the incident reports will come from.
## Operating procedure
Order matters: the primary metric and baselines are chosen before results are examined, or the report is p-hacked by construction.
### Step 1: Gather inputs
Collect from the requester before computing anything; label unknowns as guesses.
1. Task type: classification, regression, or ranking.
2. The business decision the model drives, and the asymmetry of errors (is a false positive or a false negative more expensive, and roughly by how much).