Menu

Individual skill

metric-design

0upvotes

Use when the user has evaluation principles or a dataset but needs help choosing the right graders, designing evaluation metrics, creating LLM-as-judge prompts, combining multiple metrics into a composite score, or building an automated evaluation pipeline. Also use when the user mentions grader selection, metric design, judge prompt engineering, rubric design, evaluation pipeline code, or "how to evaluate [X] automatically." Outputs executable OpenJudge pipeline code.

skills/eval_pipeline/02-metric-design/SKILL.md

Read pinned source

Source commit eb34c3b94c908d875eda025e7863bc765c44ad76

What was checked

Matches the observed repository HEAD
Yes
Scanner rules are current
Yes
Repository scan complete
No
Skill scan complete
No
Observed behavior
Network or broad shell
Last checked
Nov 10, 2026

These are observations at the recorded commit, not a guarantee of safety. Review the source before running it.

Install this skill

Choose an agent in your Stack or copy a command below. Commands fetch the pinned commit and copy only this skill directory. Existing destinations are refused. Copying a command does not install anything.

An install command is unavailable: current rules, complete repository and skill scans, an eligible behavior level and a permitted license are required.

Repository context

Repository stars (not skill ratings)
1,086
Repository last updated
Nov 9, 2026
License
Apache-2.0

Observed agent integration: No specific integration evidence. Standard SKILL.md installation does not imply verified compatibility.

Files and excerpt

File metadata and permitted, redacted excerpts come from the recorded scan. Source links are pinned to that commit.

Browse files
Read excerpt

# Metric Design

Select, configure, and combine evaluation graders into a working pipeline. You choose
the right tool for each evaluation dimension — from zero-cost code checks to LLM judges
— and produce executable `GradingRunner` code that runs on OpenJudge.

> **Requires OpenJudge** (`pip install py-openjudge`). This skill is intentionally
> SDK-centric — grader selection, `GradingRunner`, and aggregators are OpenJudge APIs. The
> design/decision logic still applies if you use another harness; only the code does not.

## When to Activate

- User has eval dimensions/principles but doesn't know which grader type to use
- User wants to write an LLM-as-judge prompt for a specific failure mode
Back to top