Individual skill
eval-engineering
Inspect an agent repository and optional traces, interview the user, write reviewed Task Specs, build and audit Harbor tasks, and bootstrap reusable project World Knowledge Skills. Use for agent evals, benchmark design, Task generation, controlled Environments, synthetic data, Verifiers, Harbor runs, calibration, or continuous benchmark maintenance.
config/skills/eval-engineering/SKILL.md
Source commit 126c384d8552dcb476b226c244172186cc3f4328
What was checked
- Matches the observed repository HEAD
- Yes
- Scanner rules are current
- Yes
- Repository scan complete
- No
- Skill scan complete
- No
- Observed behavior
- Network or broad shell
- Last checked
- Nov 9, 2026
These are observations at the recorded commit, not a guarantee of safety. Review the source before running it.
Install this skill
Choose an agent in your Stack or copy a command below. Commands fetch the pinned commit and copy only this skill directory. Existing destinations are refused. Copying a command does not install anything.
An install command is unavailable: current rules, complete repository and skill scans, an eligible behavior level and a permitted license are required.
Repository context
- Repository stars (not skill ratings)
- 1,333
- Repository last updated
- Nov 10, 2026
- License
- MIT
Observed agent integration: No specific integration evidence. Standard SKILL.md installation does not imply verified compatibility.
- engineering Derived category
- design Derived category
- data Derived category
- security Derived category
Files and excerpt
File metadata and permitted, redacted excerpts come from the recorded scan. Source links are pinned to that commit.
Browse files
SKILL.mdcore · 15,945 bytesSKILL.mdcore · 15,945 bytesSKILL.mdcore · 15,945 bytesSKILL.mdcore · 15,945 bytesSKILL.mdcore · 15,945 bytesSKILL.mdcore · 15,945 bytesSKILL.mdcore · 15,945 bytesSKILL.mdcore · 15,945 bytesagents/openai.yamlasset · 378 bytesassets/task/Task.md.templateasset · 1,872 bytesassets/world-skill/SKILL.md.templateasset · 2,987 bytesreferences/calibration.mddoc · 3,190 bytesreferences/discovery.mddoc · 12,347 bytesreferences/environment-building.mddoc · 4,125 bytesreferences/examples/service-desk.mddoc · 2,989 bytesreferences/harbor.mddoc · 6,958 bytesreferences/multi-turn-simulation/guide.mddoc · 5,177 bytesreferences/multi-turn-simulation/harbor_example.pyruns · 10,278 bytesreferences/multi-turn-simulation/model_user.pyruns · 5,117 bytesreferences/multi-turn-simulation/runner.pyruns · 7,155 bytesreferences/patterns.mddoc · 37,633 bytesreferences/synthetic-data.mddoc · 2,953 bytesreferences/task-design.mddoc · 5,657 bytesreferences/task-implementation.mddoc · 8,435 bytesreferences/verifier-design.mddoc · 3,048 bytesreferences/world-knowledge.mddoc · 11,575 bytesscripts/compare_tool_schemas.pyruns · 14,119 bytesscripts/snapshot_sqlite_state.pyruns · 5,375 bytes
Read excerpt
# Eval Engineering
## Flow
- Inspect all inputs first: the repository, Harness, optional traces, existing
Tasks and runs, existing World knowledge, and the human goal. Identify the
source files and skill references that apply before proposing work.
- Create or update the small project World Knowledge Skill from reusable facts
in those inputs. Use it to propose one grounded Task.
- Draft the Task Spec and World Skill together. Show both to the user, keep exact
Task truth only in `Task.md`, and refine both until the user approves them.
- Implement the approved Task, validate its Environment and Verifier, run the
real Harness, inspect the full evidence, and fix only non-agent failures.
- Reconcile the World Skill with what the run proved, then repeat this flow for