Individual skill
model-evaluation
Generates python code that evaluates SageMaker models. Supports two evaluation types: LLM-as-Judge and Custom Scorer. Use when the user says "evaluate my model", "run a benchmark", "test model performance", "how did my model perform", "compare models", or other similar requests.
plugins/sagemaker-ai/skills/model-evaluation/SKILL.md
Source commit fbefb3764e67558fe10ac834dd85db49734aa9b6
What was checked
- Matches the observed repository HEAD
- Yes
- Scanner rules are current
- Yes
- Repository scan complete
- No
- Skill scan complete
- No
- Observed behavior
- Network or broad shell
- Last checked
- Nov 4, 2026
These are observations at the recorded commit, not a guarantee of safety. Review the source before running it.
Install this skill
Choose an agent in your Stack or copy a command below. Commands fetch the pinned commit and copy only this skill directory. Existing destinations are refused. Copying a command does not install anything.
An install command is unavailable: current rules, complete repository and skill scans, an eligible behavior level and a permitted license are required.
Repository context
- Repository stars (not skill ratings)
- 937
- Repository last updated
- Nov 5, 2026
- License
- Apache-2.0
Observed agent integration: No specific integration evidence. Standard SKILL.md installation does not imply verified compatibility.
- agent-skills Repository topic
- agents Repository topic
- coding-agent-skills Repository topic
- coding-agents Repository topic
- agent-plugins Repository topic
- aws Repository topic
- testing Derived category
Files and excerpt
File metadata and permitted, redacted excerpts come from the recorded scan. Source links are pinned to that commit.
Browse files
SKILL.mdcore · 5,333 bytesSKILL.mdcore · 5,333 bytesSKILL.mdcore · 5,333 bytescode_templates/custom_scorer_evaluator.pyruns · 2,536 bytescode_templates/llmaaj_evaluator.pyruns · 2,500 bytesreferences/code_output_guide.mddoc · 3,346 bytesreferences/create-reward-function.mddoc · 3,062 bytesreferences/custom-lambda-scorer.mddoc · 4,472 bytesreferences/custom-scorer-evaluation.mddoc · 10,444 bytesreferences/evaluation-type-guide.mddoc · 8,156 bytesreferences/llmaaj-builtin-evaluation.mddoc · 5,086 bytesreferences/llmaaj-custom-evaluation.mddoc · 2,405 bytesreferences/llmaaj-evaluation.mddoc · 14,074 bytesreferences/supported-judge-models.mddoc · 2,522 bytesscripts/nova_reward_function_source_template.pyruns · 13,341 bytesscripts/reward_function_source_template.pyruns · 9,544 bytesscripts/validate_custom_metrics.pyruns · 3,694 bytes
Read excerpt
# Model Evaluation
Generate code that evaluates a SageMaker model.
## Prerequisites
- The SDK environment has been verified (SDK version, region, execution role). If not done, activate the `sdk-getting-started` skill first.
## Principles
1. **One thing at a time.** Each response advances exactly one decision. Never combine multiple questions in a single turn.
2. **Confirm before proceeding.** Wait for the user to agree before moving to the next step.
3. **Don't read files until you need them.** Only read reference files when you've reached the step that requires them.
4. **Don't ask what you already know.** If the answer is in conversation history, workflow_state.json, plan.md, or any file you've already read — use it. Confirm if unsure, but don't re-ask.