Individual skill
llm-evals
Use when validating, benchmarking, or monitoring LLM application performance. Keywords: RAG evaluation, LLM-as-a-judge, CI/CD gating, trajectory scoring, test suites, prompt quality.
skills/engineering/llm-evals/SKILL.md
Source commit 1c04afc2a760ed58cea1308851e7626f76c4ff44
What was checked
- Matches the observed repository HEAD
- Yes
- Scanner rules are current
- Yes
- Repository scan complete
- No
- Skill scan complete
- No
- Observed behavior
- Instructions only
- Last checked
- Nov 11, 2026
These are observations at the recorded commit, not a guarantee of safety. Review the source before running it.
Install this skill
Choose an agent in your Stack or copy a command below. Commands fetch the pinned commit and copy only this skill directory. Existing destinations are refused. Copying a command does not install anything.
An install command is unavailable: current rules, complete repository and skill scans, an eligible behavior level and a permitted license are required.
Repository context
- Repository stars (not skill ratings)
- 27
- Repository last updated
- Oct 24, 2026
- License
- Apache-2.0
Observed agent integration: No specific integration evidence. Standard SKILL.md installation does not imply verified compatibility.
- claude-code Repository topic
- agent-skills Repository topic
- agentic-workflow Repository topic
- skill-md Repository topic
- ai-agent Repository topic
- ai-workflows Repository topic
- codex Repository topic
- coding-agent Repository topic
- developer-tools Repository topic
- gemini-cli Repository topic
- devops Derived category
- testing Derived category
Files and excerpt
File metadata and permitted, redacted excerpts come from the recorded scan. Source links are pinned to that commit.
Read excerpt
# LLM Evaluation & Validation Framework
## Overview
The `llm-evals` skill provides a systematic framework for evaluating and monitoring Large Language Model (LLM) applications throughout the development lifecycle. It bridges the gap between ad-hoc testing and production-grade reliability by implementing rigorous evaluation pipelines, metric-driven gating, and dataset versioning.
## When To Use
* Setting up automated evaluation pipelines for LLM chains.
* Developing or refining RAG (Retrieval-Augmented Generation) systems.
* Preparing LLM applications for production deployment.
* Investigating performance regressions in complex, multi-step agent trajectories.
* Defining custom rubrics for LLM-as-a-Judge scenarios.
## Core Concepts
* **Trajectory-Level Evaluation**: Scoring intermediate reasoning steps, tool choices, and argument precision, moving beyond just final output metrics.