Individual skill
rag-eval
Filesystem RAG benchmarks: corpus/, train.json, evaluate_rag.py (RAGAS quality). Not for prod monitoring, latency/throughput benchmarking (use rag-perf), or evals outside this repo layout.
skills/rag-eval/SKILL.md
Source commit 25340619336e89bcc40365c28bfe9e790430468e
What was checked
- Matches the observed repository HEAD
- Yes
- Scanner rules are current
- No
- Repository scan complete
- No
- Skill scan complete
- No
- Observed behavior
- Not checked yet
- Last checked
- Nov 2, 2026
These are observations at the recorded commit, not a guarantee of safety. Review the source before running it.
Install this skill
Choose an agent in your Stack or copy a command below. Commands fetch the pinned commit and copy only this skill directory. Existing destinations are refused. Copying a command does not install anything.
An install command is unavailable: current rules, complete repository and skill scans, an eligible behavior level and a permitted license are required.
Repository context
- Repository stars (not skill ratings)
- 3,897
- Repository last updated
- Nov 6, 2026
- License
- Apache-2.0
Observed agent integration: No specific integration evidence. Standard SKILL.md installation does not imply verified compatibility.
- agent-skills Repository topic
- agentic-ai Repository topic
- ai-agents Repository topic
- claude-code Repository topic
- codex Repository topic
- coding-agents Repository topic
- cuda Repository topic
- developer-tools Repository topic
- llm Repository topic
- omniverse Repository topic
- nvidia Declared skill tag
- blueprint Declared skill tag
- rag Declared skill tag
- evaluation Declared skill tag
- ragas Declared skill tag
- benchmarking Declared skill tag
- nvidia-rag-blueprint Declared skill tag
- engineering Derived category
Files and excerpt
File metadata and permitted, redacted excerpts come from the recorded scan. Source links are pinned to that commit.
Browse files
BENCHMARK.mddoc · 2,915 bytesSKILL.mdcore · 9,415 bytesSKILL.mdcore · 9,415 bytesSKILL.mdcore · 9,415 bytesSKILL.mdcore · 9,415 bytesSKILL.mdcore · 9,415 bytesSKILL.mdcore · 9,415 bytesSKILL.mdcore · 9,415 byteseval/h100.jsonasset · 2,893 byteseval/nvidia_hosted.jsonasset · 1,963 bytesreferences/benchmark-execution.mddoc · 8,315 bytesreferences/dataset-and-conversion.mddoc · 8,361 bytesreferences/evaluate-rag-cli.mddoc · 4,283 bytesreferences/result-analysis.mddoc · 2,868 bytesskill-card.mddoc · 2,673 bytesskill.oms.sigasset · 5,797 bytes
Read excerpt
# On-disk RAG evaluation (`corpus/` + `train.json`)
## Purpose
Guide agents through NVIDIA RAG Blueprint **filesystem** benchmarks: preparing `corpus/` and `train.json`, running `scripts/eval/evaluate_rag.py`, tuning retrieval and generation flags for **quality** comparisons, interpreting RAGAS JSON outputs, and triaging failures (HTTP/stream errors, empty contexts, collection mismatch, judge API).
For **latency, throughput, and load testing**, use the **rag-perf** skill (`scripts/rag-perf`, `docs/performance-benchmarking.md`) — not this skill.
## When not to use
Do **not** use this skill for: deploying or repairing services (use rag-blueprint); evaluating APIs without the `corpus/` + `train.json` layout; general ML experimentation unrelated to this evaluator; production monitoring/alerting; or latency/throughput benchmarking (use **rag-perf**).
## Prerequisites
