Individual skill
evaluating-code-models
Evaluates code generation models across HumanEval, MBPP, MultiPL-E, and 15+ benchmarks with pass@k metrics. Use when benchmarking code models, comparing coding abilities, testing multi-language support, or measuring code generation quality. Industry standard from BigCode Project used by HuggingFace leaderboards.
plugins/KalarisLabs/research-agent-skills/skills/evaluating-code-models/SKILL.md
Source commit d780783eba4ecc7c8b9429f95c9362df95f07c61
What was checked
- Matches the observed repository HEAD
- Yes
- Scanner rules are current
- Yes
- Repository scan complete
- No
- Skill scan complete
- No
- Observed behavior
- Network or broad shell
- Last checked
- Nov 4, 2026
These are observations at the recorded commit, not a guarantee of safety. Review the source before running it.
Install this skill
Choose an agent in your Stack or copy a command below. Commands fetch the pinned commit and copy only this skill directory. Existing destinations are refused. Copying a command does not install anything.
An install command is unavailable: current rules, complete repository and skill scans, an eligible behavior level and a permitted license are required.
Repository context
- Repository stars (not skill ratings)
- 1,531
- Repository last updated
- Nov 6, 2026
- License
- Apache-2.0
Observed agent integration: No specific integration evidence. Standard SKILL.md installation does not imply verified compatibility.
- ai-coding Repository topic
- awesome Repository topic
- awesome-list Repository topic
- codex Repository topic
- codex-plugins Repository topic
- openai Repository topic
- codex-skill Repository topic
- codex-skills Repository topic
- marketplace Repository topic
- codex-marketplace Repository topic
- engineering Derived category
- testing Derived category
Declared tags are partial or unavailable.
Files and excerpt
File metadata and permitted, redacted excerpts come from the recorded scan. Source links are pinned to that commit.
Browse files
SKILL.mdcore · 13,782 bytesSKILL.mdcore · 13,782 bytesSKILL.mdcore · 13,782 bytesSKILL.mdcore · 13,782 bytesSKILL.mdcore · 13,782 bytesSKILL.mdcore · 13,782 bytesSKILL.mdcore · 13,782 bytesSKILL.mdcore · 13,782 bytesSKILL.mdcore · 13,782 bytesSKILL.mdcore · 13,782 bytesSKILL.mdcore · 13,782 bytesSKILL.mdcore · 13,782 bytesSKILL.mdcore · 13,782 bytesSKILL.mdcore · 13,782 bytesSKILL.mdcore · 13,782 bytesSKILL.mdcore · 13,782 bytesSKILL.mdcore · 13,782 bytesSKILL.mdcore · 13,782 bytesSKILL.mdcore · 13,782 bytesreferences/benchmarks.mddoc · 9,168 bytesreferences/custom-tasks.mddoc · 10,378 bytesreferences/issues.mddoc · 8,085 bytes
Read excerpt
# BigCode Evaluation Harness - Code Model Benchmarking
## Quick Start
BigCode Evaluation Harness evaluates code generation models across 15+ benchmarks including HumanEval, MBPP, and MultiPL-E (18 languages).
**Installation**:
```bash
git clone https://github.com/bigcode-project/bigcode-evaluation-harness.git
cd bigcode-evaluation-harness
pip install -e .
accelerate config
```
