Individual skill

distillation

Distill large models into smaller ones — teacher-student training, data generation, loss design, and quality validation. Use when you need big-model behavior at small-model cost.

skills/distillation/SKILL.md

Read pinned source

Source commit faf37c0859ebf627bfcb5225e8057d088691e0ae

What was checked

Matches the observed repository HEAD: Yes
Scanner rules are current: No
Repository scan complete: No
Skill scan complete: Yes
Observed behavior: Not checked yet
Last checked: Nov 8, 2026

These are observations at the recorded commit, not a guarantee of safety. Review the source before running it.

Install this skill

Choose an agent in your Stack or copy a command below. Commands fetch the pinned commit and copy only this skill directory. Existing destinations are refused. Copying a command does not install anything.

An install command is unavailable: current rules, complete repository and skill scans, an eligible behavior level and a permitted license are required.

Repository context

Repository stars (not skill ratings): 54
Repository last updated: Oct 10, 2026
License: MIT

Observed agent integration: No specific integration evidence. Standard SKILL.md installation does not imply verified compatibility.

agent-skills Repository topic
ai-agents Repository topic
ai-agents-automation Repository topic
ai-skills Repository topic
awesome Repository topic
awesome-list Repository topic
chatgpt Repository topic
claude-skills Repository topic
meta-ai Repository topic
muse-ai Repository topic
design Derived category
data Derived category

Files and excerpt

File metadata and permitted, redacted excerpts come from the recorded scan. Source links are pinned to that commit.

Browse files

SKILL.mdcore · 4,116 bytes

Read excerpt


# Model Distillation

Distillation transfers capability from a large teacher model to a smaller student: train the 
student to mimic the teacher's outputs (or output distributions) on a representative dataset. The 
result runs cheaper and faster while retaining much of the teacher's behavior.

## Overview

The core idea: the teacher's soft outputs (probability distributions, reasoning traces) contain 
more information than hard labels — "dark knowledge" about similarities between classes and 
plausible alternatives. The student trains on teacher-generated data, often with a loss combining 
teacher imitation and ground-truth labels. Modern practice extends this to distilling reasoning: 
training students on teachers' chain-of-thought traces to transfer reasoning ability, not just 
answers.