Individual skill
training-data-pipeline
Build training datasets for LLM specialization from production data, frontier model distillation, and synthetic bootstrapping. Use when formatting production logs into SFT data, distilling from frontier APIs, or preparing data for fine-tuning. Covers JSONL formatting, data quality validation, deduplication, and train/eval splitting.
backend/cli/skills/ml-training/training-data-pipeline/SKILL.md
Source commit a08e0ad0ad3d6448123039ee4659163eb2e35d99
What was checked
- Matches the observed repository HEAD
- Yes
- Scanner rules are current
- Yes
- Repository scan complete
- No
- Skill scan complete
- No
- Observed behavior
- Network or broad shell
- Last checked
- Nov 2, 2026
These are observations at the recorded commit, not a guarantee of safety. Review the source before running it.
Install this skill
Choose an agent in your Stack or copy a command below. Commands fetch the pinned commit and copy only this skill directory. Existing destinations are refused. Copying a command does not install anything.
An install command is unavailable: current rules, complete repository and skill scans, an eligible behavior level and a permitted license are required.
Repository context
- Repository stars (not skill ratings)
- 4,353
- Repository last updated
- Nov 6, 2026
- License
- Apache-2.0
Observed agent integration: No specific integration evidence. Standard SKILL.md installation does not imply verified compatibility.
- agent Repository topic
- ai Repository topic
- bun Repository topic
- cli Repository topic
- llm Repository topic
- ml Repository topic
- open-source Repository topic
- research Repository topic
- science Repository topic
- ai-agent Repository topic
- training-data Declared skill tag
- data-pipeline Declared skill tag
- fine-tuning Declared skill tag
- distillation Declared skill tag
- synthetic-data Declared skill tag
- production-data Declared skill tag
- jsonl Declared skill tag
- data-quality Declared skill tag
- data Derived category
Files and excerpt
File metadata and permitted, redacted excerpts come from the recorded scan. Source links are pinned to that commit.
Browse files
SKILL.mdcore · 15,691 bytesSKILL.mdcore · 15,691 bytesSKILL.mdcore · 15,691 bytesSKILL.mdcore · 15,691 bytesSKILL.mdcore · 15,691 bytesSKILL.mdcore · 15,691 bytesSKILL.mdcore · 15,691 bytesSKILL.mdcore · 15,691 bytesSKILL.mdcore · 15,691 bytesSKILL.mdcore · 15,691 bytesSKILL.mdcore · 15,691 bytesSKILL.mdcore · 15,691 bytesreferences/data-quality.mddoc · 4,194 bytesreferences/frontier-distillation.mddoc · 4,450 bytesreferences/production-data-formatting.mddoc · 3,857 bytes
Read excerpt
# Training Data Pipeline
## When to Use This Skill
Use this skill when you need to:
- **Format production logs** into SFT training data (API logs, user corrections, accept/reject signals)
- **Distill from frontier models** using batch APIs (OpenAI, Anthropic) to label production inputs
- **Bootstrap synthetic data** when fewer than 1000 real examples exist
- **Validate data quality** before training (dedup, schema check, diversity metrics)
- **Split data** into train/eval sets with production data reserved for evaluation
### Three Data Paths
| Path | When to Use | Data Source | Cost |