Individual skill
preference-optimization
Align a fine-tuned model with preference data using DPO, ORPO, KTO, or SimPO. Use when preference pairs or thumbs-up/down feedback exist, when choosing between preference-optimization methods, or when a DPO run needs hyperparameters or debugging.
plugins/llm-finetuning/skills/preference-optimization/SKILL.md
Source commit 4bde3168b43c0655c4d00ea2726e631719908a6d
What was checked
- Matches the observed repository HEAD
- Yes
- Scanner rules are current
- No
- Repository scan complete
- No
- Skill scan complete
- No
- Observed behavior
- Not checked yet
- Last checked
- Nov 7, 2026
These are observations at the recorded commit, not a guarantee of safety. Review the source before running it.
Install this skill
Choose an agent in your Stack or copy a command below. Commands fetch the pinned commit and copy only this skill directory. Existing destinations are refused. Copying a command does not install anything.
An install command is unavailable: current rules, complete repository and skill scans, an eligible behavior level and a permitted license are required.
Repository context
- Repository stars (not skill ratings)
- 42,309
- Repository last updated
- Nov 5, 2026
- License
- MIT
Observed agent integration: No specific integration evidence. Standard SKILL.md installation does not imply verified compatibility.
- anthropic Repository topic
- agent-skills Repository topic
- agentic-ai Repository topic
- ai-agents Repository topic
- cursor Repository topic
- cursor-rules Repository topic
- mcp Repository topic
- multi-agent Repository topic
- opencode Repository topic
- claude-code Repository topic
- data Derived category
- testing Derived category
Files and excerpt
File metadata and permitted, redacted excerpts come from the recorded scan. Source links are pinned to that commit.
Browse files
SKILL.mdcore · 7,910 bytesreferences/method-configs.mddoc · 6,602 bytes
Read excerpt
# Preference Optimization
This skill assumes `finetuning-method-selection`
already routed here because the data shape is
preference pairs or unpaired thumbs-up/down
feedback, not demonstrations (that's
`lora-qlora-recipes`) or a verifiable reward
signal (that's `grpo-rlvr-training`). What
follows is method selection among the DPO family,
the evidence for how much that selection actually
matters, the production training pattern, and how
to build the pairs in the first place.
**Input:** a routing decision (preference