fcakyon/labrat

# Compare: same-epoch run comparison across trackers

The most common comparison error is reporting "run A is 4 percentage points behind baseline" when run A is at epoch 11 of 100 and the baseline number is from epoch 100. The student is _still training_; the comparison is meaningless. This skill enforces same-epoch alignment.

The agentic Stop hook routes here from `reason` when an assistant reports a delta without aligning the runs.

If the question is "why did X fail / diverge / underperform", go to `/labrat:debug` first.

## Auto-detect the tracker

Check in this order:

1. `WANDB_API_KEY` env var set, or `wandb` imports or a `*.yaml` config with wandb keys in the project → **wandb**
2. `NEPTUNE_API_TOKEN` env var set, or a `*.yaml` config with neptune keys → **neptune**

# Dataset Curation Methodology

You are helping a researcher curate, analyze, or expand a dataset with attention to bias, fairness, and quality.

## Step 1: Distribution Analysis

Before any curation action, understand the current state:

### Per-Class Distribution
- Count instances per class/label/tag
- Compute imbalance ratio (max_count / min_count)
- Identify severely underrepresented classes (< 5% of max class)
- Visualize: bar chart of class frequencies sorted by count

# Debug: evidence-before-action investigation

The most expensive class of mistake in ML debugging is asserting a cause based on plausibility, then attempting a "fix" that masks the real problem. This skill enforces the discipline of probe → hypothesis → smoke → controls → claim, in that order.

The agentic Stop hook routes here from `reason` when an assistant claims a cause without backing tool output.

## When to run

The user just said any of:

- "why is X failing / diverging / NaN / OOM / hung / slow / crashed"
- "the loss is going up", "metrics look weird", "GPU util is 0"
- "debug this", "diagnose", "troubleshoot", "investigate this run"
- pasted a log excerpt asking what's wrong

# Experiment Design Methodology

You are helping a researcher design rigorous experiments. Follow this methodology systematically.

## Step 1: Understand the Research Question

Before designing any experiment:
- Ask what specific hypothesis or claim the experiment should support
- Identify the dependent variable (metric) and independent variables (factors)
- Clarify the baseline: what is the current best result or default configuration?

## Step 2: Single-Variable Isolation

Every ablation study must change exactly ONE variable at a time. For each factor:

# Launch: pre-flight checklist for long ML training jobs

Long training jobs are expensive to fail. A 12-hour run that crashes on epoch 3 from a missing dataset path or a default `workers=8` against an NFS mount is a full day lost. This skill walks six quick checks before you commit the GPUs.

The agentic Stop hook in this plugin will route here from `reason` when an assistant tries to launch a run without going through the checklist.

## When to run

The user just asked to:

- launch / kick off / start / fire up a training run
- restart a run that died
- kill a current run (also runs the cleanup half of the checklist)
- review a launch command before submitting

# Literature Research Methodology

You are helping a researcher conduct systematic literature research. Follow this methodology to ensure thorough, accurate coverage.

## Step 1: Scope Definition

Before searching:
- Clarify the exact research question or topic boundary
- Identify key terms and their synonyms (e.g., "content moderation" = "safety filtering" = "NSFW detection")
- Define inclusion/exclusion criteria (year range, venue type, methodology type)
- Ask if the user has seed papers to start from

## Step 2: Systematic Search

# Paper Check

Every number, method claim and citation in the paper must trace to code, data, results or a cited source. Output a prioritized list of issues with specific fixes.

For a code release request, follow [references/code-release.md](references/code-release.md) and skip Steps 1-4.

## Step 1: Discover Files

1. Find the paper's .tex files (Glob for `**/*.tex`)
2. Identify the main .tex file (usually imports other sections) and the document structure
3. Find .bib files for citation checking
4. Locate result files (JSON, CSV, logs), configs, evaluation scripts and training logs
5. Identify the key code modules referenced by the paper's methods

# Paper Write: academic paper writing and LaTeX setup

For LaTeX environment setup, biber vs bibtex, venue templates and compilation, follow [references/latex.md](references/latex.md).

## Core Principles

1. **Precision over elegance.** Every claim traces to code, data or a cited source.
2. **Conciseness.** Remove words that don't add information.

## Writing Process

1. **Read first.** Always read the existing section before writing.
2. **Identify the claim.** What is this paragraph trying to say?
3. **Find the evidence.** Where in code/results does this come from?

# Rebuttal: Prepare for Peer Review

Venue: $ARGUMENTS

## Step 1: Read the Paper and Results

Discover the paper, result and config files as in Step 1 (Discover Files) of `/labrat:paper-check`, including wandb logs and checkpoint directories. Then list the paper's claims (contributions and experiments), its metrics and baselines, and every available run and its outcome, including the ablation variants the configs define.

## Step 2: Vulnerability Analysis

Identify weaknesses from a reviewer's perspective:

### Technical Weaknesses
- Missing baselines that reviewers would expect

# Reproduce: paper reproduction from scratch

Reproducing an ML paper often means filling gaps the authors didn't ship, training scripts, hyperparameter tables, augmentation specifics, exact dataset splits. This skill walks seven stages from "I have an arxiv link" to "I have a replication run with measurable delta vs the paper's number."

Each stage has a separate reference file under `references/` so this overview stays scannable.

## When to run

The user just said any of:

- "reproduce / implement / replicate / re-run paper X"
- pasted an arxiv URL with reproduction intent ("can you redo this", "let's try this approach")
- pointed at an OpenReview / proceedings link with the same intent
- said "the paper has no code, can we build it"

# Lab Rat Setup

The plugin works correctly the moment it is installed. Most users never need to run setup. This skill exists to (1) show what was auto-detected on the user's system, and (2) let users opt into a few extras (notifications, allowlist, project CLAUDE.md, LaTeX).

## Step 0: Auto-detection tour (always run first)

Before asking any questions, print what the plugin already knows. No prompts, no decisions, just visibility.

Run these probes silently and present the results:

```bash
# Last few long-job launches the plugin would have flagged
recent_launches=$(history 200 2> /dev/null | grep -E '(python.*train|accelerate launch|torchrun|deepspeed|sbatch|tmux.*python)' | tail -5)