Individual skill

programmatic-eda

Systematic exploratory data analysis. Activate when a dataset needs profiling — structure check, nulls, outliers, distributions, correlations — before deeper analysis begins.

01-data-quality-validation/programmatic-eda/SKILL.md

Read pinned source

Source commit 9449d363e1ae43cf1706c73bebc16774e339c427

What was checked

Matches the observed repository HEAD: Yes
Scanner rules are current: Yes
Repository scan complete: No
Skill scan complete: Yes
Observed behavior: Local scripts
Last checked: Nov 6, 2026

These are observations at the recorded commit, not a guarantee of safety. Review the source before running it.

Install this skill

Choose an agent in your Stack or copy a command below. Commands fetch the pinned commit and copy only this skill directory. Existing destinations are refused. Copying a command does not install anything.

An install command is unavailable: current rules, complete repository and skill scans, an eligible behavior level and a permitted license are required.

Repository context

Repository stars (not skill ratings): 518
Repository last updated: Nov 7, 2026
License: MIT

Observed agent integration: No specific integration evidence. Standard SKILL.md installation does not imply verified compatibility.

claude-code Repository topic
codex Repository topic
data-analytics Repository topic
skills Repository topic
data Derived category

Files and excerpt

File metadata and permitted, redacted excerpts come from the recorded scan. Source links are pinned to that commit.

Browse files

SKILL.mdcore · 2,352 bytes
assets/eda_report_template.mddoc · 2,640 bytes
assets/findings_summary.mddoc · 1,458 bytes
references/eda_checklist.mddoc · 3,590 bytes
references/pandas_polars_recipes.mddoc · 3,192 bytes
references/quality_thresholds.mddoc · 2,366 bytes
scripts/correlation_explorer.pyruns · 2,715 bytes
scripts/data_overview.pyruns · 2,283 bytes
scripts/distribution_summary.pyruns · 2,308 bytes
scripts/null_profiler.pyruns · 2,275 bytes
scripts/outlier_detector.pyruns · 2,900 bytes

Read excerpt


# When to use
- You receive a new dataset and need to understand its shape and quality before analysis
- An analysis produces surprising numbers and you want to verify the underlying data first
- A stakeholder asks "is this data reliable?" or "what's in this table?"
- You're about to run a model or statistical test and need data-quality assurance

# Process
1. **Load and overview** — run `scripts/data_overview.py` to get row count, dtypes, memory usage, and a sample. Confirm grain (what one row represents).
2. **Null profile** — run `scripts/null_profiler.py`; compare output against thresholds in `references/quality_thresholds.md` and flag columns above limits.
3. **Outlier detection** — run `scripts/outlier_detector.py` (IQR + z-score) on numeric columns; document flagged values and decide: real signal or data error?
4. **Distribution summary** — run `scripts/distribution_summary.py` for descriptive stats and univariate histograms on each numeric column.
5. **Correlation exploration** — run `scripts/correlation_explorer.py`; flag pairs with |r| > 0.8 as potential multicollinearity or redundancy.
6. **EDA checklist sign-off** — work through `references/eda_checklist.md` and confirm each item before declaring the dataset profiled.
7. **Write findings** — fill `assets/eda_report_template.md` with full profiling output; distil top issues into `assets/findings_summary.md`.