nimrodfisher/data-analytics-skills

# When to use
- A data pipeline has just loaded new data and needs validation before downstream reports consume it
- A stakeholder has flagged data quality concerns (wrong totals, unexpected nulls, stale data)
- You need to produce a formal data quality scorecard for a data asset as part of a data governance process
- You are onboarding a new data source and need to understand its quality profile before building on it

# Process
1. **Null and completeness audit** — run `scripts/null_counter.py` for a column-by-column null profile. Flag columns above acceptable thresholds for the business context.
2. **Duplicate detection** — run `scripts/duplicate_finder.py` to identify full-row and key-level duplicates. Determine if duplicates are intentional (versioning) or errors (pipeline fan-out).
3. **Referential integrity check** — run `scripts/referential_integrity.py` to validate that foreign key values in child tables exist in parent tables. Report orphan rate per relationship.
4. **Value range validation** — run `scripts/value_range_validator.py` with business rules defined in `references/business_rule_patterns.md`. Flag values outside acceptable ranges.
5. **Freshness check** — run `scripts/freshness_check.py` to verify the dataset is up to date — compare the latest record timestamp against the expected lag for this pipeline.
6. **Score and classify findings** — map each finding to a quality dimension using `references/quality_dimensions.md`. Assign severity (CRITICAL / HIGH / MEDIUM / LOW).
7. **Produce deliverables** — fill `assets/audit_report_template.html` for a shareable report; fill `assets/quality_rubric.md` for a concise scorecard.

# When to use
- You receive a new dataset and need to understand its shape and quality before analysis
- An analysis produces surprising numbers and you want to verify the underlying data first
- A stakeholder asks "is this data reliable?" or "what's in this table?"
- You're about to run a model or statistical test and need data-quality assurance

# Process
1. **Load and overview** — run `scripts/data_overview.py` to get row count, dtypes, memory usage, and a sample. Confirm grain (what one row represents).
2. **Null profile** — run `scripts/null_profiler.py`; compare output against thresholds in `references/quality_thresholds.md` and flag columns above limits.
3. **Outlier detection** — run `scripts/outlier_detector.py` (IQR + z-score) on numeric columns; document flagged values and decide: real signal or data error?
4. **Distribution summary** — run `scripts/distribution_summary.py` for descriptive stats and univariate histograms on each numeric column.
5. **Correlation exploration** — run `scripts/correlation_explorer.py`; flag pairs with |r| > 0.8 as potential multicollinearity or redundancy.
6. **EDA checklist sign-off** — work through `references/eda_checklist.md` and confirm each item before declaring the dataset profiled.
7. **Write findings** — fill `assets/eda_report_template.md` with full profiling output; distil top issues into `assets/findings_summary.md`.

# When to use
- A SQL query is about to be promoted to a production dashboard or report
- A query is returning surprising or incorrect results
- A query is running slowly and needs performance review
- You want to catch anti-patterns (implicit conversions, SELECT *, unbounded CTEs) before they cause incidents

# Process
1. **Lint the query** — run `scripts/sql_lint.py` (sqlglot-based) to catch syntax errors, unsupported functions for the target engine, and style violations. Fix hard errors before continuing.
2. **Review anti-patterns** — compare the query structure against `references/sql_anti_patterns.md`. Flag any present anti-patterns with a severity rating.
3. **Parse the explain plan** — if an EXPLAIN or query profile output is available, run `scripts/explain_plan_parser.py` to extract slow steps (full table scans, missing indexes, high row estimates).
4. **Estimate cardinality** — run `scripts/cardinality_estimator.py` if schema stats are available to flag joins that might fan-out unexpectedly.
5. **Check engine-specific behaviour** — consult `references/engine_specific_guide.md` for the target engine (Snowflake / BigQuery / Postgres / Redshift) to verify date functions, window behaviour, and clustering assumptions.
6. **Produce review output** — fill in `assets/query_review_template.md` with findings; for any performance issues found, complete `assets/optimization_recommendations.md`.

# Analysis Assumptions Log

# When to use
- Starting an analysis with significant scope, method, or data quality choices
- Preparing work for peer review or stakeholder sign-off
- Returning to an old analysis and needing to understand prior decisions
- Working in a regulated environment where auditability is required
- Handing off an analysis to another analyst

# Process
1. **Initialize the log** — create a log entry for the analysis with its name, date, analyst, and the decision it informs. Use `scripts/assumptions_tracker.py` to initialise a structured JSON log.
2. **Enumerate data assumptions** — document representativeness, completeness, how missing values are handled, and any known quality issues. For each assumption, record the rationale and confidence level (high/medium/low). See `references/assumption_categories.md` for the full taxonomy.
3. **Enumerate business logic assumptions** — record metric definitions, time windows, inclusion/exclusion rules, and any definitions provided by stakeholders. Note alternatives considered.
4. **Enumerate statistical assumptions** — record distribution assumptions, independence claims, stationarity, or model assumptions relevant to the methods used.

# Analysis Documentation

# When to use
- Finalising an analysis before sharing it with stakeholders
- Handing off an analysis to another team member or to a future self
- Archiving recurring analyses so they can be run again consistently
- Preparing for peer review or a formal audit
- Converting an exploratory notebook into a reference document

# Process
1. **Confirm audience and scope** — determine whether the primary reader is technical (data team), business (stakeholders), or both. For mixed audiences, use a tiered structure. See `references/audience_depth_guide.md` for calibration.
2. **Write the business context section** — state the business question, the stakeholders who requested the analysis, the decisions it informs, and the success criteria.
3. **Document data sources** — for each source, record the table or file, date range, row count, key columns, and any known quality issues or exclusions applied.
4. **Write the methodology section** — describe the analytical approach, tools and library versions, key assumptions, and important decisions made (and alternatives considered). Reference the assumptions log if one exists.

# Data Catalog Entry

# When to use
- A new table, view, or dataset has been created and needs to be discoverable
- Analysts keep asking the same questions about a table's meaning or ownership
- A compliance or audit requirement mandates documentation of sensitive data
- Onboarding new team members who need to understand available data assets
- Auditing catalog completeness to find undocumented tables

# Process
1. **Extract technical metadata** — pull schema, column names, types, primary keys, foreign keys, and row count from `INFORMATION_SCHEMA` or the source system. Use `scripts/catalog_extractor.py` to automate this for database tables.
2. **Collect business context** — interview the data owner to capture the business purpose, owning team, criticality (critical / high / medium / low), and known use cases. Record the business-friendly display name.
3. **Write column descriptions** — for each column, write a one-sentence plain-language description, note example values, and document any business rules (valid values, constraints, format requirements).
4. **Assess data quality** — calculate or estimate completeness, freshness (hours since last update), and duplicate rate. Document known issues and how they affect downstream use.

# Metric Reconciliation

# When to use
- Two dashboards or reports show different values for the same KPI
- A metric changed unexpectedly after a data pipeline update
- Stakeholders question a number and need an authoritative explanation
- Preparing to merge or deprecate a legacy reporting source
- Onboarding analysts to a new data model and validating it against the old one

# Process
1. **Define the metric and scope** — confirm the exact definition (numerator, denominator, filters, time zone) and the period under investigation. Mismatched definitions are the most common cause of discrepancy.
2. **Pull values from both sources** — extract the metric values for the same period from each source. Record absolute values, row counts, and the query or calculation path used.
3. **Compute the gap** — calculate the absolute difference and percentage gap. If the gap is within an agreed tolerance (e.g., ±0.1%), document it as accepted and close. See `references/reconciliation_patterns.md` for tolerance guidelines.
4. **Trace the computation path** — walk each source's query or pipeline step by step. Common divergence points: different join types, filter order, null handling, date truncation, or deduplication logic.

# Schema Mapper

# When to use
- Integrating a new data source and need to map its fields to the existing data model
- Designing an ETL or dbt transformation and need to document the logic
- Auditing what happened to a field during a migration
- Onboarding a new analyst who needs to understand where columns come from
- Preparing a data catalog entry that requires lineage at the column level

# Process
1. **Collect the source schema** — list every column name, data type, nullable flag, and a brief description. Pull from `INFORMATION_SCHEMA`, a data dictionary, or the source API documentation.
2. **Collect the target schema** — same structure for the destination table or model. If the target doesn't exist yet, draft it based on the analytical requirements.
3. **Map source columns to target columns** — for each target column, identify the source column(s) that feed it. Record direct mappings (rename only) and derived mappings (calculation, type cast, lookup join). Use `scripts/schema_compare.py` to automate direct-name matches.
4. **Document transformation rules** — for each derived mapping, write the exact transformation (e.g., `CAST(amount_cents AS FLOAT) / 100.0`, `COALESCE(first_name, email)`).

# When to use
- A stakeholder asks "how is [metric] calculated?" and no canonical definition exists
- You're setting up dbt Semantic Layer and need YAML metric/dimension/entity definitions
- Multiple teams are using different SQL queries for the same metric — you need to codify the one true definition
- You're building a data catalog entry for a core model and need structured metadata

# Process
1. **Identify the object type** — decide whether you're documenting a metric, a dimension, or an entity. Use the frameworks in `references/metric_definition_framework.md` for metrics and `references/dimension_hierarchy_patterns.md` for dimensions.
2. **Gather the definition inputs** — collect: calculation logic (SQL or formula), business context, data source(s), grain, edge cases, and known gotchas. Ask the data owner if anything is unclear.
3. **Generate the YAML template** — run `scripts/metric_template_generator.py` to scaffold the initial YAML structure for the object type. Fill in the generated template.
4. **Validate the YAML** — run `scripts/model_yaml_validator.py` to check required fields, type constraints, and reference integrity (referenced dimensions exist in the same file).
5. **Add dbt context** — if this will be deployed to dbt Semantic Layer, consult `references/dbt_semantic_layer_guide.md` for the exact field names and constraints for your dbt version.
6. **Save final definitions** — save metrics to `assets/metric_definition.yaml`, dimensions to `assets/dimension_definition.yaml`, entities to `assets/entity_definition.yaml`.

# SQL to Business Logic Translator

# When to use
- A stakeholder asks "what exactly does this query calculate?"
- Documenting a query library or a dbt model for non-technical readers
- Reviewing a query for correctness by comparing its logic to the business requirement
- Onboarding new analysts to existing SQL patterns
- Translating legacy undocumented queries before refactoring

# Process
1. **Receive the query and context** — obtain the SQL and the business question it answers. Also collect any schema notes (what the key tables and columns represent in business terms).
2. **Translate the FROM/JOIN structure** — describe in plain language which data sources are combined and what type of join is used (inner keeps only matches; left keeps all rows from the left side). Note if the join type seems inconsistent with the stated purpose.
3. **Translate WHERE filters** — list each filter condition as a business rule in plain language (e.g., `status = 'completed'` → "only includes orders that have been paid and fulfilled").
4. **Explain GROUP BY and aggregations** — describe what each aggregation computes and at what grain. Use `scripts/sql_explainer.py` to automate a first-pass structural parse.

# A/B Test Analysis

# When to use
- An experiment has finished and the team needs a ship / no-ship recommendation
- Results look directionally positive but the team is unsure if they're statistically significant
- A test has been running for weeks without a clear winner and someone needs to decide whether to continue
- A new experiment needs sample-size planning before launch
- Results are disputed and need a rigorous, documented analysis

# Process
1. **Confirm test design** — verify the hypothesis, the control and treatment definitions, the randomisation unit (user/session/device), the primary metric, any guardrail metrics, and the target split ratio.
2. **Check for sample ratio mismatch (SRM)** — run a chi-square test on the actual vs. expected split. If SRM is detected, stop and investigate the randomisation pipeline before interpreting results. Use `scripts/ab_test_analyzer.py --check-srm`.
3. **Calculate per-variant metrics** — compute the rate (or mean) and 95% confidence interval for the primary metric in each variant. Document absolute and relative difference.
4. **Run the significance test** — execute a two-proportion z-test (for rates) or Welch's t-test (for means). Record z-score, p-value, and 95% CI for the effect. Use `references/statistical_tests_reference.md` if unsure which test applies.

# Business Metrics Calculator

# When to use
- Preparing a board or investor deck and need accurately defined metrics
- The team disagrees on how a key metric (e.g., churn) should be calculated
- Benchmarking performance against industry standards
- Building a metrics report for a new business or new metric set
- Validating that existing metric calculations match the standard definition

# Process
1. **Identify the business model and period** — confirm the model type (SaaS subscription, e-commerce, marketplace, product/app) and the calculation period (month, quarter, trailing 12M). Model type determines which metrics apply. See `references/metric_definitions.md`.
2. **Load and validate the underlying data** — check for expected row counts, missing values, and plausible date ranges. A metrics report is only as good as the data feeding it.
3. **Calculate primary metrics** — for SaaS: MRR, ARR, new MRR, churned MRR, expansion MRR, customer churn rate, revenue churn rate. For e-commerce: GMV, AOV, conversion rate, ROAS. Use `scripts/saas_metrics.py` or adapt for other models.
4. **Calculate unit economics** — LTV (simple average and cohort-based), CAC, LTV:CAC ratio, payback period, and quick ratio. Document which assumptions were used for LTV lifetime.

# When to use
- A stakeholder asks "are we retaining users better than last quarter?"
- You need to measure N-day, weekly, or monthly retention for a product or feature
- You want to compare how different acquisition cohorts (by channel, plan, or signup date) perform over their lifetime
- You're investigating churn and need to identify at which period users typically leave

# Process
1. **Define the cohort and activity** — clarify: cohort grouping (signup month, first purchase date, etc.) and retention event (login, purchase, feature use). Document in the report header.
2. **Pull or build the data** — if starting from a database, use `scripts/cohort_query.sql` as the starting point. Adapt the `cohort_date` and `activity_date` columns to your schema.
3. **Build the cohort table** — run `scripts/cohort_builder.py` to produce a cohort × period membership table from event data. Output is a CSV with `user_id`, `cohort_period`, `activity_period`.
4. **Compute the retention matrix** — run `scripts/retention_matrix.py` on the cohort table to generate the period-over-period retention rates. Output is an N×M matrix (cohort × period).
5. **Visualise** — run `scripts/cohort_visualizer.py` to render a heatmap of the retention matrix and a time-series of retention curves per cohort.
6. **Interpret findings** — consult `references/retention_metrics_glossary.md` for metric definitions and `references/cohort_definition_patterns.md` for pattern recognition.
7. **Write the report** — fill `assets/cohort_report_template.md`. For a visual deliverable, fill in the `assets/retention_matrix.html` heatmap template.

# Funnel Analysis

# When to use
- Conversion is low and the team needs to know where users are dropping off
- A product change may have affected a specific funnel step
- Comparing conversion rates across channels, devices, or user cohorts
- Designing an A/B test and needing a baseline to set a meaningful MDE
- Building a regular funnel monitoring report

# Process
1. **Define funnel steps and time window** — list the ordered sequence of events or pages that constitute the funnel. Agree on how long a user has to complete the funnel (session, 24 hours, 7 days). Ambiguous definitions here will invalidate the analysis.
2. **Build the user-level funnel dataset** — for each user who reached step 1, record which subsequent steps they completed and when, within the time window. Use `scripts/funnel_analyzer.py` to compute this from an events log.
3. **Calculate conversion rates** — compute step-to-step conversion (users reaching step N ÷ users reaching step N−1) and overall conversion (step 1 to last step). Record absolute drop-off counts at each step.
4. **Analyse time-to-convert** — for users who completed each step, calculate median, P75, and P95 time between steps. Long gaps can signal friction even without high drop-off.

# Root Cause Investigation

# When to use
- A key metric dropped (or spiked) unexpectedly and the team needs an explanation
- Stakeholders are asking "why did X happen?" and need an evidence-based answer
- A metric change has been observed but the team is unsure whether it's noise or signal
- Preparing a post-mortem after an incident that affected business metrics
- A trend change happened weeks ago and needs retrospective investigation

# Process
1. **Validate the change** — confirm the metric changed beyond normal variance using a z-score or simple comparison to the rolling average. If the change is within ±1.5 standard deviations, document it as within normal range and close. Use `scripts/drilldown_analyzer.py --validate`.
2. **Establish a timeline** — plot the metric over time to pinpoint when the change started. A sudden step change suggests a specific event; a gradual drift suggests a structural shift.
3. **Decompose the metric** — break the metric into its constituent parts (e.g., revenue = volume × price × mix). Determine which component is driving the change before drilling into dimensions.
4. **Drill down systematically** — compare the metric before vs. after the change across available dimensions (geography, platform, channel, product category, user segment). Sort by absolute contribution to identify the primary driver. Use `scripts/drilldown_analyzer.py --drilldown`. See `references/rca_framework.md` for the structured approach.

# Segmentation Analysis

# When to use
- The team needs to understand who the best customers are and what distinguishes them
- Marketing wants distinct groups to target with different messages or offers
- Product needs to prioritise features based on high-value user behaviour patterns
- Churn is high and the team needs to identify at-risk users before they leave
- An existing segmentation feels arbitrary and needs data validation or improvement

# Process
1. **Define the segmentation goal** — clarify what decisions the segments will inform (product roadmap, marketing campaigns, retention programs). The goal determines which variables matter and how many segments are useful (typically 3–7). See `references/segmentation_approaches.md`.
2. **Select and prepare variables** — choose 3–7 attributes or behaviours that vary across users and relate to the business outcome. Handle missing values and scale continuous variables. Remove outliers only if they would distort cluster centroids.
3. **Run the segmentation** — for data-driven segmentation, use k-means clustering via `scripts/segmentation_runner.py`. For rule-based segmentation, apply the business logic rules and validate that segments are distinct and non-overlapping.
4. **Profile each segment** — compute the mean and median for each variable by segment, expressed as % above/below the overall average. Identify the 2–3 defining characteristics of each segment and assign a descriptive name.

# Time Series Analysis

# When to use
- Building a forecast for operational planning (staffing, inventory, infrastructure capacity)
- Identifying whether a trend is genuine or driven by seasonality
- Detecting anomalies in a metric stream (traffic spikes, revenue dips, error rate surges)
- Providing a "what would have happened" baseline for measuring initiative impact
- Presenting year-over-year growth in a way that accounts for seasonal patterns

# Process
1. **Load and inspect the time series** — confirm regular intervals (fill gaps if needed), check for obvious data quality issues (negative values, zeros in non-zero series), and identify the natural granularity (daily, weekly, monthly).
2. **Test for stationarity** — run an ADF test. If non-stationary (trend or seasonality present), note this — it informs decomposition and model choice rather than blocking analysis. See `references/ts_patterns_guide.md`.
3. **Decompose into components** — separate the time series into trend, seasonal, and residual using additive or multiplicative decomposition. Measure the strength of each component (0–1). Strong seasonality (>0.6) means raw values are misleading without seasonal adjustment.
4. **Detect anomalies** — flag points more than 3 standard deviations from the rolling median. Investigate the top 5 anomalies against the event log (product releases, campaigns, incidents). Use `scripts/ts_analyzer.py --detect-anomalies`.

# Dashboard Specification

# When to use
- A new dashboard is being built and developers need a clear brief before starting
- An existing dashboard is confusing or underused and needs a structured redesign
- Stakeholders and the data team have different ideas about what a dashboard should show
- Documenting dashboard requirements as part of a broader data product process
- Creating a self-service analytics specification that can be handed off without multiple Q&A rounds

# Process
1. **Define the purpose** — write one sentence: "This dashboard answers [question] for [audience] who need to [decision or action]." If it can't be stated in one sentence, the scope needs narrowing first. See `references/dashboard_design_principles.md`.
2. **Profile target users** — for each audience (executive, manager, IC), document their visit frequency, the primary question they come to answer, and their technical comfort level. Users with different needs usually need different dashboards, not more filters on one.
3. **Define the metric hierarchy** — list primary KPIs (hero numbers at the top), secondary supporting metrics, and detail-level breakdowns. A dashboard with more than 10–12 distinct metrics is trying to do too much.
4. **Design the information architecture** — sketch the layout using the hero → trends → breakdowns → details pattern. Position the most important information in the top-left. Use `references/dashboard_requirements_guide.md` for layout patterns.

# Data Narrative Builder

# When to use
- A presentation exists but feels like a data dump rather than a story
- The analysis has a clear finding but the team doesn't know how to make it compelling
- The audience is senior and will not read more than 5 slides
- A decision needs to be made and the data needs to make a clear, emotion-aware argument
- A recurring report needs to be restructured around a narrative rather than a metric list

# Process
1. **Identify the central message** — write the single most important thing the audience should know and do after seeing this. Everything else is supporting material. If there's more than one message, there are multiple presentations.
2. **Choose a narrative framework** — select the structure that fits the context. Situation–Complication–Resolution works for most problem/solution stories. Before–After–Bridge is strong for demonstrating impact. See `references/narrative_frameworks.md` for all patterns with examples.
3. **Assign an emotional arc** — map each section to an intended emotional state: establish comfort (Situation), introduce tension (Complication), offer confidence (Resolution). Tone and data emphasis should match the intended emotion at each stage.
4. **Draft each section with data woven in** — apply the pyramid principle: lead with the conclusion, then support it with evidence. Numbers must serve the narrative, not interrupt it. Round large numbers for recall; humanise by expressing impact in terms the audience cares about.

# Executive Summary Generator

# When to use
- A detailed analysis needs to be condensed to 1–2 pages for a senior audience
- An executive asks "what's the bottom line?" and a full report won't be read
- Preparing a board deck section that summarises a longer analytical workstream
- A recurring report needs to lead with the key message rather than data tables
- A decision needs to be made by end of day and the executive has 10 minutes

# Process
1. **Extract the top 3–5 insights** — go through the full analysis and identify only the findings that change or reinforce a decision. Filter out interesting-but-not-actionable findings. If you have more than 5 insights, you haven't prioritised yet.
2. **Quantify the business impact of each insight** — every insight must carry a number: revenue at risk, cost saving, users affected, time to payback. Vague impact ("significant") does not belong in an executive summary. See `references/executive_communication.md`.
3. **Write a one-paragraph situation statement** — explain why this analysis was done, what the question was, and why the timing matters. One paragraph, no jargon.
4. **Apply the pyramid principle** — lead each insight with the conclusion ("Mobile churn is causing $800K ARR loss"), then the evidence, then the supporting detail. Never bury the finding at the end of a paragraph.

# Insight Synthesis

# When to use
- An analysis has produced many statistics but no clear "so what"
- The team has findings but is struggling to prioritise which ones to act on
- Stakeholders are asking "what does this mean for us?" rather than "what did you find?"
- Multiple analyses need to be synthesised into a unified set of recommendations
- Preparing an insight briefing for a team that doesn't have time to review the full analysis

# Process
1. **List all findings** — enumerate every statistically meaningful finding: trends, comparisons, correlations, anomalies, surprises. Write each as a factual statement. Don't interpret yet.
2. **Apply So What → Why → Now What to each finding** — convert each fact into an insight by answering: So what (why does this matter to the business?), Why (what is the most likely explanation?), Now what (what specific action should follow?). See `references/insight_framework.md`.
3. **Quantify business impact** — for each insight, estimate the financial, customer, or operational magnitude. An insight without a number is an observation. Use order-of-magnitude estimates if precise data is not available.
4. **Prioritise by impact × confidence × actionability** — score each insight on these three dimensions (1–3 scale). Insights that score high on all three are the ones to lead with. Deprioritise insights that are high-impact but low-confidence until validated.

# Visualization Builder

# When to use
- Choosing the right chart type for a specific analytical message
- A chart exists but is cluttered, misleading, or failing to make the point
- Building a chart for an executive presentation that must work without verbal explanation
- Producing consistent, branded visualisations across a report or dashboard
- Creating accessible charts that work for colorblind viewers or screen readers

# Process
1. **Identify the message type** — classify the chart's purpose: comparison (bar), trend over time (line), composition / part-of-whole (stacked bar, pie only for 2–3 categories), distribution (histogram, box plot), or relationship (scatter). The message type determines the chart type. See `references/chart_selection_guide.md`.
2. **Select and load the data** — confirm the data is at the right grain for the chart. Aggregations (e.g., groupby month) should happen before plotting, not inside the chart library.
3. **Build the base chart** — use `scripts/chart_builder.py` with pre-set professional styling (whitegrid, sans-serif, accessible color palette). Set axes, ticks, and scale deliberately — default settings are often wrong.
4. **Apply visual hierarchy** — make the most important data element visually dominant (bolder line, darker bar, distinct color). De-emphasise secondary series. Remove every element that doesn't contribute to the message (gridlines at 0.2 alpha, no top/right spines). See `references/visual_design_principles.md`.

# When to use

Before sharing any analysis output with a stakeholder — dashboard, report, ad-hoc query result, model output, or written findings. Run this every time, not just for big projects. The cost of a post-delivery correction is always higher than the cost of a pre-delivery check.

# Process

1. **Run automated checks** — use `scripts/qa_runner.py` against the output file to catch numeric, structural, and formatting issues programmatically.
2. **Complete the logic checklist** — work through `references/qa_checklist_master.md` section by section: question framing, data sourcing, transformations, statistical validity, findings, and presentation.
3. **Review for common errors** — cross-check against `references/common_analysis_errors.md`; pay special attention to the top-frequency mistakes for the analysis type.
4. **Validate assumptions explicitly** — for every assumption in the analysis, verify it has a source, is documented, and the output is sensitivity-tested where the assumption is uncertain.
5. **Check the narrative** — confirm the conclusion follows from the data, caveats are stated, and the recommendation is actionable.
6. **Record sign-off** — complete `assets/qa_signoff_template.md` with reviewer, issues found, resolution status, and delivery decision.

# Inputs the skill needs

# When to use

After an analytical finding surfaces a potential action, change, or opportunity. Use to produce a defensible numeric estimate that stakeholders can act on. Also use when prioritizing a backlog of initiatives — quantified impact is the primary ranking signal.

# Process

1. **Classify the impact type** — revenue growth, cost reduction, risk reduction, or efficiency gain. Each type has a different formula family (see `references/impact_quantification_framework.md`).
2. **Gather inputs** — collect baseline metrics, affected population size, expected lift/reduction, time horizon, and confidence level.
3. **Build the point estimate** — use `scripts/revenue_impact.py` for revenue/growth scenarios or `scripts/cost_savings.py` for cost/efficiency scenarios.
4. **Add uncertainty bounds** — use `scripts/confidence_interval.py` to produce low/base/high estimates. Never deliver a single number without a range.
5. **Document assumptions** — fill in `references/assumption_documentation.md` for every input that is estimated rather than directly measured; note the sensitivity of the output to each.
6. **Package the estimate** — complete `assets/impact_estimate_template.md` with the range, assumptions, confidence, and recommended action; optionally build the full `assets/business_case_template.md` for larger decisions.

# Inputs the skill needs

# When to use

Any time you deliver findings that require the audience to trust the method — A/B tests, attribution models, forecasts, statistical analyses, or anything where "how did you get that?" is a likely question. Write the methodology section before distributing results, not after questions arrive.

# Process

1. **Identify the audience tier** — use `references/audience_depth_guide.md` to determine the appropriate level: executive (why/what), business analyst (what/how at high level), or technical peer (full detail).
2. **Select the explanation pattern** — use `references/methodology_explanation_patterns.md` to pick the structure: narrative, layered (short summary + appendix), or Q&A format.
3. **Draft the core explanation** — cover: what question was asked, what data was used, what method was applied, what assumptions were made, and what the key limitation is.
4. **Apply plain-language rewrites** — replace statistical terms with business equivalents per the translation table in `references/methodology_explanation_patterns.md`.
5. **Add a limitations paragraph** — every methodology explanation must include at least one honest limitation and what it means for the conclusions.
6. **Produce deliverables** — write-up using `assets/methodology_writeup_template.md`; if the methodology will be presented, use `assets/methodology_slide_template.md`.

# Inputs the skill needs

# When to use

At the start of any non-trivial analysis request, especially when the ask is vague ("can you look into X?"), when multiple stakeholders have a stake in the outcome, or when the result will drive an important decision. Spending 30 minutes on requirements prevents days of rework.

# Process

1. **Run the intake interview** — use the question guide in `assets/interview_guide.md` to surface: the business decision being made, who the audience is, what "done" looks like, and what constraints exist.
2. **Identify the decision type** — apply `references/decision_maker_framework.md` to classify the decision (strategic / operational / tactical) and calibrate the required rigour and format.
3. **Document requirements** — fill in `assets/requirements_doc_template.md` covering: business question, success criteria, scope inclusions/exclusions, data sources, and timeline.
4. **Resolve ambiguities** — use the elicitation techniques in `references/elicitation_techniques.md` for any requirement still unclear after the interview (5-whys, scenario walkthrough, MoSCoW prioritisation).
5. **Get explicit sign-off** — send the requirements doc to the requestor for confirmation before starting work; update based on feedback.
6. **Produce the analysis brief** — convert approved requirements into `assets/analysis_brief_template.md`, which becomes the authoritative scope document for the project.

# Inputs the skill needs

# When to use

When technical output (model results, statistical tests, query findings) needs to be understood by a business audience. Also use to review your own writing before sending — it is easy to slip into jargon without noticing.

# Process

1. **Detect jargon** — run `scripts/jargon_detector.py` on the draft text to flag technical terms that need translation.
2. **Score readability** — run `scripts/readability_scorer.py` to get Flesch-Kincaid grade level and sentence complexity metrics; target ≤ grade 10 for executive audiences.
3. **Identify the audience persona** — use `references/stakeholder_personas.md` to select the persona that best matches your reader; each persona has vocabulary preferences and typical questions.
4. **Apply translation patterns** — use `references/translation_pattern_library.md` to swap technical language for business equivalents (e.g., "p-value < 0.05" → "we're 95% confident this isn't random chance").
5. **Replace with metaphors where needed** — for complex statistical concepts, pick an appropriate metaphor from `references/metaphor_bank.md`.
6. **Draft the translated version** — use `assets/translation_template.md` to produce the parallel technical/business version; keep the original in an appendix for technical reviewers.

# Inputs the skill needs

# When to use

After requirements are gathered and before any data is touched. Planning is especially important when the analysis involves multiple steps, uncertain data availability, or a tight deadline where sequencing matters. A 15-minute planning session prevents hours of wrong-direction work.

# Process

1. **Decompose the question** — break the business question into sub-questions using `references/scoping_framework.md`; each sub-question should be answerable with a single data pull or calculation.
2. **Identify data dependencies** — for each sub-question, list the required tables/datasets and assess availability (confirmed / likely / unknown); flag blockers early.
3. **Sequence the work** — order sub-questions so that each output feeds the next; identify which steps can run in parallel.
4. **Estimate effort** — use `references/effort_estimation.md` to assign time estimates per step; sum to a total and compare against the deadline.
5. **Log risks and dependencies** — use `references/risks_dependencies.md` to document anything that could delay or invalidate the plan (data gaps, external approvals, methodology uncertainty).
6. **Produce the plan** — fill in `assets/analysis_plan_template.md`; for projects with stakeholder kickoffs use `assets/kickoff_doc_template.md`.

# Inputs the skill needs

# When to use

Within one week of completing a significant analysis project — while the details are still fresh. Also use after an analysis that went wrong (late delivery, stakeholder rejection, data error discovered post-delivery) to prevent recurrence. Run team retros quarterly even without a specific incident.

# Process

1. **Time-box the retro** — 30 minutes for solo, 60 minutes for team. Use the structured format in `references/retro_frameworks.md` to stay focused (Start/Stop/Continue or 4Ls: Liked/Lacked/Learned/Longed for).
2. **Review the project against plan** — compare actual timeline, scope, and effort to what was planned; note the gaps.
3. **Identify what went well** — capture at least two things that worked and should be repeated; these are as important as problems.
4. **Identify root causes of issues** — for each problem, apply 5-whys to find the actual cause rather than the symptom.
5. **Capture reusable learning** — use `references/learning_capture.md` to decide which learnings belong in: templates, reference docs, checklists, or team norms.
6. **Record and track actions** — fill in `assets/retrospective_template.md` with owners and due dates; log durable learnings in `assets/learnings_log_template.md`.

# Inputs the skill needs

# When to use

Before starting an AI-assisted analysis session when the task requires more than a single prompt — complex investigations, multi-step analyses, or work that depends on project-specific knowledge. A well-packaged context bundle reduces back-and-forth and produces better first responses.

# Process

1. **Identify required context layers** — use `references/context_layering_guide.md` to decide which layers are needed: task definition, business context, data schema, prior findings, constraints, and output format.
2. **Collect and deduplicate sources** — run `scripts/context_bundler.py` to merge multiple context files into a single structured bundle; it deduplicates and applies the layering order.
3. **Check token budget** — run `scripts/token_counter.py` on the bundle to estimate token count; trim lower-priority layers if over budget (see `references/context_layering_guide.md` for trimming priority).
4. **Score context quality** — evaluate the bundle against `references/context_quality_rubric.md`; a good bundle scores ≥ 7/10 on completeness, clarity, and relevance.
5. **Write the prompt header** — prepend a clear task statement to the bundle: what you need, what output format you expect, and any hard constraints.
6. **Save the package** — store the bundle using `assets/context_package_template.md` so it can be reused or updated for follow-up sessions.

# Inputs the skill needs

# When to use

Before any analysis that will influence a significant decision is delivered to stakeholders. Peer review should be part of the standard delivery checklist for: dashboards going into production, reports used for strategic decisions, A/B test conclusions, and any analysis that will be cited externally.

# Process

1. **Agree scope of review** — clarify with the author what kind of review is needed: logic check, statistical validity, code review, or presentation clarity. Use `references/peer_review_framework.md` to set expectations.
2. **Review analytical rigour** — work through `references/analytical_rigor_checklist.md`: are the question and method aligned? Are assumptions valid? Is the conclusion supported by the data?
3. **Review code or SQL** — if the analysis involves code, apply `references/code_review_for_analysis.md`: reproducibility, correctness, readability, and performance.
4. **Write feedback** — use the feedback structure in `assets/peer_review_template.md`: must-fix issues, should-fix suggestions, and optional improvements. Be specific; "this is unclear" is not actionable.
5. **Author responds** — the author addresses each point and notes disposition (fixed / accepted as-is with rationale / deferred); use `assets/review_response_template.md`.
6. **Close the review** — reviewer confirms must-fix items are resolved and signs off; document the outcome in `assets/peer_review_template.md`.

# Inputs the skill needs