From data-analysis
Quality-checks data analyses before sharing: verifies joins, aggregations, denominators, time ranges, metrics. Detects pitfalls like survivorship bias, average-of-averages, join explosion. Provides checklists and reproducible docs templates.
npx claudepluginhub vm0-ai/vm0-skills --plugin user-researchThis skill uses the workspace's default tool permissions.
Pre-delivery review process for catching errors, validating results, and ensuring analyses are reproducible and trustworthy.
QA data analyses for methodology, accuracy, biases, and pitfalls before stakeholder sharing. Spot-checks calculations, SQL results, visualizations, and conclusions.
Validates CSV/TSV/Excel files and data analyses for quality, completeness, uniqueness, accuracy, consistency, outliers, and bias using qsv stats and frequency tools.
Reviews data analysis methodology and quality in Phase 4 of /ds workflow. Chooses single or parallel reviewer strategies, monitors context, and controls tool usage via hooks.
Share bugs, ideas, or general feedback.
Pre-delivery review process for catching errors, validating results, and ensuring analyses are reproducible and trustworthy.
Work through every section below before presenting findings to stakeholders.
What goes wrong: Joining two tables with a many-to-many relationship silently multiplies rows, blowing up counts and sums.
Detection method:
-- Compare row counts before and after the join
SELECT COUNT(*) FROM orders; -- 1,000
SELECT COUNT(*) FROM orders o
JOIN line_items li ON o.id = li.order_id; -- 3,500 (unexpected inflation)
Prevention:
COUNT(DISTINCT o.id) to count entities accurately through multi-row joinsWhat goes wrong: The analysis only covers entities that still exist, ignoring those that were removed, churned, or failed.
Typical scenarios:
Prevention: Before drawing conclusions, ask: "Who is absent from this dataset, and would their presence change the story?"
What goes wrong: A month, week, or quarter that is still in progress gets compared to a completed one.
Typical scenarios:
Prevention: Restrict comparisons to completed periods, or normalize by matching the same number of elapsed days.
What goes wrong: The population used as a denominator changes between periods, making rate comparisons invalid.
Typical scenarios:
Prevention: Lock in consistent definitions across every period being compared. Flag any definition changes.
What goes wrong: Taking the mean of group-level averages ignores differences in group size, producing an incorrect overall figure.
Illustration:
Prevention: Always compute averages from individual records. Never take the mean of already-aggregated means.
What goes wrong: Different source systems record timestamps in different zones, causing misaligned daily rollups and join mismatches.
Typical scenarios:
Prevention: Convert all timestamps to a single reference zone (UTC is the safest default) before any analysis. State the timezone in the deliverable.
What goes wrong: Segments are defined using the very outcome being measured, creating tautological findings.
Typical scenarios:
Prevention: Base segment definitions on characteristics measured before the outcome period, not on the outcome itself.
| Metric Category | Validation Approach |
|---|---|
| User counts | Cross-reference against known DAU/MAU benchmarks |
| Revenue totals | Compare to known ARR or recent financial reports |
| Conversion rates | Must be 0-100%; compare to dashboard baselines |
| Growth rates | Is 50%+ month-over-month realistic, or does it signal a data problem? |
| Averages | Given the distribution, does this number feel right? |
| Segment shares | Do percentage breakdowns sum to approximately 100%? |
Every substantial analysis should ship with this documentation:
## Analysis: [Title]
### Business Question
[The precise question this work answers]
### Sources
- Table: [schema.table_name] (snapshot date: [date])
- Table: [schema.other_table] (snapshot date: [date])
- External file: [filename] (origin: [description])
### Metric and Segment Definitions
- [Metric A]: [Precise calculation formula]
- [Segment X]: [Exact inclusion/exclusion criteria]
- [Time window]: [Start] through [end], [timezone]
### Analytical Approach
1. [First step and its purpose]
2. [Second step]
3. [Third step]
### Assumptions and Known Limitations
- [Assumption and why it holds]
- [Limitation and its potential effect on conclusions]
### Results
1. [Finding with supporting evidence]
2. [Finding with supporting evidence]
### Queries
[All SQL and code used, annotated with comments]
### Warnings for the Reader
- [Anything the audience should weigh before acting on these results]
For SQL or Python that others may reuse:
"""
Title: Monthly Cohort Retention
Author: [Name]
Created: [Date]
Sources: events, users
Last cross-checked: [Date] — matched dashboard within 2%
Objective:
Build monthly retention cohorts anchored on each user's first event date.
Assumptions:
- "Active" = at least one recorded event in the calendar month
- Internal and test accounts excluded (user_type != 'internal')
- All timestamps normalized to UTC
Output:
Retention grid: rows are cohort months, columns are months since first event.
Cell values are retention percentages (0-100).
"""