outcome-review

Outcome Review

Query analytics for a shipped feature, synthesize an outcome report, and optionally create follow-up Jira work. Entered independently after shipping — days or weeks later.

Step 1: Detect Available Tools

Check which MCP tools are available:

Tier 1 — PostHog MCP: If you have access to query-run, get-experiment, list-experiments, get-feature-flag, or create-annotation as MCP tools, use Tier 1.

Tier 2 — Manual Metrics: If no PostHog MCP tools are available, ask the user to provide metrics directly:

"I don't have PostHog MCP access. Please share any of the following:

• Dashboard screenshots or metric summaries
• Adoption numbers, funnel data, or error rates
• Experiment results if applicable
• Any specific concerns about the shipped feature"

Step 2: Identify the Feature

1. Check for a learn baseline file in ~/.claude/.skill-learn-baselines/:
   • List files, match by feature name or branch name from the user's prompt
   • If found, use the baseline's shipped_at, ship_method, hypotheses, and jira_ticket fields
   • If ship_method is "pull_request", verify the PR was actually merged before proceeding (check pr_url via gh pr view)
2. If no baseline found, ask the user:

   "Which feature should I review? Please provide the feature name, branch name, or Jira ticket ID."

Step 3: Gather Metrics

Tier 1 (PostHog MCP available):

1. Query adoption metrics via query-run with HogQL:
   • Event counts for the feature's key events since shipped_at
   • Compare to the period before shipping (same duration)
   • If the baseline has non-null hypotheses, use each hypothesis's metric field to target specific events/properties instead of generic adoption queries
2. Check experiment results if applicable:
   • list-experiments to find experiments linked to the feature
   • get-experiment for results, significance, and variant performance
3. Check feature flag status:
   • get-feature-flag for rollout percentage and targeting rules
4. Check error rates:
   • query-run for error events associated with the feature

Tier 2 (Manual):

1. Ask the user to share metrics from their dashboards
   • If the baseline has hypotheses, present each hypothesis and its metric to the user: "For H1 ([description]), I need the current value of [metric]. What is it?"
2. Ask about any observed regressions or improvements
3. Synthesize from what the user provides

Step 4: Synthesize Outcome Report

Present a structured report:

Outcome Report

Feature: [name] | Shipped: [date] | Branch: [name]

Adoption: [metrics summary — event counts, trend direction, comparison to pre-ship baseline]

Quality: [error rates, regression indicators]

Experiments: [results if applicable — significance, winning variant, effect size]

Assessment: One of:

• Positive — Metrics improved, no regressions. Close the loop.
• Regression detected — [specific metric] degraded by [amount]. Investigate.
• Inconclusive — Insufficient data. Revisit in [N] days.
• Mixed — [positive metrics] improved but [negative metrics] regressed. Judgment call.

Hypothesis Validation (when baseline has non-null hypotheses):

ID	Hypothesis	Metric	Baseline	Target	Actual	Status
H1	[description]	[metric]	[baseline]	[target]	[measured value]	[status]

Status values:

• Confirmed — Actual meets or exceeds target
• Not confirmed — Actual does not meet target
• Inconclusive — Insufficient data, or validation window has not elapsed
• Partially confirmed — Directionally correct but below target threshold

When hypotheses is null in the baseline (or no baseline found): skip this section entirely. Fall back to the existing generic metrics flow with no behavioral change.

Recommendations: Specific next actions based on the assessment.

Step 5: User Decision Gate

Present the report and ask:

"Based on this outcome review, would you like me to:

1. Close the loop — no follow-up needed
2. Create follow-up Jira tickets — I'll draft tickets for the recommended actions (requires your approval before creation)
3. Investigate further — dig deeper into a specific metric or regression"

Wait for the user's choice.

Step 6: Follow-Up Actions

If "Create follow-up tickets" (and Atlassian MCP available):

1. Draft the ticket(s) — title, description, acceptance criteria, priority
2. Present each draft to the user for approval
3. Only after explicit approval: createJiraIssue to create the ticket
4. addCommentToJiraIssue on the original ticket with the outcome summary

If Atlassian MCP unavailable:

"I don't have Atlassian MCP access. Here are the recommended follow-up tickets — please create them manually: [formatted ticket descriptions]"

Step 7: Transition

If follow-up work was identified:

"If follow-up work is needed, invoke Skill(auto-claude-skills:product-discovery) or Skill(superpowers:brainstorming) to begin the next cycle."

If the loop is closed:

"Outcome review complete. The feature loop is closed."