ctx-prompt-audit

Analyze recent session transcripts to identify prompts that led to unnecessary clarification back-and-forth.

Before Auditing

1. Check for session data: look in the journal directory for transcripts to analyze
2. Need at least 3 sessions: fewer than that gives too small a sample; tell the user to try again later
3. Confirm scope: if the user specifies sessions or a date range, use that; otherwise default to the 5 most recent

When to Use

• Periodically to help users improve their prompting
• When the user asks for feedback on their prompting style
• After noticing many clarification cycles in recent sessions
• After a session with unusually high back-and-forth

When NOT to Use

• Immediately after a user's first session (not enough data)
• When the user is frustrated; coaching lands poorly when someone is already annoyed
• Unsolicited; only run when the user invokes it or explicitly asks for feedback

Usage Examples

/ctx-prompt-audit
/ctx-prompt-audit --sessions 10
/ctx-prompt-audit 2026-01-24

Data Sources

Session transcripts are stored in the journal:

Source	Format
Journal directory	Exported session journals (richer)

Journal entries contain full turn-by-turn conversation and are the best source for pattern detection.

Process

1. Gather transcripts: read 3-5 recent sessions from the journal
2. Extract user prompts: isolate the human turns
3. Identify vague prompts: flag those that caused clarifying questions (see criteria below)
4. Cross-reference patterns: look for repeated habits across sessions, not one-off mistakes
5. Generate coaching report: use the output format below
6. Present and discuss: share the report, ask if the user wants to dig into any example

What Makes a Prompt "Vague"

Look for prompts where the agent asked clarifying questions instead of acting:

• Missing file context: "fix the bug" without specifying which file or error
• Ambiguous scope: "optimize it" without what to optimize or success criteria
• Undefined targets: "update the component" when multiple components exist
• Missing error details: "it's not working" without symptoms
• Vague action words: "make it better", "clean this up"

Important Nuance

Not every short prompt is vague. Consider context:

• "fix the bug" after discussing a specific error: not vague
• "fix the bug" as the first message: vague
• "same:" after a pattern is established: not vague (the user set a convention and is being efficient)
• Shorthand that references shared context is good prompting, not lazy prompting

Output Format

## Prompt Audit Report

**Sessions analyzed**: 5
**User prompts reviewed**: 47
**Vague prompts found**: 4 (8.5%)

---

### Example 1: Missing File Context

**Your prompt**: "fix the bug"

**What happened**: I had to ask which file and what error.

**Better prompt**: "fix the authentication error in
src/auth/login.ts where JWT validation fails with 401"

---

## Patterns to Watch

Based on your sessions, you tend to:
1. Skip mentioning file paths (3 occurrences)
2. Use "it" without establishing what "it" refers to
   (2 occurrences)

## What You Do Well

- You provide error output when debugging (4 of 5 sessions)
- You reference specific files by path in most prompts

## Tips

- Start prompts with the **file path** when discussing
  specific code
- Include **error messages** when debugging
- Specify **success criteria** for optimization tasks

Guidelines

• Constructive, not critical: frame suggestions as improvements, not corrections
• Show actual prompts: quote from their sessions so examples are concrete, not hypothetical
• Explain the consequence: what happened because the prompt was vague (extra round-trip, wrong file edited, etc.)
• Provide rewrites: show a concrete better alternative for each example
• Acknowledge strengths: include a "What You Do Well" section; people learn better when not purely criticized
• Look for patterns: one vague prompt is noise; three of the same kind is a habit worth addressing
• End with actionable tips: 3-5 specific, memorable tips

Quality Checklist

Before presenting the report, verify:

• At least 3 sessions were analyzed (not a tiny sample)
• Every "vague" example includes the actual quoted prompt
• Every example has a concrete rewrite (not just "be more specific")
• Context was considered (short != vague)
• Report includes positive observations, not just criticism
• Tips are specific to this user's patterns, not generic advice