pr-review-loop

PR Review Loop

⛔ STOP - READ THIS FIRST ⛔

1. Locate the scripts directory

FIRST, use the Glob tool to find the scripts:

Glob pattern: **/pr-review-loop/*/scripts/commit-and-push.sh
Path: ~/.claude/plugins/cache

This gives you the full absolute path to the scripts directory.

Then use the full literal path for every script call. For example:

/home/user/.claude/plugins/cache/devon-claude-skills/pr-review-loop/1.0.0/skills/pr-review-loop/scripts/commit-and-push.sh "msg"

NEVER use variables like $SCRIPTS/commit-and-push.sh — this breaks permission matching. NEVER use compound commands like export PATH=... && commit-and-push.sh — this also breaks permissions. Always inline the full absolute path in every Bash call.

2. No raw git commands

❌ FORBIDDEN	✅ USE INSTEAD
git commit	scripts/commit-and-push.sh "msg"
git commit -m "..."	scripts/commit-and-push.sh "msg"
git push	scripts/commit-and-push.sh "msg"
git push origin	scripts/commit-and-push.sh "msg"

If you use git commit or git push directly, it will be BLOCKED.

---

⚠️ Do NOT Run as a Background Task Agent

The PR review loop MUST run in the main conversation, NOT as a background Task.

Tasks cannot spawn sub-Tasks. If the review loop runs as a Task, agent reviewers (from AGENT-REVIEWERS.md) will silently fail to spawn — they require the Task tool, which is only available in the main conversation.

What to do instead: Execute the review loop steps directly in the main conversation. This allows you to spawn agent reviewer Tasks in parallel (step 5 of each round) while keeping Gemini/bot comment handling inline.

Individual agent reviewers (leaf-level Tasks that don't need to spawn further Tasks) should still be spawned via the Task tool — that works because they're called from the main conversation, not from within another Task.

---

Streamline the push-review-fix cycle for PRs with automated reviewers.

Supported Review Bots

Bot	Trigger	Priority Format
Gemini Code Assist	/gemini review comment	![critical], ![high], ![medium], ![low]
Cursor Bugbot	Auto on push	<!-- **High Severity** -->, ### Bug:
Claude	Manual via script	**Critical**, ### Critical Issues

Priority detection automatically parses all formats when summarizing and fetching comments.

Comment Formats: Line Comments vs PR Comments

Different bots use different comment formats:

Line Comments (Gemini, Cursor)

• Posted on specific lines of the diff
• Each issue is a separate comment thread
• Reply using reply-to-comment.sh <PR> <comment-id> "Fixed"
• Thread auto-resolves when replied to

PR Comments (Claude)

• Single large comment on the PR (not on specific lines)
• Contains multiple issues organized by sections (e.g., "Issues & Concerns")
• Issues reference file:line locations in prose (e.g., "Location: role_summaries_controller.rb:60")
• Reply to the comment addressing multiple issues at once

Handling Claude's PR comments:

1. Fetch the comment to get the comment ID:

   gh pr view <PR> --json comments --jq '.comments[] | select(.author.login == "claude") | {id: .id, body: .body}'
   

2. Parse issues from the structured markdown:

   • Look for numbered headings like ### **1. BREAKING CHANGE: ...**
   • Extract file:line references from **Location:** lines
   • Note priority indicators: 🚨 (critical), ⚠️ (warning), etc.

3. Reply with a consolidated response addressing each issue:

   gh pr comment <PR> --body "## Response to Claude Review
   
   **Issue 1 (Breaking API change):** Fixed - added deprecation support for old parameter name
   
   **Issue 2 (Missing validation):** Fixed - added ALLOWED_VIEW_ROLES validation
   
   **Issue 3 (Logic change):** Won't fix - this is intentional, documented in PR description
   
   **Issue 4 (Missing tests):** Fixed - added RSpec tests for SystemRoles methods
   "
   

4. Individual issues can also be addressed via separate replies if preferred:

   gh pr comment <PR> --body "**Re: Issue #2 (Missing Input Validation):** Fixed in commit abc123 - added validation for view_role parameter"
   

The key difference: Claude comments don't have "threads" to resolve - you reply to the main PR comment referencing which issues you're addressing.

Critical: Be Skeptical of Reviews

Not all suggestions are good. Evaluate each review comment critically:

• Does this actually improve the code, or is it pedantic?
• Is this suggestion appropriate for the project's context?
• Would implementing this introduce unnecessary complexity?

Skip suggestions that are:

• Platform-specific when not applicable (Windows comments for Linux-only code)
• Overly defensive (excessive null checks, unlikely edge cases)
• Stylistic preferences that don't match project conventions
• Adding documentation for self-explanatory code

When in doubt, ask the user rather than blindly applying changes.

Diminishing Returns Detection

Track review rounds. After 2-3 iterations, evaluate:

• Are new comments addressing real issues or nitpicks?
• Are we fixing the same type of issue repeatedly?
• Are reviewers finding fewer/lower-priority issues?

ONE MORE LOOP rule: When a full round (Gemini + other bots + agent reviewers) produces no actionable feedback (or only "Won't fix" responses), do ONE additional round to catch any final feedback.

Tracking state: Use TodoWrite to track whether you're in the "final verification round". Create a todo like "Final verification round - if no actionable feedback, ready to merge".

Reset condition: If the final verification round produces feedback you actually fix (not just "Won't fix"), remove the "final verification round" todo - you need a fresh "one more" after pushing those fixes.

Exit condition: If the "final verification round" todo exists AND the round produces no actionable feedback (only nitpicks/won't-fix responses), you're done - ask about merge.

After the final round, ask the user: "We've completed the review cycles. Ready to merge, or want to address more?"

Autonomous Loop Workflow

CRITICAL RULES - NEVER VIOLATE THESE:

1. ALWAYS use full absolute paths for scripts - Glob once to find the scripts directory, then inline the full path in every Bash call. NEVER use variables or compound commands (see setup at top of document)
2. ALWAYS use commit-and-push.sh - NEVER git commit or git push (see table at top of document)
3. ALWAYS reply to EVERY comment:
   • Line comments (Gemini, agents): use reply-to-comment.sh
   • PR comments (Claude): use gh pr comment with consolidated response
4. ALWAYS use --wait flag when checking for comments - this ensures proper 5-minute polling
5. PR creation automatically triggers Gemini review - use get-review-comments.sh --wait to wait for the first review

The Loop

⚠️ CRITICAL: Each round includes ALL reviewer types before triggering the next cycle.

DO NOT run multiple Gemini rounds before checking other reviewers. After addressing Gemini comments, you MUST check for other bot comments and run agent reviewers BEFORE triggering another Gemini review. This interleaving is essential because:

• Fixes made for Gemini may resolve issues other reviewers would have flagged
• Running all Gemini rounds first makes other reviewer feedback stale and irrelevant

EACH ROUND (all steps before next review trigger):
┌─────────────────────────────────────────────────────────────┐
│ 1. Get Gemini comments (--wait only on first check)         │
│ 2. Address ALL Gemini comments                              │
│ 3. Check for other bot PR comments (Claude, Cursor, etc.)   │
│ 4. Address ALL other bot comments                           │
│ 5. Run agent reviewers (AGENT-REVIEWERS.md)                 │
│ 6. Address ALL agent comments                               │
│ 7. Commit and push (if ANY fixes were made in steps 2,4,6)  │
│ 8. Trigger next review (--wait)                             │
└─────────────────────────────────────────────────────────────┘
                              │
                              ▼
                    Go to step 1 (next round)

Step details:

1. Get unresolved Gemini LINE comments:
   • First round after PR creation/push: use --wait to poll up to 5 minutes
   • Subsequent rounds: --wait is already triggered by step 8
2. Address Gemini line comments:
   • Fix it → reply "Fixed - ..."
   • Bad suggestion → reply "Won't fix - ..."
   • Good but out of scope → create beads ticket (if available), reply "Out of scope - tracked in BD-XXX"
3. Check for other bot PR comments (Claude, Cursor, Copilot):
   • These are single PR comments containing multiple issues
   • Parse the structured markdown to extract individual issues
4. Address other bot comments:
   • Reply using reply-to-comment.sh <PR> <comment-id> "response" (handles PR comments)
5. Run agent reviewers (if AGENT-REVIEWERS.md exists and agents not retired):
   • Spawn non-retired agents as parallel Tasks
   • Wait for all agents to return
6. Address agent comments:
   • Same fix/wontfix/out-of-scope flow as Gemini
   • Track per-agent diminishing returns, retire unproductive agents
7. Commit and push (NEVER raw git commands) - if ANY fixes were made in this round
8. Trigger next review: trigger-review.sh <PR> --wait
9. Go to step 1

COMPLETION: When a full round produces no actionable feedback (Gemini + other bots + agents all stable) AND this was the "final verification" round:

• Report all beads tickets created during the loop (if any)
• Ask user about merge

Step-by-step (Each Round)

1. Check for unresolved Gemini line comments (ALWAYS use --wait for first check after PR creation or push):

scripts/summarize-reviews.sh <PR>
scripts/get-review-comments.sh <PR> --with-ids --wait

The --wait flag polls every 30s for up to 5 minutes, waiting for Gemini to respond. Do NOT skip this or use a shorter timeout.

The --with-ids flag outputs comment IDs needed for replies. Example output:

=== Comment ID: 2710906366 | Node ID: PRRC_kwDOD3ZsRc6hlSX- ===
File: pfsrd2/equipment.py:84
Priority: high

The function appears to duplicate functionality...

Use the Node ID (PRRC_...) when replying to comments. The Node ID is required for reply-to-comment.sh to properly attach your reply to the review thread.

2. Address Gemini line comments (MANDATORY - never skip this):

• Evaluate if suggestion is worthwhile
• Apply fix locally OR decide to skip
• ALWAYS reply using the script with the Node ID - this resolves the thread:

# Use the Node ID (PRRC_...) from get-review-comments.sh --with-ids output
scripts/reply-to-comment.sh <PR> PRRC_kwDOD3ZsRc6hlSX- "Fixed - description"
# OR for bad/inappropriate suggestions:
scripts/reply-to-comment.sh <PR> PRRC_kwDOD3ZsRc6hlSX- "Won't fix - reason"
# OR for good suggestions outside PR scope - see "Out of Scope Suggestions" section:
scripts/reply-to-comment.sh <PR> PRRC_kwDOD3ZsRc6hlSX- "Out of scope - tracked in BD-XXX"

3. Check for other bot PR comments (Claude, Cursor, Copilot):

These bots post single PR comments (not line comments) containing multiple issues:

# Find Claude's review comment
gh pr view <PR> --json comments --jq '.comments[] | select(.author.login == "claude") | {id: .id, body: .body[:500]}'

For each issue in the comment:

• Parse the structured markdown (numbered issues, file:line references)
• Apply fix/wontfix/out-of-scope decision for each issue
• Reply to the PR comment with a consolidated response:

gh pr comment <PR> --body "## Response to Claude Review

**Issue 1 (name):** Fixed - description
**Issue 2 (name):** Won't fix - reason
**Issue 3 (name):** Out of scope - tracked in BD-XXX
"

4. Run agent reviewers (if AGENT-REVIEWERS.md exists):

• Spawn non-retired agents as parallel Tasks
• Wait for all agents to return
• Address agent comments (fix/wontfix/out-of-scope flow)
• Track per-agent diminishing returns

5. Commit and push (ALWAYS use the script, NEVER raw git) - if any fixes were made:

scripts/commit-and-push.sh "fix: description"

This script runs pre-commit, commits with proper footer, and pushes.

6. Trigger next review and wait for response:

scripts/trigger-review.sh <PR> --wait

The --wait flag polls for up to 5 minutes until new comments appear. Do NOT use sleep or manual polling.

7. When new reviews detected, go to step 1

Reply Templates

For Line Comments (Gemini, agent reviewers)

ALWAYS reply using reply-to-comment.sh. Templates:

• Fixed: "Fixed - [description]"
• Won't fix (bad suggestion): "Won't fix - [reason]"
• Out of scope (good suggestion): "Out of scope - tracked in BD-XXX" (see below)
• Deferred: "Good catch, tracking in #issue"
• Acknowledged: "Acknowledged - [explanation]"

For PR Comments (Claude)

Reply using gh pr comment with a consolidated response:

## Response to Claude Review

**Issue 1 (Breaking API change):** Fixed - added backward compatibility

**Issue 2 (Missing validation):** Fixed - added input validation in controller

**Issue 3 (Behavior change):** Won't fix - intentional, see PR description

**Issue 4 (Missing tests):** Out of scope - tracked in BD-XXX

Out of Scope Suggestions (Beads Integration)

When a review suggestion is good but outside the scope of this PR, capture it for later rather than losing it.

Decision tree:

1. Is this suggestion valid and would improve the code? → If NO, use "Won't fix"
2. Does it belong in this PR? → If YES, fix it
3. It's a good suggestion but out of scope → Create a beads ticket if available

Check if beads is installed:

bd --version

If beads is available, create a ticket:

bd create

Include in the ticket:

• Title: Brief description of the suggested improvement
• Description: Must include ALL context needed to fix the issue later:
  • The original review comment text
  • File path and line number(s) affected
  • PR number and link for context
  • The gh command to fetch the exact comment:

    gh api /repos/{owner}/{repo}/pulls/{PR}/comments/{comment-id}
    

  • Any relevant code snippets or context from the current PR

Then reply to the comment:

scripts/reply-to-comment.sh <PR> <comment-id> "Out of scope - tracked in BD-XXX"

Track the ticket for reporting at the end of the review loop. Keep a list of all created tickets (ID and title) to include in the completion summary.

If beads is NOT installed, just reply without creating a ticket:

scripts/reply-to-comment.sh <PR> <comment-id> "Out of scope for this PR"

Triggering Reviews by Bot Type

Gemini Code Assist (Default)

scripts/trigger-review.sh <PR> --gemini --wait

Gemini has a daily quota. When exceeded, the skill automatically detects this and suggests alternatives.

Cursor Bugbot

scripts/trigger-review.sh <PR> --cursor --wait

Cursor auto-reviews on push, so --cursor just waits for comments to appear (typically 1-2 minutes).

Claude Agent (Fallback)

scripts/trigger-review.sh <PR> --claude

Uses a Claude agent to review the PR and post comments. Useful when:

• Gemini is rate-limited
• You want a different perspective
• Cursor isn't configured on the repo

Claude Review Workflow

When using Claude fallback:

1. Run the script to get the prompt:

   scripts/claude-review.sh <PR>
   

2. Use the Task tool with the generated prompt:

   Task tool:
     subagent_type: general-purpose
     description: Review PR #<PR>
     prompt: (copy from /tmp/claude_review_prompt_<PR>.txt)
   

3. The agent will post the review as a PR comment

4. Continue the normal review loop - address comments using reply-to-comment.sh

Agent Reviewers (Custom Focused Reviews)

Run custom agent reviewers defined in AGENT-REVIEWERS.md files as part of each review round.

AGENT-REVIEWERS.md Format

Two types of H1 sections:

1. # Agents - Defines custom agent reviewers (H2 headings become agent names)
2. Any other H1 section - Treated as context/instructions for the main review loop (like CLAUDE.md)

# Agents

## security-reviewer
Focus on security vulnerabilities: injection attacks, auth bypasses, secrets in code,
unsafe deserialization, path traversal, etc. Flag anything that could be exploited.

## dry-reviewer
Identify code duplication. Look for repeated logic that could be abstracted.
Consider whether abstraction would actually improve maintainability.

# Guidelines

When reviewing this codebase:
- All database queries must use parameterized statements
- Prefer composition over inheritance
- Error messages should never expose internal paths or stack traces

# Context

This is a Ruby on Rails API backend. The frontend is a separate React app.
Authentication uses JWT tokens stored in httpOnly cookies.

How sections are used:

• # Agents → H2 headings define agent reviewers with their specialized instructions
• Other H1 sections (# Guidelines, # Context, etc.) → Passed to the main pr-review-loop task as context, similar to CLAUDE.md content

Hierarchical Scoping

AGENT-REVIEWERS.md files can exist at any directory level. Agents and context sections are scoped to their directory and below.

Resolution rules:

1. For each changed file, collect AGENT-REVIEWERS.md files from its directory up to repo root
2. Resolve sections by H1 name - lower directory definitions override same-named sections from above
3. For # Agents: each agent only reviews changes within its directory scope and below
4. For other H1 sections: winning version of each section (per override rule) is combined and passed to the review loop
5. Root-level definitions see all changes; subdirectory definitions see only their subtree

Example structure:

/AGENT-REVIEWERS.md                    # Agents: clarity-reviewer, dry-reviewer
                                       # Guidelines: general repo rules
/plugins/AGENT-REVIEWERS.md            # Agents: dry-reviewer (overrides root), plugin-structure-reviewer
                                       # Guidelines: plugin-specific rules (overrides root)
/plugins/auth/AGENT-REVIEWERS.md       # Agents: security-reviewer
                                       # Context: auth-specific context

Result for a PR touching plugins/auth/login.py and plugins/utils.py:

Agents:

• clarity-reviewer (from root) → sees all changes
• dry-reviewer (from /plugins/, overrides root) → sees all changes under /plugins/
• plugin-structure-reviewer (from /plugins/) → sees all changes under /plugins/
• security-reviewer (from /plugins/auth/) → sees only changes under /plugins/auth/

Context for main review loop:

• For plugins/auth/login.py: combines # Guidelines from /plugins/ + # Context from /plugins/auth/
• For plugins/utils.py: uses # Guidelines from /plugins/

Discovery algorithm:

# For each unique directory containing changed files:
# 1. Walk up to repo root collecting AGENT-REVIEWERS.md paths
# 2. Parse each file, building agent map (lower wins on name collision)
# 3. Track each agent's scope (directory where it was defined)

When to Run Agent Reviewers

Agent reviewers run as part of each review round, after addressing Gemini and other bot comments.

On the first round (or when new agents are discovered), discover agent reviewers:

# Get list of changed files
gh pr diff <PR> --name-only

# For each unique parent directory, check for AGENT-REVIEWERS.md up to repo root

Spawn non-retired agents in parallel at step 4 of each round. Track per-agent state to avoid re-running retired agents.

Spawning Agent Reviewers

For each agent discovered (after merging and scoping), spawn a Task:

Task tool:
  subagent_type: general-purpose
  model: sonnet
  description: <agent-name> review for PR #<PR>
  prompt: |
    You are the "<agent-name>" code reviewer for PR #<PR>.

    Your focus: <instructions from AGENT-REVIEWERS.md>

    **Your scope: <scope-path>/**
    You should ONLY review changes to files under this path. Ignore changes outside your scope.

    ## Your workflow:

    1. **Check your prior comments** (if any):
       ```bash
       scripts/get-agent-comments.sh <PR> <agent-name> --with-replies
       ```

    2. **Evaluate replies to your prior comments**:
       - If the response is reasonable (good explanation, valid fix, or acceptable tradeoff), do NOT re-raise
       - If the response is UNREASONABLE (dismissive, incorrect, or ignores the issue), reopen:
         ```bash
         scripts/reopen-comment.sh <PR> <comment-id> <agent-name> "Reopening - <reason why response is insufficient>"
         ```

    3. **Review the PR diff** (filter to your scope):
       ```bash
       gh pr diff <PR> -- '<scope-path>/*'
       ```
       If scope is repo root, use `gh pr diff <PR>` without path filter.

    4. **Post NEW findings** as line comments (only issues not already raised, only files in your scope):
       ```bash
       scripts/post-line-comment.sh <PR> <file> <line> <agent-name> "Issue description and suggestion"
       ```
       The script automatically adds `<!-- Agent: <agent-name> -->` signature.

    5. **Return** - Do NOT fix anything. Your job is review only.

    ## Important:
    - Be thorough but not pedantic - only flag real issues within your focus area
    - Consider project context - not all best practices apply everywhere
    - If you have no findings and all prior comments have reasonable responses, simply return "No issues found"

    Available scripts: See pr-review-loop skill documentation for full script reference.

Spawn all agents in parallel - use multiple Task tool calls in a single message.

Main Loop Integration

Agent reviewers are part of each round, running after Gemini and other bot comments are addressed:

1. After agent Tasks return, address agent comments using the same flow as Gemini:

   • Fix → reply "Fixed - ..."
   • Won't fix (bad) → reply "Won't fix - ..."
   • Out of scope (good) → create beads ticket if available, reply "Out of scope - tracked in BD-XXX"

2. Update per-agent tracking based on results (productive / final-verification / retired)

3. After all comments in the round are addressed (Gemini + other bots + agents):

   • Commit and push if any fixes: scripts/commit-and-push.sh "fix: address review comments"
   • Trigger next review: scripts/trigger-review.sh <PR> --wait
   • Start the next round

Diminishing Returns for Agent Reviewers

Apply the same heuristic as Gemini, but per agent:

• Track each agent with TodoWrite: "<agent-name>: final verification loop"
• After 2-3 cycles where an agent produces only nitpicks or "Won't fix" responses, enter final verification for that agent
• If an agent's final verification produces actual fixes, reset its state
• If an agent's final verification produces no actionable feedback, stop calling that agent

Example tracking:

- "security-reviewer: 2 cycles, still productive" (keep calling)
- "dry-reviewer: final verification loop" (one more, then retire if no actionable feedback)
- "error-handling-reviewer: retired" (no longer called)

Selective Agent Execution

As agents reach diminishing returns, stop calling them. The main loop should:

1. Parse AGENT-REVIEWERS.md to get all agent names
2. Track per-agent state (productive / final-verification / retired)
3. Only spawn Tasks for non-retired agents
4. Update state after each cycle based on results

This keeps the review loop efficient while ensuring all perspectives are heard at least once.

Rate Limit Detection

The scripts automatically detect Gemini quota limits by checking for:

"You have reached your daily quota limit"

When detected, the script suggests:

1. Use Cursor if available: --cursor --wait
2. Use Claude fallback: --claude

Scripts

Script	Purpose
commit-and-push.sh "msg"	ALWAYS USE - Never use raw git commit/push
reply-to-comment.sh <PR> <id> "msg"	ALWAYS USE - Reply and auto-resolve every comment
get-review-comments.sh <PR> [--with-ids] [--wait]	USE --wait - Fetch comments, polls 5min if --wait
trigger-review.sh [PR] [--gemini|--cursor|--claude] [--wait]	USE --wait - Trigger review and poll for response
summarize-reviews.sh <PR> [--all]	Summary of unresolved by priority/file
watch-pr.sh <PR>	Background monitor (optional, for long-running watches)
claude-review.sh <PR>	Generate Claude agent prompt for code review
check-gemini-quota.sh <PR>	Check if Gemini is rate-limited
resolve-comment.sh <node-id> [reason]	Manually resolve a thread
post-line-comment.sh <PR> <file> <line> <agent> "msg"	Post line comment with agent signature
get-agent-comments.sh <PR> <agent> [--with-replies]	Fetch agent's own comments and replies
reopen-comment.sh <PR> <comment-id> <agent> "reason"	Reply to resolved thread with Claude attribution

Permission Setup

To enable autonomous loops, user should grant access:

Bash(scripts/commit-and-push.sh:*)
Bash(scripts/reply-to-comment.sh:*)
Bash(scripts/trigger-review.sh:*)
Bash(scripts/post-line-comment.sh:*)
Bash(scripts/get-agent-comments.sh:*)
Bash(scripts/reopen-comment.sh:*)

Prerequisites

• gh CLI authenticated
• jq - JSON processor for parsing GitHub API responses Install with: apt install jq (Ubuntu/Debian) or brew install jq (macOS)
• pre-commit (optional) - If .pre-commit-config.yaml exists in the repo, pre-commit will be run. If pre-commit is not installed, a warning is shown but commits proceed. Install with: pip install pre-commit && pre-commit install
• bd (beads) (optional) - Issue tracker for capturing out-of-scope suggestions as tickets. If not installed, out-of-scope suggestions are handled with "Out of scope for this PR" replies. See: https://github.com/steveyegge/beads