**Description**: Review project history and coordinate agents to identify 3-5 specific process improvements, outputting recommendations as an ADR in IMPROVEMENTS.md
Review project history and coordinate agents to identify 3-5 specific process improvements, outputting recommendations as an ADR in IMPROVEMENTS.md
/plugin marketplace add laird/agents/plugin install modernize@plugin-marketplaceDescription: Review project history and coordinate agents to identify 3-5 specific process improvements, outputting recommendations as an ADR in IMPROVEMENTS.md
Version: 1.0
Purpose: Coordinate agents to review project history and identify process improvements
Team: All specialist agents (Migration Coordinator, Security, Architect, Coder, Tester, Documentation)
Output: IMPROVEMENTS.md (ADR format with 3-5 specific recommendations)
Duration: 2-4 hours
Note: Time estimates are based on typical human execution times and may vary significantly based on project complexity, team experience, and AI assistance capabilities.
This protocol orchestrates a multi-agent retrospective to analyze project history, identify inefficiencies, bottlenecks, risks, and agent behavioral issues, then produce a unified set of 3-5 specific, actionable recommendations for process improvement.
Improvements Target:
Core Principle: Continuous improvement through systematic reflection, learning from mistakes, and evidence-based recommendations.
CRITICAL: User interruptions and corrections are the strongest signal that agents need behavioral improvement. Every user correction should be analyzed and potentially trigger a recommendation.
Objective: Gather data from project history to understand what happened
Migration Coordinator (Lead):
# Analyze HISTORY.md for patterns
- Read complete HISTORY.md
- Identify all phases completed
- Extract timeline data (estimated vs actual)
- Note any blockers or delays
- Document quality gate failures
Data to Extract:
Architect Agent:
# Find and analyze all ADRs
find . -name "ADR-*.md" -o -name "adr-*.md" -o -path "*/docs/adr/*"
Analysis:
Tester Agent:
# Analyze test patterns
- Review test pass rates over time
- Identify flaky tests
- Check coverage evolution
- Note test infrastructure issues
Patterns to Identify:
Security Agent:
# Analyze security remediation
- Review vulnerability scan results over time
- Check security score progression
- Identify CVEs that took longest to fix
- Note dependency security issues
Analysis:
Coder Agent:
# Analyze git history
git log --all --oneline --graph
git log --all --numstat --pretty="%H" | awk 'NF==3 {plus+=$1; minus+=$2} END {printf("+%d, -%d\n", plus, minus)}'
git shortlog -sn
Patterns to Identify:
Documentation Agent:
# Analyze documentation completeness
- Check CHANGELOG.md
- Review MIGRATION-GUIDE.md
- Analyze ADRs
- Check README updates
Analysis:
All Agents:
# Analyze conversation history for agent issues
- Review git commit messages for user corrections
- Check HISTORY.md for user interventions
- Identify cases where user had to interrupt
- Find instances of wasted effort or wrong approaches
- Note misunderstandings of requirements
CRITICAL Signals of Agent Problems:
Examples of Agent Mistakes to Identify:
Wrong Tool Usage:
find command instead of Glob toolgrep instead of Grep toolcat instead of Read toolWasted Effort:
Context Ignorance:
Requirement Misunderstanding:
Poor Planning:
Communication Issues:
Data Sources for Agent Error Analysis:
# Git commit messages with corrections
git log --all --grep="fix\|correct\|actually\|oops\|mistake"
# Search HISTORY.md for user interventions
grep -i "user:\|correction\|fix\|reverted\|undo" HISTORY.md
# Look for reverted commits
git log --all --oneline | grep -i "revert\|undo"
# Find large time gaps (might indicate stuck agent)
git log --all --format="%ai %s" | awk '{print $1, $2}' | sort
IMPORTANT: User interventions are the strongest signal that agents need behavioral improvement. Every user correction should trigger a recommendation.
Objective: Each agent identifies problems and opportunities from their perspective
Format: Each agent creates a structured list of observations
## [Agent Name] Observations
### What Went Well
1. [Positive observation with evidence]
2. [Positive observation with evidence]
### What Could Be Improved
1. [Problem/inefficiency with specific examples]
2. [Problem/inefficiency with specific examples]
3. [Problem/inefficiency with specific examples]
### Specific Recommendations
1. [Actionable recommendation]
2. [Actionable recommendation]
All Agents Contribute:
Objective: Migration Coordinator synthesizes all agent observations to identify common themes
Process:
Common Pattern Examples:
Protocol Issues:
Agent Behavioral Issues (CRITICAL):
LLM-to-Code Opportunities:
Context Window Optimization:
Objective: Collaboratively develop 3-5 specific, actionable recommendations
Each recommendation MUST be:
### Recommendation [N]: [Title]
**Problem**: [What inefficiency or risk does this address?]
**Evidence**: [Specific examples from project history]
**Proposed Change**: [Exact change - can be protocol update, agent behavior, or automation]
**Change Type**: [Protocol Update / Agent Behavior / Automation / Tool Usage / Documentation]
**Expected Impact**: [Quantifiable improvement in time, quality, or risk]
**Implementation Complexity**: [Low/Medium/High]
**Affected Components**:
- **Agents**: [List of agents that will change behavior]
- **Protocols**: [List of protocol files to update]
- **Automation**: [Scripts, hooks, CI/CD to add]
- **Commands**: [Command files to modify]
Objective: Document recommendations as an Architectural Decision Record
Format: MADR 3.0.0 (Markdown Architectural Decision Records)
# Process Improvement Recommendations
**Date**: [YYYY-MM-DD]
**Status**: Proposed
**Decision Makers**: All Agent Team
**Project**: [Project Name]
**Retrospective Period**: [Start Date] - [End Date]
---
## Context and Problem Statement
Following completion of [phases/milestones], the agent team conducted a retrospective analysis of project history to identify opportunities for process improvement. This document presents evidence-based recommendations to improve efficiency, quality, and robustness of the modernization process.
**Analysis Sources**:
- HISTORY.md (project timeline and events)
- Git commit history ([N] commits analyzed)
- ADRs ([N] decisions reviewed)
- Test results and quality metrics
- Security scan results
- Documentation artifacts
**Key Metrics from This Project**:
- Timeline: [Estimated X days, Actual Y days, Delta Z%]
- Quality Gate Failures: [N instances]
- Fix-and-Retest Cycles: [N cycles]
- Security Remediation: [N CVEs, X days to resolve]
- Test Coverage: [Start X% → End Y%]
- [Other relevant metrics]
---
## Decision Drivers
* **Efficiency**: Reduce time to complete modernization phases
* **Quality**: Improve first-time quality, reduce rework
* **Risk Reduction**: Catch issues earlier in the process
* **Coordination**: Improve agent collaboration and handoffs
* **Automation**: Reduce manual effort and human error
---
## Recommendations
### Recommendation 1: [Title]
**Status**: Proposed
#### Problem
[Detailed description of the inefficiency, bottleneck, or risk]
#### Evidence
[Specific examples from project history with data]
**Examples from this project**:
- [Example 1 with timestamp/reference]
- [Example 2 with timestamp/reference]
- [Example 3 with timestamp/reference]
**Quantified Impact**:
- [Metric 1]: [Value]
- [Metric 2]: [Value]
#### Proposed Change
[Exact change to protocols, process, or agent behavior]
**Protocol Changes**:
- **File**: [protocol file or section]
- **Current**: [What happens now]
- **Proposed**: [What should happen]
**Agent Behavior Changes**:
- **[Agent 1]**: [Specific behavior change]
- **[Agent 2]**: [Specific behavior change]
#### Expected Impact
**Efficiency Gains**:
- Estimated time savings: [X hours/days per phase]
- Reduced rework: [Y% reduction]
**Quality Improvements**:
- [Specific quality metric improvement]
**Risk Reduction**:
- [Specific risk mitigation]
#### Implementation
**Effort**: [Low/Medium/High] - [X hours estimated]
**Steps**:
1. [Step 1]
2. [Step 2]
3. [Step 3]
**Validation**:
- [ ] [How to verify the change was effective]
- [ ] [Metric to track improvement]
#### Affected Components
- **Agents**: [List of affected agents]
- **Protocols**: [List of protocol documents]
- **Tools**: [Any new tools or automation needed]
---
[Repeat for Recommendations 2-5]
---
## Summary
| Recommendation | Impact | Effort | Priority | Estimated Savings |
|----------------|--------|--------|----------|-------------------|
| 1. [Title] | High | Medium | P0 | [X days/phase] |
| 2. [Title] | High | Low | P0 | [Y days/phase] |
| 3. [Title] | Medium | Low | P1 | [Z days/phase] |
| 4. [Title] | Medium | Medium | P1 | [W days/phase] |
| 5. [Title] | Low | Low | P2 | [V days/phase] |
**Total Estimated Impact**: [Combined efficiency gains, quality improvements, risk reduction]
---
## Implementation Plan
### Phase 1: Immediate Changes (Apply First)
- Recommendation [N]: [Title] - Can be applied immediately
- Recommendation [M]: [Title] - Can be applied immediately
### Phase 2: Short-term Changes (Next Project)
- Recommendation [X]: [Title] - Apply at start of next modernization
- Recommendation [Y]: [Title] - Apply at start of next modernization
### Phase 3: Long-term Changes (Strategic)
- Recommendation [Z]: [Title] - Requires tooling/infrastructure
---
## Next Steps
1. Review and approve recommendations (Team consensus)
2. Use `/retro-apply` to implement approved changes
3. Track effectiveness in next modernization project
4. Update this document with lessons learned
---
## References
- HISTORY.md: Complete project timeline
- Git log: [Commit range analyzed]
- ADRs: [List of ADRs reviewed]
- Test reports: [Test result references]
- Security scans: [Scan result references]
---
**Document Status**: Proposed
**Approval Required**: Team consensus
**Apply Using**: `/retro-apply`
**Next Review**: After next modernization project
Problem: Dependency conflicts discovered mid-migration caused 2 week delay
Evidence:
Proposed Change: Add comprehensive dependency analysis to Phase 0
Expected Impact:
Problem: Documentation created in Phase 6 caused 4 days of archaeology to recreate decisions
Evidence:
Proposed Change: All agents document continuously
Expected Impact:
Problem: Security issues discovered in Phase 1 security scan should have been caught earlier
Evidence:
Proposed Change: Security Agent integrates automated scanning
Expected Impact:
Problem: Coder Agent frequently modified files without reading them first, causing conflicts and requiring user corrections
Evidence:
Proposed Change: Enforce "Read before Write" rule for all agents
Change Type: Agent Behavior
Expected Impact:
Affected Components:
Problem: Agents frequently used Bash commands instead of specialized tools, violating tool usage policy
Evidence:
cat instead of Read: 23 instancesfind instead of Glob: 15 instancesgrep instead of Grep tool: 12 instancesProposed Change: Strict tool usage enforcement
Change Type: Agent Behavior + Protocol Update
Expected Impact:
Affected Components:
Problem: Coder Agent implemented features based on assumptions without confirming requirements with user
Evidence:
Proposed Change: Require explicit confirmation for significant changes
Change Type: Agent Behavior + Protocol Update
Expected Impact:
Affected Components:
Problem: Agents used LLM calls for tasks that could be accomplished more efficiently, reliably, and cost-effectively with scripts or CLI programs
Evidence:
jq could do it instantlyfind + wc -l does it in 0.2 secondsawk/sed handle perfectlyProposed Change: Identify and replace LLM calls with code/scripts where appropriate
Guidelines for LLM-to-Code Replacement:
Decision Tree:
Is this task:
├─ Deterministic with clear logic? → Consider script/code
├─ Simple data transformation? → Consider CLI tools (jq, awk, sed)
├─ File system operation? → Consider bash/python script
├─ Repeated analysis of same pattern? → Consider caching script
└─ Requires reasoning/judgment? → Keep LLM call
Replacement Candidates:
jq instead of LLMawk, sed, tr instead of LLMcloc, wc, grep -c) instead of LLMRecommended Tools by Task:
jq, python -m json.toolawk, sed, grep, tr, cutcloc, tokei, scc (lines of code counters)npm list --json, pip freeze, language-specific CLIscurl + jq, Python requestsChange Type: Agent Behavior + Automation
Expected Impact:
Implementation Complexity: Medium
Implementation Steps:
scripts/utilities/ directory with reusable task-specific scriptsparse-json.sh - JSON parsing with jqcount-code.sh - Lines of code analysisextract-deps.sh - Dependency extractiontransform-text.sh - Common text transformationsValidation:
Affected Components:
scripts/utilities/Important Caveats:
Do NOT replace LLM when:
DO replace LLM when:
Problem: Agents loaded excessive context into the conversation window and used significantly more tokens than necessary for tasks, leading to slower responses, higher costs, and potential context limit issues
Evidence:
Proposed Change: Implement strict context window efficiency and token minimization practices
Core Principle: Use the minimum context necessary to accomplish the task with high quality
Guidelines for Context Window Optimization:
Decision Tree for Context Loading:
Before reading any file, ask:
├─ Do I know the exact file and location?
│ ├─ Yes, specific line range → Use Read with offset/limit
│ └─ No → Use Grep/Glob to find it first
├─ Have I already read this file in this conversation?
│ ├─ Yes → Reference earlier context, don't re-read
│ └─ No → Proceed with selective read
├─ Is this a large file (>500 lines)?
│ ├─ Yes → Use offset/limit to read only relevant sections
│ └─ No → Read full file if needed
└─ Can I answer the question without reading the file?
├─ Yes → Don't read it
└─ No → Read minimum necessary
Best Practices:
1. Search-First Strategy:
# ❌ WRONG: Load multiple files hoping to find something
Read "src/file1.js"
Read "src/file2.js"
Read "src/file3.js"
# ... discover target was in file2.js
# ✅ CORRECT: Find first, then read precisely
Grep "function targetFunction" --type js --output_mode files_with_matches
# Result: src/file2.js
Grep "function targetFunction" --type js --output_mode content -B 2 -A 10
# Read only the relevant function
2. Selective Reading with Offset/Limit:
# ❌ WRONG: Read entire 3000-line file repeatedly
Read "large-file.ts" # All 3000 lines loaded
# ... later in conversation ...
Read "large-file.ts" # All 3000 lines loaded AGAIN
# ✅ CORRECT: Read specific sections
Grep "class UserManager" large-file.ts --output_mode content -n
# Result: Found at line 1247
Read "large-file.ts" --offset 1247 --limit 100 # Read only relevant class
# Later: Reference the earlier read instead of re-reading
"Based on the UserManager class I read earlier at line 1247..."
3. Progressive Context Loading:
# ❌ WRONG: Load everything upfront
Read all package.json files across project
Read all config files
Read all source files in directory
# ... then realize only needed one specific file
# ✅ CORRECT: Start narrow, expand if needed
Glob "package.json" # Find all package.json files
Read "./package.json" # Read only root package.json
# Only if that doesn't answer the question:
Read "packages/*/package.json" # Expand search
4. Leverage Earlier Context:
# ❌ WRONG: Re-read same file multiple times in conversation
[Turn 5] Read "config.json"
[Turn 12] Read "config.json" # Already read at Turn 5!
[Turn 20] Read "config.json" # Already read twice!
# ✅ CORRECT: Reference earlier context
[Turn 5] Read "config.json"
[Turn 12] "Based on the config.json I read earlier, the API endpoint is..."
[Turn 20] "As we saw in the config.json earlier..."
5. Use Grep for Quick Answers:
# ❌ WRONG: Load entire codebase to count occurrences
Read all .ts files to find how many times "deprecated" appears
# ✅ CORRECT: Use Grep with count mode
Grep "deprecated" --type ts --output_mode count
# Instant answer without loading any files into context
Token Efficiency Strategies:
| Task | Inefficient Approach | Efficient Approach | Token Savings |
|---|---|---|---|
| Find function | Read 20 files (50K tokens) | Grep then Read 1 section (500 tokens) | 99% |
| Count imports | Read all files (100K tokens) | Grep with count (0 tokens to context) | 100% |
| Check config value | Read full file 5 times (2500 tokens) | Read once, reference later (500 tokens) | 80% |
| Find file with pattern | Read 30 files (75K tokens) | Glob + Grep (minimal context) | 95% |
| Get specific function | Read 3000-line file (15K tokens) | Read with offset/limit (500 tokens) | 97% |
Change Type: Agent Behavior + Protocol Update
Expected Impact:
Implementation Complexity: Low-Medium
Implementation Steps:
Context Efficiency Checklist (add to all protocols):
Validation:
Affected Components:
Metrics to Track:
# Token usage per conversation
- Baseline: ~500K tokens per modernization project
- Target: ~200K tokens per modernization project (60% reduction)
# File read efficiency
- Baseline: 150 file reads per project, 40% redundant
- Target: 90 file reads per project, <5% redundant
# Context window size
- Baseline: Average 80K tokens in context at any time
- Target: Average 20K tokens in context at any time
Important Caveats:
Do NOT sacrifice quality for token efficiency:
DO optimize when:
Balance:
The retrospective is successful when:
✅ Comprehensive Analysis:
✅ Evidence-Based Recommendations:
✅ Actionable Output:
/retro-apply implementation✅ High Impact:
❌ Blame Culture: Focus on process, not individuals ❌ Vague Recommendations: "Communicate better" → Specify HOW ❌ Unmeasurable Improvements: "Be more efficient" → Quantify ❌ Analysis Paralysis: 3-5 recommendations, not 50 ❌ No Evidence: Opinions without data are not recommendations ❌ Unrealistic Changes: Must be implementable with available resources ❌ Ignoring Small Wins: Not all improvements need to be massive
User: /retro
User: Our modernization project just completed. Review the history and recommend improvements.
Claude: I'll coordinate the agent team to conduct a comprehensive retrospective...
[Migration Coordinator] Analyzing HISTORY.md...
- Project: .NET 6 → .NET 9 migration
- Timeline: Estimated 22 days, Actual 28 days (+27%)
- Quality gate failures: 3 instances
- [Detailed analysis]
[Security Agent] Analyzing security remediation...
- Phase 1: 47 vulnerabilities, 5 days to resolve
- Dependency issues: 12 packages had conflicts
- [Detailed findings]
[Architect Agent] Reviewing ADRs...
- 5 ADRs created, 3 created retroactively
- [Analysis]
[Each agent provides observations]
[All Agents] Analyzing user interactions and agent errors...
- Found 18 user corrections in git log
- Identified 12 instances of wrong tool usage
- User interrupted agents 7 times to provide guidance
- 3 commits reverted due to agent mistakes
- [Detailed findings]
[Migration Coordinator] Synthesizing patterns...
**Protocol Issues**:
- Pattern 1: Testing started too late (appeared in 3 phases)
- Pattern 2: Dependency conflicts discovered mid-migration
- Pattern 3: Documentation created in batches
**Agent Behavioral Issues** (CRITICAL):
- Pattern 4: Coder Agent used Bash cat instead of Read tool (23 instances)
- Pattern 5: Agent implemented features without confirming (4 major cases)
- Pattern 6: Agent didn't read files before modifying (8 instances)
Developing recommendations...
Top 6 recommendations by impact/effort ratio:
1. Always read before write - agent behavior (Impact: 9/10, Effort: 1/10) ← Agent behavior
2. Front-load dependency analysis - protocol (Impact: 9/10, Effort: 3/10)
3. Use appropriate tools enforcement - agent behavior (Impact: 8/10, Effort: 2/10) ← Agent behavior
4. Continuous documentation - protocol (Impact: 8/10, Effort: 2/10)
5. Confirm before implementing - agent behavior (Impact: 9/10, Effort: 2/10) ← Agent behavior
6. Automated security scanning - automation (Impact: 9/10, Effort: 5/10)
Creating IMPROVEMENTS.md...
✅ Retrospective complete: IMPROVEMENTS.md created
Document Owner: Migration Coordinator
Protocol Version: 1.0
Last Updated: 2025-11-01
Companion Command: /retro-apply
Remember: Retrospectives are worthless without action. Review, approve, then apply. ✅