check-redundancy

Check Redundancy Skill

Analyzes markdown files for redundant content by detecting repeated phrases and calculating redundancy percentage.

When to Use This Skill

Use this skill when:

• Reviewing generated Memory Bank documentation for quality
• Checking if a file needs optimization (redundancy > 10%)
• Validating documentation before committing
• During the file generation process to ensure optimal content

Invocation

From target project, run:

python ${CLAUDE_PLUGIN_ROOT}/skills/check-redundancy/scripts/check-redundancy.py <file>

What This Skill Does

1. Analyzes repeated phrases (2-5 words) in markdown files
2. Filters common phrases (articles, conjunctions, domain terms)
3. Calculates redundancy percentage: (repeated occurrences - unique count) / total phrases × 100
4. Reports top repeated phrases for manual review
5. Exit codes:
   • 0: Optimal (≤10% redundancy)
   • 1: High redundancy (>10%, needs optimization)
   • 2: Error

Usage

Command Line

python ${CLAUDE_PLUGIN_ROOT}/skills/check-redundancy/scripts/check-redundancy.py <file>

Example Output

File: architecture.md
Lines: 89
Total phrases analyzed: 1247
Redundancy: 3.2%
✓ Redundancy optimal (3.2% ≤ 10%)

Or if redundancy is high:

File: verbose-file.md
Lines: 150
Total phrases analyzed: 2341
Redundancy: 15.8%

⚠️  High redundancy detected (15.8% > 10%)

Top repeated phrases:
  - 'memory bank documentation' (12 times)
  - 'project analysis json' (8 times)
  - 'generation plan md' (7 times)
  - 'static files to' (6 times)
  - 'prompt templates in' (5 times)

Integration Examples

In Generation Workflows

After generating a file:

# Generate file
python generate-doc.py > .memory_bank/file.md

# Check redundancy
python ${CLAUDE_PLUGIN_ROOT}/skills/check-redundancy/scripts/check-redundancy.py .memory_bank/file.md

# If exit code is 1, optimize the file
if [ $? -eq 1 ]; then
    echo "Optimizing file..."
    # Run optimization
fi

In Batch Processing

for file in .memory_bank/**/*.md; do
    python ${CLAUDE_PLUGIN_ROOT}/skills/check-redundancy/scripts/check-redundancy.py "$file"
done

How It Works

1. Extract phrases: Tokenizes text into 2-5 word phrases
2. Filter noise: Removes markdown syntax, code blocks, common English phrases
3. Count frequencies: Tracks how often each unique phrase appears
4. Calculate redundancy:
   • If "memory bank" appears 5 times, that's 4 redundant occurrences
   • Redundancy % = (total redundant occurrences) / (total phrases) × 100
5. Report results: Shows percentage and top repeated phrases

Threshold

• ≤10% redundancy: Optimal, file is concise
• >10% redundancy: High, file likely has repetitive content that can be consolidated

Related

• /memento:optimize-memory-bank: Optimizes Memory Bank files with high redundancy
• /memento:fix-broken-links: Validates and fixes broken links in Memory Bank files
• .memory_bank/guides/code-review-guidelines.md: Documentation quality standards